Intelligent sand excavation supervision system based on multi-source data fusion

Abstract

The invention discloses an intelligent sand excavation supervision system based on multi-source data fusion, and relates to the technical field of machine learning, and the system collects target river reach data in real time through a multi-source sensing network module; the spatial-temporal feature fusion module generates a dynamic state fingerprint matrix; the adaptive baseline monitoring module establishes and dynamically updates a normal state baseline under multiple conditions, and triggers an abnormal disturbance alarm by calculating a mahalanobis distance between a real-time fingerprint and the baseline and combining collaborative deviation verification of acoustics, turbidity and water flow characteristics, and the multi-task analysis module adopts a parallel neural network architecture, so that a multi-task analysis result is obtained. The illegal operation type probability, the strength estimation value and the environment disturbance level are synchronously output; the three-dimensional visualization early warning module generates an early warning interface based on the analysis result; the dynamic knowledge management module and the self-adaptive optimization module are used for improving the analysis accuracy and continuously optimizing the system performance by using historical experience; the method has the advantages that abnormal disturbance events such as illegal sand excavation and the like can be accurately and intelligently supervised in real time, and powerful capability is provided.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an intelligent sand mining supervision system based on multi-source data fusion. Background Art

[0002] River sand mining can have significant negative impacts on riverbed stability, aquatic ecosystems, flood control, and water resource utilization. Illegal or disorderly sand mining is extremely difficult to regulate due to its concealed nature, mobility, and environmental damage. Existing methods for regulating river sand mining primarily rely on manual inspections, video surveillance, satellite remote sensing, and monitoring of single physical quantities such as turbidity and ship AIS signals.

[0003] However, these methods have numerous limitations: video surveillance is easily affected by weather and lighting, and underwater activities cannot be directly observed; satellite remote sensing has limited resolution and timeliness, making it difficult to capture small, short-term, or nighttime illegal operations; and monitoring of a single physical quantity, such as turbidity, is susceptible to natural factors such as heavy rain, changes in upstream water flow, or interference from other legal water activities such as shipping, resulting in a high false alarm rate and difficulty distinguishing the specific nature, intensity, and potential impact of abnormal disturbances. Furthermore, existing data analysis methods are often simplistic and lack in-depth exploration of the interrelationships between multiple environmental factors, making it difficult to accurately identify the characteristic, multi-dimensional, coordinated anomalies caused by illegal sand mining within a complex context. Summary of the Invention

[0004] The present invention provides an intelligent sand mining supervision system based on multi-source data fusion to solve the technical problem in the existing technology that it is difficult to accurately identify, deeply analyze and effectively warn of abnormal disturbance events such as illegal sand mining that are highly concealed and have complex characteristics, especially how to overcome the limitations and high false alarm rate of a single information source, and provide rich event attribute information to support efficient supervision decision-making.

[0005] The present invention provides an intelligent sand mining supervision system based on multi-source data fusion, which is characterized by comprising: A multi-source perception network module is used to collect underwater mechanical vibration spectrum, suspended matter concentration gradient, and three-dimensional water flow vector data in the target river section; A spatiotemporal feature fusion module receives data collected by the multi-source perception network module and generates a dynamic state fingerprint matrix including acoustic frequency domain features, turbidity spatiotemporal gradients, and water flow turbulence index through a graph attention mechanism and a time series model; An adaptive baseline monitoring module monitors the dynamic state fingerprint matrix in real time through an online learning adaptive baseline model. When it detects the coordinated deviation of acoustic energy anomalies, turbidity gradient mutations, and water flow vector disturbances, it triggers an abnormal disturbance alarm. The multi-task analysis module extracts the time-frequency feature sequences corresponding to the abnormal disturbance events that trigger abnormal disturbance alarms, and uses a parallel neural network architecture to simultaneously output the probability distribution of illegal operation types, operation intensity estimates, and bed disturbance level assessment results; The three-dimensional visualization warning module generates a three-dimensional visualization warning interface based on the analysis results of the multi-task analysis module, dynamically displaying the location coordinates of the abnormal disturbance source, the suspended sediment diffusion simulation path, and the superimposed warning layer of the ecologically sensitive area.

[0006] Preferably, the multi-source sensing network module includes an acoustic vibration sensor array, a distributed turbidity monitoring node and a flow profiler; The spatiotemporal feature fusion module generates the dynamic state fingerprint matrix, which is configured as follows: the acoustic vibration sensor array, distributed turbidity monitoring node and flow profiler included in the multi-source perception network module are represented as nodes in a graph network, and the node attributes include: For a node corresponding to a monitoring point of the acoustic vibration sensor array: its attribute is 1 / 3 octave spectrum energy distribution; For the node corresponding to the distributed turbidity monitoring node: its attributes include the current turbidity value and the time change rate of turbidity concentration calculated based on a preset time window; For a node corresponding to the velocity profiler: its attributes include a three-dimensional velocity vector and information on the angle between the water flow vector of the node and at least one adjacent node; The feature association weights are calculated based on the spatial distance between nodes, the relative direction of water flow and the signal propagation characteristics, and the weights are used to aggregate the features of adjacent nodes through the graph attention mechanism; a temporal convolutional network is used to process the node timing features, and the dynamic state fingerprint matrix that integrates the spatiotemporal information is output. The dimension of the dynamic state fingerprint matrix is ​​used to encode the comprehensive state information of the river.

[0007] Preferably, the adaptive baseline monitoring module triggers an abnormal alarm based on the dynamic state fingerprint matrix, and its working principle is: Using historical monitoring data from the period without sand mining interference, a baseline fingerprint database that distinguishes different seasons and meteorological conditions was established; Applying a sliding time window and adjusting the update rate accordingly according to the rate of change of real-time hydrological conditions to dynamically update the statistical characteristics of the benchmark fingerprint library, including the mean and covariance matrix; Calculate the Mahalanobis distance between the currently acquired real-time state fingerprint and the selected benchmark library; The abnormal alarm is triggered when the Mahalanobis distance exceeds a preset statistical significance threshold and the following two collaborative verification conditions are met at the same time: first, a statistically significant shift of the acoustic feature is detected within a specific preset frequency band; second, the observed spatial distribution direction of the turbidity gradient is inconsistent with the normal diffusion pattern predicted based on the water flow diffusion model.

[0008] Preferably, the multi-task parsing module outputs the analysis results of the abnormal disturbance event, and a multi-branch parallel neural network architecture is adopted internally. The multi-branch parallel neural network architecture technology includes a branch specifically for processing time-frequency features, which is implemented using a convolutional neural network; and a branch for processing spatial propagation features based on the sensor topology structure, which is implemented using a graph neural network. And use the attention mechanism to handle the correlation between cross-modal features; The convolution processing branch adopts a depth-separable convolution structure to extract local sensitive features and reduce the computational burden; The graph neural network processing branch is used to capture the propagation path information of abnormal signals in the monitoring network based on the connection relationship between sensors; The attention mechanism dynamically adjusts the weights of features from different modalities or different processing branches, enabling the multi-branch parallel neural network architecture to focus on the information most critical to the current parsing task; Furthermore, the multi-branch parallel neural network architecture is designed to be trained by adopting a joint loss function to simultaneously optimize the classification task for distinguishing the types of abnormal disturbance events and the levels of environmental effects and the regression task for estimating the work intensity.

[0009] Preferably, the three-dimensional visualization warning module generates a warning view based on the received abnormal disturbance event analysis results and related data, and its generation logic includes: Apply nonlinear dimensionality reduction technology to map high-dimensional state fingerprint data or anomaly scores into two-dimensional space, forming a heat map representing the health status of the river section; Based on the real-time monitored turbidity gradient change rate data, a turbidity contour map is dynamically drawn, and the density of the contour lines is used to reflect the rate and range of abnormal suspended matter diffusion; Driving a suspended sediment particle simulation system, wherein the motion trajectory of particles in the suspended sediment particle simulation system is determined based on a three-dimensional water flow vector field acquired in real time, and the visual attributes of the particles are associated with turbidity concentration information associated with their location; Run the spatial topology analysis program to calculate in real time the shortest water flow path distance between the analyzed abnormal operation point location and the calibrated ecological sensitive area, and trigger different levels of early warning indications based on the shortest water flow path and the intensity of the abnormal disturbance event.

[0010] As an advantage, a dynamic knowledge management module is also included to assist in improving the accuracy of analysis, and its operating mechanism is as follows: Continuously store the key features of historical abnormal disturbance events and the ecological and environmental impact monitoring data actually observed after the abnormal disturbance events occurred; When a new abnormal disturbance event is detected, the dynamic time warping algorithm or similar sequence matching technology is used to compare the characteristic sequence of the new abnormal disturbance event with the historical pattern library, and prioritize the association with similar patterns that are historically known to have higher ecological risks; Based on the matched historical abnormal disturbance event patterns and their known consequence information, the confidence score of the output results of the currently running multi-task parsing module is dynamically adjusted or the results are corrected a posteriori.

[0011] Preferably, an adaptive optimization module is further included to maintain and improve the system monitoring performance, which is implemented as follows: Continuously track the system's early warning performance indicators, including false alarm rate and missed alarm rate. Once the performance is detected to be below the preset standard, it will automatically trigger the retraining process of the baseline model in the adaptive baseline monitoring module or the parsing model in the multi-task parsing module; Automatically integrate real sand mining events and their detailed attribute information verified by manual confirmation or on-site verification into the sample database used for model training; In addition, based on the results of the frequency analysis of the geographical distribution of historical alarm events, the sampling strategy of sensors deployed in the multi-source perception network is dynamically adjusted. The adjustment may include increasing the sampling frequency, monitoring sensitivity or data transmission priority of relevant sensors located in high-frequency alarm areas to achieve enhanced monitoring of specific areas.

[0012] Preferably, the deployment mode of the sensors in the multi-source perception network module is configured as follows: Laying a distributed optical fiber vibration sensor array along the riverbed area of ​​the target river section, wherein the distributed optical fiber vibration sensor array is suitable for monitoring underwater mechanical vibration signals within a specific frequency range, wherein the frequency range covers the expected characteristic frequency band of sand mining equipment activity; High-frequency turbidity monitoring nodes are deployed to form a gradient tracking network. The spatial spacing of the nodes is dynamically adjusted according to the geometric characteristics of the river channel at the location, and the minimum distance between adjacent nodes is set to no more than 1 / 5 of the river channel width at that location.

[0013] The technical solution provided by this application has at least the following technical effects or advantages: By integrating multi-dimensional, multi-modal information such as underwater vibration, turbidity gradients, and water flow vectors, and employing collaborative deviation-based judgment logic and adaptive baselines, the system can more effectively distinguish actual sand mining anomalies from complex background interference, significantly reducing false alarm and missed alarm rates. Furthermore, the system not only detects anomalies but also utilizes multi-task neural networks to simultaneously analyze the type, intensity, and potential level of environmental disturbance, providing richer and more valuable information for regulatory decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is the architecture diagram of the intelligent sand mining supervision system based on multi-source data fusion of the present invention. DETAILED DESCRIPTION

[0015] The present invention relates to an intelligent sand mining supervision system based on multi-source data fusion to solve the technical problem in the existing technology that it is difficult to accurately identify, deeply analyze and effectively warn of abnormal disturbance events such as illegal sand mining that are highly concealed and have complex characteristics. In particular, it solves the technical problem of how to overcome the limitations and high false alarm rate of a single information source and provide rich event attribute information to support efficient supervision decision-making.

[0016] The above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods of the specification to better understand the above technical solution. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention. It should be understood that the present invention is not limited to the example embodiments used only to explain the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, it should be noted that, for the convenience of description, only the parts related to the present invention, rather than all, are shown in the drawings.

[0017] Example 1

[0018] like Figure 1 The intelligent sand mining supervision system architecture based on multi-source data fusion is shown in the figure. The system includes: A multi-source perception network module is used to collect underwater mechanical vibration spectrum, suspended matter concentration gradient, and three-dimensional water flow vector data in the target river section; A spatiotemporal feature fusion module receives data collected by the multi-source perception network module and generates a dynamic state fingerprint matrix including acoustic frequency domain features, turbidity spatiotemporal gradients, and water flow turbulence index through a graph attention mechanism and a time series model; An adaptive baseline monitoring module monitors the dynamic state fingerprint matrix in real time through an online learning adaptive baseline model. When it detects the coordinated deviation of acoustic energy anomalies, turbidity gradient mutations, and water flow vector disturbances, it triggers an abnormal disturbance alarm. A multi-task analysis module extracts the time-frequency feature sequence corresponding to the abnormal disturbance alarm and uses a parallel neural network architecture to simultaneously output the probability distribution of illegal operation types, operation intensity estimation, and bed disturbance level assessment results; The three-dimensional visualization warning module generates a three-dimensional visualization warning interface based on the results of the multi-task analysis module, dynamically displaying the location coordinates of the abnormal disturbance source, the simulated path of suspended sediment diffusion, and the superimposed warning layer of the ecologically sensitive area.

[0019] Embodiments of the present invention aim to provide an intelligent sand mining monitoring system based on multi-source data fusion. This system integrates and deploys a multi-type sensor network to acquire comprehensive information about the river's environment. It utilizes spatiotemporal data fusion and feature extraction techniques to generate a dynamic fingerprint representing the river's state. Anomalies are detected by comparing these fingerprints with an adaptively updated baseline state. A multi-task machine learning model analyzes the detailed attributes of abnormal disturbance events, ultimately presenting warning information and potential impacts in an intuitive three-dimensional visualization. The system also incorporates dynamic knowledge management and adaptive optimization mechanisms to continuously improve monitoring accuracy and efficiency.

[0020] 1. The multi-source perception network module is responsible for real-time, continuous or high-frequency collection of key physical and environmental parameters of the target regulated river section: Acoustic vibration sensor array: A distributed fiber optic vibration sensor (DFVS or DAS) array, deployed along the riverbed of the target river section, is preferred. This array utilizes optical fiber as a sensor, capable of detecting vibration information at multiple consecutive points along the fiber path. Its configuration requires effective monitoring of underwater vibration signals generated by the mechanical operations of sand mining vessels (such as engines, pumps, excavators, and material transport) within a specific frequency range. Based on experience or prior knowledge, this frequency range is set to cover the expected characteristic frequency band of primary sand mining equipment activity, for example, between 10 Hz and 2 kHz. The raw data collected is a time series of vibration signals from different locations along the fiber. These signals are subsequently processed (e.g., through Fourier transform) to extract the spectral characteristics of the underwater mechanical vibration.

[0021] Distributed turbidity monitoring nodes: Multiple turbidity monitoring sensor nodes are deployed within the target river section, using either wired or wireless connections. To effectively capture the changes in suspended matter concentration and its spatial diffusion caused by sand mining activities, these nodes are deployed as a network capable of tracking suspended matter concentration gradients. Specifically, the node distribution must reflect the spatial variation in concentration. The spatial spacing of the nodes is dynamically adjusted based on the specific river channel geometry (such as curvature radius and width variations) to ensure the coverage and sensitivity of the monitoring network.

[0022] In one specific implementation, to effectively capture near-source diffusion characteristics, the minimum distance between adjacent turbidity monitoring nodes is set to no more than a specific fraction of the river width at that location, for example, no more than one-fifth. These nodes can measure turbidity values ​​(e.g., in NTUs) in real time or at a high frequency (e.g., minute-by-minute or higher), enabling the calculation of spatial suspended matter concentration gradients (the direction and speed of concentration change).

[0023] Current profilers: Acoustic Doppler current profilers (ADCPs) or other devices capable of measuring water velocity profiles are deployed at key locations along the river, such as sections that may affect suspended sediment transport. These devices measure three-dimensional water velocity vectors (including magnitude and direction) at different depths within the water column, providing information on the three-dimensional water flow vector field. This information is crucial for understanding background flow conditions, predicting suspended sediment transport pathways, and distinguishing natural flow turbulence from flow anomalies caused by sand mining.

[0024] The multi-source perception network module runs continuously and transmits the various types of raw sensor data it collects (vibration time domain signals, turbidity value sequences, and flow profile data) to the subsequent spatiotemporal feature fusion module for processing.

[0025] 2. Spatiotemporal feature fusion module: Receives heterogeneous, spatiotemporally distributed data from the multi-source perception network module, effectively integrates and extracts features, and ultimately generates a dynamic state fingerprint matrix that can fully represent the comprehensive state of the current target river section. Its configuration and workflow are described in detail: Step 1: Construct a graph network representation. First, the physically distributed sensors need to be abstracted into a mathematical graph structure. The system logically represents each acoustic sensor (or virtual sensor point on an optical fiber), each turbidity monitoring node, and each current profiler (or its representative measurement point) deployed in the river section as a node in the graph network. Each node is then assigned attributes that represent its own measurement information. These attributes may include: for acoustic nodes, the energy distribution of the monitored signal within key frequency bands (for example, the power spectral density values ​​in different frequency bands); for turbidity nodes, the measured turbidity value itself, or more importantly, the rate of change of turbidity over time; and for flow nodes, these attributes may reflect the magnitude and direction of the measured water velocity, or the vector angle with the water flow direction of other adjacent nodes, and other characteristics that reflect the water flow state.

[0026] Step 2: Feature aggregation using a graph attention mechanism. Information between nodes in a graph exhibits spatial correlation. To capture this correlation, the algorithm calculates weights for feature associations between adjacent nodes (or pairs of nodes with potential mutual influence). This weighting should take into account multiple factors, including the physical distance between nodes (e.g., closer proximity may result in stronger correlations), the relative direction of water flow between them (e.g., turbidity changes at upstream nodes may affect downstream nodes), and the propagation characteristics of signals (such as sound waves and water disturbances) in the water column (e.g., the attenuation model of sound waves). These factors can be combined through a function to determine the initial strength of the associations between nodes. Next, a graph attention mechanism is introduced, which enables each node to automatically learn which neighboring node features to pay more attention to based on the features of its neighbors and the calculated correlation weights. Through this weighted aggregation (e.g., weighted summation of neighboring node features or more complex aggregation operations), each node can integrate information from its neighborhood to obtain an updated feature representation that is more context-aware.

[0027] Step 3: Use a temporal convolutional network to process temporal features. In addition to spatial correlation, the data from each sensor node is itself a sequence that changes over time. In order to effectively extract long-term dependencies in these sequences (i.e., the current state may be affected by earlier events), the algorithm uses a temporal convolutional network (TCN) to process the feature time series of each node (or aggregated nodes). A key advantage of TCN is its use of a dilated convolution structure. This convolution kernel can exponentially increase its receptive field without increasing the amount of computation by inserting "holes" (dilation factors) between elements, effectively capturing dependency patterns across longer time spans. This is very helpful for understanding events such as sand mining that may last for a period of time or whose impacts may gradually become apparent.

[0028] Step 4: Output the dynamic state fingerprint matrix. After the aforementioned graph attention aggregates spatial information and the temporal convolutional network extracts temporal dependencies, the system generates a final output for the current moment (or a short time window). This output is defined as the dynamic state fingerprint matrix (or alternatively, a high-dimensional vector). The dimensions of this matrix (or vector) are designed to encode the river's current state, integrating spatiotemporal information from multiple sensors. It is no longer a single sensor reading, but rather a compact mathematical representation of the overall "health" or "behavior pattern" of the entire monitored river section, forming the foundation for subsequent anomaly detection and analysis.

[0029] 3. Adaptive baseline monitoring module: This module is responsible for comparing the real-time generated status fingerprint with the normal background status to determine whether there is an anomaly and issue an alarm when specific conditions are met. Its working principle is explained in detail, including the key mechanisms of adaptability and multi-dimensional verification: Step 1: Establish a multi-condition baseline fingerprint library. The normal background state of a river is not constant. It is affected by factors such as season (flood season, dry season), weather (sunny, rainy), and specific hydrological events (such as upstream flood discharge). Therefore, a single fixed benchmark cannot be used. The system first needs to use historical monitoring data from periods without known sand mining activities (this data is also processed by the module in Part 2 to generate corresponding state fingerprints). According to different seasonal divisions (such as spring, summer, autumn, and winter) and major meteorological conditions (such as sunny, rainy, strong winds, etc.), corresponding baseline state fingerprint libraries are established. Each library represents the normal state pattern under a specific external environment. The initial data source needs to be reliable to ensure the validity of the benchmark.

[0030] Step 2: Dynamically update the baseline. The natural environment itself changes slowly or rapidly, and the baseline database cannot remain static. The system uses a sliding time window approach to dynamically update the baseline. That is, only monitoring data from a recent period (such as the past few days or weeks) that is considered normal (a mechanism is required to exclude data from known abnormal periods) is considered to calculate and update the baseline. At the same time, the update rate (or the length of the time window, the update frequency) is adapted to the rate of change of the currently monitored hydrological environment (such as flow rate and flow velocity). For example, during periods of drastic changes in hydrological conditions (such as floods), the update rate should be accelerated; during stable periods, it can be slowed down. The update mainly involves the statistical characteristics of the state fingerprints under the corresponding conditions in the baseline fingerprint database. The core is to update its mean vector and covariance matrix, which together describe the center and fluctuation range of the normal state.

[0031] Step 3: Calculate the distance from the benchmark and make a preliminary judgment. For each newly generated real-time status fingerprint, the system first determines the current environmental conditions (season, weather), then selects a corresponding benchmark from the benchmark library. Next, it calculates the Mahalanobis distance between this real-time status fingerprint and the selected benchmark library (represented by its mean vector and covariance matrix). Compared to the simple Euclidean distance, the Mahalanobis distance accounts for possible correlations and dimensionality differences between data dimensions, making it a more robust distance metric for anomaly detection. The system sets a preset statistical significance threshold (this threshold can be determined by statistically analyzing the Mahalanobis distance distribution of historical normal data, such as the 99th percentile). When the calculated Mahalanobis distance exceeds this threshold, it indicates that the current status has significantly deviated from the normal range under those conditions, and the system initiates the next step of multi-dimensional collaborative verification. A distance exceeding the limit alone is not sufficient to directly determine a sand mining anomaly.

[0032] Step 4: Perform multi-dimensional collaborative verification to confirm anomalies. To improve the accuracy of alerts and avoid misidentifying natural anomalies (such as landslides and normal ship passages) as sand mining, the system has designed a multi-dimensional collaborative verification rule. This rule requires that before triggering an anomaly alert, in addition to meeting the Mahalanobis distance limit condition in the previous step, the following two specific verification conditions must also be met: Condition 1 (Acoustic Signature Verification): By analyzing the characteristic components of the acoustic sensor in the real-time state fingerprint, it must be detected that these acoustic characteristics have a statistically significant deviation within a specific pre-defined frequency band related to sand mining activities (such as the characteristic frequency bands covering excavation, screening, and pumping noise). For example, the energy in these frequency bands exceeds the corresponding baseline value by a certain multiple, or specific frequency components are present that are not expected under the baseline state.

[0033] Condition 2 (verification of turbidity-flow correlation): This requires analysis based on data from both the turbidity sensor and the current profiler. The spatial distribution direction of the observed turbidity gradient (i.e., the direction of the most rapid increase in suspended solids concentration) is compared with the diffusion direction predicted by a model for the normal diffusion of substances (such as suspended solids) based on the current real-time flow field. This condition is met if the observed turbidity gradient direction is significantly inconsistent with the model's predicted normal diffusion pattern (e.g., primarily downstream diffusion along the current) (e.g., evidence of countercurrent diffusion or, at specific locations, a diffusion source direction that deviates significantly from the mainstream direction). This helps rule out natural causes of elevated turbidity, such as riverbank erosion.

[0034] Only when the three conditions of Mahalanobis distance exceeding the limit, significant abnormal acoustic characteristics, and abnormal turbidity-water flow correlation are met simultaneously, will the adaptive baseline monitoring module finally confirm and trigger a valid abnormal alarm signal and pass the relevant information to the next module for analysis.

[0035] 4. Multi-task parsing module: After receiving the abnormal alarm signal and related status fingerprint data confirmed by the baseline monitoring module, the multi-task parsing module is responsible for conducting an in-depth analysis of the abnormal disturbance event corresponding to the abnormal alarm signal to determine its specific nature, that is, outputting the abnormal disturbance event parsing results, including the type of sand mining operation (such as pump suction, chain bucket or grab bucket, if the model can distinguish, or at least determine whether it is a sand mining activity), intensity (such as estimating the scale or rate of the operation) and the potential effect level on the environment (such as low, medium, and high risk). In order to complete these different parsing tasks (classification tasks and regression tasks) at the same time, the module adopts a multi-branch parallel neural network architecture. Its technical principle is: Build a multi-branch parallel architecture: Design a neural network model with multiple parallel processing branches. Each branch is designed to process and extract feature information of different types or levels to meet the needs of multi-modal input and multi-task output. For example: There can be a branch dedicated to processing time-frequency features. This branch may mainly receive spectrograms or acoustic feature sequences obtained after processing acoustic sensor data, and use convolutional neural networks (CNNs), especially those CNN structures that are good at processing local patterns in images or sequences, to extract time-frequency pattern features related to specific mechanical noise.

[0036] Another branch could focus on processing spatial propagation characteristics. This branch might leverage the topological structure of the sensor graph network constructed in Part 2. When an abnormal signal (such as a vibration wave or turbidity diffusion) occurs, the signal propagates through the sensor network. Graph neural networks (GNNs), such as graph convolutional networks (GCNs), can be used to capture the path and pattern characteristics of how this signal propagates spatially and along the connections of the sensor network.

[0037] Furthermore, mechanisms are required to handle cross-modal correlations. Sand mining activities often leave multiple, interrelated imprints on acoustics, turbidity, and water flow. The system utilizes attention mechanisms (such as self-attention or cross-modal attention) to learn and analyze the interdependencies between features from different modalities (e.g., acoustic and turbidity features) or between features output by different processing branches (e.g., CNN and GNN branches).

[0038] To improve efficiency and effectiveness, some specific techniques can be adopted in the above branches: In the convolution processing branch, the depthwise separable convolution structure can be preferably used to replace the traditional standard convolution. This structure decomposes the convolution operation into depthwise convolution (independent convolution for each input channel) and point-by-point convolution (1x1 convolution for channel fusion). It can significantly reduce the number of model parameters and computational complexity while maintaining or sometimes even improving the model's ability to extract local spatial features.

[0039] The core of the graph neural network processing branch lies in leveraging known spatial connectivity between sensors (for example, which sensors are adjacent, their distance, and their relative orientation). When an anomaly occurs, the graph neural network can effectively simulate the propagation of signals (such as vibration energy or turbidity anomalies) from the source node to its neighboring nodes, thereby capturing spatial features that reflect the propagation path, speed, and range.

[0040] The attention mechanism enables the model to process information with precision. It dynamically calculates and assigns weights to input features from different sensor modalities, or to intermediate features generated by different processing branches. This means that for a specific parsing task (such as determining the type of work), the model may automatically learn to pay more attention to acoustic features, while determining the level of environmental effects may require more attention to turbidity and water flow characteristics. This ability to dynamically focus on key information improves the model's parsing accuracy.

[0041] Implementation of Multi-Task Learning: Because this module needs to simultaneously output multiple results—job type (classification), intensity (regression), and effect level (classification or ranking)—a multi-task learning framework is required. This is achieved by setting multiple output heads (one for each task) at the end of the neural network and designing a joint loss function. This loss function is a weighted sum of the losses of each individual task (for example, cross-entropy loss for classification and mean squared error loss for regression). During model training, minimizing this joint loss function allows the network to simultaneously address the optimization objectives of all tasks and potentially leverage inter-task correlations to improve overall performance.

[0042] 5. 3D Visualization Warning Module: This module is responsible for converting the abstract data and results analyzed by the previous modules into a 3D visualization warning view that users (such as supervisors) can intuitively understand and quickly respond to. It receives the abnormal alarm signal event analysis results from the multi-task analysis module and combines them with real-time data from the perception network and fusion modules. The view generation logic is as follows: Generating a Health Heatmap: To comprehensively understand the health of a river section, the system can apply nonlinear dimensionality reduction techniques, such as t-distributed stochastic neighbor embedding (t-SNE) or uniform manifold approximation and projection (UMAP). These techniques map the high-dimensional dynamic state fingerprint data generated by the spatiotemporal feature fusion module, or the anomaly scores (such as Mahalanobis distance) calculated by the baseline monitoring module, onto a two-dimensional plane. The "health" of each area is then rendered on this two-dimensional plane using color (e.g., from green for normal to red for abnormal), generating a heatmap that visually illustrates which areas of the entire target river section may be experiencing anomalies or high-risk conditions.

[0043] Dynamic turbidity contour mapping: To clearly demonstrate the potential spread of suspended matter pollution caused by sand mining, the system dynamically calculates and plots lines connecting points of equal turbidity in the water body, known as contour lines, based on real-time turbidity data and its spatial gradient rate of change. A contour map is distinguished by its density, which intuitively reflects the speed and spatial extent of turbidity changes: densely packed areas indicate large turbidity gradients and dramatic changes, potentially near pollution sources; sparsely packed areas indicate more gradual concentration changes. These contour lines should dynamically change as real-time data is updated, demonstrating the real-time spread of pollution.

[0044] Driven Suspended Sediment Particle Simulation System: To more vividly simulate the transport paths of suspended matter in water flow, a virtual suspended sediment particle system can be established. The system releases virtual particles near detected anomaly sources (or source areas inferred based on turbidity gradients). The movement of these particles is driven entirely by real-time or predicted three-dimensional water flow vector field data, simulating their transport by the current. Furthermore, the visual properties of the particles (such as color depth, size, and density) can be correlated with real-time turbidity concentration measurements or estimates associated with their spatial location. This allows users to intuitively understand the transport direction, speed, and impact range of suspended sediment pollution by observing the drift paths and color changes of a large number of particles.

[0045] Run spatial topology analysis and trigger warnings: The system stores the geographic spatial extent information of known ecologically sensitive areas (such as fish spawning grounds, rare aquatic habitats, and drinking water source protection areas). After the multi-task analysis module locates the abnormal operation point, the system runs a spatial topology analysis program. This program uses the river's geometric network relationships and real-time flow field information to calculate the shortest water flow path distance (not the straight-line distance) from the abnormal operation point to the boundaries of each ecologically sensitive area. The system then triggers different levels of warning indications based on this calculated water flow distance and the abnormal disturbance event intensity or environmental impact level assessment results provided by the multi-task analysis module. For example, a very close distance and a high intensity abnormal disturbance event may trigger the highest level of emergency alert; a longer distance or a lower intensity may trigger a lower level of attention or inspection recommendation.

[0046] 6. Dynamic Knowledge Management Module: This module can be included to enable the system to learn and evolve, especially to improve the accuracy of analyzing complex or new sand mining behaviors. Its operating mechanism is to use historical experience to assist current decision-making: Data Storage: This module requires a database to continuously store information about past, confirmed abnormal disturbance events (perhaps verified by manual verification or subsequent investigation). Key data stored should include feature representations of the event at the time of occurrence (e.g., the feature matrix or fingerprint sequence for that period output by the spatiotemporal feature fusion module in Part 2) and, most importantly, actual observed ecological and environmental impact monitoring data after the event (e.g., downstream water quality monitoring results, biodiversity survey reports, etc.).

[0047] Pattern matching: This module is activated when the system detects and analyzes a new abnormal disturbance event. It uses sequence matching algorithms, such as Dynamic Time Warping (DTW), or other distance- or similarity-based techniques (such as deep learning-based sequence embedding similarity calculations), to compare the feature sequence extracted from this new event (e.g., the evolution of state fingerprints over a period of time) with the various event patterns stored in the historical pattern library. During the matching process, the algorithm should be designed to prioritize and associate similar event patterns that have historically been flagged as posing higher ecological risks.

[0048] Result Adjustment: Based on the matching results, i.e., which historical similar events have been found and the known impacts of these historical events, this module can dynamically adjust the preliminary results (such as the determination of operation type, intensity, and effect level) output by the currently running multi-task analysis module (Part 4). For example, if the characteristic pattern of the current event is highly similar to a historical event known to have caused serious ecological damage, even if the preliminary analysis results indicate that its intensity is not high, this module can increase the confidence level of its risk level assessment or directly make a posterior probability correction to the analysis results, thereby making the final warning more valuable and forward-looking.

[0049] 7. Adaptive Optimization Module: To ensure the long-term stable operation of the system and continuously improve performance, this adaptive optimization module can be added. Its implementation method focuses on the self-maintenance and performance improvement of the system: Performance monitoring and retraining triggering: This module needs to continuously track and evaluate the system's warning accuracy. This usually requires combining manual feedback or on-site verification results to calculate the system's false alarm rate (misreporting normal situations as abnormalities) and missed alarm rate (failure to detect real abnormal events). The system sets acceptable thresholds for these performance indicators. Once the actual performance (for example, the average false alarm rate over a period of time) is lower than this preset standard, indicating that the system performance has deteriorated (possibly due to environmental changes, baseline drift, or the emergence of new interference patterns), the module will automatically trigger a retraining process. This process may be for the baseline model in the adaptive baseline monitoring module in Part 3 (for example, forcing the baseline statistics to be recalculated using updated data), or for the task parsing model in Part 4 (initiating incremental training or complete retraining).

[0050] Automatic sample library expansion: To enable continuous model learning, the system requires a mechanism to continuously acquire new, high-quality training samples. This module automatically and structuredly integrates real sand mining events verified by manual confirmation or on-site law enforcement inspections (including detailed attribute information such as the exact time, location, operation type, and intensity estimate), as well as sensor data and status fingerprints recorded by the system during the same period, into the sample database used for model training (whether baseline or analytical model). This ensures that the model learns the latest and most realistic abnormal patterns.

[0051] Dynamic adjustment of sensing strategies: The system can identify river sections that are "hotspots" with high incidence of sand mining activities by analyzing the geographical distribution frequency of historical alarm events. Based on this analysis result, the module can dynamically adjust the operating strategies of some sensors deployed in the multi-source sensing network to achieve optimal resource allocation and more effective monitoring of key areas. For example, for river sections identified as "hotspots", the system can automatically increase the sampling frequency of relevant sensors (such as turbidity meters and acoustic sensors) located in the area (to capture faster changes), improve their monitoring sensitivity (if the hardware is adjustable), or give them a higher priority in the data transmission network to ensure that data in key areas can be collected and processed in a priority and timely manner, thereby achieving enhanced monitoring of these high-risk areas.

[0052] This detailed description of the various modules defined in the claims and their interrelationships is intended to provide a clear and complete description that will enable those skilled in the art to understand and implement the technical solutions of the present invention. Any equivalent substitutions or adaptive improvements based on the core concepts and technical principles of the present invention that do not depart from the spirit and scope of this specification shall be deemed to fall within the scope of protection of the present invention.

Claims

1. Intelligent sand mining supervision system based on multi-source data fusion, characterized by: include: A multi-source perception network module is used to collect underwater mechanical vibration spectrum, suspended matter concentration gradient, and three-dimensional water flow vector data in the target river section; A spatiotemporal feature fusion module receives data collected by the multi-source perception network module and generates a dynamic state fingerprint matrix including acoustic frequency domain features, turbidity spatiotemporal gradients, and water flow turbulence index through a graph attention mechanism and a time series model; An adaptive baseline monitoring module monitors the dynamic state fingerprint matrix in real time through an online learning adaptive baseline model. When it detects the coordinated deviation of acoustic energy anomalies, turbidity gradient mutations, and water flow vector disturbances, it triggers an abnormal disturbance alarm. The multi-task analysis module extracts the time-frequency feature sequences corresponding to the abnormal disturbance events that trigger abnormal disturbance alarms, and uses a parallel neural network architecture to simultaneously output the probability distribution of illegal operation types, operation intensity estimates, and bed disturbance level assessment results; The three-dimensional visualization warning module generates a three-dimensional visualization warning interface based on the analysis results of the multi-task analysis module, dynamically displaying the location coordinates of the abnormal disturbance source, the suspended sediment diffusion simulation path, and the superimposed warning layer of the ecologically sensitive area.

2. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: The multi-source sensing network module includes an acoustic vibration sensor array, a distributed turbidity monitoring node and a flow profiler; The spatiotemporal feature fusion module generates the dynamic state fingerprint matrix, which is configured as follows: the acoustic vibration sensor array, distributed turbidity monitoring node and flow profiler included in the multi-source perception network module are represented as nodes in a graph network, and the node attributes include: For a node corresponding to a monitoring point of the acoustic vibration sensor array: its attribute is 1 / 3 octave spectrum energy distribution; For the node corresponding to the distributed turbidity monitoring node: its attributes include the current turbidity value and the time change rate of turbidity concentration calculated based on a preset time window; For a node corresponding to the velocity profiler: its attributes include a three-dimensional velocity vector and information on the angle between the water flow vector of the node and at least one adjacent node; The feature association weights are calculated based on the spatial distance between nodes, the relative direction of water flow and the signal propagation characteristics, and the weights are used to aggregate the features of adjacent nodes through the graph attention mechanism; a temporal convolutional network is used to process the node timing features, and the dynamic state fingerprint matrix that integrates the spatiotemporal information is output. The dimension of the dynamic state fingerprint matrix is ​​used to encode the comprehensive state information of the river.

3. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: The adaptive baseline monitoring module triggers an abnormal alarm based on the dynamic state fingerprint matrix. Its working principle is: Using historical monitoring data from the period without sand mining interference, a baseline fingerprint database that distinguishes different seasons and meteorological conditions was established; Applying a sliding time window and adjusting the update rate accordingly according to the rate of change of real-time hydrological conditions to dynamically update the statistical characteristics of the benchmark fingerprint library, including the mean and covariance matrix; Calculate the Mahalanobis distance between the currently acquired real-time state fingerprint and the selected benchmark library; The abnormal alarm is triggered when the Mahalanobis distance exceeds a preset statistical significance threshold and the following two collaborative verification conditions are met at the same time: first, a statistically significant deviation of the acoustic feature is detected within a specific preset frequency band; Second, the observed spatial distribution direction of the turbidity gradient is inconsistent with the normal diffusion pattern predicted based on the water flow diffusion model.

4. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: The multi-task parsing module outputs the analysis results of abnormal disturbance events. It adopts a multi-branch parallel neural network architecture. The multi-branch parallel neural network architecture technology includes a branch specifically for processing time-frequency features, which is implemented using a convolutional neural network; and a branch for processing spatial propagation features based on the sensor topology structure, which is implemented using a graph neural network. And use the attention mechanism to handle the correlation between cross-modal features; The convolution processing branch adopts a depth-separable convolution structure to extract local sensitive features and reduce the computational burden; The graph neural network processing branch is used to capture the propagation path information of abnormal signals in the monitoring network based on the connection relationship between sensors; The attention mechanism dynamically adjusts the weights of features from different modalities or different processing branches, enabling the multi-branch parallel neural network architecture to focus on the information most critical to the current parsing task; Furthermore, the multi-branch parallel neural network architecture is designed to be trained by adopting a joint loss function to simultaneously optimize the classification task for distinguishing the types of abnormal disturbance events and the levels of environmental effects and the regression task for estimating the work intensity.

5. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: The three-dimensional visualization warning module generates a warning view based on the received abnormal disturbance event analysis results and related data, and its generation logic includes: Apply nonlinear dimensionality reduction technology to map high-dimensional state fingerprint data or anomaly scores into two-dimensional space, forming a heat map representing the health status of the river section; Based on the real-time monitored turbidity gradient change rate data, a turbidity contour map is dynamically drawn, and the density of the contour lines is used to reflect the rate and range of abnormal suspended matter diffusion; Driving a suspended sediment particle simulation system, wherein the motion trajectory of particles in the suspended sediment particle simulation system is determined based on a three-dimensional water flow vector field acquired in real time, and the visual attributes of the particles are associated with turbidity concentration information associated with their location; Run the spatial topology analysis program to calculate in real time the shortest water flow path distance between the analyzed abnormal operation point location and the calibrated ecological sensitive area, and trigger different levels of early warning indications based on the shortest water flow path and the intensity of the abnormal disturbance event.

6. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: It also includes a dynamic knowledge management module to help improve the accuracy of analysis. Its operating mechanism is: Continuously store the key features of historical abnormal disturbance events and the ecological and environmental impact monitoring data actually observed after the abnormal disturbance events occurred; When a new abnormal disturbance event is detected, the dynamic time warping algorithm or similar sequence matching technology is used to compare the characteristic sequence of the new abnormal disturbance event with the historical pattern library, and prioritize the association with similar patterns that are historically known to have higher ecological risks; Based on the matched historical abnormal disturbance event patterns and their known consequence information, the confidence score of the output results of the currently running multi-task parsing module is dynamically adjusted or the results are corrected a posteriori.

7. The intelligent sand mining supervision system based on multi-source data fusion according to claim 1 is characterized in that: It further includes an adaptive optimization module to maintain and improve system monitoring performance, which is achieved by: Continuously track the system's early warning performance indicators, including false alarm rate and missed alarm rate. Once the performance is detected to be below the preset standard, it will automatically trigger the retraining process of the baseline model in the adaptive baseline monitoring module or the parsing model in the multi-task parsing module; Automatically integrate real sand mining events and their detailed attribute information verified by manual confirmation or on-site verification into the sample database used for model training; In addition, based on the results of the frequency analysis of the geographical distribution of historical alarm events, the sampling strategy of sensors deployed in the multi-source perception network is dynamically adjusted. The adjustment may include increasing the sampling frequency, monitoring sensitivity or data transmission priority of relevant sensors located in high-frequency alarm areas to achieve enhanced monitoring of specific areas.

8. The intelligent sand mining supervision system based on multi-source data fusion according to claim 2 is characterized in that: The deployment mode of the sensors in the multi-source perception network module is configured as follows: Laying a distributed optical fiber vibration sensor array along the riverbed area of ​​the target river section, wherein the distributed optical fiber vibration sensor array is suitable for monitoring underwater mechanical vibration signals within a specific frequency range, wherein the frequency range covers the expected characteristic frequency band of sand mining equipment activity; High-frequency turbidity monitoring nodes are deployed to form a gradient tracking network. The spatial spacing of the nodes is dynamically adjusted according to the geometric characteristics of the river channel at the location, and the minimum distance between adjacent nodes is set to no more than 1 / 5 of the river channel width at that location.

Images