Multi-modal sensing and edge intelligence water area monitoring unmanned ship and graded early warning system

Abstract

The invention discloses a multi-modal perception and edge intelligence water area monitoring unmanned ship and a grading early warning system, a grading early warning mechanism is realized based on an artificial neural network (ANN) water quality model through a sensor array formed by a water quality sensor, a hyperspectral camera and a multi-beam wave depth finder, the early warning mechanism is triggered if abnormity is found after data analysis, and the water area monitoring unmanned ship can be used for monitoring the water area. The time from data acquisition to early warning triggering is less than 3 seconds. The water area monitoring and patrol operation and maintenance cost can be reduced, and the system is suitable for real-time environment monitoring and pollution early warning of rivers, lakes and offshore water areas.

Description

Technical Field

[0001] The present invention relates to the field of water area monitoring, and in particular to an unmanned water area monitoring vessel and a graded early warning system with multimodal perception and edge intelligence. Technical Background

[0002] Current water monitoring technologies generally employ a single-sensor detection model, relying on discrete parameter collection and lacking the ability to analyze the spatiotemporal correlations of multi-source data. Traditional monitoring vessels, constrained by centralized data processing architectures, must transmit raw data back to cloud servers for pollution assessment. This results in decision delays often exceeding 10 minutes, making it difficult to meet the real-time response requirements for sudden pollution incidents. Existing early warning systems often utilize a single threshold trigger mechanism, failing to establish a tiered response strategy based on pollution type and spread rate, which can easily lead to misallocation of emergency resources. For example, patent number 202411119703.6, named as a tap water detection method, system and storage medium, obtains current turbidity data, pH data, water temperature data and dissolved oxygen concentration data, and inputs them into a standard water quality detection model to obtain a water quality detection score; when it is determined that the water quality detection score is less than a preset score threshold, an alarm instruction is generated and sent to the control end, so that the control end closes the corresponding abnormal water source according to the alarm instruction; the standard water quality detection model training process includes: obtaining historical turbidity data, pH data, water temperature data and dissolved oxygen concentration data and performing data preprocessing and PCA dimensionality reduction operations to obtain reduced dimensionality data; based on the reduced dimensionality data and preset standard water quality data as input data, an initial water quality detection model is constructed based on the particle swarm algorithm, and training is performed according to the membership matrix and the possibility partition matrix to obtain a standard water quality detection model. Currently, there are also various detection methods to predict water quality, such as patent number 202410596742.9, named VMD-based bidirectional long short-term memory network water quality prediction method and device, VMD-based bidirectional long short-term memory network water quality prediction method; collect historical water quality data, and construct a water quality data set based on the historical water quality data; preprocess the water quality data set to generate preprocessed water quality data; set the parameters of VDM decomposition, use the VDM algorithm to decompose the preprocessed water quality data, generate a set of IMF data, and arrange each IMF data from high frequency to low frequency according to frequency; filter and fuse the arranged IMF data; build a bidirectional LSTM network structure, and use IMF data to construct an LSTM water quality prediction model; predict future water quality through the LSTM water quality prediction model. Summary of the Invention

[0003] Based on the above problems, the present invention proposes a water area monitoring unmanned boat and graded early warning system with multimodal perception and edge intelligence. The sensor array formed by water quality sensors, hyperspectral cameras and multi-beam echo sounders, the water quality model based on artificial neural network (ANN), and the surface unmanned boat system with graded early warning mechanism take less than 3 seconds from data collection to early warning triggering. The present invention can reduce the cost of water area monitoring and patrol operation and maintenance, and is suitable for real-time environmental monitoring and pollution early warning of rivers, lakes and offshore waters.

[0004] The technical solution adopted is: a multimodal perception and edge intelligence water area monitoring unmanned boat and graded early warning system, including a multimodal perception subsystem, an artificial neural network water quality early warning subsystem and a graded response subsystem; the water quality parameters and spectral images obtained by the multimodal perception subsystem are aligned in time and space by establishing a unified time and space coordinate system through GPS / INS, and the water quality parameters and spectral images are fused using a cross-attention mechanism. The pollution is judged through the artificial neural network water quality early warning subsystem to achieve a graded response to pollution.

[0005] The multimodal sensing subsystem includes a water quality sensor, a hyperspectral camera and a multibeam echo sounder; the hyperspectral camera is installed at the bow of the unmanned boat, and the multibeam echo sounder is installed at the bottom of the unmanned boat; a water sampling auxiliary device is installed at the stern of the unmanned boat; a water sampling mechanism, a water sample box and an emergency reagent cabin are installed in the cabin of the unmanned boat, and a water quality sensor is installed in the water sample box. The water sampling mechanism stores the collected water samples in the water sample box and detects them through the water quality sensor.

[0006] The artificial neural network water quality early warning subsystem uses a dual-stream parallel architecture to simultaneously process water quality parameters obtained by water quality sensors and hyperspectral data obtained by hyperspectral cameras. The model architecture includes an input layer, a hidden layer, and an output layer.

[0007] The input layer includes water quality parameters obtained by the water quality sensor and hyperspectral data obtained by the hyperspectral camera;

[0008] The hidden layer uses the 1D-CNN time series processing module and the ResNet image feature extraction module;

[0009] The output layer uses the Softmax classifier to generate a three-level warning;

[0010] There are three levels of warning: Level I, which is divided into Level I, Level II and Level III; Level I, if a single indicator exceeds the threshold by 10%, high-density sampling will be initiated; Level II, if the pollution index is >0.7, sound and light alarms + direct data reporting will be implemented; Level III, if toxic substances are detected, neutralizers will be released and the aircraft will return home autonomously.

[0011] Furthermore, the water quality sensor includes a pH sensor, a turbidity sensor, a dissolved oxygen sensor, an ammonia nitrogen sensor and a conductivity sensor.

[0012] Furthermore, the artificial neural network water quality early warning subsystem,model architecture includes input layer, hidden layer and output layer;

[0013] The input layer includes water quality parameters obtained by the water quality sensor and hyperspectral data obtained by the hyperspectral camera.

[0014] The hidden layer uses the 1D-CNN time series processing module and the ResNet image feature extraction module;

[0015] The 1D-CNN time series processing module uses the water quality parameters obtained by the water quality sensor as the input structure. The dilated convolution chain uses 3 layers of 1D convolution, expanding the receptive field to 29 time steps. The adaptive gating mechanism then dynamically adjusts the feature weights of each dimension and outputs a 256-dimensional time series feature vector.

[0016] The ResNet image feature extraction module uses hyperspectral data obtained by the hyperspectral camera as input, adopts spectral response normalization, introduces a band attention mechanism to address the differences in response between different bands, and uses a strided convolutional layer after the fourth residual block to output a 512-dimensional image feature vector, preserving spatial details.

[0017] The output layer uses a Softmax classifier to generate a three-level warning, concatenating the 256-dimensional time series feature vector and the 512-dimensional image feature vector into a 768-dimensional feature vector. A primary classifier is used locally, and through two layers of fully connected layers (256→128), a preliminary probability P' is output. The ultimate classifier is used in the cloud, introducing a temporal convolutional layer (TCN) to fuse historical data. TensorRT is used to convert the FP32 model to INT8 format. 80% of the computation is performed locally, and 20% of the complex operations are supplemented in the cloud for deployment.

[0018] Furthermore, binocular vision is installed on the unmanned ship, and the pollution source is located through binocular vision positioning and pollution diffusion simulation. The feature points of the binocular image are extracted through the SIFT algorithm, and the disparity map is calculated for feature matching. The parallax is used to obtain depth information, and the depth information is fused with GPS / INS data to locate the pollution source coordinates in the UTM coordinate system.

[0019] A multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system, the working steps are as follows:

[0020] Step 1: At a certain working point, the hyperspectral camera on the unmanned vessel obtains hyperspectral data; the water sample collected by the water sampling mechanism in the cabin of the unmanned vessel is placed in a water sample box, and the water quality sensor in the water sample box obtains water quality parameters. Specifically, the water quality sensors are pH sensors, turbidity sensors, dissolved oxygen sensors, ammonia nitrogen sensors and conductivity sensors, and the water quality parameters obtained are pH value, turbidity, DO, NH3-N, and EC;

[0021] Step 2: Perform PCA dimensionality reduction on the hyperspectral data obtained in step 1 to obtain the spectral features H after dimensionality reduction. proj :

[0022] 1) Convert the original digital quantization value DN into the radiation brightness L through radiation correction. The radiation correction formula is:

[0023]

[0024] In formula (1), L is the radiant brightness, DN is the original digital quantization value, G = 120 is the system gain,

[0025] R cal =0.12W·m -2 ·sr -1 nm -1 is the radiation calibration factor;

[0026] 2) Calculate the water surface reflectivity ρ using the atmospheric correction FLAASH model toa ,

[0027] Atmospheric correction formula FLAASH model,

[0028]

[0029] In formula (2), the distance between the sun and the earth is d = 1.0 AU, the cosine of the solar zenith angle μ0 = 0.866, and the solar irradiance Es = 1367 W·m -2 , L is the radiance;

[0030] 3) Hyperspectral data matrix: H hyper =[ρ toa,400 ,ρ toa,401 ,...,ρ toa,1000 ] T ∈R 270 (3)

[0031] From H hyper Extract the water surface reflectance R680=ρ in the 680nm band toa ,680,

[0032] 4) Calculate its normalized difference vegetation index variant NDWI chl , by combining bands to enhance chlorophyll a signals, optimizing the key features of 270-dimensional spectral data, and improving principal component analysis results,

[0033] Normalized Difference Vegetation Index variants:

[0034]

[0035] In formula (4), R680 and R750 are extracted from formula (3);

[0036] R680=ρ toa ,680,R750=ρ toa ,750;

[0037] 5) Using principal component analysis PCA, H hyper Dimensionality reduction from 270 dimensions to 10 dimensions:

[0038] H hyper Standardization to obtain H norm , standardization formula:

[0039]

[0040] In formula (5), μ is the spectral data matrix H hyper The mean vector of each band, σ is the spectral data matrix H hyper The standard deviation vector of each band;

[0041] Calculate the covariance matrix C, the covariance matrix formula is:

[0042]

[0043] In formula (6), T is the transposed sign, N is the number of samples, that is, the total number of samples of hyperspectral data;

[0044] 6) Solve the eigenvalues of the covariance matrix λ1≥λ2≥…≥λ270 and the corresponding eigenvectors v1,v2,…,v270, and take the first 10 principal component eigenvectors The cumulative variance contribution rate is ≥95%, and the dimensionality reduction mapping is performed to 10 dimensions to obtain Hproj ,

[0045] Dimensionality reduction mapping formula: In formula (7), W PCA is the principal component eigenvector, and the spectral feature H after dimension reduction is obtained. proj ;

[0046] Step 3: The water quality parameters obtained in step 1 are converted into 12-dimensional time series features. The single time step raw data of the pH sensor, turbidity sensor, dissolved oxygen sensor, ammonia nitrogen sensor and conductivity sensor are 5-dimensional vectors Xt.

[0047]

[0048] In formula (8), T is the transposed symbol, t is the time step, which represents the current sampling moment (such as t = 1, 2, ...). Then, a sliding window with 3 time steps of 90 seconds is used to expand the time window to obtain the data matrix Ft.

[0049]

[0050] Then perform a 3-point moving average on each parameter and generate a smoothed data matrix.

[0051]

[0052] Expand the data matrix by column into 15-dimensional time series features,

[0053]

[0054] Redundant parameters were screened by the Pearson correlation coefficient. The correlation coefficient between turbidity and conductivity was r = 0.87, p < 0.01. Conductivity was retained and turbidity was removed to avoid redundancy. After removing turbidity, the 15-dimensional time series feature was transformed into a 12-dimensional time series feature, which is a 3-time step data F containing 4 parameters: PH, DO, NH3-N, and EC. seq , in

[0055]

[0056] Step 4: Use three layers of dilated 1D convolution with kernel = (5, 1) and dilation_rate = [1, 2, 4] to expand the receptive field of the time series data to 29 time steps. The dilation_rate is 1, 2, and 4, respectively. The three layers of dilated convolution gradually expand the receptive field through the exponential dilation rate to cover short-term, medium-term, and long-term time series features.

[0057] The calculation formula of single-layer receptive field is:

[0058] RF=kernel_size+(kernel_size-1)×(dilation_rate-1) (14)

[0059] Cumulative receptive field calculation results:

[0060] Number of layers: 1, dilation_rate: 1, single layer receptive field: 5, cumulative receptive field: 5;

[0061] Number of layers: 2, dilation_rate: 3, single layer receptive field: 9, cumulative receptive field: 13;

[0062] Number of layers: 3, dilation_rate: 4, single layer receptive field: 17, cumulative receptive field: 29;

[0063] Input the spectral feature Hproj after dimension reduction in step 2, map it to the query space through the fully connected layer, and obtain the query vector Perform 3 layers of dilated 1D convolution on the time series water quality parameters to extract multi-scale time series features. Then, after dilated convolution, the 12-dimensional time series features are mapped to generate key K water quality and value V water quality. The feature dimension is dk, which matches Q.

[0064]

[0065] K 水质 =W K DilatedConv(X 水质 ) (16)

[0066] V 水质 =W v DilatedConv(X 水质 ) (17)

[0067] is the learnable weight matrix; W Q is the query weight matrix, which is used to map the spectral features to the query space, W v is the value weight matrix, which is used to extract the value representation of time series features; 光谱 , K 水质 、V 水质 Divide into 8 sub-heads to calculate single-head attention,

[0068]

[0069] Scale the dot product attention to get the attention weights generated by each sub-head Then concatenate the outputs of the 8 sub-heads along the feature dimension to finally obtain the output dimension MultiHead(Q,K,V).

[0070]

[0071] MultiHead(Q,K,V)=Concat(Attention1,…,Attention8)W O (20)

[0072] Step 5: Enhance local features by dilating the convolution chain.

[0073] The dilated convolution chain structure is designed, and the dilation=1 in the first layer is used to capture the local features of adjacent time steps, so that the output dimension is changed from becomes

[0074] Output=DilatedConv(Attention(Q,K,V)) (21)

[0075] Among them, the expansion convolution operation of each layer is defined as

[0076] DilatedConv(X)=ReLU(W conv*dil X+b) (22)

[0077] *dil is the dilated convolution operation, W conv is the convolution kernel weight, b is the bias term;

[0078] Step 6: Fuse the spectral features with the enhanced temporal features for feature concatenation to construct a joint feature vector;

[0079] The spectral features after PCA dimensionality reduction The 258-dimensional vector output by the dilated convolution chain Directly splice along the channel dimension to obtain the spliced feature vector F

[0080]

[0081] Perform layer normalization on the concatenated feature vector F to eliminate the dimensionality differences between different features, accelerate model convergence and improve generalization ability;

[0082] Normalize the concatenated feature vector F.

[0083] Where: ∈ is the minimum value, set to 10 -5 ;Prevent division by zero error;

[0084] Calculate the mean μ and variance σ by channel dimension 2 ,

[0085]

[0086] Introducing learnable parameters to scale γ and offset β to output Outputi,

[0087]

[0088] γ and

[0089] Step 7: Design the model architecture of the artificial neural network, using a dual-stream parallel branching structure; the water quality parameters obtained by the water quality sensor and the hyperspectral data obtained by the hyperspectral camera are used as the input layer; the hidden layer uses the 1D-CNN time series processing module and the ResNet image feature extraction module;

[0090] The 1D-CNN time series processing module uses the water quality parameters obtained by the water quality sensor as the input structure. The dilated convolution chain uses 3 layers of 1D convolution, expanding the receptive field to 29 time steps. The adaptive gating mechanism then dynamically adjusts the feature weights of each dimension and outputs a 256-dimensional time series feature vector.

[0091] The structural weighted feature of the adaptive gating mechanism is X gate ,

[0092] X gate =σ(G)☉X (28)

[0093] б is the Siqmoid function, ⊙ is the element-wise multiplication;

[0094] Image feature extraction module, hyperspectral data obtained by hyperspectral camera The input data is H×W=640×640, and spectral response normalization is used. A band attention mechanism is introduced to address the differences in responses between different bands. A strided convolutional layer is used after the fourth residual block to output a 512-dimensional image feature vector to preserve spatial details.

[0095] The ResNet module modifies its band attention mechanism and generates channel descriptors by global average pooling GAP on the input spectral image. The fully connected layer and Sigmoid generate band weights W∈[0,1] 270 , W=σ(W2·ReLU(W1·S)) (29)

[0096] (29)W2 represents the weight matrix of the fully connected layer, which is used to generate the band attention weight; ReLU is the piecewise linear activation function; S is the channel descriptor;

[0097] The reweighted output is H att , H att =W⊙H hyper (30),

[0098] The residual block design has a structure in which each residual block contains two convolutional layers with 3×3 kernels and skip connections, and introduces batch normalization BatchNorm and ReLU activation, so that it can output 512-dimensional image features after passing through four residual blocks.

[0099] At the output layer, a Softmax classifier is used to generate a three-level warning, concatenating the 256-dimensional time series feature vector and the 512-dimensional image feature vector into a 768-dimensional feature vector;

[0100] When performing feature splicing, the 256-dimensional time series feature vector and the 512-dimensional image feature vector are spliced along the channel dimension. Perform layer normalization LayerNorm on the concatenated feature vector.

[0101]

[0102] A primary classifier is used locally. The fully connected layer 1 activates the function ReLU by converting 768 dimensions to 256 dimensions. The fully connected layer 2 activates the function ReLU by converting 256 dimensions to 128 dimensions, and outputs the preliminary probability P'.

[0103]

[0104] Where b1 is the bias term of the first fully connected layer of the primary classifier, and b2 is the bias term of the second fully connected layer of the primary classifier;

[0105] The ultimate classifier is used in the cloud, and a temporal convolutional layer (TCN) is introduced. Through four layers of causal dilation convolution, with a dilation_rate of [1, 2, 4, 8], 168 hours of historical data is covered.

[0106] Through P' and historical data P 历史 Calculate P, P 历史 Sliding window length = 24 hours × 7 days,

[0107] P=σ(W2·TCN(W1·[P′,′,P 历史 ])+b) (34),

[0108] Step 8: Use TensorRT to convert the FP32 model to INT8 format, perform 80% of the computation locally, and add 20% of the complex computation in the cloud for deployment;

[0109] A representative dataset was used to calculate the dynamic range of activation values and generate a quantization table. The FP32 weights were then mapped to the INT8 range of -128 to 127 for calibration. TensorRT Builder was then configured, with the optimization profiles fp16_mode = False, int8_mode = True set. The calibrated quantization table was loaded, and the INT8 engine file .engine was generated. Finally, a collaborative computing strategy was adopted, with the Jetson AGX Orin at the edge performing 80% of the computation and the cloud supplementing the remaining 20% of the complex operations. This reduced the size of the ANN water quality model from 300MB of FP32 to 120MB, a 60% reduction. Furthermore, the inference latency on the Jetson AGX Orin at the edge was reduced from 5.2 seconds to 2.8 seconds.

[0110] Step 9: Calculation of pollution index and setting of graded warnings,

[0111] Calculate the pollution index PI, Level I warning is defined as a potential risk: when any indicator exceeds the threshold by 10%, high-density sampling is initiated, and the sampling interval is reduced from 30 seconds to 5 seconds;

[0112] Level II warning, defined as moderate pollution: When PI>0.7, sound and light alarms are activated, and data is directly reported to the supervision platform;

[0113] Level III warning, defined as severe pollution: When toxic substances are detected, the neutralizer is released at 5L / min and the aircraft returns to port autonomously;

[0114] Step 10: Pollution source positioning: binocular vision positioning on the unmanned vessel and pollution diffusion simulation are used to locate the pollution source. The SIFT algorithm is used to extract binocular image feature points, and the disparity map is calculated for feature matching. The depth information is obtained through the disparity, and the depth information is integrated with the GPS / INS data to locate the pollution source coordinates in the UTM coordinate system.

[0115] Where: GSD = 0.1m is the ground sampling distance, (X0, Y0) is the coordinate of the image center;

[0116] The major axis of the error ellipse is ≤0.6m, and the pollution diffusion simulation is carried out using the advection-diffusion equation.

[0117]

[0118] Diffusion coefficient Dx = Dy = 0.3 m 2 / s is the diffusion coefficient, and flow velocity vx = 0.8m / s is the flow velocity.

[0119] Furthermore, the Level II warning response: triggering sound and light alarms, the unmanned boat whistles and flashes warning lights; uploading pollution coordinates and diffusion prediction maps via NB-IoT; starting propeller disturbance oxygenation, with a propeller speed of 2000 rpm and an oxygenation rate of 5 mg / L·min.

[0120] Furthermore, the Level III warning response: the concentration of benzene series detected is greater than the threshold, triggering a Level III warning; the neutralizer NaOH solution is released at a rate of 5L / min; the return route is planned, returning to the base station to charge and upload complete data.

[0121] This invention uses multimodal data fusion to improve pollution identification accuracy, achieving an algae identification accuracy of 92.3% ± 1.7%. It also optimizes system response speed, reducing the time from data acquisition to alert triggering to less than 3 seconds. It also reduces operational costs, with predictive maintenance reducing equipment downtime by 30%. Compared to traditional LSTM water quality prediction models, the TCN time-domain convolutional network employed in this solution reduces computational latency by 42%.

[0122] The multimodal sensing subsystem of the present invention includes a water quality sensor, a hyperspectral camera, and a multibeam echo sounder. The water quality sensor can be increased or changed as needed based on historical water quality conditions. The water body is photographed by the hyperspectral camera, and the water quality and algae growth conditions are inverted based on the reflected band information. The multibeam echo sounder performs underwater topography measurement while working, thereby drawing a real-time underwater topography map.

[0123] The present invention has a wide range of applications and can be equipped with various required sensors to detect and analyze water quality according to actual water quality conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] Figure 1 The structure of the unmanned boat in the present invention;

[0125] Figure 2 This is the overall architecture diagram of the system of the present invention;

[0126] Figure 3 This is a flow chart of the multimodal data fusion algorithm in the present invention;

[0127] Figure 4 This is a structural diagram of the artificial neural network ANNS model in the present invention;

[0128] Figure 5 This is the state transition diagram of the graded warning in the present invention. DETAILED DESCRIPTION

[0129] The present invention will be further described in detail below with reference to the accompanying drawings and through examples. The following examples are intended to explain the present invention but the present invention is not limited to the following examples.

[0130] See also Figures 1 to 5 As shown, a multimodal perception and edge intelligence water area monitoring unmanned boat and graded warning system includes a multimodal perception subsystem, an artificial neural network water quality warning subsystem and a graded response subsystem; the water quality parameters and spectral images obtained by the multimodal perception subsystem are aligned in time and space by establishing a unified time and space coordinate system through GPS / INS, and the water quality parameters and spectral images are fused using a cross-attention mechanism. The artificial neural network water quality warning subsystem determines pollution and realizes a graded response to pollution.

[0131] The multimodal sensing subsystem includes a water quality sensor, a hyperspectral camera 5, and a multibeam echo sounder 2. The hyperspectral camera 5 is installed at the bow of the unmanned boat. The spectral range of the hyperspectral camera is 400-1000nm, and the number of bands is 270. It can obtain hyperspectral cube data with a spatial resolution of 0.1m at a height of 50cm from the water surface. The multibeam echo sounder 2 is installed at the bottom of the unmanned boat. A water sampling auxiliary device 1 is installed at the stern of the unmanned boat. A water sampling mechanism 6 and a water sample box 7 are installed in the cabin of the unmanned boat. The water sample box 7 is equipped with a water quality sensor. The water sampling mechanism 6 stores the collected water samples in the sample storage bottles 8. The water samples in the sample storage bottles 8 enter the water sample box 7 and are detected by the water quality sensor to obtain water quality parameters. The working steps are as follows:

[0132] Step 1: At a certain working point, the hyperspectral camera on the unmanned vessel obtains hyperspectral data; the water sample collected by the water sampling mechanism in the cabin of the unmanned vessel is placed in a water sample box, and the water quality parameters are obtained by the water quality sensors in the water sample box. Specifically, the water quality sensors are pH sensors, turbidity sensors, dissolved oxygen sensors, ammonia nitrogen sensors and conductivity sensors. The water quality parameters obtained are pH value, turbidity, DO, NH3-N, and EC;

[0133] Step 2: Perform PCA dimensionality reduction on the hyperspectral data obtained in step 1 to obtain the spectral features H after dimensionality reduction. proj :

[0134] 1) The original digital quantization value DN is converted into the radiation brightness L through radiation correction. The radiation correction formula is as follows:

[0135]

[0136] In formula (1), L is the radiant brightness, DN is the original digital quantization value, G = 120 is the system gain, R cal =0.12W·m -2 ·sr -1 nm -1 is the radiation calibration factor;

[0137] 2) Calculate the water surface reflectivity ρ using the atmospheric correction FLAASH model toa ,

[0138] Atmospheric correction formula FLAASH model,

[0139] In formula (2), the distance between the sun and the earth is d = 1.0 AU, the cosine of the solar zenith angle μ0 = 0.866, and the solar irradiance Es = 1367 W·m -2 , L is the radiance;

[0140] 3) Hyperspectral data matrix: H hyper =[ρtoa,400 ,ρ toa,401 ,...,ρ toa,1000 ] T ∈R 270 (3)

[0141] From H hyper Extract the water surface reflectance R680=ρ in the 680nm band toa ,680;

[0142] 4) Calculate its normalized difference vegetation index variant NDWI chl , by combining bands to enhance chlorophyll a signals, optimize the key features of 270-dimensional spectral data, and improve the principal component analysis effect. R680 is the sensitive band of chlorophyll a, and R750 is the near-infrared band, which enhances chlorophyll signals;

[0143] Normalized Difference Vegetation Index variants:

[0144]

[0145] In formula (4), R680 and R750 are extracted from formula (3); R680 = ρ toa ,680,R750=ρ toa ,750;

[0146] 5) Using principal component analysis PCA, H hyper Dimensionality reduction from 270 dimensions to 10 dimensions:

[0147] H hyper Standardization to obtain H norm , standardization formula:

[0148]

[0149] In formula (5), μ is the spectral data matrix H hyper The mean vector of each band, σ is the spectral data matrix H hyper The standard deviation vector of each band;

[0150] Calculate the covariance matrix C, the covariance matrix formula is:

[0151]

[0152] In formula (6), T is the transposed sign, N is the number of samples, that is, the total number of samples of hyperspectral data;

[0153] 6) Solve the eigenvalues of the covariance matrix λ1≥λ2≥…≥λ270 and the corresponding eigenvectors v1,v2,…,v270, and take the first 10 principal component eigenvectors The cumulative variance contribution rate is ≥95%, and the dimension is reduced to 10 dimensions to obtain Hproj ,

[0154] Dimensionality reduction mapping formula:

[0155] In formula (7), W PCA The principal component eigenvector is used to obtain the spectral feature H after dimension reduction. proj ;

[0156] Step 3: The water quality parameters obtained in step 1 are converted into 12-dimensional time series features. The single time step raw data of the pH sensor, turbidity sensor, dissolved oxygen sensor, ammonia nitrogen sensor and conductivity sensor are 5-dimensional vectors Xt.

[0157] In formula (8), T is the transposition symbol, and t is the time step, which represents the current sampling moment, for example, t = 1, 2, ...,

[0158] Then, a sliding window with 3 time steps of 90 seconds is used to expand the time window to obtain the data matrix Ft.

[0159]

[0160] Then perform a 3-point moving average on each parameter and generate a smoothed data matrix.

[0161]

[0162] Expand the data matrix by column into 15-dimensional time series features,

[0163]

[0164] Redundant parameters were screened by the Pearson correlation coefficient. The correlation coefficient between turbidity and conductivity was r = 0.87, p < 0.01. Conductivity was retained and turbidity was removed to avoid redundancy. After removing turbidity, the 15-dimensional time series feature was transformed into a 12-dimensional time series feature, which is a 3-time step data F containing 4 parameters: PH, DO, NH3-N, and EC. seq , in

[0165]

[0166] Step 4: Use three layers of dilated 1D convolution with kernel = (5, 1) and dilation_rate = [1, 2, 4] to expand the receptive field of the time series data to 29 time steps. The dilation_rate is 1, 2, and 4, respectively. The three layers of dilated convolution gradually expand the receptive field through the exponential dilation rate to cover short-term, medium-term, and long-term time series features.

[0167] The calculation formula of single-layer receptive field is:

[0168] RF=kernel_size+(kernel_size-1)×(dilation_rate-1) (14)

[0169] Cumulative receptive field calculation results:

[0170] Number of layers: 1, dilation_rate: 1, single layer receptive field: 5, cumulative receptive field: 5;

[0171] Number of layers: 2, dilation_rate: 3, single layer receptive field: 9, cumulative receptive field: 13;

[0172] Number of layers: 3, dilation_rate: 4, single layer receptive field: 17, cumulative receptive field: 29;

[0173] Input the spectral feature Hproj after dimension reduction in step 2, map it to the query space through the fully connected layer, and obtain the query vector Perform 3 layers of dilated 1D convolution on the time series water quality parameters to extract multi-scale time series features. Then, after dilated convolution, the 12-dimensional time series features are mapped to generate key K water quality and value V water quality. The feature dimension is dk, which matches Q.

[0174]

[0175] K 水质 =W K DilatedConv(X 水质 ) (16)

[0176] V 水质 =W v DilatedConv(X 水质 ) (17)

[0177] W K ,W V ∈ is the learnable weight matrix; W Q Query is the query weight matrix, which is used to map the spectral features to the query space. Wv is the value weight matrix, which is used to extract the value representation of the time series features.

[0178] Q 光谱 , K 水质 、V 水质 Divide into 8 sub-heads to calculate single-head attention,

[0179]

[0180] Scale the dot product attention to get the attention weights generated by each sub-head Then concatenate the outputs of the 8 sub-heads along the feature dimension to finally obtain the output dimension MultiHead(Q,K,V).

[0181]

[0182] MultiHead(Q,K,V)=Concat(Attention1,...,Attention8)W O (20)

[0183] Step 5: Enhance local features by dilating the convolution chain.

[0184] The dilated convolution chain structure is designed, and the dilation=1 in the first layer is used to capture the local features of adjacent time steps, so that the output dimension is changed from becomes

[0185] Output=DilatedConv(Attention(Q,K,V)) (21)

[0186] Among them, the expansion convolution operation of each layer is defined as

[0187] DilatedConv(X)=ReLU(W conv*dil X+b) (22)

[0188] *dil is the dilated convolution operation, W conv is the convolution kernel weight, b is the bias term;

[0189] Step 6: Fuse the spectral features with the enhanced temporal features for feature concatenation to construct a joint feature vector;

[0190] The spectral features after PCA dimensionality reduction The 258-dimensional vector output by the dilated convolution chain Directly splice along the channel dimension to obtain the spliced feature vector F,

[0191]

[0192] Perform layer normalization on the concatenated feature vector F to eliminate the dimensionality differences between different features, accelerate model convergence and improve generalization ability;

[0193] Normalize the concatenated feature vector F.

[0194] Where: ∈ is the minimum value, set to 10 -5 ;Prevent division by zero error;

[0195] Calculate the mean μ and variance σ by channel dimension2 ,

[0196]

[0197] Introducing learnable parameters to scale γ and offset β to output Outputi,

[0198]

[0199] γ and

[0200] Step 7: Design the model architecture of the artificial neural network, using a dual-stream parallel branching structure; the water quality parameters obtained by the water quality sensor and the hyperspectral data obtained by the hyperspectral camera are used as the input layer; the hidden layer uses the 1D-CNN time series processing module and the ResNet image feature extraction module;

[0201] The 1D-CNN time series processing module uses the water quality parameters obtained by the water quality sensor as the input structure. The dilated convolution chain uses 3 layers of 1D convolution, expanding the receptive field to 29 time steps. The adaptive gating mechanism then dynamically adjusts the feature weights of each dimension and outputs a 256-dimensional time series feature vector.

[0202] The structural weighted feature of the adaptive gating mechanism is X gate ,

[0203] X gate =σ(G)☉X (28)

[0204] б is the Siqmoid function, ⊙ is the element-wise multiplication;

[0205] Image feature extraction module, hyperspectral data obtained by hyperspectral camera The input data is H×W=640×640, and spectral response normalization is used. A band attention mechanism is introduced to address the differences in responses between different bands. A strided convolutional layer is used after the fourth residual block to output a 512-dimensional image feature vector to preserve spatial details.

[0206] The ResNet module modifies its band attention mechanism and generates channel descriptors by global average pooling GAP on the input spectral image. The fully connected layer and Sigmoid generate band weights W∈[0,1] 270 , W=σ(W2·ReLU(W1·S)) (29)

[0207] (29)W2 represents the weight matrix of the fully connected layer, which is used to generate the band attention weight; ReLU is the piecewise linear activation function; S is the channel descriptor;

[0208] The reweighted output is Hatt , H att =W⊙H hyper (30),

[0209] The residual block design has a structure in which each residual block contains two convolutional layers with 3×3 kernels and skip connections, and introduces batch normalization BatchNorm and ReLU activation, so that it can output 512-dimensional image features after passing through four residual blocks.

[0210] At the output layer, a Softmax classifier is used to generate a three-level warning, concatenating the 256-dimensional time series feature vector and the 512-dimensional image feature vector into a 768-dimensional feature vector;

[0211] When performing feature splicing, the 256-dimensional time series feature vector and the 512-dimensional image feature vector are spliced along the channel dimension. Perform layer normalization LayerNorm on the concatenated feature vector.

[0212]

[0213] A primary classifier is used locally. The fully connected layer 1 activates the function ReLU by converting 768 dimensions to 256 dimensions. The fully connected layer 2 activates the function ReLU by converting 256 dimensions to 128 dimensions, and outputs the preliminary probability P'.

[0214]

[0215] Where b1 is the bias term of the first fully connected layer of the primary classifier, and b2 is the bias term of the second fully connected layer of the primary classifier;

[0216] The ultimate classifier is used in the cloud, and a temporal convolutional layer (TCN) is introduced. Through four layers of causal dilation convolution, with a dilation_rate of [1, 2, 4, 8], 168 hours of historical data is covered.

[0217] Through P' and historical data P 历史 Calculate P, P 历史 Sliding window length = 24 hours × 7 days,

[0218] P=σ(W2·TCN(W1·[P′,P 历史 ])+b) (34),

[0219] Step 8: Use TensorRT to convert the FP32 model to INT8 format, perform 80% of the computation locally, and add 20% of the complex computation in the cloud for deployment;

[0220] A representative dataset was used to calculate the dynamic range of activation values and generate a quantization table. The FP32 weights were then mapped to the INT8 range of -128 to 127 for calibration. TensorRT Builder was then configured, with the optimization profiles fp16_mode = False, int8_mode = True set. The calibrated quantization table was loaded, and the INT8 engine file .engine was generated. Finally, a collaborative computing strategy was adopted, with the Jetson AGX Orin at the edge performing 80% of the computation and the cloud supplementing the remaining 20% of the complex operations. This reduced the size of the ANN water quality model from 300MB of FP32 to 120MB, a 60% reduction. Furthermore, the inference latency on the Jetson AGX Orin at the edge was reduced from 5.2 seconds to 2.8 seconds.

[0221] Step 9: Calculation of pollution index and setting of graded warnings,

[0222] Calculate the pollution index PI, Level I warning is defined as a potential risk: when any indicator exceeds the threshold by 10%, high-density sampling is initiated, and the sampling interval is reduced from 30 seconds to 5 seconds;

[0223] Level II warning, defined as moderate pollution: When PI>0.7, sound and light alarms are activated, and data is directly reported to the supervision platform;

[0224] Level III warning, defined as severe pollution: When toxic substances are detected, such as benzene series > 0.1 mg / L, the neutralizer will be released at 5 L / min and the aircraft will return to port autonomously;

[0225] Step 10: Pollution source positioning: binocular vision positioning on the unmanned vessel and pollution diffusion simulation are used to locate the pollution source. The SIFT algorithm is used to extract binocular image feature points, and the disparity map is calculated for feature matching. The depth information is obtained through the disparity, and the depth information is integrated with the GPS / INS data to locate the pollution source coordinates in the UTM coordinate system.

[0226] Where: GSD = 0.1m is the ground sampling distance, (X0, Y0) is the coordinate of the image center;

[0227] The major axis of the error ellipse is ≤0.6m, and the pollution diffusion simulation is carried out using the advection-diffusion equation.

[0228]

[0229] Diffusion coefficient Dx = Dy = 0.3 m 2 / s is the diffusion coefficient, and flow velocity vx = 0.8m / s is the flow velocity.

[0230] When a Level II warning response occurs, an audible and visual alarm is triggered, the unmanned vessel sounds its horn and flashes its warning lights; pollution coordinates and diffusion prediction maps are uploaded via NB-IoT; and propeller disturbance oxygenation is started, with a propeller speed of 2000 rpm and an oxygenation rate of 5 mg / L·min.

[0231] When a Level III warning response occurs, the detected benzene concentration is greater than the threshold, triggering a Level III warning; the neutralizer NaOH solution is released at a rate of 5L / min; the return route is planned, returning to the base station to charge and upload complete data.

Claims

1. A multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system, characterized by include: Multimodal sensing subsystem, artificial neural network water quality early warning subsystem, and graded response subsystem. The water quality parameters and spectral images obtained by the multimodal sensing subsystem are aligned with each other through GPS / INS to establish a unified spatiotemporal coordinate system. The water quality parameters and spectral images are fused using a cross-attention mechanism. The artificial neural network water quality early warning subsystem then determines pollution and implements a graded response to pollution. The multimodal sensing subsystem includes a water quality sensor, a hyperspectral camera, and a multibeam echosounder. The hyperspectral camera is installed at the bow of the unmanned boat, and the multibeam echosounder is installed at the bottom of the unmanned boat. A water sampling auxiliary device is installed at the stern of the unmanned boat. A water sampling mechanism and a water sample box are installed in the cabin of the unmanned boat. The water sample box is equipped with a water quality sensor. The water sampling mechanism stores the collected water samples in the water sample box and detects them through the water quality sensor. The artificial neural network water quality early warning subsystem adopts a dual-stream parallel architecture to synchronously process water quality parameters obtained by the water quality sensor and hyperspectral data obtained by the hyperspectral camera; the model architecture includes an input layer, a hidden layer, and an output layer; The input layer includes water quality parameters obtained by the water quality sensor and hyperspectral data obtained by the hyperspectral camera; The hidden layer uses a 1D-CNN time series processing module and a ResNet image feature extraction module; The output layer uses a Softmax classifier to generate a three-level warning; The three-level warning is divided into Level I, Level II and Level III; Level I, when a single indicator exceeds the threshold by 10%, high-density sampling is initiated; Level II: pollution index > 0.7, sound and light alarm + direct data reporting; Level III: toxic substances are detected, neutralizer is released and the aircraft returns home autonomously.

2. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claim 1 is characterized in that The water quality sensor includes a pH sensor, a turbidity sensor, a dissolved oxygen sensor, an ammonia nitrogen sensor and a conductivity sensor.

3. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claim 1 is characterized in that The artificial neural network water quality early warning subsystem has a model architecture including an input layer, a hidden layer, and an output layer; The input layer includes water quality parameters obtained by the water quality sensor and hyperspectral data obtained by the hyperspectral camera. The hidden layer uses a 1D-CNN time series processing module and a ResNet image feature extraction module; The 1D-CNN time series processing module uses the water quality parameters obtained by the water quality sensor as the input structure. The dilated convolution chain uses 3 layers of 1D convolution, expanding the receptive field to 29 time steps. The adaptive gating mechanism then dynamically adjusts the feature weights of each dimension and outputs a 256-dimensional time series feature vector. The ResNet image feature extraction module uses hyperspectral data obtained by the hyperspectral camera as input, adopts spectral response normalization, introduces a band attention mechanism to address the differences in response between different bands, and uses a strided convolutional layer after the fourth residual block to output a 512-dimensional image feature vector, preserving spatial details. The output layer uses a Softmax classifier to generate a three-level warning, concatenating the 256-dimensional time series feature vector and the 512-dimensional image feature vector into a 768-dimensional feature vector; a primary classifier is used locally, through two layers of fully connected layers 256→128, to output a preliminary probability P'; an ultimate classifier is used in the cloud, introducing a temporal convolutional layer TCN to fuse historical data, and using TensorRT to convert the FP32 model to INT8 format. 80% of the computation is performed locally, and 20% of complex operations are supplemented on the cloud for deployment.

4. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claims 1 to 4 is characterized in that Binocular vision is installed on the unmanned vessel, and the pollution source is located through binocular vision positioning and pollution diffusion simulation. The SIFT algorithm is used to extract the feature points of the binocular image, and the disparity map is calculated for feature matching. The parallax is used to obtain depth information, which is then integrated with GPS / INS data to locate the pollution source coordinates in the UTM coordinate system.

5. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claims 1 to 4 is characterized in that The working steps are as follows: Step 1: At a certain working point, the hyperspectral camera on the unmanned vessel obtains hyperspectral data; the water sample collected by the water sampling mechanism in the cabin of the unmanned vessel is placed in a water sample box, and the water quality sensor in the water sample box obtains water quality parameters. Specifically, the water quality sensors are pH sensors, turbidity sensors, dissolved oxygen sensors, ammonia nitrogen sensors and conductivity sensors, and the water quality parameters obtained are pH value, turbidity, DO, NH3-N, and EC; Step 2: Perform PCA dimensionality reduction on the hyperspectral data obtained in step 1 to obtain the spectral features H after dimensionality reduction. proj : 1) Convert the original digital quantization value DN into the radiation brightness L through radiation correction. The radiation correction formula is: In formula (1), L is the radiant brightness, DN is the original digital quantization value, G = 120 is the system gain, R cal =0.12W·m -2 ·sr -1 nm -1 is the radiation calibration factor; 2) Calculate the water surface reflectivity ρ using the atmospheric correction FLAASH model toa , Atmospheric correction formula FLAASH model, In formula (2), the distance between the sun and the earth is d = 1.0 AU, the cosine of the solar zenith angle μ0 = 0.866, and the solar irradiance Es = 1367 W·m -2 , L is the radiance; 3) Hyperspectral data matrix: H hyper =[ρ toa,400 ,ρ toa,401 ,...,ρ toa,1000 ] T ∈R 270 (3) From H hyper Extract the water surface reflectance R680=ρ in the 680nm band toa ,680, 4) Calculate its normalized difference vegetation index variant NDWI chl , by combining bands to enhance chlorophyll a signals, optimizing the key features of 270-dimensional spectral data, and improving principal component analysis results, Normalized Difference Vegetation Index variants: In formula (4), R680 and R750 are extracted from formula (3); R680=ρ toa ,680,R750=ρ toa ,750; 5) Using principal component analysis PCA, H hyper Dimensionality reduction from 270 dimensions to 10 dimensions: H hyper Standardization to obtain H norm , standardization formula: In formula (5), μ is the spectral data matrix H hyper The mean vector of each band, σ is the spectral data matrix H hyper The standard deviation vector of each band; Calculate the covariance matrix C, the covariance matrix formula is: In formula (6), T is the transposed sign, N is the number of samples, that is, the total number of samples of hyperspectral data; 6) Solve the eigenvalues of the covariance matrix λ1≥λ2≥…≥λ270 and the corresponding eigenvectors v1,v2,…,v270, and take the first 10 principal component eigenvectors The cumulative variance contribution rate is ≥95%, and the dimension is reduced to 10 dimensions to obtain H proj , Dimensionality reduction mapping formula: In formula (7), W PCA is the principal component eigenvector, and the spectral feature H after dimension reduction is obtained. proj ; Step 3: The water quality parameters obtained in step 1 are converted into 12-dimensional time series features. The single time step raw data of pH sensor, turbidity sensor, dissolved oxygen sensor, ammonia nitrogen sensor and conductivity sensor is a 5-dimensional vector X t , In formula (8), T is the transposition symbol, t is the time step, which represents the current sampling moment, Then, a sliding window with 3 time steps of 90 seconds is used to expand the time window to obtain the data matrix Ft. Then perform a 3-point moving average on each parameter to obtain And generate the smoothed data matrix Expand the data matrix into 15-dimensional time series features by column Redundant parameters were screened by the Pearson correlation coefficient. The correlation coefficient between turbidity and conductivity was r = 0.87, p < 0.

01. Conductivity was retained and turbidity was removed to avoid redundancy. After removing turbidity, the 15-dimensional time series feature was transformed into a 12-dimensional time series feature, which is a 3-time step data Fseq containing 4 parameters: pH, DO, NH3-N, and EC. in Step 4: Dilated convolution using 3 layers of dilated 1D convolution kernel=(5,1),dilation_rate=[1,2,4] expands the receptive field of time series data to 29 time steps; where dilation_rate is 1, 2, and 4, corresponding to three layers of dilated convolution, the receptive field is gradually expanded through the exponential dilation rate to cover short-term, medium-term, and long-term time series features; The calculation formula of single-layer receptive field is: RF=kernel_size+(kernel_size-1)×(dilation_rate-1) (14) Cumulative receptive field calculation results: Number of layers: 1, dilation_rate: 1, single layer receptive field: 5, cumulative receptive field: 5; Number of layers: 2, dilation_rate: 3, single layer receptive field: 9, cumulative receptive field: 13; Number of layers: 3, dilation_rate: 4, single layer receptive field: 17, cumulative receptive field: 29; Input the spectral feature H after dimension reduction in step 2 proj , mapped to the query space through the fully connected layer, and the query vector is obtained Perform 3-layer expansion 1D convolution on the time series water quality parameters to extract multi-scale time series features, and then map the 12-dimensional time series features after expansion convolution to generate the key K 水质 Sum V 水质 , the feature dimension is d k , matches Q; K 水质 =W K ·DilatedConv(X 水质 ) (16) V 水质 =W V ·DilatedConv(X 水质 ) (17) is the learnable weight matrix; W Q Query weight matrix, used to map spectral features to query space, W v is the value weight matrix, which is used to extract the value representation of time series features; Q 光谱 , K 水质 、V 水质 Divide into 8 sub-heads to calculate single-head attention, Scale the dot product attention to get the attention weights generated by each sub-head Then concatenate the outputs of the 8 sub-heads along the feature dimension to finally obtain the output dimension MultiHead(Q,K,V). MultiHead(Q,K,V)=Concat(Attention1,...,Attention8)W O (20) Step 5: Enhance local features by dilating the convolution chain. The dilated convolution chain structure is designed, and the dilation=1 in the first layer is used to capture the local features of adjacent time steps, so that the output dimension is changed from becomes Output=DilatedConv(Attention(Q,K,V)) (21) Among them, the expansion convolution operation of each layer is defined as DilatedConv(X)=ReLU(W conv*dil X+b) (22) *dil is the dilated convolution operation, W conv is the convolution kernel weight, b is the bias term; Step 6: Fuse the spectral features with the enhanced temporal features for feature concatenation to construct a joint feature vector; The spectral features after PCA dimensionality reduction The 258-dimensional vector output by the dilated convolution chain Along the channel The spliced feature vector F is obtained by directly splicing Perform layer normalization on the concatenated feature vector F to eliminate the dimensionality differences between different features, accelerate model convergence and improve generalization ability; Normalize the concatenated feature vector F. Where: ∈ is the minimum value, set to 10 -5 ;Prevent division by zero error; Calculate the mean μ and variance σ by channel dimension 2 , Introducing learnable parameters to scale γ and offset β output i , Step 7: Design the model architecture of the artificial neural network, using a dual-stream parallel branching structure; the water quality parameters obtained by the water quality sensor and the hyperspectral data obtained by the hyperspectral camera are used as the input layer; the hidden layer uses the 1D-CNN time series processing module and the ResNet image feature extraction module; The 1D-CNN time series processing module uses the water quality parameters obtained by the water quality sensor as the input structure. The dilated convolution chain uses 3 layers of 1D convolution, expanding the receptive field to 29 time steps. The adaptive gating mechanism then dynamically adjusts the feature weights of each dimension and outputs a 256-dimensional time series feature vector. The structural weighted feature of the adaptive gating mechanism is X gate , X gate =σ(G)⊙X (28) б is the Siqmoid function, ⊙ is the element-wise multiplication; Image feature extraction module, hyperspectral data obtained by hyperspectral camera The input data is H×W=640×640, and spectral response normalization is used. A band attention mechanism is introduced to address the differences in responses between different bands. A strided convolutional layer is used after the fourth residual block to output a 512-dimensional image feature vector to preserve spatial details. The ResNet module modifies its band attention mechanism and generates channel descriptors by global average pooling GAP on the input spectral image. The fully connected layer and Sigmoid generate band weights W∈[0,1] 270 , W=σ(W2·ReLU(W1·S)) (29) (29)W2 represents the weight matrix of the fully connected layer, which is used to generate the band attention weight; ReLU is the piecewise linear activation function; S is the channel descriptor; The reweighted output is H att , H att =W⊙H hyper (30), The residual block design has a structure in which each residual block contains two convolutional layers with 3×3 kernels and skip connections, and introduces batch normalization BatchNorm and ReLU activation, so that it can output 512-dimensional image features after passing through four residual blocks. The output layer uses a Softmax classifier to generate a three-level warning, concatenating the 256-dimensional time series feature vector and the 512-dimensional image feature vector into a 768-dimensional feature vector; When performing feature splicing, the 256-dimensional time series feature vector and the 512-dimensional image feature vector are spliced along the channel dimension. Perform layer normalization LayerNorm on the concatenated feature vector. A primary classifier is used locally. The fully connected layer 1 activates the function ReLU by converting 768 dimensions to 256 dimensions. The fully connected layer 2 activates the function ReLU by converting 256 dimensions to 128 dimensions, and outputs the preliminary probability P'. Where b1 is the bias term of the first fully connected layer of the primary classifier, and b2 is the bias term of the second fully connected layer of the primary classifier; The ultimate classifier is used in the cloud, and a temporal convolutional layer (TCN) is introduced. Through four layers of causal dilation convolution, with a dilation_rate of [1, 2, 4, 8], 168 hours of historical data is covered. Through P' and historical data P 历史 Calculate P, P 历史 Sliding window length = 24 hours × 7 days, P=σ(W2·TCN(W1·[P′,P 历史 ])+b) (34), Step 8: Use TensorRT to convert the FP32 model to INT8 format, perform 80% of the computation locally, and add 20% of the complex computation in the cloud for deployment; A representative dataset was used to calculate the dynamic range of activation values and generate a quantization table. The FP32 weights were then mapped to the INT8 range of -128 to 127 for calibration. TensorRT Builder was then configured, with the optimization profiles fp16_mode = False, int8_mode = True set. The calibrated quantization table was loaded, and the INT8 engine file .engine was generated. Finally, a collaborative computing strategy was adopted, with the Jetson AGX Orin at the edge performing 80% of the computation and the cloud supplementing the remaining 20% of the complex operations. This reduced the size of the ANN water quality model from 300MB of FP32 to 120MB, a 60% reduction. Furthermore, the inference latency on the Jetson AGX Orin at the edge was reduced from 5.2 seconds to 2.8 seconds. Step 9: Calculation of pollution index and setting of graded warnings, Calculate the pollution index PI, Level I warning is defined as a potential risk: when any indicator exceeds the threshold by 10%, high-density sampling is initiated, and the sampling interval is changed from 30 seconds to 10 seconds; Level II warning, defined as moderate pollution: When PI>0.7, sound and light alarms are activated, and data is directly reported to the supervision platform; Level III warning, defined as severe pollution: When toxic substances are detected, the neutralizer is released at 5L / min and the aircraft returns to port autonomously; Step 10: Pollution source positioning: binocular vision positioning on the unmanned vessel and pollution diffusion simulation are used to locate the pollution source. The SIFT algorithm is used to extract binocular image feature points, and the disparity map is calculated for feature matching. The depth information is obtained through the disparity, and the depth information is integrated with the GPS / INS data to locate the pollution source coordinates in the UTM coordinate system. Where: GSD = 0.1m is the ground sampling distance, (X0, Y0) is the coordinate of the image center; The major axis of the error ellipse is ≤0.6m, and the pollution diffusion simulation is carried out using the advection-diffusion equation. Diffusion coefficient Dx = Dy = 0.3 m 2 / s is the diffusion coefficient, and flow velocity vx = 0.8m / s is the flow velocity.

6. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claim 1 or 5 is characterized in that The Level II warning response described above includes: triggering an audible and visual alarm, with the unmanned vessel sounding its horn and flashing its warning lights; uploading pollution coordinates and a diffusion prediction map via NB-IoT; and starting propeller disturbance oxygenation, with a propeller speed of 2000 rpm and an oxygenation rate of 5 mg / L·min.

7. The multimodal perception and edge intelligence water monitoring unmanned vessel and graded warning system according to claim 1 or 5 is characterized in that The aforementioned Level III warning response: when the concentration of benzene series substances detected is greater than the threshold, a Level III warning is triggered; Release the neutralizer NaOH solution at a rate of 5L / min; plan the return route, return to the base station to charge and upload complete data.

Images