Water supply pipeline leakage multi-domain feature extraction and fusion recognition method

Abstract

The application discloses a water supply pipeline leakage multi-domain feature extraction and fusion recognition method, relates to the field of water supply pipeline leakage detection, and comprises the following steps: firstly, constructing different types of water supply pipeline leakage event signal data sets; then, extracting artificial features and deep features from the leakage signals contained in the signal data sets; then, fusing the artificial features and the deep features based on an attention module, constructing a feature fusion network recognition model containing an artificial feature extraction module, a deep feature extraction module and an attention fusion module, and performing offline training on the feature fusion network recognition model; and finally, using the obtained feature fusion network recognition model to recognize the data signal set of a water supply pipeline to-be-detected event, so as to determine whether a leakage event occurs in the water supply pipeline.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of water supply pipeline leakage detection, in particular to a water supply pipeline leakage multi-domain feature extraction and fusion recognition method. BACKGROUND

[0002] With the continuous deepening of urbanization in China, the infrastructure of the city has gradually become a solid foundation for survival and development. As an important project of infrastructure construction, the city water supply pipeline is related to the normal operation of the city and the daily life of residents. However, due to the natural aging of the water supply pipeline, the continuous expansion of the water supply network scale and the gradual increase of the water supply pressure, the leakage problem of the water supply pipeline is becoming increasingly serious. The automatic recognition of water supply pipeline leakage can effectively save costs, improve leakage detection accuracy, reduce the average pipe network leakage rate, improve water resource utilization, and prevent and control water loss, drinking water accidents, and water supply problems caused by pipeline leakage. It has important significance for the normal operation of the city water supply system.

[0003] At present, the mainstream method is a recognition method based on machine learning or deep learning. If machine learning is used, the extraction of artificial features requires a lot of professional knowledge and experience, which is time-consuming and laborious, and many high-level features of signals cannot be obtained. However, if deep learning is used, high-order features of signals can be automatically extracted; however, due to the problem of few signal sample types, deep learning network is prone to overfitting, which will cause the model to fail in some complex environments. SUMMARY

[0004] In view of the above problems existing in the prior art, the purpose of the present application is to provide a water supply pipeline leakage multi-domain feature extraction and fusion recognition method. Artificial distinguishable features and deep learning features are extracted from the leakage signal, wherein the artificial features refer to the leakage distinguishable features obtained in multiple dimensions, and the deep learning features refer to the high-dimensional features obtained by any suitable one-dimensional deep convolution network. At the same time, the attention mechanism is used to deeply fuse the features of the two, so that the network automatically focuses on the key information of the multi-domain features, forms the main leakage feature expression, and makes the recognition network have faster convergence speed, lower training loss and higher recognition accuracy. In addition, using this method can not only realize the recognition of leakage and non-leakage, but also realize the recognition of large leakage and small leakage.

[0005] In order to solve the above technical problems, the present application adopts the following technical scheme:

[0006] The water supply pipeline leakage multi-domain feature extraction and fusion recognition method comprises the following steps:

[0007] Step 1: Constructing different types of water supply pipeline leakage event signal data sets;

[0008] Step 2: extracting artificial features and deep features from the leakage signals contained in the signal data set; then, fusing the artificial features and the deep features based on an attention module to construct a feature fusion network recognition model containing an artificial feature extraction module, a deep feature extraction module and an attention fusion module, and performing offline training on the feature fusion network recognition model;

[0009] The artificial feature extraction module is used to extract artificial features in the leakage signal, the deep feature extraction module is used to extract deep features in the leakage signal, and the attention fusion module is used to fuse the artificial features and the deep features.

[0010] Step 3: using the feature fusion network recognition model obtained in step 2 to recognize the data signal set of the water supply pipeline to be tested, so as to determine whether a leakage event occurs in the water supply pipeline.

[0011] Considering the above problems existing in the prior art, the present application proposes a recognition method of fusing artificial features and deep features, which not only considers the reliability of artificial features, but also considers deep features extracted by deep learning, and combines the advantages of the two features, enriches the information of the signal, so that the model has stronger generalization ability. The extraction of deep features can use any suitable one-dimensional deep convolutional network, but the commonly used deep feature extraction network convolutional neural network (CNN) has some problems, such as overfitting when the data volume is small; with the increase of network layers, the problems of gradient dispersion and performance degradation are prone to occur. The residual network (ResNet) based on CNN increases the important basic unit - residual block, which realizes the cross-layer connection of intermediate features through the way of jump connection, which well avoids the problems of gradient explosion or gradient disappearance and network degradation caused by deepening depth, not only speeds up the convergence speed of deep neural network, but also greatly improves the accuracy of deep network, that is, compared with the traditional convolutional neural network CNN, ResNet has the advantages of deeper network structure, faster convergence speed, better generalization ability, etc. Therefore, the deep feature extraction network in the present application is mainly described by taking ResNet network as an example.

[0012] As an optional technical solution, the step 1 specifically comprises the following steps:

[0013] Step 1.1: using piezoelectric acceleration sensors to adsorb at different collection points near the leakage point to collect original signals of the water supply pipeline in a complex background environment;

[0014] Step 1.2: The single original signal is expanded by using the offset sampling method, the total length of the signal is L sampling points, the required sample length is N sampling points, and the starting point of the sample is N0. If the offset sampling technology is used, the starting point of the next sample is not the N0+N+1 sampling point, but the offset ΔN is added to the starting point of the previous sample, that is, the N0+ΔN sampling point is started;

[0015] Step 1.3: The expanded data set is randomly divided into a training set and a test set according to a 7:3 ratio, and corresponding labels are set, thereby obtaining different types of event signal data sets, including a two-class signal data set and a three-class signal data set. The two-class signal data set includes leakage signals and non-leakage signals, and the three-class signal data set includes large leakage signals, small leakage signals and non-leakage signals.

[0016] As an optional technical solution, in step 2, the manual feature extraction module analyzes the multi-domain features of the leakage signal data to obtain manual features.

[0017] As an optional technical solution, the manual features include a spectral width parameter, a singular spectrum value, a wavelet packet energy, an AR model coefficient, and an MFCC coefficient feature.

[0018] The spectral width parameter is calculated by the following formula:

[0019] In the formula, ε is the spectral width parameter of the signal, m0, m2 and m4 are the 0th, 2nd and 4th spectral moments of the signal, respectively.

[0020] The singular spectrum value is obtained by the following process: the one-dimensional time signal is cut and reorganized to construct a two-dimensional trajectory matrix, the trajectory matrix is singular value decomposed, the first r singular values are selected as the useful components of the original signal to represent the singular value features of the signal, and the last d-r singular values are discarded as noise, and r

[0021] The wavelet packet energy is obtained by the following process: a wavelet basis is first selected, then the number of decomposition layers is set, an n-layer wavelet tree is constructed, and the energy of each node in the last layer is extracted as the wavelet packet energy feature of the signal.

[0022] The AR model coefficient is obtained by the following process: the Yule-Walk equation is derived from the difference equation of the model to obtain the autocorrelation function of the signal, and the Yule-Walk equation is as follows:

[0023] In the formula, R xx(m) is the autocorrelation function of the signal, a k is the AR model parameter of the signal, and p is the model order;

[0024] The AR model coefficients of the signal are obtained by solving the Y-W equation using the Levinson-Durbin algorithm, and the recursive formula of the Levinson-Durbin algorithm is as follows:

[0025] a m (k) = a m-1 (k) + a m (m) a m-1 (m-k), k = 1, 2, …, m-1, wherein a m (k) is the prediction coefficient, a m (m) is the reflection coefficient.

[0026] The MFCC coefficient feature is obtained by the following formula:

[0027] wherein C(n) is the MFCC coefficient of the signal, s(m) is the logarithm of the energy of the output signal of each filter, M is the number of triangular bandpass filters, and L is the order of the MFCC coefficient.

[0028] As an optional technical solution, in step 2, the composition of the deep feature extraction module includes a convolution layer C1, a ReLU layer, a pooling layer P1, a residual block R1, a convolution layer C2, a ReLU layer, a pooling layer P2, and a residual block R2.

[0029] The deep feature module processing process is specifically implemented as follows: the signal spectrum in the water supply pipeline leakage event is taken as the input signal of the network, and then sequentially passes through the convolution layer C1, the ReLU layer, the pooling layer P1, the residual block R1, the convolution layer C2, the ReLU layer, the pooling layer P2, and the residual block R2 for processing. The output signal of the residual block R2 is the deep feature of the leakage signal.

[0030] As an optional technical solution, in step 2, the artificial feature and the deep feature obtained are input into the AFF module for feature fusion, and the output of the AFF module sequentially passes through two fully connected layers, and then the final classification result is obtained by using the SoftMax method.

[0031] As an optional technical solution, step 2 specifically includes the following steps:

[0032] Step 3.1: input the extracted artificial feature and deep feature into the attention module for fusion, construct a fusion network model based on the attention module, and set the network initialization parameters;

[0033] Step 3.2: training the fusion network model based on the attention module, updating parameters and network optimization, if the iteration is ended, saving the best model as the final water pipeline leakage event identification model; otherwise, jumping to step 3.2.

[0034] As an optional technical solution, the step 3.2 specifically comprises the following steps:

[0035] Step 3.2.1: initializing the fusion network model based on the attention module, including matrix weight and bias;

[0036] Step 3.2.2: performing fast Fourier transform on the sample signal in the training set, inputting the signal spectrum into the fusion network model based on the attention module, and obtaining the predicted label of the signal sample through forward propagation;

[0037] Step 3.2.3: calculating the back propagation error according to the set target function and the obtained predicted label, and updating and optimizing the parameters of the whole network using the error.

[0038] As an optional technical solution, in the step 3.2.2, the attention module adopts an AFF module, and the specific implementation is as follows: setting the deep feature as X and the artificial feature as Y; the deep feature and the artificial feature are subjected to a corresponding convolution operation respectively, so that they have the same dimension, and the deep feature after the convolution operation is X', and the artificial feature is Y'; then the deep feature X' and the artificial feature Y' are added to obtain the local attention feature and the global attention feature respectively; setting the input feature as C, the calculation process of the corresponding local attention feature and the global attention feature is as follows:

[0039] L(C)=B(Conv2(δ(B(Conv1(C))))),

[0040] g(C)=B(Conv2(δ(B(Conv1(Avg(C)))))),

[0041] Wherein, L(C) is the local attention feature, g(C) is the global attention feature, Conv1 is the convolution operation, the convolution size is 1x1, B represents the BatchNorm layer, δ represents the ReLU activation function, and Avg represents the average pooling operation; then the deep feature X' and the artificial feature Y' are added to obtain the local attention feature and the global attention feature, and the result M of the two through the sigmod function is as follows:

[0042]

[0043] Wherein, δ' represents the sigmod activation function;

[0044] The mixed feature after the attention fusion module of the deep feature X' and the artificial feature Y' is denoted as Z, and the result is as follows:

[0045]

[0046] As an optional technical solution, the step 3.2.3 specifically comprises the following steps:

[0047] Step 3.2.3.1: the loss value of the predicted label and the real label is calculated by using the cross information entropy loss function, and the loss value L is calculated according to the following formula:

[0048]

[0049] Wherein: x represents a sample, n represents the total number of samples, a represents the predicted label of the sample, and y represents the real label of the sample.

[0050] Step 3.2.3.2: the parameter gradient of the fusion network model based on the attention module is reversely calculated by using the loss value, the network model is updated by using the parameter gradient, and the Adam algorithm is used for optimization;

[0051] Step 3.2.3.3: after updating the fusion network model based on the attention module with the model parameter theta, it is judged whether the updated network model converges by using the training loss value, if it converges, the best model is saved as the final event recognition model, otherwise, it is jumped to step 3.2.

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] 1. The present application proposes a synchronous extraction and fusion of artificial features and deep features in water supply pipeline leakage detection, increases the richness of information extraction, avoids the risk of incomplete information extraction when relying solely on artificial features and overfitting when relying solely on deep networks, and effectively improves the accuracy of leakage detection and recognition and the generalization ability of the algorithm.

[0054] 2. The method of the present application applies the attention module to the fusion network of artificial features and deep features, so that the leakage recognition model can automatically focus on the key information part of the event signal, form a strong representation of the leakage signal, and improve the leakage event recognition rate while accelerating the network convergence speed.

[0055] 3. The method of the present application uses an end-to-end network to realize the fusion of artificial features and deep features, automatically adjusts the key information attention mode of the fused high-dimensional features through network updating, and has stronger portability than other traditional fusion methods such as simple splicing.

[0056] 4. The method can realize intelligent identification of leakage and non-leakage, and accurate identification of different leakage conditions such as large leakage and small leakage. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The signal identification algorithm structure and signal processing flowchart of the application;

[0058] Figure 2 The signal collection schematic diagram of the application;

[0059] Figure 3 The data augmentation and data set expansion algorithm principle diagram of the application;

[0060] Figure 4 The wavelet packet decomposition tree principle diagram in the artificial recognizable feature of the leakage signal of the application;

[0061] Figure 5 The energy feature extraction flow of wavelet packet decomposition in the artificial recognizable feature of the leakage signal of the application;

[0062] Figure 6 The AR model parameter estimation flowchart based on L-D algorithm in the artificial recognizable feature of the leakage signal of the application;

[0063] Figure 7 The feature fusion network 2 (artificial feature and deep feature splicing based on spatial attention feature fusion network) algorithm structure and signal processing flowchart of the application;

[0064] Figure 8 The test set confusion matrix of different networks in the two-class signal data set of the application;

[0065] Figure 9 The test set accuracy histogram of different networks in the two-class signal data set of the application;

[0066] Figure 10 The network blind measurement accuracy histogram of different networks under different signal-to-noise ratios in the two-class of the application;

[0067] Figure 11 The test set confusion matrix of different networks in the three-class signal data set of the application;

[0068] Figure 12 The test set accuracy histogram of different networks in the three-class signal data set of the application. DETAILED DESCRIPTION

[0069] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0070] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0071] Embodiment 1

[0072] Taking the application of water supply pipe network leakage detection in urban background as an example, the water supply pipe leakage identification method based on multi-domain feature fusion, the whole signal processing flow and network structure are as shown in Figure 1 The main steps are as follows:

[0073] Step 1: Data preparation. The piezoelectric acceleration sensor is respectively adsorbed at different collection points near the leakage point to collect the original signal of the water supply pipe in a complex background environment, which is divided into leakage signal and non-leakage signal. The offset sampling method is used to expand a single original signal, so as to achieve the purpose of expanding the data set. The expanded data set is randomly divided into a training set and a test set according to a 7:3 ratio, and corresponding labels are set, 0 represents a leakage signal, and 1 represents a non-leakage signal, to obtain a two-class signal data set. The signal data set is further adjusted, the data is divided into three classes according to the size of the leakage degree, and a three-class signal data set is constructed.

[0074] Step 2: Extraction of artificial recognizable features of leakage signals and deep features of leakage signals. Based on the signal data set, the spectral width parameter, singular spectrum, wavelet packet energy, AR model coefficient and MFCC coefficient feature are extracted as artificial recognizable features. The deep features can be obtained by any suitable one-dimensional deep convolution network, and the one-dimensional residual network is taken as an example in the embodiment. Fast Fourier transform is performed on the signal data set as the input signal of the network, a one-dimensional residual network (Residual Network) with two-layer residual block (Residual block) structure, referred to as 1D-ResNet, is constructed, and the deep features of the signal are obtained.

[0075] Step 3: Fusion network construction. By using the attention fusion module, a new network for fusing artificial features and deep features is constructed, and the feature fusion network is trained and tested. The network parameters are adjusted to make the network performance optimal, and the final network model is obtained.

[0076] Step 4: Leak identification and performance testing. The measured leakage and non-leakage signals are classified. In order to verify the effectiveness of artificial features, an artificial feature SVM classification network is added. The test set of the two-class signal data set and the three-class signal data set obtained in step 1 are input into the trained artificial feature + SVM network, 1D-Resnet network and feature fusion new network based on different fusion strategies, among which there are three kinds of fusion new networks: feature fusion network 1 is only a network for splicing artificial features and deep features without attention module; feature fusion network 2 is a feature fusion network based on spatial attention after splicing artificial features and deep features; feature fusion network 3 is a fusion network based on attention module for artificial features and deep features, and the results obtained by different network models are compared.

[0077] On the basis of the above, the present application is further described in detail.

[0078] As an embodiment of the present application, the data acquisition principle is shown in Figure 2 The piezoelectric acceleration sensor is used to absorb different collection points near the leakage point, and the pipeline vibration sound signals before and after the leakage point repair (with leakage and without leakage) are collected, the sampling frequency fs = 20480Hz, and the sampling time is about 1s.

[0079] As an embodiment of the present application, in order to enhance the generalization ability of the identification algorithm, data augmentation and data set expansion are carried out based on the collected data before the network is constructed, and the specific process is as follows:

[0080] The number of data set samples has a great influence on the classification ability of the deep learning network model. The larger the number of samples used for training the model, the stronger the generalization ability, and the more applicable the application scenarios. Based on the reason that the current sample data amount is small, it is hoped to expand the number of data set samples by data set enhancement technology. However, since the pipeline leakage signal is a one-dimensional time series signal, the current common data set enhancement technology, such as rotation, flipping, cropping and scaling, is often aimed at two-dimensional images and is not applicable. According to the characteristics of the pipeline leakage signal, the offset sampling method is used to expand the signal data set.

[0081] The principle of offset sampling is shown in Figure 3As shown, the total length of the signal is L sample points, the required sample length is N sample points, the sample start point is N0, if the offset sampling technology is used, the start point of the next sample is no longer the N0+N+1 sample point, but is added to the offset ΔN based on the start point of the previous sample, that is, from the N0+ΔN sample point; it is ensured that the original signal can be intercepted into more samples, which greatly expands the data set capacity and does not damage the timing of the signal.

[0082] The original length of the pipeline leakage signal collected in this paper is about 1s, and the sampling frequency fs=20480Hz. In the preprocessing stage, in order to avoid the instability of the starting point of collection, the first 0.001s signal is discarded, the sample length N=4096 is taken as 0.2s, the offset length 0.05s is taken as the offset ΔN=1024, and the overlap length N-ΔN=3072. Therefore, each original signal can be expanded to 16 samples. According to the signal category and the different leakage degrees, two-class (leakage and no leakage) signal data sets and three-class (large leakage, small leakage and no leakage) signal data sets are constructed. The overall signal data set is randomly divided into a training set and a test set in a ratio of 7:3, wherein the two-class signal data set is shown in Table 1, and the three-class signal data set is shown in Table 2.

[0083] Table 1 Two-class signal data set

[0084] Event type Training set (number) Test set (number) Total Leakage 1243 565 1808 No leakage 1041 415 1456 Total 2284 980 3264

[0085] Table 2 Three-class signal data set

[0086] Event type Training set (number) Test set (number) Total Large leakage 528 240 768 Small leakage 353 143 496 No leakage 597 251 848 Total 1478 634 2112

[0087] As an embodiment four of the present application, the artificial recognizable feature extraction process of the leakage signal is as follows:

[0088] Based on the above obtained signal data set, the spectral width parameter, the singular spectrum value, the wavelet packet energy, the AR model coefficient and the MFCC coefficient feature of the signal are extracted as artificial recognizable features.

[0089] (1) Spectral width parameter feature

[0090] The spectral width parameter is used to quantify the characteristics of the energy distribution of the signal frequency domain. The smaller the value is, the higher the energy concentration degree of the signal in a specific frequency range is, otherwise the larger the value is, and the energy is more dispersed. The calculation process is as follows:

[0091] For a signal x(n) with a length of N, the one-sided power spectral density S(ω) is first obtained, which can be expressed as

[0092]

[0093] where Rx is the autocorrelation function of the signal, ω is the angular frequency of the signal. The kth spectral moment of the signal m k is

[0094]

[0095] f H is the upper limit frequency of integration, respectively, k = 0, 2, 4, m0, m2, m4 are obtained, then the spectral width parameter ε can be obtained according to the following formula:

[0096]

[0097] From formula 1-3, the value range of ε is 0-1.

[0098] Under different working conditions, the center frequency of the pipeline leakage sound signal is different, and the frequency band of the signal spectral width parameter analysis changes accordingly. Set the initial value of the integral upper limit fHfHmin=100Hz, the terminal value fHmax=10kHz, the interval step 100Hz, and the spectral width parameters of different upper limit frequencies are solved in turn, so as to obtain the spectral width parameter characteristics of the leakage signal and the non-leakage signal.

[0099] (2) Singular spectrum characteristics

[0100] Singular spectrum analysis decomposes nonlinear time series and predicts sequence characteristics. This method realizes the decomposition of trend, oscillation components and noise components from time series by constructing a specific matrix on the time series and performing singular value decomposition.

[0101] The singular spectrum analysis steps are as follows:

[0102] ① Embedding, that is, cutting and recombining one-dimensional time signal to construct two-dimensional trajectory matrix. For the original signal X N = (x1, x2, …, x N ) of length N, given window length L (1 < L < N), the original signal is mapped to K = N-L+1 vectors of length L, that is

[0103] X i = (x i ,x i+1 ,…,x i+L-1 ) T ,1≤i≤K (1-4)

[0104] These vectors form a trajectory matrix, which is represented as

[0105]

[0106] ② Decomposition (singular value). This step performs singular value decomposition of the trajectory matrix X, denoted as

[0107]

[0108] where d is the number of nonzero singular values of X, λ i are singular values of X and λ1≥ λ2≥…≥ λ d > 0, U i and V i are left and right singular vectors of X.

[0109] ③Grouping. This step is to extract the useful components of the signal, discard the useless amount and interference terms. In the field of signal processing, usually take the first r(r < d) singular values as the useful components of the original signal, and the last d-r singular values are discarded as noise. Where d is the number of nonzero singular values of the trajectory matrix, r is mainly determined according to the contribution rate of the singular value, by comparing the sum of the contribution rates of the first r submatrices with the size of the pre-set threshold, the number of submatrices that meet the threshold is obtained.

[0110] ④Reconstruction. The purpose of this step is to reconstruct the two-dimensional matrix Y after the foregoing grouping into a new sequence with the same length as the original signal. Let y i,j (1≤i≤L,1≤j≤K) be each element in Y, L * = min(L,K), K * = max(L,K), when L < K, y * i,j = y i,j , otherwise y * i,j = y j,i . Then Y can be transformed into a reconstructed sequence y rc1 of length N by diagonal averaging, y rc2 , … y rcN , the calculation formula is as follows

[0111]

[0112] The larger the window length L selected by the singular spectrum analysis of the signal, the more refined the signal decomposition, but too large L will also bring a large amount of meaningless operation amount. When the singular spectrum analysis is performed on the pipeline leakage signals under different working conditions, the window length L is in the range of [1, 50], and the largest singular value is selected as the singular spectrum feature of the signal, so as to obtain the singular spectrum features of the leakage signal and the non-leakage signal.

[0113] (3) Wavelet packet energy feature

[0114] Wavelet packet analysis is a multi-resolution optimization analysis method for signal spectrum. Compared with wavelet transform, wavelet packet analysis divides the signal multiple times, so that it can be uniformly adapted to the original signal spectrum, and a more detailed signal decomposition is obtained. Each wavelet packet decomposition divides the signal into high-frequency and low-frequency two parts, and two sub-signals are obtained, which detailedly and completely show the information of the frequency band in the signal, and facilitate subsequent feature extraction and time-frequency localization analysis. Figure 4 is a three-layer wavelet packet decomposition tree, where (i, j) represents the signal component of the i-th layer node j of the wavelet packet decomposition tree.

[0115] Unlike the Fourier transform based on trigonometric functions, the wavelet function used in wavelet packet analysis is not unique. In other words, even if the same number of layers is used to decompose the same signal, the results will not be the same if different wavelet bases are used. The Haar wavelet base is selected in this embodiment, which can form the simplest orthogonal normalized wavelet family. Common wavelet bases also include Morlet (morl) wavelet, Daubechies (dbN) wavelet, Meyer wavelet, etc.

[0116] Because the signal intensity is different under different working conditions, the absolute value of the signal energy is different, so the energy of each node of the leakage signal after three-layer wavelet packet decomposition is taken as the wavelet packet energy feature in this embodiment, so that the wavelet packet energy feature has higher recognition under different working conditions. The energy feature extraction process of wavelet packet decomposition is as shown in Figure 5

[0117] (4) AR model coefficient feature

[0118] The AR model is used to analyze the time series of the signal. This method represents the random signal x(n) as a linear superposition of its past values x(n-k) and the current excitation value w(n), that is,

[0119]

[0120] where p represents the order of the model, a k is a constant coefficient, and the system function of the model is

[0121]

[0122] This is a system function with only poles and no zeros, also known as an all-pole model, represented by AR(p). The distribution of the poles relates to the stability of the system.

[0123] The coefficients of the AR model reflect the stability of the sequence, and there are three commonly used estimation methods: moment estimation, least squares estimation, and maximum likelihood estimation. To reduce complexity and simplify calculation, the moment estimation is used in this embodiment, and the autocorrelation function of x(n) is derived from the difference equation of the model ​

[0124] R xx (m) = E[x(n)x(n+m)] (1-10)

[0125] Substituting equation 1-8, we have

[0126]

[0127]

[0128] It can be seen that the autocorrelation function of the AR model output signal has a recursive property. Equation (1-12) can be transformed into

[0129]

[0130] Equation (1-13) is the Yule-Walker equation (Y-W equation). The Y-W equation shows that only a small amount of observed data is needed to obtain the AR model parameters {a k}.

[0131] In the calculation of solving the Y-W equation, the most commonly used is the Levinson-Durbin algorithm (L-D algorithm). The L-D algorithm is derived from the Y-W equation and the recursive property of the autocorrelation sequence. Each order parameter is calculated from the previous order parameter until the desired accuracy is met. This recursive method can effectively reduce the amount of calculation and conveniently find the optimal order. The L-D algorithm recursive formula can be expressed as

[0132] a m (k) = a m-1 (k) + a m (m)a m-1 (m-k), k = 1, 2, …, m-1 (1-14)

[0133]

[0134]

[0135] where a m (k) is called the prediction coefficient, and a m (m) is called the reflection coefficient. Before starting the calculation, set the initial value E0= R(0), a0(0) = 1, and take the order p as a suitable value according to actual needs. The system recursive process is shown in Figure 6 .

[0136] After weighing the model coefficient calculation amount and identification effect, the L-D algorithm is used to solve the 5th order AR model coefficient of the signal, which is the distinguishable feature of the pipeline leakage signal.

[0137] (5) MFCC coefficient feature

[0138] MFCC analysis method, in the analyzed frequency band, a set of Mel filter is designed in order from dense to sparse, the filtered signal energy as the signal characteristics, for subsequent signal analysis.

[0139] MFCC is extracted under the Mel scale. Mel scale is a nonlinear frequency scale determined by the human ear's judgment of equal distance hearing. Its relationship with linear frequency can be described as

[0140]

[0141] If the signal is uniformly distributed in the Mel scale, the distance between linear frequencies will be larger and larger.

[0142] The leakage signal MFCC feature extraction process is as follows:

[0143] ① Pre-emphasis: improve the proportion of high frequency part, so as to make the signal spectrum flat. Pre-emphasis operation is to pass the original signal through a high-pass filter

[0144] H(z) = 1 - μz -1 (1-18)

[0145] Usually, μ = 0.97.

[0146] ② Frame and window: for simplicity, take N sampling points as a frame, usually 20ms-40ms. There is an overlapping area between adjacent two frames, the overlapping length is about 1 / 2 or 1 / 3 of N, to realize smooth transition and avoid excessive change. Before FFT operation, each frame will be multiplied by a Hamming window to reduce the size of sidelobe and spectral leakage.

[0147] ③ Frequency domain conversion: to observe the energy distribution of the signal, use FFT to realize the conversion from time domain to frequency domain.

[0148] ④ Calculate the Mel filter bank: define a Mel filter bank of M triangular band-pass filters (M usually takes 22-26), the frequency response of the filter is

[0149]

[0150] Among them, f(m) is the center frequency of the filter, and the interval between f(m) widens as m increases.

[0151] ⑤ Logarithmic operation: calculate the energy of each filter output signal and take the logarithm for cepstrum analysis.

[0152]

[0153] ⑥ Discrete Cosine Transform: Discrete cosine transform is performed on the above logarithmic energy to obtain L-order MFCC, and L is usually 12-16.

[0154]

[0155] In this embodiment, the frame length N is 2048 points, at this time the length of each frame is about 0.1s, the number of Mel filter banks M is 26, and the order of MFCC L is 13. The Mel frequency cepstrum coefficient of each leakage point signal is extracted as the distinguishable feature of the pipeline leakage signal according to the above process.

[0156] As the fifth embodiment of the present application, the deep feature extraction process is as follows:

[0157] The network structure of the present application adopts a synchronous and parallel extraction route of artificial features (spectral width parameters, singular spectrum, wavelet packet energy, AR model coefficients, and MFCC coefficients) and deep features. As shown in Figure 1 , the leakage signal data set collected on site is subjected to fast Fourier transform to obtain a leakage signal spectrum data set, which is then divided into two routes. One is the extraction of deep features, that is, the leakage signal spectrum data passes through the convolution layer C1, the ReLU layer, the pooling layer P1, the residual block R1, the convolution layer C2, the ReLU layer, the pooling layer P2, and the residual block R2 to obtain the deep features of the leakage signal. The other is the extraction of artificial features, that is, the multi-domain feature analysis of the leakage signal data obtains the artificial features of the leakage signal.

[0158] In the deep feature extraction, the signal data set obtained above is subjected to fast Fourier transform to obtain the spectrum data set of all signals. As shown in Figure 1 , the spectrum of the leakage signal is taken as the input signal of the network, which sequentially passes through the convolution layer C1, the ReLU layer, the pooling layer P1, the residual block R1, the convolution layer C2, the ReLU layer, the pooling layer P2, and the residual block R2, and the output of the residual block R2 is taken as the deep features of the leakage signal.

[0159] As the sixth embodiment of the present application, the fusion network construction process is as follows:

[0160] The deep features of the leakage signal are extracted from the spectrum samples, and the artificial features and the deep features are fused based on the attention module. Based on the same objective function, the deep feature extraction module and the attention fusion module are trained offline to obtain the optimal model, and the feature fusion recognition model is obtained.

[0161] The specific process of the feature fusion recognition network design includes three links of network structure design and parameter setting, network initialization and network training, and parameter updating and optimization. The specific method is as follows:

[0162] (1) Network structure design and parameter setting

[0163] The deep feature extraction of the leakage signal spectrum can be any suitable one-dimensional deep convolutional network. In this embodiment, the deep feature extraction part of the leakage signal spectrum is taken as an example to be constructed in a one-dimensional ResNet network.

[0164] After the deep features and artificial features of the leakage signal are extracted, they are input into the attention module for fusion to obtain a fusion network model based on the attention module, specifically: the artificial features and deep features obtained are input into the AFF module for feature fusion, and the output of the AFF module is sequentially passed through two fully connected layers and then the final classification result is obtained by using the SoftMax method. The specific network structure and parameter settings are shown in Table 3.

[0165] Table 3 Fusion network structure parameters based on attention module

[0166]

[0167] (2) Network initialization and network training

[0168] After the fusion network based on the attention module is initialized, the training data set is input into it, the prediction class probability distribution is obtained through the forward propagation of the network, the loss value between the class probability output by the fully connected layer and the true probability is calculated using the cross-entropy loss function, and the gradient of each learning parameter is calculated using the loss value. Finally, according to the specified learning rate, the model parameters θ including the matrix weight W and the bias b are updated according to the principle of gradient descent. Taking the first iteration learning process of the model as an example for explanation:

[0169] Initialize the parameters of the fusion network based on the attention module, and good initialization parameters make the model easier to learn and converge faster. The present application uses the Xavier initialization method to initialize the network structure parameters. In order to ensure that the variance of each layer is consistent during forward propagation and backward propagation, the distribution range of the random initialization of the parameters is a uniform distribution in the range of in , the output parameter number n in , the distribution range formula is:

[0170]

[0171] The sample signals in the training set are input into the fusion network based on the attention module for forward propagation to obtain the predicted labels of the signal samples. This process mainly includes a 2-layer convolution structure, an attention fusion module and an output layer:

[0172] Convolution structure: taking convolution structure 1 (convolution layer C1- ReLU layer-pooling layer P1-residual block) as an example. The calculation process is as follows:

[0173] Convolutional layer C1: assuming that the convolution kernel size of the convolutional layer is m, the step is S, the boundary padding is p, the sequence length of the input is L, and the input feature channel number is K in , the output feature channel number is K out , the input training data is The jth convolution kernel is initialized after the weight matrix W j = {W j1 , W j2 ,..., W jKin},

[0174]

[0175]

[0176] ReLU layer: the ReLU activation function calculation process is as follows:

[0177] conv_out = max {0, conv u} (1-25)

[0178] Pooling layer P1: assuming that the pooling layer step is s, the calculation process is as follows:

[0179]

[0180] Residual block: there are two 1x5 convolutional layers with the same output channel number in the residual block, and a ReLU activation function is followed after each convolutional layer. The output result of the second convolutional layer is added to the input of the residual block, and then processed through a ReLU activation function, which realizes the cross-connection. Therefore, the original differential of H(x) mapping will be transformed into the differential of F(x)+x, which reduces the complexity of problem optimization and avoids gradient vanishing and other problems. The calculation process is as follows:

[0181] y = F(x, {W i}+x) (1-27)

[0182] Where y represents the output of the residual structure, x represents the input of the structure, {W i} represents the convolution or feature extraction operation performed, and F(x) represents the nonlinear superposition activation of the linearly extracted features, i.e. the ReLU activation function.

[0183] AFF attention module: the deep feature is set as X, and the artificial feature is set as Y; the deep feature and the artificial feature are respectively subjected to a corresponding convolution operation, so that they have the same dimension. The deep feature after the convolution operation is X', and the artificial feature is Y'; as Figure 1The attention module AFF module in the fusion network based on the attention module is shown, and the local attention feature and the global attention feature are calculated after the deep feature X' and the artificial feature Y' are added; assuming that the input feature is C, the calculation process of the corresponding local attention feature is shown in formula 1-28, and the calculation of the global attention feature is shown in formula 1-29:

[0184] L(C)=B(Conv2(δ(B(Conv1(C))))) (1-28)

[0185] g(C)=B(Conv2(δ(B(Conv1(Avg(C)))))) (1-29)

[0186] Wherein, L(C) is a local attention feature, g(C) is a global attention feature, Conv1 is a convolution operation, the convolution size is 1*1, B represents a BatchNorm layer, δ represents a ReLU activation function, and Avg represents an average pooling operation; then the deep feature X' and the artificial feature Y' are calculated to obtain the local attention feature and the global attention feature, and the result M of the sum of the two through the sigmod function is as follows:

[0187]

[0188] The mixed feature Z after the deep feature X' and the artificial feature Y' pass through the attention fusion module is recorded as Z, and the result is as follows:

[0189]

[0190] The full connection layer: assuming that the input is x i , the weight matrix W i , the bias y i , and the final classification output y i The calculation process is as follows:

[0191] y i =W i ×x i +b i (1-32)

[0192] (3) Parameter updating and optimization

[0193] According to the set target function and the obtained prediction label, the back propagation error is calculated, and the error is used to update and optimize the constructed fusion network based on the attention module. The specific steps are as follows:

[0194] The loss function used in the application is a cross-entropy loss function, and the cross-entropy loss function is used to calculate the loss value of the predicted label and the real label: the distance between the predicted event label and the real label is calculated according to the cross-entropy loss function, and the loss value L is calculated according to the following formula:

[0195]

[0196] Wherein: x represents a sample, n represents the total number of samples, a represents a sample predicted label, and y represents a sample real label;

[0197] The loss value is used to calculate the parameter gradient of the fusion network model based on the attention mechanism, and the parameter gradient is used to update the fusion network deep learning model based on the attention mechanism; the application adopts Adam algorithm for updating optimization, and the calculation steps are as follows:

[0198]

[0199] m t =u*m t-1 +(1-u)*g t (1-35)

[0200]

[0201] Wherein: g t is the calculated target gradient, m t , n t are the first and second order moment estimates of the gradient, respectively, and u, v∈[0,1) are the exponential decay rates of the first and second order moments of the gradient.

[0202] Let the tth iteration be, Then the network parameter update formula is:

[0203]

[0204] Wherein: α is the learning rate, and ε is a very small number to prevent division by zero.

[0205] In the fusion network model based on the attention module, the network model is first updated using the model parameters θ. Then, according to the change of the training loss value, it is judged whether the network model has converged. Once the convergence state is reached, the training process is stopped; otherwise, jump to step (2) and continue to update iteratively until the maximum number of iterations is reached. When the loss function value is less than a certain threshold or the iteration exceeds the preset threshold, it is considered that the model has converged, and the above iteration process is stopped. Finally, the best model is selected as the final event recognition model.

[0206] As an embodiment six of the application, the recognition results and performance of different networks are as follows:

[0207] (1) Identification results of two categories of leaked datasets

[0208] The optimal model of the fusion network based on the attention module obtained above is used as the event recognition model to identify typical events to be tested, realizing online monitoring of water supply pipeline leakage signals. It is compared with a pre-trained artificial feature + SVM network, a 1D-ResNet network, and fusion networks based on different fusion strategies under the same conditions (same input conditions). There are three types of fusion networks: Feature Fusion Network 1 is simply a network that concatenates artificial features and deep features without an attention module; Feature Fusion Network 2 is a feature fusion network based on spatial attention after concatenating artificial features and deep features; Feature Fusion Network 3 is a fusion network of artificial features and deep features based on an attention module. The comparison includes recognition accuracy (TestAccuracy) and recognition precision (TestPrecision). The network structure parameters of 1D-ResNet are shown in Table 4. The artificial feature and deep feature concatenation network is identical to 1D-ResNet except that it adds artificial features before the fully connected layer. The network structure of Feature Fusion Network 2 is shown in Table 4. Figure 7 As shown, the structure after removing the SpaceAttention module is the structure of Feature Fusion Network 1, and its network structure parameters are shown in Table 5:

[0209] Table 4 shows the structural parameters of the 1D-ResNet networks for comparison.

[0210]

[0211] Table 5 Structural parameters of Feature Fusion Network 2

[0212]

[0213] Confusion matrices of binary classification test sets for different networks, as follows Figure 8 As shown, the histogram of the accuracy distribution on the test set is as follows: Figure 9 As shown in Table 6, the accuracy, precision, recall, and F1-score of different networks on the test set after their respective model parameters have been optimized. It can be seen that when only manual features are used, i.e., the manual feature + SVM network has the lowest recognition accuracy. Among deep networks, the recognition accuracy is relatively high, reaching over 99%. Furthermore, the feature fusion network 3, i.e., the fusion network based on the attention module, achieves a test accuracy, precision, and recall of up to 100%, which is the best recognition result compared to other networks.

[0214] Table 6. Comparison of overall performance of event recognition for different network models

[0215]

[0216] Since the test set recognition accuracy of the artificial feature + SVM network is low, and this method depends on the different artificial feature extraction methods, the artificial features extracted under different signal conditions may be different, thereby causing the model to be difficult to generalize. The following discussion is mainly based on deep feature networks and artificial feature and deep feature fusion networks. In order to further analyze the recognition effect and stability of different networks, the pipeline leakage signals collected in the new scene are added with different degrees of noise for further testing of different networks, i.e. new scene blind test. The signals in the new scene are divided into leakage signals and non-leakage signals, and the number of samples of the leakage and non-leakage signals is 80, and the total number of samples is 160. The recognition results are shown in Table 7, and the distribution histogram is shown in Figure 10 Since the leakage in the new scene is obvious, the difference between the leakage and non-leakage signals is large, so under the condition of high signal-to-noise ratio, the test results of different networks are all 100%, but as the signal-to-noise ratio decreases, the networks show different effects. At SNR = 0, it can be seen that the recognition accuracy of the mixed feature network is still very high, especially the mixed feature network 2 (artificial feature and deep feature splicing based on spatial attention feature fusion network) and the mixed feature network 3 (fusion network based on attention module) have a correct rate of more than 98%, and at SNR = -5, the low signal-to-noise ratio, the mixed feature network 3 has more obvious advantages, and the correct rate is still more than 90%, and the network has good anti-noise ability.

[0217] Table 7 Comparison of new scene blind test results of different network models

[0218] Network model snr = -5 snr = 0 snr = 5 1Dresnet 50.63％ 90.63％ 100.00％ Feature fusion network 1 53.13％ 91.25％ 100.00％ Feature fusion network 2 52.50％ 98.75％ 100.00％ Feature fusion network 3 94.38％ 98.75％ 100.00％

[0219] (2) Three-class leakage data set recognition results

[0220] Since the current data set is ideal after selection, i.e. the difference between the leakage signal and the non-leakage signal is obvious, and the recognition accuracy of different networks is relatively high, in order to better test the performance of the network, the data set is further adjusted. The leakage signals in the data set are divided into large leaks and small leaks according to the different leakage conditions, i.e. the new data set has three categories, large leaks, small leaks and non-leaks. The specific categories and sample quantities are shown in Table 2.

[0221] Figure 11 The test set confusion matrix of different networks in the three-class signal data set is shown in Table 8. Figure 12The accuracy rate distribution histogram of the test set of different networks is shown. It can be found that in the three-class test, due to the increase of the category and the difference in the complexity of the signal, the overall recognition accuracy of the network is significantly lower than that of the two-class test, and in many cases, some leakage signals are recognized as non-leakage signals, that is, it is difficult to distinguish some small leakage conditions. However, through the test of the feature fusion network, the recognition accuracy of the feature fusion network is higher than that of the pure deep feature network, and different fusion methods lead to differences in network performance, among which the feature fusion network 3, that is, the fusion network based on the attention module, has the best recognition effect, with a recognition accuracy of more than 94%, that is, the fusion network based on the attention module not only has good anti-noise ability and high recognition accuracy, but also can distinguish between large leaks and small leaks. Figure 12 It can be obviously found that the feature fusion network has higher test accuracy than the pure deep feature network, and different fusion methods lead to differences in network performance, among which the feature fusion network 3, that is, the fusion network based on the attention module, has the best recognition effect, with a recognition accuracy of more than 94%, that is, the fusion network based on the attention module not only has good anti-noise ability and high recognition accuracy, but also can distinguish between large leaks and small leaks.

[0222] The above only describes the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement and improvement made by those skilled in the art within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for multi-domain feature extraction and fusion recognition of water supply pipeline leakage, characterized in that, The method comprises the following steps: Step 1: constructing different types of water supply pipeline leakage event signal data sets; Step 2: extracting artificial features and deep features from the leakage signals contained in the signal data set; Then, based on the attention module, the artificial features and the deep features are fused to construct a feature fusion network recognition model comprising an artificial feature extraction module, a deep feature extraction module and an attention fusion module, and the feature fusion network recognition model is trained offline; Wherein, the artificial feature extraction module is used to extract artificial features in the leakage signal, the deep feature extraction module is used to extract deep features in the leakage signal, and the attention fusion module is used to fuse the artificial features and the deep features; Step 3: using the feature fusion network recognition model obtained in step 2 to recognize the data signal set of the water supply pipeline to be tested, so as to determine whether a leakage event occurs in the water supply pipeline.

2. The method according to claim 1, wherein, The step 1 specifically comprises the following steps: Step 1.1: using a piezoelectric acceleration sensor to adsorb at different collection points near the leakage point to collect the original signal of the water supply pipeline in a complex background environment; Step 1.2: using offset sampling method to expand a single original signal, the total length of the signal is L sampling points, the required sample length is N sampling points, the sample starting point is N0, if offset sampling technology is used, the starting point of the next sample is no longer the N0+N+1 sampling point, but is the starting point of the previous sample plus an offset ΔN, that is, from the N0+ΔN sampling point; Step 1.3: dividing the expanded data set into a training set and a test set according to a 7:3 ratio, setting corresponding labels, and obtaining different types of event signal data sets, including a two-class signal data set and a three-class signal data set, wherein: the two-class signal data set includes leakage signals and non-leakage signals, and the three-class signal data set includes large leakage signals, small leakage signals and non-leakage signals.

3. The method according to claim 1, wherein, In the step 2, the artificial feature extraction module obtains artificial features through multi-domain feature analysis of the leakage signal data.

4. The method of claim 1, wherein, The artificial features include: spectral width parameters, singular spectrum values, wavelet packet energy, AR model coefficients, and MFCC coefficient features; Wherein, the spectral width parameters are calculated by the following formula: wherein is a spectral width parameter of the signal, m 0, m 2, m 4 are 0th, 2nd, 4th spectral moments of the signal, respectively. The singular spectrum values are obtained by the following process: the one-dimensional time signal is cut and reorganized to construct a two-dimensional trajectory matrix, the trajectory matrix is singular value decomposed, the first r singular values are selected as the useful components of the original signal, the singular value features of the signal, and the last d-r singular values are discarded as noise, and r < d; Wherein d is the number of non-zero singular values of the trajectory matrix, and r is determined by the contribution rate of the singular value; The wavelet packet energy is obtained by the following process: first, a wavelet base is selected, then the number of decomposition layers is set, then an n-layer wavelet tree is constructed, and the energy of each node in the last layer is extracted as the wavelet packet energy feature of the signal; The AR model coefficients are obtained by solving the following formula: where is the autocorrelation function of the signal, are the AR model parameters of the signal, and p is the model order. The MFCC coefficient features are calculated by the following formula: wherein is the MFCC coefficient of the signal, is the log of the energy of each filter output signal, is the number of triangular bandpass filters, is the order of the MFCC coefficients.

5. The method of claim 1, wherein, In step 2, the deep feature extraction module comprises a convolution layer C1, a ReLU layer, a pooling layer P1, a residual block R1, a convolution layer C2, a ReLU layer, a pooling layer P2 and a residual block R2. The deep feature module processing process is specifically implemented as follows: the signal spectrum in the water supply pipeline leakage event is taken as an input signal of a network, and then sequentially processed through a convolution layer C1, a ReLU layer, a pooling layer P1, a residual block R1, a convolution layer C2, a ReLU layer, a pooling layer P2, and a residual block R2, and an output signal of the residual block R2 is a deep feature of the leakage signal.

6. The method of claim 1, wherein, In step 2, the artificial feature and the deep feature obtained are input into an AFF module for feature fusion, and an output of the AFF module is sequentially input into two full connection layers, and finally a classification result is obtained by using a SoftMax method.

7. The method according to claim 6, wherein, In step 2, the feature fusion specifically includes the following steps: Step 3.1: The artificial feature and the deep feature obtained are input into an attention module for fusion, a fusion network model based on the attention module is constructed, and network initialization parameters are set; Step 3.2: The fusion network model based on the attention module is trained, and parameter updating and network optimization are performed, if iteration is ended, the best model is saved as a final water supply pipeline leakage event identification model, otherwise, the step 3.2 is jumped to.

8. The method according to claim 7, wherein, The step 3.2 specifically includes the following steps: Step 3.2.1: The fusion network model based on the attention module is initialized, including matrix weight and bias; Step 3.2.2: A sample signal in a training set in the data set is subjected to fast Fourier transform, and a signal spectrum is input into the fusion network model based on the attention module, and a predicted label of the sample signal is obtained by forward propagation; Step 3.2.3: A back propagation error is calculated according to a set target function and the predicted label, and the error is used for updating and optimizing parameters of the entire network.

9. The water supply pipeline leakage multi-domain feature extraction and fusion recognition method according to claim 8, characterized in that, In the step 3.2.2, the attention module adopts the AFF module, and the specific implementation is: the deep feature is set as , and the artificial feature is set as ; Both the deep feature and the manually generated feature are subjected to a corresponding convolution operation to make them have the same dimension. Let the deep feature after the convolution operation be... Artificial features are Then the depth features and artificial features After summing, calculate the local attention features and the global attention features separately; let... The input features are Then the corresponding local attention feature and global attention feature calculation process is as follows: ， ， wherein, is a local attention feature, is a global attention feature, is a convolution operation with a convolution size of 1x1, denotes a BatchNorm layer, denotes a ReLU activation function, denotes an average pooling operation; then the deep feature and the artificial feature is the result of adding the local attention feature and the global attention feature and passing the sum through a sigmod function as follows: ， wherein, denotes a sigmod activation function; deep features and artificial features The mixed features after the attention fusion module are denoted as The results are as follows: 。 10. The method of claim 8, wherein the method further comprises: The step 3.2.3 specifically includes the following steps: Step 3.2.3.1: Calculate the loss value of the predicted label and the real label by using the cross information entropy loss function, the loss value The calculation formula is as follows: ， Wherein: x represents a sample, n represents a total number of samples, a represents a sample predicted label, and y represents a sample real label; Step 3.2.3.2: The parameter gradient of the fusion network model based on the attention module is calculated reversely by using the loss value, the network model is updated by using the parameter gradient, and the network model is optimized by using an Adam algorithm; Step 3.2.3.3: Using model parameters After updating the fusion network model based on the attention module, the training loss value is used to determine whether the updated network model converges. If it converges, the best model is saved as the final event recognition model; otherwise, go to step 3.2.

Images