Vibration event classification method and device based on multi-modal features and double-branch network

Abstract

The invention discloses a vibration event classification method and device based on multi-modal features and a double-branch network, and relates to the technical field of structural health monitoring and safety early warning. The method comprises the steps that original vibration signals are collected and preprocessed, and normalized signals are obtained; extracting multi-scale time-domain features through empirical mode decomposition, extracting sub-band frequency-domain features through wavelet packet decomposition, screening optimal sub-band frequency-domain features in combination with information entropy, integrating the optimal sub-band frequency-domain features with the multi-scale time-domain features, and obtaining dimensionality-reduced features through principal component analysis; converting the normalized signal into a two-dimensional image matrix by using a Markov conversion field; constructing a back propagation algorithm-multilayer perceptron and self-attention transformation network encoder double-branch network, and respectively extracting local multi-scale features and global time sequence features; and fusing the local multi-scale features with the global time sequence features to obtain a fused feature vector, and outputting a final classification result through a classification module. According to the invention, the accuracy and robustness of ship collision event classification can be improved.

Description

Technical Field

[0001] This invention relates to the field of structural health monitoring and safety early warning technology, and in particular to a vibration event classification method and device based on multimodal characteristics and a dual-branch network. Background Technology

[0002] With the rapid development of global transportation networks, large bridges, as key transportation hubs, directly impact regional economic and social stability through their structural safety. However, bridges face various potential threats during operation, among which ship collisions (vessel strikes) have become a major factor leading to sudden bridge damage and even collapse due to their large impact loads, short duration, and random location of damage. Ship collisions can not only cause partial or overall structural damage to bridges but also lead to serious consequences such as traffic disruptions, environmental pollution, and casualties. Therefore, developing an intelligent monitoring system capable of accurately identifying and classifying ship collision vibration events is of great significance for ensuring bridge safety and reducing accident risks.

[0003] Traditional bridge vibration monitoring methods mainly rely on point sensors (such as accelerometers and strain gauges) or manual inspections, which have drawbacks such as limited coverage, low spatial resolution, poor real-time performance, and susceptibility to environmental interference (such as electromagnetic noise and changes in humidity and temperature). Point sensors can only capture local vibration signals and cannot comprehensively reflect the propagation characteristics and spatiotemporal distribution patterns of vibration events in the bridge structure; manual inspections are inefficient and cannot provide real-time early warnings. These methods are unable to identify ship collision events in a timely and accurate manner and cannot meet the needs of modern bridge safety monitoring.

[0004] Against this backdrop, distributed fiber optic sensing technology offers a new possibility for solving this challenge. This system uses optical fiber as the sensing medium, transmitting laser pulses and analyzing the phase changes of the returned signals to achieve continuous vibration monitoring along the entire length of the bridge structure. Compared to traditional methods, the distributed fiber optic sensing system has full-area coverage capability, enabling blind-spot-free monitoring of key components such as piers, decks, and towers; it possesses high sensitivity, capable of detecting minute vibration signals such as minor ship collisions and anchor chain scraping; and it exhibits excellent anti-interference performance, unaffected by electromagnetic interference, corrosion, or extreme weather conditions. Furthermore, the system offers significant advantages such as low deployment cost, no need for additional sensor nodes, and suitability for large-scale infrastructure applications.

[0005] These technical characteristics enable the distributed fiber optic vibration sensing system to effectively capture the unique vibration features of ship collision events, including high-frequency, high-amplitude transient signals and spatial and temporal differences that distinguish it from other vibration sources (such as vehicle traffic and wind vibration). Furthermore, combining the distributed fiber optic vibration sensing system with machine learning algorithms can further improve the accuracy and reliability of vibration event classification. By constructing a vibration feature library based on historical data or simulation experiments, and training models using deep learning networks (such as convolutional neural networks and long short-term memory networks) or traditional machine learning algorithms (such as support vector machines and random forests), the system can effectively distinguish ship collision events from other types of vibration. This combination of technologies not only enhances the system's ability to identify abnormal vibrations in the overall bridge structure but also effectively collaborates with other monitoring methods such as cameras, radar, and water level sensors to jointly build a more reliable and intelligent bridge safety monitoring system.

[0006] Although distributed fiber optic sensing systems have the above advantages, there are still some problems in their use. These defects severely restrict the reliability and real-time classification of the monitoring system when it is applied to monitor bridge damage.

[0007] At the feature extraction level, existing technologies mostly employ single-mode feature extraction methods, which have significant shortcomings. For example, while time-domain feature analysis methods can extract basic statistical features of signals, such as mean, variance, and zero-crossing rate, they cannot capture the dynamic changes of frequency components over time. Frequency-domain analysis methods can obtain global spectral features through Fourier transform, but they lose time-dimensional information, making it difficult to capture transient events. Simple time-frequency analysis methods, such as short-time Fourier transform, have an inherent contradiction between time resolution and frequency resolution due to their fixed window length design, making it impossible to achieve an optimal balance between the two. These limitations directly affect the accuracy and reliability of vibration event classification.

[0008] At the level of temporal modeling capabilities, existing technologies also face numerous challenges in handling the temporal correlations of vibration signals. Commonly used statistical classification models such as support vector machines and random forests cannot effectively model dependencies in long-term series; simple recurrent neural networks, while possessing some temporal processing capabilities, often suffer from the vanishing gradient problem, making it difficult to capture long-distance temporal correlations; traditional convolutional neural network models, due to their limited receptive field, also have significant limitations in recognizing global temporal patterns. These technical shortcomings often render existing methods inadequate when processing vibration signals with temporal characteristics.

[0009] At the model architecture design level, existing deep learning solutions generally adopt a single network architecture, which has inherent limitations. While a single convolutional neural network (CNN) architecture excels at extracting local features, it has weak global perception capabilities; a single recurrent neural network (RNN) architecture has certain advantages in temporal modeling, but its parallel computing efficiency is low; and the simple fusion strategies used by most methods, such as feature concatenation or voting fusion, lack deep feature interaction mechanisms, making it difficult to achieve true complementary advantages.

[0010] In general, existing distributed fiber optic sensing systems suffer from difficulties in feature extraction, insufficient temporal modeling capabilities, and simplistic model architecture designs at the vibration event classification level. To address these issues, it is urgent to propose an intelligent classification algorithm that integrates comprehensive feature extraction, advanced temporal modeling, and a rationally designed architecture with a distributed fiber optic sensing system to monitor bridge structures and achieve accurate identification and real-time early warning of ship collision events. Summary of the Invention

[0011] The purpose of this invention is to provide a vibration event classification method and device based on multimodal features and dual-branch networks, which solves the problems of vibration event classification defects caused by difficulties in multidimensional feature extraction, simple time series modeling, and single model architecture design in the prior art, improves the accuracy and robustness of ship collision event classification, and provides reliable technical support for the practical application of distributed optical fiber sensing systems in bridge safety monitoring.

[0012] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a vibration event classification method based on multimodal features and a dual-branch network is provided, the method comprising the following steps: S1. Acquire the original vibration signal and preprocess it to obtain the normalized signal; S2. For the normalized signal, multi-scale time-domain features are extracted by empirical mode decomposition, sub-band frequency-domain features are extracted by wavelet packet decomposition, the optimal sub-band frequency-domain features are selected by information entropy, integrated with multi-scale time-domain features, and dimensionality is reduced by principal component analysis to obtain the dimensionality-reduced features. S3. Use Markov transform field to convert the normalized signal into a two-dimensional image matrix; S4. Construct a dual-branch network of backpropagation algorithm-multilayer perceptron and self-attention transform network encoder. Use backpropagation algorithm-multilayer perceptron to process the dimensionality-reduced features to obtain local multi-scale features. Use self-attention transform network encoder to process the two-dimensional image matrix to obtain global temporal features. S5. Fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

[0013] Optionally, step S1 specifically includes: The original vibration signal was acquired based on a distributed fiber optic sensing system, denoted as... x ( t ),in t Indicates time; The original vibration signal was preprocessed using wavelet thresholding denoising. The Symlet-8 wavelet basis function was then used to perform a 5-level decomposition of the original vibration signal, resulting in a low-frequency approximation coefficient. and 5 high-frequency detail coefficients , j = 1,2,3,4,5; A hierarchical thresholding algorithm is designed to process the high-frequency detail coefficients of each layer, with the highest frequency detail coefficient being... The corresponding threshold for the first layer is For the remaining high-frequency detail coefficients , j = 2,3,4,5, no. j Layer threshold is ; An improved soft thresholding method is used for noise reduction when the absolute value of the high-frequency detail coefficients is less than the threshold. When the absolute value of the high-frequency detail coefficient is greater than or equal to the threshold, it is considered as zero noise; At that time, the coefficients are shrunk to obtain the first value after soft thresholding. j High-frequency detail factor ; Low-frequency approximation coefficients High-frequency detail coefficients after soft thresholding The denoised signal is obtained by reconstructing the signal using inverse wavelet transform. ; For denoised signals Perform Z-score normalization to obtain the normalized signal. ,in It is a denoised signal The i One sampling point.

[0014] Optionally, step S2 specifically includes: Normalized signals are obtained through empirical mode decomposition. It can be decomposed into 6 intrinsic mode functions. Indicates the first j The eigenmode function of the eigenmode function i One sampling point, j = 1,2,...,6; The normalized signal is decomposed using wavelet packet decomposition. Full-frequency domain decomposition yields eight non-overlapping full-frequency domain sub-band coefficients; Calculate the information entropy for each subband, select the top 3 subbands with the highest information entropy, and denote the corresponding subband coefficients as the optimal subband coefficients. ; The three selected optimal subband coefficients Compared with the output of empirical mode decomposition The features are integrated to form a multi-scale feature input set S, denoted as... S ={ S 1, S 2,..., S 9}, of which - Corresponding intrinsic mode functions, - Corresponding optimal sub-band coefficient; By using the root mean square (RMS) value of each feature to reflect the signal's strength, a 9-dimensional time-domain feature set is obtained. ; Dimensionality reduction was performed using principal component analysis (PCA) to obtain the dimensionality-reduced features. .

[0015] Optionally, step S3 specifically includes: Normalized signal The amplitude range is divided into Q=8 equally probable quantile intervals, and the normalized signal is traversed. Each sampling point i Determine its quantile interval. Generate discrete state sequences s ; Based on discrete state sequences s Construct the Markov state transition matrix M Quantify the transition probabilities between different states; Based on discrete state sequence s The state transition order is used to extract the corresponding transition probabilities and generate a 1-dimensional Markov state sequence (MTF). The Markov state sequence MTF is resampled into an 8×8 two-dimensional image matrix. .

[0016] Optionally, step S4 specifically includes: Backpropagation algorithm - Multilayer perceptron uses a structure with 3 hidden layers and ReLU activation function to process the dimensionality-reduced features. Processing is performed to obtain local multi-scale features. ; For two-dimensional image matrices The self-attention transform network encoder captures the global dependencies of state transition probabilities in an image through the Transformer self-attention mechanism, thus obtaining global temporal features. .

[0017] Optionally, step S5 specifically includes: The features of the dual-branch network are integrated by vector concatenation, combining the local multi-scale features output by the backpropagation algorithm-multilayer perceptron. Global temporal features of the encoder output of the self-attention transform network Perform dimensional concatenation to obtain the fused feature vector. ; The feature vectors are fused using a fully connected layer and a softmax function. Convert to probability vectors of each category ; Take probability vector The category corresponding to the maximum value in the middle is used as the final classification label to complete the identification and classification of vibration events.

[0018] On the other hand, a vibration event classification device based on multimodal features and a dual-branch network is provided to implement the method described in any of the above embodiments, the device comprising: The signal acquisition and preprocessing module is used to acquire the raw vibration signal and preprocess it to obtain the normalized signal. The multi-scale decomposition and feature extraction module is used to extract multi-scale time-domain features from normalized signals through empirical mode decomposition, extract sub-band frequency-domain features through wavelet packet decomposition, select the optimal sub-band frequency-domain features by combining information entropy, integrate them with multi-scale time-domain features, and perform dimensionality reduction through principal component analysis to obtain the dimensionality-reduced features. The Markov Transform Field module is used to convert normalized signals into two-dimensional image matrices using Markov transform fields. The dual-branch neural network module is used to construct a dual-branch network of backpropagation algorithm-multilayer perceptron and self-attention transform network encoder. The backpropagation algorithm-multilayer perceptron is used to process the dimensionality-reduced features to obtain local multi-scale features, and the self-attention transform network encoder is used to process the two-dimensional image matrix to obtain global temporal features. The feature fusion and classification output module is used to fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

[0019] On the other hand, an electronic device is provided, the electronic device comprising: processor; A memory storing computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the vibration event classification method described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the steps of the vibration event classification method described above.

[0021] The beneficial effects of the technical solution provided by this invention include at least the following: This invention employs an improved adaptive hierarchical wavelet soft-threshold denoising method. Through robust noise estimation based on the median, a threshold dynamically adjusted with the number of decomposition levels, and a soft-threshold function incorporating a shape coefficient, it achieves a balance between noise suppression and effective signal preservation. Compared to traditional hard-thresholding or fixed-threshold denoising methods, this method eliminates high-frequency environmental noise while preserving the unique transient impact characteristics and local details of ship collision events to the greatest extent possible. This effectively improves the signal-to-noise ratio, enabling clear separation of transient impact signals such as ship collisions from complex environmental noise in actual bridge monitoring, significantly reducing false alarms and laying the foundation for accurate early warning.

[0022] This invention employs a multi-scale feature extraction system combining Empirical Mode Decomposition (EMD) and Wavelet Packet Decomposition (WPD). EMD adaptively decomposes the intrinsic modes of the signal at different time scales, characterizing the time-varying patterns of vibration energy. WPD captures high-frequency details and frequency band energy distribution through full-frequency domain fine decomposition. The optimal sub-band of WPD is selected by information entropy and fused with EMD components, overcoming the information limitations of single-mode features. Simultaneously, principal component analysis (PCA) is used for dimensionality reduction, improving subsequent processing efficiency. This method enables the system to simultaneously capture the high-frequency instantaneous impact of ship collisions and the low-frequency overall response of bridge structures, effectively distinguishing between wind-induced vibrations, vehicle traffic, and other conventional vibrations, significantly improving the accuracy and reliability of event identification.

[0023] This invention introduces Markov Transformer Fields (MTFs) to convert one-dimensional vibration time-series signals into two-dimensional images, addressing the difficulty of directly capturing long-range temporal dependencies. By discretizing the signal amplitude into a state sequence and constructing a state transition probability matrix, the signal is encoded into a visual image that can be processed by the model. This preserves the temporal structure information of the original signal and transforms it into a spatially correlated feature map, providing a novel and efficient data input format for subsequent mining of global temporal patterns using Transformer models. The spatially correlated feature map enhances the system's early prediction capability of potential hazards.

[0024] This invention designs a dual-branch network consisting of a backpropagation algorithm, a multilayer perceptron (BP-MLP), and a self-attention transform encoder. The BP-MLP branch specifically handles local multi-scale features, capturing local nonlinear correlations between features; the Transformer Encoder branch handles global temporal features generated by the MTF, leveraging its powerful global perception capability through its self-attention mechanism to uncover long-distance dependencies in the state transition image. Finally, through feature concatenation and deep fusion, the advantages of local fine features and global temporal patterns are complemented. This model avoids the inherent limitations of a single network model, significantly improves the model's comprehensive discrimination ability for complex vibration modes, and achieves more comprehensive and accurate intelligent identification and classification of ship collision events, reducing the risk of missed detections. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a vibration event classification method based on multimodal features and a dual-branch network provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the Markov conversion field process and its effects provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a dual-branch network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a vibration event classification device based on multimodal features and a dual-branch network provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0028] This invention provides a vibration event classification method based on multimodal features and a dual-branch network. This method can be implemented by an electronic device, such as a terminal or a server. Figure 1 As shown, the processing flow of this method may include the following steps:

[0029] S1. Acquire the original vibration signal and preprocess it to obtain the normalized signal.

[0030] A distributed fiber optic sensing system is installed in critical, easily damaged sections of bridges, such as piers, decks, and towers, forming a high-density vibration monitoring network using phase-sensitive optical time-domain reflectometry (DAS). This distributed fiber optic sensing system boasts meter-level spatial resolution and a sampling frequency exceeding 10 kHz, enabling real-time capture of vibration wave signals propagating along the optical fiber. Vibration signals are acquired using a phase-sensitive optical time-domain reflectometer (Φ-OTDR) and a DAS system. The raw vibration signals acquired by the distributed fiber optic sensing system are denoted as... x ( t ),in t The time is represented by N, and the signal length, i.e., the number of sampling points, is N.

[0031] To address the issue of high noise and complex information in the original vibration signal under complex working conditions, a wavelet threshold denoising method is used for preprocessing the original signal to suppress noise interference. The Symlet-8 wavelet basis function is selected to perform a 5-level decomposition of the original vibration signal, yielding a low-frequency approximation coefficient. and 5 high-frequency detail coefficients , j = 1,2,3,4,5.

[0032] Low-frequency approximation coefficients reflect the overall trend of the signal, such as a slowly changing baseline, and do not include high-frequency noise. High-frequency detail coefficients contain signal noise and high-frequency vibration information. The decomposition expression is:

[0033] (1) in These are the low-frequency approximation coefficients of the 5th layer. It is the first j High-frequency detail factor of layer, j The smaller the value, the higher the frequency.

[0034] To accurately quantify the noise intensity in the high-frequency detail coefficients of each layer, a robust median estimation method is used to estimate the noise standard deviation. Compared to traditional mean estimation, this method is more robust to outlier interference and avoids the influence of extreme noise points on the estimation results. The noise standard deviation is expressed as:

[0035] (2) in It is the first j The noise standard deviation of the high-frequency detail coefficients of the layer. It is the first j The median of the absolute value sequence of high-frequency detail coefficients of the layer, where G is the standard deviation adjustment coefficient under Gaussian distribution, and G=0.6745.

[0036] Due to the different high-frequency detail coefficients Since the proportion of noise to effective signal is different, using a single threshold for denoising will result in residual noise in the high-frequency layer or loss of effective features in the mid-to-low-frequency layer. Therefore, a hierarchical threshold algorithm is designed to process the high-frequency detail coefficients of each layer.

[0037] For the highest frequency detail factor The corresponding first-level threshold is expressed as: (3) in It is the first-level threshold. It is the signal length. It is the noise standard deviation of the first layer of high-frequency detail coefficients (i.e., the highest frequency detail coefficients), and ln represents the logarithmic function.

[0038] For high-frequency detail (relative to the highest frequency detail) in the mid-to-low frequency range , j =2, 3, 4, 5, and their corresponding first... j The layer threshold is expressed as: (4) in It is the first j Layer threshold, log is a logarithmic function.

[0039] To address the unsatisfactory smoothing effect of traditional hard thresholding methods, this invention employs an improved soft thresholding method that incorporates a shape coefficient. This method not only better preserves the local characteristics of the signal but also achieves a superior smoothing effect. The high-frequency detail coefficients of each layer are subjected to threshold shrinkage using the aforementioned threshold. When the absolute value of a high-frequency detail coefficient is less than the threshold... When the absolute value of the high-frequency detail coefficient is greater than or equal to the threshold, it is considered as zero noise; At that time, the coefficients are shrunk to obtain the first value after soft thresholding. j High-frequency detail factor This preserves the effective signal components.

[0040] The soft thresholding function is expressed as: (5) in It is the first one after soft thresholding. j High-frequency detail coefficients of the layer, that is, preserving effective features and suppressing noise. It is a shape coefficient function, which can better preserve the local characteristics of the coefficients and the smoothing effect. sgn() is the sign function, and || is the absolute value.

[0041] After completing the thresholding of high-frequency detail coefficients at each layer, the low-frequency approximation coefficients are... High-frequency detail coefficients after soft thresholding The denoised signal is obtained by reconstructing the signal using inverse wavelet transform. ; Denoising signal Represented as: (6) in It is the denoised signal of the original vibration signal x(t). It is the Symlet-8 wavelet inverse transform.

[0042] Because the amplitude differences at different vibration times can lead to excessive focus on signal features with large amplitudes during subsequent feature extraction, while effective features with small amplitudes are ignored, this is problematic for denoising signals. Z-score normalization is performed to eliminate the influence of amplitude differences on subsequent feature extraction, resulting in a normalized signal. The normalized signal's first... i Each sampling point is represented as:

[0043] (7) in It is the first normalized signal i One sampling point, It is a denoised signal The i One sampling point, It is the mean of the denoised signal within a single sampling period. It is the standard deviation of the denoised signal within a single sampling period.

[0044] S2. For the normalized signal, multi-scale time-domain features are extracted by empirical mode decomposition, sub-band frequency-domain features are extracted by wavelet packet decomposition, the optimal sub-band frequency-domain features are selected by combining information entropy, integrated with multi-scale time-domain features, and dimensionality is reduced by principal component analysis to obtain the dimensionality-reduced features.

[0045] Since the characteristics of different vibration events are often distributed across different time-frequency scales, such as the instantaneous high-frequency impact of a ship collision and the low-frequency response of a bridge structure, single-dimensional features cannot fully characterize the event characteristics. Therefore, this method extracts multi-scale features in the time domain through empirical mode decomposition (EMD), extracts fine features in the frequency domain through wavelet packet decomposition (WPD), combines information entropy to select the optimal features, and eliminates redundancy through dimensionality reduction.

[0046] To address the nonlinear and nonstationary characteristics of vibration signals, an empirical mode adaptive decomposition method is used to decompose the signal into several intrinsic mode functions (IMFs) and a residual function. Each IMF corresponds to a vibration characteristic at a specific scale. The decomposition formula is expressed as follows: (8) in It is the first j The first of the intrinsic mode functions (IMFs) i Each sampling point reflects For features at different time scales, j The smaller the value, the higher the frequency; j = 1, 2, ..., 6. It is the th residual function i The sampling points, i.e. the low-frequency trend terms remaining after decomposition, have no effective vibration features and need to be removed in subsequent feature extraction.

[0047] To compensate for the lack of frequency domain precision in EMD, wavelet packet decomposition (WPD) is used. Full-frequency domain decomposition is performed, simultaneously decomposing low-frequency and high-frequency coefficients to achieve fine-grained full-frequency domain decomposition and capture high-frequency details of vibration signals.

[0048] First, define the wavelet packet decomposition (WPD) input, and then input the signal after Z-score normalization. The input signal WPD is represented as follows: (9) in, is the input signal for wavelet packet decomposition (WPD), and N is the signal length.

[0049] Next, wavelet packet decomposition (WPD) iterative decomposition is performed, using the Symlet-8 wavelet basis function and setting the decomposition level L=3. The decomposition yields... There are 8 sub-bands, and each sub-band corresponds to a set of sub-band coefficients. WPD achieves full-frequency domain coverage by iteratively decomposing low-frequency and high-frequency coefficients. The recursive relationship of the decomposition process is as follows:

[0050] Layer 0 (Original Signal Layer): (10) in It is the coefficient of the 0th sub-band of the 0th layer. p It is the coefficient index within the sub-band. p =1,2,..., , It is the first l Number of layer coefficients , l It is a decomposition level index. l =0,1,2,3, where l =0 is the initial layer. l =3 represents the final decomposition layer.

[0051] Recursive decomposition (level l = 1, 2, 3): For the l The coefficients of each subband in layer -1 Iterative decomposition using low-pass and high-pass filters into the th... l The coefficients of the two sub-bands of the layer are decomposed using the following formulas: (11) in It is the first l Layer The coefficient of each sub-band These are the coefficients of the low-pass filter. n It is the filter coefficient index. It is the first l The layers are indexed.

[0052] (12) in It is the first l Layer The coefficient of each sub-band These are the coefficients of the high-pass filter. n It is the filter coefficient index. It is the first l -1 level sub-band index.

[0053] Final output: Decomposed to level 3 ( l After (=3), eight non-overlapping full-frequency sub-band coefficients are obtained, which serve as the final output of wavelet packet decomposition (WPD). The final output is the first... l The subband coefficient of a layer subband is expressed as: , p = 1,2,..., (L=3) (13) in It is the first l Subband coefficient of the subband.

[0054] Some of the eight subbands obtained by the wavelet packet decomposition (WPD) may contain only noise, such as the high-frequency subband when there is no vibration. If they are directly used for feature construction, they will introduce redundancy. Therefore, the feature information content of the subband is quantified by information entropy. The larger the entropy value, the more uneven the distribution of the subband coefficients, indicating that the proportion of effective features is higher, thus selecting the optimal subband.

[0055] First, to avoid generating log0 and numerical overflow, the sub-band coefficients... Normalization. The normalized subband coefficients are expressed as:

[0056] (14) in It is the first l Normalized subband coefficients of the subband.

[0057] After normalization, the information entropy of each subband is calculated to quantify the amount of feature information in that subband. The information entropy of a subband is expressed as:

[0058] (15) in It is the first l Information entropy of sub-bands It is a logarithmic function with base 2.

[0059] The above results Information entropy of each sub-band , , ... , Sort by largest to smallest, and select the first K=3. The largest subband is denoted as the optimal subband coefficient. .

[0060] To integrate time-domain and frequency-domain characteristics, the three optimal sub-band coefficients were selected. Compared with the output of empirical mode decomposition (j=1,2,3,...,6) are integrated to form a multi-scale feature input set S, denoted as: (16) Where S is the multi-scale feature input set, denoted as S ={ S 1, S 2,..., S 9}, of which - Corresponding intrinsic mode functions, - The corresponding optimal sub-band coefficient.

[0061] Since the optimal subband coefficient length is The length N of the IMF is inconsistent with the length N of the IMF, so linear interpolation is used to... - The length is padded to N to ensure that all feature vectors have a uniform length. The root mean square (RMS) value of each feature vector is used to reflect the signal's energy strength, such as the difference between the high energy of a ship collision and the low energy of wind vibration, resulting in a 9-dimensional time-domain feature set, represented as: (17) in It is the time-domain feature set, and RMS() is the root mean square value of the feature vector.

[0062] The 9-dimensional features may exhibit correlations; for example, the RMS values ​​of different IMFs may reflect features with similar energies, leading to feature redundancy and reduced processing efficiency. Therefore, principal component analysis (PCA) is used for dimensionality reduction. A threshold of ≥95% variance percentage for each principal component is set to ensure that most of the effective information is retained. The top k=33 principal components are selected, yielding the dimensionality-reduced features, represented as follows:

[0063] (18) in These are the features after dimensionality reduction, and PCA() is the dimensionality reduction operation of principal component analysis.

[0064] S3. Use Markov transform field to convert the normalized signal into a two-dimensional image matrix.

[0065] The temporal correlation of vibration events is a key feature for classification, but traditional feature extraction methods struggle to directly capture this global temporal pattern. Therefore, this method utilizes Markov transform fields to convert temporal state relationships into a two-dimensional image, facilitating subsequent capture of global dependencies. The process is as follows: Figure 2 As shown, the detailed path is as follows: Normalized signal The amplitude range is divided into Q = 8 equally probable quantile intervals, the first... q intervals ( q =1,2,..., Q ) is represented as: (19) in It is the first q quantile intervals, It is a normalized signal of p Quantiles .

[0066] Traversal of normalized signals Each sampling point ( Given a subset of integers (e.g., 1, 2, ..., N), determine the quantile interval to which it belongs. Generate a discretized state sequence It intuitively reflects the change in signal amplitude over time.

[0067] Based on discrete state sequences s Construct the Markov state transition matrix M Quantify the transition probabilities between different states, from state i Transition to state j The probability is expressed as: (20) in From state i Transition to state j The probability, yes t The state at any given moment (the previous state). yes t The state at time +1 (current state).

[0068] According to the state sequence s The transition order is determined, and the corresponding transition probabilities are extracted to generate a 1D Markov state sequence (MTF), represented as: (twenty one) Where MTF is a Markov state sequence. It is the first t Secondary transfer ( Transferred to The probability corresponding to ).

[0069] To accommodate subsequent Transformer Encoder model inputs, the MTF was resampled into an 8×8 two-dimensional image matrix. , represented as: (twenty two) in It is a Markov two-dimensional image matrix. ( ) indicates the use of bilinear interpolation resampling operation.

[0070] S4. Construct a dual-branch network consisting of a backpropagation algorithm-multilayer perceptron (BP-MLP) and a self-attention transform network encoder. Use the backpropagation algorithm-multilayer perceptron to process the dimensionality-reduced features to obtain local multi-scale features, and use the self-attention transform network encoder to process the two-dimensional image matrix to obtain global temporal features.

[0071] To simultaneously capture local multi-scale features (from X after PCA dimensionality reduction)PCA ) and global temporal characteristics (MTF from Markov transformed field module) img This invention employs a dual-branch network of "Backpropagation Algorithm-Multilayer Perceptron" (BP-MLP) and "Self-Attention Transformer Encoder," abbreviated as "BP-MLP + Transformer Encoder Dual-Branch Network," to achieve deep mining and complementarity of two types of features. The network structure is as follows: Figure 3 As shown.

[0072] Leveraging BP-MLP's strength in handling low-dimensional structured features and capturing local nonlinear correlations, the temporal features are processed, and the input layer features are represented as follows: (twenty three) in It is the feature vector of the input layer. .

[0073] BP-MLP employs a "3-layer hidden layer + ReLU activation function" structure to address the problem of traditional shallow networks struggling to capture complex feature relationships. Simultaneously, the ReLU activation function alleviates gradient vanishing and helps capture complex feature relationships. The 3 hidden layers are represented as follows:

[0074] Hidden Layer 1: Through linear transformation and nonlinear activation, the input features are mapped to a higher-dimensional feature space. The output feature vector of Hidden Layer 1 is represented as: (twenty four) in It is the output feature vector of hidden layer 1. , It is the weight matrix from the input layer to hidden layer 1. , It is the bias vector of hidden layer 1. ReLU is the activation function.

[0075] Hidden layer 2: Based on the output feature vector of hidden layer 1 To further deepen feature extraction, the output feature vector of hidden layer 2 is represented as: (25) in It is the output feature vector of hidden layer 2. , It is the weight matrix from hidden layer 1 to hidden layer 2. , It is the bias vector of hidden layer 2. .

[0076] Hidden Layer 3: Building upon Hidden Layer 2, this layer further deepens the BP-MLP branch, serving as the final output layer and fully capturing the local nonlinear correlations of multi-scale features. The output feature vector of Hidden Layer 3 is represented as:

[0077] (26) in It is the output feature vector of hidden layer 3, i.e., local multi-scale features, which serves as the final output of the BP-MLP branch. , It is the weight matrix from hidden layer 2 to hidden layer 3. , It is the bias vector of hidden layer 3. .

[0078] For Markov two-dimensional image matrices It captures the global dependencies of state transition probabilities in images through the Transformer self-attention mechanism, thus processing image-type temporal features.

[0079] First, use the Flatten operation to... Flattened into a one-dimensional vector, it is transformed into an embedding feature matrix E through an embedding layer, as follows: (27) Where E is the embedding feature matrix. , It is the embedding layer weight matrix. Is it according to the row Flattened into a one-dimensional vector, It is the embedding layer bias vector.

[0080] After mapping image pixel values ​​to high-dimensional feature vectors, the embedded feature matrix E is used to capture the dependencies between different subspaces through a multi-head attention mechanism.

[0081] First, the query, key, and value matrices are represented as follows: (28) in i It is the attention head index. , They are the first i A matrix of Query, Key, and Value elements. , , It is the first i The weight matrices of each head correspond to... , , .

[0082] The embedding feature matrix E is divided into multiple subspaces and computed in parallel. The output of the single attention head is represented as follows: (29) in It is the first i The output of each attention head, , yes The transpose of the matrix, It is the scaling factor.

[0083] By concatenating the single attention heads of subspaces, a multi-head attention is output, integrating the global dependencies of different subspaces and improving the comprehensiveness of feature representation. The final output of multi-head self-attention is represented as:

[0084] (30) in It is the final output of the bulls' self-attention, making , ( ) concatenates the outputs of multiple attention heads. It is the output weight matrix.

[0085] To enhance the nonlinear expressive power of features and stabilize the training process, the multi-head self-attention output is processed. First, a nonlinear transformation is performed using a feedforward network (FFN), expressed as:

[0086] (31) in It is a multi-head self-attention output transformed by a feedforward network. It is the multi-head self-attention output after the splicing operation, that is... , , These are the weight matrices of the first and second layers of the feedforward network, respectively. , These are the biases of the first and second layer feedforward networks, respectively. ( ) is the activation function.

[0087] Then through layer normalization ( After eliminating feature distribution offsets, the final output of the Transformer Encoder branch is obtained. The final output feature vector is represented as: (32) in It is the final output feature vector, i.e., the global temporal features. , It is a layer normalization operation.

[0088] S5. Fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

[0089] The two-branch network outputs local multi-scale features respectively. With global temporal features To achieve complementary advantages of the two features, this invention integrates the two types of information through feature fusion and outputs the final result through the classification module.

[0090] The dual-branch features are integrated using vector concatenation, combining the output of BP-MLP. With the output of TransformerEncoder Perform dimensional concatenation to obtain the fused feature vector. , represented as: (33) in It is a fusion of feature vectors. , It is a vector concatenation operation.

[0091] This fused vector simultaneously incorporates local multi-scale energy features and global temporal transition features, avoiding information loss from a single feature dimension and providing comprehensive feature support for subsequent classification. The fused features are converted into probabilities for each category using a fully connected layer and a softmax function. The output of the fully connected layer is represented as follows:

[0092] (34) Where z is the original output of the output layer. It is the weight matrix of the output layer. It is the bias vector of the output layer.

[0093] The softmax function normalizes the original output into class probabilities, intuitively reflecting the model's confidence in each class. The output probabilities are expressed as:

[0094] (35) in These are the predicted probability vectors for each category, where C is the number of categories. It is the original output of the output layer corresponding to a single category. It is the traversal variable of the original output of all categories of output layers.

[0095] Take probability vector The category corresponding to the maximum value in the middle is used as the final classification label to complete the identification and classification of vibration events.

[0096] In summary, this invention constructs a set of intelligent identification and classification algorithms for vibration events, encompassing adaptive signal preprocessing, fusion of multimodal time-frequency and image-based temporal features, and a dual-branch neural network. Its core innovation lies in employing an improved adaptive hierarchical wavelet soft thresholding denoising method to effectively enhance signal quality. A multi-scale feature extraction strategy combining Empirical Mode Decomposition (EMD) and Wavelet Packet Decomposition (WPD) fully exploits the rich information of vibration signals in both the time and frequency domains. Furthermore, a Markov Transformer Field (MTF) is introduced to convert temporal state relationships into two-dimensional image features. Through a dual-branch network layer of "BP-MLP + TransformerEncoder," effective feature fusion and complementarity are achieved, significantly improving the model's comprehensive discrimination ability against complex vibration modes and enabling more comprehensive and accurate intelligent identification and classification of ship collision events.

[0097] Accordingly, embodiments of the present invention also provide a vibration event classification device based on multimodal features and a dual-branch network, such as... Figure 4 As shown, the device includes: The signal acquisition and preprocessing module 201 is used to acquire the original vibration signal and preprocess it to obtain a normalized signal. The multi-scale decomposition and feature extraction module 202 is used to extract multi-scale time-domain features from normalized signals through empirical mode decomposition, extract sub-band frequency-domain features through wavelet packet decomposition, select the optimal sub-band frequency-domain features by combining information entropy, integrate them with multi-scale time-domain features, and perform dimensionality reduction through principal component analysis to obtain the dimensionality-reduced features. Markov Transform Field Module 203 is used to convert a normalized signal into a two-dimensional image matrix using a Markov transform field; The dual-branch neural network module 204 is used to construct a dual-branch network of backpropagation algorithm-multilayer perceptron and self-attention transform network encoder. The backpropagation algorithm-multilayer perceptron is used to process the dimensionality-reduced features to obtain local multi-scale features, and the self-attention transform network encoder is used to process the two-dimensional image matrix to obtain global temporal features. The feature fusion and classification output module 205 is used to fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

[0098] For ease of explanation, Figure 4 Only the main components of the device are shown. The device of this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0099] In an exemplary embodiment, the present invention also provides an electronic device, the electronic device comprising: processor; A memory storing computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the vibration event classification method described above.

[0100] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the electronic device 300 may include a first processor 3001 and a memory 3002. Optionally, the electronic device 300 may also include a transceiver 3003. The first processor 3001, memory 3002, and transceiver 3003 may be connected via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the first processor 3001, implement the steps of the vibration event classification method described above.

[0101] In a specific implementation, as one example, the first processor 3001 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0102] In a specific implementation, as one example, the electronic device 300 may also include multiple processors, for example... Figure 5 The first processor 3001 and the second processor 3004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0103] The memory 3002 is used to store the software program that executes the present invention, and is controlled by the first processor 3001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0104] The transceiver 3003 is used to communicate with network devices or with terminal devices.

[0105] Optionally, the transceiver 3003 may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0106] Optionally, the transceiver 3003 can be integrated with the first processor 3001 or exist independently and be coupled to the first processor 3001 through the interface circuit of the electronic device 300. This embodiment of the invention does not specifically limit this.

[0107] It should be noted that, Figure 5The structure of the electronic device 300 shown is not intended to limit the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects of the electronic device 300 can be understood by referring to the technical effects of the above-described method embodiments, and will not be repeated here.

[0108] In an exemplary embodiment, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the steps of the vibration event classification method described above. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0109] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0110] The use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0111] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0112] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0114] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0115] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vibration event classification method based on multimodal features and a dual-branch network, characterized in that, Includes the following steps: S1. Acquire the original vibration signal and preprocess it to obtain the normalized signal; S2. For the normalized signal, multi-scale time-domain features are extracted by empirical mode decomposition, sub-band frequency-domain features are extracted by wavelet packet decomposition, the optimal sub-band frequency-domain features are selected by information entropy, integrated with multi-scale time-domain features, and dimensionality is reduced by principal component analysis to obtain the dimensionality-reduced features. S3. Use Markov transform field to convert the normalized signal into a two-dimensional image matrix; S4. Construct a dual-branch network of backpropagation algorithm-multilayer perceptron and self-attention transform network encoder. Use backpropagation algorithm-multilayer perceptron to process the dimensionality-reduced features to obtain local multi-scale features. Use self-attention transform network encoder to process the two-dimensional image matrix to obtain global temporal features. S5. Fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

2. The vibration event classification method according to claim 1, characterized in that, Step S1 specifically includes: The original vibration signal was acquired based on a distributed fiber optic sensing system, denoted as... x ( t ),in t Indicates time; The original vibration signal was preprocessed using wavelet thresholding denoising. The Symlet-8 wavelet basis function was then used to perform a 5-level decomposition of the original vibration signal, resulting in a low-frequency approximation coefficient. and 5 high-frequency detail coefficients , j = 1,2,3,4,5; A hierarchical thresholding algorithm is designed to process the high-frequency detail coefficients of each layer, with the highest frequency detail coefficient being... The corresponding threshold for the first layer is For the remaining high-frequency detail coefficients , j = 2,3,4,5, no. j Layer threshold is ; An improved soft thresholding method is used for noise reduction when the absolute value of the high-frequency detail coefficients is less than the threshold. When the absolute value of the high-frequency detail coefficient is greater than or equal to the threshold, it is considered as zero noise; At that time, the coefficients are shrunk to obtain the first value after soft thresholding. j High-frequency detail factor ; Low-frequency approximation coefficients High-frequency detail coefficients after soft thresholding The denoised signal is obtained by reconstructing the signal using inverse wavelet transform. ; For denoised signals Perform Z-score normalization to obtain the normalized signal. ,in It is a denoised signal The i One sampling point.

3. The vibration event classification method according to claim 1, characterized in that, Step S2 specifically includes: Normalized signals are obtained through empirical mode decomposition. It can be decomposed into 6 intrinsic mode functions. Indicates the first j The eigenmode function of the eigenmode function i One sampling point, j = 1,2,...,6; The normalized signal is decomposed using wavelet packet decomposition. Full-frequency domain decomposition yields eight non-overlapping full-frequency domain sub-band coefficients; Calculate the information entropy for each subband, select the top 3 subbands with the highest information entropy, and denote the corresponding subband coefficients as the optimal subband coefficients. ; The three selected optimal subband coefficients Compared with the output of empirical mode decomposition The features are integrated to form a multi-scale feature input set S, denoted as... S ={ S 1, S 2,..., S 9}, of which - Corresponding intrinsic mode functions, - Corresponding optimal sub-band coefficient; By using the root mean square (RMS) value of each feature to reflect the signal's strength, a 9-dimensional time-domain feature set is obtained. ; Dimensionality reduction was performed using principal component analysis (PCA) to obtain the dimensionality-reduced features. .

4. The vibration event classification method according to claim 1, characterized in that, Step S3 specifically includes: Normalized signal The amplitude range is divided into Q=8 equally probable quantile intervals, and the normalized signal is traversed. Each sampling point i Determine its quantile interval. Generate discrete state sequences s ; Based on discrete state sequences s Construct the Markov state transition matrix M Quantify the transition probabilities between different states; Based on discrete state sequence s The state transition order is used to extract the corresponding transition probabilities and generate a 1-dimensional Markov state sequence (MTF). The Markov state sequence MTF is resampled into an 8×8 two-dimensional image matrix. .

5. The vibration event classification method according to claim 1, characterized in that, Step S4 specifically includes: Backpropagation algorithm - Multilayer perceptron uses a structure with 3 hidden layers and ReLU activation function to process the dimensionality-reduced features. Processing is performed to obtain local multi-scale features. ; For two-dimensional image matrices The self-attention transform network encoder captures the global dependencies of state transition probabilities in an image through the Transformer self-attention mechanism, thus obtaining global temporal features. .

6. The vibration event classification method according to claim 1, characterized in that, Step S5 specifically includes: The features of the dual-branch network are integrated by vector concatenation, combining the local multi-scale features output by the backpropagation algorithm-multilayer perceptron. Global temporal features of the encoder output of the self-attention transform network Perform dimensional concatenation to obtain the fused feature vector. ; The feature vectors are fused using a fully connected layer and a softmax function. Convert to probability vectors of each category ; Take probability vector The category corresponding to the maximum value in the middle is used as the final classification label to complete the identification and classification of vibration events.

7. A vibration event classification device based on multimodal features and a dual-branch network, the device being used to implement the method as described in any one of claims 1 to 6, characterized in that, The device includes: The signal acquisition and preprocessing module is used to acquire the raw vibration signal and preprocess it to obtain the normalized signal. The multi-scale decomposition and feature extraction module is used to extract multi-scale time-domain features from normalized signals through empirical mode decomposition, extract sub-band frequency-domain features through wavelet packet decomposition, select the optimal sub-band frequency-domain features by combining information entropy, integrate them with multi-scale time-domain features, and perform dimensionality reduction through principal component analysis to obtain the dimensionality-reduced features. The Markov Transform Field module is used to convert normalized signals into two-dimensional image matrices using Markov transform fields. The dual-branch neural network module is used to construct a dual-branch network of backpropagation algorithm-multilayer perceptron and self-attention transform network encoder. The backpropagation algorithm-multilayer perceptron is used to process the dimensionality-reduced features to obtain local multi-scale features, and the self-attention transform network encoder is used to process the two-dimensional image matrix to obtain global temporal features. The feature fusion and classification output module is used to fuse local multi-scale features with global temporal features to obtain a fused feature vector, and output the final classification result through the classification module.

8. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when loaded and executed by the processor, implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images