Intelligent acoustic emission sensing device and method for scaffold safety monitoring

Abstract

The invention relates to an intelligent acoustic emission sensing device and method for scaffold safety monitoring, and belongs to the technical field of construction safety monitoring. The signal sensing module captures an abnormal sound wave signal of a scaffold structure and converts the abnormal sound wave signal into an electric signal; the electric signal is amplified, filtered and subjected to analog-to-digital conversion through the data conditioning module; extracting the characteristics of the digital signal through an intelligent abnormity judgment module and judging the abnormal state of the scaffold structure in real time; the central processor receives the discrimination result, and then the wireless transmission module sends the discrimination result to a remote terminal. Wherein the intelligent anomaly discrimination module adopts VMD, K-means, LSTM and other algorithms to carry out efficient feature extraction and real-time classification discrimination on scaffold acoustic emission signals, false alarm and missing alarm can be effectively reduced, and automatic monitoring can be realized in an early warning stage. According to the invention, the time from abnormal occurrence to information transmission can be obviously shortened, and the safety monitoring efficiency of the scaffold structure is enhanced.

Description

Technical Field

[0001] The invention belongs to the technical field of building construction safety monitoring, and relates to an intelligent acoustic emission sensor device and method for scaffold safety monitoring. Background Art

[0002] Scaffolding is a temporary support structure widely used in construction, and its safety is directly related to the life and property safety of construction workers. However, during long-term use or construction, scaffolding may be affected by external impact, material aging, loose connections or construction vibration, resulting in abnormal vibration or structural damage, and in severe cases even collapse accidents, causing major safety accidents.

[0003] At present, scaffolding safety monitoring technology usually relies on manual inspection, which requires multiple people to work, is time-consuming and labor-intensive, and is heavily dependent on the professional level of the inspectors. In addition, simple mechanical sensor devices can also be used for safety monitoring, but the sensor devices are large in size, have poor real-time performance, low sensitivity, and rely on experience. Therefore, in complex construction environments, traditional monitoring methods are difficult to detect potential safety hazards in a timely manner, which often leads to the failure to effectively prevent safety accidents in the early stages.

[0004] In order to solve the above problems, there is an urgent need for a miniaturized, highly sensitive and adaptable scaffolding safety monitoring device to improve the real-time and accuracy of detection. If the initial judgment of the abnormal state of the scaffolding can be achieved through intelligent algorithms during the data collection stage, the response speed and automation level of the scaffolding abnormality detection system will be greatly improved, and false alarms and missed alarms will be reduced. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an intelligent acoustic emission sensing device and method for scaffolding safety monitoring, which combines MEMS acoustic emission sensors with advanced VMD, K-means clustering and LSTM intelligent algorithms to effectively capture and analyze abnormal acoustic emission signals of scaffolding, realize real-time and accurate monitoring and early warning of abnormal conditions, and meet the needs of modern construction scenarios for high-precision and intelligent safety monitoring equipment.

[0006] To achieve the above object, the present invention provides an intelligent acoustic emission sensor device for scaffold safety monitoring, which includes:

[0007] A signal sensing module is used to capture abnormal acoustic wave signals of the scaffolding structure and convert them into electrical signals;

[0008] A data conditioning module, used to amplify, filter and perform analog-to-digital conversion on the electrical signal output by the signal sensing module;

[0009] Intelligent abnormality identification module, used to extract the characteristics of the output signal of the data conditioning module and identify the abnormal state of the scaffolding structure in real time;

[0010] A central processing unit, used for receiving the output result of the intelligent abnormality discrimination module;

[0011] A wireless transmission module, used to send the processing results to a remote terminal;

[0012] The signal sensing module includes a housing, a matching layer and a piezoelectric film layer. The matching layer is bonded and fixed to the bottom of the housing; the piezoelectric film layer is bonded and fixed above the matching layer. The acoustic emission signal generated by the scaffolding structure is transmitted to the piezoelectric film layer through the housing and the matching layer in turn, thereby generating a weak electrical signal.

[0013] Among them, the data conditioning module, intelligent abnormality judgment module, central processing unit and wireless transmission module are all integrated in the shell.

[0014] The data conditioning module includes a voltage amplifier, a low-pass filter and an AD converter. The voltage amplifier is used to amplify the weak electrical signal generated by the signal sensing module to improve the accuracy of subsequent processing. The low-pass filter is used to filter out high-frequency interference signals to improve the stability and accuracy of the data. The AD converter is used to convert the amplified analog signal into a digital signal so that the intelligent abnormality discrimination module can analyze and distinguish it.

[0015] The intelligent abnormality discrimination module includes a signal decomposition subunit, a feature clustering subunit and a timing discrimination subunit. The signal decomposition subunit uses the VMD algorithm to decompose the digital signal output by the data conditioning module into multiple intrinsic mode functions with different frequency characteristics, and extracts the key features in the signal; the feature clustering subunit uses the K-means clustering algorithm to cluster the sub-modes of each frequency band decomposed by the signal decomposition subunit, and assigns weights to each category of sub-modes; the timing discrimination subunit uses the weighted category features output by the feature clustering subunit as the input of LSTM, and the output layer classifies the signal through the activation function, and judges it as "normal" or "abnormal" state, thereby realizing the preliminary identification of the abnormal state of the scaffolding structure.

[0016] Furthermore, in the feature clustering subunit, weights are assigned to sub-modalities in each category, including: first, selecting sub-modalities with larger mean and smaller variance in each category after clustering as evaluation criteria; calculating the Euclidean distance between the evaluation criteria of each category and the original signal, and mapping the Euclidean distance to the weight value of each category sub-modal relative to the original signal through a mapping function.

[0017] Furthermore, the shell is made of stainless steel; the matching layer is made of alumina; and the piezoelectric film layer is made of aluminum nitride.

[0018] In another aspect, the present invention provides a method for scaffold safety monitoring, the method comprising:

[0019] S1, firstly collect the acoustic emission signal generated by the abnormal scaffold structure and convert it into an electrical signal;

[0020] S2, amplifying, filtering and analog-to-digital conversion of the electrical signal;

[0021] S3, using VMD decomposition algorithm to decompose the digital signal into a series of intrinsic mode functions IMF with narrowband spectrum, so as to extract high-frequency, medium-frequency and low-frequency characteristic information in the signal, so as to analyze abnormal information from different frequency bands;

[0022] S4. Perform K-means clustering on the high-frequency, medium-frequency and low-frequency sub-modes obtained after decomposition. After clustering, select the sub-mode with larger mean and smaller variance in each category as the evaluation standard, and use the evaluation standard to determine the weight of each category of sub-mode;

[0023] S5. The weighted category features obtained after clustering are used as the input of LSTM. The outputs of multiple LSTM layers are weighted and fused to obtain a comprehensive feature. The weighted fused comprehensive features are then classified through the activation function to determine whether they are in a "normal" or "abnormal" state, thereby achieving preliminary identification of the abnormal state of the scaffolding structure.

[0024] S6. Transmit the discrimination signal output by the LSTM network to the remote monitoring terminal through the wireless network, so that the user can understand the scaffolding structure status in real time.

[0025] Furthermore, in step S4, the weights of each category of sub-modality are determined by using the evaluation criteria, including first calculating the Euclidean distance between the evaluation criteria of each category and the original signal:

[0026]

[0027] In the formula, D(P i ,P * ) represents the evaluation criteria P of the i-th category i With the original signal P * The Euclidean distance between i Indicates the evaluation criteria P i Mean * Represents the mean of the original signal, Var i Indicates the evaluation criteria P i The variance of * is the variance of the original signal;

[0028] The inverse of the Euclidean distance is mapped to the weight value of each category sub-modality relative to the original signal through a mapping function:

[0029]

[0030] In the formula, a i represents the weight value of the ith sub-modality relative to the original signal, and F(·) represents the mapping function.

[0031] Furthermore, in step S6, when the LSTM network outputs a judgment signal of an "abnormal" state, a warning signal is simultaneously generated and transmitted to the remote monitoring terminal.

[0032] The beneficial effects of the present invention are as follows: the present invention realizes the miniaturization and high-sensitivity design of the sensor device through MEMS technology, so that it can adapt to various complex construction environments, including construction scenes with high noise and high vibration. The high sensitivity and low power consumption characteristics of MEMS piezoelectric film, combined with the use of intelligent algorithms, can greatly improve the reliability and real-time performance of anomaly detection; in addition, the intelligent anomaly discrimination module uses algorithms such as VMD, K-means and LSTM to efficiently extract features and classify and discriminate scaffolding acoustic emission signals in real time, which can effectively reduce false alarms and missed alarms, and can realize automatic monitoring in the early warning stage.

[0033] Compared with traditional scaffold monitoring equipment, the present invention significantly shortens the time from abnormal occurrence to information transmission, and improves the efficiency of scaffold structure safety monitoring. The real-time transmission function also enables monitoring personnel to obtain scaffold status information instantly, so as to quickly take corresponding measures to reduce the risk of construction accidents and ensure construction safety.

[0034] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0036] Figure 1 A schematic diagram of the structure of an intelligent acoustic emission sensor device for scaffold safety monitoring provided by one embodiment of the present invention;

[0037] Figure 2 It is a preliminary leakage identification flow chart of the intelligent acoustic emission sensor device;

[0038] Figure 3 This is a flow chart of intelligent abnormality identification of intelligent acoustic emission sensor device.

[0039] Figure numerals: 1-housing; 2-matching layer; 3-piezoelectric film layer; 4-data conditioning module; 5-intelligent abnormality judgment module; 6-central processing unit; 7-wireless transmission module; 8-adhesive layer. DETAILED DESCRIPTION

[0040] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0042] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0043] like Figure 1 As shown, an intelligent acoustic emission sensor device for scaffold safety monitoring provided by an embodiment of the present invention includes a signal sensing module, a data conditioning module, an intelligent abnormality discrimination module, a central processing unit and a wireless transmission module.

[0044] Among them, the signal sensing module is used to capture the acoustic emission signal generated by the abnormal scaffolding structure and convert it into an electrical signal. It includes a sensing device housing 1, a matching layer 2 and a MEMS piezoelectric film layer 3. The matching layer 2 is fixed at the bottom of the housing 1, and the piezoelectric film layer 3 is fixed above the matching layer 2. The matching layer 2 and the housing 1, and the piezoelectric film layer 3 and the matching layer 2 are fixed by an adhesive layer 8. Among them, the housing 1 is made of stainless steel material, which has corrosion resistance and good mechanical strength, and is suitable for use in a construction environment. The acoustic emission signal generated by the scaffolding structure is transmitted to the piezoelectric film layer 3 through the housing 1 and the matching layer 2 in turn, generating a weak electrical signal. The matching layer 2 uses alumina material, which can reduce the reflection of sound waves between the housing 1 and the piezoelectric film layer 3 and enhance the coupling effect of the signal; the piezoelectric film layer 3 uses ALN (aluminum nitride) material, which has the characteristics of high sensitivity and low power consumption, and can accurately detect weak sound wave signals.

[0045] The data conditioning module 4 is used to amplify, filter and digitize the electrical signal collected by the signal sensing module so as to be suitable for subsequent intelligent analysis. Among them, the voltage amplifier adopts a low-noise amplifier (such as MAX40078) for amplification, which can amplify the weak electrical signal generated by the piezoelectric film layer 3 to a processable range. The bandpass filter adopts the LT6004 chip design to filter out useless noise signals, thereby improving the accuracy and stability of the signal. The AD converter (such as AD7606) converts the analog signal into a digital signal, providing a clear digital timing signal for the intelligent abnormality discrimination module 5.

[0046] The intelligent abnormality discrimination module 5 adopts an embedded AI chip, equipped with VMD, K-means and LSTM algorithms, which can realize the decomposition, clustering and real-time discrimination of abnormal signals of scaffolding structures. The intelligent abnormality discrimination module is divided into sub-units such as signal decomposition, feature clustering, and time series discrimination. The signal decomposition sub-unit adopts the VMD decomposition algorithm to perform variational modal decomposition on the digital signal, decomposing the signal into a series of intrinsic mode functions (IMFs) with narrowband spectrum, thereby extracting high-frequency, medium-frequency and low-frequency information in the signal, so as to analyze abnormal information from different frequency bands; the feature clustering sub-unit adopts the K-means clustering algorithm to perform K-means clustering on the high-frequency, medium-frequency and low-frequency sub-modes obtained after VMD decomposition. After clustering, the sub-mode with larger mean and smaller variance in each category is selected as the evaluation standard, and the weight of the sub-mode in each category is determined by using the evaluation standard, thereby weakening the influence of points with larger differences on the clustering results, and finally obtaining the weight of each sub-modal component after VMD decomposition. The temporal discrimination subunit takes the weighted submodal features output by the feature clustering subunit as the input of LSTM. The output layer classifies the signal through the activation function and distinguishes it as "normal" or "abnormal" state, thereby achieving preliminary identification of the abnormal state of the scaffolding structure.

[0047] The central processor 6 (such as STM32) is responsible for receiving the output result of the intelligent abnormality discrimination module 5. When a scaffold structure abnormality signal is detected, the central processor 6 generates an alarm signal and transmits the signal to the wireless transmission module 7. The adopted STM32 processor has the characteristics of low power consumption and high performance, is suitable for application in real-time monitoring devices, and can efficiently process and quickly respond to scaffold state changes.

[0048] The wireless transmission module 7 adopts NB-IOT technology to transmit the processing results to the remote monitoring terminal. The NB-IOT module has the characteristics of low power consumption and wide coverage. When the scaffolding structure is abnormal, it can quickly and accurately transmit the detection results to the monitoring center to achieve real-time monitoring and remote response.

[0049] Another embodiment of the present invention provides a method for scaffold safety monitoring, such as Figure 2 and Figure 3 As shown, the method includes:

[0050] 1. Capture the acoustic emission signal generated by the abnormal scaffold structure through the signal sensing module and convert it into an electrical signal;

[0051] 2. Amplify the electrical signal output by the signal sensing module, thereby amplifying the weak electrical signal to a processable range; perform bandpass filtering on the amplified electrical signal to filter out useless noise signals; and convert the filtered electrical signal into a digital signal;

[0052] 3. Use VMD decomposition algorithm to decompose the digital signal into multiple intrinsic mode functions (IMFs) with different frequency characteristics, and extract key features in the signal, including:

[0053] 1) First, decompose the original digital signal into several intrinsic mode functions u k , so that each modal component has a narrowband spectrum characteristic, and at the same time set the number of modes K and the penalty factor α; initialize the modal component u k , center frequency ω k , Lagrangian operator λ.

[0054] 2) For u k Perform Hlibert transform to obtain a single-sided spectrum, and then multiply it by the exponential harmonic to tune to the corresponding center frequency ω k , so that the spectrum is shifted to baseband, and Gauss smoothing is used to estimate ω k The bandwidth of . Then the VMD constrained variational model is as follows:

[0055]

[0056]

[0057] in, represents the gradient; δ(t) represents the unit pulse function; j is the imaginary part, t represents the time variable, f(t) represents the original digital signal, ω k t Represents the center frequencies of the K modes decomposed by the VMD algorithm.

[0058] 3) While requiring the IMF bandwidth to be as narrow as possible, it is also necessary to prevent serious distortion of the IMF reconstruction. Therefore, it is necessary to use a quadratic penalty term to optimize the reconstruction fidelity and introduce the Lagrange multiplier λ(t) to keep the conditional constraints strict. The above formula is transformed as follows:

[0059]

[0060] Where λ(t) is the Lagrange multiplier.

[0061] Then continue to use the alternating direction multiplier algorithm to alternately change u k ,ω k , λ. Finally, the accuracy is calculated according to the following formula (ε is the set accuracy threshold), and the K modal components solved are outputted after the requirements are met.

[0062]

[0063] 4. Perform K-means clustering on each sub-modality obtained after VMD decomposition; after clustering, select the sub-modality with larger mean and smaller variance in each category as the evaluation criterion, use the evaluation criterion as the representative of each category of sub-modality and calculate the weight of each category to weaken the influence of points with larger differences on the clustering results.

[0064] Specifically, the similarity between each category sub-modality and the real data is determined by calculating the Euclidean distance between the evaluation criteria of each category and the original signal. The smaller the difference in distance, the closer the category sub-modality is to the real data value. The Euclidean distance is mapped to the weight value of the category sub-modality relative to the original signal through a mapping function.

[0065] The Euclidean distance calculation formula for each category of sub-modality is:

[0066]

[0067]

[0068]

[0069] Among them, D(P i ,P * ) represents the evaluation criteria P of the i-th category i With the original signal P* The Euclidean distance between two data is used to measure the difference between the two data, that is, the difference between their mean and variance. The smaller the distance, the more similar the evaluation standard is to the original signal, and the weight will be relatively large. i The evaluation criteria P i Mean * is the mean of the original data, Var i The evaluation criteria P i The variance of * is the variance of the original data. F(x) is the mapping function used to convert the Euclidean distance D(P i ,P * ) is mapped to weights, and larger Euclidean distances can be mapped to smaller weight values, thereby highlighting categories with higher similarity (smaller distances). When F(x) approaches 1, it means that this type of sub-modality is very similar to the original data; when F(x) approaches 0, it means that this type of sub-modality is very similar to the original data.

[0070] By selecting sub-modes that are more representative and less variable as evaluation criteria, the influence of abnormal sub-modes with smaller means and larger variances can be weakened. The purpose of selecting the evaluation criteria is to ensure that the selection of sub-modes can better reflect the main characteristics of the signal while avoiding the influence of outliers on the weighted results.

[0071] 5. Use the weighted sub-modal features obtained after clustering as the input of LSTM.

[0072] LSTM uses its built-in memory unit to perform temporal modeling on the input weighted features and learn the temporal dependencies between features. The trained LSTM network can judge the features of real-time input, and the outputs of multiple LSTM layers are integrated through weighted fusion to generate a comprehensive feature. Then, the comprehensive feature classifies the weighted fused signal through the activation function and judges it as "normal" or "abnormal", thereby realizing the preliminary identification of the abnormal state of the scaffolding structure.

[0073] 6. The central processor transmits the discrimination signal output by LSTM to the remote monitoring terminal through the wireless transmission terminal, so that the user can understand the scaffolding structure status in real time. If the LSTM outputs a discrimination signal of "abnormal" state, the central processor generates an alarm signal at the same time and transmits the alarm signal to the remote monitoring terminal through wireless transmission.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. An intelligent acoustic emission sensor device for scaffolding safety monitoring, characterized in that: The device includes: A signal sensing module is used to capture abnormal acoustic wave signals of the scaffolding structure and convert them into electrical signals; A data conditioning module, used for amplifying, filtering and analog-to-digital conversion of the electrical signal output by the signal sensing module; An intelligent abnormality identification module is used to extract the characteristics of the output signal of the data conditioning module and identify the abnormal state of the scaffolding structure in real time; A central processing unit, used for receiving the result output by the intelligent abnormality discrimination module; A wireless transmission module, used to send the processing results to a remote terminal; The intelligent abnormality discrimination module includes a signal decomposition subunit, a feature clustering subunit and a timing discrimination subunit; the signal decomposition subunit uses the VMD algorithm to decompose the digital signal output by the data conditioning module into multiple intrinsic mode functions with different frequency characteristics, and extracts key features in the signal; the feature clustering subunit uses the K-means clustering algorithm to cluster the frequency band sub-modes decomposed by the signal decomposition subunit, and assigns weights to each category of sub-modes; the timing discrimination subunit uses the weighted category features output by the feature clustering subunit as the input of the LSTM, and the output layer classifies the signal through the activation function, and judges it as a "normal" or "abnormal" state, thereby realizing the preliminary identification of the abnormal state of the scaffolding structure.

2. The intelligent acoustic emission sensor device according to claim 1, characterized in that: In the feature clustering subunit, weights are assigned to sub-modalities of each category, including: first, selecting sub-modalities with larger mean and smaller variance in each category after clustering as evaluation criteria; calculating the Euclidean distance between the evaluation criteria of each category and the original signal, and mapping the Euclidean distance to the weight value of each category sub-modal relative to the original signal through a mapping function.

3. The intelligent acoustic emission sensor device according to claim 1, characterized in that: The signal sensing module includes a shell, a matching layer and a piezoelectric film layer; the matching layer is bonded and fixed to the bottom of the shell; the piezoelectric film layer is bonded and fixed above the matching layer; the acoustic emission signal generated by the scaffolding structure is transmitted to the piezoelectric film layer through the shell and the matching layer in turn, thereby generating a weak electrical signal.

4. The intelligent acoustic emission sensor device according to claim 3, characterized in that: The shell is made of stainless steel; the matching layer is made of aluminum oxide; and the piezoelectric film layer is made of aluminum nitride.

5. The intelligent acoustic emission sensor device according to claim 1 or 3, characterized in that: The data conditioning module, intelligent abnormality identification module, central processing unit and wireless transmission module are all integrated in the housing.

6. A method for scaffold safety monitoring, characterized in that: The method comprises: firstly collecting the acoustic emission signal generated by the abnormal scaffold structure and converting it into an electrical signal; Amplify, filter and perform analog-to-digital conversion on electrical signals; The VMD decomposition algorithm is used to decompose the digital signal into a series of intrinsic mode functions (IMFs) with narrowband spectra, thereby extracting high-frequency, medium-frequency and low-frequency characteristic information from the signal, so as to analyze abnormal information from different frequency bands; K-means clustering is performed on the high-frequency, medium-frequency and low-frequency sub-modes obtained after decomposition. After clustering, the sub-mode with larger mean and smaller variance in each category is selected as the evaluation standard, and the weight of each category of sub-mode is determined by using the evaluation standard; The weighted category features obtained after clustering are used as the input of LSTM. The outputs of multiple LSTM layers are weighted and fused to obtain a comprehensive feature. The weighted fused comprehensive features are then classified through the activation function to determine whether they are in a "normal" or "abnormal" state, thereby achieving preliminary identification of the abnormal state of the scaffolding structure. The discrimination signal output by the LSTM network is transmitted to the remote monitoring terminal through a wireless network, so that users can understand the scaffolding structure status in real time.

7. The method according to claim 6, characterized in that The method of using the evaluation criteria to determine the weights of each category of sub-modalities includes first calculating the Euclidean distance between the evaluation criteria of each category and the original signal: In the formula, D(P i ,P * ) represents the evaluation criteria P of the i-th category i With the original signal P * The Euclidean distance between i Indicates the evaluation criteria P i Mean * Represents the mean of the original signal, Var i Indicates the evaluation criteria P i The variance of * is the variance of the original signal; The inverse of the Euclidean distance is mapped to the weight value of each category sub-modality relative to the original signal through a mapping function: In the formula, a i represents the weight value of the ith sub-modality relative to the original signal, and F(·) represents the mapping function.

8. The method according to claim 6, characterized in that When the LSTM network outputs a judgment signal of an "abnormal" state, an early warning signal is simultaneously generated and transmitted to the remote monitoring terminal.

Images