Method and system for predicting and diagnosing abnormal operation of elevator based on Internet of Things

Abstract

The invention provides an elevator operation abnormity prediction and diagnosis method and system based on the Internet of Things, and relates to the technical field of elevator fault diagnosis. The method comprises the following steps: acquiring historical audio data, performing time-frequency conversion to generate a plurality of first time-frequency diagrams, and clustering the historical audio data to obtain a plurality of elevator running state class clusters; generating a plurality of second time-frequency diagrams of each elevator running state class cluster; performing event flow detection on each second time-frequency diagram to obtain event flow characteristic data of each second time-frequency diagram, performing time sequence abnormity diagnosis on the second time-frequency diagrams to generate an abnormity diagnosis result, and performing characteristic optimization on the second time-frequency diagrams to generate third time-frequency diagrams corresponding to the second time-frequency diagrams; and a training set is obtained through construction, the elevator operation analysis model is trained, the elevator operation audio data are analyzed through the trained elevator operation analysis model, and an elevator operation anomaly detection result is generated. According to the invention, efficient prediction and fault diagnosis of abnormal operation of the elevator are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator fault diagnosis, and in particular to an elevator operation abnormality prediction and diagnosis method and system based on the Internet of Things. Background Art

[0002] With the rapid development of Internet of Things technology, intelligent management of elevators has become an indispensable part of urban construction. Modern elevator systems rely on a large amount of sensor data to achieve real-time monitoring and predictive analysis of operating status. Among them, audio signals, as diagnostic signals that cannot be ignored during elevator operation, can reflect the mechanical condition, load changes and potential faults of elevators under different working conditions. By analyzing the audio data of elevator operation, abnormal modes in elevator operation can be identified in advance, and faults can be predicted and diagnosed, thereby improving the safety, reliability and operating efficiency of elevators.

[0003] For the collected elevator audio signals, the spectrum features are extracted by combining traditional signal processing methods such as time-frequency analysis, and machine learning algorithms are used for fault detection. This method can identify some abnormal modes in elevator operation to a certain extent, but because it does not fully consider the characteristic changes of different types of faults in different working conditions of the elevator, it is easy to lead to limited diagnostic accuracy. Summary of the invention

[0004] The purpose of the present invention is to propose a method and system for predicting and diagnosing elevator operation abnormalities based on the Internet of Things, to solve the problem that some methods cannot flexibly adapt to variable operating states and load changes, to consider the differences in elevator operating states and the complex changes in abnormal events, and to improve the accuracy of elevator fault diagnosis.

[0005] To achieve the above-mentioned object, the first aspect of the present invention provides an elevator operation abnormality prediction and diagnosis method based on the Internet of Things, comprising: Acquire historical audio data collected by the IoT sensor during the operation of the elevator, and perform time-frequency conversion on the historical audio data to generate a plurality of first time-frequency graphs; Extract multiple spectral features corresponding to each first time-frequency graph and construct a spectral feature vector corresponding to each time-frequency graph, cluster the historical audio data according to the spectral feature vector to obtain multiple elevator operation status clusters, each elevator operation status cluster contains multiple groups of audio signals; Based on the variational modal decomposition algorithm, adaptive time-frequency analysis is performed on each elevator operation state cluster to generate multiple target modal signals of each elevator operation state cluster, and time-frequency conversion and feature map splicing are performed on the multiple target modal signals of each elevator operation state cluster to obtain multiple second time-frequency maps of each elevator operation state cluster; Performing event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, performing time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, performing feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generating a third time-frequency graph corresponding to each second time-frequency graph; A training set for training an elevator operation analysis model is constructed based on multiple third time-frequency graphs, and the elevator operation analysis model is trained. After the elevator operation audio data to be analyzed is collected, the elevator operation audio data is analyzed by the trained elevator operation analysis model to generate an elevator operation abnormality detection result.

[0006] Preferably, for each elevator operation status cluster, the plurality of second time-frequency graphs further include: The multiple groups of audio signals in each elevator operation status cluster are modally decomposed by a variational modal decomposition algorithm to obtain multiple sub-signals corresponding to each group of audio signals, and the total energy value of each sub-signal is calculated. The sub-signal whose total energy value of each group of audio signals is greater than a preset energy threshold is recorded as a target modal signal; Short-time Fourier transform is performed on multiple target modal signals of each elevator operation status cluster to obtain a local time-frequency diagram of each target modal signal. Feature fusion is performed on the local time-frequency diagrams corresponding to the multiple target modal signals of each group of audio signals to obtain a global time-frequency diagram of each group of audio signals, which is recorded as the second time-frequency diagram of the audio signal.

[0007] Preferably, event flow detection is performed on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, and time series abnormality diagnosis is performed on multiple second time-frequency graphs of each elevator operation status cluster to generate abnormality diagnosis results, including: Determine the fault type label of each second time-frequency graph according to the historical audio data, filter out multiple groups of audio signals belonging to the normal operating state in each elevator operation state cluster according to the fault type label, and construct a multi-band event abnormal threshold for each elevator operation state cluster according to the filtered multiple groups of audio signals; Perform sliding window analysis on each second time-frequency graph, mark the second time-frequency graph with abnormal events according to the multi-band event abnormality threshold corresponding to the elevator operation status cluster to which the second time-frequency graph belongs, and obtain event stream feature data containing multiple abnormal event streams for each second time-frequency graph; The event timing data and corresponding reference data of each abnormal event are extracted according to the preset time step, the event timing difference vector of each abnormal event flow is extracted based on the reference data corresponding to the event timing data, the timing difference parameters of each abnormal event flow are determined according to the event timing difference vector, and the abnormal diagnosis result in which each second time-frequency diagram contains the corresponding timing difference parameters is obtained.

[0008] Preferably, the feature optimization of multiple second time-frequency graphs of each elevator operation status cluster is performed according to the event stream feature data and the abnormality diagnosis result to generate a third time-frequency graph corresponding to each second time-frequency graph, including: Determine a fault type parameter of each second time-frequency graph according to a plurality of timing difference parameters and a fault type label, determine an operating state parameter of each second time-frequency graph according to a plurality of timing difference parameters and an operating state label, and determine an abnormal event factor of each abnormal event according to the fault type parameter and the operating state parameter; Based on the event flow feature data, the abnormal event impact interval of each second time-frequency graph is determined, and the abnormal event impact interval of each second time-frequency graph is feature optimized according to the abnormal event factor to generate a third time-frequency graph corresponding to each second time-frequency graph.

[0009] Preferably, constructing a training set for training an elevator operation analysis model according to a plurality of third time-frequency graphs and training the elevator operation analysis model comprises: Perform pseudo-color mapping on the multiple third time-frequency graphs of each elevator operation status cluster to obtain multiple target frequency spectrum graphs of each elevator operation status cluster; Determine the fault type label and the operating status label of each target spectrum graph, and construct a training set including multiple target spectrum graphs and the fault type label and the operating status label corresponding to each target spectrum graph; The multiple target spectrograms in the training set and the operating status label of each spectrogram are used as the input of the elevator operation analysis model, and the fault type label of each spectrogram is used as the training target of the elevator operation analysis model. The elevator operation analysis model is obtained by training the training set.

[0010] Preferably, the elevator operation audio data is analyzed by a trained elevator operation analysis model to generate an elevator operation abnormality detection result, including: An adaptive time-frequency analysis is performed on the elevator operation audio data based on the variational mode decomposition algorithm, and time-frequency conversion and feature map splicing are performed to generate a target time-frequency map. The motion feature data corresponding to the elevator operation audio data is obtained to determine the target operation state label of the elevator operation audio data. The target time-frequency map containing the target operation state label is input into the elevator operation analysis model to generate an elevator operation abnormality detection result corresponding to the elevator operation audio data.

[0011] The second aspect of the present invention provides an elevator operation abnormality prediction and diagnosis system based on the Internet of Things, which is used to implement the above-mentioned elevator operation abnormality prediction and diagnosis method based on the Internet of Things, including: A data acquisition module is used to obtain historical audio data collected by IoT sensors during the operation of the elevator; The operation status analysis module is used to perform time-frequency conversion on the historical audio data to generate multiple first time-frequency graphs, extract multiple frequency spectrum features corresponding to each first time-frequency graph and construct a frequency spectrum feature vector corresponding to each time-frequency graph, cluster the historical audio data according to the frequency spectrum feature vector to obtain multiple elevator operation status clusters, each elevator operation status cluster contains multiple groups of audio signals; A time-frequency feature analysis module is used to perform adaptive time-frequency analysis on each elevator operation status cluster based on a variational modal decomposition algorithm, generate multiple target modal signals of each elevator operation status cluster, perform time-frequency conversion and feature graph splicing on the multiple target modal signals of each elevator operation status cluster, and obtain multiple second time-frequency graphs of each elevator operation status cluster; A time-frequency feature optimization module is used to perform event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, perform time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, perform feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generate a third time-frequency graph corresponding to each second time-frequency graph; The operation abnormality analysis module is used to construct a training set for training the elevator operation analysis model based on multiple third time-frequency graphs and train the elevator operation analysis model. After collecting the elevator operation audio data to be analyzed, the elevator operation audio data is analyzed by the trained elevator operation analysis model to generate the elevator operation abnormality detection result.

[0012] Preferably, for the operation abnormality analysis module, the elevator operation audio data is analyzed by a trained elevator operation analysis model to generate an elevator operation abnormality detection result, including: An adaptive time-frequency analysis is performed on the elevator operation audio data based on the variational mode decomposition algorithm, and time-frequency conversion and feature map splicing are performed to generate a target time-frequency map. The motion feature data corresponding to the elevator operation audio data is obtained to determine the target operation state label of the elevator operation audio data. The target time-frequency map containing the target operation state label is input into the elevator operation analysis model to generate an elevator operation abnormality detection result corresponding to the elevator operation audio data.

[0013] The beneficial effects of the present invention are as follows: The present invention performs adaptive time-frequency analysis on elevator audio signals through variational mode decomposition (VMD), and can decompose the complex audio signals of the elevator into multiple modal signals. The effective features of the time-frequency graph can be further enhanced by feature graph splicing, so as to better capture the signal changes in different frequency bands. By performing event flow detection and timing anomaly diagnosis on the time-frequency graph, potential abnormal events in the elevator operation process can be extracted, and the timing information in the elevator operation process can be fully utilized to help the model not only identify the anomalies of single events, but also capture the cumulative effects of abnormal events in different time periods. Taking into account the interactive effects between different fault types and elevator operation states, the abnormal event factors are calculated and feature optimization is performed, so that the time-frequency graph can contain more accurate fault and operation state information, thereby improving the model's ability to identify and locate different faults, and realizing efficient prediction and fault diagnosis of elevator operation anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an elevator operation abnormality prediction and diagnosis method based on the Internet of Things is provided in an embodiment of the present invention.

[0015] Figure 2 A structural block diagram of an elevator operation abnormality prediction and diagnosis system based on the Internet of Things provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make those skilled in the art better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] like Figure 1 As shown, an embodiment of the present invention provides an elevator operation abnormality prediction and diagnosis method based on the Internet of Things, which specifically includes the following steps: Step S1, obtaining historical audio data collected during the operation of the elevator based on the Internet of Things sensor, performing time-frequency conversion on the historical audio data to generate multiple first time-frequency graphs, extracting multiple spectral features corresponding to each first time-frequency graph and constructing a spectral feature vector corresponding to each time-frequency graph, and clustering the historical audio data according to the spectral feature vectors to obtain multiple elevator operation status clusters.

[0018] In this embodiment, the historical audio data during the operation of the elevator can be collected through different IoT sensors such as microphone arrays, vibration sensors, etc. These sensors can monitor the sound, vibration and other related parameters of the elevator in real time, and generate high-quality audio data, which includes the audio characteristics of the elevator in different operating states, and characterizes the audio performance of the elevator in normal operation or in the presence of a fault. The time-frequency conversion of these historical audio data can be performed by using short-time Fourier transform (STFT), wavelet transform or other time-frequency analysis methods, or further extracting Mel spectrum diagrams to obtain the first time-frequency diagram of the time-frequency characteristics of the audio signal of the historical audio data, which is used to characterize the changes in different frequency components of the audio signal at different time points during the operation of the elevator.

[0019] Considering that the actual performance of different operating faults will be different under different operating states of the elevator, in order to improve the accuracy of fault detection, a specific analysis is performed on each type of operating state. After obtaining the first time-frequency graph, the spectrum features in each first time-frequency graph are further extracted to construct the corresponding spectrum feature vector. Taking the Mel spectrum as an example, the spectrum features such as energy, zero crossing rate, spectrum centroid, and spectrum roll-off are extracted to construct the feature vector, which is used to effectively reflect the characteristics of the elevator audio signal. The spectrum feature vector will be used for subsequent clustering analysis. Through clustering algorithms such as K-means and DBSCAN, the historical audio data is divided into multiple elevator operating state clusters. Here, the K-means algorithm is taken as an example. Multiple operating states of the elevator can be determined according to prior knowledge, such as starting, accelerating, decelerating, uniform speed, parking, etc., so as to set the corresponding K value, and realize the classification of historical audio data based on the operating state to obtain multiple elevator operating state clusters. Each elevator operating state cluster contains multiple groups of audio signals, which represent the data obtained from multiple detections of the elevator in this operating state.

[0020] Step S2: Based on the variational modal decomposition algorithm, adaptive time-frequency analysis is performed on each elevator operation state cluster to generate multiple target modal signals for each elevator operation state cluster, and time-frequency conversion and feature graph splicing are performed on the multiple target modal signals for each elevator operation state cluster to obtain multiple second time-frequency graphs for each elevator operation state cluster.

[0021] In this embodiment, after the audio data is clustered, the variational mode decomposition (VMD) algorithm is used to perform further adaptive time-frequency analysis on the audio signal for each elevator operation status cluster. The VMD algorithm can decompose complex non-stationary signals into multiple intrinsic mode functions (IMFs), each of which represents a specific frequency component of the signal, characterizing the different frequency components of the audio signal of the elevator in this operating state, and through adaptive processing, determine the representative IMF signal and record it as the target modal signal. Then, for each target modal signal, time-frequency conversion and feature map splicing are performed, with the aim of fusing multiple target modal signals under each group of audio signals to obtain a second time-frequency map containing rich time-frequency information for each group of audio signals.

[0022] Step S3, perform event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, perform timing anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, perform feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generate a third time-frequency graph corresponding to each second time-frequency graph.

[0023] In this embodiment, event flow detection is performed on each second time-frequency graph, the purpose of which is to extract sudden events or abnormal behaviors during the operation of the elevator from the time-frequency graph, and event flow analysis methods such as sliding detection based on time windows are used to obtain event flow feature data of each time-frequency graph, which contains the specific location of the abnormal event in the second time-frequency graph corresponding to the data with some operating faults. At the same time, the context information of each abnormal event is further analyzed to extract the spectrum feature change information within a certain period of time before and after the abnormal event occurs, to achieve the time sequence abnormality diagnosis of each abnormal event, to obtain the abnormal diagnosis result of the context spectrum feature information characterizing the abnormal event, and finally to optimize the features of each time-frequency graph in combination with the event flow feature data. In order to enhance the effective signal components in the time-frequency graph, a more accurate third time-frequency graph is generated to obtain feature data containing more accurate elevator operation status and fault characteristics.

[0024] Step S4: construct a training set for training the elevator operation analysis model based on multiple third time-frequency graphs and train the elevator operation analysis model. After collecting the elevator operation audio data to be analyzed, analyze the elevator operation audio data through the trained elevator operation analysis model to generate an elevator operation abnormality detection result.

[0025] In this embodiment, the multiple third time-frequency graphs in the different elevator operation state clusters obtained by the above steps contain characteristic information of the audio signal of the elevator in different operation states. The multiple third time-frequency graphs are used as input data to construct a training set containing multiple elevator operation states and their abnormal modes. It includes not only data of normal operation states, but also abnormal data under various fault types. The training set is used to train the elevator operation analysis model, and is trained using deep learning models such as convolutional neural networks (CNN) and deep neural networks (DNN), so as to achieve accurate prediction of elevator faults.

[0026] In actual applications, when the elevator operation audio data to be analyzed is collected, the audio data can be directly analyzed through the trained elevator operation analysis model. The model can generate corresponding elevator operation abnormality detection results based on the input audio data, analyze the minor anomalies in the data, diagnose and warn possible operation faults in advance, and accurately identify possible abnormal events in the elevator operation process.

[0027] Optionally, multiple second time-frequency graphs of each elevator operation status cluster are generated in the following manner: The multiple groups of audio signals in each elevator operation status cluster are modally decomposed by the variational modal decomposition algorithm to obtain multiple sub-signals corresponding to each group of audio signals. The total energy value of each sub-signal is calculated, and the sub-signal whose total energy value of each group of audio signals is greater than the preset energy threshold is recorded as the target modal signal.

[0028] In this embodiment, for multiple groups of audio signals in each elevator operation status cluster, each audio signal is first decomposed using the VMD algorithm to obtain a number of sub-signals (IMF), which can represent the signal components of each frequency band during the operation of the elevator. After obtaining multiple sub-signals, multiple target modal signals under each audio signal are further screened out. The target modal signal usually contains important spectral components related to abnormal operation or failure of the elevator. Specifically, by calculating the energy of each sub-signal, for example, by calculating the average value of the square of its amplitude to obtain the total energy value of each sub-signal, and then filtering out the target modal signal through a preset energy threshold, the sub-signal with a total energy value greater than the preset energy threshold is selected because it usually contains more significant features and may represent a key fault or abnormal signal during the operation of the elevator.

[0029] Short-time Fourier transform is performed on multiple target modal signals of each elevator operation status cluster to obtain the local time-frequency diagram of each target modal signal. Feature fusion is performed on the local time-frequency diagrams corresponding to the multiple target modal signals of each group of audio signals to obtain the global time-frequency diagram of each group of audio signals.

[0030] In this embodiment, for each target modal signal, a feature graph containing spectral information is further extracted. Specifically, a short-time Fourier transform (STFT) is used to generate a local time-frequency graph of each target modal signal. This process divides the audio signal into short time periods through a sliding window technique, and performs a Fourier transform on each short time period, thereby obtaining information on how the frequency distribution of the signal changes over time.

[0031] Since each group of audio signals contains multiple target modal signals, after generating the local time-frequency graph of each target modal signal, the multiple local time-frequency graphs are feature fused to finally obtain the global time-frequency graph of each group of audio signals, which is recorded as the second time-frequency graph of the audio signal, thereby obtaining multiple second time-frequency graphs of each elevator operation status cluster. In this process, the multiple local time-frequency graphs of each target modal signal can be weighted fused. For example, by calculating the energy values ​​of multiple local time-frequency graphs and determining the corresponding weights, the multiple local time-frequency graphs are weighted fused to strengthen those time-frequency features that are more critical to fault diagnosis. The final generated global time-frequency graph, i.e., the second time-frequency graph, is a two-dimensional matrix of higher dimension, which integrates the time-frequency features of multiple target modal signals. Each graph represents the comprehensive performance of the audio signal in the cluster after modal decomposition, short-time Fourier transform and feature fusion, which can better characterize the elevator operation status or fault characteristics.

[0032] Optionally, event flow detection is performed on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, and time series anomaly diagnosis is performed on multiple second time-frequency graphs of each elevator operation status cluster to generate an anomaly diagnosis result, including: The fault type label of each second time-frequency graph is determined according to the historical audio data, and multiple groups of audio signals belonging to the normal operating state in each elevator operating state cluster are filtered out according to the fault type label. The multi-band event abnormal threshold of each elevator operating state cluster is constructed according to the filtered multiple groups of audio signals.

[0033] In this embodiment, the historical audio data contains information related to the fault status of the elevator under different data, such as a fault type label that characterizes whether the elevator has an operating fault. Through the known fault type label, multiple groups of audio signals belonging to the normal operating state can be screened out from each elevator operating state cluster. These audio signals usually do not contain abnormal or fault information, but can be used to assist in establishing baseline features under normal conditions. Specifically, based on the audio signal data under normal operating conditions, the multi-band event abnormality threshold of each elevator operating state cluster is calculated. This threshold represents the amplitude range of the elevator audio signal in different frequency bands under normal conditions. Statistical methods such as mean and standard deviation can be used to determine the normal fluctuation range of each frequency band, and the threshold can be set to the mean plus or minus a certain multiple of the standard deviation.

[0034] Exemplarily, a sliding window analysis method is used to calculate the mean and standard deviation of the amplitude in different frequency bands in each fixed window, and obtain the local thresholds in different frequency bands in each fixed window, wherein the frequency bands can be divided into low frequency, medium frequency, high frequency and other types based on the frequency value range. The event anomaly threshold in each frequency band can be obtained by averaging the local thresholds of multiple windows in the same frequency band. The multi-band event anomaly threshold finally generated includes the event anomaly threshold in each frequency band. The local threshold and the event anomaly threshold can be specifically in the form of a value range, for example, centered on the mean and n times the standard deviation as the value range, so as to determine the final local threshold and event anomaly threshold.

[0035] A sliding window analysis is performed on each second time-frequency graph, and abnormal events are marked on the second time-frequency graph according to the multi-band event abnormality threshold corresponding to the elevator operation status cluster to which the second time-frequency graph belongs, so as to obtain event stream feature data containing multiple abnormal event streams for each second time-frequency graph.

[0036] In this embodiment, a sliding window analysis method is used to gradually detect each second time-frequency graph and mark the abnormal event stream therein. Specifically, a fixed window of the same size as that in the analysis process of the multi-band event abnormal threshold is used to perform sliding window analysis on the second time-frequency graph. In each sliding window, the mean of the amplitude in different frequency bands is determined, and based on the multi-band event abnormal threshold, the signal in the current time period is compared. If the signal value of a certain frequency band in the time period exceeds the threshold range under the normal state, it is marked as an abnormal window. Finally, the multiple abnormal windows generated in the sliding window process constitute abnormal events. In this way, the abnormal events that may exist in each second time-frequency graph can be determined, and the event stream feature data containing multiple abnormal event streams of each second time-frequency graph can be obtained.

[0037] On this basis, the event timing data and the corresponding reference data of each abnormal event are extracted according to the preset time step, the event timing difference vector of each abnormal event flow is extracted based on the reference data corresponding to the event timing data, and the timing difference parameters of each abnormal event flow are determined according to the event timing difference vector, so as to obtain the abnormal diagnosis result in which each second time-frequency diagram contains the corresponding timing difference parameters.

[0038] In this embodiment, the preset time step is used to determine the key event segment before and after the occurrence of the abnormal event, and the characteristic data within a period of time before and after each abnormal event in the second time-frequency diagram is extracted to construct the event time series data of the abnormal event, and in the same time period range, the corresponding reference data in the normal sample data is extracted, and then the event time series difference vector of each abnormal event flow is extracted according to the reference data corresponding to the event time series data. The event time series difference vector contains the characteristic differences between the event time series data and the corresponding reference data at multiple time nodes, such as the energy difference. Finally, the time series difference parameters of the abnormal event flow can be calculated based on the event time series difference vector, wherein the time series difference parameters can be analyzed based on the benchmark template vector. For example, the value of each element in the benchmark template vector is 0, and the Euclidean distance between the benchmark template vector and the event time series difference vector is calculated to obtain the time series difference parameters of each abnormal event flow, and generate an abnormal diagnosis result in which each second time-frequency diagram contains the corresponding time series difference parameters.

[0039] It is worth noting that when the elevator has the same or different faults in different operating states, there will be differences in the data before and after the fault occurs. For example, the potential time of the same fault in different operating states is different. This analysis method using a fixed time step can characterize the difference between the change amplitude of the data before and after the fault occurs. The final calculated timing difference parameter can not only reflect the degree of difference between the abnormal event and the normal operating state, but also can well characterize the difference between different operating faults in different operating states, thereby realizing the feature analysis of abnormal events based on contextual information.

[0040] Optionally, feature optimization is performed on multiple second time-frequency graphs of each elevator operation status cluster according to the event stream feature data and the abnormality diagnosis result to generate a third time-frequency graph corresponding to each second time-frequency graph, including: The fault type parameter of each second time-frequency graph is determined according to multiple timing difference parameters and fault type labels, the operating state parameter of each second time-frequency graph is determined according to multiple timing difference parameters and operating state labels, and the abnormal event factor of each abnormal event is determined according to the fault type parameter and the operating state parameter.

[0041] In this embodiment, the fault type parameter is used to characterize the significance of the fault performance of different fault types under the same operating state. For each operating state, that is, each elevator operating state cluster, the abnormal diagnosis result includes the time series difference parameters of different operating faults in the operating state. After summarizing and averaging, the performance level of each fault type in the operating state can be obtained. Then, the parameters representing the average level of each fault type in the elevator operating state cluster in the operating state are normalized to obtain the fault type parameters of each fault type in the operating state. By analyzing each elevator operating state cluster in this way, the fault type parameters of each second time-frequency graph can be obtained.

[0042] The operating state parameters are used to characterize the significance of the fault performance of the same fault type under different operating states. For each fault type, each elevator operating state cluster represents an elevator operating state. Multiple time series difference parameters of each fault type under different operating states are summarized, and the time series difference parameters of the same fault type under multiple operating states are normalized to obtain the fault type parameters of each fault type under different operating states. Thus, the fault type parameters of the second time-frequency graph are determined according to the operating state and fault type corresponding to the second time-frequency graph.

[0043] The above-mentioned fault type parameters and operating status parameters take into account the different manifestations of different fault types under the same operating state of the elevator, and consider the manifestations of each group of fault types when the elevator is in different operating states. Combining the fault type parameters and the operating status parameters can help identify the manifestations of different fault types under different operating states. The product of the two is taken as the abnormal event factor of the abnormal event, which integrates the interactive influence between the fault type and the operating state of the elevator, and reflects the characteristic importance of different abnormal events from the two levels of fault type and operating state.

[0044] Finally, after determining the abnormal event impact interval of each second time-frequency graph based on the event flow feature data, the abnormal event impact interval of each second time-frequency graph is optimized according to the abnormal event factor to generate the third time-frequency graph corresponding to each second time-frequency graph. Among them, the abnormal event impact interval specifically includes the time period interval determined by each abnormal event in the event flow detection process, and the previous and next associated intervals involved in the abnormal event in the time series abnormality diagnosis process. The two together constitute the abnormal event impact interval of the second time-frequency graph. The abnormal event factor of the second time-frequency graph is used as the weight parameter for feature optimization of the abnormal event impact interval, and multiple abnormal events are weighted separately to achieve feature optimization. This feature optimization method that considers the interactive influence between the fault type and the operating state of the elevator generates a third time-frequency graph that integrates multi-dimensional information such as fault type, operating state and time series difference, effectively improving the feature expression ability of the time-frequency graph, and can provide a more detailed and reliable basis for the accurate diagnosis of elevator faults.

[0045] Optionally, a training set for training an elevator operation analysis model is constructed according to a plurality of third time-frequency graphs, including: Pseudo-color mapping is performed on the multiple third time-frequency graphs of each elevator operation status cluster to obtain multiple target frequency spectrum graphs of each elevator operation status cluster.

[0046] Specifically, it is necessary to first perform pseudo-color mapping on multiple third time-frequency graphs in each elevator operation status cluster to generate multiple target spectrum graphs. Pseudo-color mapping enhances the original grayscale image, i.e., the time-frequency graph, by color, making the signal changes more intuitive and significant. Figure 2 The 3D frequency-time distribution diagram contains the characteristic performance of the audio signal at different frequencies and time points during the elevator operation. Through pseudo-color mapping, mapping these grayscale values ​​to color values ​​can help highlight the key areas in the image, making important frequency bands and time intervals easier to identify.

[0047] Then, the fault type label and the operating status label of each target spectrum graph are determined, and a training set including multiple target spectrum graphs and the fault type label and the operating status label corresponding to each target spectrum graph is constructed.

[0048] In this embodiment, each target spectrogram corresponds to a specific fault type label and operating status label. The fault type label indicates the fault category to which the spectrogram belongs, such as motor fault, control system fault, etc., indicating which fault type the abnormal information contained in the time-frequency graph belongs to. The operating status label indicates the elevator operating state corresponding to the spectrogram, such as start, acceleration, deceleration, etc. This label helps the model understand the characteristic differences of the elevator under different operating states. Each sample pair, namely the target spectrogram, the fault type label and the operating status label, constitutes part of the training set.

[0049] ResNet-50 is used as the elevator operation analysis model for training. ResNet-50 is a deep convolutional neural network that solves the gradient vanishing problem in deep network training through residual connections and achieves excellent performance in computer vision tasks. In the process of training the elevator operation analysis model through the training set, multiple target spectrograms in the training set and the operating status label of each spectrogram are used as the input of the elevator operation analysis model, and the fault type label of each spectrogram is used as the training target of the elevator operation analysis model, that is, training ResNet-50 can predict the corresponding fault type based on the input spectrogram and the corresponding operating status label. The weight of the network can be optimized through the back propagation algorithm, and the accuracy of the model can be continuously improved through multiple rounds of training. Data enhancement, regularization and other technologies can also be used during the training process to prevent overfitting and ensure the generalization ability of the model. The training of the ResNet-50 network is not specifically limited here.

[0050] The elevator operation analysis model obtained through training can analyze new elevator audio data and generate fault diagnosis results of elevator operation. Specifically, after collecting new elevator operation audio signals, that is, elevator operation audio data to be analyzed, the steps of analyzing historical audio data are adopted, including adaptive time-frequency analysis of elevator operation audio data based on variational mode decomposition algorithm, and time-frequency conversion and feature map splicing to generate target time-frequency map, and obtain motion feature data corresponding to elevator operation audio data, such as elevator load, operation speed and other data, to determine the current operation status of the elevator, obtain the target operation status label of the elevator operation audio data, and then input the target time-frequency map containing the target operation status label into the elevator operation analysis model, and generate the elevator operation abnormality detection result corresponding to the elevator operation audio data through the elevator operation analysis model.

[0051] An embodiment of the present invention provides an elevator operation abnormality prediction and diagnosis method based on the Internet of Things. By performing adaptive time-frequency analysis on elevator audio signals through variational mode decomposition (VMD), the complex audio signals of the elevator can be decomposed into multiple modal signals. The effective features of the time-frequency graph can be further enhanced by feature graph splicing, so that signal changes in different frequency bands can be better captured. By performing event stream detection and timing anomaly diagnosis on the time-frequency graph, potential abnormal events in the elevator operation process can be extracted, and the timing information in the elevator operation process can be fully utilized to help the model not only identify the abnormality of a single event, but also capture the cumulative effect of abnormal events in different time periods. Considering the interactive influence between different fault types and elevator operation states, the abnormal event factor is calculated and the feature optimization is performed, so that the time-frequency graph can contain more accurate fault and operation state information, thereby improving the model's ability to identify and locate different faults. This analysis based on different elevator operation states and fault types can provide strong support for the accurate diagnosis of elevator faults, realize efficient prediction and fault diagnosis of elevator operation abnormalities, and can detect potential faults that may exist in the elevator at an early stage, and provide real-time warnings for elevator maintenance personnel, thereby effectively avoiding sudden failures of the elevator and reducing maintenance costs and downtime.

[0052] like Figure 2 As shown, an elevator operation abnormality prediction and diagnosis system based on the Internet of Things provided by an embodiment of the present invention is used to implement the above-mentioned elevator operation abnormality prediction and diagnosis method based on the Internet of Things, and specifically includes the following structure: A data acquisition module is used to obtain historical audio data collected by IoT sensors during the operation of the elevator; The operation status analysis module is used to perform time-frequency conversion on the historical audio data to generate multiple first time-frequency graphs, extract multiple frequency spectrum features corresponding to each first time-frequency graph and construct a frequency spectrum feature vector corresponding to each time-frequency graph, cluster the historical audio data according to the frequency spectrum feature vector to obtain multiple elevator operation status clusters, each elevator operation status cluster contains multiple groups of audio signals; A time-frequency feature analysis module is used to perform adaptive time-frequency analysis on each elevator operation status cluster based on a variational modal decomposition algorithm, generate multiple target modal signals of each elevator operation status cluster, perform time-frequency conversion and feature graph splicing on the multiple target modal signals of each elevator operation status cluster, and obtain multiple second time-frequency graphs of each elevator operation status cluster; Specifically, a variational modal decomposition algorithm is used to perform modal decomposition on multiple groups of audio signals in each elevator operation state cluster to obtain multiple sub-signals corresponding to each group of audio signals, and the total energy value of each sub-signal is calculated, and the sub-signal whose total energy value of each group of audio signals is greater than a preset energy threshold is recorded as a target modal signal; Short-time Fourier transform is performed on multiple target modal signals of each elevator operation status cluster to obtain a local time-frequency diagram of each target modal signal. Feature fusion is performed on the local time-frequency diagrams corresponding to the multiple target modal signals of each group of audio signals to obtain a global time-frequency diagram of each group of audio signals, which is recorded as the second time-frequency diagram of the audio signal.

[0053] A time-frequency feature optimization module is used to perform event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, perform time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, perform feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generate a third time-frequency graph corresponding to each second time-frequency graph; An operation abnormality analysis module is used to construct a training set for training an elevator operation analysis model according to a plurality of third time-frequency graphs and train the elevator operation analysis model. After collecting the elevator operation audio data to be analyzed, the elevator operation audio data is analyzed by the trained elevator operation analysis model to generate an elevator operation abnormality detection result; Specifically, an adaptive time-frequency analysis is performed on the elevator operation audio data based on the variational mode decomposition algorithm, and time-frequency conversion and feature map splicing are performed to generate a target time-frequency map. The motion feature data corresponding to the elevator operation audio data is obtained to determine the target operation status label of the elevator operation audio data. The target time-frequency map containing the target operation status label is input into the elevator operation analysis model to generate an elevator operation abnormality detection result corresponding to the elevator operation audio data.

[0054] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A method for predicting and diagnosing abnormal elevator operation based on the Internet of Things, characterized in that: include: Acquire historical audio data collected by the IoT sensor during the operation of the elevator, and perform time-frequency conversion on the historical audio data to generate a plurality of first time-frequency graphs; Extract multiple spectral features corresponding to each first time-frequency graph and construct a spectral feature vector corresponding to each time-frequency graph, cluster the historical audio data according to the spectral feature vector to obtain multiple elevator operation status clusters, each elevator operation status cluster contains multiple groups of audio signals; Based on the variational modal decomposition algorithm, adaptive time-frequency analysis is performed on each elevator operation state cluster to generate multiple target modal signals of each elevator operation state cluster, and time-frequency conversion and feature map splicing are performed on the multiple target modal signals of each elevator operation state cluster to obtain multiple second time-frequency maps of each elevator operation state cluster; Performing event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, performing time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, performing feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generating a third time-frequency graph corresponding to each second time-frequency graph; A training set for training an elevator operation analysis model is constructed based on multiple third time-frequency graphs, and the elevator operation analysis model is trained. After the elevator operation audio data to be analyzed is collected, the elevator operation audio data is analyzed by the trained elevator operation analysis model to generate an elevator operation abnormality detection result.

2. The method for predicting and diagnosing abnormal elevator operation based on the Internet of Things according to claim 1, characterized in that: For each of the multiple second time-frequency diagrams of the elevator operation status cluster, the second time-frequency diagram further includes: The multiple groups of audio signals in each elevator operation status cluster are modally decomposed by a variational modal decomposition algorithm to obtain multiple sub-signals corresponding to each group of audio signals, and the total energy value of each sub-signal is calculated. The sub-signal whose total energy value of each group of audio signals is greater than a preset energy threshold is recorded as a target modal signal; Short-time Fourier transform is performed on multiple target modal signals of each elevator operation status cluster to obtain a local time-frequency diagram of each target modal signal. Feature fusion is performed on the local time-frequency diagrams corresponding to the multiple target modal signals of each group of audio signals to obtain a global time-frequency diagram of each group of audio signals, which is recorded as the second time-frequency diagram of the audio signal.

3. The method for predicting and diagnosing abnormal elevator operation based on the Internet of Things according to claim 2 is characterized in that: Perform event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, and perform time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, including: Determine the fault type label of each second time-frequency graph according to the historical audio data, filter out multiple groups of audio signals belonging to the normal operating state in each elevator operation state cluster according to the fault type label, and construct a multi-band event abnormal threshold for each elevator operation state cluster according to the filtered multiple groups of audio signals; Perform sliding window analysis on each second time-frequency graph, mark the second time-frequency graph with abnormal events according to the multi-band event abnormality threshold corresponding to the elevator operation status cluster to which the second time-frequency graph belongs, and obtain event stream feature data containing multiple abnormal event streams for each second time-frequency graph; The event timing data and corresponding reference data of each abnormal event are extracted according to the preset time step, the event timing difference vector of each abnormal event flow is extracted based on the reference data corresponding to the event timing data, the timing difference parameters of each abnormal event flow are determined according to the event timing difference vector, and the abnormal diagnosis result in which each second time-frequency diagram contains the corresponding timing difference parameters is obtained.

4. The method for predicting and diagnosing abnormal elevator operation based on the Internet of Things according to claim 3 is characterized in that: According to the event stream feature data and the abnormal diagnosis result, the features of multiple second time-frequency graphs of each elevator operation status cluster are optimized to generate a third time-frequency graph corresponding to each second time-frequency graph, including: Determine a fault type parameter of each second time-frequency graph according to a plurality of timing difference parameters and a fault type label, determine an operating state parameter of each second time-frequency graph according to a plurality of timing difference parameters and an operating state label, and determine an abnormal event factor of each abnormal event according to the fault type parameter and the operating state parameter; Based on the event flow feature data, the abnormal event impact interval of each second time-frequency graph is determined, and the abnormal event impact interval of each second time-frequency graph is feature optimized according to the abnormal event factor to generate a third time-frequency graph corresponding to each second time-frequency graph.

5. The method for predicting and diagnosing abnormal elevator operation based on the Internet of Things according to claim 1, characterized in that: Constructing a training set for training an elevator operation analysis model according to a plurality of third time-frequency graphs and training the elevator operation analysis model includes: Perform pseudo-color mapping on the multiple third time-frequency graphs of each elevator operation status cluster to obtain multiple target frequency spectrum graphs of each elevator operation status cluster; Determine the fault type label and the operating status label of each target spectrum graph, and construct a training set including multiple target spectrum graphs and the fault type label and the operating status label corresponding to each target spectrum graph; Multiple target spectrum graphs in the training set and the operating status label of each spectrum graph are used as the input of the elevator operation analysis model, and the fault type label of each spectrum graph is used as the training target of the elevator operation analysis model. The elevator operation analysis model is obtained by training the training set, wherein the elevator operation analysis model is built based on the ResNet-50 network.

6. The method for predicting and diagnosing abnormal elevator operation based on the Internet of Things according to claim 1, characterized in that: The elevator operation audio data is analyzed through the trained elevator operation analysis model to generate elevator operation anomaly detection results, including: An adaptive time-frequency analysis is performed on the elevator operation audio data based on the variational mode decomposition algorithm, and time-frequency conversion and feature map splicing are performed to generate a target time-frequency map. The motion feature data corresponding to the elevator operation audio data is obtained to determine the target operation state label of the elevator operation audio data. The target time-frequency map containing the target operation state label is input into the elevator operation analysis model to generate an elevator operation abnormality detection result corresponding to the elevator operation audio data.

7. An elevator operation abnormality prediction and diagnosis system based on the Internet of Things, characterized in that: The system is used to implement the elevator operation abnormality prediction and diagnosis method based on the Internet of Things as described in any one of claims 1 to 6, including: A data acquisition module is used to obtain historical audio data collected by IoT sensors during the operation of the elevator; The operation status analysis module is used to perform time-frequency conversion on the historical audio data to generate multiple first time-frequency graphs, extract multiple frequency spectrum features corresponding to each first time-frequency graph and construct a frequency spectrum feature vector corresponding to each time-frequency graph, cluster the historical audio data according to the frequency spectrum feature vector to obtain multiple elevator operation status clusters, each elevator operation status cluster contains multiple groups of audio signals; A time-frequency feature analysis module is used to perform adaptive time-frequency analysis on each elevator operation status cluster based on a variational modal decomposition algorithm, generate multiple target modal signals of each elevator operation status cluster, perform time-frequency conversion and feature graph splicing on the multiple target modal signals of each elevator operation status cluster, and obtain multiple second time-frequency graphs of each elevator operation status cluster; A time-frequency feature optimization module is used to perform event flow detection on each second time-frequency graph to obtain event flow feature data of each second time-frequency graph, perform time series anomaly diagnosis on multiple second time-frequency graphs of each elevator operation status cluster to generate anomaly diagnosis results, perform feature optimization on multiple second time-frequency graphs of each elevator operation status cluster according to the event flow feature data and the anomaly diagnosis results, and generate a third time-frequency graph corresponding to each second time-frequency graph; The operation abnormality analysis module is used to construct a training set for training the elevator operation analysis model based on multiple third time-frequency graphs and train the elevator operation analysis model. After collecting the elevator operation audio data to be analyzed, the elevator operation audio data is analyzed by the trained elevator operation analysis model to generate the elevator operation abnormality detection result.

8. The elevator operation abnormality prediction and diagnosis system based on the Internet of Things according to claim 7 is characterized in that: For the abnormal operation analysis module, the elevator operation audio data is analyzed through the trained elevator operation analysis model to generate the elevator operation abnormality detection results, including: An adaptive time-frequency analysis is performed on the elevator operation audio data based on the variational mode decomposition algorithm, and time-frequency conversion and feature map splicing are performed to generate a target time-frequency map. The motion feature data corresponding to the elevator operation audio data is obtained to determine the target operation state label of the elevator operation audio data. The target time-frequency map containing the target operation state label is input into the elevator operation analysis model to generate an elevator operation abnormality detection result corresponding to the elevator operation audio data.

Images