Abnormal jump behavior detection method based on multi-source acceleration signal feature analysis

Abstract

The application discloses an abnormal jumping behavior detection method based on multi-source acceleration signal feature analysis, comprising the following steps: acquiring multi-source acceleration time series data; pre-processing the multi-source acceleration time series data to obtain clean and aligned acceleration signals; detecting candidate segments of abnormal jumping behaviors in the clean and aligned acceleration signals, and constructing a multi-source acceleration feature matrix by using the candidate segments; inputting the multi-source acceleration feature matrix into a convolutional neural network model to identify and classify the abnormal jumping behaviors, and obtaining output probabilities corresponding to the abnormal jumping behavior categories; and determining and deciding the passenger abnormal jumping behaviors according to the output probabilities of the abnormal jumping behavior categories. The application is trained in combination with a self-supervised learning strategy, and is updated and optimized offline by using continuously accumulated new data samples, so that the accuracy and robustness of the abnormal jumping behavior detection in the elevator are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of elevator safety monitoring, and particularly relates to an abnormal jumping behavior detection method based on multi-source acceleration signal feature analysis. BACKGROUND

[0002] As a core vertical transportation tool in high-rise buildings, the running safety of an elevator is of great importance. In recent years, with the increasing frequency of elevator use, elevator failures and safety accidents caused by improper passenger behavior occur from time to time. For example, passengers intentionally jumping in the elevator car can cause unexpected elevator stop, people being trapped, and even component damage, threatening the safety of life and property. Such abnormal jumping behavior poses a hidden danger to the safe operation of the elevator, and more intelligent monitoring and protection means are urgently needed to detect and stop it in time.

[0003] Currently, common elevator safety monitoring mainly relies on video monitoring means, that is, a camera is installed in the car to take pictures of passenger behavior, and abnormal behavior is identified through manual review of the video or computer vision algorithm. However, such image-based monitoring methods are easily affected by factors such as lighting conditions, passenger shielding, camera angle blind area, and image clarity. In the small and closed environment of an elevator, relying solely on camera detection often has problems of low accuracy and insufficient real-time performance, making it difficult to timely and reliably capture dangerous actions such as passenger jumping.

[0004] With the development of artificial intelligence and Internet of Things technology, some elevator monitoring systems have begun to introduce machine learning or deep learning algorithms to intelligently analyze elevator operation data in order to achieve automatic identification of abnormal behavior. However, such systems still face problems such as single data source, insufficient training samples, and limited model generalization ability. For example, it is difficult to comprehensively reflect the stress state of the elevator using data from a single sensor, and insufficient multi-source data fusion limits the performance of the algorithm; at the same time, a large number of labeled samples covering various abnormal behaviors are lacking, resulting in low recognition accuracy of the trained model when faced with complex and variable actual scenarios. In addition, the adaptability of existing models to different elevators and environments needs to be improved, and when the elevator operating conditions or passenger behavior patterns change, the detection performance of the model may decrease significantly. It can be seen that the existing technology is still insufficient in detecting abnormal behavior (especially passenger jumping) in elevators. SUMMARY

[0005] The application proposes an abnormal jumping behavior detection method based on multi-source acceleration signal feature analysis to solve the problem of being unable to timely and accurately identify passenger jumping behavior in existing elevator monitoring.

[0006] The technical solution for achieving the purpose of the application is: an abnormal jumping behavior detection method based on multi-source acceleration signal feature analysis, comprising:

[0007] a plurality of acceleration sensors are arranged in the elevator car, and multi-source acceleration time series data are acquired by using the acceleration sensors;

[0008] clean and aligned acceleration signals are obtained by preprocessing the multi-source acceleration time series data, and the preprocessing mode includes filtering noise and calibrating and aligning multi-sensor data;

[0009] candidate segments of abnormal jumping behaviors are detected in the clean and aligned acceleration signals, and a multi-source acceleration feature matrix is constructed by using the candidate segments;

[0010] the multi-source acceleration feature matrix is input into a convolutional neural network model for identification and classification of abnormal jumping behaviors, and output probabilities corresponding to the abnormal jumping behavior categories are obtained;

[0011] abnormal jumping behaviors of passengers are determined and decided according to the output probabilities corresponding to the abnormal jumping behavior categories output by the convolutional neural network model.

[0012] Compared with the prior art, the present application has the following advantages:

[0013] 1. By arranging high-sensitivity three-axis acceleration sensors at multiple points on the floor, ceiling and other positions of the elevator car, and synchronously collecting by using a unified clock or time stamp, the overall coverage and accurate alignment of multi-source signals in space and time are ensured, thereby significantly improving the capture ability and data integrity of abnormal jumping behaviors.

[0014] 2. A multi-stage signal preprocessing method combining band-pass filtering, zero-point calibration and time alignment is proposed, which effectively filters out the direct current drift and high-frequency noise generated during the operation of the elevator, and corrects the amplitude and time delay differences between the multi-sensors, thereby ensuring the high signal-to-noise ratio and consistency of the input signals, and laying a solid foundation for subsequent feature extraction.

[0015] 3. A double-threshold detection strategy is used to identify acceleration mutation points, and short-time window signal segments are intercepted before and after the candidate time, thereby constructing a two-dimensional multi-source feature matrix of time x channel, which can completely retain the multi-dimensional time sequence mode of the whole jumping event, and is more targeted and robust compared with the fixed sliding window method.

[0016] 4. Multi-layer convolution stacking, residual connection and multi-scale convolution fusion are introduced into the convolutional neural network structure, and a space-channel joint attention mechanism is combined, thereby adaptively highlighting the key sensor channel and time position features, realizing deep discriminative expression of the jumping behavior, and effectively improving the detection accuracy and generalization ability.

[0017] 5、Combined with the hybrid training paradigm of self-supervised pre-training and supervised fine-tuning, the potential representation of elevator normal working condition is learned by using a large amount of unlabeled running data, and then the classification boundary is optimized by a small amount of labeled samples, which solves the problem of sample scarcity and improves the adaptability of the model to abnormal behavior in complex environment.

[0018] 6、Based on the Softmax probability output and dynamic threshold determination mechanism, the system can flexibly adjust the alarm sensitivity, realize the hierarchical response of abnormal jump behavior combined with the real-time running state of the elevator, ensure the sensitivity of detection, and effectively reduce the false alarm and missed alarm rate.

[0019] 7、Continuous monitoring and closed-loop optimization mechanism is constructed, the detection result and corresponding feature matrix are stored in the database each time, and self-supervised update and supervised fine-tuning are periodically performed, so that the detection model is continuously optimized with the change of elevator running environment and passenger behavior mode, and the long-term stability and adaptability of the system are guaranteed.

[0020] The application will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are used to provide further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application. In the drawings:

[0022] Figure 1 is the overall framework diagram of the elevator internal abnormal jump behavior detection method based on multi-source acceleration signal feature tensor and time-frequency gated attention of the application.

[0023] Figure 2 is a multi-point three-axis acceleration sensor layout schematic diagram of the application.

[0024] Figure 3 is a multi-stage filtering and signal alignment process flowchart of the application.

[0025] Figure 4 is a candidate event detection and feature tensor construction diagram of the application.

[0026] Figure 5 is an abnormal jump recognition model structure diagram based on CNN and time-frequency gated attention (TF-GA) of the application.

[0027] Figure 6 is a Softmax output and hysteresis threshold determination mechanism diagram. DETAILED DESCRIPTION

[0028] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0029] The specific embodiments of the present application are further described below with reference to the drawings.

[0030] An abnormal jump behavior detection method based on multi-source acceleration signal feature analysis, which synchronously collects signals by deploying multiple acceleration sensors in the elevator car, then performs multi-stage filtering preprocessing, fuses various signal features, and finally realizes real-time detection of abnormal jump behavior by combining convolutional neural network analysis. The method comprises the following steps:

[0031] S1: acceleration signal acquisition: multiple acceleration sensors are arranged at appropriate positions of the elevator car, and each sensor acquires three axial acceleration signals. Each sensor continuously acquires acceleration data during the operation of the elevator at a preset sampling rate, and synchronously transmits the obtained multi-source acceleration signals to a data processing unit. This step outputs the original multi-source acceleration time series data, providing a basis for subsequent analysis.

[0032] Further, in the step S1, it comprises:

[0033] S101, sensor arrangement and initialization: selecting appropriate sensor installation positions and fixing multiple acceleration sensors so that their measurement axes correspond to the direction of elevator movement. A uniform sampling frequency is set for each sensor and initial calibration is performed to ensure the comparability of the collected data. At this time, the system input is the motion state of the elevator car, and the output is the original acceleration signals monitored by each sensor in real time.

[0034] S102, three-axis acceleration data acquisition: during the operation of the elevator, each sensor continuously acquires acceleration values along three orthogonal axes (denoted as 、 、 axis) to form three independent time series. For any time , the th sensor outputs a three-dimensional vector covering the acceleration of the three axes of the sensor. The sampling of all sensors is synchronized, obtaining a multi-source acceleration observation vector at time , where is the number of sensors. If a total of A total of acceleration signal channel number is one at each sampling time .

[0035] S103, data synchronization and transmission: the acceleration signals collected by each sensor are transmitted in real time to the data processing unit in the elevator through wireless means. Time stamp mechanism is adopted in the transmission process to align the multi-sensor data, ensuring that the acceleration data of different sensors correspond accurately on the time axis. At the same time acceleration from all channels forms a multi-source acceleration observation vector . After synchronization processing, the data processing unit receives the complete multi-source acceleration time series , is the total number of sampling points.

[0036] S104, output raw data: the output of step S1 is the raw multi-source acceleration time series data, which includes the sequence of acceleration raw measurement values of each axis of all sensors over time. The output is represented in the form of a matrix or multi-dimensional array, for example, a matrix with a size of , with rows as time sampling points and columns as each sensor axis signal. The raw data will be directly input into the next pre-processing module.

[0037] S2: signal preprocessing: the raw multi-source acceleration time series data obtained in step S1 is pre-processed to improve signal quality and consistency. The pre-processing includes noise filtering and multi-sensor data calibration and alignment, outputting clean and aligned acceleration signals, ready for feature extraction.

[0038] Further, in the step S2, it includes:

[0039] S201, denoising filtering: taking the raw acceleration time series of each channel output by step S1 as input, band-pass filtering is performed on each channel to remove noise and drift. The lower limit frequency of the filter is selected to be higher than the ramp acceleration component when the elevator is running smoothly, and the upper limit frequency is selected to be lower than the environmental high-frequency noise band, so as to filter out the low-frequency drift generated by the elevator start / stop and the high-frequency interference in the environment, and retain the transient vibration signal generated when the passenger jumps. Mathematically, the filtering can be represented as a convolution operation: for example, for the channel signal , the filtering result is , where is the impulse response of the band-pass filter (length ). After this step, the output acceleration signal sequence no longer contains obvious DC drift and high-frequency noise, and the signal-to-noise ratio of the signal is improved.

[0040] S202, Sensor Calibration and Alignment: Perform inter-sensor consistency calibration and time alignment on the filtered multi-source acceleration signals. First, correct the zero-point bias and scale factor of each sensor: obtain the output offset of each sensor in the elevator static state , then subtract (or linearly transform by the pre-calibrated coefficient) from the data in the acceleration signal sequence to obtain the corrected acceleration signal sequence , ensuring that the acceleration amplitudes are comparable. Second, calibrate the data time axis of each sensor: if there is a sampling delay or relative clock bias in some sensors, through interpolation, resampling or correction method based on synchronization signal, adjust corresponding shift or interpolation to ensure that the observations of each sensor at the same time point remain synchronized. The input of this sub-step is the filtered signal sequence , and the output is the calibrated zero-bias and time-aligned signal sequence .

[0041] S203, Output Preprocessing Signal: The output of step S2 is clean, aligned multi-source acceleration signal data. Formally represented as an acceleration matrix , where each column corresponds to the calibrated acceleration time sequence of a sensor. Compared to the original data, the preprocessed signal eliminates noise and sensor differences, more accurately reflects the instantaneous vibration received by the elevator car, and provides high-quality input for subsequent feature extraction and recognition.

[0042] S3: Feature Extraction and Data Construction: Detect candidate segments of abnormal jump behavior in the clean and aligned acceleration signals obtained in step S2, and construct multi-source feature data for classification analysis. This step finds the possible jump event occurrence time through threshold detection, intercepts the multi-source signal segment near this time, and organizes it into a feature matrix.

[0043] Further, in the step S3, it includes:

[0044] S301, Abnormal Acceleration Change Detection: The input is the multi-source acceleration time sequence output by step S2. The algorithm monitors the dramatic changes in the acceleration signal on the time axis to capture the instantaneous impact features generated by passenger jump behavior. Calculate the amplitude and instantaneous change of each channel acceleration at each time sampling point, and compare it with the preset threshold. When a certain sensor channel satisfies or , it is determined that a passenger abnormal jump event may have occurred at , where represents the acceleration amplitude threshold, This represents the acceleration increment threshold. If there is more than one channel and time that meets the threshold condition, the most significant event time can be selected as the candidate jump event center time based on the magnitude of the acceleration peak or the chronological order. .

[0045] S302, Time window signal segment extraction: using the center time of the detected candidate jump event Based on this, a short time window segment is extracted from the clean-aligned acceleration signal. The length of this time window is selected according to the duration of the jump behavior (e.g., using...). Centered before and after milliseconds, total length (Millisecond window). Let the sampling frequency be... The time window then contains approximately There are 10 acceleration data points, among which The corresponding centered position within the window. The window cropping operation applies the acceleration sequence to each sensor channel. This process is performed synchronously to obtain a multi-source acceleration segment containing complete waveforms before and after the event. The input for this sub-step is the candidate event time. And multi-source acceleration sequences, the output is around Candidate segments of multi-source acceleration signals (time length is) ).

[0046] S303, Multi-source Feature Tensor Construction: The candidate segments of the multi-source acceleration signal obtained in sub-step S302 are subjected to time-frequency transformation processing to construct a more discriminative three-dimensional feature tensor. Specifically, within the time window... Inside, each axial channel of each sensor generates Several acceleration sample values. These data are first organized into an initial matrix in the form of "time × channel". Each row of the matrix represents the multi-channel acceleration observations at a certain moment within the window, and the columns represent different sensors and their axial channels, totaling... Columns. Next, for each column of this two-dimensional matrix... (i.e., the first) Perform multi-scale discrete wavelet transform (DWT) on each sensor axial channel to extract its values ​​at multiple frequency scales. The frequency domain coefficients are denoted as follows: , indicating the first At the wavelet scale, the first Channel at time The acceleration frequency domain characteristic response. Stacking the wavelet coefficients of all channels and all scales along the scale dimension constitutes a three-dimensional tensor: .in, Indicates the first time window The first channel in the Wavelet scale and the Signal characteristics at each sampling point. Here... This represents the total number of scales used in discrete wavelet decomposition, equivalent to the number of wavelet decomposition layers, and is usually set as a constant based on the length or spectral distribution of the original signal. A larger value indicates a finer decomposition frequency, allowing for the extraction of lower-frequency but longer-time-span features, suitable for joint modeling of transient and background phenomena in elevator jumping behavior. In this way, the original acceleration data is fused and unified across the three dimensions of "time, channel, and frequency," forming a feature tensor with multi-scale time-frequency representation capabilities. Compared to traditional two-dimensional feature matrices, this tensor significantly improves the ability and robustness of the abnormal behavior recognition model to represent non-stationary signals, providing more complete event representation information for subsequent classification models.

[0047] S304. Output candidate feature tensor: The output of step S3 is the multi-source acceleration feature matrix of the candidate event (suspected abnormal jump). This tensor incorporates data from each sampling time step ( ), each sensor channel ( ) and multiple wavelet decomposition scales ( The acceleration variation characteristics on the surface fully characterize the temporal, channel, and frequency response patterns before and after the jump event. This three-dimensional tensor possesses strong expressive power and high temporal-frequency resolution, serving as the input basis for subsequent feature classification models. In practical applications, if multiple candidate jump events are detected within the same elevator operating cycle, steps S302–S303 can be executed for each event to generate the corresponding feature tensor, which is then fed into the downstream model for identification and judgment.

[0048] S4: Feature Classification Based on Convolutional Neural Network: The multi-source acceleration feature tensor obtained in step S3 is input into a convolutional neural network (CNN) model for the identification and classification of abnormal jumping behavior. The CNN incorporates a time-frequency gated attention mechanism to highlight key features and can be optimized using a self-supervised / pre-training strategy, thereby improving the accuracy and robustness of detecting abnormal jumping behavior within elevators. This CNN sequentially includes an input layer, a first convolutional module consisting of two sets of first convolutional layers and a first activation layer, a first pooling layer, a second convolutional module consisting of two sets of second convolutional layers and a second activation layer, a second pooling layer, a third convolutional module consisting of two sets of third convolutional layers and a third activation layer, a third pooling layer, a TF-GA module, a first fully connected layer, a second fully connected layer, and an output layer.

[0049] Furthermore, step S4 includes:

[0050] S401, First convolutional layer processing: In the first convolutional layer, the size is... The multi-source acceleration feature tensor (considered to have) "Time-frequency multi-channel image" with multiple channels, among which The length of the time window. Number of sensor channels Using wavelet scale / subband number as input, through 64 The convolution operation is performed using convolution kernels. The kernels slide sequentially across the feature tensor with a stride of 1, and zero-padding is applied at the edges to ensure the temporal dimension of the convolution output. and sensor dimensions Same as the input. Each convolutional kernel extracts local "temporal-sensor-scale" correlation patterns, outputting a total of 64 feature map channels, forming a shape of size [size missing]. The three-dimensional feature map tensor.

[0051] S402, First Activation Layer Processing: Processing the output of the previous layer. A non-linear activation function is applied element-wise to the feature map. Preferably, a method is used... Activation, that is, performing an operation on each element in the feature map. The nonlinear transformation truncates negative values ​​to 0 while leaving positive values ​​unchanged, thus introducing sparsity and nonlinear discrimination capabilities. After mapping, the feature map size is preserved. constant.

[0052] S403, Repeated Convolution and Activation: Repeat sub-steps S401 and S402 once. That is, use another set of 64... The convolution kernel convolves the feature map of the previous layer and applies... Nonlinear transformation. The stacking of two "convolution + activation" layers allows the network to extract more complex and abstract local feature patterns in the first convolutional layer, while the output feature map dimension remains the same. .

[0053] S404, First Pooling Layer Processing: Performs dimensionality reduction on the feature map obtained from the previous layer. The size of the max-pooling filter slides on the feature map with a step size of 2, for each The maximum value within the neighborhood is used as the pooling result. Pooling downsamples the feature map in both time and sensor dimensions, reducing the feature map size to its original value. For example, by dropped to about (If N and M are odd numbers, zero-padding can be used to make the size divisible). Pooling layers preserve the most salient feature responses in local regions, reducing data dimensionality and improving feature invariance and robustness.

[0054] S405, Second Convolutional Layer Processing: The downsampled feature map output from the first pooling layer is used as input, and processed through 128... convolution operation. The convolution kernel slides with a step size of 1, and the plane size of the convolution kernel is , and the depth is consistent with the channel number of the input feature map 64 (i.e., each convolution kernel contains weight parameters). After the convolution operation, feature patterns are learned at a more abstract level, and the output size is a feature map tensor (the channel number increases to 128).

[0055] S406, second activation layer processing: applying an activation function to the above 128-channel feature map element by element, obtaining a feature map after nonlinear mapping, and the size remains .

[0056] S407, repeat convolution and activation: after the second convolution layer, continue to deepen the network level, repeat the operation of sub-step S405 and sub-step S406 twice. That is, two rounds of 128-channel convolution and activation are performed, so that the second convolution layer accumulates three times of convolution extraction. After this series of convolution stacking, the network can extract more complex spatiotemporal feature combinations, and the output feature map dimension is still .

[0057] S408, second pooling layer processing: the same 2x2 max pooling (step size 2) as S404 is used to downsample the feature map of the last layer. The feature map size is further reduced by half, to about . Through the pooling again, the extracted features have a higher level of abstraction, and the data scale is greatly reduced, laying a foundation for subsequent deeper convolution extraction.

[0058] S409, third convolution layer processing: taking the feature map output by the second pooling layer as input, using 256 convolution kernels for convolution operation. The convolution kernel scans the input feature map with a step size of 1, and also pads zero at the edge to keep the size unchanged. Since the channel number of the input feature map is 128 at this time, the depth of each convolution kernel is also 128, so that the convolution can fuse all the channel feature information extracted by the previous layer. The convolution operation outputs a feature map tensor with a size of (the channel number is expanded to 256).

[0059] S410, third activation layer processing: applying a nonlinear activation to the 256-channel feature map element by element, obtaining an activated feature map, providing nonlinear capabilities for the combination of subsequent higher-level features.

[0060] S411, Repeat convolution and activation: Increase the network depth by stacking in the third convolutional layer, repeat the sub-step S409 and the sub-step S410 twice, and perform two rounds of 256-channel 3x3 convolution and activation on the feature map . .

[0061] S412, Third pooling layer processing: Perform the last 2x2 max pooling (the step is still 2) on the above-mentioned 256-channel feature map, and reduce the size of the feature map by half to about At this time, the extracted features have stronger robustness and smaller size, and are ready for subsequent fully connected layer processing.

[0062] S413, Time-frequency gated attention (TF-GA) fusion module processing:

[0063] Before entering the full connection layer conversion, the TF-GA module is introduced to weight the fusion of the feature map. This module jointly models the "channel importance (corresponding to different sensor / scale sub-band)" and "spatial and temporal position importance (corresponding to different time and different channel position)", and performs sparse selective enhancement through the gating coefficient.

[0064] Implementation:

[0065] 1. Channel gating: Perform global average pooling and global variance pooling on the feature map to form a channel statistical vector, and generate a channel weight vector through two fully connected layers + Sigmoid. ;

[0066] 2. Spatial and temporal attention: Apply convolution operation to the feature map (size is ) output by the third pooling layer (S412) to obtain a spatial attention map .

[0067] 3. Gated fusion: Broadcast the channel weight vector to the spatial dimension, multiply it with the spatial attention map to form a gating mask , and element-wise multiply it with the original feature map input to the TF-GA module to form a weighted feature map: . Wherein is the element at position , channel ​original feature values on the feature map. The module adds two designs of "variance statistics" and "gate coefficient" to the traditional attention, which can more sensitively capture the energy distribution difference of "sudden jump" and suppress redundant subbands and irrelevant channels at the same time, and the output size remains .

[0068] S414, the first full connection layer processing: the three-dimensional feature map weighted by TF-GA is flattened into a one-dimensional feature vector, and feature compression and combination are performed. Specifically, the feature map with a size of is unfolded into a vector with a length of , and then through full connection operation, the dimensional feature vector is linearly mapped into a new feature vector with a length of (e.g., let ). The full connection layer performs affine transformation of the form on the input vector (weight matrix , bias vector ), realizes dimension reduction and fusion of high-dimensional features. The sub-step outputs the feature with compact representation.

[0069] S415, the second full connection layer processing: map to the target output dimension vector . For binary classification tasks, you can map to a 2-dimensional vector . and respectively represent the feature scores of the corresponding "normal state" and "abnormal jump" classes extracted by the model.

[0070] S416, output layer classification: apply the Softmax normalization function to to calculate the output probability corresponding to the abnormal jump behavior. For , the Softmax output is . The output of the Softmax layer is denoted as the probability vector , where corresponds to the probability of the "abnormal jump" class.

[0071] S417, model training and self-supervised strategy: the above convolutional neural network model is optimized by supervised training combined with self-supervised pre-training strategy. In the offline training phase, labeled samples containing elevator normal operation and jump events are collected, and the feature tensor of each sample is input into the model for forward calculation to obtain the class probability vector of the output layer after Softmax normalization , where total number of classes, probability that the sample belongs to the class, index denotes the output class number, the value range is . The true class label is represented as a one-hot encoded vector , when the sample belongs to the class , otherwise 0.

[0072] The training loss adopts a cross-information measure function (formally equivalent to cross-entropy) to measure the information difference between the model prediction distribution and the true distribution: where is the class index ).

[0073] For binary classification tasks , it can be written as: where , denotes the abnormal jump class, denotes the normal class, , and are the normalized probabilities of the two classes, respectively.

[0074] Subsequently, the error backpropagation algorithm is used to update the gradients of the convolution kernel, fully connected layer and the gating weights in the TF-GA module according to the loss , and the model is iteratively trained until it converges.

[0075] In the training process, a self-supervised pre-training task is introduced to utilize a large amount of unlabeled elevator running acceleration data to assist in optimizing model parameters. For example, the model can be allowed to predict the future evolution of the acceleration sequence without artificial labels, or to reconstruct the obscured signal segment, and by minimizing the error between the predicted value and the true value, effective time-frequency feature representation is extracted. After completing the self-supervised pre-training, combined with a small amount of labeled jump events for supervised fine-tuning, the model achieves high-precision identification and strong generalization performance for abnormal jump behavior under the condition of limited labeled samples.

[0076] S5: Abnormal behavior judgment and post-processing: according to the probability vector output by the convolutional neural network through Softmax, the abnormal jump probability is calculated and smoothed to realize stable time series judgment. This step converts the continuous probability sequence output by the model into a clear binary classification event decision through probability smoothing, hysteresis threshold comparison and state retention mechanism, avoiding misjudgment caused by short-term fluctuations.

[0077] Further, in the step S5, it includes:

[0078] Further, in the step S5, it includes:S501, Obtain anomaly probability: read the probability vector output by step S416 , extract the anomaly jump confidence in the time window . In the continuous running state, the system calculates the corresponding , forming a time series probability sequence for subsequent dynamic determination.

[0079] S502, Probability smoothing processing: to reduce short-term noise interference, use sliding average or exponential weighted smoothing method to process the probability sequence: where is the smoothing coefficient (preferably 0.2-0.3). The smoothed sequence represents the stable confidence after time filtering, providing input for subsequent threshold comparison.

[0080] S503, Hysteresis threshold determination: preset high threshold and low threshold ( , typically 0.6 and 0.4), using a double-threshold hysteresis mechanism to prevent frequent switching:

[0081] When and lasts for windows, it is determined that an anomaly jump event occurs;

[0082] When and lasts for windows, it is determined that the normal state is restored;

[0083] When is in the interval, the previous state remains unchanged.

[0084] This hysteresis determination method can be equivalent to a time-gated logic, effectively suppressing false triggers caused by random jitter under multiple window continuous judgment.

[0085] S504, Event confirmation and output: if an anomaly jump event is detected, the system records the event start and end time, peak probability and duration , and outputs the event identification signal to the alarm module (S6) and the recording module (S7). If it is determined to be a normal state, a state retention signal is output and the monitoring cache is updated. The determination result output through this step is the basis for subsequent alarm and control.

[0086] S6: Alarm prompt and linkage control: According to the abnormal jump determination result of step S5, the system triggers multi-level alarm prompt and linkage control. This step realizes the closed loop from intelligent recognition result to safety disposal response, with real-time, grading and traceability.

[0087] Further, in step S6, it includes:

[0088] S601, Grading alarm strategy: According to the state determination result output by step S5 and smooth probability , the system adopts a double-threshold grading alarm strategy:

[0089] When , trigger the early warning level prompt;

[0090] When and continue determination window, trigger the formal alarm.

[0091] This grading strategy can adjust the threshold and duration according to the elevator use scene, so as to balance sensitivity and stability.

[0092] S602, On-site sound and light prompt and voice warning: When the system determines to enter the early warning or alarm state, control the sound and light module and voice prompt system in the elevator car. In the early warning stage, play a flexible prompt (such as "Please pay attention to safety, do not jump"), and light the orange indicator light; In the formal alarm stage, start the red warning light and play the emergency voice prompt, and can send a pop-up window information to the car screen. This way can make immediate feedback to abnormal behavior without disturbing the normal passengers.

[0093] S603, Remote linkage control: When the system enters the alarm state, automatically sends an alarm message to the elevator monitoring host and cloud maintenance platform through the communication interface, the message content includes: elevator number, floor, timestamp, alarm level, peak value and event duration. The remote system can automatically perform one of the following operations: record event log and trigger maintenance personnel notification; Perform safety mode (such as temporarily suspend door closing, reduce car running speed) on the continuous abnormal state; Enable the cloud self-diagnosis module to analyze the similarity of abnormal patterns and historical samples.

[0094] S604, Data feedback and confirmation mechanism: On-site alarm state and cloud reporting state are recorded in the system log. When the monitoring center or maintenance personnel confirms the abnormality, it can feedback the confirmation signal through the cloud interface. After the system receives the feedback, it automatically updates the state label of the corresponding event in the local database, realizing alarm-confirmation-closed loop tracking.

[0095] If the cloud does not respond within a limited time, the system enters a local cache mode, and the alarm records are transmitted after the communication is restored to ensure data integrity.

[0096] S605, alarm cancellation and self-reset logic: when the number of consecutive windows is less than , the system automatically cancels the alarm state and records the "cancellation timestamp". To prevent frequent alarms, set a minimum interval time , and do not trigger new alarms within this time period. After the alarm is canceled, the system automatically resets the sound and light module and returns to the monitoring standby state.

[0097] S606, abnormal event identification and log tracking: each alarm event generates a unique event ID, and the following fields are stored: event time period, peak probability , duration , alarm level, feedback status and disposal measures. The log can be used for subsequent analysis, statistics and model maintenance, providing data input for S7.

[0098] S7: result storage and model update: this step realizes long-term storage, statistical analysis and model adaptive maintenance of abnormal detection results. By continuously updating the sample database and periodically retraining, the system maintains the ability to self-learn different elevator operation characteristics and environmental changes.

[0099] Further, in step S7, it includes:

[0100] S701, event data archiving and index management: structurally store each detected abnormal event and corresponding judgment information. The archived content includes: time interval, event number, probability sequence , peak probability, duration, alarm level, feedback confirmation status and field log. All records are indexed (elevator number, date, event type, etc.) to support fast query and statistical analysis.

[0101] S702, normal sample and background sample extraction: to prevent model overfitting of abnormal samples, the system periodically extracts acceleration segments from non-abnormal segments as a background sample library for comparative learning during retraining. Background samples and abnormal samples are associated by timestamp and can be used to balance sample distribution.

[0102] S703, model self-learning and dynamic optimization: combined with archived abnormal samples and new normal samples, the system periodically performs model adaptive update:

[0103] Self-supervised stage: use the latest unlabeled data to perform unlabeled pre-training tasks (such as signal prediction, masking segment reconstruction) to optimize model bottom features; ​

[0104] Fine-tuning phase: Perform supervised fine-tuning with newly added labeled samples to optimize classification boundaries. The loss function remains the cross-entropy metric function, and the training strategy remains consistent with S417. Through this two-stage learning mechanism, the model can adapt to elevator equipment aging, sensor drift, or environmental noise changes.

[0105] S704, Threshold adaptive adjustment: According to the false alarm rate and false alarm rate statistics in a period of time, the system automatically adjusts the threshold parameter 、 、 、 , etc., to maintain the false alarm rate within the target range (such as <2%). The adjustment process is performed offline and synchronized to the online system after manual review and confirmation.

[0106] S705, Version management and rollback mechanism: All model and parameter updates are recorded with version number, timestamp, training data source, and performance indicators. If the performance of the new version model decreases or is abnormal, the system can roll back to the last stable version with one key, ensuring safe and reliable operation.

[0107] S706, Data export and interface support: Provide standardized interfaces (such as JSON, CSV, Modbus or MQTT protocol) to realize data interaction with maintenance platform, regulatory system. Support export statistical report according to time interval, event type or elevator number, facilitate safety supervision and maintenance decision.

[0108] S707, Privacy and security protection: Implement end-to-end encryption and access control during data transmission and storage; event database uses anonymized identifier, does not contain personal information, complies with data security and privacy protection specifications.

[0109] S708, System cycle optimization: After completing the above steps, the system returns to the monitoring state. Through continuous learning and dynamic updating, realize the closed-loop mechanism of "detection-response-learning-optimization", make the elevator abnormal jump behavior detection system have long-term self-adaptation evolution ability.

[0110] In further embodiments, in step S1, three-axis acceleration sensors (as shown in Figure 2 ) are arranged at multiple positions on the top and bottom of the elevator car to collect vibration or acceleration signals in real time. Each sensor can measure acceleration in X, Y, and Z directions to ensure complete spatial coverage. The sensors work synchronously through a unified sampling clock to collect multi-source acceleration signals at a preset frequency (such as ≥100Hz), and transmit data to the data processing unit in real time to form the original multi-source acceleration time series matrix.

[0111] In a further embodiment, in step S2, the raw acceleration signals collected by each sensor are subjected to multi-level filtering (e.g., Figure 3 As shown, the process includes bandpass filtering, zero-point calibration, and dynamic interference suppression. Hierarchical filtering removes low-frequency drift and high-frequency environmental noise during elevator operation, preserving the transient components related to abnormal passenger jumps. The filtered signal output is an acceleration sequence with a higher signal-to-noise ratio.

[0112] In a further embodiment, in step S3, the filtered multiple signals are aligned to a unified time base (e.g., Figure 3 (As shown on the right). The data from each sensor are synchronized on the time axis, eliminating sampling delay and zero-point differences, and outputting a calibrated multi-source acceleration signal matrix to provide consistent input for candidate event detection.

[0113] In a further embodiment, in step S4, the system performs candidate event detection on the aligned signal (e.g., Figure 4 (As shown). Through threshold comparison and a dual-window strategy, when the acceleration peak value of a certain channel exceeds a preset threshold or multiple channels exhibit abnormal fluctuations at the same time, that moment is determined to be a candidate event. The signal segment corresponding to the candidate event is extracted for further feature analysis.

[0114] In a further embodiment, in step S5, various feature parameters are extracted from the candidate event segments, such as peak value, duration, and frequency band energy, and the multiple features from different sensors are organized into a two-dimensional feature matrix in a "time × channel" manner (e.g., Figure 4 (As shown in the matrix on the right). This matrix integrates information from multiple sources and serves as the input to the intelligent recognition module.

[0115] In a further embodiment, in step S6, the feature matrix is ​​input into a convolutional neural network model for classification and recognition (e.g., Figure 5 (As shown). CNNs include multi-layer convolutional and pooling structures to progressively extract pattern information in the spatial and temporal dimensions. An attention mechanism is introduced into the convolutional network to weight the feature maps, enhancing attention to signals related to anomalous jumps. After Flattening and two fully connected layers, the final Softmax output vector [P] is generated. normal ,P jump ], where P jump This represents the probability that the candidate event is a jump behavior.

[0116] In a further embodiment, in step S7, the final judgment is made by combining the Softmax output result with the dynamic decision mechanism (e.g., Figure 6 As shown). When P jump Greater than the threshold At this time, it is determined as an abnormal jumping behavior; otherwise, it is determined as a normal state. For the abnormal jumping determination, the system further triggers a hierarchical response, including: ① an audible and visual and voice alarm prompt in the elevator car; ② remote reporting to the elevator management center; ③ archiving key information such as event time, floor position, peak size, etc. Through this dynamic threshold determination and multi-level response, the application realizes high-precision identification and real-time disposal of abnormal jumping behavior.

[0117] In summary, through multi-point sensor layout, hierarchical filtering and alignment, candidate event detection and feature matrix construction, and attention mechanism enhanced convolutional neural network analysis and Softmax dynamic decision, the application realizes real-time and intelligent detection of passenger abnormal jumping behavior in the elevator, and has the advantages of strong robustness, low false alarm rate and scalable update.

Claims

1. A method for detecting abnormal jumping behavior based on multi-source acceleration signal feature analysis, characterized in that, The method comprises the following steps: a plurality of acceleration sensors are arranged in an elevator car, and multi-source acceleration time series data are obtained by using the acceleration sensors; clean and aligned acceleration signals are obtained by preprocessing the multi-source acceleration time series data, and the preprocessing method comprises filtering noise and calibrating and aligning multi-sensor data; candidate segments of abnormal jumping behaviors are detected in the clean and aligned acceleration signals, and a multi-source acceleration feature matrix is constructed by using the candidate segments, and the specific method is as follows: The amplitude and instantaneous change of each channel acceleration at each time sampling point are calculated and compared with preset threshold values. When there are sensor channels satisfying or , it is determined that a passenger abnormal jumping event possibly occurs at time, represents an acceleration amplitude threshold value, represents an acceleration increment threshold value, is the amplitude of acceleration at time, is the amplitude of acceleration at time, if there are more than one channel and time satisfying the threshold condition, the event time corresponding to the acceleration peak with the largest amplitude or the earliest time is selected as the candidate jumping event center time according to the acceleration peak size or time sequence with the detected candidate jump event center time construct a time window centered on the candidate jump event center time, and extract a time windowed signal segment from the clean aligned acceleration signal, the windowing operation being applied to the acceleration sequence for each sensor channel synchronously, resulting in a multi-source acceleration segment containing the complete waveform before and after the event; time-frequency transformation is performed on the multi-source acceleration segments to construct a three-dimensional feature tensor, i.e., a multi-source acceleration feature matrix, and the specific method is as follows: Within a time window Each axial channel of each sensor produces a multi-source acceleration segment; The multi-source acceleration segments are arranged into an initial matrix in time x channel form where each row of the matrix represents the multi-channel acceleration observations at a certain time instant within the window, and the columns represent different sensors and their axial channels, totaling columns. For each column of the initial matrix Perform multi-scale discrete wavelet transform to extract frequency domain coefficients at multiple frequency scales. , Indicates the first At the wavelet scale, the first Channel at time The acceleration frequency domain characteristic response; After stacking all the channels, all the scale's frequency domain coefficients along the scale dimension, a three-dimensional tensor is formed: wherein the element in the three-dimensional tensor represents the signal feature of the th channel at the th wavelet scale and the th sampling point within the time window, represents the total number of scales used in the discrete wavelet decomposition; the multi-source acceleration feature matrix is input into a convolutional neural network model to identify and classify abnormal jumping behaviors, and output probabilities corresponding to the abnormal jumping behavior categories are obtained, and the convolutional neural network model comprises an input layer, a first convolutional module composed of two groups of first convolutional layers and first activation layers, a first pooling layer, a second convolutional module composed of two groups of second convolutional layers and second activation layers, a second pooling layer, a third convolutional module composed of two groups of third convolutional layers and third activation layers, a third pooling layer, a time-frequency gated attention fusion module, a first full connection layer, a second full connection layer, and an output layer; the specific process of obtaining the output probabilities corresponding to the abnormal jumping behavior categories is as follows: the multi-source acceleration feature matrix is input through the input layer; the multi-source acceleration feature matrix is first processed by the first convolutional module, and the specific processing process of each group of first convolutional layers and first activation layers is as follows: In the first convolutional layer, the multi-source acceleration feature matrix is convoluted by 64 convolutional kernels ; The convolution kernel slides on the feature matrix with a step of 1, and zero padding is performed at the matrix edge to ensure the time dimension of the convolution output and sensor dimensions As the input, each convolution kernel extracts the associated pattern in the local time-sensor dimension of the feature matrix, and a total of 64 feature map channels are output, forming a three-dimensional feature map tensor with a size of ​ In the first activation layer, the output of the previous layer is applied element-wise to a non-linear activation function the feature map is applied element-wise to a non-linear activation function The output of the first convolution module is input into a first pooling layer, the first pooling layer performs a dimension reduction operation on the feature map obtained by the first convolution module, adopts a maximum pooling filter with a size of 2*2 to slide on the feature map with a step of 2, takes the maximum value of the values in each 2*2 neighborhood as the pooling result, and performs down-sampling on the feature map in the time and sensor dimensions to reduce the size of the feature map to 1 / 2 of the original size. The output of the first convolution module is input into a first pooling layer, the first pooling layer performs a dimension reduction operation on the feature map obtained by the first convolution module, adopts a maximum pooling filter with a size of 2*2 to slide on the feature map with a step of 2, takes the maximum value of the values in each 2*2 neighborhood as the pooling result, and performs down-sampling on the feature map in the time and sensor dimensions to reduce the size of the feature map to 1 / 2 of the original size. The output the down-sampling feature map output by the first pooling layer is input into the second convolutional module, and the specific processing process of each group of second convolutional layers and second activation layers is as follows: The second convolutional layer, through 128... The convolution operation is performed using a convolution kernel that slides with a stride of 1. The kernel plane size is [missing information]. The depth is consistent with the number of channels in the input feature map (64). After convolution, feature patterns are learned at a more abstract level, and the output size is [size missing]. Feature map tensor; the output of the second convolutional layer is input into the second activation layer for processing, and a ReLU activation function is applied to the 128-channel feature map element by element to obtain a feature map after nonlinear mapping; The output of the second convolution module is input into a second pooling layer for processing, which adopts The max pooling reduces the sampling of the feature map of the previous layer. the output of the second pooling layer is input into the third convolutional module, and the specific processing process of each group of third convolutional layers and third activation layers is as follows: In the third convolutional layer, the feature map output from the second pooling layer is used as input, utilizing 256... The convolution operation is performed using a convolution kernel, which scans the input with a stride of 1. The feature map is padded with zeros at the edges to maintain its size, and the output size is [size missing]. Feature map tensor; In the third activation layer, element-wise ReLU non-linear activation is applied to the 256-channel feature map to obtain the activated feature map; The third convolutional module inputs the third pooling layer processing, and performs the last time on the 256-channel feature map max-pooling, which reduces the feature map size by half to about ; The input and output of the third pooling layer are processed by the time-frequency gated attention fusion module, and the output of the third pooling layer is... The feature map is subjected to global average pooling and global variance pooling to form channel statistical vectors, which are then passed through two fully connected layers to generate channel weight vectors; the output of the third pooling layer... A spatial attention map is obtained by applying a 1×1 convolution to the feature map. Broadcast the channel weight vector to the spatial dimension, and associate it with the spatial attention graph. Multiplication to form a gated mask The gated mask is multiplied element-wise with the feature map output from the third pooling layer to obtain the fused features: ; The output of the time-frequency gate attention fusion module is input into a first full connection layer. In the first full connection layer, the fusion features are flattened into one-dimensional feature vectors, and feature compression and combination are performed to obtain a first feature vector ; The output of the first fully connected layer is input to a second fully connected layer, in which the first feature vector is further mapped to a second feature vector of a target output dimension ; The output of the second full connection layer is input into an output layer for classification, and a 2-dimensional feature vector obtained by the second full connection layer is classified An output probability corresponding to the abnormal jump behavior is calculated by applying a Softmax normalization function.

2. The method of claim 1, wherein, the specific method of obtaining the multi-source acceleration time series data by using the acceleration sensors is as follows: ， In the formula, is the moment multi-source acceleration observation vector, , is the total number of samples, wherein is the number of sensors.

3. The method of claim 1, wherein, the specific method of calibrating and aligning the multi-sensor data is as follows: Obtaining an output offset of each sensor in an elevator stationary state ; Data in the filtered acceleration signal sequence is subtracted by the offset ; the acceleration signal sequence after the offset is subtracted is shifted or adjusted in difference.

4. The method of claim 1, wherein, the convolutional neural network model is optimized by using a supervised training combined with a self-supervised learning strategy, and the specific method is as follows: In the offline training phase, labeled data samples containing normal operation and jump events of the elevator are collected, a forward calculation is performed on the feature matrix of each sample by a convolutional neural network model to obtain the Softmax probability of the output layer , and the real class label of the sample is compared , i represents the output category number, and the value range is , the total number of categories; The error back propagation algorithm is used to update the weights of the convolution kernel and the full connection layer according to the loss The gradient of the weights of the convolution kernel and the full connection layer is updated, and the training is iterated until the model converges. a self-supervised learning strategy is introduced in the training process, i.e., a large amount of unlabeled elevator running acceleration data is used to design a pre-training task to assist in optimizing the model parameters; the passenger abnormal jumping behavior is determined and decided according to the output probabilities corresponding to the abnormal jumping behavior categories output by the convolutional neural network model.

Images