Elevator car abnormity intelligent monitoring device and method based on multi-sensor fusion

Abstract

The invention discloses an elevator car abnormity intelligent monitoring device and method based on multi-sensor fusion, and relates to the technical field of vibration monitoring. Static data and multi-modal data of the elevator are obtained, and a simulation model of the elevator car is constructed; determining abnormal vibration configuration through the abnormal vibration type and updating the simulation model to obtain a target model; determining a preset operation mode and inputting the preset operation mode into the target model for simulation to obtain vibration data; and converting the vibration data into time frequency data, performing feature extraction to obtain simulation data, determining an abnormal area according to the simulation data, and performing real-time monitoring on the abnormal area. Static data and multi-modal data are fused to construct a three-dimensional model, and abnormal vibration of the elevator is accurately simulated. Comprehensive vibration data are generated in combination with multiple preset operation modes, key features are extracted through time-frequency analysis and a lightweight network, and efficient detection is achieved. The sensor layout is optimized, the monitoring comprehensiveness is enhanced, the fault risk is reduced through real-time early warning, and the safety and reliability of elevator operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of vibration monitoring technology, and in particular to an intelligent monitoring device and method for elevator car abnormalities based on multi-sensor fusion. Background Art

[0002] With the acceleration of urbanization, the installation of new elevators and the continued rise in the number of existing elevators, safe elevator operation has become a key issue concerning public safety. Elevator vibration is a significant factor affecting safe operation, and abnormalities often indicate potential malfunctions, potentially leading to accelerated component wear or even sudden failure, which in turn affects the normal operation of the elevator.

[0003] Patent No.: CN118047275A, discloses a method for predicting abnormal vibration of an elevator car; the elevator car vibration signal is decomposed by the local mean decomposition method to obtain high-frequency, medium-frequency and low-frequency components, and the above three components are predicted using the improved support vector regression machine prediction model, the improved LightGBM prediction model and the improved GRU prediction model respectively to obtain the high-frequency component prediction result, the medium-frequency component prediction result and the low-frequency component prediction result; finally, the high-frequency component prediction result, the medium-frequency component prediction result and the low-frequency component prediction result are superimposed to obtain the elevator car vibration signal prediction result. The method for predicting abnormal vibration of an elevator car of the present invention achieves the technical effect of improving the accuracy of abnormality prediction by predicting abnormality of the elevator car vibration signal through multiple models.

[0004] In the existing technology, the training of the prediction model requires the use of a large amount of data, so the amount of data calculation is large, which leads to increased costs and reduced efficiency. When the model predicts a vibration anomaly, it is also necessary to verify the actual situation of the elevator. At this time, since the elevator is in real-time operation, there is a time lag from predicting the anomaly to actually obtaining the data, which causes errors in subsequent data analysis. Summary of the Invention

[0005] The purpose of the present invention is to solve the problem that the training of the prediction model requires the use of a large amount of data, so the amount of data calculation is large, which leads to increased costs and reduced efficiency; when the model predicts a vibration abnormality, it is also necessary to verify the actual situation of the elevator. At this time, since the elevator is in real-time operation, there is a time lag in the process from predicting the abnormality to actually obtaining data, which makes the subsequent data analysis erroneous. Therefore, an intelligent monitoring device and method for elevator car abnormalities based on multi-sensor fusion are proposed.

[0006] The purpose of the present invention can be achieved through the following technical solutions: First, an intelligent monitoring device for elevator car abnormalities based on multi-sensor fusion is proposed. The device includes: a data acquisition module, a model update module, a mathematical simulation module, and an area monitoring module: The data acquisition module is used to acquire static data and multimodal data of the elevator, and construct a simulation model of the elevator car based on the static data and multimodal data; the static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; the multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor; The model updating module is used to determine the abnormal vibration configuration according to the abnormal vibration type, and update the simulation model according to the abnormal vibration configuration to obtain the target model; the abnormal vibration type corresponds to the abnormal vibration configuration one by one, and the abnormal vibration type includes: deformation, bending and dislocation; The mathematical simulation module is used to determine a preset operating mode and input the preset operating mode into the target model for simulation to obtain vibration data; the preset operating mode includes: starting acceleration, running at a constant speed, and stopping deceleration; The area monitoring module is used to convert the vibration data into time-frequency data, extract features from the time-frequency data through a lightweight network to obtain simulation data, determine the abnormal area based on the simulation data, and monitor the abnormal area in real time.

[0007] Optionally, the device is further configured to obtain deformation data of the guide rail, obtain a guide rail equation by fitting the deformation data, and determine the type of abnormal vibration according to the type of the guide rail equation; the types of guide rail equations include deformation equations, bending equations, and misalignment equations; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amax is the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: e b (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=ecmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

[0008] Optionally, the lightweight network includes: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; The working principle of the lightweight network is as follows: Taking the time-frequency data as input of the lightweight network, segmenting the time-frequency data through a feature channel to obtain a first feature map and a second feature map; Inputting the first feature map into a 1×1 convolution module to obtain a third feature map, inputting the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and inputting the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into a 3×3 convolution module for downsampling to obtain a fourth feature map, the fourth feature map is input into a 1×1 convolution module to obtain a sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain an eighth feature map, and the eighth feature map is subjected to a channel shuffling operation to obtain a ninth feature map, and the ninth feature map is used as the output of the lightweight network.

[0009] Optionally, the working principle of the multi-attention module includes: Obtain a feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain a first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; Permuting the channel and height dimensions in the first feature tensor to obtain a second feature tensor, and permuting the channel and width dimensions in the first feature tensor to obtain a third feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , the attention weight ω ch and the attention weight ω cw The feature tensors are element-wise multiplied to obtain the final feature tensor.

[0010] Optionally, the area monitoring module includes: an image acquisition module and a wear judgment module: The image acquisition module is configured to determine an image acquisition frequency according to the abnormal area, obtain an image set of the abnormal area according to the image acquisition frequency, and determine a proportion of the wear area of ​​each image in the image set; The wear judgment module is configured to use the value of the largest wear area ratio as a wear score. If the wear score is less than or equal to a wear threshold, the wear condition is determined to be normal wear; otherwise, the wear condition is determined to be abnormal wear.

[0011] An intelligent monitoring method for elevator car abnormalities based on multi-sensor fusion is proposed, the method comprising: Obtaining static data and multimodal data of the elevator, and constructing a simulation model of the elevator car based on the static data and multimodal data; the static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; the multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor; The abnormal vibration configuration is determined by the abnormal vibration type, and the simulation model is updated according to the abnormal vibration configuration to obtain the target model; the abnormal vibration type corresponds to the abnormal vibration configuration one by one, and the abnormal vibration types include deformation, bending and misalignment; Determine a preset operating mode, and input the preset operating mode into the target model for simulation to obtain vibration data; the preset operating mode includes: start acceleration, run at a constant speed, and stop deceleration; The vibration data is converted into time-frequency data, features of the time-frequency data are extracted through a lightweight network to obtain simulation data, abnormal areas are determined based on the simulation data, and the abnormal areas are monitored in real time.

[0012] Optionally, before determining the abnormal vibration configuration according to the abnormal vibration type, the method includes: Obtaining deformation data of the guide rail, fitting the deformation data to obtain a guide rail equation, and determining the type of abnormal vibration based on the type of the guide rail equation; the types of guide rail equations include: deformation equation, bending equation, and misalignment equation; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amaxis the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: e b (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=e cmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

[0013] Optionally, after converting the vibration data into time-frequency data through continuous wavelet transform, performing feature extraction on the time-frequency data through a lightweight network to obtain simulation data includes: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; Working principle: Taking the time-frequency data as input of the lightweight network, segmenting the time-frequency data through a feature channel to obtain a first feature map and a second feature map; Inputting the first feature map into a 1×1 convolution module to obtain a third feature map, inputting the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and inputting the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into a 3×3 convolution module for downsampling to obtain a fourth feature map, the fourth feature map is input into a 1×1 convolution module to obtain a sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain an eighth feature map, and the eighth feature map is subjected to a channel shuffling operation to obtain a ninth feature map, and the ninth feature map is used as the output of the lightweight network.

[0014] Optionally, the working principle of the multi-attention module includes: Obtain a feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain a first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; Permuting the channel and height dimensions in the first feature tensor to obtain a second feature tensor, and permuting the channel and width dimensions in the first feature tensor to obtain a third feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , the attention weight ω ch and the attention weight ω cw The feature tensors are element-wise multiplied to obtain the final feature tensor.

[0015] Optionally, performing real-time monitoring on the abnormal area includes: determining an image acquisition frequency according to the abnormal area, acquiring an image set of the abnormal area according to the image acquisition frequency, and determining a proportion of the wear area of ​​each image in the image set; The value with the largest proportion of the wear area is used as the wear score. If the wear score is ≤ the wear threshold, the wear condition is determined to be normal wear; otherwise, the wear condition is determined to be abnormal wear.

[0016] Beneficial effects of the present invention: This invention proposes an intelligent monitoring method for elevator car anomalies based on multi-sensor fusion. The method acquires static and multimodal data from the elevator and constructs a simulation model of the elevator car. The abnormal vibration configuration is determined based on the abnormal vibration type, and the simulation model is updated to obtain a target model. A preset operating mode is determined and input into the target model to simulate vibration data. The vibration data is converted into time-frequency data and features are extracted to obtain simulation data. Based on the simulation data, abnormal areas are identified and monitored in real time. Static and multimodal data are integrated to construct a three-dimensional model, accurately simulating abnormal elevator vibrations. Comprehensive vibration data is generated by combining multiple preset operating modes. Time-frequency analysis and a lightweight network are used to extract key features for efficient detection. Optimizing sensor layout enhances monitoring comprehensiveness, and real-time warnings reduce the risk of failures, significantly improving the safety and reliability of elevator operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the structure of an intelligent monitoring device for elevator car abnormalities based on multi-sensor fusion provided by an embodiment of the present invention; Figure 2 A flowchart of an intelligent monitoring method for elevator car abnormalities based on multi-sensor fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] Based on the same inventive concept, the present invention also provides an intelligent monitoring device for elevator car abnormalities based on multi-sensor fusion. Figure 1 , Figure 1 A schematic diagram of the structure of an intelligent monitoring device for elevator car abnormalities based on multi-sensor fusion provided by an embodiment of the present invention, comprising: a data acquisition module, a model update module, a mathematical simulation module, and an area monitoring module: The data acquisition module is used to obtain static data and multimodal data of the elevator and construct a simulation model of the elevator car based on the static data and multimodal data. The static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; the multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor. A model update module is used to determine the abnormal vibration configuration based on the abnormal vibration type and update the simulation model according to the abnormal vibration configuration to obtain the target model; the abnormal vibration type corresponds to the abnormal vibration configuration one by one, and the abnormal vibration types include deformation, bending, and misalignment; A mathematical simulation module is used to determine a preset operating mode and input the preset operating mode into the target model for simulation to obtain vibration data; the preset operating modes include: start acceleration, constant speed operation, and stop deceleration; The regional monitoring module is used to convert vibration data into time-frequency data, extract features from the time-frequency data through a lightweight network to obtain simulation data, determine the abnormal area based on the simulation data, and monitor the abnormal area in real time.

[0020] The intelligent elevator car anomaly monitoring device based on multi-sensor fusion, provided by the embodiments of the present invention, accurately simulates abnormal elevator vibrations by fusing static and multimodal data to construct a three-dimensional model. Comprehensive vibration data is generated by combining multiple preset operating modes. Time-frequency analysis and a lightweight network extract key features for efficient detection. Optimizing sensor layout enhances comprehensive monitoring, while real-time warnings reduce the risk of failures, significantly improving the safety and reliability of elevator operation.

[0021] In one embodiment, the device is further configured to obtain deformation data of the guide rail, obtain a guide rail equation by fitting the deformation data, and determine the type of abnormal vibration according to the type of the guide rail equation; the types of guide rail equations include deformation equations, bending equations, and misalignment equations; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amax is the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: e b (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=e cmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

[0022] In one embodiment, the lightweight network includes: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; How lightweight networks work: The time-frequency data is used as the input of the lightweight network, and the time-frequency data is segmented through the feature channel to obtain the first feature map and the second feature map; Input the first feature map into a 1×1 convolution module to obtain a third feature map, input the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and input the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into the 3×3 convolution module for downsampling to obtain the fourth feature map, the fourth feature map is input into the 1×1 convolution module to obtain the sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain the eighth feature map, and the eighth feature map is subjected to a channel shuffle operation to obtain the ninth feature map, which is used as the output of the lightweight network.

[0023] In one embodiment, the working principle of the multi-attention module includes: Get the feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain the first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; The second feature tensor is obtained by permuting the channel and height dimensions in the first feature tensor, and the third feature tensor is obtained by permuting the channel and width dimensions in the first feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , attention weight ω ch and attention weight ω cw Perform element-wise multiplication on the feature tensors to obtain the final feature tensor.

[0024] In one embodiment, the area monitoring module includes: an image acquisition module and a wear determination module: An image acquisition module is used to determine an image acquisition frequency according to the abnormal area, obtain an image set of the abnormal area according to the image acquisition frequency, and determine the proportion of the wear area of ​​each image in the image set; The wear judgment module is used to use the value with the largest proportion of the wear area as the wear score. If the wear score is ≤ the wear threshold, the wear condition is judged to be normal wear; otherwise, the wear condition is judged to be abnormal wear.

[0025] The embodiment of the present invention provides an intelligent monitoring method for elevator car abnormalities based on multi-sensor fusion. Figure 2 , Figure 2 A flowchart of an intelligent monitoring method for elevator car abnormalities based on multi-sensor fusion provided by an embodiment of the present invention. The method includes the following steps: S101, acquiring static data and multimodal data of an elevator, and constructing a simulation model of an elevator car based on the static data and the multimodal data; S102, determining an abnormal vibration configuration according to the abnormal vibration type, and updating the simulation model according to the abnormal vibration configuration to obtain a target model; S103, determining a preset operating mode, inputting the preset operating mode into a target model for simulation to obtain vibration data; S104: Convert the vibration data into time-frequency data, extract features from the time-frequency data using a lightweight network to obtain simulation data, determine abnormal areas based on the simulation data, and monitor the abnormal areas in real time.

[0026] Static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor; abnormal vibration types correspond to abnormal vibration configurations one by one, and abnormal vibration types include: deformation, bending and misalignment; preset operating modes include: start acceleration, constant speed operation and stop deceleration; The intelligent elevator car anomaly monitoring method based on multi-sensor fusion, provided by the embodiments of this invention, accurately simulates abnormal elevator vibrations by fusing static and multimodal data to construct a three-dimensional model. Comprehensive vibration data is generated by combining multiple preset operating modes. Time-frequency analysis and a lightweight network are used to extract key features, enabling efficient detection. Optimizing sensor layout enhances comprehensive monitoring, while real-time warnings reduce the risk of failures, significantly improving the safety and reliability of elevator operation.

[0027] In one implementation, multiple sensors are installed on the elevator car to enhance the realism of the simulation environment and provide richer data support for the construction of the simulation model. These sensors can be used to monitor key parameters such as the elevator's speed, acceleration, noise, and ambient temperature. The data collected by different sensors is used as multimodal data; static data is the basic parameter for building the simulation environment.

[0028] In one implementation, displacement sensors are installed on both sides of the elevator guide rails to measure the displacement of the guide rails in different directions, including vertical and horizontal directions, which can be used to analyze the deformation of the guide rails; by changing the speed v (preset operating mode), the vibration of the elevator at different operating speeds can be simulated; the change in speed will affect the frequency and amplitude of the vibration signal, so more comprehensive vibration data can be generated by simulating different speeds; the preset operating modes include: starting acceleration, running at a constant speed and stopping deceleration. For example, the speed of uniform speed 1 is 1m / s, the speed of uniform speed 2 is 1.5m / s, the speed of uniform speed 3 is 1.75m / s, and the speed of uniform speed 4 is 2m / s. Each uniform speed corresponds to starting acceleration and stopping deceleration.

[0029] In one implementation, the abnormal vibration configuration is determined based on the abnormal vibration type, and the simulation model is updated accordingly to obtain the target model, enabling targeted simulation of different abnormal vibration types. Abnormal vibration types correspond one-to-one to abnormal vibration configurations, including common fault types such as deformation, bending, and misalignment. This targeted configuration enables the model to accurately reflect the vibration characteristics of different fault conditions, enhancing the model's adaptability and recognition capabilities for various abnormal vibration types, providing technical support for subsequent fault diagnosis and early warning.

[0030] In one implementation, vibration data is converted into time-frequency data and feature extracted using a lightweight network, efficiently extracting key features from complex data. Time-frequency analysis captures both the time and frequency information of vibration signals, while the lightweight network significantly reduces computational complexity and model size while maintaining feature extraction accuracy. This efficient feature extraction method enables the model to run quickly on resource-constrained edge devices, improving the real-time performance and response speed of abnormal vibration detection.

[0031] In one implementation, abnormal areas are identified based on simulation data and monitored in real time, enabling the timely detection of potential elevator failures. By extracting feature data from a lightweight network, the model can quickly identify areas of abnormal vibration and provide real-time feedback on the monitoring results. This real-time monitoring and early warning mechanism effectively reduces the risk of failures, improves elevator safety and reliability, and avoids the time lag in guide rail detection caused by signal propagation.

[0032] In one embodiment, before determining the abnormal vibration configuration according to the abnormal vibration type, the method includes: Obtain the deformation data of the guide rail, fit the deformation data to obtain the guide rail equation, and determine the type of abnormal vibration based on the guide rail equation type; the guide rail equation types include: deformation equation, bending equation, and misalignment equation; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amax is the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: eb (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=e cmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

[0033] In one implementation, the system accurately identifies abnormal vibration types by acquiring guide rail deformation data and fitting them to guide rail equations. The deformation equation, bending equation, and misalignment equation each correspond to a different abnormality (the deformation equation identifies a guide rail folding in the same direction at a joint, the bending equation identifies an S-shaped deformation, and the misalignment equation identifies misalignment at a joint). This allows the system to quickly identify specific guide rail issues based on the equation type. This precise identification capability improves the efficiency and accuracy of fault diagnosis.

[0034] In one implementation, the guide rail deformation data is combined with elevator operating parameters (such as time t and speed v) to achieve dynamic monitoring of the elevator operating status. a (x), bending amount e b (x) and the displacement e c (x) The system can provide real-time feedback of abnormal situations during the operation of the elevator, thus enhancing the safety and reliability of the elevator operation.

[0035] In one implementation, the deformation, bending, and misalignment equations are designed to account for various guide rail anomalies, enabling the model to adapt to different types of faults. The introduction of multiple equations improves the model's generalization, enabling it to accurately detect and identify abnormal vibrations under diverse operating conditions, reducing the likelihood of false positives and missed detections. This also improves the accuracy of the mathematical model (simulation model).

[0036] In one embodiment, after converting the vibration data into time-frequency data through continuous wavelet transform, extracting features from the time-frequency data through a lightweight network to obtain simulation data includes: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; Working principle: The time-frequency data is used as the input of the lightweight network, and the time-frequency data is segmented through the feature channel to obtain the first feature map and the second feature map; Input the first feature map into a 1×1 convolution module to obtain a third feature map, input the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and input the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into the 3×3 convolution module for downsampling to obtain the fourth feature map, the fourth feature map is input into the 1×1 convolution module to obtain the sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain the eighth feature map, and the eighth feature map is subjected to a channel shuffle operation to obtain the ninth feature map, which is used as the output of the lightweight network.

[0037] In one implementation, the second feature map is input into a 3×3 convolution module (depthwise separable convolution (DWConv, 3x3stride=2)) for downsampling to obtain a fourth feature map: a 3×3 depthwise separable convolution kernel is used with a stride of 2 for spatial downsampling to reduce the feature map size and extract local features (DWConv can reduce the number of parameters by separating intra-channel and cross-channel calculations); the fourth feature map is input into a 1×1 convolution module to obtain a sixth feature map (1×1 standard convolution adjusts the channel dimension and enhances feature expression capabilities); the third to seventh feature maps, as the outputs of each module, also need to undergo batch normalization processing (standardizing the convolution output, accelerating training convergence and improving generalization capabilities).

[0038] In one implementation, the Mish function: Mish(X)=x*tanh(ln(1+e x )), where x is the input value (i.e., the input signal of a neuron in the neural network, which can be any real number), ln(1+e x ) is the softplus function, tanh is the hyperbolic tangent function, and ln is the natural logarithm function. The Mish function is continuously differentiable and has no upper bound, which improves the efficiency of gradient propagation in deep networks. Mish's nonlinear expression is smoother, retaining more information during feature calibration (such as before and after the attention module), improving the model's accuracy in distinguishing complex time-frequency features.

[0039] In one implementation, the eighth feature map after splicing is shuffled to promote cross-dimensional interaction of features of each branch (first feature map and second feature map); channel shuffling is carried out through multi-branch parallel processing (Z-Pool compresses redundant information, convolution learns cross-channel associations, and Sigmoid generates dynamic weights). After fusion, the average operation (Avg) is performed to generate attention features, realize collaborative calibration of channels and spatial dimensions, dynamically strengthen key feature areas (vibration signals), suppress noise interference, and enhance the model's representation ability and task robustness for complex features. It is particularly suitable for visual tasks that require multi-scale feature interaction (such as classification and detection), and achieves an optimal balance between accuracy and efficiency.

[0040] In one embodiment, the working principle of the multi-attention module includes: Get the feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain the first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; The second feature tensor is obtained by permuting the channel and height dimensions in the first feature tensor, and the third feature tensor is obtained by permuting the channel and width dimensions in the first feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , attention weight ω ch and attention weight ω cw Perform element-wise multiplication on the feature tensors to obtain the final feature tensor.

[0041] In one implementation, the multi-attention module works as follows: Z pool (X)=[MaxPool(X),AvgPool(X)] X′=(ω c ⊙X+ω ch ⊙X+ω cw ⊙X)) / 3 ω c =σ(f 7×7 (Z pool (X))) ω ch =σ(f 7×7 (Z pool (X wz1 ))) ω cw =σ(f 7×7 (Z pool (X wz2 ))) Among them, X represents the feature tensor, MaxPool(X) represents the maximum pooling operation on the feature tensor X, AvgPool(X) represents the average pooling operation on the feature tensor X, and Z pool (X) represents the concatenation of the pooling results of MaxPool(X) and AvgPool(X) in the channel dimension, ω c represents the channel attention weight, ω ch Indicates the attention weight of the high interaction between channel and space, ω cw represents the attention weight of the interaction between channel and spatial width, ⊙ represents element-by-element multiplication, X′ represents the final feature tensor after weighted fusion, f 7×7 represents the convolution operation of the 7×7 convolution module, X wz1 Represents the dimension permutation of the input feature tensor in terms of channel and spatial height, X wz2 It represents the dimension permutation of the input feature tensor in terms of channel and spatial width, and σ represents the Sigmoid activation function.

[0042] In one implementation, X represents a feature tensor (i.e., a feature map, which is stored and transferred as a tensor; the size is H × W × C, where H is the height, W is the width, and C is the number of channels); MaxPool(X) is used to preserve the salient features of the feature tensor X, AvgPool(X) is used to preserve the global features of the feature map X, and the Sigmoid activation function is used to map the weights to [0, 1].

[0043] In one implementation, channel attention (ωc): generates channel weights through mixed pooling (Max+Avg Pool) to highlight important channels; spatial-channel interaction attention (ωc) ch ,ω cw ): Height-channel branch: permutes the feature map dimension to C×H×W to capture the spatial correlation of the channel in the height direction; width-channel branch: permutes the feature map dimension to C×W×H to capture the spatial correlation of the channel in the width direction; feature fusion: weighted fusion of the three sets of attention weights to enhance the diversity of feature expression.

[0044] In one implementation, the 7×7 convolution module introduces only a small number of parameters (7×7 convolution + Sigmoid), reducing computational complexity. Multi-dimensional perception: Simultaneously modeling the dependencies between the three dimensions of channel, height, and width helps to extract local features from the vibration time-frequency graph. Improved generalization: Reduces the risk of overfitting through cross-dimensional interaction. In elevator vibration detection, the multiple attention modules help locate abnormal frequency bands in the time-frequency graph (such as the high-frequency pulse area corresponding to guide rail misalignment), suppress noise interference (such as random vibration in the elevator operating environment), and enhance the feature expression of low-frequency energy concentrated areas (such as periodic low-frequency vibration caused by guide rail bending).

[0045] In one embodiment, real-time monitoring of abnormal areas includes: Determine an image acquisition frequency according to the abnormal area, obtain an image set of the abnormal area according to the image acquisition frequency, and determine the proportion of the wear area of ​​each image in the image set; The value with the largest proportion of the wear area is taken as the wear score. If the wear score is ≤ the wear threshold, the wear condition is judged to be normal wear; otherwise, the wear condition is judged to be abnormal wear.

[0046] In one implementation, the image acquisition frequency is dynamically adjusted based on the abnormal area (for example, the acquisition frequency of the elevator displacement segment is increased to 30 times / second, and the normal segment is 10 times / second), reducing the amount of redundant data while ensuring high-resolution monitoring of key areas and improving system resource utilization.

[0047] In one implementation, a set of images of the abnormal area is acquired according to the image acquisition frequency. That is, when the elevator is about to run into the abnormal area, the image sensor installed on the elevator will acquire images according to the image acquisition frequency to prevent the subsequent images acquired by the image sensor from being deviated from the vibration occurrence point due to the time lag of the signal when the abnormal vibration signal of the elevator car is received.

[0048] In one implementation method, the wear area ratio of each image in the image set is determined, and the guide rail is identified for each image in the image set through the YOLOv8 model. The features of the wear area and normal area of ​​the guide rail are extracted, and the ratio of the wear area of ​​the guide rail to the total area of ​​the guide rail is calculated to obtain the wear area ratio. The wear area ratio with the largest value is used as the wear score. For example, 10 images are collected for a certain section of displacement guide rail, and the wear area ratios of each image are 2%, 5%, 7%, 3%, 12%, 4%, 6%, 1%, 9%, and 2%, respectively. The wear threshold is set to 8%. Since the largest wear area ratio is 12%, 12% is used as the wear score. If 12% is greater than 8%, the wear condition is determined to be abnormal wear.

[0049] In one implementation, 1. For normal wear (wear score ≤ threshold), maintenance monitoring is performed: images continue to be collected at a normal frequency (e.g., 10 times / second), and wear trend reports are generated regularly (weekly / monthly). Preventive maintenance reminders are issued: if the score approaches the threshold (e.g., ≥6%) three times in a row, a local area maintenance report is pushed to the operation and maintenance platform. 2. For abnormal wear (wear score > threshold), real-time alarms are triggered: audio and visual alarms are triggered (inside the elevator car) and push notifications are sent to the IoT platform (SMS / APP), with the abnormal location marked (e.g., "guide rail section 3, coordinate X=5.2m"). Forced speed reduction or elevator stop: If the wear score exceeds 15%, the linkage control system limits the elevator speed to 0.5m / s, stops it at the nearest floor, opens the doors, and generates a maintenance work order. The order is automatically dispatched to the maintenance team, requiring on-site processing within 24 hours, and the repaired inspection images must be uploaded to the cloud.

[0050] It should be noted that, in this document, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements that are inherent to such process, method, article or apparatus.

[0051] While the embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various changes, modifications, substitutions, and alterations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. An intelligent monitoring device for elevator car abnormalities based on multi-sensor fusion, characterized in that: The device includes: a data acquisition module, a model updating module, a mathematical simulation module and a regional monitoring module: The data acquisition module is used to acquire static data and multimodal data of the elevator, and construct a simulation model of the elevator car based on the static data and multimodal data; the static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; the multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor; The model updating module is used to determine the abnormal vibration configuration according to the abnormal vibration type, and update the simulation model according to the abnormal vibration configuration to obtain the target model; the abnormal vibration type corresponds to the abnormal vibration configuration one by one, and the abnormal vibration type includes: deformation, bending and dislocation; The mathematical simulation module is used to determine a preset operating mode and input the preset operating mode into the target model for simulation to obtain vibration data; the preset operating mode includes: starting acceleration, running at a constant speed, and stopping deceleration; The area monitoring module is used to convert the vibration data into time-frequency data, extract features from the time-frequency data through a lightweight network to obtain simulation data, determine the abnormal area based on the simulation data, and monitor the abnormal area in real time.

2. The intelligent monitoring device for elevator car abnormality based on multi-sensor fusion according to claim 1 is characterized in that: The device is also used to obtain deformation data of the guide rail, obtain a guide rail equation by fitting the deformation data, and determine the type of abnormal vibration according to the type of the guide rail equation; the types of guide rail equations include: deformation equation, bending equation and misalignment equation; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amax is the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: e b (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=e cmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

3. The intelligent monitoring device for elevator car abnormality based on multi-sensor fusion according to claim 1 is characterized in that: The lightweight network includes: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; The working principle of the lightweight network is as follows: Taking the time-frequency data as input of the lightweight network, segmenting the time-frequency data through a feature channel to obtain a first feature map and a second feature map; Inputting the first feature map into a 1×1 convolution module to obtain a third feature map, inputting the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and inputting the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into a 3×3 convolution module for downsampling to obtain a fourth feature map, the fourth feature map is input into a 1×1 convolution module to obtain a sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain an eighth feature map, and the eighth feature map is subjected to a channel shuffling operation to obtain a ninth feature map, and the ninth feature map is used as the output of the lightweight network.

4. The intelligent monitoring device for elevator car abnormality based on multi-sensor fusion according to claim 1 is characterized in that: The working principle of the multi-attention module includes: Obtain a feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain a first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; Permuting the channel and height dimensions in the first feature tensor to obtain a second feature tensor, and permuting the channel and width dimensions in the first feature tensor to obtain a third feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , the attention weight ω ch and the attention weight ω cw The feature tensors are element-wise multiplied to obtain the final feature tensor.

5. The intelligent monitoring device for elevator car abnormality based on multi-sensor fusion according to claim 1 is characterized in that: The area monitoring module includes: an image acquisition module and a wear judgment module: The image acquisition module is configured to determine an image acquisition frequency according to the abnormal area, obtain an image set of the abnormal area according to the image acquisition frequency, and determine a proportion of the wear area of ​​each image in the image set; The wear judgment module is configured to use the value of the largest wear area ratio as a wear score. If the wear score is less than or equal to a wear threshold, the wear condition is determined to be normal wear; otherwise, the wear condition is determined to be abnormal wear.

6. An intelligent monitoring method for elevator car abnormalities based on multi-sensor fusion, characterized in that: The method comprises: Obtaining static data and multimodal data of the elevator, and constructing a simulation model of the elevator car based on the static data and multimodal data; the static data includes: guide rail dimensions, car mass, guide shoe equivalent stiffness and damping; the multimodal data includes: acceleration sensor, noise sensor, strain sensor, temperature sensor and displacement sensor; The abnormal vibration configuration is determined by the abnormal vibration type, and the simulation model is updated according to the abnormal vibration configuration to obtain the target model; the abnormal vibration type corresponds to the abnormal vibration configuration one by one, and the abnormal vibration types include deformation, bending and misalignment; Determine a preset operating mode, and input the preset operating mode into the target model for simulation to obtain vibration data; the preset operating mode includes: start acceleration, run at a constant speed, and stop deceleration; The vibration data is converted into time-frequency data, features of the time-frequency data are extracted through a lightweight network to obtain simulation data, abnormal areas are determined based on the simulation data, and the abnormal areas are monitored in real time.

7. The method for intelligent monitoring of elevator car abnormalities based on multi-sensor fusion according to claim 6 is characterized in that: Before determining the abnormal vibration configuration according to the abnormal vibration type, include: Obtaining deformation data of the guide rail, fitting the deformation data to obtain a guide rail equation, and determining the type of abnormal vibration based on the type of the guide rail equation; the types of guide rail equations include: deformation equation, bending equation, and misalignment equation; Deformation equation: When (2i-2)L≤x≤(2i-1)L, e a (x)=e a (x / L)-(2i-2)e amax ; When (2i-1)L≤x≤2iL, e a (x)=-e amax (x / L)+2i*e mmax ; Among them, e a (x) is the guide rail deformation, e amax is the maximum deformation of the guide rail, x=t*v, x is the distance from the starting point of the guide rail to the current position, t is the elevator running time, v is the elevator running speed, i is a positive integer, and L is the length of a single guide rail segment; Bending equation: e b (x)=e bmax sin(πx); Among them, e b (x) is the bending amount of the guide rail, e bmax is the maximum bending amount of the guide rail; Dislocation equation: When (2i-1)L≤x≤2iL, then e c (x)=e cmax ;When (2i-2)L≤x≤(2i-1)L, e c (x)=0; Among them, e c (x) is the misalignment of the guide rail, e cmax is the maximum misalignment of the guide rail.

8. The method for intelligent monitoring of elevator car abnormalities based on multi-sensor fusion according to claim 6 is characterized in that: After the vibration data is converted into time-frequency data through continuous wavelet transform, feature extraction of the time-frequency data is performed through a lightweight network to obtain simulation data, including: Improvements to the lightweight network include: The ReLU function in ShuffleNetV2 is replaced by the Mish function, and multiple attention modules are added to each module in ShuffleNetV2 to obtain a lightweight network; Working principle: Taking the time-frequency data as input of the lightweight network, segmenting the time-frequency data through a feature channel to obtain a first feature map and a second feature map; Inputting the first feature map into a 1×1 convolution module to obtain a third feature map, inputting the third feature map into a 3×3 convolution module for downsampling to obtain a fifth feature map, and inputting the fifth feature map into a 1×1 convolution module to obtain a seventh feature map; The second feature map is input into a 3×3 convolution module for downsampling to obtain a fourth feature map, the fourth feature map is input into a 1×1 convolution module to obtain a sixth feature map, the seventh feature map and the sixth feature map are concatenated to obtain an eighth feature map, and the eighth feature map is subjected to a channel shuffling operation to obtain a ninth feature map, and the ninth feature map is used as the output of the lightweight network.

9. The method for intelligent monitoring of elevator car abnormalities based on multi-sensor fusion according to claim 6, characterized in that: The working principle of the multi-attention module includes: Obtain a feature tensor, perform the maximum pooling operation and the average pooling operation on the feature tensor in sequence to obtain a first feature tensor, and calculate the attention weight ω based on the first feature tensor c ; Permuting the channel and height dimensions in the first feature tensor to obtain a second feature tensor, and permuting the channel and width dimensions in the first feature tensor to obtain a third feature tensor; The attention weight ω is calculated based on the second feature tensor ch , the attention weight ω is calculated based on the third feature tensor cw , according to the attention weight ω c , the attention weight ω ch and the attention weight ω cw The feature tensors are element-wise multiplied to obtain the final feature tensor.

10. The method for intelligent monitoring of elevator car abnormalities based on multi-sensor fusion according to claim 6, characterized in that: Real-time monitoring of the abnormal area includes: determining an image acquisition frequency according to the abnormal area, acquiring an image set of the abnormal area according to the image acquisition frequency, and determining a proportion of the wear area of ​​each image in the image set; The value with the largest proportion of the wear area is used as the wear score. If the wear score is ≤ the wear threshold, the wear condition is determined to be normal wear; otherwise, the wear condition is determined to be abnormal wear.

Images