Escalator safety monitoring system and method based on multi-feature data fusion decision

Abstract

The invention is suitable for the technical field of escalator monitoring, and provides an escalator safety monitoring system and method based on multi-feature data fusion decision making. The system comprises a sensor, an image acquisition device, an edge node and a cloud platform, the sensor acquires running state data of the escalator, and the image acquisition device acquires escalator video images; the edge node performs threshold detection on the operation state data, performs passenger behavior detection on the escalator video image, and sends abnormal operation state data exceeding a threshold range and abnormal behavior information to the cloud platform; and the cloud platform determines a health degree characteristic index of the escalator according to the abnormal operation state data and the abnormal behavior information, calculates the probability that the escalator is in various preset health states according to the health degree characteristic index, determines the comprehensive health degree of the escalator according to the probability, and carries out safety monitoring on the escalator according to the comprehensive health degree of the escalator. According to the invention, automatic identification and active early warning of potential safety hazards of the escalator can be realized, and various potential safety hazards are effectively avoided.

Description

Technical Field

[0001] The present application relates to the technical field of escalator monitoring, and in particular to an escalator safety monitoring system and method based on multi-feature data fusion decision-making. Background Art

[0002] Escalators are a widely used means of transportation in public places, which can facilitate people to move quickly between different floors. Escalators run continuously along an inclined track at a fixed speed through a series of steps and handrails. When passengers step on the steps, the escalator automatically takes them to their destination. Due to its high carrying capacity and high operating efficiency, escalators have become an indispensable vertical transportation equipment in large shopping malls, subway stations, airports and other places.

[0003] However, as escalators age, their components are prone to aging, wear and tear, such as broken steps, loose handrails, and drive system failures. If these problems are not discovered and handled in a timely manner, they can easily lead to serious safety accidents. At the same time, some passengers may engage in unsafe behaviors such as going in the opposite direction, climbing, and pushing when riding escalators, which may also bring risks of injury to themselves and others. Therefore, how to achieve real-time monitoring of escalator equipment status and passenger behavior, and timely warning and handling of safety hazards have become key issues that must be solved to ensure the safe operation of escalators.

[0004] At present, there are two main common means of escalator safety assurance: one is regular manual inspection, in which professionals conduct comprehensive safety inspections and maintenance on escalators; the other is video monitoring, which uses cameras to record the operation of escalators, and manually checks video images to identify abnormal situations. However, both methods have certain limitations: manual inspections have a long cycle, slow results, and are difficult to detect early signs of failure; video monitoring relies on the human eye to identify dangers, which is greatly affected by subjective factors, and its real-time and accuracy cannot be guaranteed. Therefore, there is an urgent need for a more efficient, intelligent, and reliable technical solution that can realize automatic identification and active warning of escalator safety hazards to minimize accidents. Summary of the invention

[0005] In view of this, the embodiments of the present application provide an escalator safety monitoring system and method based on multi-feature data fusion decision-making, which can comprehensively evaluate and accurately monitor the escalator status, effectively reduce the false alarm rate and missed alarm rate, optimize maintenance strategies, and improve the safety and reliability of escalator equipment operation.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an escalator safety monitoring system based on multi-feature data fusion decision-making, including a sensor, an image acquisition device, an edge node and a cloud platform;

[0008] The sensor is arranged at a key position of the escalator, and is used to collect the running status data of the escalator, wherein the running status data includes the vibration information of the steps, the temperature information of the handrails and the noise information of the driving system, and sends the running status data to the edge node;

[0009] The image acquisition device is arranged above the steps and the entrance and exit of the escalator, and is used to acquire the video image of the escalator and send it to the edge node;

[0010] The edge node is used to receive the running status data and the escalator video image, perform threshold detection on the running status data, send abnormal running status data exceeding the threshold range to the cloud platform, and perform passenger behavior detection on the escalator video image, and when it is determined that the passenger has abnormal behavior, send the abnormal behavior information to the cloud platform;

[0011] The cloud platform is used to determine the health characteristic index of the escalator according to the abnormal operation status data and the abnormal behavior information, calculate the probability of the escalator being in multiple preset health states according to the health characteristic index, and determine the comprehensive health of the escalator according to the probability of the escalator health state being in multiple preset health states, and perform safety monitoring of the escalator according to the comprehensive health of the escalator.

[0012] In a second aspect, the embodiment of the present application provides an escalator safety monitoring method based on multi-feature data fusion decision-making, including:

[0013] Acquire the running status data and video images of the escalator, wherein the running status data includes the vibration information of the steps, the temperature information of the handrails and the noise information of the driving system, and the video images of the escalator are acquired by an image acquisition device arranged above the steps and the entrance and exit of the escalator;

[0014] Performing threshold detection on the operating status data based on spectrum energy and spectrum entropy to determine abnormal operating status data exceeding a threshold range;

[0015] Performing passenger behavior detection on the escalator video image to determine abnormal behavior information of the passenger in the video image, wherein the abnormal behavior information includes at least one of falling, walking in the opposite direction, and climbing;

[0016] Determine the health characteristic index of the escalator according to the abnormal operation state data and the abnormal behavior information, wherein the health characteristic index includes a vibration noise related index, a temperature anomaly index, a behavior influence coefficient, a trend index and an environmental influence coefficient;

[0017] Calculating the probability of the escalator being in a plurality of preset health states according to the health characteristic index, wherein the plurality of preset health states include normal, slightly abnormal, moderately abnormal and severely abnormal, and the escalator health states represented by normal, slightly abnormal, moderately abnormal and severely abnormal are getting worse in sequence;

[0018] The comprehensive health of the escalator is determined according to the probability that the health state of the escalator is in multiple preset health states, and the escalator is safely monitored based on the comprehensive health of the escalator.

[0019] Based on the second aspect, in some embodiments, determining the health characteristic index of the escalator according to the abnormal operation status data and the abnormal behavior information includes:

[0020] The Pearson correlation coefficient of the vibration data sequence and the noise data sequence is calculated by r=cov(V,N) / (std(V)*std(N)) to obtain the vibration-noise correlation index r, where V is the vibration information sequence and N is the noise information sequence; wherein the vibration-noise correlation index r reflects the correlation strength between the vibration information and the noise information;

[0021] The temperature anomaly index TAI is calculated by TAI=(T-Tref) / Trange, where T is the current temperature of the escalator, Tref is the normal reference temperature of the escalator, and Trange is the normal temperature variation range of the handrail; wherein the temperature anomaly index TAI reflects the degree of abnormality of the escalator temperature;

[0022] The behavior impact coefficient BIC is calculated by BIC=Σ(Wi×Bi), where Wi is the weight of the i-th abnormal behavior and Bi is the number of occurrences of the abnormal behavior; wherein BIC reflects the comprehensive impact of the passenger's abnormal behavior;

[0023] pass Calculate the trend index TI, Xt is the current value of the operating status data, is the historical average value of the operating status data, and σ is the standard deviation; where the trend index TI reflects the changing trend of the monitored operating status data;

[0024] The environmental impact coefficient EIC is calculated by EIC=(w1T+w2H+w3L)×S, where T is the temperature factor, H is the humidity factor, L is the load factor, S is the season factor, w1, w2, and w3 are the weight coefficients of temperature, humidity, and load, respectively, and w1+w2+w3=1; among which, the environmental impact coefficient EIC reflects the degree of influence of environmental conditions on the operation of the escalator.

[0025] Based on the second aspect, in some embodiments, the calculating the probability of the escalator being in multiple preset health states according to the health characteristic index includes:

[0026] Determine the basic probability distribution of each health characteristic indicator under various preset health states, and adjust the corresponding basic probability distribution based on the preset weights of various health characteristic indicators;

[0027] Calculate the conflict degree based on the basic probability distribution of the adjusted health characteristic indicators;

[0028] When the conflict degree is less than a first conflict degree threshold, the probability of the escalator being in each preset health state is calculated according to the adjusted basic probability distribution;

[0029] When the conflict degree is greater than or equal to the first conflict degree threshold and less than the second conflict degree threshold, the preset weights of various health characteristic indicators are adjusted according to the average conflict degree between each basic probability distribution and other basic probability distributions until the conflict degree is less than the first conflict degree threshold;

[0030] When the conflict degree is greater than or equal to a second conflict degree threshold, an escalator abnormality alarm is generated; wherein the second conflict degree threshold is greater than the first conflict degree threshold.

[0031] Based on the second aspect, in some embodiments, the determining the basic probability distribution of each health characteristic indicator under various preset health states includes: for each preset health state, determining a calculation formula for the basic probability distribution of the health characteristic indicator under each preset health state based on the value of the health characteristic indicator, substituting the value of the health characteristic indicator into the calculation formula to calculate the basic probability distribution of the health characteristic indicator under each preset health state;

[0032] The method of calculating the probability of the escalator being in each preset health state according to the adjusted basic probability distribution includes: for a first preset health state, determining all combinations of a first event and a second event, the intersection of the first event and the second event being the first preset health state, and the first preset health state being one of normal, slightly abnormal, moderately abnormal, and severely abnormal; for each combination of the first event and the second event, multiplying the basic probability distribution corresponding to the first event by the basic probability distribution corresponding to the second event, and adding the products corresponding to each combination of the first event and the second event to obtain the probability of the escalator being in the first preset health state.

[0033] Based on the second aspect, in some embodiments, determining the comprehensive health of the escalator according to the probability that the escalator health state is in multiple preset health states includes:

[0034] according to Calculate the credibility Bel(A) of event A, and calculate the uncertainty interval U(A) of event A according to U(A)=Pl(A)-Bel(A). Event A means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal; and event B means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal.

[0035] The comprehensive health of the escalator is determined based on the three probabilities of the escalator being in a slight abnormality, a moderate abnormality, and a serious abnormality and the credibility Bel(A);

[0036] When the credibility Bel(A) is greater than the credibility threshold and the uncertainty interval U(A) is less than the uncertainty interval threshold, the escalator monitoring plan corresponding to the comprehensive health of the escalator is executed; wherein the escalator monitoring plan includes the escalator being in good condition and requiring no intervention, increasing the inspection frequency, and shutting down for maintenance and overhaul.

[0037] Based on the second aspect, in some embodiments, performing threshold detection on the operating status data based on spectrum energy and spectrum entropy to determine abnormal operating status data exceeding a threshold range includes:

[0038] The running status data is subjected to spectrum analysis by fast Fourier transform, and the spectrum energy and spectrum entropy of the running status data are calculated; wherein, Calculate the spectrum energy E of the operating status data by Calculate the spectrum entropy H of the operating status data, x(t) is the running status data;

[0039] Comparing the spectrum energy with a preset spectrum energy range, and comparing the spectrum entropy with a preset spectrum entropy range;

[0040] If the spectrum energy exceeds the preset spectrum energy range, and / or the spectrum entropy exceeds the preset spectrum entropy range, the operating status data is regarded as abnormal operating status data.

[0041] Based on the second aspect, in some embodiments, before performing spectrum analysis on the operating status data by fast Fourier transform, performing threshold detection on the operating status data to determine abnormal operating status data exceeding the threshold range further includes:

[0042] Extracting local features for characterizing the health status of the escalator from the operating status data by means of a convolutional neural network;

[0043] The local features of various escalator operation status data are adaptively integrated through the attention mechanism to obtain the global health status representation of the escalator. By integrating the local features of various escalator operation status data, the global health status representation of the escalator is obtained; (a) is the global health status representation vector of the escalator, represents the feature vector of the data collected by the i-th sensor, is the attention weight of the i-th sensor, N is the number of sensors, and is the hidden representation of the data collected by the i-th sensor, u (a) is the attention vector, and are the preset parameters for linear transformation;

[0044] Afterwards, a spectrum analysis is performed on the global health status characterization of the escalator through fast Fourier transform, the spectrum energy and spectrum entropy of the global health status characterization of the escalator are calculated, and the above-mentioned steps of comparing the spectrum energy with the preset spectrum energy range and comparing the spectrum entropy with the preset spectrum entropy range are performed.

[0045] Based on the second aspect, in some embodiments, the performing passenger behavior detection on the escalator video image to determine abnormal behavior information of the passenger in the video image includes:

[0046] Inputting the escalator video image into a convolutional neural network to generate a feature map, and performing human body detection based on the feature map;

[0047] After detecting the human body, determining the position coordinates of multiple key points of the human body;

[0048] Mapping the position coordinates of the multiple key points to the three-dimensional space coordinate system of the escalator to obtain the position and posture of the passenger on the escalator;

[0049] Determining abnormal behavior information of the passenger in the video image according to the position and posture;

[0050] The determining of abnormal behavior information of the passenger in the video image according to the position and posture includes: calculating the moving speed and average speed of the passenger based on the position and posture, and determining whether the passenger has abnormal retrograde behavior according to the average speed; determining the hand key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal climbing behavior according to the coordinate values ​​of the hand key points and the foot key points on the vertical coordinate axis; determining the head key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal falling behavior according to the coordinate values ​​of the head key points and the foot key points on the vertical coordinate axis;

[0051] The method of determining abnormal behavior information of a passenger in a video image based on the position and posture includes: constructing multiple vector representations between adjacent key points, wherein the two key points corresponding to each vector representation are key points of the same limb of the human body; if there is an angle change between two vector representations that is greater than a threshold, determining the abnormal behavior based on the angles before and after the two vector changes and a preset abnormal behavior vector angle set.

[0052] Based on the second aspect, in some embodiments, after performing passenger behavior detection on the escalator video image and determining abnormal behavior information of passengers in the video image, the method further includes:

[0053] Determine the area where the passenger is located; wherein the area includes the first area, the second area and the third area of ​​the escalator, and the first area, the second area and the third area of ​​the escalator are sequentially connected and include the entire escalator;

[0054] Generate an escalator control strategy based on the abnormal behavior, the area where the passengers are located, and the distribution of passengers on the escalator;

[0055] Among them, the escalator control strategy includes:

[0056] If the abnormal behavior is climbing, the escalator will be controlled to slow down and issue a warning. If the climbing behavior lasts longer than the preset time, the escalator will be controlled to temporarily stop running.

[0057] When the abnormal behavior is falling, if there are other passengers in the adjacent area behind the area where the passenger who fell is located and the passenger density is greater than the density threshold, or the current time is within the preset time period, the escalator is controlled to stop running; otherwise, the escalator is controlled to slow down and issue a voice warning; wherein, the area behind the area is determined based on the escalator running direction, and the opposite direction of the escalator running direction is the back.

[0058] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0059] In an embodiment of the present application, first, the operating status data of the escalator is obtained and the escalator video image is collected, and a threshold detection is performed on the operating status data through the edge node, and abnormal operating status data exceeding the threshold range is sent to the cloud platform, and the passenger behavior detection is performed on the escalator video image through the edge node. When it is determined that the passenger has abnormal behavior, the abnormal behavior is sent to the cloud platform, and the cloud platform determines the health characteristic index of the escalator based on the abnormal operating status data and the abnormal behavior information, calculates the probability of the escalator being in multiple preset health states based on the health characteristic index, and determines the comprehensive health of the escalator based on the probability of the escalator health state being in multiple preset health states, and performs safety monitoring on the escalator based on the comprehensive health of the escalator.

[0060] The embodiment of the present application is based on the escalator operation status data and escalator video images collected by sensors, determines the escalator's health characteristic indicators according to the abnormal operation status data and abnormal behavior information, and then calculates the probability of the escalator being in multiple preset health states, and then determines the escalator's comprehensive health according to the probability of the escalator's health state being in multiple preset health states, thereby performing safety monitoring of the escalator based on the escalator's comprehensive health, achieving a comprehensive assessment of the escalator's state and precise safety monitoring, and having the capabilities of automated data collection, intelligent state assessment, and predictive maintenance decision-making, which can effectively reduce the false alarm rate and missed alarm rate, optimize maintenance strategies, and improve the safety and reliability of escalator equipment operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0062] Figure 1 is a schematic diagram of an escalator safety monitoring system based on multi-feature data fusion decision-making provided in an embodiment of the present application;

[0063] Figure 2 It is a flowchart of an escalator safety monitoring method based on multi-feature data fusion decision-making provided in an embodiment of the present application;

[0064] Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The present application is described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present application, but are not intended to limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present application. These all fall within the scope of protection of the present application.

[0066] In order to make the purpose, technical solutions and advantages of the present application clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0067] See also Figure 1 The escalator safety monitoring system based on multi-feature data fusion decision-making provided in the embodiment of the present application may include sensors, image acquisition devices, edge nodes and cloud platforms.

[0068] The sensor is arranged at a key position of the escalator to collect the running status data of the escalator, wherein the running status data includes the vibration information of the steps, the temperature information of the handrails and the noise information of the driving system, and sends the running status data to the edge node.

[0069] The image acquisition device is set above the steps and entrances and exits of the escalator to collect escalator video images and send them to the edge nodes.

[0070] The edge node is used to receive the running status data and the escalator video image, perform threshold detection on the running status data, send abnormal running status data exceeding the threshold range to the cloud platform, and perform passenger behavior detection on the escalator video image, and when it is determined that the passenger has abnormal behavior, send the abnormal behavior information to the cloud platform;

[0071] The cloud platform is used to determine the health characteristic indicators of the escalator based on abnormal operation status data and abnormal behavior information, calculate the probability of the escalator being in multiple preset health states based on the health characteristic indicators, and determine the comprehensive health of the escalator based on the probability of the escalator health state being in multiple preset health states, and perform safety monitoring of the escalator based on the comprehensive health of the escalator.

[0072] Among them, there are multiple sensors, which can be set in key parts of the escalator such as the steps, handrails, and drive system to collect escalator operation status data in real time.

[0073] Specifically, vibration sensors and temperature sensors can be placed on the steps, vibration sensors can be placed on the handrails, and noise sensors can be placed near the drive system. When selecting sensors, vibration sensors use accelerometers with a frequency response range of 0.1Hz to 1kHz and a sensitivity better than 0.1m / s 2 The noise sensor uses a capacitive microphone with a frequency range of 20Hz to 8kHz and a sensitivity better than 60dB. The temperature sensor uses a platinum resistor PT100 with a measurement range of -50℃ to 100℃ and a resolution better than 0.1℃. The protection level of all sensors reaches IP68, and the long-term stability drift is less than 0.2% / year.

[0074] For example, the escalator video image can be collected by a camera, and two cameras can be installed at the steps and entrances and exits of the escalator, and the camera field of view can completely cover the step surface and handrails. Among them, the minimum resolution of the camera is 1080P, the frame rate is not less than 25fps, it has infrared night vision function, and the video stream is encoded in H.265.

[0075] The following combination Figure 1 The escalator safety monitoring method based on multi-feature data fusion decision-making provided in an embodiment of the present application is described.

[0076] See also Figure 2 The escalator safety monitoring method based on multi-feature data fusion decision-making provided in the embodiment of the present application may include the following steps:

[0077] Step 201, obtaining the running status data and video image of the escalator.

[0078] The operating status data includes vibration information of the steps, temperature information of the handrails and noise information of the driving system.

[0079] For example, the escalator operation status data can be collected in real time by installing sensors in key parts such as the steps, handrails, and drive system of the escalator. For example, vibration sensors and temperature sensors can be arranged on the steps, vibration sensors can be arranged on the handrails, and noise sensors can be arranged near the drive system. For specific requirements for sensors, please refer to the above content on sensor selection.

[0080] For example, the escalator video image can be collected by a camera, and two cameras can be installed at the steps and entrances and exits of the escalator, and the camera angle of view can completely cover the step surface and handrails. For specific requirements for the camera, please refer to the parameter requirements of the camera mentioned above.

[0081] Step 202: Perform threshold detection on the operating status data based on spectrum energy and spectrum entropy to determine abnormal operating status data exceeding the threshold range.

[0082] In some embodiments, step 202 may specifically include: performing spectral analysis on the operating status data through fast Fourier transform, calculating the spectral energy and spectral entropy of the operating status data; comparing the spectral energy with a preset spectral energy range, and comparing the spectral entropy with a preset spectral entropy range; if the spectral energy exceeds the preset spectral energy range, and / or the spectral entropy exceeds the preset spectral entropy range, then treating the operating status data as abnormal operating status data.

[0083] For example, you can Calculate the spectrum energy E of the operating status data by Calculate the spectrum entropy H of the operating status data, x(t) is the running status data.

[0084] If the spectrum energy E or spectrum entropy H of a certain operation status data frame exceeds the normal range, the operation status data frame will be marked as suspected abnormality and uploaded to the cloud platform first.

[0085] The operating status data is a continuous signal that changes over time and directly reflects the real-time status of each escalator component. After receiving this data, the edge node can perform some basic preprocessing (such as removing obvious outliers), and then apply fast Fourier transform to these time domain data for processing, especially for periodic signals such as vibration and noise. This processing method can retain the characteristics of the original data to the greatest extent, which is conducive to timely detection of subtle changes in the escalator status.

[0086] In some embodiments, a reasonable warning threshold can be set for each operating status data, and a warning message is generated when the monitoring value exceeds the threshold. The warning threshold can be set as follows: the step vibration speed threshold is 0.5m / s, the noise threshold is 75dB, and the temperature threshold is 50°C. i Exceeding the warning threshold The degree can be quantified by the following formula:

[0087]

[0088] In the formula, Δ i is the percentage of parameter i exceeding the standard. i When it is greater than the preset threshold (such as 10%), an early warning message is generated and sent to the cloud platform and the terminals of relevant personnel. The early warning message contains metadata such as the name of the exceeded parameter, the exceeded percentage, the measurement point location, and the timestamp.

[0089] In this embodiment, the operating status data collected by the sensor can be uploaded to the edge node via industrial Ethernet or 5G network. The sensor nodes converge to the edge nodes nearby, and the edge nodes form a local Ethernet and access the cloud platform through 5G base stations or optical fiber dedicated lines. The data frame format of the operating status data complies with the OPC UA (Open Platform Communications Unified Architecture) protocol standard, with a transmission delay of less than 10ms and a jitter of less than 1ms.

[0090] In some embodiments, before performing spectrum analysis on the operating status data by fast Fourier transform, step 202 may further include:

[0091] Extracting local features for characterizing the health status of the escalator from the operating status data by means of a convolutional neural network;

[0092] The local features of various escalator operation status data are adaptively integrated through the attention mechanism to obtain the global health status representation of the escalator. By integrating the local features of various escalator operation status data, the global health status representation of the escalator is obtained; (a) is the global health status representation vector of the escalator, represents the feature vector of the data collected by the i-th sensor, is the attention weight of the i-th sensor, N is the number of sensors, and is the hidden representation of the data collected by the i-th sensor, u (a) is the attention vector, and are the preset parameters for linear transformation;

[0093] Afterwards, a spectrum analysis is performed on the global health status characterization of the escalator through fast Fourier transform, the spectrum energy and spectrum entropy of the global health status characterization of the escalator are calculated, and the above-mentioned steps of comparing the spectrum energy with the preset spectrum energy range and comparing the spectrum entropy with the preset spectrum entropy range are performed.

[0094] Specifically, the edge node can extract local features used to characterize the health status of the escalator from the operating status data through a convolutional neural network, and adaptively fuse the local features of various escalator operating status data through an attention mechanism to obtain the global health status representation of the escalator. Threshold detection is performed based on the global health status representation of the escalator, and abnormal operating status data that exceeds the threshold range is sent to the cloud platform.

[0095] Specifically, an independent CNN subnetwork can be designed for each type of sensor data to extract local features, and then the features extracted by different subnetworks can be adaptively fused through the attention mechanism to obtain the global health status representation of the escalator.

[0096] For example, you can By integrating the local features of various escalator operation status data, the global health status representation of the escalator is obtained; where v (a) is the global health status representation vector of the escalator, represents the feature vector of the data collected by the i-th sensor, is the attention weight of the i-th sensor, N is the number of sensors, and is the hidden representation of the data collected by the i-th sensor, u (a) is the attention vector, and The default parameters for linear transformation.

[0097] Through the attention mechanism, the system can adaptively adjust the importance of different sensor features. For example, in some cases, the vibration feature may receive a higher attention weight, indicating that the current health status of the escalator is mainly determined by the vibration data. This dynamic adjustment capability enables the system to more flexibly and accurately evaluate the overall health status of the escalator.

[0098] Step 203: Perform passenger behavior detection on the escalator video image to determine abnormal behavior information of passengers in the video image.

[0099] In some embodiments, step 203 may specifically include: inputting the escalator video image into a convolutional neural network to generate a feature map, and performing human body detection based on the feature map; after detecting the human body, determining the position coordinates of multiple key points of the human body; mapping the position coordinates of the multiple key points to the escalator three-dimensional space coordinate system to obtain the position and posture of the passenger on the escalator; and determining abnormal behavior information of the passenger in the video image based on the position and posture.

[0100] Exemplarily, the above-mentioned determination of abnormal behavior information of a passenger in a video image based on the position and posture may include: calculating the passenger's moving speed and average speed based on the position and posture, and determining whether the passenger has abnormal retrograde behavior based on the average speed; determining the hand key points and foot key points among the multiple key points, and determining whether the passenger has abnormal climbing behavior based on the coordinate values ​​of the hand key points and the foot key points on the vertical coordinate axis; determining the head key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal falling behavior based on the coordinate values ​​of the head key points and the foot key points on the vertical coordinate axis.

[0101] For example, the affine transformation matrix obtained in advance through calibration is used to transform the 2D pixel coordinates into coordinates in the 3D escalator coordinate system. The transformation formula is as follows:

[0102]

[0103] Where: (x, y) is the pixel coordinate of the key point in the 2D image, (X, Y) is the coordinate in the escalator's three-dimensional space coordinate system after transformation, and a 11 ,a 12 ,a 21 ,a 22 are the scaling, rotation, and shearing components of the transformation matrix, (t x ,t y ) is the translation component.

[0104] Exemplarily, the above-mentioned calculation of the moving speed and average speed of the passenger based on the position and posture, and determination of whether the passenger has abnormal retrograde behavior according to the average speed specifically includes:

[0105] pass Calculate the passenger's moving speed, is the passenger's velocity vector, is the position vector of the target key point in the position and posture at time t1, is the position vector of the target key point at time t2, t1 and t2 are the timestamps of two consecutive frames respectively;

[0106] pass Calculate the average speed of the passengers, is the average velocity vector, n is the total number of frames of the escalator video image, is the velocity vector of the passenger corresponding to the i-th frame of the escalator video image;

[0107] If the value of the average speed of the passenger is greater than the speed threshold, it is determined that the passenger has abnormal retrograde behavior, and the direction of the speed threshold is opposite to the running direction of the escalator.

[0108] For example, the above-mentioned determining the hand key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal climbing behavior based on the hand key points and the foot key points, specifically includes: determining a hand key point set and a foot key point set from the multiple key points; if there are a first target position point and a second target position point, determining that the passenger has abnormal climbing behavior; wherein the first target position point is a position point in the hand key point set, the second target position point is a position point in the foot key point set, and the coordinate value of the first target position point on the vertical coordinate axis is less than the coordinate value of the second target position point on the vertical coordinate axis.

[0109] For example, the above-mentioned determination of the head key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal falling behavior according to the coordinate values ​​of the head key points and the foot key points on the vertical coordinate axis, specifically includes: determining a head key point set, a knee key point set and a foot key point set from the multiple key points; if there are a third target position point, a fourth target position point and a fifth target position point, determining that the passenger has abnormal falling behavior; wherein the third target position point is a position point in the head key point set, the fourth target position point is a position point in the knee key point set, the fifth target position point is a position point in the foot key point set, and the difference in the coordinate values ​​of the third target position point, the fourth target position point and the fifth target position point on the vertical coordinate axis becomes smaller and is less than a threshold value.

[0110] Exemplarily, the above-mentioned determination of abnormal behavior information of passengers in the video image based on the position and posture may include: constructing multiple vector representations between adjacent key points, the two key points corresponding to each vector representation are key points of the same limb part of the human body; if there is an angle change between two vector representations greater than a threshold, determining the abnormal behavior based on the angles before and after the two vector changes and a preset abnormal behavior vector angle set.

[0111] Specifically, an abnormal behavior vector angle set can be pre-constructed: the key points of each limb of the passenger are connected into multiple vector representations, the angles between the multiple vector representations before the passenger's abnormal behavior occurs are determined, and then the angles between the multiple vector representations after the passenger's abnormal behavior occurs are determined, and then the specific abnormal behavior is marked. After the abnormal behavior vector angle set is constructed, the constructed abnormal behavior vector angle set is used to detect abnormal behavior of passengers: multiple vector representations between adjacent key points of each passenger are constructed, and the two key points corresponding to each vector representation are the key points of the same limb of the human body; if the angle change between two vector representations of a certain passenger is greater than a threshold, the abnormal behavior of the passenger is determined based on the angles before and after the two vectors change and the preset abnormal behavior vector angle set.

[0112] In some embodiments, after step 203, the above-mentioned escalator safety monitoring method based on multi-feature data fusion decision may also include: determining the area where the passenger is located; wherein the area includes the first area, the second area and the third area of ​​the escalator, and the first area, the second area and the third area of ​​the escalator are connected in sequence and include the entire escalator; generating an escalator control strategy based on the abnormal behavior, the area where the passenger is located and the distribution of passengers on the escalator.

[0113] Among them, the escalator control strategy includes:

[0114] If the abnormal behavior is climbing, the escalator will be controlled to slow down and issue a warning. If the climbing behavior lasts longer than the preset time, the escalator will be controlled to temporarily stop running.

[0115] When the abnormal behavior is falling, if there are other passengers in the adjacent area behind the area where the passenger who fell is located and the passenger density is greater than the density threshold, or the current time is within the preset time period, the escalator is controlled to stop running; otherwise, the escalator is controlled to slow down and issue a voice warning; wherein, the area behind the area is determined based on the escalator running direction, and the opposite direction of the escalator running direction is the back.

[0116] In some embodiments, a sliding window method may be used to detect abnormal behaviors in M ​​consecutive frames of escalator video images; if abnormal behaviors are detected more than K times, the above step of determining the area where the passenger is located is performed.

[0117] Step 204: determining the health characteristic index of the escalator according to the abnormal operation status data and the abnormal behavior information.

[0118] In some embodiments, the implementation process of step 204 may be:

[0119] The Pearson correlation coefficient of the vibration data sequence and the noise data sequence is calculated by r=cov(V,N) / (std(V)*std(N)) to obtain the vibration-noise correlation index r, where V is the vibration information sequence and N is the noise information sequence; wherein the vibration-noise correlation index r reflects the correlation strength between the vibration information and the noise information;

[0120] The temperature anomaly index TAI is calculated by TAI=(T-Tref) / Trange, where T is the current temperature of the escalator, Tref is the normal reference temperature of the escalator, and Trange is the normal temperature variation range of the handrail; wherein the temperature anomaly index TAI reflects the degree of abnormality of the escalator temperature;

[0121] The behavior impact coefficient BIC is calculated by BIC=Σ(Wi×Bi), where Wi is the weight of the i-th abnormal behavior and Bi is the number of occurrences of the abnormal behavior; wherein BIC reflects the comprehensive impact of the passenger's abnormal behavior;

[0122] pass Calculate the trend index TI, Xt is the current value of the operating status data, is the historical average value of the operating status data, and σ is the standard deviation; where the trend index TI reflects the changing trend of the monitored operating status data;

[0123] The environmental impact coefficient EIC is calculated by EIC=(w1T+w2H+w3L)×S, where T is the temperature factor, H is the humidity factor, L is the load factor, S is the season factor, w1, w2, and w3 are the weight coefficients of temperature, humidity, and load, respectively, and w1+w2+w3=1; among which, the environmental impact coefficient EIC reflects the degree of influence of environmental conditions on the operation of the escalator.

[0124] Step 205, calculating the probability of the escalator being in a plurality of preset health states according to the health characteristic index.

[0125] In some embodiments, the implementation process of step 205 may be:

[0126] Determine the basic probability distribution of each health characteristic indicator under various preset health states, and adjust the corresponding basic probability distribution based on the preset weights of various health characteristic indicators;

[0127] Calculate the conflict degree based on the basic probability distribution of the adjusted health characteristic indicators;

[0128] When the conflict degree is less than a first conflict degree threshold, the probability of the escalator being in each preset health state is calculated according to the adjusted basic probability distribution;

[0129] When the conflict degree is greater than or equal to the first conflict degree threshold and less than the second conflict threshold, adjust the preset weights of various health characteristic indicators according to the average conflict degree between each basic probability assignment and other basic probability assignments until the conflict degree is less than the first conflict degree threshold;

[0130] When the conflict degree is greater than or equal to the second conflict threshold, generate an escalator abnormality alarm; wherein, the second conflict degree threshold is greater than the first conflict degree threshold.

[0131] Exemplarily, the above determination of the basic probability assignment of each health characteristic indicator in various preset health states may include: for each preset health state, determine the calculation formula for the basic probability assignment of the health characteristic indicator in each preset health state based on the value of the health characteristic indicator, and substitute the value of the health characteristic indicator into the calculation formula to calculate the basic probability assignment of the health characteristic indicator in each preset health state. Wherein, the basic probability assignment represents the probability of judging that the escalator is in various preset health states based on a certain health characteristic indicator.

[0132] The following gives an example of how to determine the basic probability assignment of each health characteristic indicator in various preset health states.

[0133] For the vibration and noise related indicator r, define r ≤ 0.5 as normal, 0.5 < r ≤ 0.8 as slightly abnormal, 0.8 < r ≤ 0.9 as moderately abnormal, and r > 0.9 as severely abnormal. Based on this:

[0134] When r ≤ 0.5, calculate the probability of judging that the escalator is in the normal state based on the vibration and noise related indicator r through m1(normal) = 1 - r 2 Calculate the probability of judging that the escalator is in the slightly abnormal state based on the vibration and noise related indicator r through m1(slightly abnormal) = 0.7r 2 Calculate the probability of judging that the escalator is in the moderately abnormal state based on the vibration and noise related indicator r through m1(moderately abnormal) = 0.2r 2 Calculate the probability of judging that the escalator is in the severely abnormal state based on the vibration and noise related indicator r through m1(severely abnormal) = 0.1r 2 Calculate the probability of judging that the escalator is in the severely abnormal state based on the vibration and noise related indicator r.

[0135] When \(0.5 \lt r \leq 0.8\), the probability of judging that the escalator is in a normal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{normal}) = 0.3(1 - r)\); the probability of judging that the escalator is in a slightly abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{slightly abnormal}) = 0.4 + 0.3r\); the probability of judging that the escalator is in a moderately abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{moderately abnormal}) = 0.2r\); the probability of judging that the escalator is in a severely abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{severely abnormal}) = 0.1r\).

[0136] When \(0.8 \lt r \leq 0.9\), the probability of judging that the escalator is in a normal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{normal}) = 0.1(1 - r)\); the probability of judging that the escalator is in a slightly abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{slightly abnormal}) = 0.2(1 - r)\); the probability of judging that the escalator is in a moderately abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{moderately abnormal}) = 0.5 + 0.3r\); the probability of judging that the escalator is in a severely abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{severely abnormal}) = 0.2r\).

[0137] When \(r \gt 0.9\), the probability of judging that the escalator is in a normal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{normal}) = 0.1(1 - r)\); the probability of judging that the escalator is in a slightly abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{slightly abnormal}) = 0.1(1 - r)\); the probability of judging that the escalator is in a moderately abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{moderately abnormal}) = 0.2(1 - r)\); the probability of judging that the escalator is in a severely abnormal state based on the vibration noise related index \(r\) is calculated by \(m_1(\text{severely abnormal}) = 0.6 + 0.4r\).

[0138] For the temperature anomaly index TAI, TAI \(\lt 0.3\) is defined as normal, \(0.3 \leq \text{TAI} \lt 0.6\) is defined as slightly abnormal, \(0.6 \leq \text{TAI} \lt 0.8\) is defined as moderately abnormal, and \(\text{TAI} \geq 0.8\) is defined as severely abnormal. Based on this:

[0139] When \(\text{TAI} \lt 0.3\), the probability of judging that the escalator is in a normal state based on the temperature anomaly index TAI is calculated by \(m_2(\text{normal}) = 0.7 + 0.3(1 - \text{TAI} / 0.3)\); the probability of judging that the escalator is in a slightly abnormal state based on the temperature anomaly index TAI is calculated by \(m_2(\text{slightly abnormal}) = 0.3(\text{TAI} / 0.3)\); the probability of judging that the escalator is in a moderately abnormal state based on the temperature anomaly index TAI is calculated by \(m_2(\text{moderately abnormal}) = 0\); the probability of judging that the escalator is in a severely abnormal state based on the temperature anomaly index TAI is calculated by \(m_2(\text{severely abnormal}) = 0\).

[0140] When 0.3≤TAI<0.6, the probability of the escalator being in a normal state based on the temperature anomaly index TAI is calculated by m2(normal)=0.3(0.6-TAI) / 0.3, the probability of the escalator being in a slightly abnormal state based on the temperature anomaly index TAI is calculated by m2(slight abnormality)=0.5+0.3(TAI-0.3) / 0.3, the probability of the escalator being in a moderately abnormal state based on the temperature anomaly index TAI is calculated by m2(moderate abnormality)=0.2(TAI-0.3) / 0.3, and the probability of the escalator being in a severely abnormal state based on the temperature anomaly index TAI is calculated by m2(severe abnormality)=0.

[0141] When 0.6≤TAI<0.8, the probability of the escalator being in a normal state based on the temperature anomaly index TAI is calculated by m2(normal)=0.1(0.8-TAI) / 0.2, the probability of the escalator being in a slightly abnormal state based on the temperature anomaly index TAI is calculated by m2(slight abnormality)=0.2(0.8-TAI) / 0.2, the probability of the escalator being in a moderately abnormal state based on the temperature anomaly index TAI is calculated by m2(moderate abnormality)=0.5+0.3(TAI-0.6) / 0.2, and the probability of the escalator being in a seriously abnormal state based on the temperature anomaly index TAI is calculated by m2(severe abnormality)=0.2(TAI-0.6) / 0.2.

[0142] When TAI≥0.8, the probability of the escalator being in a normal state based on the temperature anomaly index TAI is calculated by m2(normal)=0, the probability of the escalator being in a slightly abnormal state based on the temperature anomaly index TAI is calculated by m2(slight abnormality)=0.1(1-TAI), the probability of the escalator being in a moderately abnormal state based on the temperature anomaly index TAI is calculated by m2(moderate abnormality)=0.2(1-TAI), and the probability of the escalator being in a severely abnormal state based on the temperature anomaly index TAI is calculated by m2(severe abnormality)=0.7+0.3TAI.

[0143] For the behavioral impact coefficient BIC, BIC < 1.2 is defined as normal, 1.2 ≤ BIC < 1.5 is defined as slightly abnormal, 1.5 ≤ BIC < 1.8 is defined as moderately abnormal, and BIC ≥ 1.8 is defined as severely abnormal. Based on this:

[0144] When BIC<1.2, the probability of the escalator being in a normal state based on the behavior influence coefficient BIC is calculated by m3(normal)=0.7+0.3(1.2-BIC) / 1.2, the probability of the escalator being in a slightly abnormal state based on the behavior influence coefficient BIC is calculated by m3(slightly abnormal)=0.3(BIC / 1.2), the probability of the escalator being in a moderately abnormal state based on the behavior influence coefficient BIC is calculated by m3(moderately abnormal)=0, and the probability of the escalator being in a severely abnormal state based on the behavior influence coefficient BIC is calculated by m3(severely abnormal)=0.

[0145] When 1.2≤BIC<1.5, the probability of the escalator being in a normal state based on the behavior influence coefficient BIC is calculated by m3(normal)=0.3(1.5-BIC) / 0.3, the probability of the escalator being in a slightly abnormal state based on the behavior influence coefficient BIC is calculated by m3(slightly abnormal)=0.5+0.3(BIC-1.2) / 0.3, the probability of the escalator being in a moderately abnormal state based on the behavior influence coefficient BIC is calculated by m3(moderately abnormal)=0.2(BIC-1.2) / 0.3, and the probability of the escalator being in a severely abnormal state based on the behavior influence coefficient BIC is calculated by m3(severely abnormal)=0.

[0146] When 1.5≤BIC<1.8, the probability of the escalator being in a normal state based on the behavior influence coefficient BIC is calculated by m3(normal)=0.1(1.8-BIC) / 0.3, the probability of the escalator being in a slightly abnormal state based on the behavior influence coefficient BIC is calculated by m3(slightly abnormal)=0.2(1.8-BIC) / 0.3, the probability of the escalator being in a moderately abnormal state based on the behavior influence coefficient BIC is calculated by m3(moderately abnormal)=0.5+0.3(BIC-1.5) / 0.3, and the probability of the escalator being in a severely abnormal state based on the behavior influence coefficient BIC is calculated by m3(severely abnormal)=0.2(BIC-1.5) / 0.3.

[0147] When BIC≥1.8, the probability of the escalator being in a normal state based on the behavior influence coefficient BIC is calculated by m3(normal)=0, the probability of the escalator being in a slightly abnormal state based on the behavior influence coefficient BIC is calculated by m3(slightly abnormal)=0.1(2.0-BIC) / 0.2, the probability of the escalator being in a moderately abnormal state based on the behavior influence coefficient BIC is calculated by m3(moderately abnormal)=0.2(2.0-BIC) / 0.2, and the probability of the escalator being in a severely abnormal state based on the behavior influence coefficient BIC is calculated by m3(severely abnormal)=0.7+0.3(BIC-1.8) / 0.2.

[0148] For the trend index TI and trend difference index TDI, first calculate the comprehensive trend index CTI =

[0149] max(|TI| / 3,|TDI| / 0.5), CTI<0.33 is defined as normal, 0.33≤CTI<0.67 is defined as slightly abnormal, 0.67≤CTI<0.9 is defined as moderately abnormal, and CTI≥0.9 is defined as severely abnormal. Based on this:

[0150] When CTI<0.33, the probability of judging that the escalator is in a normal state based on the comprehensive trend indicator CTI is calculated by m4(normal)=0.7+0.3(0.33-CTI) / 0.33, and the probability of judging that the escalator is in a normal state based on m4(slightly abnormal)=

[0151] 0.3(CTI / 0.33) is used to calculate the probability that the escalator is in a slightly abnormal state based on the comprehensive trend indicator CTI. m4 (moderate abnormality) = 0 is used to calculate the probability that the escalator is in a moderately abnormal state based on the comprehensive trend indicator CTI. m4 (severe abnormality) = 0 is used to calculate the probability that the escalator is in a seriously abnormal state based on the comprehensive trend indicator CTI.

[0152] When 0.33≤CTI<0.67, the probability of judging that the escalator is in a normal state based on the comprehensive trend indicator CTI is calculated by m4(normal)=0.3(0.67-CTI) / 0.34, and the probability of judging that the escalator is in a normal state based on m4(slightly abnormal)=0.5+

[0153] The probability of the escalator being in a slightly abnormal state based on the comprehensive trend indicator CTI is calculated by 0.3(CTI-0.33) / 0.34, the probability of the escalator being in a moderately abnormal state based on the comprehensive trend indicator CTI is calculated by m4(moderate abnormality)=0.2(CTI-0.33) / 0.34, and the probability of the escalator being in a seriously abnormal state based on the comprehensive trend indicator CTI is calculated by m4(severe abnormality)=0.

[0154] When 0.67≤CTI<0.9, the probability of the escalator being in a normal state based on the comprehensive trend indicator CTI is calculated by m4(normal)=0.1(0.9-CTI) / 0.23, the probability of the escalator being in a slightly abnormal state based on the comprehensive trend indicator CTI is calculated by m4(slightly abnormal)=0.2(0.9-CTI) / 0.23, the probability of the escalator being in a moderately abnormal state based on the comprehensive trend indicator CTI is calculated by m4(moderately abnormal)=0.5+0.3(CTI-0.67) / 0.23, and the probability of the escalator being in a seriously abnormal state based on the comprehensive trend indicator CTI is calculated by m4(severely abnormal)=0.2(CTI-0.67) / 0.23.

[0155] When CTI≥0.9, the probability of the escalator being in a normal state based on the comprehensive trend indicator CTI is calculated by m4(normal)=0, the probability of the escalator being in a slightly abnormal state based on the comprehensive trend indicator CTI is calculated by m4(slight abnormality)=0.1(1-CTI) / 0.1, the probability of the escalator being in a moderately abnormal state based on the comprehensive trend indicator CTI is calculated by m4(moderate abnormality)=0.2(1-CTI) / 0.1, and the probability of the escalator being in a seriously abnormal state based on the comprehensive trend indicator CTI is calculated by m4(severe abnormality)=0.7+0.3(CTI-0.9) / 0.1.

[0156] Among them, the trend difference index TDI reflects the degree of deviation between the current trend and the trend during historical normal operation. The calculation formula is TDI = (TIcurrent-TInormal) / TInormal, TIcurrent is the trend index currently calculated, and TInormal is the average trend index during historical normal operation. TI focuses on short-term changes, and TDI focuses on the comparison with the historical normal state.

[0157] For the environmental impact coefficient EIC and the seasonal difference index SDI, first calculate the comprehensive environmental index CEI = max(|1-EIC| / 0.6,|SDI| / 0.5), define CEI < 0.33 as normal, 0.33 ≤ CEI < 0.67 as slightly abnormal, 0.67 ≤ CEI < 0.9 as moderately abnormal, and CEI ≥ 0.9 as severely abnormal. Based on this:

[0158] When CEI<0.33, the probability of the escalator being in a normal state based on the comprehensive environmental index CEI is calculated by m5(normal)=0.7+0.3(0.33-CEI) / 0.33, the probability of the escalator being in a slightly abnormal state based on the comprehensive environmental index CEI is calculated by m5(slight abnormality)=0.3(CEI / 0.33), the probability of the escalator being in a moderately abnormal state based on the comprehensive environmental index CEI is calculated by m5(moderate abnormality)=0, and the probability of the escalator being in a seriously abnormal state based on the comprehensive environmental index CEI is calculated by m5(severe abnormality)=0.

[0159] When 0.33≤CEI<0.67, the probability of the escalator being in a normal state based on the comprehensive environmental index CEI is calculated by m5(normal)=0.3(0.67-CEI) / 0.34, the probability of the escalator being in a slightly abnormal state based on the comprehensive environmental index CEI is calculated by m5(slightly abnormal)=0.5+0.3(CEI-0.33) / 0.34, the probability of the escalator being in a moderately abnormal state based on the comprehensive environmental index CEI is calculated by m5(moderately abnormal)=0.2(CEI-0.33) / 0.34, and the probability of the escalator being in a seriously abnormal state based on the comprehensive environmental index CEI is calculated by m5(severely abnormal)=0.

[0160] When 0.67≤CEI<0.9, the probability of the escalator being in a normal state based on the comprehensive environmental index CEI is calculated by m5(normal)=0.1(0.9-CEI) / 0.23, the probability of the escalator being in a slightly abnormal state based on the comprehensive environmental index CEI is calculated by m5(slight abnormality)=0.2(0.9-CEI) / 0.23, the probability of the escalator being in a moderately abnormal state based on the comprehensive environmental index CEI is calculated by m5(moderate abnormality)=0.5+0.3(CEI-0.67) / 0.23, and the probability of the escalator being in a seriously abnormal state based on the comprehensive environmental index CEI is calculated by m5(severe abnormality)=0.2(CEI-0.67) / 0.23.

[0161] When CEI≥0.9, the probability of the escalator being in a normal state based on the comprehensive environmental index CEI is calculated by m5(normal)=0, the probability of the escalator being in a slightly abnormal state based on the comprehensive environmental index CEI is calculated by m5(slight abnormality)=0.1(1-CEI) / 0.1, the probability of the escalator being in a moderately abnormal state based on the comprehensive environmental index CEI is calculated by m5(moderate abnormality)=0.2(1-CEI) / 0.1, and the probability of the escalator being in a seriously abnormal state based on the comprehensive environmental index CEI is calculated by m5(severe abnormality)=0.7+0.3(CEI-0.9) / 0.1.

[0162] Exemplarily, the above-mentioned calculation of the probability of the escalator being in each preset health state according to the adjusted basic probability distribution may include: for a first preset health state, determining all combinations of a first event and a second event, the intersection of the first event and the second event being the first preset health state, and the first preset health state being one of normal, slightly abnormal, moderately abnormal, and severely abnormal; for each combination of the first event and the second event, multiplying the basic probability distribution corresponding to the first event by the basic probability distribution corresponding to the second event, and adding the products corresponding to each combination of the first event and the second event to obtain the probability of the escalator being in the first preset health state.

[0163] Exemplarily, for the probability that the escalator is in event A (such as a minor abnormal state), first traverse all combinations where the intersection of the first event (such as event B) and the second event (such as event C) is a minor abnormality. For example, combinations like B = {normal, minor abnormal} and C = {minor abnormal}, B = {minor abnormal, moderate abnormal} and C = {minor abnormal}, B = {minor abnormal} and C = {minor abnormal}, etc. Then, calculate the product of the basic probability assignment corresponding to the first event and the basic probability assignment corresponding to the second event in each combination, and then sum up the calculated products to obtain the probability that the escalator is in a minor abnormal state.

[0164] For example, the following takes "calculating the probability that the escalator is in a minor abnormal state through two health characteristic indicators, namely the relevant index r of vibration and noise analysis and the temperature anomaly index TAI" as an example for illustration.

[0165] The relevant index r of vibration and noise analysis is 0.75. Since 0.5 < r ≤ 0.8, according to the formula, calculate: m1(normal) = 0.3(1 - 0.75) = 0.075, m1(minor abnormal) = 0.4 + 0.3 * 0.75 = 0.625, m1(moderate abnormal) = 0.2 * 0.75 = 0.15, m1(severe abnormal) = 0.1 * 0.75 = 0.075.

[0166] The temperature anomaly index TAI is 0.45. Since 0.3 ≤ TAI < 0.6, according to the formula, calculate: m2(normal) = 0.3(0.6 - 0.45) / 0.3 = 0.15, m2(minor abnormal) = 0.5 + 0.3(0.45 - 0.3) / 0.3 = 0.65, m2(moderate abnormal) = 0.2(0.45 - 0.3) / 0.3 = 0.2, m2(severe abnormal) = 0.

[0167] Calculate the probability that the escalator is in a minor abnormal state, considering all combinations where the intersection is a minor abnormal state: 1. m1(minor abnormal) * m2(minor abnormal) = 0.625 * 0.65 = 0.40625, 2. m1(minor abnormal) * m2(normal) = 0.625 * 0.15 = 0.09375, 3. m1(normal) * m2(minor abnormal) = 0.075 * 0.65 =

[0168] 0.04875. Summing up the above three products, the probability that the escalator is in a minor abnormal state is mγ(minor abnormal) = 0.40625 + 0.09375 + 0.04875 = 0.54875.

[0169] Based on the above method, the probability that the escalator is in a normal state is mγ(normal) = 0.11, the probability that the escalator is in a slightly abnormal state is mγ(slightly abnormal) = 0.54875, the probability that the escalator is in a moderately abnormal state is mγ(moderately abnormal) = 0.27125, and the probability that the escalator is in a severely abnormal state is mγ(severely abnormal) = 0.07. Since the probability that the escalator is in a slightly abnormal state is the highest, it can be determined that the escalator is most likely in a slightly abnormal state.

[0170] For the case of five health characteristic indicators, based on the above content, we can first calculate the vibration noise analysis related indicators and the temperature anomaly index to calculate the probability of the escalator being in multiple preset health states, and then regard the calculated probability as a new health characteristic indicator and the behavior influence coefficient to calculate the probability of the escalator being in multiple preset health states. The calculation is repeated in this way until all health characteristic indicators are taken into account, and the final probability of the escalator being in multiple preset health states can be obtained.

[0171] In some embodiments, the conflict degree con=K / 5, where 5 represents the number of health characteristic indicators, and K is the conflict coefficient. K represents the product of the basic probability distributions corresponding to all events B and C whose intersections are empty sets. When calculating the K value, the basic probability distributions of the two health characteristic indicators (i.e., which two of m1 to m5) are selected for combination. Generally, they can be selected in the order of m1→m2→m3→m4→m5, or the combination order can be adjusted according to the reliability and importance of m1 to m5.

[0172] In addition, when the intersection of event B and event C is an empty set, event B and event C are usually mutually exclusive. Specifically, whether event B and event C are mutually exclusive can be determined based on the basic mutual exclusion rule, the mutual exclusion of states separated by distance, and the non-mutual exclusion of adjacent states.

[0173] 1. Basic mutual exclusion rules: Examples are given to illustrate that the normal state and the severe abnormal state are mutually exclusive, the normal state and the moderate abnormal state are mutually exclusive, and the mild abnormal state and the severe abnormal state are mutually exclusive.

[0174] 2. Distance-separated states are mutually exclusive: If two health states are separated by more than one health state in {normal, slightly abnormal, moderately abnormal, severely abnormal}, the two health states are mutually exclusive. For example, in {normal, slightly abnormal, moderately abnormal, severely abnormal}, the normal state is mutually exclusive with the moderately abnormal state, the normal state is mutually exclusive with the severely abnormal state, and the slightly abnormal state is mutually exclusive with the severely abnormal state.

[0175] 3. Adjacent states are not mutually exclusive: If two health states are adjacent in {normal, slightly abnormal, moderately abnormal, severely abnormal}, then the two health states are not mutually exclusive. For example, in {normal, slightly abnormal, moderately abnormal, severely abnormal}, the normal state and the slightly abnormal state are not mutually exclusive, the slightly abnormal state and the moderately abnormal state are not mutually exclusive, and the moderately abnormal state and the severely abnormal state are not mutually exclusive.

[0176] The above mutually exclusive rules are based on the continuous characteristics of the escalator state. There may be transitions between adjacent states, but the interval states cannot occur at the same time.

[0177] For example, the conflict coefficient K can also be calculated based on the basic probability distribution of the five health characteristic indicators (i.e., m1-m5). Calculate the conflict coefficient K between m1 and m2 12 , then the conflict degree Con is calculated 12 =K 12 / 5. If the conflict degree Con 12 If it is less than 0.2, you can Calculate m 12 The conflict coefficient K with m3 123 , m 12 (B) = m1(B)*m2(B), then the conflict degree Con is calculated 123 =K 123 / 5. If the conflict degree Con 123 If it is less than 0.2, you can Calculate m 123 The conflict coefficient K with m4 1234 , m 123 (B) = m1(B)*m2(B)*m3(B), then the conflict degree Con is calculated 1234 =K 1234 / 5. If the conflict degree Con 1234 If it is less than 0.2, you can Calculate m 1234 The conflict coefficient K with m5 12345 , m 123 (B) = m1(B)*m2(B)*m3(B)*m4(B), then the conflict degree Con is calculated 12345 =K 12345 / 5. If Con 12 ~Con 12345 If both are less than 0.2, the probability of the escalator being in each preset health state is calculated according to the adjusted basic probability distribution.

[0178] If Con 12 ~Con 12345If a certain degree of conflict is greater than or equal to 0.2, the preset weights of various health characteristic indicators are adjusted according to the average degree of conflict between each basic probability distribution and other basic probability distributions until the degree of conflict is less than the first conflict threshold. For example, the preset weights of health characteristic indicators with a degree of conflict greater than the average degree of conflict are reduced, and the preset weights of health characteristic indicators with a degree of conflict less than the average degree of conflict are increased, and finally the sum of the preset weights of various health characteristic indicators is ensured to be 1. This can reduce the impact of the basic probability distribution of health characteristic indicators that may be unreliable and enhance the role of the basic probability distribution of more reliable health characteristic indicators.

[0179] Step 206, determining the comprehensive health of the escalator according to the probability that the escalator health state is in a plurality of preset health states, and performing safety monitoring on the escalator based on the comprehensive health of the escalator.

[0180] In some embodiments, determining the comprehensive health of the escalator according to the probability of the escalator health state being in multiple preset health states may include:

[0181] according to Calculate the credibility Bel(A) of event A, and calculate the uncertainty interval U(A) of event A according to U(A)=Pl(A)-Bel(A). Event A means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal; event B means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal; m Y (B) represents the probability that the escalator is in event state B;

[0182] The comprehensive health of the escalator is determined based on the three probabilities of the escalator being in a slight abnormality, a moderate abnormality, and a serious abnormality and the credibility Bel(A);

[0183] When the credibility Bel(A) is greater than the credibility threshold and the uncertainty interval U(A) is less than the uncertainty interval threshold, the escalator monitoring plan corresponding to the comprehensive health of the escalator is executed; wherein the escalator monitoring plan includes the escalator being in good condition and requiring no intervention, increasing the inspection frequency, and shutting down for maintenance and overhaul.

[0184] For example, the comprehensive health of an escalator, CHS, can be expressed by the formula

[0185] CHS=100-(30m Y (Minor abnormality)+60m Y (Moderate Abnormal) +90m Y (Severe abnormality)) Calculated, m Y (Minor abnormality) indicates the probability that the escalator is in a minor abnormal state, m Y(Moderate Abnormality) indicates the probability that the escalator is in a moderate abnormal state, m Y (Severe abnormality) indicates the probability that the escalator is in a severe abnormal state, and these probabilities are calculated in step 205.

[0186] In this embodiment, when the credibility Bel(A) is greater than the credibility threshold (for example, 0.7) and the uncertainty interval U(A) is less than the uncertainty interval threshold (for example, 0.2), the escalator monitoring scheme corresponding to the comprehensive health of the escalator can be executed. That is, if CHS≥90, it means that the escalator is in good condition. If 80≤CHS<90, it is necessary to increase the inspection frequency, including mechanical inspection frequency, electrical inspection frequency, sensor inspection frequency and parameter recording frequency. If 70≤CHS<80, it is necessary to arrange for recent maintenance of the escalator. If CHS<70, the escalator can be immediately inspected or shut down.

[0187] In addition, the CHS threshold, Bel(A) threshold, and U(A) threshold can be dynamically adjusted according to the equipment operating conditions, as follows:

[0188] 1. CHS threshold adjustment: adjust the reference value according to the operating conditions: Reference value = standard threshold × (1 +

[0189] k1L+k2T+k3E), L is the load factor (actual load / rated load), T is the operating time factor (current operating time / design life), E is the environmental factor (based on temperature, humidity, etc.), and k1, k2 and k3 are influence coefficients.

[0190] 2. Bel(A) threshold adjustment: The default threshold is 0.7. Under high load, Bel(A) can be increased to 0.75-0.8. Under harsh environment, Bel(A) can be increased to 0.75-0.8. Under normal working conditions, Bel(A) is maintained at 0.7.

[0191] 3. U(A) threshold adjustment: The default threshold is 0.2. Under high load, Bel(A) is reduced to 0.15-0.1. Under harsh environment, Bel(A) is reduced to 0.15-0.1. Under normal working conditions, Bel(A) is maintained at 0.2.

[0192] In some embodiments, when the comprehensive health of the escalator indicates that the escalator is in normal operation, the sensor collects the escalator operation status data at a first sampling frequency and transmits the escalator operation status data at a first transmission period; when the comprehensive health of the escalator indicates that the escalator is abnormal, the sensor collects the escalator operation status data at a second sampling frequency and transmits the escalator operation status data at a second transmission period. The second sampling frequency is greater than the first sampling frequency, and the second transmission period is less than the first transmission period. That is, when the escalator is in normal operation, the sensor reduces the sampling frequency and transmission period to reduce the amount of data and transmission load.

[0193] In some embodiments, when the information entropy of the escalator operation status data increases, the sensor increases the sampling frequency and reduces the transmission period. When adjusting the sensor sampling frequency and transmission period, not only the overall health of the escalator can be considered, but also the information entropy of the data. When the information entropy of the data is high, it indicates that the data contains more useful information. At this time, the sampling frequency can be appropriately increased and the transmission period can be reduced; conversely, when the information entropy of the data is low, it indicates that the data redundancy is high. At this time, the sampling frequency can be reduced and the transmission period can be increased to reduce the amount of data.

[0194] Specifically, the information entropy of the escalator operation status data can be X represents the data collected by the sensor, i represents the i-th value of the data, p(x i ) represents x i Probability of occurrence.

[0195] In addition, when the sensor transmits data, wavelet transform can be used to compress the time series data of the escalator operation status, and principal component analysis can be used to select the most important features for transmission.

[0196] During data transmission, this embodiment introduces data compression and feature selection technology to reduce the amount of data transmitted. Specifically, methods such as wavelet transform can be used to compress the time series data of the escalator operation status to reduce the data length. Afterwards, methods such as principal component analysis (PCA) can be used to select the most important features from the compressed data for transmission to reduce the data dimension. Assume that the original data is X∈R n×m , where n is the number of sampling points and m is the feature dimension. Perform wavelet transform on X to get the compressed data where n c <n. for X c Perform PCA to obtain the data after dimensionality reduction Where m r <m. The final transmitted data X r While retaining key information, the amount of data is greatly reduced, reducing transmission load and storage overhead.

[0197] In some embodiments, a distributed big data platform (such as Hadoop, Spark, etc.) can be used to store and manage the massive data collected by sensors, supporting efficient data reading, writing and query operations. By designing a reasonable data storage format and indexing mechanism, it can meet the real-time writing of data and support fast data retrieval and analysis.

[0198] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0199] (1) By placing multi-parameter sensors on key components of the escalator, real-time data on operating status such as vibration, noise, and temperature are collected, and scientific warning thresholds are set. This enables timely warnings to be issued when equipment fails or becomes abnormal, allowing maintenance personnel to take measures before the problem worsens, thus avoiding accidents and extending the service life of the escalator.

[0200] (2) Computer vision technology is used to intelligently analyze the video images collected by the escalator camera. Target detection, posture recognition and other algorithms are used to monitor abnormal passenger behavior in real time. When abnormal behaviors such as going against the flow, climbing, and falling are detected, an alarm is immediately issued and the escalator is stopped, effectively curbing unsafe behaviors and protecting the personal safety of passengers.

[0201] (3) A cloud-edge collaborative data processing architecture is adopted, with edge nodes responsible for data preprocessing and real-time analysis, and the cloud platform responsible for data aggregation and intelligent modeling. This not only ensures a rapid response to abnormal situations on site, but also realizes the centralized management and mining of massive data, greatly improving the efficiency and level of escalator safety management.

[0202] (4) The escalator health risk is assessed by integrating operating status data and video images containing passenger behavior, achieving a comprehensive assessment of the escalator status and accurate safety monitoring. It has the capabilities of automated data collection, intelligent status assessment, and predictive maintenance decision-making, which can effectively reduce the false alarm rate and missed alarm rate, optimize maintenance strategies, and improve the safety and reliability of escalator equipment operation.

[0203] (5) This scheme adopts a multi-source data fusion method, which has significant advantages over traditional expert systems, fuzzy reasoning and gray systems. Traditional expert systems rely on subjective experience and often use "black or white" binary judgments, which are difficult to handle complex uncertainty problems; fuzzy reasoning methods require manual setting of fuzzy rules and membership functions, and parameter selection is highly subjective; gray systems are limited by limited data samples and lack the ability to process multi-dimensional information.

[0204] (5) This scheme optimizes the traditional DS evidence theory through the improved YAGER rule, effectively solves the problem of handling high-conflict evidence, and avoids the defects of the traditional DS theory in the normalization process. At the same time, a dynamic adaptive mechanism is introduced to continuously optimize the fusion model based on historical data, thereby improving the adaptability and reliability of the system. The multi-level conflict detection and processing mechanism, as well as the complete credibility evaluation system, provide reliable guarantees for system decision-making.

[0205] (6) This solution comprehensively utilizes multiple sources of information such as sensor data, historical operation records, environmental factors, and human behavior characteristics, and realizes a comprehensive assessment of the escalator status through standardized feature extraction and standardized fusion processes. The system has the capabilities of automated data collection, intelligent status assessment, and predictive maintenance decision-making, which can effectively reduce the false alarm rate and missed alarm rate, optimize maintenance strategies, and improve the safety and reliability of equipment operation.

[0206] The multi-feature data fusion of this scheme not only overcomes the limitations of traditional methods, but also provides more comprehensive and reliable decision support through collaborative analysis of multi-source data, providing a scientific basis for the safe operation and preventive maintenance of escalators, and has important theoretical significance and practical application value.

[0207] In summary, the method provided by the present invention can realize comprehensive evaluation and accurate safety monitoring of the escalator status, has the capabilities of automated data collection, intelligent status evaluation and predictive maintenance decision-making, can effectively reduce the false alarm rate and missed alarm rate, optimize maintenance strategies, and improve the operating safety and reliability of escalator equipment, creating a safer, more comfortable and reliable riding environment for passengers, and promoting the development of escalator operation and maintenance management towards intelligence, refinement and specialization, and has broad application prospects.

[0208] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0209] Figure 3 is a schematic diagram of an electronic device provided by an embodiment of the present invention, and the electronic device may be the above-mentioned edge node or a server of a cloud platform. Figure 3 As shown, the electronic device 300 of this embodiment includes a processor 310 and a memory 320 , in which a computer program that can be executed on the processor 310 is stored.

[0210] The electronic device 300 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 310 and a memory 320. Those skilled in the art will appreciate that Figure 3 It is only an example of the electronic device 300 and does not constitute a limitation of the electronic device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0211] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0212] The memory 320 may be an internal storage unit of the electronic device 300, such as a hard disk or memory of the electronic device 300. The memory 320 may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 300. Further, the memory 320 may also include both an internal storage unit of the electronic device 300 and an external storage device. The memory 320 is used to store the computer program and other programs and data required by the electronic device. The memory 320 may also be used to temporarily store data that has been output or is to be output.

[0213] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An escalator safety monitoring system based on multi-feature data fusion decision-making, characterized in that: Including sensors, image acquisition devices, edge nodes and cloud platforms; The sensor is arranged at a key position of the escalator, and is used to collect the running status data of the escalator, wherein the running status data includes the vibration information of the steps, the temperature information of the handrails and the noise information of the driving system, and sends the running status data to the edge node; The image acquisition device is arranged above the steps and the entrance and exit of the escalator, and is used to acquire the video image of the escalator and send it to the edge node; The edge node is used to receive the running status data and the escalator video image, perform threshold detection on the running status data, send abnormal running status data exceeding the threshold range to the cloud platform, and perform passenger behavior detection on the escalator video image, and when it is determined that the passenger has abnormal behavior, send the abnormal behavior information to the cloud platform; The cloud platform is used to determine the health characteristic index of the escalator according to the abnormal operation status data and the abnormal behavior information, calculate the probability of the escalator being in multiple preset health states according to the health characteristic index, and determine the comprehensive health of the escalator according to the probability of the escalator health state being in multiple preset health states, and perform safety monitoring of the escalator according to the comprehensive health of the escalator.

2. An escalator safety monitoring method based on multi-feature data fusion decision-making, characterized in that: include: Acquire the running status data and video images of the escalator, wherein the running status data includes the vibration information of the steps, the temperature information of the handrails and the noise information of the driving system, and the video images of the escalator are acquired by an image acquisition device arranged above the steps and the entrance and exit of the escalator; Performing threshold detection on the operating status data based on spectrum energy and spectrum entropy to determine abnormal operating status data exceeding a threshold range; Performing passenger behavior detection on the escalator video image to determine abnormal behavior information of the passenger in the video image, wherein the abnormal behavior information includes at least one of falling, walking in the opposite direction, and climbing; Determine the health characteristic index of the escalator according to the abnormal operation state data and the abnormal behavior information, wherein the health characteristic index includes a vibration noise related index, a temperature anomaly index, a behavior influence coefficient, a trend index and an environmental influence coefficient; Calculating the probability of the escalator being in a plurality of preset health states according to the health characteristic index, wherein the plurality of preset health states include normal, slightly abnormal, moderately abnormal and severely abnormal, and the escalator health states represented by normal, slightly abnormal, moderately abnormal and severely abnormal are getting worse in sequence; The comprehensive health of the escalator is determined according to the probability that the health state of the escalator is in multiple preset health states, and the escalator is safely monitored based on the comprehensive health of the escalator.

3. The escalator safety monitoring method based on multi-feature data fusion decision according to claim 2 is characterized in that: The determining of the health characteristic index of the escalator according to the abnormal operation status data and the abnormal behavior information includes: The Pearson correlation coefficient of the vibration data sequence and the noise data sequence is calculated by r=cov(V,N) / (std(V)*std(N)) to obtain the vibration-noise correlation index r, where V is the vibration information sequence and N is the noise information sequence; wherein the vibration-noise correlation index r reflects the correlation strength between the vibration information and the noise information; The temperature anomaly index TAI is calculated by TAI=(T-Tref) / Trange, where T is the current temperature of the escalator, Tref is the normal reference temperature of the escalator, and Trange is the normal temperature variation range of the handrail; wherein the temperature anomaly index TAI reflects the degree of abnormality of the escalator temperature; The behavior impact coefficient BIC is calculated by BIC=Σ(Wi×Bi), where Wi is the weight of the i-th abnormal behavior and Bi is the number of occurrences of the abnormal behavior; wherein BIC reflects the comprehensive impact of the passenger's abnormal behavior; pass Calculate the trend index TI, Xt is the current value of the operating status data, is the historical average value of the operating status data, and σ is the standard deviation; where the trend index TI reflects the changing trend of the monitored operating status data; The environmental impact coefficient EIC is calculated by EIC=(w1T+w2H+w3L)×S, where T is the temperature factor, H is the humidity factor, L is the load factor, S is the season factor, w1, w2, and w3 are the weight coefficients of temperature, humidity, and load, respectively, and w1+w2+w3=1; among which, the environmental impact coefficient EIC reflects the degree of influence of environmental conditions on the operation of the escalator.

4. The escalator safety monitoring method based on multi-feature data fusion decision according to claim 3 is characterized in that: The calculating the probability of the escalator being in a plurality of preset health states according to the health characteristic index includes: Determine the basic probability distribution of each health characteristic indicator under various preset health states, and adjust the corresponding basic probability distribution based on the preset weights of various health characteristic indicators; Calculate the conflict degree based on the basic probability distribution of the adjusted health characteristic indicators; When the conflict degree is less than a first conflict degree threshold, the probability of the escalator being in each preset health state is calculated according to the adjusted basic probability distribution; When the conflict degree is greater than or equal to the first conflict degree threshold and less than the second conflict degree threshold, the preset weights of various health characteristic indicators are adjusted according to the average conflict degree between each basic probability distribution and other basic probability distributions until the conflict degree is less than the first conflict degree threshold; When the conflict degree is greater than or equal to a second conflict degree threshold, an escalator abnormality alarm is generated; wherein the second conflict degree threshold is greater than the first conflict degree threshold.

5. The escalator safety monitoring method based on multi-feature data fusion decision according to claim 4 is characterized in that: The determining of the basic probability distribution of each health characteristic index under various preset health states includes: for each preset health state, determining a calculation formula for the basic probability distribution of the health characteristic index under each preset health state based on the value of the health characteristic index, substituting the value of the health characteristic index into the calculation formula to calculate the basic probability distribution of the health characteristic index under each preset health state; The method of calculating the probability of the escalator being in each preset health state according to the adjusted basic probability distribution includes: for a first preset health state, determining all combinations of a first event and a second event, the intersection of the first event and the second event being the first preset health state, and the first preset health state being one of normal, slightly abnormal, moderately abnormal, and severely abnormal; for each combination of the first event and the second event, multiplying the basic probability distribution corresponding to the first event by the basic probability distribution corresponding to the second event, and adding the products corresponding to each combination of the first event and the second event to obtain the probability of the escalator being in the first preset health state.

6. The escalator safety monitoring method based on multi-feature data fusion decision-making according to claim 4 is characterized in that: The method of determining the comprehensive health of the escalator according to the probability that the escalator health state is in a plurality of preset health states includes: according to Calculate the credibility Bel(A) of event A, and calculate the uncertainty interval U(A) of event A according to U(A)=Pl(A)-Bel(A). Event A means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal; and event B means that the escalator is in any of the following states: normal, slightly abnormal, moderately abnormal, or severely abnormal. The comprehensive health of the escalator is determined based on the three probabilities of the escalator being in a slight abnormality, a moderate abnormality, and a serious abnormality and the credibility Bel(A); When the credibility Bel(A) is greater than the credibility threshold and the uncertainty interval U(A) is less than the uncertainty interval threshold, the escalator monitoring plan corresponding to the comprehensive health of the escalator is executed; wherein the escalator monitoring plan includes the escalator being in good condition and requiring no intervention, increasing the inspection frequency, and shutting down for maintenance and overhaul.

7. The escalator safety monitoring method based on multi-feature data fusion decision-making according to claim 2 is characterized in that: The performing threshold detection on the operating status data based on spectrum energy and spectrum entropy to determine abnormal operating status data exceeding the threshold range includes: The running status data is subjected to spectrum analysis by fast Fourier transform, and the spectrum energy and spectrum entropy of the running status data are calculated; wherein, Calculate the spectrum energy E of the operating status data by Calculate the spectrum entropy H of the operating status data, x(t) is the running status data; Comparing the spectrum energy with a preset spectrum energy range, and comparing the spectrum entropy with a preset spectrum entropy range; If the spectrum energy exceeds the preset spectrum energy range, and / or the spectrum entropy exceeds the preset spectrum entropy range, the operating status data is regarded as abnormal operating status data.

8. The escalator safety monitoring method based on multi-feature data fusion decision-making according to claim 7 is characterized in that: Before performing spectrum analysis on the operating status data by fast Fourier transform, performing threshold detection on the operating status data to determine abnormal operating status data exceeding the threshold range further includes: Extracting local features for characterizing the health status of the escalator from the operating status data by means of a convolutional neural network; The local features of various escalator operation status data are adaptively integrated through the attention mechanism to obtain the global health status representation of the escalator. By integrating the local features of various escalator operation status data, the global health status representation of the escalator is obtained; (a) is the global health status representation vector of the escalator, represents the feature vector of the data collected by the i-th sensor, is the attention weight of the i-th sensor, N is the number of sensors, and is the hidden representation of the data collected by the i-th sensor, u (a) is the attention vector, and are the preset parameters for linear transformation; Afterwards, a spectrum analysis is performed on the global health status characterization of the escalator through fast Fourier transform, the spectrum energy and spectrum entropy of the global health status characterization of the escalator are calculated, and the above-mentioned steps of comparing the spectrum energy with the preset spectrum energy range and comparing the spectrum entropy with the preset spectrum entropy range are performed.

9. The escalator safety monitoring method based on multi-feature data fusion decision-making according to claim 2 is characterized in that: The performing passenger behavior detection on the escalator video image to determine abnormal behavior information of passengers in the video image includes: Inputting the escalator video image into a convolutional neural network to generate a feature map, and performing human body detection based on the feature map; After detecting the human body, determining the position coordinates of multiple key points of the human body; Mapping the position coordinates of the multiple key points to the escalator three-dimensional space coordinate system to obtain the position and posture of the passenger on the escalator; Determining abnormal behavior information of the passenger in the video image according to the position and posture; The determining of abnormal behavior information of the passenger in the video image according to the position and posture includes: calculating the moving speed and average speed of the passenger based on the position and posture, and determining whether the passenger has abnormal retrograde behavior according to the average speed; determining the hand key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal climbing behavior according to the coordinate values ​​of the hand key points and the foot key points on the vertical coordinate axis; determining the head key points and the foot key points among the multiple key points, and determining whether the passenger has abnormal falling behavior according to the coordinate values ​​of the head key points and the foot key points on the vertical coordinate axis; Alternatively, determining the abnormal behavior information of a passenger in a video image based on the position and posture includes: constructing multiple vector representations between adjacent key points, where the two key points corresponding to each vector representation are key points of the same limb part of the human body; if the angle change between two vector representations is greater than a threshold, determining the passenger's abnormal behavior based on the angles before and after the two vectors change and a preset abnormal behavior vector angle set.

10. The escalator safety monitoring method based on multi-feature data fusion decision-making according to claim 9 is characterized in that: After performing passenger behavior detection on the escalator video image and determining abnormal behavior information of passengers in the video image, the method further includes: Determine the area where the passenger is located; wherein the area includes the first area, the second area and the third area of ​​the escalator, and the first area, the second area and the third area of ​​the escalator are sequentially connected and include the entire escalator; Generate an escalator control strategy based on the abnormal behavior, the area where the passengers are located, and the distribution of passengers on the escalator; Among them, the escalator control strategy includes: If the abnormal behavior is climbing, the escalator will be controlled to slow down and issue a warning. If the climbing behavior lasts longer than the preset time, the escalator will be controlled to temporarily stop running. When the abnormal behavior is falling, if there are other passengers in the adjacent area behind the area where the passenger who fell is located and the passenger density is greater than the density threshold, or the current time is within the preset time period, the escalator is controlled to stop running; otherwise, the escalator is controlled to slow down and issue a voice warning; wherein, the area behind the area is determined based on the escalator running direction, and the opposite direction of the escalator running direction is the back.

Images