Abnormal behavior detection system and method in elevator car

By combining RFID and sensors, a feature matrix is ​​generated and an abnormal behavior detection is detected in the elevator car using deep learning model, which solves the problems of unsatisfactory detection results and insufficient real-time performance in the existing technology, and achieves more accurate and robust abnormal behaviors judge.

CN118619037BActive Publication Date: 2024-11-22CHANGSHU INSTITUTE OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411093330.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

The existing abnormal behavior detection system in the elevator car relies on camera to collect data and is easily affected by ambient light, camera angle and occlusion, resulting in unsatisfactory detection results, especially in complex environments, insufficient real-time performance and high false alarm rate.

Method used

Using the method of combining RFID and sensors, the position monitoring module, data acquisition module and feature fusion module generates the feature matrix of position changes, temperature, pressure and sound, and uses deep learning network to build an abnormal behavior judgment model, and comprehensive judgment is made based on attention weights. .

Benefits of technology

It improves the accuracy and robustness of detection, reduces the impact on light changes and occlusions, does not rely on visual information, and meets the immediate response needs of elevator detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118619037B_ABST
    Figure CN118619037B_ABST
Patent Text Reader

Abstract

This invention provides an elevator car abnormal behavior detection system and method, belonging to the field of cross-modal fusion technology. The invention achieves multimodal detection of personnel and environmental conditions within the elevator by collecting and generating data on the position change rate of RFID tags carried by passengers, temperature change data inside the elevator, and abnormal pressure conditions inside the car. Through feature fusion and deep learning networks, the real-time collected position, temperature, pressure, and sound data are converted into corresponding feature matrices. Attention weights are calculated using a cross-modal attention mechanism to generate a fusion matrix. A trained abnormal behavior judgment model is then used to determine in real time whether abnormal behavior exists inside the elevator car, thereby improving the accuracy and comprehensiveness of elevator safety monitoring and enhancing passenger safety.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of cross-modal fusion, and in particular to a system and method for detecting abnormal behavior in an elevator car. Background Art

[0002] With the continuous advancement of urbanization, elevators have become an indispensable vertical transportation tool in high-rise buildings. At the same time, the safety issues inside elevator cars are increasingly attracting people's attention. Traditional elevator safety monitoring systems are mainly based on video monitoring, which detects abnormal behavior by capturing and analyzing images inside the elevator through monitoring cameras. However, this method is limited by light and viewing angle, and is easily affected by factors such as changes in ambient light, obstructions, and image noise, resulting in unsatisfactory detection results. In addition, a single video monitoring method has limitations in dealing with complex abnormal behaviors. Nowadays, RFID and sensor technology are relatively mature. RFID communicates through radio waves, can read tag information without physical contact, and can read multiple tag information at the same time, which is very suitable for environments such as elevators with closed spaces and multiple passengers; sensors can cover multiple modal data, such as temperature, pressure, light, etc. in the car, ensuring the comprehensiveness of detection. Combining the two, a more accurate and comprehensive abnormal behavior detection system inside the elevator car can be developed.

[0003] In the prior art, the publication number CN111539265B discloses a method for detecting abnormal behavior in an elevator car, which preprocesses the original image of the elevator monitoring video, and screens the preprocessed video. The optical flow method is used to segment the video segments where people, pets and electric vehicles appear. The screened video segments are input into the trained yolov3 model for recognition, and the number of people and pets in the video is obtained, and it is judged whether an electric vehicle appears. The energy value of each frame of the image is calculated based on the motion history graph of each frame in the screened video segment, and an energy threshold is preset. Whether abnormal behavior occurs is judged based on the energy value and the energy threshold.

[0004] The main problems with the above methods are: relying on cameras to collect data, the data is affected by factors such as ambient light, camera angle, and obstructions in the elevator, resulting in inaccurate image data. The optical flow method has reduced accuracy when processing fast-moving or drastically changing images, and is prone to errors in the complex environment of the elevator. In addition, the elevator detection is long-term and continuous, and the optical flow method has a large amount of calculation, which may lead to insufficient real-time performance. In addition, the energy value judgment is not accurate enough when judging, which is prone to missed reports and false alarms. For example, normal heavy lifting may be mistaken for abnormality.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The object of the present invention is to provide a system and method for detecting abnormal behavior in an elevator car to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] An abnormal behavior detection system in an elevator car, specifically comprising:

[0009] The position monitoring module is used to set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate;

[0010] The data acquisition module is used to monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature characteristic vector according to the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure characteristic vector, monitor the sound loudness in the car, and generate a sound characteristic vector;

[0011] The feature fusion module is used to reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. The four feature matrices are combined in pairs to generate a total of six combinations. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices.

[0012] A model building module is used to build a deep learning network. After marking the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, it is input into the deep learning network for training to obtain an elevator car abnormal behavior judgment model whose input value is the fusion feature matrix and the output value is the type of abnormal behavior in the elevator car;

[0013] The comprehensive judgment module is used to generate a fusion feature matrix in real time according to the processing process of the position detection module, data acquisition module and feature fusion module based on the real-time RFID tag position, real-time temperature, real-time pressure, real-time light and real-time sound of the elevator car to be identified, and then input the fusion feature matrix into the elevator car abnormal behavior judgment model to determine whether there is abnormal behavior in the current elevator car.

[0014] Furthermore, when setting the reference position of the RFID tag in the elevator, an RFID tag is installed at the center point of the ceiling inside the elevator car as the reference position of the RFID tag, and an RFID reader / writer that can cover the entire space of the car is installed inside the elevator. The frequency offset of the position of the RFID tag carried by the passenger relative to the reference position of the RFID tag is collected by the RFID reader / writer, and the position change rate of the RFID tag carried by the passenger is generated based on the Doppler effect.

[0015] Furthermore, the principle for generating the position change feature vector is:

[0016] The formula for generating the rate of change of the position of the RFID tag is:

[0017] △x(t i )=x(t i )-x0

[0018] △y(t i )=y(t i )-y0

[0019] △z(t i )=z(t i )-z0

[0020] △t i =t i -t i-1

[0021]

[0022] Among them, △x(t i ) represents the horizontal deviation of the RFID tag position read at time i relative to the reference position, x(t i ) represents the horizontal coordinate of the RFID tag position read at time i, x0 represents the horizontal coordinate of the reference position, △y(t i ) represents the vertical deviation of the RFID tag position read at time i relative to the reference position, y(t i ) represents the vertical coordinate of the RFID tag position read at time i, y0 represents the vertical coordinate of the reference position, △z(t i ) represents the depth deviation of the RFID tag position read at time i relative to the reference position, z(t i ) represents the depth coordinate of the RFID tag position read at time i, z0 represents the depth coordinate of the reference position, △t i Represents the time difference, t i represents the time at time i, t i-1 represents the time at time i-1, △v x (t i) represents the rate of change of the horizontal position at the i-th moment, △v y (t i ) represents the vertical position change rate at the i-th moment, △v z (t i ) represents the depth position change rate at the i-th moment, D i represents the position change rate of the RFID tag read at the i-th moment, i represents the index of the read RFID tag, and i=1, 2, ..., I, I represents the number of times the RFID tag is read;

[0023] The position change feature vector generated by the position change rate is:

[0024] l=(D2、D3、......、D I )

[0025] Among them, l represents the position change feature vector.

[0026] Furthermore, the principle for generating the temperature feature vector is:

[0027] First, based on the collected car internal temperature, the standard temperature change rate is generated. The formula is:

[0028]

[0029] Among them, μ T represents the mean temperature, n represents the number of temperature acquisitions, T j represents the temperature data collected for the jth time, σ T represents the standard deviation of temperature, Z Tj represents the standard temperature change rate at the jth temperature collection time, j represents the temperature index, and j=1, 2, ..., n;

[0030] The temperature change characteristic value is generated by combining the mean and standard deviation of the standard temperature change rate, based on the formula:

[0031]

[0032] in, represents the mean value of the standard temperature change rate, Indicates the standard deviation of the standard temperature change rate, F j Indicates the characteristic value of temperature change;

[0033] The temperature eigenvector generated by the temperature change eigenvalue is:

[0034] u=(F1、F2、……、F n )

[0035] Where u represents the temperature eigenvector.

[0036] Furthermore, the principle for generating the pressure characteristic vector is:

[0037] First, the standard pressure change rate is generated based on the principle:

[0038]

[0039] Among them, △P(k) represents the standard pressure change rate at the kth pressure collection, P(k) represents the pressure value collected at time k, P(k-1) represents the pressure value collected at time k-1, △t k Indicates the collection time interval;

[0040] The mean and standard deviation of the standard pressure change rate are generated based on the principle that:

[0041]

[0042] Among them, μ △P represents the mean value of the standard pressure change rate, m represents the number of times the standard pressure change rate is generated, △P k represents the standard pressure change rate generated for the kth time, k represents the index of the standard pressure change rate, and k = 1, 2, ..., m, σ △P It represents the standard deviation of the standard pressure change rate;

[0043] The formula for generating the pressure change signature based on the mean and standard deviation of the standard pressure change rate is:

[0044]

[0045] Among them, R k Indicates the characteristics of pressure changes;

[0046] The pressure feature vector generated by the pressure change feature is:

[0047] v=(R1、R2、……、R m )

[0048] Where v represents the pressure eigenvector.

[0049] Furthermore, the principle for generating the sound feature vector is:

[0050] Set a time window and calculate the short-time energy of the sound based on the formula:

[0051]

[0052] Wherein, w(q) represents the Hamming window function at the qth time, q represents the time index of the Hamming window function, N represents the window length, E(p) represents the short-time energy of the sound at the pth moment, p represents the time index of the short-time energy of the sound, and p=1, 2, ..., N-1, x represents the amplitude of the sound signal, and x(pq) represents the amplitude of the collected sound signal at the pq moment;

[0053] Taking the short-time energy of sound as the loudness feature of the sound, the generated sound feature vector is:

[0054] y=(E(1),E(2),...,E(N-1))

[0055] Where y represents the sound feature vector.

[0056] Furthermore, the principle for generating the fusion feature matrix is:

[0057] The position change feature matrix generated according to the position change feature vector is:

[0058] L=[D2、D3、……、D I ]

[0059] Where L represents the position change feature matrix;

[0060] The temperature feature matrix generated according to the temperature feature vector is:

[0061] U=[F1、F2、……、F n ]

[0062] Where U represents the temperature characteristic matrix;

[0063] The pressure feature matrix generated according to the pressure feature vector is:

[0064] V=[R1、R2、……、R m ]

[0065] Where V represents the pressure feature matrix;

[0066] The sound feature matrix generated according to the sound feature vector is:

[0067] Y=[E(1),E(2),…,E(N-1)]

[0068] Where Y represents the sound feature matrix;

[0069] The principle for calculating the correlation matrix and generating the attention weight for each combination is as follows:

[0070] Taking the position change feature matrix and the temperature feature matrix as examples, the same number of elements are intercepted in the matrix to generate the similarity between the two. The formula is:

[0071] S a,b =L a ·U b

[0072] Among them, S a,b Indicates the similarity between the elements of the corresponding position change feature matrix and the temperature feature matrix, L a represents the element of the ath column of the position change feature matrix, a represents the column index of the element in the position change feature matrix, and a=1, 2, ..., I, U b represents the element of the b-th row of the temperature feature matrix, b represents the element row number index in the temperature feature matrix, and b=1, 2, ..., n;

[0073] All calculated similarity values ​​are arranged in sequence to form a matrix S, which represents the correlation matrix between the position change feature matrix and the temperature feature matrix. a,b represents the element in the ath row and bth column of the correlation matrix S;

[0074] The attention weights are generated based on the correlation matrix according to the formula:

[0075]

[0076] in, represents the attention weight of the position change feature matrix in the e1 group, represents the attention weight of the temperature feature matrix in the e1 group, e1 represents the index of the combination of the position change feature matrix and the temperature feature matrix, S represents the correlation matrix, S T represents the transposed matrix of the correlation matrix;

[0077] After combining the four feature matrices in pairs, each feature matrix participates in three combinations and has three group attention weights. Combining the group attention weights, the formula for generating the corresponding total attention weight is:

[0078]

[0079] Wherein, B(L) represents the attention weight of the position change feature matrix, B(U) represents the attention weight of the temperature feature matrix, B(V) represents the attention weight of the pressure feature matrix, B(Y) represents the attention weight of the sound feature matrix, e represents the index of the combination including a certain feature matrix, and a certain feature matrix is ​​a position change feature matrix or a temperature feature matrix or a pressure feature matrix or a sound feature matrix, and e=1, 2, 3;

[0080] The feature fusion is performed by calculating the generated attention weights to generate a fusion feature matrix based on the formula:

[0081] K=B(L)·L+B(U)·U+B(V)·V+B(Y)·Y

[0082] Among them, K represents the fusion weight matrix.

[0083] The present invention also provides a method for detecting abnormal behavior in an elevator car, which is performed by the above-mentioned abnormal behavior detection system in the elevator car, and the specific steps include:

[0084] Step 1: Set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate;

[0085] Step 2: Monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature feature vector based on the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure feature vector, monitor the sound loudness in the car, and generate a sound feature vector;

[0086] Step 3: Reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. Four feature matrices are combined in pairs to generate six combinations in total. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices.

[0087] Step 4: Construct a deep learning network, mark the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, and then input it into the deep learning network for training to obtain an elevator car abnormal behavior judgment model with the input value being the fusion feature matrix and the output value being the type of abnormal behavior in the elevator car;

[0088] Step 5: The real-time RFID tag position, real-time temperature, real-time pressure, real-time light and real-time sound of the elevator car to be identified are used to generate a fusion feature matrix in real time according to the above steps, and then the fusion feature matrix is ​​input into the elevator car abnormal behavior judgment model to determine whether there is abnormal behavior in the current elevator car.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] The present invention judges abnormal behavior from two major directions and four detection parameters. The major direction includes reading the RFID tags carried by passengers and the sensors inside the elevator. The specific detection parameters include the RFID tag position, temperature, pressure and sound. Through the fusion of multiple data, more comprehensive information can be provided, and the accuracy and robustness of detection can be increased. It is independent of visual information and is not affected by factors such as lighting changes and obstructions. It is a significant change compared to the common method of collecting data through cameras.

[0091] The present invention also generates a feature vector for the detection parameters, which is more concise than the original data and extracts the key features in the data, and further generates a feature matrix, calculates the attention weights in pairs, simplifies the calculation process, and inputs the generated feature vector into the trained judgment model to meet the needs of instant response, so that the entire system can better adapt to the instant elevator detection process. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 This is a schematic diagram of the system modules of an embodiment of the present invention;

[0093] Figure 2 The figure is a schematic diagram of the method flow of an embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0095] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0096] Example:

[0097] See also Figure 1 , the present invention provides a technical solution:

[0098] An abnormal behavior detection system in an elevator car, specifically comprising:

[0099] The position monitoring module is used to set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate;

[0100] In this embodiment, when setting the reference position of the RFID tag in the elevator, an RFID tag is installed at the center point of the ceiling inside the elevator car as the reference position of the RFID tag, and an RFID reader / writer that can cover the entire space of the car is installed inside the elevator. The frequency offset of the position of the RFID tag carried by the passenger relative to the reference position of the RFID tag is collected by the RFID reader / writer, and the position change rate of the RFID tag carried by the passenger is generated based on the Doppler effect.

[0101] In the RFID system, the Doppler effect will affect the communication between the tag and the reader. When the RFID tag moves relative to the reader, the frequency of the transmitted radio wave will shift. If the tag moves toward the reader, the received signal frequency will increase. If the tag moves away from the reader, the received signal frequency will decrease. The movement path of the RFID tag can be determined based on the change in signal frequency.

[0102] In this embodiment, the principle for generating the position change feature vector is:

[0103] The formula for generating the rate of change of the position of the RFID tag is:

[0104] △x(t i )=x(t i )-x0

[0105] △y(t i )=y(t i )-y0

[0106] △z(t i )=z(t i )-z0

[0107] △t i =t i -t i-1

[0108]

[0109]

[0110] Among them, △x(t i ) represents the horizontal deviation of the RFID tag position read at time i relative to the reference position, x(t i) represents the horizontal coordinate of the RFID tag position read at time i, x0 represents the horizontal coordinate of the reference position, △y(t i ) represents the vertical deviation of the RFID tag position read at time i relative to the reference position, y(t i ) represents the vertical coordinate of the RFID tag position read at time i, y0 represents the vertical coordinate of the reference position, △z(t i ) represents the depth deviation of the RFID tag position read at time i relative to the reference position, z(t i ) represents the depth coordinate of the RFID tag position read at time i, z0 represents the depth coordinate of the reference position, △t i Represents the time difference, t i represents the time at time i, t i-1 represents the time at time i-1, △v x (t i ) represents the rate of change of the horizontal position at the i-th moment, △v y (t i ) represents the vertical position change rate at the i-th moment, △v z (t i ) represents the depth position change rate at the i-th moment, D i represents the position change rate of the RFID tag read at the i-th moment, i represents the index of the read RFID tag, and i=1, 2, ..., I, I represents the number of times the RFID tag is read;

[0111] The position change rate reflects the amplitude and frequency of the passenger's movement in the elevator car. The higher the frequency and amplitude of the passenger's movement in the elevator, the more violent the movement amplitude, and the greater the corresponding position change rate. If the rate is close to zero, it means that the passenger is basically stationary, which is a normal elevator riding behavior. If the rate is moderate and stable, it means that the passenger has normal movement in the elevator. If the rate suddenly increases, it means that the passenger may fall, run, or other abnormal behaviors. The position change rate is proportional to the length of the path the passenger moves per unit time;

[0112] The position change feature vector generated by the position change rate is:

[0113] l=(D2、D3、……、D I )

[0114] Among them, l represents the position change feature vector.

[0115] In this embodiment, the method can only be used for the case where passengers carrying devices with RFID tags enter the car. Passengers who do not carry corresponding devices cannot be detected. Abnormal behavior detection can be performed through the sensors mentioned below.

[0116] The data acquisition module is used to monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature characteristic vector according to the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure characteristic vector, monitor the sound loudness in the car, and generate a sound characteristic vector;

[0117] In this embodiment, the principle for generating the temperature characteristic vector is:

[0118] First, based on the collected car internal temperature, the standard temperature change rate is generated. The formula is:

[0119]

[0120]

[0121] Among them, μ T represents the mean temperature, n represents the number of temperature acquisitions, T j represents the temperature data collected for the jth time, σ T represents the standard deviation of temperature, Z Tj represents the standard temperature change rate at the jth temperature collection time, j represents the temperature index, and j=1, 2, ..., n;

[0122] The standard temperature change rate reflects the change of temperature in unit time. The smaller the temperature fluctuation, the more stable the temperature and the smaller the standard temperature change rate. The standard temperature change rate is inversely proportional to the standard deviation of the temperature and directly proportional to the temperature difference between the current temperature and the average temperature.

[0123] The temperature change characteristic value is generated by combining the mean and standard deviation of the standard temperature change rate, based on the formula:

[0124]

[0125] in, represents the mean value of the standard temperature change rate, Indicates the standard deviation of the standard temperature change rate, F j Indicates the characteristic value of temperature change;

[0126] The temperature variation coefficient is used as the temperature variation characteristic value to reflect the temperature variation characteristics and the temperature dispersion. The greater the temperature dispersion, the greater the temperature fluctuation and the greater the possibility of abnormality in the car. The temperature variation characteristic value is inversely proportional to the mean of the standard temperature change rate and is proportional to the standard deviation of the standard temperature change rate.

[0127] The temperature eigenvector generated by the temperature change eigenvalue is:

[0128] u=(F1、F2、……、F n )

[0129] Where u represents the temperature eigenvector.

[0130] In this embodiment, the principle for generating the pressure characteristic vector is:

[0131] First, the standard pressure change rate is generated based on the principle:

[0132]

[0133] Among them, △P(k) represents the standard pressure change rate at the kth pressure collection, P(k) represents the pressure value collected at time k, P(k-1) represents the pressure value collected at time k-1, △t k Indicates the collection time interval;

[0134] The mean and standard deviation of the standard pressure change rate are generated based on the principle that:

[0135]

[0136] Among them, μ △P represents the mean value of the standard pressure change rate, m represents the number of times the standard pressure change rate is generated, △P k represents the standard pressure change rate generated for the kth time, k represents the index of the standard pressure change rate, and k = 1, 2, ..., m, σ △P It represents the standard deviation of the standard pressure change rate;

[0137] The formula for generating the pressure change signature based on the mean and standard deviation of the standard pressure change rate is:

[0138]

[0139] Among them, R k Indicates pressure change characteristics;

[0140] The pressure change characteristic reflects the load change in the car. When people enter and exit the elevator car or there are people moving inside the elevator car, the pressure will change. Stable pressure changes mean that the elevator is operating normally, and irregular pressure fluctuations indicate that there may be abnormal human activities in the car. The pressure change characteristic is proportional to the standard deviation of the pressure change rate and inversely proportional to the mean of the pressure change rate.

[0141] The pressure feature vector generated by the pressure change feature is:

[0142] v=(R1、R2、……、R m )

[0143] Where v represents the pressure eigenvector.

[0144] In this embodiment, the principle for generating the sound feature vector is:

[0145] Set a time window and calculate the short-time energy of the sound based on the formula:

[0146]

[0147] Wherein, w(q) represents the Hamming window function at the qth time, q represents the time index of the Hamming window function, N represents the window length, E(p) represents the short-time energy of the sound at the pth moment, p represents the time index of the short-time energy of the sound, and p=1, 2, ..., N-1, x represents the amplitude of the sound signal, and x(pq) represents the amplitude of the collected sound signal at the pq moment;

[0148] The short-time energy of sound is used as the loudness feature of sound. The loudness feature of sound reflects the size of the sound loudness in the elevator. During normal operation, the sound loudness in the elevator is maintained at a moderate and stable value. When the sound loudness changes suddenly, it may be due to elevator failure, quarrels between people or collisions between objects. The short-time energy of sound is proportional to the size of the sound signal.

[0149] The generated sound feature vector is:

[0150] y=(E(1),E(2),...,E(N-1))

[0151] Where y represents the sound feature vector.

[0152] The feature fusion module is used to reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. The four feature matrices are combined in pairs to generate a total of six combinations. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices.

[0153] In this embodiment, the position change feature matrix generated according to the position change feature vector is:

[0154] L=[D2、D3、……、D I ]

[0155] Where L represents the position change feature matrix;

[0156] The temperature feature matrix generated according to the temperature feature vector is:

[0157] U=[F1、F2、……、F n ]

[0158] Where U represents the temperature characteristic matrix;

[0159] The pressure feature matrix generated according to the pressure feature vector is:

[0160] V=[R1、R2、……、R m ]

[0161] Where V represents the pressure characteristic matrix;

[0162] The sound feature matrix generated according to the sound feature vector is:

[0163] Y=[E(1),E(2),…,E(N-1)]

[0164] Where Y represents the sound feature matrix;

[0165] The principle for calculating the correlation matrix and generating the attention weight for each combination is as follows:

[0166] Taking the position change feature matrix and the temperature feature matrix as examples, the same number of elements are intercepted in the matrix to generate the similarity between the two. The formula is:

[0167] S a,b =L a ·U b

[0168] Among them, S a,b Indicates the similarity between the elements of the corresponding position change feature matrix and the temperature feature matrix, L a represents the element of the ath column of the position change feature matrix, a represents the column index of the element in the position change feature matrix, and o = 1, 2, ..., I, U b represents the element of the b-th row of the temperature feature matrix, b represents the element row number index in the temperature feature matrix, and b=1, 2, ..., n;

[0169] All calculated similarity values ​​are arranged in sequence to form a matrix S, which represents the correlation matrix between the position change feature matrix and the temperature feature matrix. a,b represents the element in the ath row and bth column of the correlation matrix S;

[0170] The attention weights are generated based on the correlation matrix according to the formula:

[0171]

[0172] in, represents the attention weight of the position change feature matrix in the e1 group, represents the attention weight of the temperature feature matrix in the e1 group, e1 represents the index of the combination of the position change feature matrix and the temperature feature matrix, S represents the correlation matrix, S Trepresents the transposed matrix of the correlation matrix;

[0173] The purpose of calculating the attention weight through the softmax function is to convert the original relevance score into a probability value ranging from 0 to 1, and to ensure that the sum of all attention weights is 1, so as to ensure that the model reasonably distributes weights between different modalities and avoid the situation where the features of a certain modality are too prominent or ignored.

[0174] After combining the four feature matrices in pairs, each feature matrix participates in three combinations and has three group attention weights. Combining the group attention weights, the formula for generating the corresponding total attention weight is:

[0175]

[0176] Wherein, B(L) represents the attention weight of the position change feature matrix, B(U) represents the attention weight of the temperature feature matrix, B(V) represents the attention weight of the pressure feature matrix, B(Y) represents the attention weight of the sound feature matrix, e represents the index of the combination including a certain feature matrix, and a certain feature matrix is ​​a position change feature matrix or a temperature feature matrix or a pressure feature matrix or a sound feature matrix, and e=1, 2, 3;

[0177] The feature fusion is performed by calculating the generated attention weights to generate a fusion feature matrix based on the formula:

[0178] K=B(L)·L+B(U)·U+B(V)·V+B(Y)·Y

[0179] Among them, K represents the fusion weight matrix;

[0180] The fusion weight matrix is ​​a fusion of the position change feature matrix, temperature feature matrix, pressure feature matrix and sound feature matrix to generate a comprehensive feature representation with richer information, reflecting the correlation between each mode and the joint effect of abnormal behavior detection in the elevator. The fusion weight matrix can reflect the changes in the four feature matrices, thereby determining whether abnormal behavior occurs in the elevator car.

[0181] In this embodiment, if the passenger is not detected to have an RFID tag, the weight of the position change fusion matrix is ​​0.

[0182] A model building module is used to build a deep learning network. After marking the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, it is input into the deep learning network for training to obtain an elevator car abnormal behavior judgment model whose input value is the fusion feature matrix and the output value is the type of abnormal behavior in the elevator car;

[0183] The comprehensive judgment module is used to generate a fusion feature matrix in real time according to the processing process of the position detection module, data acquisition module and feature fusion module based on the real-time RFID tag position, real-time temperature, real-time pressure and real-time sound of the elevator car to be identified, and then input the fusion feature matrix into the elevator car abnormal behavior judgment model to determine whether there is abnormal behavior in the current elevator car.

[0184] In this embodiment, the input features of the deep learning network include: a position change feature matrix, a temperature feature matrix, a pressure feature matrix, and a sound feature matrix;

[0185] The structure of the deep learning network is:

[0186] Input layer: used to accept fusion feature matrix input;

[0187] The first hidden layer: contains 128 neurons and uses the ReLU activation function;

[0188] The second hidden layer: contains 64 neurons and uses the ReLU activation function;

[0189] Input layer: Contains 8 neurons and uses the sigmoid activation function to determine whether an anomaly is detected based on the feature matrix.

[0190] In this embodiment, the fusion feature matrix is ​​input into the elevator car abnormal behavior judgment model to judge whether there is abnormal behavior in the current elevator car based on the principle of:

[0191] During the model training process, the data received by the model includes the fusion feature matrix under normal behavior and the fusion feature matrix under abnormal behavior. By analyzing a large amount of historical data, the model establishes the boundary between normal behavior and abnormal behavior, and uses the boundary as the fusion matrix threshold. When it is actually applied, the Euclidean distance between the current input fusion feature matrix and the fusion matrix threshold is calculated to determine whether the current input fusion feature matrix is ​​abnormal. The formula is:

[0192]

[0193] Among them, f represents the fusion matrix threshold, g represents the current input fusion feature matrix, d(f, g) represents the Euclidean distance between matrices f and g, c represents the index of the element in the matrix, n1×n2 represents the dimension of the matrix, and f c represents the cth element of the fusion matrix threshold, g c Represents the cth element of the current input fusion feature matrix;

[0194] When d(f, g) ≥ d0, it means that there is a large gap between the current input fusion matrix and the fusion matrix threshold, which can be judged as abnormal behavior;

[0195] When 0≤d(f, g)<d0, it means that the current input fusion matrix is ​​close to the fusion matrix threshold, and it can be judged that there is no abnormal behavior;

[0196] Among them, d0 represents the Euclidean distance threshold, which is used to judge the difference between the input matrix and the matrix threshold. The smaller the Euclidean distance, the smaller the difference between the matrices. The specific Euclidean distance threshold value is analyzed and obtained in actual applications.

[0197] See also Figure 2 The present invention also provides a method for detecting abnormal behavior in an elevator car. The method is performed by the above-mentioned abnormal behavior detection system in the elevator car, and the specific steps include:

[0198] Step 1: Set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate;

[0199] Step 2: Monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature feature vector based on the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure feature vector, monitor the sound loudness in the car, and generate a sound feature vector;

[0200] Step 3: Reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. Four feature matrices are combined in pairs to generate six combinations in total. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices.

[0201] Step 4: Construct a deep learning network, mark the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, and then input it into the deep learning network for training to obtain an elevator car abnormal behavior judgment model with the input value being the fusion feature matrix and the output value being the type of abnormal behavior in the elevator car;

[0202] Step 5: The real-time RFID tag position, real-time temperature, real-time pressure, real-time light and real-time sound of the elevator car to be identified are used to generate a fusion feature matrix in real time according to the above steps, and then the fusion feature matrix is ​​input into the elevator car abnormal behavior judgment model to determine whether there is abnormal behavior in the current elevator car.

[0203] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0204] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An abnormal behavior detection system in an elevator car, characterized in that: Specifically include: The position monitoring module is used to set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate; The data acquisition module is used to monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature characteristic vector according to the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure characteristic vector, monitor the sound loudness in the car, and generate a sound characteristic vector; The feature fusion module is used to reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. The four feature matrices are combined in pairs to generate a total of six combinations. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices. A model building module is used to build a deep learning network. After marking the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, it is input into the deep learning network for training to obtain an elevator car abnormal behavior judgment model whose input value is the fusion feature matrix and the output value is the type of abnormal behavior in the elevator car; The comprehensive judgment module is used to generate a fusion feature matrix in real time according to the processing process of the position detection module, the data acquisition module and the feature fusion module based on the real-time RFID tag position, real-time temperature, real-time pressure, real-time light and real-time sound of the elevator car to be identified, and then input the fusion feature matrix into the elevator car abnormal behavior judgment model to judge whether there is abnormal behavior in the current elevator car; The principle for generating position change feature vectors is: The formula for generating the rate of change of the position of the RFID tag is: △x(t i )=x(t i )-x0 △y(t i )=y(t i )-y0 △z(t i )=z(t i )-z0 △t i =t i -t i-1 Among them, △x(t i ) represents the horizontal deviation of the RFID tag position read at time i relative to the reference position, x(t i ) represents the horizontal coordinate of the RFID tag position read at time i, x0 represents the horizontal coordinate of the reference position, △y(t i ) represents the vertical deviation of the RFID tag position read at time i relative to the reference position, y(t i ) represents the vertical coordinate of the RFID tag position read at time i, y0 represents the vertical coordinate of the reference position, △z(t i ) represents the depth deviation of the RFID tag position read at time i relative to the reference position, z(t i ) represents the depth coordinate of the RFID tag position read at time i, z0 represents the depth coordinate of the reference position, △t i Represents the time difference, t i represents the time at time i, t i-1 represents the time at time i-1, △v x (t i ) represents the rate of change of the horizontal position at the i-th moment, △v y (t i ) represents the vertical position change rate at the i-th moment, △v z (t i ) represents the depth position change rate at the i-th moment, D i represents the position change rate of the RFID tag read at the i-th moment, i represents the index of the read RFID tag, and i=1, 2, ..., I, I represents the number of times the RFID tag is read; The position change feature vector generated by the position change rate is: <h2 style=";text-align:left;direction:ltr">l=(D2、D3、……、D<h2 style=";text-align:left;direction:ltr"> I <h2 style=";text-align:left;direction:ltr"> ) Among them, l represents the position change feature vector.

2. The abnormal behavior detection system in an elevator car according to claim 1, characterized in that: When the reference position of the RFID tag in the elevator is set in the position monitoring module, an RFID tag is installed at the center point of the ceiling inside the elevator car as the reference position of the RFID tag, and an RFID reader / writer that can cover the entire space of the car is installed inside the elevator. The frequency offset of the position of the RFID tag carried by the passenger relative to the reference position of the RFID tag is collected by the RFID reader / writer, and the position change rate of the RFID tag carried by the passenger is generated based on the Doppler effect.

3. The abnormal behavior detection system in an elevator car according to claim 1, characterized in that: The principle for generating the temperature feature vector in the data acquisition module is: First, based on the collected car internal temperature, the standard temperature change rate is generated. The formula is: Among them, μ T represents the mean temperature, n represents the number of temperature acquisitions, T j represents the temperature data collected for the jth time, σ T represents the standard deviation of temperature, Z Tj represents the standard temperature change rate at the jth temperature collection time, j represents the temperature index, and j=1, 2, ..., n; The temperature change characteristic value is generated by combining the mean and standard deviation of the standard temperature change rate, based on the formula: in, represents the mean value of the standard temperature change rate, Indicates the standard deviation of the standard temperature change rate, F j Indicates the characteristic value of temperature change; The temperature eigenvector generated by the temperature change eigenvalue is: u=(F1、F2、……、F n ) Where u represents the temperature eigenvector.

4. The abnormal behavior detection system in an elevator car according to claim 1, characterized in that: The principle for generating the pressure characteristic vector in the data acquisition module is: First, the standard pressure change rate is generated based on the principle: Among them, △P(k) represents the standard pressure change rate at the kth pressure collection, P(k) represents the pressure value collected at time k, P(k-1) represents the pressure value collected at time k-1, △t k Indicates the collection time interval; The mean and standard deviation of the standard pressure change rate are generated based on the principle that: Among them, μ △P represents the mean value of the standard pressure change rate, m represents the number of times the standard pressure change rate is generated, △P k represents the standard pressure change rate generated for the kth time, k represents the index of the standard pressure change rate, and k = 1, 2, ..., m, σ △P It represents the standard deviation of the standard pressure change rate; The formula for generating the pressure change signature based on the mean and standard deviation of the standard pressure change rate is: Among them, R k Indicates pressure change characteristics; The pressure feature vector generated by the pressure change feature is: v=(R1、R2、……、R m ) Where v represents the pressure eigenvector.

5. The abnormal behavior detection system in an elevator car according to claim 1, characterized in that: The principle for generating the sound feature vector in the data acquisition module is: Set a time window and calculate the short-time energy of the sound based on the formula: Wherein, w(q) represents the Hamming window function at the qth time, q represents the time index of the Hamming window function, N represents the window length, E(p) represents the short-time energy of the sound at the pth moment, p represents the time index of the short-time energy of the sound, and p=1, 2, ..., N-1, x represents the amplitude of the sound signal, and x(pq) represents the amplitude of the collected sound signal at the pq moment; Taking the short-time energy of sound as the loudness feature of the sound, the generated sound feature vector is: y=(E(1),E(2),...,E(N-1)) Where y represents the sound feature vector.

6. The abnormal behavior detection system in an elevator car according to claim 1, characterized in that: The principle for generating the fusion feature matrix in the feature fusion module is as follows: The position change feature matrix generated according to the position change feature vector is: <h2 style=";text-align:left;direction:ltr">L=[D2、D3、……、D<h2 style=";text-align:left;direction:ltr"> I <h2 style=";text-align:left;direction:ltr"> ] Where L represents the position change feature matrix; The temperature feature matrix generated according to the temperature feature vector is: U=[F1、F2、……、F n ] Where U represents the temperature characteristic matrix; The pressure feature matrix generated according to the pressure feature vector is: V=[R1、R2、……、R m ] Where V represents the pressure feature matrix; The sound feature matrix generated according to the sound feature vector is: Y=[E(1),E(2),…,E(N-1)] Where Y represents the sound feature matrix; The principle for calculating the correlation matrix and generating the attention weight for each combination is as follows: Taking the position change feature matrix and the temperature feature matrix as examples, the same number of elements are intercepted in the matrix to generate the similarity between the two. The formula is: S a,b =L a ·U b Among them, S a,b Indicates the similarity between the elements of the corresponding position change feature matrix and the temperature feature matrix, L a represents the element of the ath column of the position change feature matrix, a represents the column index of the element in the position change feature matrix, and a=1, 2, ..., I, U b represents the element of the b-th row of the temperature feature matrix, b represents the element row number index in the temperature feature matrix, and b=1, 2, ..., n; All calculated similarity values ​​are arranged in sequence to form a matrix S, which represents the correlation matrix between the position change feature matrix and the temperature feature matrix. a,b represents the element in the ath row and bth column of the correlation matrix S; The attention weights are generated based on the correlation matrix according to the formula: in, represents the attention weight of the position change feature matrix in the e1 group, represents the attention weight of the temperature feature matrix in the e1 group, e1 represents the index of the combination of the position change feature matrix and the temperature feature matrix, S represents the correlation matrix, S T represents the transposed matrix of the correlation matrix; After combining the four feature matrices in pairs, each feature matrix participates in three combinations and has three group attention weights. Combining the group attention weights, the formula for generating the corresponding total attention weight is: Wherein, B(L) represents the attention weight of the position change feature matrix, B(U) represents the attention weight of the temperature feature matrix, B(V) represents the attention weight of the pressure feature matrix, B(Y) represents the attention weight of the sound feature matrix, e represents the index of the combination including a certain feature matrix, and a certain feature matrix is ​​a position change feature matrix or a temperature feature matrix or a pressure feature matrix or a sound feature matrix, and e=1, 2, 3; The feature fusion is performed by calculating the generated attention weights to generate a fusion feature matrix based on the formula: K=B(L)·L+B(U)·U+B(V)·V+B(Y)·Y Among them, K represents the fusion weight matrix.

7. A method for detecting abnormal behavior in an elevator car, characterized in that: The method is performed by the abnormal behavior detection system in the elevator car according to any one of claims 1 to 6, and the specific steps include: Step 1: Set the reference position of the RFID tag in the elevator, read the position of the RFID tag carried by the passenger in real time and compare it with the reference position of the RFID tag, generate the position change rate of the RFID tag carried by the passenger, and generate the position change feature vector according to the position change rate; Step 2: Monitor the temperature change in the elevator, generate a standard temperature change rate, use the coefficient of variation of the standard temperature change rate as the temperature change characteristic, generate a temperature feature vector based on the temperature change characteristic, monitor the abnormal pressure in the car, generate a pressure feature vector, monitor the sound loudness in the car, and generate a sound feature vector; Step 3: Reshape the position change feature vector, temperature feature vector, pressure feature vector and sound feature vector into corresponding feature matrices. Four feature matrices are combined in pairs to generate six combinations in total. For each combination, the correlation matrix is ​​calculated and the attention weight is generated. The attention weights of all combinations are combined to generate a fusion feature matrix of the four feature matrices. Step 4: Construct a deep learning network, mark the labels of the types of abnormal behaviors in the elevator car on the fusion feature matrix, and then input it into the deep learning network for training to obtain an elevator car abnormal behavior judgment model with the input value being the fusion feature matrix and the output value being the type of abnormal behavior in the elevator car; Step 5: The real-time RFID tag position, real-time temperature, real-time pressure, real-time light and real-time sound of the elevator car to be identified are used to generate a fusion feature matrix in real time according to the above steps, and then the fusion feature matrix is ​​input into the elevator car abnormal behavior judgment model to determine whether there is abnormal behavior in the current elevator car.