An elevator door blocking behavior recognition method based on a two-stage detection network

Abstract

The application provides an elevator door blocking behavior recognition method based on a two-stage detection network, which uses a two-stage feature extraction network to recognize the real-time opening degree and change trend of the elevator door, so as to recognize the occurrence of the door blocking behavior. The first-stage elevator door area detection network uses a Yolox algorithm to focus the feature extraction on the elevator door area, the second-stage elevator door opening degree recognition network uses an EfficientNet b6 network trunk to perform feature extraction, and linear regression is performed at the output end to obtain the real-time opening degree of the elevator door. Finally, the recognition of the door blocking behavior is completed through logical judgment and screening. The method greatly improves the accuracy of the elevator door blocking behavior recognition, and can intercept the door blocking segment, thereby improving the elevator security efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of behavior recognition, and particularly to an elevator door blocking behavior recognition method. BACKGROUND

[0002] The emergence of the elevator has brought great convenience to our life. However, the safety of the elevator is also worth special attention while improving our quality of life. Especially in high-rise residential areas, the usage rate of the elevator is very high, and the incidence of some unsafe behaviors in the elevator is also increasing. Among them, the behavior of passengers blocking the elevator door is one of the most common unsafe behaviors, which can easily cause the elevator door to fail to open and close normally, thereby causing danger.

[0003] In order to timely discover some bad behaviors and emergencies in the elevator, most elevator cars are equipped with monitoring cameras, but the traditional video monitoring system is difficult to accurately and timely discover the uncivilized behavior of passengers in the elevator. With the emergence of intelligent video monitoring technology and the development of artificial intelligence, artificial intelligence algorithms based on computer vision have also become a research hotspot in recent years. Therefore, using artificial intelligence algorithms to recognize behaviors in monitoring videos has gradually become a common method.

[0004] The existing patents on elevator behavior recognition methods can be roughly divided into two categories. One is a behavior detection method based on machine learning algorithm. Some researchers use the background subtraction method + Sobel operator edge detection method to extract the edge features of passengers in the elevator and analyze their actions. Some other researchers calculate the Hausdorff distance of a fixed feature point between two adjacent frames of images to determine the opening and closing state of the elevator door. In addition, in order to solve the problem that there are no significant feature points in the elevator door area, some patents propose to paste a mark on the elevator car door, and add feature points artificially to enhance the feature points. Although this method solves the problem of labor consumption of traditional monitoring technology to some extent, it is often affected by environmental light, has poor generalization ability, low detection accuracy, and has great limitations in actual application. The other is a behavior detection method based on deep learning. This kind of method mostly uses dual-stream convolutional network or three-dimensional convolutional network to extract the spatio-temporal features of monitoring videos. Some patents also integrate the optical flow method into the three-dimensional convolutional network to improve the network recognition accuracy. Compared with the machine learning method, the above method greatly improves the detection accuracy and generalization ability, but its shortcomings are network calculation complexity and heavy network framework. Moreover, the small window score fusion method makes the time relationship between multiple small windows not optimally related. In addition, for the behavior of passengers blocking the door, its duration is long, and the behavior features between distant frames have correlation, so it is difficult to accurately extract features and recognize them by relying on a simple single neural network.

[0005] Therefore, the application provides an elevator door blocking behavior recognition method based on a two-stage detection network, which adopts a two-stage feature extraction network to indirectly determine whether a door blocking behavior occurs by recognizing the change trend of the elevator door opening degree in a period of time. The first stage is an elevator door area detection network, which uses the Yolox algorithm to focus the feature extraction on the elevator door area. The second stage is an elevator door opening degree recognition network, which uses the EfficientNet b6 network backbone for feature extraction and performs linear regression at the output end to obtain the real-time opening degree of the elevator door. Finally, the recognition of the door blocking behavior is completed through logical judgment and screening. Tests show that the method effectively improves the recognition accuracy of the elevator door blocking behavior. SUMMARY

[0006] The application aims to provide an elevator door blocking behavior recognition method based on a two-stage detection network, which sequentially completes area cropping and elevator door opening degree recognition through a two-stage feature extraction network, and outputs the door blocking behavior segments through logical judgment and screening after regression at the output end, thereby improving the recognition accuracy of the elevator door blocking behavior.

[0007] Technical scheme: To achieve this object, the application adopts the following technical scheme:

[0008] The elevator door blocking behavior recognition method based on the two-stage detection network comprises the following steps:

[0009] S1: Collect and screen the elevator monitoring video in a high-rise building, select monitoring videos of different scenes and different camera angles, frame them, crop the elevator door area and label them to make an elevator door area detection dataset, and classify and make an elevator door opening degree recognition dataset according to the elevator door opening degree;

[0010] S2: Build the first-stage elevator door area detection network, use the Yolox algorithm to build the network, the input end is a picture tensor of 224x224 size, and the output end is the rectangular coordinates of the elevator door area in the picture, and the elevator door area is cropped from the original picture by using the coordinates;

[0011] S3: Build the second-stage elevator door opening degree recognition network, use the EfficientNet b6 network backbone as the feature extraction network, and extract the features of the elevator door area cropped in step S2, and use the linear regression method at the output end to obtain a floating-point number representing the elevator door opening degree;

[0012] S4: Train the two-stage network using the two data sets in step S1 respectively, obtain the trained model, and connect and fuse the two-stage network model, then connect a set of logical judgment statements at the output end to form a two-stage detection network for identifying and screening the door blocking behavior;

[0013] S5: Pass the video stream to be detected through a sliding window, then continuously send the video segments cut by the sliding window into the two-stage network model constructed in step S4, output the judgment result, and cut the video segments containing the door blocking behavior.

[0014] In step S1, when making the elevator door opening recognition data set, the pictures are divided into 0%, 20%, 40%, 60%, 80%, 90%, and 100% according to the elevator door opening size.

[0015] In step S2, the first-stage elevator door area detection network is constructed including the following steps:

[0016] S21: Use the Yolox-Darknet53 model in the Yolox series network as the backbone network part;

[0017] S22: At the output end, combine the coordinates of the elevator door target frame recognized by the network model to perform cutting operation on the original picture using the ffmpeg function tool;

[0018] S23: The cutting result is input into the second-stage network in jpg picture format.

[0019] In step S3, the second-stage elevator door opening recognition network is constructed including the following steps:

[0020] S31: Use the first 17 layers of the EfficientNet b6 network as the feature extraction network to extract features from the cut elevator door area picture input in step S23;

[0021] S32: After the 17-layer feature extraction network, connect a pooling layer and a fully connected layer to change the channel number to 960;

[0022] S33: Use a linear regression function at the last output end to regress the 960-channel information into a floating-point number as the real-time opening value of the elevator door for output.

[0023] In step S4, the logical judgment statement combination at the output end of the two-stage detection network is as follows: calculate the gradient of the elevator door real-time opening value obtained in step S33, if the gradient changes from the negative threshold -L to the positive threshold L within 20 consecutive frames, and the opening value of the elevator door does not decrease to 0 within this time period, it is judged that the door blocking behavior occurs in this video interval, which is screened and cut.

[0024] Beneficial effects: The application discloses an elevator door blocking behavior recognition method based on a two-stage detection network, adopts a two-stage feature extraction network, the first stage uses a Yolox algorithm to crop an original picture, focuses feature extraction on an elevator door area, plays an important role in subsequent feature extraction, and is beneficial to excluding interference of non-feature areas. The second stage uses an EfficientNet b6 network trunk to perform feature extraction on the elevator door area, performs linear regression at an output end, obtains real-time opening of the elevator door, and finally completes recognition of the door blocking behavior through logical judgment and screening. The method solves problems of low detection precision of a machine learning method and complex parameters of a three-dimensional neural network, and tests show that the method effectively improves recognition accuracy of the elevator door blocking behavior, and detection precision of the door blocking behavior in an elevator car reaches more than 90%. In actual application, the method greatly improves efficiency of the elevator door blocking behavior detection, saves manpower and material resources consumed by a traditional video monitoring technology, is not easy to be affected by light inside and outside the elevator, has strong generalization ability, has low model calculation amount and complexity, and saves cost. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the method in the specific embodiment of the application.

[0026] Figure 2 is a logical judgment and recognition diagram in the specific embodiment of the application. DETAILED DESCRIPTION

[0027] The technical solutions of the application are further described below in combination with specific embodiments and drawings.

[0028] Embodiment: The specific embodiment discloses an elevator door blocking behavior recognition method based on a two-stage detection network, as shown in the figure, including the following steps: Figure 1

[0029] S1: Collect and screen elevator monitoring videos in a high-rise building, select 200 monitoring videos of different scenes and different camera angles, frame all the monitoring videos, crop elevator door areas of a part of the monitoring videos, and label the part as "door" to manufacture elevator door area detection data sets, and divide the other part of the monitoring videos into seven categories of 0%, 20%, 40%, 60%, 80%, 90% and 100% according to elevator door opening sizes to manufacture elevator door opening recognition data sets;

[0030] ​S2: Constructing the first stage of the elevator door area detection network, using the Yolox algorithm to build the network, selecting Yolox-Darknet53 as the basic model, the input end being a picture tensor with a size of 224x224, and the coordinates of the elevator door target frame recognized by the network model being combined at the output end to perform a cutting operation on the original picture by using an ffmpeg function tool to cut down the elevator door area;

[0031] S3: Constructing the second stage of the elevator door opening recognition network, using the main part of the EfficientNet b6 network as a feature extraction network, performing feature extraction on the elevator door area cut down in step S2, and using a linear regression method at the output end to obtain a floating point number representing the opening of the elevator door, the steps being constructed as follows:

[0032] S31: Using the first 17 layers of the EfficientNet b6 network as a feature extraction network to perform feature extraction on the cut elevator door area picture input in step S2;

[0033] S32: After the 17-layer feature extraction network, a pooling layer and a fully connected layer are connected to change the channel number to 960;

[0034] S33: Using a linear regression function at the last output end to regress the information of 960 channels into a floating point number as the real-time opening value of the elevator door for output.

[0035] S4: Training the two-stage network using the two data sets in step S1 respectively to obtain a trained model, and connecting and fusing the two-stage network model, and then connecting a set of logical judgment statements at the output end, the combination of the logical judgment statements being: calculating the gradient of the real-time opening value of the elevator door obtained in step S33, if the gradient changes from a negative threshold -L to a positive threshold L within 20 consecutive frames, and the opening value of the elevator door does not decrease to 0 in this time period, it is judged that the door blocking behavior occurs in this video interval, and the video segment is screened and intercepted;

[0036] S5: Passing the video stream to be detected through a sliding window, and then continuously feeding the video segment intercepted by the sliding window into the two-stage network model constructed in step S4 to output a judgment result, and intercepting the video segment containing the door blocking behavior.

[0037] It should be noted that the above embodiments are not intended to limit the protection scope of the present application, and equivalent transformations or substitutions made on the basis of the above technical solutions all fall within the scope of the claims of the present application.

Claims

1. A method for recognizing elevator door-blocking behavior based on a two-stage detection network, characterized in that, The method includes the following steps: S1: Collect and filter elevator monitoring videos in high-rise buildings. Select monitoring videos with different scenes and camera angles, extract frames from them, and create an elevator door area detection dataset by cropping the elevator door area and labeling it. Another part is classified according to the size of the elevator door opening to create an elevator door opening recognition dataset. S2: Construct the elevator door region detection network in the first stage. The network is built using the Yolox algorithm. The input is a 224×224 image tensor, and the output is the rectangular coordinates of the elevator door region in the image. The coordinates are used to crop the elevator door region from the original image. S3: Construct the elevator door opening recognition network for the second stage. Use the backbone of the EfficientNet b6 network as the feature extraction network to extract features from the elevator door area cropped in step S2. Use linear regression at the output to obtain a floating-point number representing the elevator door opening. S4: Train the two-stage networks using the two datasets from step S1 respectively to obtain trained models. Connect and fuse the two-stage network models, and then connect a set of logical judgment statements to the output to form a two-stage detection network for the identification and screening of door-blocking behavior. S5: Pass the video stream to be detected through a sliding window, and then continuously feed the video segments extracted from the sliding window into the two-stage network model constructed in step S4, output the judgment result, and extract the video segments containing the door-blocking behavior.

2. The elevator door-blocking behavior recognition method based on a two-stage detection network according to claim 1, characterized in that, In step S1, when creating the elevator door opening recognition dataset, the images are divided into seven categories according to the size of the elevator door opening: 0%, 20%, 40%, 60%, 80%, 90%, and 100%.

3. The elevator door-blocking behavior recognition method based on a two-stage detection network according to claim 1, characterized in that: In step S2, constructing the elevator door area detection network for the first stage includes the following steps: S21: The Yolox-Darknet53 model from the Yolox series of networks is used as the backbone network. S22: Combine the coordinates of the elevator door target box obtained by the network model at the output end, and use the ffmpeg function tool to perform a cropping operation on the original image; S23: The cropping result is input into the second-stage network as a JPG image.

4. The elevator door-blocking behavior recognition method based on a two-stage detection network according to claim 1, characterized in that: In step S3, constructing the elevator door opening recognition network for the second stage includes the following steps: S31: Use the first 17 layers of the EfficientNet b6 network as the feature extraction network to extract features from the cropped elevator door area image input in step S23; S32: After the 17-layer feature extraction network, a pooling layer and a fully connected layer are added to change the number of channels to 960. S33: At the final output, a linear regression function is used to regress the information from 960 channels into a floating-point number, which is then output as the real-time elevator door opening value.

5. The elevator door-blocking behavior recognition method based on a two-stage detection network according to claim 1, characterized in that: In step S4, the logical judgment statement combination connected to the output end of the two-stage detection network is as follows: calculate the gradient of the real-time opening value of the elevator door obtained in step S33. If the gradient changes from the negative threshold -L to the positive threshold L within 20 consecutive frames, and the opening value of the elevator door does not drop to 0 during this period, it is determined that the video interval has a door blocking behavior, and it is filtered and captured.

Images