Escalator passenger abnormal behavior detection method and device, medium and equipment

Abstract

The invention discloses an escalator passenger abnormal behavior detection method and device, a medium and equipment, and relates to the technical field of abnormal behavior detection.The escalator passenger abnormal behavior detection method comprises the steps that an obtained to-be-detected video stream is divided into continuous frames, and an initial three-dimensional tensor used for representing passenger behavior information is obtained according to the continuous frames; constructing a joint optimization model based on a tensor ring low-rank model TRLRD and a tensor total variation model TTV; decomposing the initial three-dimensional tensor through a joint optimization model, determining a low-rank background and a dynamic foreground, and separating the low-rank background from the dynamic foreground to obtain a target foreground tensor; and inputting the target foreground tensor into a multi-person attitude estimation algorithm AlphaPose, performing attitude estimation on the passengers, determining an attitude detection result, inputting the attitude detection result into a trained convolutional neural network for abnormal behavior recognition, and obtaining a recognition result.

Description

Technical Field

[0001] The present invention relates to the technical field of abnormal behavior detection, and in particular to a method, device, medium and equipment for detecting abnormal behavior of escalator passengers. Background Art

[0002] With the rapid development of my country's economy, escalators can be seen everywhere in the country, but they also bring many safety issues. Traditional abnormal behavior detection methods mainly rely on manual real-time monitoring in front of the surveillance video. This method not only consumes a lot of manpower, but also requires the monitoring personnel to concentrate their attention as the flow of people increases. They are often responsible for multiple detections at the same time, making it difficult to take care of the overall situation. Existing methods are mainly based on two types of technologies: (1) Traditional image processing: extracting moving targets through optical flow or background difference method, but dynamic lighting changes (such as glass curtain wall reflections) and dense passenger occlusion can easily lead to false detection. (2) YOLO-based abnormal behavior detection: relying on large-scale labeled data training to improve model accuracy, and has high computing power requirements, making it difficult to deploy on edge devices.

[0003] Object detection algorithms based on tensor low-rank decomposition have a natural advantage over other methods in capturing static backgrounds. Video data is inherently high-order tensor data. Traditional methods typically reduce the dimensionality of video data before processing it, but this method loses the spatial information of the video data itself. Traditional tensor decomposition methods (such as CP decomposition and Tucker decomposition) are insufficient for low-rank modeling of multimodal spatiotemporal data and cannot effectively separate escalator operating status (low-rank background) from passenger behavior (sparse foreground). Existing methods also ignore the spatiotemporal correlation between the periodicity of escalator operation and passenger motion trajectories, resulting in low sensitivity to minor anomalies (such as sudden stops), making it difficult to accurately detect abnormal passenger behavior. Summary of the Invention

[0004] The present invention provides a method, device, medium and equipment for detecting abnormal behavior of escalator passengers, which are used to solve the above-mentioned problems existing in the prior art, namely, how to improve the accuracy of detecting abnormal behavior of passengers in the prior art. The present invention provides a method for detecting abnormal behavior of escalator passengers, which includes: Divide the acquired video stream to be tested into continuous frames, and obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames; Construct a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV; The initial three-dimensional tensor is decomposed through a joint optimization model to determine the low-rank background and dynamic foreground, and the low-rank background and dynamic foreground are separated to obtain the target foreground tensor; The target foreground tensor is input into the multi-person pose estimation algorithm AlphaPose to estimate the passenger's pose and determine the pose detection result. The pose detection result is then input into the EfficientNet network for abnormal behavior recognition to obtain the recognition result.

[0005] Optionally, separating the low-rank background and the dynamic foreground specifically includes: The augmented Lagrange multiplier method ALM and alternating direction multiplier method ADMM algorithms are used to separate low-rank background and dynamic foreground.

[0006] Optionally, estimating the passenger's posture specifically includes: Extract the passenger's joint key point information and perform posture detection on the passenger based on the joint key point information.

[0007] Optionally, the convolutional neural network is an efficient neural network EfficientNet.

[0008] Optionally, the abnormal behavior specifically includes: Walking against traffic, falling or crowding.

[0009] Optionally, the video stream is divided into frames according to time windows and normalized.

[0010] The present invention provides an escalator passenger abnormal behavior detection device, comprising: An acquisition module, configured to divide the acquired video stream to be tested into continuous frames, and to obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames; A construction module for constructing a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV; A decomposition module is used to decompose the initial three-dimensional tensor through a joint optimization model, determine the low-rank background and dynamic foreground, separate the low-rank background and dynamic foreground, and obtain the target foreground tensor; The abnormal behavior recognition module is used to input the target foreground tensor into the multi-person pose estimation algorithm AlphaPose, estimate the passenger's pose, determine the pose detection result, and input the pose detection result into the trained convolutional neural network for abnormal behavior recognition to obtain the recognition result.

[0011] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for detecting abnormal behavior of escalator passengers is implemented.

[0012] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned method for detecting abnormal behavior of escalator passengers is implemented.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a method for detecting abnormal behavior of escalator passengers, which effectively separates low-rank background, such as escalator operation status, and sparse foreground, such as passenger behavior, through tensor ring low-rank decomposition, and can more accurately identify abnormal behavior of passengers; the ability to capture dynamic foreground is improved by adopting the TTV model; by combining the ALM and ADMM algorithms, the low-rank background and dynamic foreground are separated to obtain a target foreground tensor. The extraction process of the target foreground tensor in the present invention takes into account the spatiotemporal characteristics of the video data, so that it can still maintain a high detection performance when facing complex dynamic backgrounds and multi-passenger interaction occlusion, further improving the accuracy of identifying abnormal behavior of passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0015] Figure 1 A flowchart of a method for detecting abnormal behavior of escalator passengers provided by an embodiment of the present invention; Figure 2 A technical roadmap for a method for detecting abnormal behavior of escalator passengers provided by an embodiment of the present invention; Figure 3 A schematic diagram of a computer device for the method for detecting abnormal behavior of escalator passengers provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0017] The following describes in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0018] like Figure 1and Figure 2 As shown, this embodiment shows a method for detecting abnormal behavior of escalator passengers, including: S1: Divide the acquired video stream to be tested into continuous frames, and obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames.

[0019] Optionally, the video stream is divided into frames according to time windows and normalized.

[0020] For example, a real-time video stream can be captured by a grayscale camera, divided into frames according to time windows and normalized to construct a three-dimensional spatiotemporal tensor to represent pedestrian motion information (dimensions: length x width x number of frames). The grayscale camera captures the video stream with a resolution of H×W and a frame rate of F. The input video stream is divided into consecutive frames in time order to construct the initial three-dimensional tensor: , normalized to the interval [0,1] frame by frame:

[0021] S2: Construct a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV.

[0022] S3: Decompose the initial three-dimensional tensor through the joint optimization model to determine the low-rank background and dynamic foreground, separate the low-rank background and dynamic foreground, and obtain the target foreground tensor.

[0023] For example, a joint optimization model is constructed using tensor robust principal component analysis (TRLRDPCA) and tensor total variation model (TTV), which decomposes the initial three-dimensional tensor into low-rank background, sparse noise and dynamic foreground. The moving target foreground tensor is separated by iterative solution using the ADMM algorithm.

[0024] Exemplarily, the tensor is divided into local spatiotemporal blocks, each of size K×Hb×Wb.

[0025] Among them, Z represents the original video data, L represents the low-rank tensor, T represents the sparse tensor, S represents the dynamic background and W represents the moving target foreground, and the TTV-A term represents the TTV-A norm of the foreground W.

[0026] The augmented Lagrange multiplier method (ALM) and alternating direction multiplier method (ADMM) can be used to solve the minimization model.

[0027] S4: Input the target foreground tensor into the multi-person pose estimation algorithm AlphaPose, estimate the passenger's pose, determine the pose detection result, and input the pose detection result into the trained convolutional neural network for abnormal behavior recognition to obtain the recognition result.

[0028] Optionally, perform posture detection on the passenger to obtain posture detection results, specifically including: By adopting the multi-person pose estimation algorithm AlphaPose, the joint key point information of the passengers is extracted, and the posture of the passengers is detected based on the joint key point information.

[0029] For example, the AlphaPose multi-person pose estimation algorithm is used to estimate the pose of passengers frame by frame using the target foreground tensor W. Pose estimation involves two steps: testing and training. During testing, the passenger targets are first fed into the SSTN spatial transformer network to modify the poor-quality passenger regions. Then, the Single Person Pose Estimator (SPPE) is used to estimate the pose of the foreground information. Finally, the parameterized pose non-maximum suppression algorithm (P-NMS) is used to remove redundant poses. During training, the SPPE module is used to train the SSTN module to avoid local optimality.

[0030] For example, posture features can be input into the EfficientNet classification network to identify abnormal behaviors such as walking against traffic, falling, and crowding, and trigger alarms or braking signals.

[0031] The above is a method for detecting abnormal behavior of escalator passengers provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding device for detecting abnormal behavior of escalator passengers, including: An acquisition module, configured to divide the acquired video stream to be tested into continuous frames, and to obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames; A construction module for constructing a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV; A decomposition module is used to decompose the initial three-dimensional tensor through a joint optimization model, determine the low-rank background and dynamic foreground, separate the low-rank background and dynamic foreground, and obtain the target foreground tensor; The abnormal behavior recognition module is used to input the target foreground tensor into the multi-person pose estimation algorithm AlphaPose, estimate the passenger's pose, determine the pose detection result, and input the pose detection result into the trained convolutional neural network for abnormal behavior recognition to obtain the recognition result.

[0032] The specific definition of the escalator passenger abnormal behavior detection device can be found in the definition of the escalator passenger abnormal behavior detection method above, and will not be repeated here. The various modules in the above-mentioned escalator passenger abnormal behavior detection device can be implemented in whole or in part through software, hardware, or a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of the above-mentioned modules.

[0033] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned method for detecting abnormal behavior of escalator passengers.

[0034] The present invention also provides Figure 3 The structural diagram of the computer equipment shown in FIG. Figure 3 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the internal memory and then runs it to implement the method for detecting abnormal escalator passenger behavior provided in the above embodiment.

[0035] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware using a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes in the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0036] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A method for detecting abnormal behavior of escalator passengers, characterized in that: include: Divide the acquired video stream to be tested into continuous frames, and obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames; Construct a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV; The initial three-dimensional tensor is decomposed through a joint optimization model to determine the low-rank background and dynamic foreground, and the low-rank background and dynamic foreground are separated to obtain the target foreground tensor; The target foreground tensor is input into the multi-person pose estimation algorithm AlphaPose to estimate the passenger's pose and determine the pose detection result. The pose detection result is then input into the trained convolutional neural network for abnormal behavior recognition to obtain the recognition result.

2. The method for detecting abnormal behavior of escalator passengers according to claim 1, characterized in that: The separation of low-rank background and dynamic foreground specifically includes: The augmented Lagrange multiplier method ALM and alternating direction multiplier method ADMM algorithms are used to separate low-rank background and dynamic foreground.

3. The method for detecting abnormal behavior of escalator passengers according to claim 1, wherein: The passenger posture estimation specifically includes: Extract the passenger's joint key point information and perform posture detection on the passenger based on the joint key point information.

4. The method for detecting abnormal behavior of escalator passengers according to claim 1, wherein: The convolutional neural network is an efficient neural network EfficientNet.

5. The method for detecting abnormal behavior of escalator passengers as claimed in claim 1, wherein: The abnormal behaviors specifically include: Walking against traffic, falling or crowding.

6. The method for detecting abnormal behavior of escalator passengers according to claim 1, wherein: The video stream is divided into frames according to time windows and normalized.

7. An escalator passenger abnormal behavior detection device, characterized in that: include: An acquisition module, configured to divide the acquired video stream to be tested into continuous frames, and to obtain an initial three-dimensional tensor for representing passenger behavior information based on the continuous frames; A construction module for constructing a joint optimization model based on the tensor ring low-rank model TRLRD and the tensor total variation model TTV; A decomposition module is used to decompose the initial three-dimensional tensor through a joint optimization model, determine the low-rank background and dynamic foreground, separate the low-rank background and dynamic foreground, and obtain the target foreground tensor; The abnormal behavior recognition module is used to input the target foreground tensor into the multi-person pose estimation algorithm AlphaPose, estimate the passenger's pose, determine the pose detection result, and input the pose detection result into the trained convolutional neural network for abnormal behavior recognition to obtain the recognition result.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the method for detecting abnormal behavior of escalator passengers according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, the method for detecting abnormal behavior of escalator passengers according to any one of claims 1 to 6 is implemented.

Images