Rail transit station passenger abnormal behavior identification method and system

Abstract

The invention provides a rail transit station passenger abnormal behavior recognition method and system based on human body postures, and belongs to the technical field of human body behavior recognition. Performing abnormal behavior recognition on the rail transit station passengers by using a rail transit station passenger abnormal behavior recognition algorithm, and finally judging the safety state of the rail transit station passengers and performing early warning. According to the rail transit station passenger abnormal behavior identification algorithm, the identification of the rail transit station passenger behavior is completed according to the rail transit station monitoring requirement, only the rail transit station passenger abnormal behavior is detected, and the safety state level of the rail transit station passenger is judged. The method improves the detection precision and speed of the rail transit station passenger abnormal behaviors, and is more suitable for the application requirements of the rail transit station passenger abnormal behaviors.

Description

Technical Field

[0001] The present invention relates to the technical field of human behavior recognition, and in particular to a method and system for recognizing abnormal behavior of passengers in a rail transit station based on human posture. Background Art

[0002] In a densely populated scene such as a rail transit station, the possibility of unexpected events is high, which increases the difficulty of operation and management for rail transit station operators. How to effectively use existing resources to ensure the safety of passengers' travel is imminent. The existing behavior recognition method based on human posture is a two-stage framework: first, the posture of the person in the video is estimated through the posture estimation algorithm, and the key point coordinate data and confidence of the passenger posture in the key frame are obtained; then the neural network is used to extract features from the spatiotemporal segments constructed from the passenger posture data of continuous frames to form a global spatiotemporal representation, and then the behavior classification is completed based on the local features extracted in these frames. How to apply the existing advanced technology to the problem of abnormal behavior identification of passengers in rail transit stations, meet the safety maintenance needs of rail transit stations, and realize the safety status analysis function of passengers in rail transit stations is a problem that needs to be solved. Summary of the invention

[0003] The object of the present invention is to provide a method and system for identifying abnormal behavior of passengers in rail transit stations based on human posture, so as to solve at least one technical problem existing in the above-mentioned background technology.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides a method for identifying abnormal behavior of passengers in a rail transit station based on human posture, comprising:

[0006] Obtain video images monitored by rail transit stations and extract each frame of the video;

[0007] The target detection model is used to detect the passengers in the video image, and the detection frame position information of each passenger target and the number of passengers in the current video are obtained. The image of each passenger in each frame is cropped according to the detection frame position information.

[0008] Use the pre-trained passenger posture estimation model to estimate the posture of the passenger image and complete the recognition of the key position information of the current passenger's body;

[0009] The Deepsort algorithm is used to track the passengers in the video images, and the passenger posture information in multiple consecutive frames is concatenated into posture sequence data based on the tracking results.

[0010] The positions of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated based on the posture sequence data, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information;

[0011] According to the obtained number of passengers, movement speed, behavior category information and behavior duration, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value.

[0012] Furthermore, the training of the pre-trained passenger posture estimation model includes: inputting the cropped passenger image into the MobilenetV2 network for feature extraction to obtain the center point heat map and key point heat map information of the passenger; after determining the center position of the character, predicting the regression values ​​of other skeletal key points to obtain a preliminary estimate of the coordinates of each key point, calculating the position of the maximum value in the key point heat map as the final key point estimation position, and adding a local 2D offset for coordinate correction.

[0013] Furthermore, the training of the pre-trained passenger behavior recognition model includes: constructing a passenger behavior data set for a rail transit station, and marking the behavior data of passengers in the rail transit station; calculating the average speed of the current passenger's head, shoulders, elbows and hips in multiple consecutive frames; inputting the key point coordinate information of the multiple consecutive frames of posture into the spatiotemporal graph convolutional network, and calculating the key point spatial feature information of the current passenger in multiple consecutive frames; inputting the coordinate transformation information of the same key points of adjacent frames in multiple consecutive frames into the spatiotemporal graph convolutional network, and calculating the key point temporal feature information of the current passenger in multiple consecutive frames; fusing the spatial feature information and the temporal feature information of the passenger, and inputting them into the fully connected layer and the sigmoid activation function to obtain the behavior category of the passenger.

[0014] Furthermore, according to the obtained number of passengers n, average speed v, behavior category information c and behavior duration t, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value M:

[0015]

[0016] Furthermore, five loss functions are used for the training of the pre-trained passenger posture estimation model, namely center point heat map loss, key point heat map loss, key point regression loss, local offset loss and skeleton loss.

[0017] Furthermore, for the center point heat map loss, the weighted mean square error loss is used; the key point heat map loss and the center point heat map loss use the weighted mean square error loss; the spatial resolution of the feature map output by the network becomes one-fourth of the original input image, which is equivalent to a 4x4 area of ​​the original image corresponding to a pixel point on the output feature map, which will bring a large error, so local offset and local offset loss are introduced. The offset refers to the difference between the key point coordinates and the center point coordinates; the bone loss is to calculate the difference between the length of the skeleton composed of the predicted key points and the length of the corresponding real key points.

[0018] In a second aspect, the present invention provides a system for identifying abnormal behavior of passengers in a rail transit station based on human posture, comprising:

[0019] An acquisition module is used to acquire video images monitored by rail transit stations and extract each frame of the video;

[0020] The detection module is used to detect the passengers in the video image using the target detection model, obtain the detection frame position information of each passenger target and the number of passengers in the current video, and crop each passenger image in each frame according to the detection frame position information;

[0021] The posture estimation module is used to estimate the posture of the passenger image using the pre-trained passenger posture estimation model to complete the recognition of the key position information of the current passenger's body;

[0022] The target tracking module is used to track the passengers in the video image using the Deepsort algorithm, and concatenate the passenger posture information in multiple consecutive frames into posture sequence data based on the tracking results;

[0023] The behavior recognition module is used to calculate the position of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips based on the posture sequence data, and input the posture sequence data into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information;

[0024] The analysis module is used to analyze the safety status of passengers in rail transit stations and obtain a safety status value based on the obtained number of passengers, movement speed, behavior category information and behavior duration.

[0025] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the method for identifying abnormal behavior of passengers in a rail transit station based on human posture as described in the first aspect is implemented.

[0026] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method for identifying abnormal behavior of passengers in rail transit stations based on human posture as described in the first aspect.

[0027] In a fifth aspect, the present invention provides an electronic device, comprising: a processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the method for identifying abnormal behavior of passengers in rail transit stations based on human posture as described in the first aspect.

[0028] The beneficial effects of the present invention are as follows: in view of the problem of detecting abnormal behavior of passengers in rail transit stations, a model for detecting abnormal behavior of passengers in rail transit stations based on human body posture is proposed in combination with the actual application scenarios of rail transit stations. The model recognizes the behavior of passengers in rail transit stations according to the monitoring needs of rail transit stations, and judges the safety status level of passengers in rail transit stations, thereby improving the detection accuracy and speed of abnormal behavior of passengers in rail transit stations, and is more suitable for the application needs of abnormal behavior of passengers in rail transit stations.

[0029] Additional advantages of the present invention will be more clearly given in the following description or learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0031] Figure 1 This is a flow chart of a method for identifying abnormal behavior of passengers in rail transit stations based on human posture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below by the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0033] It should be understood by those skilled in the art that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0034] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with that in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0035] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or groups thereof.

[0036] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. Different embodiments or examples described in this specification and features of different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

[0037] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0038] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0039] Example 1

[0040] In this embodiment 1, a system for identifying abnormal behavior of passengers in a rail transit station is first provided, including: an acquisition module, which is used to acquire video images monitored by a rail transit station and extract each frame of the video. A detection module, which is used to detect the passengers in the video image using a target detection model, obtain the detection frame position information of each passenger target and the number of passengers in the current video, and crop each passenger image in each frame of the image according to the detection frame position information. A posture estimation module, which is used to estimate the posture of the passenger image using a pre-trained passenger posture estimation model, and complete the identification of the key position information of the human body of the current passenger. A target tracking module, which is used to track the passengers in the video image using a Deepsort algorithm, and concatenate the passenger posture information in multiple consecutive frames into posture sequence data according to the tracking results. A behavior recognition module, which is used to calculate the position of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips according to the posture sequence data, and input the posture sequence data into a pre-trained passenger behavior recognition model to obtain the behavior category information of the passenger. The analysis module is used to analyze the safety status of passengers in rail transit stations and obtain a safety status value based on the obtained number of passengers, movement speed, behavior category information and behavior duration.

[0041] In this embodiment, the above-mentioned system is used to implement a method for identifying abnormal behavior of passengers in a rail transit station, including: obtaining a video image monitored by a rail transit station and extracting each frame of the video; using a target detection model to perform target detection on passengers in the video image, obtain the detection frame position information of each passenger target and the number of passengers in the current video, and crop each passenger image in each frame according to the detection frame position information; using a pre-trained passenger posture estimation model to perform posture estimation on the passenger image, and complete the identification of the key position information of the human body of the current passenger; using a Deepsort algorithm to track the passengers in the video image, and according to the tracking results, the passenger posture information in multiple consecutive frames is serially connected into posture sequence data; according to the posture sequence data, the position of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the behavior category information of the passenger; according to the obtained number of passengers, movement speed, behavior category information and duration of the behavior, the safety status analysis of the passengers in the rail transit station is performed to obtain the safety status value.

[0042] Among them, the training of the pre-trained passenger posture estimation model includes: inputting the cropped passenger image into the MobilenetV2 network for feature extraction to obtain the passenger's center point heat map and key point heat map information; after determining the center position of the character, predicting the regression values ​​of other skeletal key points, obtaining a preliminary estimate of the coordinates of each key point, calculating the position of the maximum value in the key point heat map as the final key point estimation position, and adding a local 2D offset for coordinate correction.

[0043] The training of the pre-trained passenger behavior recognition model includes: constructing a passenger behavior dataset for rail transit stations and marking the behavior data of passengers in rail transit stations; calculating the average speed of the current passenger's head, shoulders, elbows and hips in multiple consecutive frames; inputting the key point coordinate information of multiple consecutive frames of posture into the spatiotemporal graph convolutional network, and calculating the key point spatial feature information of the current passenger in multiple consecutive frames; inputting the coordinate transformation information of the same key points of adjacent frames in multiple consecutive frames into the spatiotemporal graph convolutional network, and calculating the key point temporal feature information of the current passenger in multiple consecutive frames; fusing the spatial feature information and the temporal feature information of the passenger, and inputting them into the fully connected layer and the sigmoid activation function to obtain the behavior category of the passenger.

[0044] According to the obtained number of passengers n, average speed v, behavior category information c and behavior duration t, the safety status of passengers in rail transit stations is analyzed to obtain the safety status value M:

[0045]

[0046] The training of the pre-trained passenger posture estimation model uses five loss functions, namely center point heat map loss, key point heat map loss, key point regression loss, local offset loss and skeleton loss. For center point heat map loss, weighted mean square error loss is used; key point heat map loss and center point heat map loss use weighted mean square error loss; the spatial resolution of the feature map output by the network becomes one-fourth of the original input image, which is equivalent to a 4x4 area of ​​the original image corresponding to a pixel on the output feature map, which will bring a large error, so local offset and local offset loss are introduced. The offset refers to the difference between the coordinates of the key point and the coordinates of the center point; the skeleton loss is to calculate the difference between the length of the skeleton composed of the predicted key points and the length of the corresponding real key points.

[0047] Example 2

[0048] In this embodiment 2, a method for identifying abnormal behavior of passengers in rail transit stations based on human posture is provided, and the posture estimation method and the abnormal behavior recognition method are applied to the rail transit scenario to solve the problems of passenger posture estimation, abnormal behavior recognition and passenger safety status analysis in the rail transit station scenario.

[0049] like Figure 1 As shown, the method for identifying abnormal behavior of passengers in rail transit stations based on human posture includes the following steps:

[0050] Step 1, obtaining a video image monitored by a rail transit station and extracting each frame of the video;

[0051] Step 2: Use the Yolov5 algorithm to detect the passengers in the video image, output the detection frame position information of each passenger target and the number of passengers n in the current video, and crop each passenger image in each frame according to the detection frame position information;

[0052] Step 3: Use the improved passenger posture estimation algorithm in the rail transit station scene to estimate the posture of the passenger image and complete the recognition of 14 key position information of the current passenger;

[0053] Therefore, the specific detection method of step 3 mainly includes the following steps:

[0054] Step 3.1, input the cropped passenger image into the MobilenetV2 network for feature extraction to obtain the center point heat map and key point heat map information of the passenger;

[0055] Step 3.2, after determining the center position of the character, predict the regression values ​​of other skeleton key points to obtain a preliminary estimate of the coordinates of each key point, calculate the position of the maximum value in the key point heat map, and use it as the final estimated position of the key point. On this basis, add the local 2D offset to perform coordinate correction to obtain the final coordinates of the posture key points;

[0056] Step 4: Use the Deepsort algorithm to track the passengers in the video image, and concatenate the passenger posture information in 30 consecutive frames into posture sequence data based on the tracking results;

[0057] Step 5, calculating the movement speed information of the head, shoulders, elbows and hips of the current passenger according to the posture sequence data, inputting the posture sequence data into the improved rail transit station passenger behavior recognition algorithm to obtain the passenger's behavior information;

[0058] Therefore, the specific detection method of step 5 mainly includes the following steps:

[0059] Step 5.1: Construct a passenger behavior dataset for rail transit stations, and label the behavior data of passengers in rail transit stations, mainly including running, falling, waving, standing, and walking, for training and testing the passenger pose estimation algorithm in the rail transit station scenario;

[0060] Step 5.2: Calculate the average speed of key points such as the current passenger's head, shoulders, elbows, and hips within 30 consecutive frames, x t+1 ,x t ,y t+1 ,y t respectively representing the horizontal and vertical coordinates of the key point at time t + 1 and time t, and W and H are the width and height of the image;

[0061] Step 5.3: Input the key point coordinate information of 30 consecutive frames of poses into the spatio-temporal graph convolutional network to calculate the key point spatial feature information of the current passenger within 30 consecutive frames;

[0062] Step 5.4: Input the coordinate transformation information of the same key points in adjacent frames among 30 consecutive frames into the spatio-temporal graph convolutional network to calculate the key point temporal feature information of the current passenger within 30 consecutive frames;

[0063] Step 5.5: Fuse the spatial feature information and temporal feature information of the passenger, and input them into the fully connected layer and the sigmoid activation function to obtain the behavior category c of the passenger;

[0064] Step 6. Based on the number n of passengers, speed v, behavior category information c, and the duration t of the behavior obtained in the above steps, perform a safety status analysis of passengers in rail transit stations to obtain a safety status value M:

[0065]

[0066] According to the safety status value M, the safety status levels of passengers in rail transit stations are divided into four levels: major level (M > 500), large level (50 < M < 500), general level (5 < M < 50), and minor level (0.5 < M < 5).

[0067] Example 3

[0068] In this Example 3, a method for identifying abnormal behaviors of passengers in rail transit stations based on videos is provided, including the following steps:

[0069] Step 1: Based on the surveillance video of the rail transit station, the YOLOv5 algorithm is used to detect the passenger targets of the rail transit station in the current video image, and the number of passengers in the current video is obtained. Each passenger target is cropped from the original image, and then input into the improved passenger posture estimation in the rail transit station scene, and the coordinate information of 14 human body key points of the passenger is detected. The posture information of each passenger is transmitted back to the original video image to obtain the posture information of all passengers in the current image;

[0070] Step 2: Use the Deepsort algorithm to track the passengers in the rail transit station. According to the tracking results, the posture data of the same passenger in multiple consecutive frames are connected into a posture sequence. According to the posture sequence data, the movement speed information of the key points of the current passenger's head, shoulders, elbows and hips is calculated. At the same time, the posture sequence data is input into the improved rail transit station passenger behavior recognition algorithm to obtain the passenger's behavior information;

[0071] Step 3: Analyze the safety status of passengers in rail transit stations based on the number of passengers, speed, behavior category information, and behavior duration obtained in the above steps.

[0072] The improved passenger posture estimation in the rail transit station scenario in step 1 is different from other posture algorithms in the training set and uses the collected rail transit station passenger posture dataset.

[0073] The improved passenger posture estimation in the rail transit station scenario in step 1 uses five loss functions, namely center heatmap loss, keypoint heatmap loss, keypoint regression loss, local offsets loss and skeleton loss.

[0074] For the center point heat map loss, the weighted mean square error loss (WMSE) is used, and the calculation formula is as follows:

[0075]

[0076] Where: N represents the width of the input heat map, M represents the height of the input heat map, and p ij represents the predicted value of the i-th column and j-th row in the predicted heat map, t ij represents the target value of the i-th column and j-th row in the real heat map, w ij Represents the weight of the target value in the i-th column and j-th row.

[0077] For the 14 key point heat map losses, the weighted mean square error loss is used similar to the center point heat map loss, but the average value of the 14 key point heat map losses is taken. The formula is as follows:

[0078]

[0079] The spatial resolution of the feature map output by the network becomes one-fourth of the original input image, which is equivalent to a pixel on the output feature map corresponding to a 4x4 area of ​​the original image. This will bring a large error, so local offset and local offset loss are introduced:

[0080]

[0081] In the formula, p represents the center point of the target box, R represents the downsampling multiple of 4, Represents the deviation value.

[0082] The offset refers to the coordinates of 14 key points of the human body (x p ,u p ) and the center point (x g ,y g ), the loss formula is as follows:

[0083]

[0084] Bone loss is the difference between the length of the bone composed of the predicted key points and the length of the corresponding real key points. The calculation formula is as follows:

[0085]

[0086] where p_bone i , t_bone i Represent the actual and predicted human skeleton lengths respectively, i∈[[0,2],[2,4],[1,3],[3,5],[7,9],[9,11],[6,8],[8,10]], indicating the skeleton between the skeleton key points represented by each serial number, as shown in Table 1. The skeleton loss takes into account the spatial relationship between the above key points and the constraints of joint connection, which can help the method to accurately estimate the key positions and ensure that the relative distance between the key points conforms to the actual human body structure.

[0087] Table 1

[0088]

[0089] Adding the above five types of losses together, the final passenger posture estimation loss function in the rail transit scenario is as follows:

[0090]

[0091] In this embodiment, the improved rail transit station passenger behavior recognition algorithm in step 1 is different from other behavior recognition algorithms in terms of training set, and uses the collected rail transit station passenger behavior data set.

[0092] In this embodiment, the improved rail transit station passenger behavior recognition algorithm in step 1, in addition to the actual human body key point association relationship, that is, [(12,13), (13,0), (13,1), (0,1), (0,2), (1,3), (2,4), (3,5), (0,6), (1,7), (6,7), (6,8), (7,9), (8,10), (9,11)], adds the human hand key points and There are some connections between the key points of the legs, namely [(12,13),(13,0),(13,1),(0,1),(0,2),(1,3),(2,4),(3,5),(0,6),(1,7),(6,7),(6,8),(7,9),(8,10),(9,11),4,10),(11,5),(2,8),(3,9),(2,3),(4,5),(9,8),(10,11)].

[0093] Example 4

[0094] This embodiment 4 provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the above-mentioned method for identifying abnormal behavior of passengers in rail transit stations based on human posture is implemented. The method includes:

[0095] Obtain video images monitored by rail transit stations and extract each frame of the video;

[0096] The target detection model is used to detect the passengers in the video image, and the detection frame position information of each passenger target and the number of passengers in the current video are obtained. The image of each passenger in each frame is cropped according to the detection frame position information.

[0097] Use the pre-trained passenger posture estimation model to estimate the posture of the passenger image and complete the recognition of the key position information of the current passenger's body;

[0098] The Deepsort algorithm is used to track the passengers in the video images, and the passenger posture information in multiple consecutive frames is concatenated into posture sequence data based on the tracking results.

[0099] The positions of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated based on the posture sequence data, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information;

[0100] According to the obtained number of passengers, movement speed, behavior category information and behavior duration, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value.

[0101] Example 5

[0102] This embodiment 5 provides a computer device, including a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, the processor calls the program instructions to execute the above-mentioned method for identifying abnormal behavior of passengers in rail transit stations based on human body posture, the method comprising:

[0103] Obtain video images monitored by rail transit stations and extract each frame of the video;

[0104] The target detection model is used to detect the passengers in the video image, and the detection frame position information of each passenger target and the number of passengers in the current video are obtained. The image of each passenger in each frame is cropped according to the detection frame position information.

[0105] Use the pre-trained passenger posture estimation model to estimate the posture of the passenger image and complete the recognition of the key position information of the current passenger's body;

[0106] The Deepsort algorithm is used to track the passengers in the video images, and the passenger posture information in multiple consecutive frames is concatenated into posture sequence data based on the tracking results.

[0107] The positions of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated based on the posture sequence data, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information;

[0108] According to the obtained number of passengers, movement speed, behavior category information and behavior duration, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value.

[0109] Example 6

[0110] This embodiment 6 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes instructions for implementing the above-mentioned method for identifying abnormal behavior of passengers in rail transit stations based on human body posture, the method comprising:

[0111] Obtain video images monitored by rail transit stations and extract each frame of the video;

[0112] The target detection model is used to detect the passengers in the video image, and the detection frame position information of each passenger target and the number of passengers in the current video are obtained. The image of each passenger in each frame is cropped according to the detection frame position information.

[0113] Use the pre-trained passenger posture estimation model to estimate the posture of the passenger image and complete the recognition of the key position information of the current passenger's body;

[0114] The Deepsort algorithm is used to track the passengers in the video images, and the passenger posture information in multiple consecutive frames is concatenated into posture sequence data based on the tracking results.

[0115] The positions of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated based on the posture sequence data, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information;

[0116] According to the obtained number of passengers, movement speed, behavior category information and behavior duration, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value.

[0117] In summary, the video-based method for identifying abnormal behaviors of passengers in rail transit stations described in the embodiment of the present invention. First, based on the monitoring of the rail transit station, the target image of passengers in the area of ​​interest is obtained, and then the abnormal behavior identification algorithm of passengers in the rail transit station is used to identify the abnormal behavior of passengers in the rail transit station, and finally the safety status of passengers in the rail transit station is determined and an early warning is issued. Compared with other methods, the present invention proposes an abnormal behavior identification algorithm for passengers in rail transit stations, which completes the identification of the behavior of passengers in rail transit stations according to the monitoring requirements of rail transit stations, only detects the abnormal behavior of passengers in rail transit stations, and determines the safety status level of passengers in rail transit stations. The detection accuracy and speed of abnormal behaviors of passengers in rail transit stations are improved, and it is more suitable for the application requirements of abnormal behaviors of passengers in rail transit stations.

[0118] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0120] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0121] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0122] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative work on the basis of the technical solution disclosed in the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying abnormal behavior of passengers in rail transit stations, characterized in that: include: Obtain video images monitored by rail transit stations and extract each frame of the video; The target detection model is used to detect the passengers in the video image, and the detection frame position information of each passenger target and the number of passengers in the current video are obtained. The image of each passenger in each frame is cropped according to the detection frame position information. Use the pre-trained passenger posture estimation model to estimate the posture of the passenger image and complete the recognition of the key position information of the current passenger's body; The Deepsort algorithm is used to track the passengers in the video images, and the passenger posture information in multiple consecutive frames is concatenated into posture sequence data based on the tracking results. The positions of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips are calculated based on the posture sequence data, and the posture sequence data is input into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information; According to the obtained number of passengers, movement speed, behavior category information and behavior duration, the safety status of the passengers in the rail transit station is analyzed to obtain the safety status value.

2. The method for identifying abnormal behavior of passengers in rail transit stations according to claim 1, characterized in that: The training of the pre-trained passenger posture estimation model includes: inputting the cropped passenger image into the MobilenetV2 network for feature extraction to obtain the passenger's center point heat map and key point heat map information; after determining the center position of the character, predicting the regression values ​​of other skeletal key points, obtaining a preliminary estimate of the coordinates of each key point, calculating the position of the maximum value in the key point heat map as the final key point estimation position, and adding a local 2D offset for coordinate correction.

3. The method for identifying abnormal behavior of passengers in rail transit stations according to claim 1, characterized in that: The training of the pre-trained passenger behavior recognition model includes: constructing a passenger behavior dataset for rail transit stations and marking the behavior data of passengers in rail transit stations; calculating the average speed of the current passenger's head, shoulders, elbows and hips in multiple consecutive frames; inputting the key point coordinate information of multiple consecutive frames of posture into the spatiotemporal graph convolutional network, and calculating the key point spatial feature information of the current passenger in multiple consecutive frames; inputting the coordinate transformation information of the same key points of adjacent frames in multiple consecutive frames into the spatiotemporal graph convolutional network, and calculating the key point temporal feature information of the current passenger in multiple consecutive frames; fusing the spatial feature information and the temporal feature information of the passenger, and inputting them into the fully connected layer and the sigmoid activation function to obtain the behavior category of the passenger.

4. The method for identifying abnormal behavior of passengers in rail transit stations according to claim 1, characterized in that: According to the obtained number of passengers n, average speed v, behavior category information c and behavior duration t, the safety status of passengers in rail transit stations is analyzed to obtain the safety status value M:

5. The method for identifying abnormal behavior of passengers in rail transit stations according to claim 2, characterized in that: The training of the pre-trained passenger posture estimation model uses five loss functions, namely center point heat map loss, key point heat map loss, key point regression loss, local offset loss and skeleton loss.

6. The method for identifying abnormal behavior of passengers in rail transit stations according to claim 5, characterized in that: For the center point heat map loss, the weighted mean square error loss is used; the key point heat map loss and the center point heat map loss use the weighted mean square error loss; the spatial resolution of the feature map output by the network becomes one-fourth of the original input image, which is equivalent to a 4x4 area of ​​the original image corresponding to a pixel point on the output feature map, which will bring a large error, so the local offset and local offset loss are introduced. The offset refers to the difference between the key point coordinates and the center point coordinates; the bone loss is to calculate the difference between the length of the skeleton composed of the predicted key points and the length of the corresponding real key points.

7. A rail transit station passenger abnormal behavior recognition system, characterized in that: include: An acquisition module is used to acquire video images monitored by rail transit stations and extract each frame of the video; The detection module is used to detect the passengers in the video image using the target detection model, obtain the detection frame position information of each passenger target and the number of passengers in the current video, and crop each passenger image in each frame according to the detection frame position information; The posture estimation module is used to estimate the posture of the passenger image using the pre-trained passenger posture estimation model to complete the recognition of the key position information of the current passenger's body; The target tracking module is used to track the passengers in the video image using the Deepsort algorithm, and concatenate the passenger posture information in multiple consecutive frames into posture sequence data based on the tracking results; The behavior recognition module is used to calculate the position of the current passenger's head, shoulders, elbows and hips and the movement speed information of the head, shoulders, elbows and hips based on the posture sequence data, and input the posture sequence data into a pre-trained passenger behavior recognition model to obtain the passenger's behavior category information; The analysis module is used to analyze the safety status of passengers in rail transit stations and obtain a safety status value based on the obtained number of passengers, movement speed, behavior category information and behavior duration.

8. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for identifying abnormal behavior of passengers in a rail transit station based on human posture as described in any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method for identifying abnormal behavior of passengers in rail transit stations based on human posture as described in any one of claims 1-6.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the method for identifying abnormal behavior of passengers in rail transit stations based on human posture as described in any one of claims 1-6.

Images