Gate turnstile fare evasion recognition method, device, computer equipment and storage medium

Abstract

Embodiments of the present invention disclose a method, device, computer device and storage medium for detecting fare evasion at a turnstile. The method includes: obtaining a video captured by a camera and processing the video to obtain a picture to be detected; inputting the picture to be detected into a picture model for human key point detection to obtain a human body frame and a human key point prediction map; inputting the human body frame into a tracking model for target tracking to obtain a tracking result; generating a posture key point sequence video according to the tracking result in combination with the human key point prediction map; inputting the posture key point sequence video into a behavior recognition model for fare evasion behavior detection to obtain a detection result; when the detection result is a fare evasion behavior, generating a warning message and sending the warning message to a terminal. By implementing the method of the embodiments of the present invention, real-time detection of fare evasion behavior can be achieved, without the need for infrared imaging technology for target positioning, saving a large amount of costs and having a high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to a fare evasion recognition method, and more specifically to a fare evasion recognition method, device, computer device and storage medium for turnstiles. Background Art

[0002] Currently, rail transit such as subways, as an important and convenient means of transportation, is favored by more and more people. Generally, people need to buy tickets before taking the subway and pass through the turnstiles for ticket checking when entering the station. However, there are many passengers who evade fares every day, for example, by jumping over the turnstiles, which causes great economic losses to the subway company.

[0003] Chinese Patent CN201910010440.8 discloses a subway fare evasion behavior detection method and system based on infrared thermal imaging, including the following steps: detecting whether a pedestrian enters the pedestrian detection range of the turnstile image; using an automatically updated background for infrared thermal imaging image background difference to extract the pedestrian infrared thermal imaging top view image; performing morphological processing on the extracted pedestrian infrared thermal imaging top view image to obtain a binary top view image of the pedestrian passing through the turnstile opening based on an appropriately automatically updated threshold; performing parallel extraction of regions of interest on the binary pedestrian top view image, setting the number of ROI regions to be the same as the number of turnstile openings (N), and obtaining N independent binary turnstile channel pedestrian top view images; performing connected component labeling on the N independent binary turnstile channel pedestrian top view Figure 2 value images respectively to obtain pedestrian parameters; determining the fare evasion behavior at N turnstile openings. Using this invention can effectively identify and avoid fare evasion behavior. However, this method has a relatively high cost for fare evasion detection based on infrared thermal imaging and requires a fixed camera, which is very unfriendly for project deployment, has certain limitations, and is based on the binary turnstile channel pedestrian top view Figure 2Judging the value image is often greatly affected by other environments such as light, and it is impossible to guarantee the accuracy of the effect. Chinese Patent CN201911224925.3 provides a subway passenger fare evasion detection system and method. The system specifically includes a passenger information marking module, which identifies and marks whether the passenger is a passenger who needs to buy a ticket to take the train, whether the passenger is carrying a baby, captures the face information, and stores the passenger's dynamic boarding information; a card swiping behavior recognition module, which judges whether the passenger swipes the card based on whether there is an intersection between the human body bone movement trajectory based on depth camera and the card swiping area; a card swiping information reading module, which reads the card swiping information and time of the turnstile. If the card swiping behavior is implemented and the card swiping is successful, it is determined that the card swiping is successful; a fare evasion behavior determination and warning module, which combines the passenger ticket purchase marking information, card swiping behavior recognition information, card swiping success record and the number of passing people to identify the fare evasion behavior and give a warning; this method is based on shooting by a depth camera, with a relatively high cost, and this system needs to combine the card swiping information reading module to obtain the fare evasion behavior, and cannot directly judge based on image information, which has certain limitations. Chinese Patent CN201510144081.7 discloses a turnstile detection system and method. The system includes: a three-dimensional image information acquisition module, which includes at least two image data acquisition devices for collecting two-dimensional image information of the same area to be detected from different positions, and is used to obtain the image information of the human body in the area to be detected; a gate state acquisition module, which is used to obtain the gate state information of the turnstile; a three-dimensional image information recognition and processing module, which is used to judge whether the human body has a fare evasion behavior by using the image information of the human body and the state information of the gate; an alarm module, which is used to give an alarm when the human body has a fare evasion behavior; this method requires a three-dimensional image information acquisition module with a relatively low cost, and needs to be based on the acquisition of the turnstile state, and cannot analyze only based on image information, and the deployment project is relatively complex. Chinese Patent CN202110192793.1 relates to a subway turnstile passage fare evasion method based on rapid estimation of passenger posture. The identification is carried out through the following steps. First, the subway turnstile video monitoring information is collected, then the subway passenger bone key points are detected, and finally the fare evasion behavior of the passengers passing through the turnstile is identified. However, this method only judges the fare evasion behavior of passengers based on the key point information. This method cannot obtain the continuous characteristics of the fare evasion behavior of passengers, and the lack of identification information affects the judgment of the fare evasion behavior of passengers. Secondly, relying solely on the key point information for logical judgment is often affected by the space captured by the camera, resulting in inaccurate identification.

[0004] Therefore, it is necessary to design a new method to realize real-time detection of fare evasion behavior, without using infrared imaging technology for target positioning, saving a large amount of costs and having a high recognition accuracy. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a turnstile fare evasion recognition method, device, computer device and storage medium.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A fare evasion recognition method for a turnstile, comprising:

[0007] Obtain the video captured by the camera and process the video to obtain a picture to be detected;

[0008] Input the picture to be detected into a picture model for human key point detection to obtain a human body frame and a human key point prediction map;

[0009] Input the human body frame into a tracking model for target tracking to obtain a tracking result;

[0010] Generate a posture key point sequence video according to the tracking result in combination with the human key point prediction map;

[0011] Input the posture key point sequence video into a behavior recognition model for fare evasion behavior detection to obtain a detection result;

[0012] When the detection result is a fare evasion behavior, generate a warning message and send the warning message to the terminal.

[0013] A further technical solution thereof is: The picture model is to add a key point mask prediction branch on the basis of the yolov5 model.

[0014] A further technical solution thereof is: The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within a continuous specified number of frames, the target is tracked by optical flow several frames in advance from the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for target tracking.

[0015] A further technical solution thereof is: The generating of the posture key point sequence video according to the tracking result in combination with the human key point prediction map includes:

[0016] Generate a corresponding posture key map from the human key point prediction map through yolov5-mp;

[0017] Determine the posture key map with an action trajectory according to the tracking result to obtain a target posture key map;

[0018] Merge the target posture key maps according to time to generate a posture key point sequence video.

[0019] A further technical solution thereof is: The behavior recognition model is formed by modifying the 3D convolution into a variable 3D convolution and adding a fast branch in parallel on the basis of the MoviNet model.

[0020] Its further technical solution is that the fast branch uses 3D convolution of the spatio-temporal context model to sample and classify the video of the pose key point sequence.

[0021] Its further technical solution is that the fare evasion behaviors include squatting, jumping and trailing behaviors.

[0022] The present invention also provides a fare evasion recognition device for a turnstile, including:

[0023] A sampling unit, configured to acquire the video captured by the camera and process the video to obtain a picture to be detected;

[0024] A key point detection unit, configured to input the picture to be detected into a picture model for human key point detection to obtain a human body frame and a human key point prediction map;

[0025] A tracking unit, configured to input the human body frame into a tracking model for target tracking to obtain a tracking result;

[0026] A video generation unit, configured to generate a pose key point sequence video according to the tracking result and the human key point prediction map;

[0027] A behavior detection unit, configured to input the pose key point sequence video into a behavior recognition model for fare evasion behavior detection to obtain a detection result;

[0028] An alarm unit, configured to generate a warning message when the detection result is a fare evasion behavior and send the warning message to the terminal.

[0029] The present invention also provides a computer device, which includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the above method is implemented.

[0030] The present invention also provides a storage medium, which stores a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0031] The beneficial effects of the present invention compared with the prior art are as follows: By acquiring the image captured by the camera and performing image cutting, the present invention inputs the generated picture to be detected into a picture model for human key point detection, combines a tracking model for target tracking, generates a pose key point sequence video, performs fare evasion behavior detection according to the pose key point sequence video, and gives an alarm when there is a fare evasion behavior, so as to realize real-time detection of fare evasion behaviors, without using infrared imaging technology for target positioning, saving a large amount of costs and having a high recognition accuracy.

[0032] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Description of the Drawings

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0034] Figure 1 Schematic diagram of the application scenario of the gate fare evasion recognition method provided by the embodiment of the present invention;

[0035] Figure 2 Schematic flowchart of the gate fare evasion recognition method provided by the embodiment of the present invention;

[0036] Figure 3 Schematic sub - flowchart of the gate fare evasion recognition method provided by the embodiment of the present invention;

[0037] Figure 4 Schematic block diagram of the gate fare evasion recognition device provided by the embodiment of the present invention;

[0038] Figure 5 Schematic block diagram of the video generation unit of the gate fare evasion recognition device provided by the embodiment of the present invention;

[0039] Figure 6 Schematic block diagram of the computer device provided by the embodiment of the present invention. Detailed Embodiments

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0041] It should be understood that when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0042] It should also be understood that the terms used in the specification of the present invention are merely for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0043] It should be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0044] Please refer to Figure 1 and Figure 2 , Figure 1 which is a schematic diagram of the application scenario of the gate fare evasion recognition method provided by the embodiment of the present invention. Figure 2 which is a schematic flowchart of the gate fare evasion recognition method provided by the embodiment of the present invention. This gate fare evasion recognition method is applied to a server, which interacts with a terminal and a camera for data. By obtaining the video captured by the camera and processing the video to form a picture to be detected, and using a picture model to perform human key point detection and a tracking model to perform target tracking, and then generating a posture key point sequence video, and inputting the posture key point sequence video into a behavior recognition model to recognize fare evasion behavior. When there is a fare evasion behavior, a warning message is generated and sent to the terminal for prompting; it can recognize the fare evasion behavior of passengers only relying on an optical camera, without fixed shooting restrictions on the camera, and only need to shoot the gate position to detect multiple people's fare evasion at the same time. The micro model adopted is very friendly to the real-time effect and can alarm the fare evasion behavior of passengers in time with high recognition accuracy.

[0045] Figure 2 is a schematic flowchart of the gate fare evasion recognition method provided by the embodiment of the present invention. As Figure 2 shown, the method includes the following steps S110 to S160.

[0046] S110. Obtain the video captured by the camera and process the video to obtain a picture to be detected.

[0047] In this embodiment, the picture to be detected refers to the picture within the set gate range captured by the camera.

[0048] Specifically, aim the lens of the optical camera at the passenger card swiping gate area, monitor this area based on the video. Here, the optical camera uses a fixed focal length, fixed angle and position, and the captured video is cut into pictures. Here, every 2 frames are cut to generate frame pictures with a certain time sequence, that is, the pictures to be detected.

[0049] Only relying on an optical camera, without the need for infrared imaging technology for target positioning, which saves a large amount of costs.

[0050] S120. Input the image to be detected into the image model for human key point detection to obtain a human body frame and a human key point prediction map.

[0051] In this embodiment, the human body frame refers to a positioning frame formed by the coordinate positions of a passenger in the image to be detected.

[0052] The image model adds a key point mask prediction branch based on the yolov5 model. This image model can support tasks of both target detection and key point detection, and is carried out in a top-down manner. The predicted regression box branch of the yolov5 model is used to locate the spatial coordinates of the human target, and then the positions and types of the key points inside it are predicted based on this coordinate position.

[0053] Specifically, for targets of the type of passengers, whose size is medium to large, the output information of the N3 layer, N4 layer, and N5 layer of the PANet network is selected in the yolov5 model, and the human key point prediction map is predicted through the key point mask prediction branch. To improve the model speed, spatial information is extracted based on the N3 layer, and semantic information is extracted based on the N5 layer. The features within the target area are intercepted from their respective feature layers through the ROI Align mechanism and the SPP method. The features intercepted from the N5 layer are upsampled through a dilated convolution group and a transposed convolution group, the features intercepted from the N3 layer are downsampled through a deformable convolution group, and the features intercepted from the N4 layer are convolved through a two-dimensional convolution group. Here, the convolution group includes multiple convolution operations, activation operations, where the Mish activation function is used, batch normalization, etc. Finally, the convolution output results of the three layers are NF3, NF4, and NF5, ensuring that they have the same feature layer size. Then, these three output results are Concated to merge the three output results to obtain the feature PE that is simultaneously used for rich semantic information and spatial information. Next, the CBAM attention mechanism operation is performed on the feature PE to make the number of its channel numbers equal to the number of key point categories. Here, the key points are key points such as the left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right waists, left and right knees, left and right ankles, etc. Finally, the feature map is restored to the original image target size through the transposed convolution of the transposed convolution and the final human key point prediction map is obtained through the activation function; here, the sigmoid activation function is used for normalization, and for each category, the key point with the highest probability is selected, and at the same time, the key points with lower probabilities are filtered out through a threshold.

[0054] The prediction and training of the image model are different for the selection of the target position in key point prediction. In prediction, the position of the target is selected based on the position of the prediction box. In training, the position of the target is selected through the information of the annotation box, and finally input into the key point detection branch to predict the position and corresponding category of the key points. The image model uses the CIOU loss function for object detection. Considering the sparse density of people in the turnstile area, the Fast-NMS process is adopted to speed up the inference of the model. For the object classification loss, the focal loss is used to reduce the impact caused by the imbalance between the foreground and background. The loss function used for key point detection is Adaloss.

[0055] First, the turnstile card swiping area is set, and detection is performed within this area. The image model mentioned above is used to predict the human body box and the corresponding human key point information, that is, the human key point prediction map, for the collected samples.

[0056] S130: Input the human body box into the tracking model for object tracking to obtain the tracking result.

[0057] In this embodiment, the tracking result refers to the movement trajectory of the human body, that is, the behavior trajectory of the passenger.

[0058] The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within a continuous specified number of frames, the target is tracked by optical flow a certain number of frames ahead from the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for object tracking.

[0059] Specifically, the tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within 5 consecutive frames, the target is tracked by optical flow 10 frames ahead from the disappearance moment until the target is found within 5 frames, and then the Bytetrack model is continued to be used for object tracking.

[0060] The tracking model is based on the Bytetrack model. On this basis, for the targets missing in detection, the optical flow method is used for target trajectory prediction. Here, the optical flow method is an algorithm based on sparse optical flow. When the target is not detected within 5 consecutive frames during tracking based on the Bytetrack model, the target is subjected to optical flow tracking 10 frames in advance from the disappearance until the target is found within 5 frames, and then the tracking algorithm of the ByteTrack model is continued to make up for the poor tracking effect in the case of missed detection in target detection, improving the target recall rate and also improving the tracking effect. Especially in the case of crowded passengers, target occlusion often occurs, and this method can well solve the problem. Among them, for the internal IOU matching algorithm, the CIOU matching is improved here to further enhance the model effect.

[0061] Track the human body boxes predicted by the image model to obtain the action trajectories of each passenger and the index information of each passenger. If the distance between the central points of two passengers is lower than the threshold, the two passengers are combined and their trajectory information is retained separately. This is to detect the situation of trailing and fare evasion.

[0062] S140. Generate a posture key point sequence video according to the tracking result in combination with the human body key point prediction map.

[0063] In this embodiment, the posture key point sequence video refers to a time sequence video of key points with action trajectories.

[0064] In one embodiment, please refer to Figure 3 , the above step S140 may include steps S141 to S143.

[0065] S141. Generate a corresponding posture key map from the human body key point prediction map through yolov5-mp.

[0066] In this embodiment, the posture key map refers to a key point trajectory picture generated from the human body key point prediction map through yolov5-mp. I

[0067] Among them, yolov5-mp is a model improved based on the yolov5 model and added with a key point mask prediction branch.

[0068] S142. Determine the posture key map with an action trajectory according to the tracking result to obtain the target posture key map.

[0069] In this embodiment, the target posture key map refers to the posture key map with an action trajectory.

[0070] S143. Merge the target posture key maps by time to generate a posture key point sequence video.

[0071] Specifically, logical judgment is performed based on the movement trajectories of each passenger and combined passengers and the position of the turnstile to determine the process of the passenger from entering the turnstile to leaving the turnstile. During this process, the pose key points of each frame of each passenger, that is, the human key point prediction map, are recorded, and the corresponding KPP (Key Point Picture) is generated. The information of the key points is set to 1, and the information of non-key points is set to 0. Different categories of key points are represented by different channels. The pose key point maps of each passenger or combined passenger with a trajectory are merged by time to generate the pose key point sequence video KPP-Video. Behavior recognition is performed not only based on image information but also based on the sequence information between images in the video, greatly enhancing the effect of fare evasion behavior recognition.

[0072] S150. Input the pose key point sequence video into a behavior recognition model for fare evasion behavior detection to obtain a detection result.

[0073] In this embodiment, the detection result refers to the recognition result of whether there is a fare evasion behavior in the current picture to be detected. Behavior recognition is performed not only based on image information but also based on the sequence information between images in the video, greatly enhancing the detection accuracy.

[0074] The behavior recognition model is formed by modifying the 3D convolution to a variable 3D convolution on the basis of the MoviNet model and adding a fast branch in parallel. The fast branch uses the 3D convolution of the spatio-temporal context model to perform frame sampling and classification on the pose key point sequence video.

[0075] Specifically, the pose key point sequence video KPP-Video is used as the input of the behavior recognition model for behavior recognition. In order to make the behavior recognition model more robust, during the training process, the pose key point sequence video frames are randomly sampled at an interval of 1 - 3 frames for training. At the same time, the 3D convolution of the behavior recognition model is modified to a variable 3D convolution to enhance the adaptation of the behavior recognition model to irregular objects, especially having a good effect on non-grid square targets. A fast branch is added in parallel. The fast branch uses the 3D convolution of the spatio-temporal context model, that is, Spatio-Temporal FAST 3DConvolutions, to perform frame sampling and classification on the pose key point sequence video. The selected frame sampling interval here is 4 frames, making it easier for the behavior recognition model to understand the global information of the action and improving the prediction accuracy of the behavior recognition model. During training, some key points will be randomly erased, 2 - 4 key points each time, so that the behavior recognition model can adapt to partial occlusion situations.

[0076] S160. When the detection result is a fare evasion behavior, generate a warning message and send the warning message to the terminal.

[0077] The fare evasion behaviors include squatting, jumping, and trailing behaviors.

[0078] Specifically, the behaviors of passengers passing through the turnstile are classified into four categories: squatting, jumping, trailing, and passing. For single-person trajectories, squatting, jumping, and passing behaviors are predicted, and for multi-person combined trajectories, trailing and passing are predicted. If squatting, jumping, or trailing occurs at the turnstile, it is regarded as fare evasion, and a warning message is generated.

[0079] This embodiment can identify the fare evasion behaviors of passengers only relying on an optical camera, without fixed shooting restrictions on the camera. Only by shooting the position of the turnstile can multi-person fare evasion be detected simultaneously. The micro model adopted is very friendly to real-time effects and can alarm the fare evasion behaviors of passengers in a timely manner. It is improved on the basis of the existing model to form an image model, a tracking model, and a behavior recognition model, greatly improving the accuracy of fare evasion behavior detection.

[0080] The above turnstile fare evasion recognition method obtains the image captured by the camera and performs image cutting, inputs the generated image to be detected into the image model for human key point detection, combines the tracking model for target tracking, and generates a video of pose key point sequences. Fare evasion behaviors are detected according to the video of pose key point sequences. When there is a fare evasion behavior, an alarm is given, realizing real-time detection of fare evasion behaviors, without using infrared imaging technology for target positioning, saving a large amount of costs, and having a high recognition accuracy.

[0081] Figure 4 It is a schematic block diagram of a turnstile fare evasion recognition device 300 provided by an embodiment of the present invention. As Figure 4 shown, corresponding to the above turnstile fare evasion recognition method, the present invention also provides a turnstile fare evasion recognition device 300. The turnstile fare evasion recognition device 300 includes units for executing the above turnstile fare evasion recognition method, and this device can be configured in a server. Specifically, please refer to Figure 4 , the turnstile fare evasion recognition device 300 includes a sampling unit 301, a key point detection unit 302, a tracking unit 303, a video generation unit 304, a behavior detection unit 305, and an alarm unit 306.

[0082] The sampling unit 301 is configured to obtain the video captured by the camera and process the video to obtain the picture to be detected; the key point detection unit 302 is configured to input the picture to be detected into the picture model for human key point detection to obtain the human body frame and the human key point prediction map; the tracking unit 303 is configured to input the human body frame into the tracking model for target tracking to obtain the tracking result; the video generation unit 304 is configured to generate a posture key point sequence video according to the tracking result in combination with the human key point prediction map; the behavior detection unit 305 is configured to input the posture key point sequence video into the behavior recognition model for ticket evasion behavior detection to obtain the detection result; the alarm unit 306 is configured to generate a warning message when the detection result is a ticket evasion behavior and send the warning message to the terminal.

[0083] In one embodiment, as Figure 5 shown, the video generation unit 304 includes a key graph generation subunit 3041, a screening subunit 3042, and a merging subunit 3043.

[0084] The key graph generation subunit 3041 is configured to generate a corresponding posture key graph from the human key point prediction map through yolov5-mp; the screening subunit 3042 is configured to determine the posture key graphs with action trajectories according to the tracking result to obtain the target posture key graphs; the merging subunit 3043 is configured to merge the target posture key graphs by time to generate a posture key point sequence video.

[0085] It should be noted that those skilled in the art can clearly understand the specific implementation processes of the above-mentioned turnstile ticket evasion recognition device 300 and each unit, which can refer to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity of description, they will not be elaborated here.

[0086] The above-mentioned turnstile ticket evasion recognition device 300 can be implemented in the form of a computer program, and this computer program can run on a computer device as Figure 6 shown.

[0087] Please refer to Figure 6 , Figure 6 which is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 may be a server. Among them, the server may be an independent server or a server cluster composed of multiple servers.

[0088] Referring to Figure 6 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501. Among them, the memory may include a non-volatile storage medium 503 and an internal memory 504.

[0089] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, can cause the processor 502 to execute a fare evasion recognition method for a turnstile.

[0090] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0091] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, it can cause the processor 502 to execute a fare evasion recognition method for a turnstile.

[0092] The network interface 505 is used for network communication with other devices. Those skilled in the art can understand that Figure 6 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device 500 to which the solution of this application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0093] Among them, the processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0094] Obtain the video captured by the camera, and process the video to obtain the picture to be detected; input the picture to be detected into the picture model for human key point detection to obtain the human body frame and the human key point prediction map; input the human body frame into the tracking model for target tracking to obtain the tracking result; generate a posture key point sequence video according to the tracking result and the human key point prediction map; input the posture key point sequence video into the behavior recognition model for fare evasion behavior detection to obtain the detection result; when the detection result is a fare evasion behavior, generate a warning message and send the warning message to the terminal.

[0095] Among them, the picture model adds a key point mask prediction branch on the basis of the yolov5 model.

[0096] The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within a continuously specified number of frames, the target is tracked by optical flow several frames ahead of the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for target tracking.

[0097] The described behavior recognition model is formed by modifying the 3D convolution in the MoviNet model into a deformable 3D convolution and adding a fast branch in parallel.

[0098] The fast branch uses 3D convolution of the spatio-temporal context model to extract and sample frames for classification of the video of the pose key point sequence.

[0099] The fare evasion behaviors include squatting, jumping, and trailing behaviors.

[0100] In one embodiment, when the processor 502 implements the step of generating a video of the pose key point sequence based on the tracking result and the human key point prediction map, the specific implementation steps are as follows:

[0101] Generate a corresponding pose key map from the human key point prediction map through yolov5-mp; determine the pose key map with an action trajectory according to the tracking result to obtain the target pose key map; merge the target pose key map by time to generate a video of the pose key point sequence.

[0102] It should be understood that in the embodiments of the present application, the processor 502 may be a central processing unit (CPU), and this processor 502 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0103] Those of ordinary skill in the art can understand that all or part of the processes in the methods of implementing the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, and this storage medium is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0104] Therefore, the present invention also provides a storage medium. This storage medium may be a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the processor is caused to execute the following steps:

[0105] Obtain the video captured by the camera and process the video to obtain the picture to be detected; input the picture to be detected into the picture model for human key point detection to obtain the human frame and the human key point prediction map; input the human frame into the tracking model for target tracking to obtain the tracking result; generate a pose key point sequence video according to the tracking result in combination with the human key point prediction map; input the pose key point sequence video into the behavior recognition model for ticket evasion behavior detection to obtain the detection result; when the detection result is a ticket evasion behavior, generate a warning message and send the warning message to the terminal.

[0106] Among them, the picture model adds a key point mask prediction branch on the basis of the yolov5 model.

[0107] The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within a continuously specified number of frames, the target is tracked by optical flow several frames ahead of the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for target tracking.

[0108] The behavior recognition model is formed by modifying the 3D convolution to a variable 3D convolution and adding a fast branch in parallel on the basis of the MoviNet model.

[0109] The fast branch uses the 3D convolution of the spatio-temporal context model to extract frames and classify the pose key point sequence video.

[0110] The ticket evasion behaviors include squatting, jumping and trailing behaviors.

[0111] In an embodiment, when the processor executes the computer program to implement the step of generating the pose key point sequence video according to the tracking result in combination with the human key point prediction map, the following steps are specifically implemented:

[0112] Generate the corresponding pose key map from the human key point prediction map through yolov5-mp; determine the pose key map with an action trajectory according to the tracking result to obtain the target pose key map; merge the target pose key map by time to generate the pose key point sequence video.

[0113] The storage medium can be various computer-readable storage media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disc that can store program codes.

[0114] Those of ordinary skill in the art will appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of the examples have been generally described in terms of function in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0115] In several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0116] The steps in the method embodiments of the present invention can be adjusted, combined, and deleted according to actual needs. The units in the apparatus embodiments of the present invention can be combined, divided, and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0118] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. Gate turnstile fare evasion recognition method, characterized in that Including: Obtain the video captured by the camera and process the video to obtain the picture to be detected; Input the picture to be detected into the picture model for human key point detection to obtain the human body frame and the human key point prediction map; Input the human body frame into the tracking model for target tracking to obtain the tracking result; Generate a posture key point sequence video according to the tracking result combined with the human key point prediction map; Input the posture key point sequence video into the behavior recognition model for ticket evasion behavior detection to obtain the detection result; When the detection result is a ticket evasion behavior, generate a warning message and send the warning message to the terminal; The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within a continuously specified number of frames, the target is tracked by optical flow several frames in advance from the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for target tracking.

2. The ticket evasion recognition method for turnstiles according to claim 1, wherein, The picture model adds a key point mask prediction branch on the basis of the yolov5 model.

3. The gate fare evasion recognition method according to claim 1, wherein The generating the posture key point sequence video according to the tracking result combined with the human key point prediction map includes: Generate the corresponding posture key graph from the human key point prediction map through yolov5-mp; Determine the posture key graph with an action trajectory according to the tracking result to obtain the target posture key graph; Merge the target posture key graphs by time to generate a posture key point sequence video.

4. The method for identifying fare evasion at a turnstile according to claim 1, wherein, The behavior recognition model is formed by modifying the 3D convolution to a variable 3D convolution and adding a fast branch in parallel on the basis of the MoviNet model.

5. The gate fare evasion recognition method according to claim 4, wherein The fast branch uses the 3D convolution of the spatio-temporal context model to extract frames and classify the posture key point sequence video.

6. The method for identifying fare evasion at a turnstile according to claim 1, characterized in that, The ticket evasion behaviors include squatting, jumping and trailing behaviors.

7. Gate turnstile fare evasion recognition device, characterized in that, Including: A sampling unit for obtaining the video captured by the camera and processing the video to obtain the picture to be detected; A key point detection unit for inputting the picture to be detected into the picture model for human key point detection to obtain the human body frame and the human key point prediction map; A tracking unit for inputting the human body frame into the tracking model for target tracking to obtain the tracking result; A video generation unit for generating a posture key point sequence video according to the tracking result combined with the human key point prediction map; A behavior detection unit for inputting the posture key point sequence video into the behavior recognition model for ticket evasion behavior detection to obtain the detection result; An alarm unit for generating a warning message and sending the warning message to the terminal when the detection result is a ticket evasion behavior; The tracking model is formed by adding the sparse optical flow method on the basis of the Bytetrack model. Among them, when the tracking model uses the Bytetrack model to track the target and the target is not detected within continuously specified number of frames, the target is tracked by optical flow several frames earlier from the disappearance moment until the target is found within the specified number of frames, and then the Bytetrack model is continued to be used for target tracking.

8. A computer device, characterized in that, The computer device includes a memory and a processor. A computer program is stored on the memory, and when the processor executes the computer program, the method described in any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

Images