Gate machine fare evasion detection method, device, computer equipment and storage medium

Through the processing of camera video and deep learning model, real-time and accurate identification of subway ticket evasion detection is achieved, solving the problems of high cost and low accuracy of the existing technology, and providing an efficient ticket evasion detection solution.

CN114549867BActive Publication Date: 2024-11-22SHENZHEN SUNWIN INTELLIGENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210151285.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-16
Publication Date
2024-11-22
Estimated Expiration
2042-02-16

AI Technical Summary

Technical Problem

The existing subway ticket evasion detection methods are costly, highly affected by the environment, and have low accuracy, so they cannot achieve real-time and accurate ticket evasion detection.

Method used

The gate ticket evasion detection method based on camera video is adopted, and through human instance segmentation, key point detection, target tracking and behavior recognition models, mask posture key point sequence video is generated for ticket evasion behavior detection, real-time alarm is achieved.

Benefits of technology

It does not require infrared imaging technology, saves costs, has high recognition accuracy, can detect ticket evasion behavior in real time, is suitable for multiple people to detect simultaneously, and has good real-time and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114549867B_ABST
    Figure CN114549867B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention discloses a method, device, computer equipment and storage medium for detecting ticket evasion at a gate. The method comprises: obtaining a video shot by a camera, and processing the video to obtain a picture to be detected; inputting the picture to be detected into a picture model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; inputting the key point prediction map and the human mask prediction map into a tracking model for target tracking to obtain a tracking result; generating a mask posture key point sequence video according to the tracking result; inputting the mask posture key point sequence video into a behavior recognition model for ticket evasion behavior detection to obtain a detection result; when the detection result is a ticket evasion behavior, generating a warning message, and sending the warning message to a terminal. By implementing the method of the embodiment of the present invention, real-time ticket evasion behavior detection can be achieved without the need for infrared imaging technology for target positioning, saving a lot of costs, and having a high recognition accuracy rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a fare evasion detection method, and more specifically to a gate fare evasion detection method, device, computer equipment and storage medium. Background Art

[0002] Currently, subway and other rail transportation are becoming more and more popular as an important and convenient means of transportation. Generally, people need to buy tickets before taking the subway and pass through the gate for ticket inspection when entering the station. However, many passengers evade fares every day, such as by crossing the gate, which brings great economic losses to the subway company.

[0003] Chinese patent CN201910010440.8 discloses a method and system for detecting subway fare evasion based on infrared thermal imaging, which includes the following steps: detecting whether a pedestrian has entered the pedestrian detection range of the gate image; performing background difference of the infrared thermal imaging image using the automatically updated background to extract the infrared thermal imaging overhead view image of the pedestrian; performing morphological processing on the extracted infrared thermal imaging overhead view image of the pedestrian to obtain a binary overhead view image of the pedestrian passing through the gate based on the automatically updated appropriate threshold; performing parallel extraction of sub-regions of interest on the binary pedestrian overhead view image, setting the ROI area to be the same as the number of gates (N), and obtaining N mutually independent binary overhead views of pedestrians in the gate channels; performing N mutually independent binary overhead views of pedestrians in the gate channels Figure 2 The connected areas of the binary image are marked to obtain pedestrian parameters; the fare evasion behavior at N gates is determined. The invention can effectively identify and avoid fare evasion behavior. However, this method is based on infrared thermal imaging for fare evasion detection, which is very costly and requires a fixed camera. This is very unfriendly to project deployment and has certain limitations. It is also based on the overhead view of pedestrians in the binary gate channel. Figure 2The judgment based on the value image is often greatly affected by other environments such as lighting, and the accuracy of the effect cannot be guaranteed. Chinese patent CN201911224925.3 provides a subway passenger fare evasion behavior detection system and method, which specifically includes a passenger information marking module, which identifies and marks whether the passenger is a passenger who needs to buy a ticket to ride, whether he is carrying a baby, and captures facial information, and stores the passenger's dynamic riding information; the card swiping behavior recognition module, based on the intersection of the human skeleton motion trajectory and the card swiping area obtained by the depth camera, determines whether the passenger swipes the card; the card swiping information reading module, reads the gate card swiping information and time, and determines that the card swiping is successful when the card swiping behavior is implemented; the fare evasion behavior judgment and warning module, combined with the passenger ticket purchase mark information, card swiping behavior recognition information, card swiping success record and the number of people passing to identify the fare evasion behavior and warn; this method is based on the depth camera for shooting, the cost is relatively high, and the system needs to combine the card swiping information reading to touch and see to obtain the fare evasion behavior, and cannot directly judge the image information, which has certain limitations. Chinese patent CN201510144081.7 discloses a gate detection system and method, which includes: a three-dimensional image information acquisition module, including at least two image data acquisition devices for acquiring two-dimensional image information of the same area to be detected from different positions, for acquiring image information of a human body in the area to be detected; a gate status acquisition module, for acquiring gate status information of the gate; a three-dimensional image information recognition and processing module, for determining whether a human body has evaded the ticket by using the image information of the human body and the status information of the gate; an alarm module, for sounding an alarm when the human body has evaded the ticket; this method requires the use of a three-dimensional image information acquisition module, which is relatively low in cost, and needs to be based on the acquisition of the gate status, and cannot be analyzed based solely on image information, and the deployment project is relatively complex. Chinese patent CN202110192793.1 relates to a method for subway gate fare evasion based on rapid estimation of passenger posture, which is identified through the following steps: first, information is collected through subway gate video surveillance, then the subway passenger skeleton key points are detected, and finally the fare evasion behavior of passengers passing through the Forbidden City gate is identified. However, this method only judges the passenger's fare evasion behavior based on key point information. This method cannot obtain the continuous characteristics of the passenger's fare evasion behavior. The lack of identification information affects the judgment of the passenger's fare evasion behavior. Secondly, logical judgment is only based on key point information, which is often affected by the space captured by the camera, resulting in inaccurate recognition.

[0004] Therefore, it is necessary to design a new method to achieve real-time detection of fare evasion, without the need for infrared imaging technology for target positioning, saving a lot of costs and with high recognition accuracy. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a gate fare evasion detection method, device, computer equipment and storage medium.

[0006] To achieve the above object, the present invention adopts the following technical solution: a gate fare evasion detection method, comprising:

[0007] Obtain the video captured by the camera and process the video to obtain the image to be detected;

[0008] Inputting the image to be detected into the image model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map;

[0009] Input the key point prediction map and the human body mask prediction map into the tracking model to perform target tracking to obtain the tracking result;

[0010] Generate a mask posture key point sequence video according to the tracking result;

[0011] Inputting the mask posture key point sequence video into the behavior recognition model to detect the ticket evasion behavior to obtain the detection result;

[0012] When the detection result is a fare evasion behavior, a warning message is generated and sent to the terminal.

[0013] Its further technical solution is: the image model is formed by adding a mask branch based on the yolov5 model, and the CIOU loss function is used for target detection, the focal loss function is used for target classification, the dice loss function is used for semantic segmentation, and the Adaloss loss function is used for key point detection.

[0014] A further technical solution is: the image to be detected is input into the image model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map, including:

[0015] The image to be detected is input into the image model, and the output information of the N3, N4 and N5 layers of the PANet network is selected from the yolov5 model of the image model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map.

[0016] Its further technical solution is: the tracking model is obtained by training the SimMask-Transformer network through a number of human mask prediction images with target detection frame labels.

[0017] A further technical solution is: generating a mask posture key point sequence video according to the tracking result includes:

[0018] The key point prediction map in the tracking result is used to generate a corresponding posture key point prediction map through yolov5-mp;

[0019] The human body mask prediction image and the posture key point prediction image in the tracking result are directly spliced ​​and merged by original features to form a fused image;

[0020] The fused images are merged in time to generate a mask pose key point sequence video.

[0021] Its further technical solution is: the behavior recognition model is obtained by training the 3D Transformer-TIN model through a number of mask posture key point sequence videos of ticket evasion behavior labels.

[0022] A further technical solution is: the ticket evasion behavior labels include squatting, jumping and following behavior labels.

[0023] The present invention also provides a gate fare evasion detection device, comprising:

[0024] A sampling unit, used to obtain a video shot by a camera and process the video to obtain a picture to be detected;

[0025] A segmentation detection unit, used for inputting the image to be detected into the image model to perform human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map;

[0026] A tracking unit, used for inputting the key point prediction map and the human body mask prediction map into the tracking model to perform target tracking to obtain a tracking result;

[0027] A video generation unit, used for generating a mask posture key point sequence video according to the tracking result;

[0028] A behavior detection unit, used for inputting the mask posture key point sequence video into the behavior recognition model to detect the fare evasion behavior to obtain a detection result;

[0029] The warning unit is used to generate a warning message when the detection result is a fare evasion behavior, and send the warning message to the terminal.

[0030] The present invention further provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the above method when executing the computer program.

[0031] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention obtains images taken by a camera and performs image cutting, so as to input the generated image to be detected into an image model for human instance segmentation and key point detection, combines the tracking model for target tracking, and generates a mask posture key point sequence video, and performs ticket evasion detection based on the mask posture key point sequence video. When ticket evasion occurs, an alarm is issued to realize real-time ticket evasion detection, without the need for infrared imaging technology for target positioning, thereby saving a lot of costs and having a high recognition accuracy.

[0033] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0035] Figure 1 A schematic diagram of an application scenario of the gate fare evasion detection method provided by an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the flow chart of a gate fare evasion detection method provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of a sub-process of a gate fare evasion detection method provided by an embodiment of the present invention;

[0038] Figure 4 A schematic block diagram of a gate fare evasion detection device provided by an embodiment of the present invention;

[0039] Figure 5 A schematic block diagram of a video generation unit of a gate fare evasion detection device provided by an embodiment of the present invention;

[0040] Figure 6 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0043] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0044] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0045] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the gate fare evasion detection method provided in an embodiment of the present invention. Figure 2 A schematic flow chart of a gate fare evasion detection method provided in an embodiment of the present invention. The gate fare evasion detection method is applied in a server. The server exchanges data with an optical camera and a terminal, obtains a video shot by a camera, processes the video to form a picture to be detected, and uses a picture model to perform human instance segmentation and key point detection, and a tracking model to perform target tracking, and then generates a mask posture key point sequence video, and inputs the mask posture key point sequence video into a behavior recognition model to perform fare evasion behavior recognition. When there is fare evasion behavior, a warning message is generated and sent to the terminal for prompting; the fare evasion behavior of passengers can be identified by relying solely on an optical camera; there is no need to have a fixed shooting limit on the camera, and multiple people can be detected by shooting the gate position at the same time; the adopted miniature model is very friendly to the real-time effect, and can promptly alarm passengers for fare evasion behavior, and the detection accuracy is high.

[0046] Figure 2 1 is a flow chart of a gate fare evasion detection method provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S160.

[0047] S110, obtaining a video captured by a camera, and processing the video to obtain a picture to be detected.

[0048] In this embodiment, the image to be detected refers to an image within a set gate range taken by a camera.

[0049] Specifically, an optical camera is used to aim the camera lens at the passenger card swiping gate area, and the area is monitored based on video. The camera adopts a fixed focal length here, and the captured video is cut into frames to generate the image to be detected. Here, the image is cut every 2 frames.

[0050] S120, inputting the image to be detected into the image model to perform human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map.

[0051] In this embodiment, the key point prediction map refers to the prediction map of the posture key points of each passenger in each frame; the human body mask prediction map refers to the human body mask map of each passenger in each frame.

[0052] Specifically, the image model is formed by adding a mask branch to the yolov5 model, and the CIOU loss function is used for target detection, the focal loss function is used for target classification, the dice loss function is used for semantic segmentation, and the Adaloss loss function is used for key point detection.

[0053] The image model is improved based on the yolov5 model and is formed by adding a mask branch. The image model is named the yolov5-mask model, so that the model can simultaneously support target detection and segmentation tasks to achieve real-time instance segmentation tasks.

[0054] In this embodiment, the picture to be detected is input into the picture model, and the output information of the N3, N4 and N5 layers of the PANet network are selected from the yolov5 model of the picture model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map.

[0055] Because it is necessary to detect targets such as passengers, which are medium to large in size, the output information of the N3, N4 and N5 layers of the PANet network is selected in the yolov5 model, and the human key point prediction map is predicted through the mask branch. In order to improve the model speed, spatial information is extracted based on the N3 layer, and semantic information is extracted based on the N5 layer. The features in the target area are intercepted from their respective feature layers through the ROI Align mechanism and the SPP method. The features intercepted by the N5 layer are upsampled by the hole convolution group and the deconvolution group, and the features intercepted by the N3 layer are downsampled by the variable convolution group. At the same time, the features intercepted by the N4 layer are convoluted by the two-dimensional convolution group. The convolution group here contains multiple convolution operations and activation operations. Here, the Mish activation function and batch normalization are used. Finally, the three-layer convolution output results NF3, NF4, and NF5 are guaranteed to have the same feature layer size. Then, the three output results are concatenated to merge them, and the feature PE is obtained for both rich semantic information and spatial information. Then, the feature PE is operated by the attention mechanism of CBAM (Convolutional Block Attention Module) to make the number of its channels equal to the number of key point categories plus the number of target categories, which is 1 here, i.e., the human category. The key points here are the left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right waists, left and right knees, and left and right ankles. Finally, the feature map is restored to the original target size through the deconvolution of the transposed convolution and normalized through the activation function, here using the sigmoid activation function, to obtain the final Gaussian distribution of the key points of the human body. For each category, the key point with the highest probability is selected, and the key points with lower probability are filtered out through the threshold, so that a multi-category key point prediction map and a human mask prediction map can be obtained.

[0056] It should be noted that the prediction and training of the image model have different selection of target positions for mask and key point prediction. The target position is selected based on the prediction box position in the prediction, and the target position is selected based on the information of the annotation box in the training. The image model uses the CIOU loss function for target detection. Considering the sparse human density in the gate area, the Fast-NMS process is used to speed up the model reasoning. The focal loss function is used for target classification loss to reduce the impact of category imbalance. The dice loss function is used for semantic segmentation, and the loss function used for key point detection is Adaloss.

[0057] First, the gate card swiping area is set, and detection is performed in the area. The collected samples are predicted to have a human body frame and the corresponding PMP (mask information, Person Mask Picture) and the key point information corresponding to the human body through the above-mentioned image model.

[0058] S130: input the key point prediction map and the human body mask prediction map into the tracking model to perform target tracking to obtain a tracking result.

[0059] In this embodiment, the tracking result refers to the movement trajectory of the human body.

[0060] Specifically, the tracking model is obtained by training a SimMask-Transformer network with a number of human mask prediction images with target detection box labels.

[0061] Specifically, the number of targets in the gate area is controlled within a certain range, so it is possible to determine whether it is the same target based solely on the similarity of the features of the detected human mask prediction map, and then track it. Here, the mask information of the human body, that is, the human mask prediction map with the foreground being the original pixels of the human body and the background RGB being 0, is input into the SimMask-Transformer network to obtain the characteristics of the target. Here, the SimMask-Transformer network is trained using triplet loss. The similarity of the same mask features is very large, and the similarity of different mask features is very small. This judgment greatly improves the model recognition speed and achieves the effect of real-time detection.

[0062] The SimMask-Transformer network refers to the human mask prediction map with pixels of the foreground being the original human body and pixels of the background RGB being 0 as input to train the Transformer network.

[0063] When tracking a target, you only need to calculate the similarity of different targets within the gate range to track them. When the similarity exceeds the set threshold, maximum matching is performed to track the target, and the movement trajectory of each passenger and the index information of each passenger are obtained. If the distance between the coordinate center points of two passengers is lower than the threshold, the two passengers are combined and their trajectory information is retained separately. This is to detect the situation of tailgating and fare evasion.

[0064] S140: Generate a mask posture key point sequence video according to the tracking result.

[0065] In this embodiment, the mask posture key point sequence video refers to a time series video with key points of action trajectories.

[0066] In one embodiment, see Figure 3, the above-mentioned step S140 may include steps S141 to S143.

[0067] S141, generating a corresponding posture key point prediction map through yolov5-mp using the key point prediction map in the tracking result.

[0068] In this embodiment, the posture key point prediction map refers to the key point trajectory image generated by the human body key point prediction map through yolov5-mp.

[0069] Among them, yolov5-mp is an improvement based on the yolov5 model, adding a model with a key point mask prediction branch.

[0070] S142: directly splicing and merging the original features of the human body mask prediction image and the posture key point prediction image in the tracking result to form a fused image.

[0071] In this embodiment, the fused image refers to an image formed by merging the features of a human body mask prediction image and a posture key point prediction image.

[0072] S143: Merge the fused images according to time to generate a mask posture key point sequence video.

[0073] Specifically, a logical judgment is made according to the motion trajectory of each passenger and combined passenger and the position of the gate to determine the process of the passenger from entering the gate to leaving the gate. In this process, the posture key points of each passenger in each frame are recorded, that is, the human body key point prediction map in the tracking result, and the corresponding KPP (posture key point prediction map, KeyPoint Picture) is generated through yolov5-mp, and the information of the key point is set to 1, and the information of the non-key point is set to 0. Different categories of key points are represented by different channels. At the same time, for the PMP (mask information, Person Mask Picture) map of the human body, the original pixel RGB three-channel information is set for the foreground content, and the RGB for the non-background content is set to (0, 0, 0) to retain the spatial information of different human bodies. Finally, the posture key point prediction map KPP and the human body mask prediction map PMP are concatenated to obtain multiple channels of PFP (prediction fusion picture, Prediction Fusion Picture), and the PFP of each passenger or combined passenger with a trajectory is merged by time to generate a mask posture key point sequence video PFP-Video.

[0074] S150, inputting the mask posture key point sequence video into a behavior recognition model to perform ticket evasion behavior detection to obtain a detection result.

[0075] In this embodiment, the detection result refers to the recognition result of whether the current image to be detected contains fare evasion behavior. Behavior recognition is performed not only based on image information, but also based on the sequence information between images in the video, and the detection accuracy is greatly enhanced.

[0076] Specifically, the behavior recognition model is obtained by training a 3D Transformer-TIN model through a number of mask posture key point sequence videos of ticket evasion behavior labels.

[0077] Among them, the 3D Transformer-TIN model refers to the model formed by combining the 3D Transformer model with TIN.

[0078] The ticket evasion behavior labels include squatting, jumping and tailing behavior labels.

[0079] Specifically, in the 3D Transformer-TIN model, the mask pose key point sequence video is processed by SPatio-temporal Representation to obtain temporal frames, and then the feature map is obtained by 3D convolution. An improvement is made to this by using the input frames as the input of the transformer mechanism, so that the temporal information can be interconnected. Not only can the features between video sequences be found, but also the interval information can be found, which further enhances the understanding of the features of the sequence image. The result of the transformer output is then used as the attention feature to generate a feature map through the 3D convolution machine, thereby determining whether there is any fare evasion behavior.

[0080] S160: When the detection result is a fare evasion behavior, a warning message is generated and sent to the terminal.

[0081] Specifically, the passenger's behavior at the gate is divided into four categories: squatting, jumping, trailing, and passing. For a single person's trajectory, squatting, jumping, and passing behavior predictions are performed, and for a multi-person combined trajectory, trailing and passing predictions are performed. If squatting, jumping, and trailing occur at the gate, it is considered as a fare evasion behavior and a warning message is generated.

[0082] The micro-model used is very friendly to real-time effects and can promptly alert passengers of fare evasion. Based on the existing models, the yolov5-mask model, SimMask-Transformer model and 3DTransformer-TIN model are used to greatly improve the accuracy of fare evasion detection.

[0083] The above-mentioned gate machine fare evasion detection method obtains the image taken by the camera and cuts it, and then inputs the generated image to be detected into the image model for human instance segmentation and key point detection, combines the tracking model to track the target, and generates a mask posture key point sequence video. Ticket evasion behavior is detected based on the mask posture key point sequence video. When ticket evasion behavior exists, an alarm is issued to realize real-time ticket evasion behavior detection, without the need for infrared imaging technology for target positioning, saving a lot of cost and with high recognition accuracy.

[0084] Figure 4 is a schematic block diagram of a gate fare evasion detection device 300 provided in an embodiment of the present invention. Figure 4 As shown, corresponding to the above gate fare evasion detection method, the present invention also provides a gate fare evasion detection device 300. The gate fare evasion detection device 300 includes a unit for executing the above gate fare evasion detection method, and the device can be configured in a server. Figure 4 The gate fare evasion detection device 300 includes a sampling unit 301, a segmentation detection unit 302, a tracking unit 303, a video generation unit 304, a behavior detection unit 305 and a warning unit 306.

[0085] The sampling unit 301 is used to obtain the video captured by the camera and process the video to obtain the picture to be detected; the segmentation detection unit 302 is used to input the picture to be detected into the picture model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; the tracking unit 303 is used to input the key point prediction map and the human mask prediction map into the tracking model for target tracking to obtain a tracking result; the video generation unit 304 is used to generate a mask posture key point sequence video according to the tracking result; the behavior detection unit 305 is used to input the mask posture key point sequence video into the behavior recognition model for ticket evasion behavior detection to obtain a detection result; the warning unit 306 is used to generate a warning message when the detection result is ticket evasion behavior, and send the warning message to the terminal.

[0086] In one embodiment, the segmentation detection unit 302 is used to input the image to be detected into the image model, select the output information of the N3, N4 and N5 layers of the PANet network from the yolov5 model of the image model, predict the human body key point prediction map through the mask branch, and generate a human body mask to obtain a key point prediction map and a human body mask prediction map.

[0087] In one embodiment, if Figure 5 As shown, the video generation unit 304 includes a key point prediction map generation subunit 3041, a splicing subunit 3042 and a merging subunit 3043.

[0088] The key point prediction map generation subunit 3041 is used to generate a corresponding posture key point prediction map through yolov5-mp from the key point prediction map in the tracking result; the splicing subunit 3042 is used to directly splice and merge the human body mask prediction map and the posture key point prediction map in the tracking result with the original features to form a fused image; the merging subunit 3043 is used to merge the fused image by time to generate a mask posture key point sequence video.

[0089] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned gate fare evasion detection device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0090] The gate fare evasion detection device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 6 Runs on the computer device shown.

[0091] See also Figure 6 , Figure 6 5 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0092] See also Figure 6 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0093] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a gate fare evasion detection method.

[0094] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0095] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a gate fare evasion detection method.

[0096] The network interface 505 is used to communicate with other devices over the network. Figure 6The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0097] The processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps:

[0098] Acquire a video captured by a camera, and process the video to obtain a picture to be detected; input the picture to be detected into a picture model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; input the key point prediction map and the human mask prediction map into a tracking model for target tracking to obtain a tracking result; generate a mask posture key point sequence video according to the tracking result; input the mask posture key point sequence video into a behavior recognition model for ticket evasion behavior detection to obtain a detection result; when the detection result is ticket evasion behavior, generate a warning message, and send the warning message to the terminal.

[0099] Among them, the image model is formed by adding a mask branch based on the yolov5 model, and the CIOU loss function is used for target detection, the focal loss function is used for target classification, the dice loss function is used for semantic segmentation, and the Adaloss loss function is used for key point detection.

[0100] The tracking model is obtained by training the SimMask-Transformer network with several human mask prediction images with target detection frame labels.

[0101] The behavior recognition model is obtained by training a 3D Transformer-TIN model with a number of mask posture key point sequence videos of ticket evasion behavior labels.

[0102] The ticket evasion behavior labels include squatting, jumping and tailing behavior labels.

[0103] In one embodiment, when the processor 502 implements the step of inputting the image to be detected into the image model to perform human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map, the processor 502 specifically implements the following steps:

[0104] The image to be detected is input into the image model, and the output information of the N3, N4 and N5 layers of the PANet network is selected from the yolov5 model of the image model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map.

[0105] In one embodiment, when the processor 502 implements the step of generating a mask pose key point sequence video according to the tracking result, the processor 502 specifically implements the following steps:

[0106] The key point prediction map in the tracking result is used to generate a corresponding posture key point prediction map through yolov5-mp; the human body mask prediction map and the posture key point prediction map in the tracking result are directly spliced ​​and merged by original features to form a fused image; the fused image is merged by time to generate a mask posture key point sequence video.

[0107] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0108] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0109] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0110] Acquire a video captured by a camera, and process the video to obtain a picture to be detected; input the picture to be detected into a picture model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; input the key point prediction map and the human mask prediction map into a tracking model for target tracking to obtain a tracking result; generate a mask posture key point sequence video according to the tracking result; input the mask posture key point sequence video into a behavior recognition model for ticket evasion behavior detection to obtain a detection result; when the detection result is ticket evasion behavior, generate a warning message, and send the warning message to the terminal.

[0111] Among them, the image model is formed by adding a mask branch based on the yolov5 model, and the CIOU loss function is used for target detection, the focal loss function is used for target classification, the dice loss function is used for semantic segmentation, and the Adaloss loss function is used for key point detection.

[0112] The image to be detected is input into the image model, and the output information of the N3, N4 and N5 layers of the PANet network is selected from the yolov5 model of the image model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map.

[0113] The tracking model is obtained by training the SimMask-Transformer network with several human mask prediction images with target detection frame labels.

[0114] The behavior recognition model is obtained by training a 3D Transformer-TIN model with a number of mask posture key point sequence videos of ticket evasion behavior labels.

[0115] The ticket evasion behavior labels include squatting, jumping and tailing behavior labels.

[0116] In one embodiment, when the processor executes the computer program to implement the step of inputting the image to be detected into the image model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map, the processor specifically implements the following steps:

[0117] In one embodiment, when the processor executes the computer program to implement the step of generating a mask pose key point sequence video according to the tracking result, the processor specifically implements the following steps:

[0118] The key point prediction map in the tracking result is used to generate a corresponding posture key point prediction map through yolov5-mp; the human body mask prediction map and the posture key point prediction map in the tracking result are directly spliced ​​and merged by original features to form a fused image; the fused image is merged by time to generate a mask posture key point sequence video.

[0119] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which are computer-readable storage media that can store program codes.

[0120] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0121] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0122] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0124] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A gate fare evasion detection method, characterized in that: include: Obtain the video captured by the camera and process the video to obtain the image to be detected; Inputting the image to be detected into the image model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; Input the key point prediction map and the human body mask prediction map into the tracking model to perform target tracking to obtain the tracking result; Generate a mask posture key point sequence video according to the tracking result; Inputting the mask posture key point sequence video into the behavior recognition model to detect the fare evasion behavior to obtain the detection result; When the detection result is a ticket evasion behavior, a warning message is generated and sent to the terminal; The image model is formed by adding a mask branch to the yolov5 model, and uses the CIOU loss function for target detection, the focalloss loss function for target classification, the dice loss function for semantic segmentation, and the Adaloss loss function for key point detection; The step of inputting the image to be detected into the image model for human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map includes: The image to be detected is input into the image model, and the output information of the N3, N4 and N5 layers of the PANet network is selected from the yolov5 model of the image model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map.

2. The gate fare evasion detection method according to claim 1, characterized in that: The tracking model is obtained by training the SimMask-Transformer network with several human mask prediction images with target detection frame labels.

3. The gate fare evasion detection method according to claim 1, characterized in that: The step of generating a mask posture key point sequence video according to the tracking result includes: The key point prediction map in the tracking result is used to generate a corresponding posture key point prediction map through yolov5-mp; The human body mask prediction image and the posture key point prediction image in the tracking result are directly spliced ​​and merged by original features to form a fused image; The fused images are merged in time to generate a mask pose key point sequence video.

4. The gate fare evasion detection method according to claim 1, characterized in that: The behavior recognition model is obtained by training a 3D Transformer-TIN model with a number of mask posture key point sequence videos of ticket evasion behavior labels.

5. The gate fare evasion detection method according to claim 4, characterized in that: The ticket evasion behavior labels include squatting, jumping and tailing behavior labels.

6. A gate fare evasion detection device, characterized in that: include: A sampling unit, used to obtain a video shot by a camera and process the video to obtain a picture to be detected; The segmentation detection unit is used to input the image to be detected into the image model to perform human instance segmentation and key point detection to obtain a key point prediction map and a human mask prediction map; the image to be detected is input into the image model, and the output information of the N3, N4 and N5 layers of the PANet network are selected from the yolov5 model of the image model to predict the human key point prediction map through the mask branch, and a human mask is generated to obtain a key point prediction map and a human mask prediction map; A tracking unit, used for inputting the key point prediction map and the human body mask prediction map into the tracking model to perform target tracking to obtain a tracking result; A video generation unit, used for generating a mask posture key point sequence video according to the tracking result; A behavior detection unit, used for inputting the mask posture key point sequence video into the behavior recognition model to detect the fare evasion behavior to obtain a detection result; a warning unit, configured to generate a warning message when the detection result is a fare evasion behavior, and send the warning message to the terminal; The image model is formed by adding a mask branch to the yolov5 model, and uses the CIOU loss function for target detection, the focalloss loss function for target classification, the dice loss function for semantic segmentation, and the Adaloss loss function for key point detection.

7. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.