Method, device, computer equipment and storage medium for detecting fare evasion at gates

Abstract

The embodiments of the present invention disclose a method, device, computer equipment and storage medium for detecting ticket evasion at a gate. The method comprises: obtaining a video captured by a camera, and processing the video to obtain a picture to be detected; inputting the picture to be detected into a picture model to perform human instance segmentation to obtain a human mask prediction map; inputting the human mask prediction map into a tracking model to perform target tracking to obtain a tracking result; generating a human mask sequence video based on the tracking result combined with the human mask prediction map; inputting the human mask sequence video into a behavior recognition model to detect ticket evasion to obtain a detection result; when the detection result is ticket evasion, generating a warning message, and sending the warning message to a terminal. By implementing the method of the embodiment of the present invention, real-time ticket evasion detection can be achieved without the need for infrared imaging technology for target positioning, saving a lot of costs, and achieving a high recognition accuracy rate.

Description

Technical Field

[0001] The present invention relates to a method for detecting fare evasion, and more specifically to a method, device, computer equipment and storage medium for detecting fare evasion at a gate. Background Art

[0002] Currently, rail transit, such as the subway, is becoming increasingly popular as an important and convenient mode of transportation. Generally, people need to purchase a ticket before boarding the subway and pass through a gate to check their tickets when entering the station. However, many passengers evade fares every day, for example by jumping over the gate, causing significant financial losses to the subway company.

[0003] Chinese patent CN201910010440.8 discloses a method and system for detecting subway fare evasion based on infrared thermal imaging, which includes the following steps: detecting whether a pedestrian has entered the pedestrian detection range of the gate image; performing background difference of the infrared thermal imaging image using the automatically updated background to extract the infrared thermal imaging overhead view image of the pedestrian; performing morphological processing on the extracted infrared thermal imaging overhead view image of the pedestrian to obtain a binary overhead view image of the pedestrian passing through the gate based on the automatically updated appropriate threshold; performing parallel extraction of sub-regions of interest on the binary pedestrian overhead view image, setting the ROI area to be the same as the number of gates (N), and obtaining N mutually independent binary overhead views of pedestrians in the gate channels; performing N mutually independent binary overhead views of pedestrians in the gate channels Figure 2 The connected areas of the image are marked respectively to obtain the pedestrian parameters; the fare evasion behavior at N gates is determined. The invention can effectively identify and avoid fare evasion behavior. However, this method is based on infrared thermal imaging for fare evasion detection. The cost is high and a fixed camera is required, which is very unfriendly for project deployment. It has certain limitations and is based on the overhead view of pedestrians in the binary gate channel. Figure 2Judging by using a value image is often greatly affected by other environmental factors such as lighting, and the accuracy of the effect cannot be guaranteed. Chinese patent CN201911224925.3 provides a subway passenger fare evasion behavior detection system and method, which specifically includes a passenger information marking module, which identifies and marks whether the passenger is a passenger who needs to purchase a ticket to ride, whether he or she is carrying an infant, and captures facial information, and stores the passenger's dynamic riding information; a card swiping behavior recognition module, which determines whether the passenger swipes the card based on whether there is an intersection between the human skeleton motion trajectory and the card swiping area obtained by the depth camera; a card swiping information reading module, which reads the gate card swiping information and time, and determines that the card swiping behavior is successful when the card swiping behavior is performed; a fare evasion behavior judgment and warning module, which combines the passenger's ticket purchase mark information, card swiping behavior recognition information, card swiping success record and the number of people passing through to identify fare evasion behavior and issue a warning; this method is based on shooting with a depth camera, which is costly, and the system needs to combine the card swiping information reading to touch and see to obtain fare evasion behavior, and cannot directly use image information to judge, which has certain limitations. Chinese patent CN201510144081.7 discloses a gate detection system and method, which includes: a three-dimensional image information acquisition module, including at least two image data acquisition devices for acquiring two-dimensional image information of the same area to be detected from different positions, for obtaining image information of the human body in the area to be detected; a gate status acquisition module, for obtaining gate status information of the gate; a three-dimensional image information recognition and processing module, for using the image information of the human body and the status information of the gate to determine whether the human body has evaded the ticket; an alarm module, for sounding an alarm when the human body has evaded the ticket; this method requires the use of a three-dimensional image information acquisition module, which is relatively low in cost, and needs to be based on the acquisition of the gate status. It cannot be analyzed based solely on image information, and the deployment project is relatively complex. Chinese patent CN202110192793.1 relates to a method for fare evasion at subway gates based on rapid estimation of passenger posture. Identification is performed through the following steps: first, information is collected through video surveillance of subway gates, then key points of subway passenger skeletons are detected, and finally, the fare evasion behavior of passengers passing through the Forbidden City gates is identified. However, this method only judges the fare evasion behavior of passengers based on key point information. This method cannot obtain the continuous characteristics of the passenger's fare evasion behavior. The lack of identification information affects the judgment of the passenger's fare evasion behavior. Secondly, relying solely on key point information for logical judgment is often affected by the space captured by the camera, resulting in inaccurate recognition.

[0004] Therefore, it is necessary to design a new method to achieve real-time detection of fare evasion, without the need for infrared imaging technology for target positioning, saving a lot of costs and with high recognition accuracy. Summary of the Invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide a method, device, computer equipment and storage medium for detecting fare evasion at a gate.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for detecting fare evasion at a gate, comprising:

[0007] Obtaining the video captured by the camera and processing the video to obtain the image to be detected;

[0008] Inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction image;

[0009] Inputting the human body mask prediction image into a tracking model to perform target tracking to obtain a tracking result;

[0010] Generate a human mask sequence video according to the tracking result and the human mask prediction image;

[0011] Inputting the human mask sequence video into a behavior recognition model to detect fare evasion behavior to obtain a detection result;

[0012] When the detection result is a fare evasion behavior, a warning message is generated and sent to the terminal.

[0013] Its further technical solution is: the image model is formed by adding a target detection branch and a mask branch on the basis of the ParNet model.

[0014] A further technical solution is: inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction image, including:

[0015] Inputting the image to be detected into the image model, and selecting the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer in the ParNet model to identify the target position in the image to be detected;

[0016] The seventh layer RepVGG SSE block of the ParNet model, the fused tenth layer, and the eleventh layer downsampling layer are selected to upsample the image to be detected and directly concatenate the original features to obtain a fused feature map;

[0017] Processing the fused feature map through deformable convolution and sigmoid activation function to obtain a mask feature map;

[0018] The mask feature map is resized and upsampled to obtain a human body mask prediction map.

[0019] Its further technical solution is: the tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model, wherein, when the tracking model uses the Fairmot model to track the target and the target is not detected within a specified number of consecutive frames, the target is tracked by optical flow for several frames in advance from the moment it disappears, until the target is found within the specified number of frames, and then the Fairmot model is continued to be used for target tracking.

[0020] A further technical solution is: generating a human mask sequence video based on the tracking result and the human mask prediction map, including:

[0021] Determining a human mask prediction map having a motion trajectory according to the tracking result to obtain a screening result;

[0022] The screening results are processed and merged according to time to obtain a human body mask sequence video.

[0023] A further technical solution is: the behavior recognition model is formed by modifying the convolution of the time pyramid network in the time dimension into a Transformer model.

[0024] A further technical solution is that the loss function used in training the behavior recognition model is a cross entropy loss function.

[0025] The present invention also provides a device for detecting fare evasion at a gate, which is characterized by comprising:

[0026] A sampling unit is used to obtain the video captured by the camera and process the video to obtain the image to be detected;

[0027] An instance segmentation unit, configured to input the image to be detected into an image model to perform human instance segmentation to obtain a human mask prediction image;

[0028] A target tracking unit, configured to input the human body mask prediction image into a tracking model to perform target tracking to obtain a tracking result;

[0029] A sequence video generating unit, configured to generate a human body mask sequence video according to the tracking result and the human body mask prediction image;

[0030] a behavior detection unit, configured to input the human mask sequence video into a behavior recognition model to detect fare evasion behavior and obtain a detection result;

[0031] The alarm unit is used to generate a warning message when the detection result is a fare evasion behavior, and send the warning message to the terminal.

[0032] The present invention further provides a computer device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0033] The present invention also provides a storage medium, wherein the storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0034] The beneficial effects of the present invention compared with the existing technology are: the present invention obtains the image taken by the camera and cuts it, so as to input the generated image to be detected into the image model for human instance segmentation, combines the tracking model to track the target, and generates a human mask sequence video. The fare evasion behavior is detected based on the human mask sequence video. When the fare evasion behavior occurs, an alarm is issued to realize real-time detection of the fare evasion behavior, and no infrared imaging technology is required for target positioning, which saves a lot of costs and has a high recognition accuracy.

[0035] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A schematic diagram of an application scenario of the method for detecting fare evasion at a gate provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the flow of a method for detecting fare evasion at a gate provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of a sub-process of a method for detecting fare evasion at a gate provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of a sub-process of a method for detecting fare evasion at a gate provided by an embodiment of the present invention;

[0041] Figure 5 A schematic block diagram of a device for detecting fare evasion at a gate provided by an embodiment of the present invention;

[0042] Figure 6 A schematic block diagram of an instance segmentation unit of a gate fare evasion detection device provided by an embodiment of the present invention;

[0043] Figure 7A schematic block diagram of a sequence video generation unit of a gate fare evasion detection device provided by an embodiment of the present invention;

[0044] Figure 8 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0046] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0047] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0049] See also Figure 1 and Figure 2 , Figure 1 Schematic diagram of an application scenario of the method for detecting fare evasion at a gate provided by an embodiment of the present invention. Figure 2The schematic flow chart of the method for detecting fare evasion at a gate provided by an embodiment of the present invention. The method for detecting fare evasion at a gate is applied in a server. The server exchanges data with the terminal and the camera, obtains the video captured by the camera, processes the video to form a picture to be detected, and uses the picture model to perform human instance segmentation and the tracking model to perform target tracking, and then generates a human mask sequence video, which is input into the behavior recognition model to perform fare evasion recognition. When fare evasion occurs, a warning message is generated and sent to the terminal for prompting. The method can identify the fare evasion behavior of passengers by relying solely on an optical camera, without the need for fixed shooting restrictions on the camera. It only needs to capture the gate position to detect multiple people evading the fare at the same time. The miniature model used is very friendly to real-time effects, and can promptly alarm passengers for fare evasion with high recognition accuracy.

[0050] Figure 2 : is a flow chart of the method for detecting ticket evasion at a gate provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S160.

[0051] S110: Obtain a video captured by a camera, and process the video to obtain a picture to be detected.

[0052] In this embodiment, the image to be detected refers to an image captured by a camera within a set gate range.

[0053] Specifically, the lens of the optical camera is aimed at the passenger card swiping gate area, and the area is monitored based on video. Here, the optical camera adopts a fixed focal length, fixed angle and position, and the captured video is cut into frames to generate pictures. Here, every 2 frames are cut to generate frame pictures with a certain time sequence, that is, the pictures to be detected.

[0054] It only relies on optical cameras and does not require infrared imaging technology for target positioning, saving a lot of costs.

[0055] S120 , inputting the image to be detected into an image model to perform human instance segmentation to obtain a human mask prediction image.

[0056] In this embodiment, the human body mask prediction image refers to the human body mask image of each passenger in each frame.

[0057] Specifically, the image model is formed by adding a target detection branch and a mask branch to the ParNet model.

[0058] In one embodiment, see Figure 3 , the above-mentioned step S120 may include steps S121 to S124.

[0059] S121: Input the image to be detected into the image model, and select the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer in the ParNet model to identify the target position in the image to be detected.

[0060] In this embodiment, the ninth-layer Fusion and the tenth-layer Fusion of the ParNet model are respectively selected to identify the target position, ie, the passenger position.

[0061] S122. Select the seventh layer RepVGG SSE block of the ParNet model, the fused tenth layer, and the eleventh layer downsampling layer, upsample the image to be detected, and directly concatenate the original features to obtain a fused feature map.

[0062] In this embodiment, the fused feature map refers to a picture obtained by processing the image to be detected by the seventh layer RepVGG SSE block of the ParNet model, the fused tenth layer, and the eleventh downsampling layer.

[0063] S123. Process the fused feature map through deformable convolution and sigmoid activation function to obtain a mask feature map.

[0064] In this embodiment, the mask feature map refers to a picture formed by processing the fused feature map through deformable convolution and sigmoid activation function.

[0065] S124: resize and upsample the mask feature map to obtain a human body mask prediction map.

[0066] In this embodiment, the ninth-layer Fusion and the tenth-layer Fusion are respectively selected in the ParNet model to identify the target (passenger) position. Nine anchor frames of different sizes and proportions are generated on the feature points of the feature layers of the two layers, just like the yolo series. The anchor frame with the largest target iou is selected during training, and the target frame with the highest confidence is selected for target detection during prediction. For semantic segmentation, the seventh-layer RepVGG SSE block, the fused tenth layer, and the eleventh-layer downsampling layer are respectively upsampled and concatenated to obtain a fused feature map, and then a mask feature map is obtained through deformable convolution and sigmoid activation function. Finally, the upsampled image is resized to be consistent with the original image target size for human mask segmentation. It should be noted here that the image model prediction and training have different target position selection in mask prediction. The target position is selected based on the prediction box position in the prediction, and the target position is selected based on the information of the annotation box in the training. The training loss functions include the foreground and background loss function, the category loss function, the target box loss function, and the segmentation loss function. This loss function is consistent with the yolo loss, and the segmentation loss is added on this basis. The segmentation loss uses Lovasz Loss.

[0067] First, the gate card swiping area is set, and detection is performed in this area. The collected samples are predicted to form a human body frame and the corresponding PMP (mask information, Person Mask Picture), that is, a human body mask prediction map, through the above-mentioned image model.

[0068] S130: Input the human body mask prediction image into a tracking model to perform target tracking to obtain a tracking result.

[0069] In this embodiment, the tracking result is the movement trajectory of the human body, that is, the movement trajectory of the passenger.

[0070] The tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model. When the tracking model uses the Fairmot model to track a target and the target is not detected within a specified number of consecutive frames, the optical flow tracking of the target is advanced by several frames from the moment it disappears until the target is found within the specified number of frames, and then the Fairmot model is continued to be used for target tracking.

[0071] Specifically, the tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model. When the tracking model uses the Fairmot model to track a target and the target is not detected within 5 consecutive frames, the target is tracked by optical flow 10 frames in advance from the moment it disappears. Until the target is found within 5 frames, the Fairmot model is continued to be used for target tracking.

[0072] Because in the process of detecting fare evasion at the gate, it is easy to have multiple targets, that is, multiple passengers, at the same time. In order to record the movement trajectories of different passengers and facilitate subsequent behavior recognition based on time series information, it is necessary to use a target tracking model, that is, the Fairmot model. For targets that are missing in multiple frame detections in the central area of ​​the gate, the dense optical flow method is used to predict the target trajectory. The principle here is that when the target is tracked based on the Fairmot model, if no object is detected in 5 consecutive frames, the target is tracked 10 frames in advance from disappearance until the target is found within 5 frames. The tracking algorithm of the Fairmot model is continued to make up for the poor tracking effect when the target detection is missed. Combining the Fairmot model and the dense optical flow method for target tracking improves the target recall rate and the tracking effect. Especially in the case of crowded passengers, target occlusion often occurs. This method can well solve the problem of occlusion in the target tracking process.

[0073] By tracking the human mask prediction map, the movement trajectory and index information of each passenger are obtained. If the distance between the center points of two passengers is lower than a threshold, the two passengers are combined and their trajectory information is retained separately to facilitate the detection of tailgating and fare evasion.

[0074] S140: Generate a human body mask sequence video according to the tracking result and the human body mask prediction image.

[0075] In this embodiment, the human mask sequence video refers to a time series video formed by processing the human mask prediction map with the human body's movement trajectory. Behavior recognition is performed not only based on image information, but also based on the sequence information between images in the video, which greatly enhances the recognition accuracy.

[0076] In one embodiment, see Figure 4 , the above-mentioned step S140 may include steps S141 to S142.

[0077] S141 , determining a human mask prediction map with a motion trajectory according to the tracking result to obtain a screening result.

[0078] In this embodiment, the screening result refers to a human body mask prediction image with a motion trajectory.

[0079] S142: Process the screening results and merge them by time to obtain a human body mask sequence video.

[0080] Based on the motion trajectory of each passenger and combination of passengers and the position of the gate, a logical judgment is made to determine the process of the passenger from entering the gate to leaving the gate. The foreground (human mask) of the human mask prediction image of each passenger or combination of passengers with a motion trajectory is set to the original image pixel, and the background RGB is set to (0,0,0). Finally, the human mask sequence video Mask-Video is generated by merging them according to time.

[0081] S150: Input the human mask sequence video into a behavior recognition model to detect fare evasion behavior to obtain a detection result.

[0082] In this embodiment, the detection result refers to the recognition result of whether the current image to be detected contains fare evasion behavior.

[0083] Specifically, the behavior recognition model is formed by modifying the convolution of the time pyramid network in the time dimension into a Transformer model. The loss function used in training the behavior recognition model is a cross entropy loss function.

[0084] Mask-Video, a sequence of human mask videos, is used as input for the action recognition model. To make the model more robust, VideoMix is ​​used for data augmentation training during training. This model replaces the temporal convolution with a Transformer, extracting features in both spatial and temporal modulation. During training, the mask-video sequence is randomly DropBlocked, meaning that the dropped mask is set as the background. This allows the model to adapt to partial occlusions and effectively predict action types. The cross-entropy loss function is used for this model.

[0085] S160: When the detection result is a fare evasion behavior, generate a warning message and send the warning message to the terminal.

[0086] Specifically, the system categorizes passenger gate behaviors into four categories: squatting, jumping, tailgating, and passing. For single-person trajectories, these behaviors are predicted, while for multi-person trajectories, these behaviors are predicted as tailgating and passing. If squatting, jumping, or tailgating occurs at the gate, it is considered fare evasion and a warning message is generated.

[0087] The micro-model used in this embodiment is highly responsive to real-time performance, enabling timely alerts to passengers attempting to evade fares. Building upon existing models, the ParNet-IS model, the Fairmot-HF model, and the TPN-Transformer model are generated. The ParNet-IS model combines ParNet with the target detection branch and the mask branch, the Fairmot model combines the HF model, and the TPN model combines the Transformer model. These models significantly improve the accuracy of fare evasion detection.

[0088] The above-mentioned method for detecting fare evasion at a turnstile obtains images captured by a camera and cuts them, inputting the generated images to be detected into an image model for human instance segmentation, combining the tracking model for target tracking, and generating a human mask sequence video. Fare evasion is detected based on the human mask sequence video. When fare evasion occurs, an alarm is issued to achieve real-time detection of fare evasion. No infrared imaging technology is required for target positioning, which saves a lot of costs and has a high recognition accuracy.

[0089] Figure 5 FIG. 3 is a schematic block diagram of a gate fare evasion detection device 300 provided by an embodiment of the present invention. Figure 5 As shown, corresponding to the above gate fare evasion behavior detection method, the present invention also provides a gate fare evasion behavior detection device 300. The gate fare evasion behavior detection device 300 includes a unit for executing the above gate fare evasion behavior detection method, and the device can be configured in a server. Figure 5 The gate fare evasion behavior detection device 300 includes a sampling unit 301, an instance segmentation unit 302, a target tracking unit 303, a sequence video generation unit 304, a behavior detection unit 305 and an alarm unit 306.

[0090] The sampling unit 301 is used to obtain the video captured by the camera and process the video to obtain the picture to be detected; the instance segmentation unit 302 is used to input the picture to be detected into the picture model to perform human instance segmentation to obtain a human mask prediction map; the target tracking unit 303 is used to input the human mask prediction map into the tracking model to perform target tracking to obtain a tracking result; the sequence video generation unit 304 is used to generate a human mask sequence video based on the tracking result and the human mask prediction map; the behavior detection unit 305 is used to input the human mask sequence video into the behavior recognition model to detect ticket evasion behavior to obtain a detection result; the alarm unit 306 is used to generate a warning message when the detection result is ticket evasion behavior, and send the warning message to the terminal.

[0091] In one embodiment, if Figure 6As shown, the instance segmentation unit 302 includes a target recognition subunit 3021 , a splicing subunit 3022 , a processing subunit 3023 and an upsampling subunit 3024 .

[0092] The target recognition subunit 3021 is used to input the image to be detected into the image model, and select the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer in the ParNet model to identify the target position in the image to be detected; the splicing subunit 3022 is used to select the seventh layer RepVGG SSE block, the fused tenth layer and the eleventh layer downsampling layer of the ParNet model, upsample the image to be detected and directly splice the original features to obtain a fused feature map; the processing subunit 3023 is used to process the fused feature map through deformable convolution and sigmoid activation function to obtain a mask feature map; the upsampling subunit 3024 is used to resize and upsample the mask feature map to obtain a human body mask prediction map.

[0093] In one embodiment, if Figure 7 As shown, the sequence video generating unit 304 includes a screening subunit 3041 and a merging subunit 3042 .

[0094] The screening subunit 3041 is used to determine the human mask prediction map with the action trajectory according to the tracking result to obtain the screening result; the merging subunit 3042 is used to process the screening result and merge it by time to obtain the human mask sequence video.

[0095] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned gate fare evasion detection device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0096] The above-mentioned gate fare evasion detection device 300 can be implemented in the form of a computer program. The computer program can be used in Figure 8 Runs on the computer equipment shown.

[0097] See also Figure 8 , Figure 8 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, wherein the server may be an independent server or a server cluster composed of multiple servers.

[0098] See Figure 8The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0099] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which, when executed, can cause the processor 502 to execute a method for detecting fare evasion at a gate.

[0100] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0101] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for detecting ticket evasion at a gate.

[0102] The network interface 505 is used to communicate with other devices through the network. Figure 8 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0103] The processor 502 is configured to execute a computer program 5032 stored in the memory to implement the following steps:

[0104] Acquire the video captured by the camera and process the video to obtain a picture to be detected; input the picture to be detected into the picture model for human instance segmentation to obtain a human mask prediction map; input the human mask prediction map into the tracking model for target tracking to obtain a tracking result; generate a human mask sequence video based on the tracking result and the human mask prediction map; input the human mask sequence video into the behavior recognition model for ticket evasion behavior detection to obtain a detection result; when the detection result is ticket evasion behavior, generate a warning message and send the warning message to the terminal.

[0105] The image model is formed by adding the target detection branch and the mask branch to the ParNet model.

[0106] The tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model. When the tracking model uses the Fairmot model to track a target and the target is not detected within a specified number of consecutive frames, the optical flow tracking of the target is advanced by several frames from the moment it disappears until the target is found within the specified number of frames, and then the Fairmot model is continued to be used for target tracking.

[0107] The behavior recognition model is formed by modifying the convolution of the time pyramid network in the time dimension into a Transformer model.

[0108] The loss function used in training the behavior recognition model is the cross entropy loss function.

[0109] In one embodiment, when the processor 502 implements the step of inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction image, the processor 502 specifically implements the following steps:

[0110] The image to be detected is input into the image model, and the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer are selected in the ParNet model to identify the target position in the image to be detected; the seventh layer RepVGG SSE block, the fused tenth layer and the eleventh layer downsampling layer of the ParNet model are selected to upsample the image to be detected and directly splice the original features to obtain a fused feature map; the fused feature map is processed by deformable convolution and sigmoid activation function to obtain a mask feature map; the mask feature map is resized and upsampled to obtain a human body mask prediction map.

[0111] In one embodiment, when the processor 502 implements the step of generating a human body mask sequence video according to the tracking result and the human body mask prediction map, the processor 502 specifically implements the following steps:

[0112] A human mask prediction map with a motion trajectory is determined according to the tracking result to obtain a screening result; the screening result is processed and merged according to time to obtain a human mask sequence video.

[0113] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0114] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0115] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0116] Acquire the video captured by the camera and process the video to obtain a picture to be detected; input the picture to be detected into the picture model for human instance segmentation to obtain a human mask prediction map; input the human mask prediction map into the tracking model for target tracking to obtain a tracking result; generate a human mask sequence video based on the tracking result and the human mask prediction map; input the human mask sequence video into the behavior recognition model for ticket evasion behavior detection to obtain a detection result; when the detection result is ticket evasion behavior, generate a warning message and send the warning message to the terminal.

[0117] The image model is formed by adding the target detection branch and the mask branch to the ParNet model.

[0118] The tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model. When the tracking model uses the Fairmot model to track a target and the target is not detected within a specified number of consecutive frames, the optical flow tracking of the target is advanced by several frames from the moment it disappears until the target is found within the specified number of frames, and then the Fairmot model is continued to be used for target tracking.

[0119] The behavior recognition model is formed by modifying the convolution of the time pyramid network in the time dimension into a Transformer model.

[0120] The loss function used in training the behavior recognition model is the cross entropy loss function.

[0121] In one embodiment, when the processor executes the computer program to implement the step of inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction map, the processor specifically implements the following steps:

[0122] The image to be detected is input into the image model, and the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer are selected in the ParNet model to identify the target position in the image to be detected; the seventh layer RepVGG SSE block, the fused tenth layer and the eleventh layer downsampling layer of the ParNet model are selected to upsample the image to be detected and directly splice the original features to obtain a fused feature map; the fused feature map is processed by deformable convolution and sigmoid activation function to obtain a mask feature map; the mask feature map is resized and upsampled to obtain a human body mask prediction map.

[0123] In one embodiment, when the processor executes the computer program to implement the step of generating a human body mask sequence video according to the tracking result and the human body mask prediction map, the processor specifically implements the following steps:

[0124] A human mask prediction map with a motion trajectory is determined according to the tracking result to obtain a screening result; the screening result is processed and merged according to time to obtain a human mask sequence video.

[0125] The storage medium may be any computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0126] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0127] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the various units is merely a logical functional division, and actual implementation may employ other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented.

[0128] The steps in the methods of the embodiments of the present invention may be adjusted in order, combined, or deleted as needed. The units in the devices of the embodiments of the present invention may be combined, divided, or deleted as needed. Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0129] If this integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting fare evasion at a gate, characterized in that: include: Obtaining the video captured by the camera and processing the video to obtain the image to be detected; Inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction image; Inputting the human body mask prediction image into a tracking model to perform target tracking to obtain a tracking result; Generate a human mask sequence video according to the tracking result and the human mask prediction image; Inputting the human mask sequence video into a behavior recognition model to detect fare evasion behavior to obtain a detection result; When the detection result is a fare evasion behavior, a warning message is generated and sent to the terminal; The image model is formed by adding the target detection branch and the mask branch to the ParNet model; The step of inputting the image to be detected into the image model to perform human instance segmentation to obtain a human mask prediction image includes: Inputting the image to be detected into the image model, and selecting the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer in the ParNet model to identify the target position in the image to be detected; The seventh layer RepVGG SSE block of the ParNet model, the fused tenth layer, and the eleventh layer downsampling layer are selected to upsample the image to be detected and directly concatenate the original features to obtain a fused feature map; Processing the fused feature map through deformable convolution and sigmoid activation function to obtain a mask feature map; The mask feature map is resized and upsampled to obtain a human body mask prediction map.

2. The method for detecting ticket evasion at a gate according to claim 1, characterized in that: The tracking model is formed by adding a dense optical flow method on the basis of the Fairmot model. When the tracking model uses the Fairmot model to track a target and the target is not detected within a specified number of consecutive frames, the optical flow tracking of the target is advanced by several frames from the moment it disappears until the target is found within the specified number of frames, and then the Fairmot model is continued to be used for target tracking.

3. The method for detecting ticket evasion at a gate according to claim 1, characterized in that: The step of generating a human body mask sequence video according to the tracking result and the human body mask prediction image includes: Determining a human mask prediction map having a motion trajectory according to the tracking result to obtain a screening result; The screening results are processed and merged according to time to obtain a human body mask sequence video.

4. The method for detecting ticket evasion at a gate according to claim 1, characterized in that: The behavior recognition model is formed by modifying the convolution of the time pyramid network in the time dimension into a Transformer model.

5. The method for detecting ticket evasion at a gate according to claim 4, characterized in that: The loss function used in training the behavior recognition model is the cross entropy loss function.

6. The gate machine fare evasion detection device is characterized by: include: A sampling unit is used to obtain the video captured by the camera and process the video to obtain the image to be detected; An instance segmentation unit, configured to input the image to be detected into an image model to perform human instance segmentation to obtain a human mask prediction image; A target tracking unit, configured to input the human body mask prediction image into a tracking model to perform target tracking to obtain a tracking result; A sequence video generating unit, configured to generate a human body mask sequence video according to the tracking result and the human body mask prediction image; a behavior detection unit, configured to input the human mask sequence video into a behavior recognition model to detect fare evasion behavior and obtain a detection result; an alarm unit, configured to generate a warning message when the detection result is a fare evasion behavior, and send the warning message to the terminal; The image model is formed by adding the target detection branch and the mask branch to the ParNet model; The instance segmentation unit includes a target recognition subunit, a splicing subunit, a processing subunit and an upsampling subunit; The target recognition subunit is used to input the image to be detected into the image model, and select the multimodal fusion of the ninth layer and the multimodal fusion of the tenth layer in the ParNet model to identify the target position in the image to be detected; The splicing subunit is used to select the seventh layer RepVGG SSE block of the ParNet model, the fused tenth layer, and the eleventh layer downsampling layer, upsample the image to be detected, and directly splice the original features to obtain a fused feature map; A processing subunit, configured to process the fused feature map through a deformable convolution and a sigmoid activation function to obtain a mask feature map; The upsampling subunit is used to resize and upsample the mask feature map to obtain a human body mask prediction map.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

8. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images