Intelligent detection and identification system and method for subway turnstile

Abstract

The invention discloses a subway turnstile intelligent detection and identification system and method, and the method comprises the steps: carrying out the key frame sampling and direction optical flow histogram calculation of a behavior monitoring video, and carrying out the feature extraction and reinforcement of each direction optical flow histogram through employing the image analysis and processing technology based on computer vision; therefore, according to the motion mode characteristics of the optical flow histograms in all directions in the full time domain, whether the passenger evades the fare or not is automatically judged, and whether a fare evasion early warning prompt signal is generated or not is judged. Therefore, through carrying out key frame sampling and direction optical flow histogram calculation on a passenger behavior monitoring video and utilizing a computer vision technology and an image analysis processing technology to carry out deep feature extraction and association on a behavior mode of a passenger, the system can more accurately capture and analyze behavior features of the passenger; and misjudgment or missed judgment caused by similar behavior modes is reduced, so that the identification accuracy of fare evasion behaviors is improved, and false alarms are reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to an intelligent detection and identification system and method for subway gates. Background Art

[0002] As a key channel for subway passengers to enter the paid area, the turnstile is used very frequently and is closely related to the safety and efficiency of passengers. Fare evasion refers to passengers entering or leaving the paid area of ​​public transportation systems such as subways by improper means without paying the corresponding fare or using invalid tickets. This fare evasion behavior not only violates traffic regulations and damages the economic interests of public transportation operators, but may also disrupt the order of riding and increase safety risks, such as causing congestion during emergency evacuation.

[0003] China's public patent CN113014870A proposes a subway gate fare evasion identification method based on rapid estimation of passenger posture. It collects video data of the gate area, cleans and segments the collected video to divide the gate channel. It also detects the key points of the passenger's skeleton, constructs skeleton information, and tracks the passenger's movements through a multi-target tracking algorithm. Then, it designs recognition criteria and thresholds, learns and identifies fare evasion behavior, and controls the gate to close until the passenger pays.

[0004] The above-mentioned subway gate pass fare evasion identification method based on rapid estimation of passenger posture judges fare evasion behavior by designing manually set identification criteria and thresholds. However, if the design of identification criteria and thresholds is too dependent on specific behavior patterns, it may lead to misjudgment or missed judgment. For example, a passenger carrying large luggage may be misjudged as diving to evade the fare because he bends down to pick up items, and thus be misjudged by the system as fare evasion. These misjudgments will not only trigger false alarms and increase operating costs, but may also cause inconvenience to the misjudged passengers and affect their riding experience. At the same time, if the system is too sensitive to normal behavior, it may reduce the staff's sensitivity to alarms in the long run, thereby affecting the response speed and accuracy to real fare evasion behavior.

[0005] Therefore, an optimized intelligent detection and identification system for subway gates is desired. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a subway gate intelligent detection and recognition system and method, which performs key frame sampling and directional optical flow histogram calculation on the behavior monitoring video, and uses computer vision-based image analysis and processing technology to extract and enhance the features of each directional optical flow histogram, so as to automatically judge whether the passenger is fare evading behavior according to the motion pattern characteristics of each directional optical flow histogram in the full time domain, and judge whether to generate a fare evasion warning prompt signal. In this way, by performing key frame sampling and directional optical flow histogram calculation on the passenger behavior monitoring video, and using computer vision technology and image analysis and processing technology to perform in-depth feature extraction and association of the passenger's behavior pattern, the system can more accurately capture and analyze the passenger's behavior characteristics, reduce misjudgment or missed judgment due to similar behavior patterns, thereby improving the accuracy of identifying fare evasion and reducing the occurrence of false alarms.

[0007] According to one aspect of the present application, a subway gate intelligent detection and identification system is provided, which includes:

[0008] A passenger behavior monitoring video acquisition module is used to obtain the behavior monitoring video of the monitored passenger object collected by the camera;

[0009] A behavior monitoring video key frame sampling module, used for performing key frame sparse sampling on the behavior monitoring video to obtain a time queue of behavior monitoring key frames;

[0010] A directional optical flow histogram calculation module, used for calculating the directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram;

[0011] A motion state feature extraction module, used for passing each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor to obtain a time queue of a neighborhood motion state feature vector;

[0012] A neighborhood motion state feature reverse reinforcement module, used for inputting the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature reinforcement selection network to obtain a time queue of reverse attention selection neighborhood motion state feature vector;

[0013] A motion pattern temporal semantic encoding module, used for inputting the time queue of the reverse attention selection neighborhood motion state feature vector into a temporal encoder to obtain a motion pattern temporal semantic feature;

[0014] A recognition result generation module, used to obtain a recognition result based on the temporal semantic features of the motion pattern, wherein the recognition result is used to indicate whether the behavior is fare evasion;

[0015] The fare evasion warning prompt module is used to generate a fare evasion warning prompt signal in response to the recognition result being a fare evasion behavior.

[0016] According to another aspect of the present application, a method for intelligent detection and identification of a subway gate is provided, which includes:

[0017] Obtaining behavior monitoring video of the monitored passenger object collected by the camera;

[0018] Performing key frame sparse sampling on the behavior monitoring video to obtain a time queue of behavior monitoring key frames;

[0019] Calculating a directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram;

[0020] Each directional optical flow histogram in the time queue of the directional optical flow histogram is passed through a motion state feature extractor to obtain a time queue of a neighborhood motion state feature vector;

[0021] Inputting the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature enhancement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector;

[0022] Inputting the time queue of the reverse attention selection neighborhood motion state feature vector into a time encoder to obtain a motion pattern temporal semantic feature;

[0023] Based on the temporal semantic features of the motion pattern, a recognition result is obtained, wherein the recognition result is used to indicate whether the behavior is ticket evasion;

[0024] In response to the identification result being a fare evasion behavior, a fare evasion warning prompt signal is generated.

[0025] Compared with the prior art, the present application provides a subway gate intelligent detection and identification system and method, which performs key frame sampling and directional optical flow histogram calculation on the behavior monitoring video, and uses computer vision-based image analysis and processing technology to extract and enhance the features of each directional optical flow histogram, so as to automatically judge whether the passenger is fare evading behavior according to the motion pattern characteristics of each directional optical flow histogram in the full time domain, and judge whether to generate a fare evasion warning prompt signal. In this way, by performing key frame sampling and directional optical flow histogram calculation on the passenger behavior monitoring video, and using computer vision technology and image analysis and processing technology to perform in-depth feature extraction and association of the passenger's behavior pattern, the system can more accurately capture and analyze the passenger's behavior characteristics, reduce misjudgment or missed judgment due to similar behavior patterns, thereby improving the accuracy of identifying fare evasion behavior and reducing the occurrence of false alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0027] Figure 1 A block diagram of a subway gate intelligent detection and identification system according to an embodiment of the present application;

[0028] Figure 2 A schematic diagram of data flow of a subway gate intelligent detection and identification system according to an embodiment of the present application;

[0029] Figure 3 The present invention is a flowchart of a method for intelligent detection and identification of subway gates according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0031] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0032] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0033] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0034] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0035] If the design of recognition criteria and thresholds is too dependent on specific behavior patterns, it may lead to misjudgment or missed judgment. For example, a passenger carrying large luggage may be misjudged as diving to evade the fare because he bends down to pick up the items, and thus be misjudged by the system as fare evasion. These misjudgments will not only trigger false alarms and increase operating costs, but may also cause inconvenience to the misjudged passengers and affect their riding experience. At the same time, if the system is too sensitive to normal behavior, it may reduce the staff's sensitivity to alarms in the long run, thereby affecting the response speed and accuracy to real fare evasion. Therefore, an optimized intelligent detection and recognition system for subway gates is desired.

[0036] In the technical solution of the present application, an intelligent detection and identification system for subway gates is proposed. Figure 1 A block diagram of a subway gate intelligent detection and identification system according to an embodiment of the present application; Figure 2 Schematic diagram of data flow of the intelligent detection and identification system for subway gates according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the intelligent detection and identification system 300 for subway gates includes: a passenger object behavior monitoring video acquisition module 310, which is used to obtain the behavior monitoring video of the monitored passenger object acquired by the camera; a behavior monitoring video key frame sampling module 320, which is used to perform key frame sparse sampling on the behavior monitoring video to obtain a time queue of behavior monitoring key frames; a directional optical flow histogram calculation module 330, which is used to calculate the directional optical flow histogram between each two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram; a motion state feature extraction module 340, which is used to extract each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor respectively. A time queue of neighborhood motion state feature vectors is obtained; a neighborhood motion state feature reverse reinforcement module 350 is used to input the time queue of neighborhood motion state feature vectors into a self-supervised cluster center inverse feature reinforcement selection network to obtain a time queue of reverse attention selection neighborhood motion state feature vectors; a motion pattern temporal semantic encoding module 360 ​​is used to input the time queue of reverse attention selection neighborhood motion state feature vectors into a time encoder to obtain motion pattern temporal semantic features; a recognition result generation module 370 is used to obtain a recognition result based on the motion pattern temporal semantic features, and the recognition result is used to indicate whether it is a fare evasion behavior; a fare evasion warning prompt module 380 is used to generate a fare evasion warning prompt signal in response to the recognition result being a fare evasion behavior.

[0037] In particular, the passenger object behavior monitoring video acquisition module 310 is used to obtain the behavior monitoring video of the monitored passenger object collected by the camera. It should be understood that the behavior monitoring video of the monitored passenger object can clearly record the passenger's behavior, including whether the card is swiped, whether the gate is tripped, etc. In the technical solution of the present application, the behavior pattern of the monitored passenger object in the behavior monitoring video of the monitored passenger object can be analyzed to determine whether there is fare evasion behavior, thereby improving the accuracy of identifying fare evasion behavior.

[0038] In particular, the behavior monitoring video key frame sampling module 320 is used to perform sparse sampling of key frames on the behavior monitoring video to obtain a time queue of behavior monitoring key frames. Considering that the behavior monitoring video contains a large amount of repeated video frame information, including image frame information that plays a key role in determining the passenger's ticket evasion behavior, but also contains some background redundant frames. Based on this, in order to reduce the number of behavior monitoring video frames that need to be processed, thereby reducing computational complexity and storage requirements, and extracting key behavior information from the video, in the technical solution of the present application, sparse sampling of key frames is performed on the behavior monitoring video to obtain a time queue of behavior monitoring key frames. In this way, sparse sampling helps to quickly identify and understand important events in the behavior monitoring video, providing a basis for subsequent analysis.

[0039] In particular, the directional optical flow histogram calculation module 330 is used to calculate the directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain the time queue of the directional optical flow histogram. Taking into account that the passenger's behavior changes very quickly and dynamically, in order to better clarify and accurately analyze the changes in passenger behavior, in the technical solution of the present application, the directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames is calculated to obtain the time queue of the directional optical flow histogram. In other words, the directional optical flow histogram can quantify the motion pattern of the pixel points between two behavior monitoring key frames, so as to help capture and analyze the motion changes of the behavior in the video, thereby distinguishing different behavioral actions and events more finely.

[0040] In particular, the motion state feature extraction module 340 is used to pass each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor to obtain a time queue of neighborhood motion state feature vectors. In a specific example of the present application, each directional optical flow histogram in the time queue of the directional optical flow histogram is passed through a motion state feature extractor based on a deep residual network to obtain a time queue of the neighborhood motion state feature vectors. That is, by using a motion state feature extractor based on a deep residual network with excellent image extraction and analysis capabilities to analyze and capture the behavior state of each directional optical flow histogram in the time queue of the directional optical flow histogram, the complexity of the behavioral motion state is better understood, and a time queue of neighborhood motion state feature vectors with the ability to express motion state characteristics is obtained. In particular, the essence of the motion state feature extractor based on a deep residual network here is a deep residual neural network model.

[0041] In particular, the neighborhood motion state feature reverse reinforcement module 350 is used to input the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature reinforcement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector. Considering that each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector represents the characteristic information of the behavioral motion state at a time point, it contains characteristic information that is relatively close to the intrinsic essential characteristics of the neighborhood motion state feature. Therefore, in the technical solution of the present application, the key motion features of each neighborhood motion state feature vector are strengthened based on the central features of the neighborhood motion state feature. Specifically, the time queue of the neighborhood motion state feature vector is input into the self-supervised cluster center reverse feature reinforcement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector. It is worth mentioning that the self-supervised cluster center reverse feature reinforcement selection network reversely strengthens each neighborhood motion state feature vector through reverse screening and adaptive weight adjustment, and based on the self-supervised cluster center representation vector of the neighborhood motion state feature.

[0042] In an embodiment of the present application, the time queue of the neighborhood motion state feature vector is input into the self-supervised cluster center reverse feature reinforcement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector, including: firstly, the time queue of the neighborhood motion state feature vector is subjected to self-supervised clustering processing based on energy measurement to obtain the neighborhood motion state feature self-supervised cluster center representation vector; that is, through self-supervised clustering processing, and based on the energy measurement coefficient of each feature vector, the time queue of the neighborhood motion state feature vector is subjected to self-supervised center representation vector calculation to capture the pattern and internal structure of the overall characteristics of the neighborhood motion state feature. Then, the anti-correlation coefficient between each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector is calculated to obtain a sequence of neighborhood motion state feature anti-correlation coefficients, so as to obtain the difference between the two in time series. Then, the sequence of the anti-correlation coefficients of the neighborhood motion state features is input into the Sigmoid activation function to obtain a sequence of normalized neighborhood motion state feature anti-correlation weight coefficients; and each normalized neighborhood motion state feature anti-correlation weight coefficient in the sequence of the normalized neighborhood motion state feature anti-correlation weight coefficients is used as a weight to weight each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector to obtain a sequence of neighborhood motion state anti-enhancement feature vectors. Here, the anti-correlation coefficient is normalized to adjust the balance of the weight coefficient, and each neighborhood motion state feature vector is weighted and modulated based on the normalized weight coefficient to obtain a neighborhood motion state anti-enhancement feature vector, so that the features with large differences from the self-supervisory center representation vector can be more accurately enhanced to enhance the distinction between related features. Furthermore, the positional difference between each group of corresponding neighborhood motion state feature vectors and neighborhood motion state anti-reinforcement feature vectors in the time queue of the neighborhood motion state feature vector and the sequence of the neighborhood motion state anti-reinforcement feature vector is calculated to obtain the time queue of the reverse attention selection neighborhood motion state feature vector. Here, by performing a differential calculation between the original neighborhood motion state feature vector and the neighborhood motion state anti-reinforcement feature vector, the most relevant and critical feature information in the self-supervised center representation vector is reversely screened and extracted, thereby obtaining the time queue of the reverse attention selection neighborhood motion state feature vector, which significantly improves the representation ability of key motion features while reducing the influence and interference of irrelevant features.

[0043] Among them, the process of performing energy measurement-based self-supervised clustering processing on the time queue of the neighborhood motion state feature vectors to obtain the neighborhood motion state feature self-supervised cluster center representation vector includes: based on the maximum value, mean and variance of each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vectors, determining the energy measurement coefficient of each neighborhood motion state feature vector to obtain a sequence of neighborhood motion state feature energy measurement coefficients; inputting the sequence of neighborhood motion state feature energy measurement coefficients into the softmax function to obtain a sequence of neighborhood motion state feature activation energy measurement coefficients; using the sequence of neighborhood motion state feature activation energy measurement coefficients as a sequence of weights, calculating the position-weighted sum of the time queue of the neighborhood motion state feature vectors to obtain the neighborhood motion state feature self-supervised cluster center representation vector. Specifically, based on the maximum value, mean value and variance of each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector, the process of determining the energy measurement coefficient of each neighborhood motion state feature vector to obtain a sequence of neighborhood motion state feature energy measurement coefficients includes: extracting the maximum value of the neighborhood motion state feature vector to obtain the neighborhood motion state feature maximum value; calculating the mean value and variance of the neighborhood motion state feature vector respectively to obtain the neighborhood motion state feature mean and the neighborhood motion state feature variance; multiplying the value obtained by adding the neighborhood motion state feature variance and the bias term by four To obtain a first energy measurement value of a neighborhood motion state feature; calculate the square of the difference between the maximum value of the neighborhood motion state feature and the mean value of the neighborhood motion state feature to obtain a neighborhood motion state feature difference value; add the modulated neighborhood motion state feature variance obtained by multiplying the neighborhood motion state feature variance by two, the value obtained by multiplying the bias term by two and the neighborhood motion state feature difference value to obtain a second energy measurement value of the neighborhood motion state feature; divide the first energy measurement value of the neighborhood motion state feature by the second energy measurement value of the neighborhood motion state feature to obtain the neighborhood motion state feature energy measurement coefficient.

[0044] More specifically, the process of calculating the anti-correlation coefficient between each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector to obtain a sequence of neighborhood motion state feature anti-correlation coefficients includes: calculating the positional difference between the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector to obtain a neighborhood motion state offset feature vector; calculating the covariance matrix between the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector, and calculating the inverse of each eigenvalue in the covariance matrix to obtain an anti-correlation covariance matrix; calculating the transposed vector of the neighborhood motion state offset feature vector and the product of the anti-correlation covariance matrix and the neighborhood motion state offset feature vector to obtain a neighborhood motion state feature correlation value; calculating the square root of the neighborhood motion state feature correlation value to obtain the neighborhood motion state feature anti-correlation coefficient.

[0045] In summary, in the above embodiment, the time queue of the neighborhood motion state feature vector is input into the self-supervised cluster center reverse feature enhancement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector, including: inputting the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature enhancement selection network, and processing it with the following reverse feature enhancement formula to obtain the time queue of the reverse attention selection neighborhood motion state feature vector; wherein, the reverse feature enhancement formula is:

[0046] X={x1,x2,...,x n}

[0047]

[0048]

[0049]

[0050] s ai =sigmoid[s i (x i , v c )]

[0051] X f ={s a1 x1,s a2 x2, ..., s an ·x n}

[0052]

[0053] Where X is the time sequence of the neighborhood motion state feature vector, xi is the ith neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector, x n is the nth neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector, μ i and σ i 2 denote the mean and variance of the motion state feature vector of the ith neighborhood, β is the bias term, max(x i ) represents the maximum value of the motion state feature vector of the i-th neighborhood, e i is the energy measurement coefficient of the ith neighborhood motion state feature vector, softmax(·) is the softmax function, n is the number of feature vectors in the time queue of the neighborhood motion state feature vector, v c is the self-supervised cluster center representation vector of the neighborhood motion state feature, (·) T is the transpose of the vector, S i is the covariance matrix between the ith neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector, s i (x i , v c ) is the anti-correlation coefficient between the ith neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector, sigmoid(·) is the sigmoid function, s ai is the normalized neighborhood motion state feature anti-correlation weight coefficient corresponding to the ith neighborhood motion state feature vector, X f is the sequence of anti-enhancement feature vectors of the neighborhood motion state, is subtracted by position, X s It is the time queue of the reverse attention selection neighborhood motion state feature vector.

[0054] In particular, the motion pattern temporal semantic encoding module 360 ​​is used to input the time queue of the reverse attention selection neighborhood motion state feature vector into a time encoder to obtain the motion pattern temporal semantic features. In a specific example of the present application, the time queue of the reverse attention selection neighborhood motion state feature vector is input into a time encoder based on an RNN model to obtain the motion pattern temporal semantic feature vector as the motion pattern temporal semantic feature. Considering that there are semantic correlations of motion state changes at different time scales between different reverse attention selection neighborhood motion state features in the time queue of the reverse attention selection neighborhood motion state feature vector, and the RNN model is particularly suitable for analyzing time series data and can capture cross-time scale dependencies and dynamic changes in the data. Therefore, in the technical solution of the present application, the time queue of the reverse attention selection neighborhood motion state feature vector is input into a time encoder based on an RNN model to extract the semantic dependencies of motion features between each time point, and obtain the motion pattern temporal semantic feature vector. In particular, the time encoder based on the RNN model here is essentially an RNN model.

[0055] In particular, the recognition result generation module 370 is used to obtain a recognition result based on the temporal semantic features of the motion pattern, and the recognition result is used to indicate whether it is a fare evasion behavior. In a specific example of the present application, the temporal semantic feature vector of the motion pattern is input into a classifier-based behavior identifier to obtain the recognition result, and the recognition result is used to indicate whether it is a fare evasion behavior. That is, the temporal semantic features of the motion pattern obtained by time encoding the time queue of the reverse attention selection neighborhood motion state feature vector are classified and processed to automatically determine whether the passenger is a fare evasion behavior, and determine whether to generate a fare evasion warning prompt signal. In this way, by sampling key frames and calculating directional optical flow histograms of passenger behavior monitoring videos, and using computer vision technology and image analysis and processing technology to perform in-depth feature extraction and association of passenger behavior patterns, the system can more accurately capture and analyze passenger behavior characteristics, reduce misjudgments or missed judgments caused by similar behavior patterns, thereby improving the accuracy of identifying fare evasion behaviors and reducing the occurrence of false alarms.

[0056] In a preferred example, when the time queue of the neighborhood motion state feature vector expresses the image semantic features of the directional optical flow histogram between the key frames of the surveillance video, the self-supervised cluster center inverse feature enhancement selection network will perform temporal center enhancement based on the local temporal image semantic feature distribution, so that the motion pattern temporal semantic feature vector obtained by further temporal encoding based on the RNN model has a distribution regression pattern offset caused by the multi-dimensional temporal feature association, that is, it causes the motion pattern temporal semantic feature vector to overflow the recognition result obtained by the classifier-based behavior recognizer, affecting its accuracy.

[0057] Based on this, the motion pattern temporal semantic feature vector is input into a classifier-based behavior identifier to obtain a recognition result, specifically including:

[0058] Determine the probability value of fare evasion corresponding to the fare evasion behavior obtained by inputting the motion pattern temporal semantic feature vector into the classifier-based behavior identifier, and pass the motion pattern temporal semantic feature vector through a probabilistic activation function to obtain a probabilistic motion pattern temporal semantic feature vector;

[0059] Subtracting the probabilistic temporal semantic feature vector of the motion pattern from the unit feature vector, and using the subtraction result vector as a base to calculate a power function with the ticket evasion probability value as an exponent;

[0060] Performing a dot multiplication of the power function vector and the probabilistic motion pattern temporal semantic feature vector to obtain a motion pattern temporal semantic intermediate feature vector;

[0061] Taking the absolute value of each eigenvalue of the probabilistic motion pattern temporal semantic feature vector as the base, calculating a power function with the inverse of the ticket evasion probability value as the exponent, and summing all eigenvalues ​​to obtain the motion pattern temporal semantic bias eigenvalue;

[0062] After multiplying the motion pattern temporal semantic bias feature value by a weight serving as a hyperparameter, the resultant is interpolated with the motion pattern temporal semantic intermediate feature vector to obtain an optimized motion pattern temporal semantic feature vector; and

[0063] The optimized motion pattern temporal semantic feature vector is input into a classifier-based behavior identifier to obtain a recognition result.

[0064] Wherein, let the probabilistic motion pattern temporal semantic feature vector be denoted as V, then the optimized motion pattern temporal semantic feature vector V' is expressed as:

[0065]

[0066] Where V is the probabilistic temporal semantic feature vector of the motion pattern, I is the unit feature vector, and v i ∈V, is the eigenvalue of the probabilistic motion pattern temporal semantic feature vector, (·) ⊙p is a power function with the probability of fare evasion as an exponent of the feature value of each position in the vector, where p represents the probability of fare evasion, and v i represents the i-th eigenvalue of the probabilistic motion pattern temporal semantic feature vector, Indicates point addition, represents point subtraction, ⊙ represents point multiplication, α represents the weight as a hyperparameter, and V' is the optimized motion pattern temporal semantic feature vector.

[0067] That is, for the motion pattern time series semantic feature vector, based on the convergence limit of the probability distribution of the Cauchy-Hadamard form power series, the feature set of the motion pattern time series semantic feature vector is represented by a power series based on the convergence probability, and a set convergence bounded bias based on each eigenvalue of the motion pattern time series semantic feature vector as a convergence radius is added, so as to avoid the invalid overflow of the classification result caused by the deviation of the distribution mode of the motion pattern time series semantic feature vector caused by the gradient propagation of the model parameters based on the convergence probability in the regression process of the classifier-based behavior identifier under the unified probability convergence limit standard, thereby improving the accuracy of the recognition result obtained by the classifier-based behavior identifier. In this way, by sampling the key frames and calculating the directional optical flow histogram of the passenger behavior monitoring video, and using computer vision technology and image analysis and processing technology to perform in-depth feature extraction and association of the passenger's behavior pattern, the system can more accurately capture and analyze the passenger's behavior characteristics, reduce the misjudgment or missed judgment caused by similar behavior patterns, thereby improving the recognition accuracy of fare evasion behavior and reducing the occurrence of false alarms.

[0068] In particular, the fare evasion warning prompt module 380 is used to generate a fare evasion warning prompt signal in response to the identification result being a fare evasion behavior. In one example, in response to the passenger behavior monitoring system identifying a fare evasion behavior, a fare evasion warning prompt signal is generated to remind law enforcement personnel or station staff to take action.

[0069] As described above, the intelligent detection and identification system 300 for subway gates according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with an intelligent detection and identification algorithm for subway gates. In a possible implementation, the intelligent detection and identification system 300 for subway gates according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the intelligent detection and identification system 300 for subway gates can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the intelligent detection and identification system 300 for subway gates can also be one of the many hardware modules of the wireless terminal.

[0070] Alternatively, in another example, the subway gate intelligent detection and identification system 300 and the wireless terminal may also be separate devices, and the subway gate intelligent detection and identification system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0071] Furthermore, a method for intelligent detection and identification of subway gates is also provided.

[0072] Figure 3 Flow chart of the intelligent detection and identification method of subway gate according to the embodiment of the present application. Figure 3 As shown, according to the subway gate intelligent detection and identification method of the embodiment of the present application, the steps include: S1, obtaining the behavior monitoring video of the monitored passenger object collected by the camera; S2, performing key frame sparse sampling on the behavior monitoring video to obtain a time queue of the behavior monitoring key frames; S3, calculating the directional optical flow histogram between each two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram; S4, respectively passing each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor to obtain a neighborhood A time queue of motion state feature vectors; S5, inputting the time queue of neighborhood motion state feature vectors into the self-supervised cluster center inverse feature enhancement selection network to obtain a time queue of reverse attention selection neighborhood motion state feature vectors; S6, inputting the time queue of reverse attention selection neighborhood motion state feature vectors into a time encoder to obtain motion pattern temporal semantic features; S7, obtaining a recognition result based on the motion pattern temporal semantic features, and the recognition result is used to indicate whether it is a fare evasion behavior; S8, in response to the recognition result being a fare evasion behavior, generating a fare evasion warning prompt signal.

[0073] In summary, the intelligent detection and identification method for subway gates according to the embodiment of the present application is explained, which performs key frame sampling and directional optical flow histogram calculation on the behavior monitoring video, and uses image analysis and processing technology based on computer vision to extract and enhance the features of each directional optical flow histogram, so as to automatically judge whether the passenger is evading the ticket based on the motion pattern characteristics of each directional optical flow histogram in the full time domain, and judge whether to generate a fare evasion warning prompt signal. In this way, by performing key frame sampling and directional optical flow histogram calculation on the passenger behavior monitoring video, and using computer vision technology and image analysis and processing technology to perform in-depth feature extraction and association of the passenger's behavior pattern, the system can more accurately capture and analyze the passenger's behavior characteristics, reduce misjudgment or missed judgment due to similar behavior patterns, thereby improving the accuracy of identifying fare evasion and reducing the occurrence of false alarms.

[0074] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A subway gate intelligent detection and identification system, characterized in that: include: A passenger object behavior monitoring video acquisition module is used to obtain the behavior monitoring video of the monitored passenger object collected by the camera; A behavior monitoring video key frame sampling module, used for performing key frame sparse sampling on the behavior monitoring video to obtain a time queue of behavior monitoring key frames; A directional optical flow histogram calculation module, used for calculating the directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram; A motion state feature extraction module, used for passing each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor to obtain a time queue of a neighborhood motion state feature vector; A neighborhood motion state feature reverse reinforcement module, used for inputting the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature reinforcement selection network to obtain a time queue of reverse attention selection neighborhood motion state feature vector; A motion pattern temporal semantic encoding module, used for inputting the time queue of the reverse attention selection neighborhood motion state feature vector into a temporal encoder to obtain a motion pattern temporal semantic feature; A recognition result generation module, used to obtain a recognition result based on the temporal semantic features of the motion pattern, wherein the recognition result is used to indicate whether the behavior is fare evasion; The fare evasion warning prompt module is used to generate a fare evasion warning prompt signal in response to the recognition result being a fare evasion behavior.

2. The intelligent detection and identification system for subway gates according to claim 1 is characterized in that: The motion state feature extraction module is used to: pass each directional optical flow histogram in the time queue of the directional optical flow histogram through a motion state feature extractor based on a deep residual network to obtain the time queue of the neighborhood motion state feature vector.

3. The intelligent detection and identification system for subway gates according to claim 2 is characterized in that: The neighborhood motion state feature reverse enhancement module includes: A neighborhood motion state feature self-supervised clustering processing unit, used for performing self-supervised clustering processing based on energy measurement on the time queue of the neighborhood motion state feature vector to obtain a neighborhood motion state feature self-supervised cluster center representation vector; A neighborhood motion state feature anti-correlation coefficient calculation unit, used to calculate the anti-correlation coefficient between each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector to obtain a sequence of neighborhood motion state feature anti-correlation coefficients; A normalization unit, used for inputting the sequence of anti-correlation coefficients of the neighborhood motion state features into a Sigmoid activation function to obtain a sequence of normalized neighborhood motion state feature anti-correlation weight coefficients; A neighborhood motion state anti-enhancement unit, configured to weight each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector using each normalized neighborhood motion state feature anti-correlation weight coefficient in the sequence of normalized neighborhood motion state feature anti-correlation weight coefficients as a weight, so as to obtain a sequence of neighborhood motion state anti-enhancement feature vectors; The reverse attention selection unit is used to calculate the positional difference between each group of corresponding neighborhood motion state feature vectors and neighborhood motion state anti-reinforcement feature vectors in the time queue of the neighborhood motion state feature vectors and the sequence of the neighborhood motion state anti-reinforcement feature vectors to obtain the time queue of the reverse attention-selected neighborhood motion state feature vectors.

4. The intelligent detection and identification system for subway gates according to claim 3 is characterized in that: The neighborhood motion state feature self-supervised clustering processing unit comprises: A neighborhood motion state feature energy measurement coefficient calculation subunit, used to determine the energy measurement coefficient of each neighborhood motion state feature vector in the time queue of the neighborhood motion state feature vector to obtain a sequence of neighborhood motion state feature energy measurement coefficients; An activation subunit, used for inputting the sequence of neighborhood motion state feature energy measurement coefficients into a softmax function to obtain a sequence of neighborhood motion state feature activation energy measurement coefficients; The neighborhood motion state feature self-supervision subunit is used to calculate the position-weighted sum of the time queue of the neighborhood motion state feature vector using the sequence of the neighborhood motion state feature activation energy measurement coefficients as the sequence of weights to obtain the neighborhood motion state feature self-supervision cluster center representation vector.

5. The intelligent detection and identification system for subway gates according to claim 4 is characterized in that: The neighborhood motion state characteristic energy measurement coefficient calculation subunit is used to: Extracting the maximum value of the neighborhood motion state feature vector to obtain the neighborhood motion state feature maximum value; Respectively calculating the mean and variance of the neighborhood motion state feature vector to obtain a neighborhood motion state feature mean and a neighborhood motion state feature variance; The value obtained by adding the neighborhood motion state feature variance and the bias term is multiplied by four to obtain a first energy metric value of the neighborhood motion state feature; Calculate the square of the difference between the maximum value of the neighborhood motion state feature and the mean value of the neighborhood motion state feature to obtain a neighborhood motion state feature difference value; The neighborhood motion state feature variance is multiplied by two to obtain a modulated neighborhood motion state feature variance, the bias term is multiplied by two to obtain a neighborhood motion state feature difference value, and the neighborhood motion state feature second energy measurement value is added together; The first energy measurement value of the neighborhood motion state feature is divided by the second energy measurement value of the neighborhood motion state feature to obtain the neighborhood motion state feature energy measurement coefficient.

6. The intelligent detection and identification system for subway gates according to claim 5 is characterized in that: The neighborhood motion state feature anti-correlation coefficient calculation unit is used to: Calculating the position difference between the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector to obtain a neighborhood motion state offset feature vector; Calculating the covariance matrix between the neighborhood motion state feature vector and the neighborhood motion state feature self-supervised cluster center representation vector, and calculating the inverse of each eigenvalue in the covariance matrix to obtain an anti-correlation covariance matrix; Calculating the transposed vector of the neighborhood motion state offset feature vector and the product of the anti-correlation covariance matrix and the neighborhood motion state offset feature vector to obtain a neighborhood motion state feature correlation value; The square root of the neighborhood motion state feature correlation value is calculated to obtain the neighborhood motion state feature anti-correlation coefficient.

7. The intelligent detection and identification system for subway gates according to claim 6 is characterized in that: The motion pattern temporal semantic encoding module is used to: input the time queue of the reverse attention selection neighborhood motion state feature vector into a time encoder based on the RNN model to obtain a motion pattern temporal semantic feature vector as the motion pattern temporal semantic feature.

8. The intelligent detection and identification system for subway gates according to claim 7 is characterized in that: The recognition result generating module is used to: input the motion pattern temporal semantic feature vector into a classifier-based behavior identifier to obtain the recognition result, and the recognition result is used to indicate whether the behavior is ticket evasion.

9. A method for intelligent detection and identification of subway gates, characterized in that: include: Obtaining behavior monitoring video of the monitored passenger object collected by the camera; Performing key frame sparse sampling on the behavior monitoring video to obtain a time queue of behavior monitoring key frames; Calculating a directional optical flow histogram between every two adjacent behavior monitoring key frames in the time queue of the behavior monitoring key frames to obtain a time queue of the directional optical flow histogram; Each directional optical flow histogram in the time queue of the directional optical flow histogram is passed through a motion state feature extractor to obtain a time queue of a neighborhood motion state feature vector; Inputting the time queue of the neighborhood motion state feature vector into the self-supervised cluster center reverse feature enhancement selection network to obtain the time queue of the reverse attention selection neighborhood motion state feature vector; Inputting the time queue of the reverse attention selection neighborhood motion state feature vector into a time encoder to obtain a motion pattern temporal semantic feature; Based on the temporal semantic features of the motion pattern, a recognition result is obtained, wherein the recognition result is used to indicate whether the behavior is ticket evasion; In response to the identification result being a fare evasion behavior, a fare evasion warning prompt signal is generated.

Images