Cross-line crowd counting method based on individual space-time coupling feature consistency

Abstract

The invention discloses a cross-line crowd counting method based on individual space-time coupling feature consistency, which comprises the following steps: 1) inputting a plurality of continuous frames into a crowd density map prediction network based on space-time coupling features, and predicting a density map by using the crowd density map prediction network; 2) extracting positioning points representing individuals by using a local maximum detection algorithm, and extracting detection features of the individuals through the positioning points; 3) constructing an individual recognition network to perform recognition training on detection features of individuals; (4) associating individuals in different frames by using an individual association strategy, skipping the frame of the missing detection individual, and eliminating the error detection individual, and (5) carrying out the cross-line crowd counting test on the test data set in different scenarios. The method has important significance for counting the crowd scale and analyzing the crowd behavior in different geographic information environments.

Description

Technical Field

[0001] The present invention relates to the field of geographic information, and in particular to a method for counting cross-line crowds based on the consistency of individual spatiotemporal coupling features. Background Art

[0002] Information on the number of people in public places plays a vital role in public safety, traffic control, urban planning, and other fields. For example, accurately counting the number of people in public places helps to implement timely control measures, avoid overcrowding and stampedes, and prevent epidemic prevention risks caused by excessive crowds. Crowd distribution in geographic space is a key task in geographic analysis. However, surveillance cameras have a limited field of view and can only reflect the distribution of people in a local area within the field of view, making it difficult to fully cover the entire venue. Furthermore, the movement of people in surveillance videos lacks directional information. Lines of interest are marked on the corresponding locations in the surveillance video and map of necessary routes (such as entrances and exits). By counting the number of people crossing the line in the surveillance video, the size of the crowd in the entire venue can be inferred. Therefore, methods for counting people crossing the line have extremely strong practical significance.

[0003] Counting people across a line is a straightforward approach, often performed by detecting or tracking each person or specific body parts (e.g., faces, shoulders, etc.). Most methods require a large number of manually labeled bounding boxes to supervise model training. However, for densely populated crowd scenes, labeling each individual's bounding box individually is prohibitively expensive. Furthermore, to monitor a larger area, individuals in a crowd are typically small objects occupying only a small pixel area in the surveillance video. Furthermore, individuals frequently occlude each other, causing detection- or tracking-based crowd counting methods to frequently fail.

[0004] For crowd scenes with severe occlusion, a regression model can still be constructed to map the scene to the crowd density map. The density map-based model can be well applied to the method of counting people crossing the line under occlusion. The mainstream methods are divided into two categories. (1) One category is based on the method of sampling and splicing lines of interest. Some methods sample the video frames and their optical flows located on the interest line, then splice the sampling lines into a map and corresponding optical flow in chronological order, then extract clusters and construct a linear mapping relationship between clusters and the number of people, and finally count the number of people crossing the line according to the direction of the optical flow. Some methods first segment the crowd in the video according to direction, then use the interest line to sample the passing frames in chronological order and splice them into a map, and then count the number of people crossing the line in different directions in the corresponding interval on the spliced map according to the start and end time of the count; this method relies on the performance of crowd segmentation in different directions, which is itself a difficult problem in scenes with dense crowds. In addition, convolutional neural networks are used to construct the corresponding relationship between the interest line sampling spliced map and the number of people, a classification model for the exit and entry states of the spliced map optical flow, and a regression model for the ratio of exit and entry people. However, for scenes with high crowd density, slow movement or severe occlusion, the cross-line crowd counting method based on interest line sampling and splicing is not stable.

[0005] (2) Another type of method is based on crowd density map integration. Some methods use convolutional neural networks to predict the crowd density map and speed map of each pair of video frames, and then integrate the crowd density at the line of interest in the direction to obtain the number of people crossing the line in different directions. Some other methods first divide each small area into different crowd density levels, then build a crowd density prediction model for each small area of each level, and finally synthesize the speed in the small area as the direction of crowd movement in the area. Methods based on crowd density map integration generally use labeled head points to supervise the training of density map prediction models. Compared with detection or tracking-based methods, the labeling cost of training data is greatly reduced. In crowd scenes, due to background clutter and frequent occlusion, the predicted crowd density and movement direction (such as optical flow) inevitably deviate from the actual crowd state. However, this type of method does not effectively utilize the correlation between frames to improve the performance of line crossing counting. In addition, the number of people crossing the line obtained by crowd density integration is almost never equal to an integer, resulting in the counting result always being greater or less than the number of people crossing the line in actual statistics. Over time, the counting deviation of the crowd density map integration method will continue to accumulate. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the prior art and provide a method for counting people crossing the line based on the consistency of individual spatiotemporal coupling features. By extracting spatiotemporal coupling features that compensate for each other's information between frames and using them to detect individual positioning points, the problem that the number of people obtained by density integration is always greater or less than the actual number of people crossing the line is avoided. Individual recognition training is performed in the feature space to make the positioning points of the same individual more similar and the positioning points of different individuals more distinguishable. On this basis, an association strategy is designed that can skip frames of missed individuals and eliminate erroneously detected individuals.

[0007] The object of the present invention is achieved as follows: a method for counting cross-line crowds based on the consistency of individual spatiotemporal coupling features, comprising the following steps:

[0008] 1) Input several consecutive frames into the crowd density map prediction network based on spatiotemporal coupling features, and use the crowd density map prediction network to predict the density map;

[0009] 2) Using the local maximum detection algorithm to extract the positioning points representing individuals, and extracting the individual detection features through the positioning points;

[0010] 3) Construct an individual recognition network to perform recognition training on individual detection features;

[0011] 4) Use the individual association strategy to associate individuals in different frames, skip frames where individuals are missed, and exclude individuals that are incorrectly detected;

[0012] 5) Conduct cross-line crowd counting tests on test datasets in different scenarios.

[0013] Furthermore, the step 1) specifically includes: a crowd density map prediction network based on spatiotemporal coupling features is composed of spatial correlation attention and temporal correlation attention, and the process of predicting the density map is:

[0014] Use the backbone network to extract features of each frame H represents the height of the feature, W represents the width of the feature, and C represents the number of channels of the feature. The feature of consecutive T frames is Fb 1,…,T ;

[0015] Next, Facebook 1,…,T After convolution processing, we get the query of attention mechanism key Sum Used to construct spatial correlation attention SCA based on transformer design;

[0016] Then, the features are convolved to obtain the query of temporal correlation attention TCA key Sum Then the spatiotemporal coupling feature Fst is obtained 1,…,T ; Then, the spatiotemporal coupling feature Fst 1,…,T and Fb after convolution 1,…,T Make connections to predict crowd density maps;

[0017] In the crowd density prediction network, focal loss is used, and the formula is as follows:

[0018]

[0019] Where S represents the number of positive samples; N = H × W represents the total number of pixels in a frame; and D t (n) represents the predicted value of the n-th pixel in the density map of the t-th frame and the actual annotation respectively; γ is the balance factor; σ is the penalty coefficient of the background area; η is the balance coefficient of positive and negative samples.

[0020] Furthermore, the step 2) specifically includes:

[0021] Use a maximum pooling layer with a kernel of 3×3 to transform the predicted crowd density map Convert to location map Loc t ; Extract the positioning point P by the following judgment t (n):

[0022]

[0023] Among them, δ represents the adaptive threshold, which is generally the maximum value in the positioning map.

[0024] From the spatiotemporal coupling feature Fst of each frame t Extract the area around the positioning point as the individual detection feature {Dst t (i)|i=1,…,I}, where I represents the number of positioning points in the observation area in the current frame.

[0025] Furthermore, the step 3) specifically includes: using a residual block RB and a downsampled residual block DRB to construct an individual recognition network;

[0026] Individual spatiotemporal coupling detection feature Dst t (i) After passing through two residual blocks, a downsampling residual block, and a residual block, the spatiotemporal coupling feature Rst for individual recognition is output t (i); Then, the individual identification feature Rst t (i) Input softmax classifier; the individual recognition network classifies the identification features of the same individual in different frames into one category as the training target to construct the cross entropy loss calculation as follows:

[0027]

[0028] Where ID represents the total number of individual identities; Y(id) and They represent the value of the idth bit of the individual identity code and the value of the idth bit of the softmax output code respectively.

[0029] Furthermore, the step 4) specifically includes: associating the features of the same individual; assuming that in the tth frame, the spatiotemporal features of the detected individual in the observation area are Rst t (i), position is Pos t (i), and the k-th trajectory segment Tr τ,…,t (k) association, where k = 1, ..., K, K represents the number of trajectories in the observation area in the t-th frame; in the t+1-th frame, the spatiotemporal characteristics of the detected individual in the observation area are Rst t+1 (j), position is Pos t+1 (j), where j = 1,…,J, J represents the number of people in the observation area in the t+1th frame;

[0030] According to the trajectory segment Tr τ,…,t (k) The position of the individual on the Kalman filter is predicted to be in the t+1 frame as follows: and the individual position Pos in the t+1 frame t+1 The Mahalanobis distance of (j) is MD(i,j); the trajectory segment Tr τ,…,t (k) and Rst t+1 The minimum cosine distance CD(i,j) of (j) is calculated as follows:

[0031] CD(i,j)=min{1-Rst t+1 (j) tr ×Tr τ,…,t (k)}

[0032] Where min{·} represents the minimum value operation in the set; tr represents the transpose operation; MD(i,j) describes the positional relationship of individuals between adjacent frames, and CD(i,j) measures the similarity of the appearance of individuals between adjacent frames. The weighted sum of the two is used to obtain the correlation cost matrix (CM), which is calculated as follows:

[0033] CM(i,j)=λ·MD(i,j)+(1-λ)·CD(i,j)

[0034] Where λ represents the weight, 0<λ<1. On CM, the Hungarian algorithm is used to construct the trajectory segment Tr τ,…,t The correlation between the detected individuals in the t+1th frame and the decision matrix DM is also needed to eliminate false detections, which is calculated as follows:

[0035]

[0036] Among them, μ and v represent the thresholds of individual position and feature association, respectively. In order to avoid missed detection and missing trajectories, individuals and trajectory segments that have not been successfully associated will still participate in the subsequent association process for a period of time to obtain individual trajectory segments.

[0037] The present invention adopts the above technical solution, and compared with the existing technology, the beneficial effects are as follows: the method of the present invention is used to predict the crowd density map by extracting spatiotemporal coupling features that can achieve inter-frame information complementarity; since the actual cross-line counting is based on the complete individual as the counting unit, a local maximum detection algorithm is designed on the basis of the predicted crowd density map to extract the positioning point of each individual; using the positioning point to represent the individual avoids the problem that the number of people obtained by density integration is always greater than or less than the actual number of people crossing the line.

[0038] Training the recognition points in feature space ensures that the identification features of the same individual are as consistent as possible, while also enhancing the ability to distinguish between different individuals. This helps improve the performance of inter-frame individual association. During the training of the crowd density map prediction network and the individual recognition network, only the less expensive head points need to be annotated. The present invention also designs a fault-tolerant individual association strategy that can skip frames containing missed individuals and exclude individuals that were incorrectly detected, effectively improving the accuracy of cross-line counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Overall framework diagram of the present invention.

[0040] Figure 2 The present invention is based on an individual positioning graph framework diagram of spatiotemporal coupling characteristics.

[0041] Figure 3 Framework diagram of the individual identification network of the present invention.

[0042] Figure 4 Some test results of the present invention on the PETS2009 dataset.

[0043] Figure 5 Some test results of the present invention on the FDST dataset.

[0044] Figure 6 Some test results of the present invention on the UCSD dataset. DETAILED DESCRIPTION

[0045] like Figure 1 The method for counting people across lines based on the consistency of individual spatiotemporal coupling features is characterized by comprising the following steps:

[0046] 1) Since the density map of each frame is predicted separately, the correlation between video frames cannot be utilized. Therefore, several consecutive frames are input into the crowd density map prediction network based on spatiotemporal coupling features (STCFNet), and the density map is predicted using the crowd density map prediction network;

[0047] like Figure 2 As shown in Figure 2, the crowd density map prediction network based on spatiotemporal coupling features consists of spatial correlation attention and temporal correlation attention. The process of predicting the density map is as follows:

[0048] Use the backbone network to extract features of each frame H represents the height of the feature, W represents the width of the feature, and C represents the number of channels of the feature. The feature of consecutive T frames is Fb 1,…,T ;

[0049] Next, Facebook 1,…,T After convolution processing, we get the query of attention mechanism key Sum Used to construct spatial correlation attention SCA based on transformer design;

[0050] Then, the features are convolved to obtain the query of temporal correlation attention TCA key Sum Then the spatiotemporal coupling feature Fst is obtained 1,…,T ; Spatial correlation attention reduces the weight of the background area so that the feature focuses on the foreground individual representation; temporal correlation attention enhances the feature's ability to describe the crowd scene by transmitting individual information in different frames; then, the spatiotemporal coupling feature Fst 1,…,T and Fb after convolution 1,…,T Make connections to predict crowd density maps;

[0051] For individual positioning in a crowd, the positive samples only have head annotations that are very sparse relative to the background. To address this serious imbalance in the ratio of positive and negative samples, it is necessary to reduce the weight of a large number of simple background samples in training to balance the distribution of positive and negative samples. In the crowd density prediction network, the focal loss Lf is used, and the formula is as follows:

[0052]

[0053] Where S represents the number of positive samples; N = H × W represents the total number of pixels in a frame; and D t (n) represents the predicted value of the n-th pixel in the density map of the t-th frame and the actual annotation respectively; γ is the balance factor; σ is the penalty coefficient of the background area; η is the balance coefficient of positive and negative samples.

[0054] 2) Using the local maximum detection (LMD) algorithm to extract the positioning points representing individuals, and extract the individual detection features through the positioning points;

[0055] Use a maximum pooling layer with a kernel of 3×3 to transform the predicted crowd density map Convert to location map Loc t ; Extract the positioning point P by the following judgment t (n):

[0056]

[0057] Among them, δ represents the adaptive threshold, which is generally the maximum value in the positioning map.

[0058] From the spatiotemporal coupling feature Fst of each frame t Extract the area around the positioning point as the individual detection feature {Dst t (i)|i=1,…,I}, where I represents the number of positioning points in the observation area in the current frame; using positioning points to represent individuals can avoid the problem that the number of people obtained by density integration is always greater or less than the actual number of people crossing the line.

[0059] 3) Construct an individual recognition network to perform recognition training on individual detection features (IRNet);

[0060] like Figure 3 As shown in the figure, in order to avoid the gradient vanishing problem caused by retraining the features, the residual block RB and the downsampling residual block DRB are used to construct the individual recognition network; the residual network introduces residual connections, which makes it easier for the gradient to propagate to the shallower layers, thus avoiding the gradient vanishing problem;

[0061] Individual spatiotemporal coupling detection feature Dst t (i) After passing through two residual blocks, a downsampling residual block, and a residual block, the spatiotemporal coupling feature Rst for individual recognition is output t (i); Then, the individual identification feature Rst t (i) Input softmax classifier; the individual recognition network classifies the identification features of the same individual in different frames into one category as the training target to construct the cross entropy loss calculation as follows:

[0062]

[0063] Where ID represents the total number of individual identities; Y(id) and They represent the value of the idth bit of the individual identity code and the value of the idth bit of the softmax output code respectively.

[0064] 4) Utilize the Individual Association Strategy (IAS) to associate individuals in different frames, skip frames where individuals are missed, and exclude individuals that are incorrectly detected;

[0065] Associating the features of the same individual; assuming that in the tth frame, the spatiotemporal features of the detected individual in the observation area are Rst t (i), position is Pos t (i), and the k-th trajectory segment Tr τ,…,t (k) association, where k = 1, ..., K, K represents the number of trajectories in the observation area in the t-th frame; in the t+1-th frame, the spatiotemporal characteristics of the detected individual in the observation area are Rst t+1 (j), position is Pos t+1 (j), where j = 1,…,J, J represents the number of people in the observation area in the t+1th frame;

[0066] According to the trajectory segment Tr τ,…,t (k) The position of the individual on the Kalman filter is predicted to be in the t+1 frame as follows: and the individual position Pos in the t+1 frame t+1 The Mahalanobis distance of (j) is MD(i,j); the trajectory segment Tr τ,…,t (k) and Rst t+1 The minimum cosine distance CD(i,j) of (j) is calculated as follows:

[0067] CD(i,j)=min{1-Rst t+1 (j) tr ×Tr τ,…,t (k)}

[0068] Where min{·} represents the minimum value operation in the set; tr represents the transpose operation; MD(i,j) describes the positional relationship of individuals between adjacent frames, and CD(i,j) measures the similarity of the appearance of individuals between adjacent frames. The weighted sum of the two is used to obtain the correlation cost matrix (CM), which is calculated as follows:

[0069] CM(i,j)=λ·MD(i,j)+(1-λ)·CD(i,j)

[0070] Where λ represents the weight, 0<λ<1. On CM, the Hungarian algorithm is used to construct the trajectory segment Tr τ,…,t The correlation between the detected individuals in the t+1th frame and the decision matrix DM is also needed to eliminate false detections, which is calculated as follows:

[0071]

[0072] Among them, μ and v represent the thresholds of individual position and feature association, respectively. In order to avoid missed detection and missing trajectories, individuals and trajectory segments that have not been successfully associated will still participate in the subsequent association process for a period of time to obtain individual trajectory segments.

[0073] 5) Conduct cross-line crowd counting tests on test datasets in different scenarios;

[0074] The cross-line crowd counting test uses three public crowd surveillance video datasets: the PETS2009 dataset, the FDST dataset, and the UCSD dataset. In crowd scenes, labeling the pixel region or bounding box of each individual is very expensive, so these three datasets only annotate the head points.

[0075] The PETS2009 dataset was captured on a university campus, with a frame resolution of 768×576 pixels. In the experiment, two crowd scenes, S0 and S2, were used for training, and a high-density crowd scene L2 from S1, totaling 201 frames, was used for testing. The difficulty of this dataset lies in the extremely high density of people in the scenes, which often results in some people being completely occluded in a video.

[0076] The FDST dataset is a video dataset with a wide variety of scenes. A scene of people crossing a zebra crossing was used for the cross-crossing counting experiment. This scene exhibits significant perspective distortion and severe mutual occlusion, making cross-crossing counting extremely challenging. In the experiment, 900 frames with a resolution of 1920×1080 pixels were used for model training, and 300 frames were used for testing.

[0077] The videos in the UCSD dataset were captured from a high-altitude camera, offering a bird's-eye view. They primarily depict traffic on campus sidewalks. The crowds in the field of view move in both directions in a natural setting, with varying density. The experiment used 800 frames at a resolution of 238 × 158 pixels as the training set and 1200 frames as the test set.

[0078] Due to the high density of people and significant mutual occlusion in the PETS2009 and FDST datasets, detection- or tracking-based methods struggle to perform effectively in these crowd scenarios. Almost all cross-line counting methods fail to report their performance in these crowd scenarios. The performance of different cross-line crowd counting methods on the PETS2009 and FDST datasets is shown in Tables 1 and 2, respectively. Method 1 is a two-stage cross-line crowd counting method based on crowd density and velocity estimation, while Method 2 is a cross-line crowd counting method based on individual positioning and tracking using HRNet. The proposed method achieves the lowest MAE and MSE on both the PETS2009 and FDST datasets, at 0.15 and 0.23, and 0.17 and 0.31, respectively. This demonstrates that the crowd density map prediction network based on spatiotemporal coupling features employed by the proposed method can achieve good performance in crowded conditions.

[0079] Table 1 Performance of different methods on the PETS2009 dataset

[0080]

[0081] Table 2 Performance of different methods on the FDST dataset

[0082]

[0083] Some test results of the method of the present invention on PETS2009, FDST and UCSD datasets are as follows: Figure 4 、 5 and 6. Figure 4-6 The red line represents the line of interest; the yellow rectangle represents the observation area; the colored dots represent the individual positioning points in the observation area; the line connecting the colored dots represents the individual trajectory; the number of people crossing the line in different directions is counted according to the direction in which the trajectory intersects the line of interest; Figure 4-6 As can be seen in the figure, the detected individual anchor points are not always located on the same body part; some are on the face, some on the top of the head, and some on the back of the head. To ensure that the features of the same individual are as consistent as possible and that different individuals are more distinguishable, it is necessary to train the anchor points for identification. Isolated individual anchor points are not shown in the figure because individuals that are incorrectly detected are excluded during the individual association process. The individual trajectory segments show that individual anchor points were missed in some frames, but this did not affect the ability to count the number of people crossing the line of interest in different directions based on the direction in which the individual trajectory segments intersect the line of interest.

[0084] The present invention provides a method for counting people crossing a line based on the consistency of individual spatiotemporal coupling features. By extracting spatiotemporal coupling features, mutual compensation of information between frames can be achieved. The extracted spatiotemporal coupling features are sequentially subjected to crowd density map prediction training and individual recognition training, so that the positioning points located at different parts of the same individual are more similar, and the distinction between positioning points of different individuals is greater. During the training process of the crowd density map prediction network and the individual recognition network, only the head points with lower cost need to be labeled. The visualization results of individual associations show that the method provided by the present invention can skip frames of missed individuals and exclude individuals that are incorrectly detected, forming robust trajectory segments for counting the number of people crossing lines in different directions. This method is of great significance for counting crowd size and analyzing crowd behavior in an environment without geographic information.

[0085] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some of the technical features therein according to the disclosed technical content without creative labor, and these substitutions and modifications are all within the protection scope of the present invention.

Claims

1. A method for counting people across lines based on the consistency of individual spatiotemporal coupling features, characterized by: The following steps are involved: 1) Input several consecutive frames into the crowd density map prediction network based on spatiotemporal coupling features, and use the crowd density map prediction network to predict the density map; 2) Using the local maximum detection algorithm to extract the positioning points representing individuals, and extracting the individual detection features through the positioning points; 3) Construct an individual recognition network to perform recognition training on individual detection features; 4) Use the individual association strategy to associate individuals in different frames, skip frames where individuals are missed, and exclude individuals that are incorrectly detected; 5) Conduct cross-line crowd counting tests on test datasets in different scenarios.

2. The method for counting people across lines based on the consistency of individual spatiotemporal coupling features according to claim 1 is characterized in that: The step 1) specifically includes: a crowd density map prediction network based on spatiotemporal coupling features is composed of spatial correlation attention and temporal correlation attention, and the process of predicting the density map is as follows: Use the backbone network to extract features of each frame H represents the height of the feature, W represents the width of the feature, and C represents the number of channels of the feature. The feature of consecutive T frames is Fb 1,…,T ; Next, Facebook 1,…,T After convolution processing, we get the query of attention mechanism key Sum Used to construct spatial correlation attention SCA based on transformer design; Then, the features are convolved to obtain the query of temporal correlation attention TCA key Sum Then the spatiotemporal coupling feature Fst is obtained 1,…,T ; Then, the spatiotemporal coupling feature Fst 1,…,T and Fb after convolution 1,…,T Make connections to predict crowd density maps; In the crowd density prediction network, focal loss is used, and the formula is as follows: Where S represents the number of positive samples; N = H × W represents the total number of pixels in a frame; and D t (n) represents the predicted value of the n-th pixel in the density map of the t-th frame and the actual annotation respectively; γ is the balance factor; σ is the penalty coefficient of the background area; η is the balance coefficient of positive and negative samples.

3. The method for counting people across lines based on the consistency of individual spatiotemporal coupling features according to claim 1 is characterized in that: The step 2) specifically includes: Use a maximum pooling layer with a kernel of 3×3 to transform the predicted crowd density map Convert to location map Loc t ; Extract the positioning point P by the following judgment t (n): Among them, δ represents the adaptive threshold, which is generally the maximum value in the positioning map. From the spatiotemporal coupling feature Fst of each frame t Extract the area around the positioning point as the individual detection feature {Dst t (i)|i=1,…,I}, where I represents the number of positioning points in the observation area in the current frame.

4. The method for counting people across lines based on the consistency of individual spatiotemporal coupling features according to claim 1 is characterized in that: The step 3) specifically includes: using the residual block RB and the down-sampled residual block DRB to construct an individual recognition network; Individual spatiotemporal coupling detection feature Dst t (i) After passing through two residual blocks, a downsampling residual block, and a residual block, the spatiotemporal coupling feature Rst for individual recognition is output t (i); Then, the individual identification feature Rst t (i) Input softmax classifier; the individual recognition network classifies the identification features of the same individual in different frames into one category as the training target to construct the cross entropy loss calculation as follows: Where ID represents the total number of individual identities; Y(id) and They represent the value of the idth bit of the individual identity code and the value of the idth bit of the softmax output code respectively.

5. The method for counting people across lines based on the consistency of individual spatiotemporal coupling features according to claim 1 is characterized in that: The step 4) specifically includes: associating the features of the same individual; assuming that in the tth frame, the spatiotemporal features of the detected individual in the observation area are Rst t (i), position is Pos t (i), and the k-th trajectory segment Tr τ,…,t (k) association, where k = 1, ..., K, K represents the number of trajectories in the observation area in the t-th frame; in the t+1-th frame, the spatiotemporal characteristics of the detected individual in the observation area are Rst t+1 (j), position is Pos t+1 (j), where j = 1,…,J, J represents the number of people in the observation area in the t+1th frame; According to the trajectory segment Tr τ,…,t (k) The position of the individual on the Kalman filter is predicted to be in the t+1 frame as follows: and the individual position Pos in the t+1 frame t+1 The Mahalanobis distance of (j) is MD(i,j); the trajectory segment Tr τ,…,t (k) and Rst t+1 The minimum cosine distance CD(i,j) of (j) is calculated as follows: CD(i,j)=min{1-Rst t+1 (j) tr ×Tr τ,…,t (k)} Where min{·} represents the minimum value operation in the set; tr represents the transpose operation; MD(i,j) describes the positional relationship of individuals between adjacent frames, and CD(i,j) measures the similarity of the appearance of individuals between adjacent frames. The weighted sum of the two is used to obtain the correlation cost matrix CM, which is calculated as follows: CM(i,j)=λ·MD(i,j)+(1-λ)·CD(i,j) Where λ represents the weight, 0<λ<1; On CM, the Hungarian algorithm is used to construct the trajectory segment Tr τ,…,t The correlation between the detected individuals in the t+1th frame and the decision matrix DM is also needed to eliminate false detections, which is calculated as follows: Among them, μ and ν represent the thresholds of individual position and feature association, respectively. In order to avoid missed detection and missing trajectories, individuals and trajectory segments that have not been successfully associated will still participate in the subsequent association process for a period of time to obtain individual trajectory segments.

Images