Park management method and system based on image recognition

Abstract

The invention discloses a park management method and system based on image recognition. The method comprises the following steps: S1, generating a preprocessed park monitoring video data set; s2, performing feature extraction on the preprocessed park monitoring video data set by using a unified encoder, constructing a shared semantic feature representation encoder, and obtaining a pre-trained shared semantic image feature set; s3, feature extraction is carried out on the pre-trained shared semantic image feature set through an identity recognition branch network, and identity behavior enhancement feature representation is generated; s4, applying a structure sparse mapper to the identity behavior enhanced feature representation to obtain a park personnel behavior label; and S5, when the identity behavior matching consistency result shows that non-consistency exists, outputting abnormal alarm information. The method globally highlights key information which contributes to classification most, so that better performance is obtained in identity and behavior dual-classification tasks.

Description

Technical Field

[0001] This invention relates to the field of park management technology, and in particular to a park management method and system based on image recognition. Background Technology

[0002] With the increasing demand for security management in industrial parks, intelligent monitoring and analysis of personnel identity and behavior at park entrances and exits using image recognition technology has become an important development direction. Currently, mainstream park entrance and exit management systems mostly rely on a combination of traditional video surveillance and manual patrols, supplemented by single recognition technologies based on facial recognition or pedestrian detection. This has initially achieved the detection of personnel entry and exit records and some abnormal behaviors. However, this monitoring method based on single-task shallow feature extraction still has many technical limitations in practical applications.

[0003] First, existing park image recognition systems typically use a single identity recognition model, such as face recognition or gait recognition, which lacks joint modeling of personnel behavioral characteristics. This makes it difficult to achieve comprehensive analysis of identity and behavior, resulting in insufficient early identification and warning capabilities for abnormal personnel behavior. When complex behaviors such as personnel disguise, substitute passage, or tailgating occur, single identity recognition methods are difficult to effectively identify, and are prone to misjudgment or missed judgment.

[0004] Secondly, the feature extraction methods used in existing technologies are mostly trained based on standard convolutional neural networks. The models are not robust enough to occlusion, viewpoint changes and background complexity. In environments such as park entrances and exits where there is dense flow of people, frequent occlusion and drastic changes in lighting conditions, the features extracted by the models have serious noise interference and limited representation capabilities, making it difficult to accurately support subsequent identity verification and behavior analysis tasks.

[0005] Furthermore, current systems generally lack modeling and mining of dynamic behavioral trajectories when processing continuous video stream information. They are mostly limited to static analysis of single frames or local continuous frames, lack the ability to judge temporal consistency, and cannot effectively identify security risks caused by abnormal behavioral trajectories. At the same time, the existing technology's understanding of the relationship between personnel identity and behavioral evolution is still at the stage of static correlation analysis, lacking linkage modeling from a dynamic evolution perspective, resulting in insufficient ability to recognize abnormal patterns across time and space.

[0006] In summary, existing park management technologies have significant shortcomings in the following aspects: First, identity and behavior analysis are separated, lacking joint modeling capabilities; second, image feature extraction has weak anti-interference capabilities and poor adaptability to actual scenarios such as occlusion and lighting changes; third, the modeling of the relationship between dynamic trajectory and behavior evolution is insufficient, making it difficult to achieve efficient and accurate anomaly detection. These problems seriously restrict the demand for intelligent, refined, and dynamic development of park entrance and exit security management, and new methods are urgently needed to make up for the deficiencies of existing technologies. Summary of the Invention

[0007] One objective of this invention is to propose a park management method and system based on image recognition. This invention highlights the key information that contributes most to classification, thereby achieving better performance in both identity and behavior dual classification tasks.

[0008] An image recognition-based park management method according to an embodiment of the present invention includes the following steps:

[0009] S1. Collect raw surveillance video data continuously captured by multiple cameras at the entrance and exit of the park, construct the park surveillance video dataset, perform image preprocessing on the park surveillance video dataset, complete frame synchronization, brightness equalization, background noise suppression and target area cropping, and generate the preprocessed park surveillance video dataset.

[0010] S2. Use a unified encoder to extract features from the preprocessed park surveillance video dataset, construct a shared semantic feature representation encoder, and design gait prediction proxy task, occlusion reconstruction proxy task and cross-frame temporal sorting proxy task based on the shared semantic feature representation encoder. Use deep self-supervised learning to perform unsupervised pre-training on the shared semantic feature representation encoder to obtain a pre-trained shared semantic image feature set.

[0011] S3. Input the pre-trained shared semantic image feature set into the identity recognition branch network, extract gait features, clothing features, and posture features from the pre-trained shared semantic image feature set, and output the identity feature vector. Input the pre-trained shared semantic image feature set into the behavior recognition branch network, extract the behavior spatiotemporal features by combining the temporal modeling structure, and output the behavior feature vector. Introduce the identity and behavior joint attention module between the identity recognition branch network and the behavior recognition branch network, calculate the joint attention weight based on the identity feature vector and the behavior feature vector, and generate the identity and behavior enhanced feature representation.

[0012] S4. Apply a structural sparse mapper to the identity behavior enhancement feature representation, use sparse regularization terms to constrain the identity behavior enhancement feature representation, obtain sparse joint feature representation, use the sparse joint feature representation to complete the identity classification of park personnel in the identity recognition branch network, obtain park personnel identity labels, use the sparse joint feature representation to complete the behavior classification of park personnel in the behavior recognition branch network, obtain park personnel behavior labels.

[0013] S5. Construct dynamic identity and behavior graph nodes based on park personnel identity tags and park personnel behavior tags, and generate dynamic identity and behavior graphs according to the spatiotemporal relationship of personnel. Track the evolution path of park personnel identity and behavior in real time according to the dynamic identity and behavior graphs, detect the consistency of park personnel identity and behavior matching, and output the identity and behavior matching consistency results. When the identity and behavior matching consistency results show that there is inconsistency, trigger the anomaly detection module, combine sparse joint feature representation and dynamic identity and behavior graphs to determine abnormal behaviors such as identity spoofing, alternative passage, tailgating entry, and abnormal stay, and output abnormal alarm information to realize park management.

[0014] Optionally, S1 includes the following steps:

[0015] S11. Simultaneously collect continuously captured raw surveillance video data by deploying multiple camera devices at the entrances and exits of the park, set a unified sampling time window Δt, and record the collected raw surveillance video data as the park's raw surveillance video dataset;

[0016] S12. Perform image frame synchronization processing on the original surveillance video dataset of the park, set a unified reference timestamp set T, and match the frame images in the video segment of each camera with the closest reference timestamp to form a synchronized image sequence.

[0017] S13. Perform brightness equalization processing on the image sequence after frame synchronization. Adjust the original brightness histogram of each frame image to a balanced brightness distribution that meets the target standard through histogram equalization. Apply a background noise suppression algorithm to separate the foreground region and background region of each frame image based on the background modeling parameter set, retain only the foreground region, remove background noise, and obtain the background-suppressed image sequence.

[0018] S14. Perform target region cropping on the background-suppressed image sequence. Based on the personnel region cropping box detected in the k-th frame image of the i-th camera in the detection box set, extract the personnel region image according to the cropping box to generate the final preprocessed park surveillance video dataset V. final :

[0019]

[0020] in, 1≤k≤K represents the Kth frame image captured by the i-th camera after cropping the target region, and M represents the number of cameras. Let K represent the video segment captured by the i-th camera within the time window, and let K represent the number of frames captured by the i-th camera within the time window Δt.

[0021] Optionally, S2 includes the following steps:

[0022] S21. Transfer the preprocessed park surveillance video dataset V final The encoder is input with shared semantic features, which represent the encoder's features for each frame of the image. Feature extraction is performed to generate a preliminary image feature set F. init ;

[0023] S22. Construct a gait prediction proxy task based on the preliminary image feature set, predict the gait features of the next frame for a continuous frame feature sequence, set a gait prediction target set, and form a gait prediction training pair. Gait prediction training pairs represent the initial image features from the (k+1)th frame of the i-th camera. Given the input, predict the gait features of the (k+1)th frame.

[0024] S23. Construct an occlusion reconstruction proxy task based on the preliminary image feature set, and analyze the preliminary image features of each frame. Apply random occlusion mask m i，k Obtain occluded image features Simultaneously define the target feature set R for reconstruction. mask , forming a training program for gait prediction based on occlusion reconstruction in, This represents the complete reconstructed target feature of the initial image features of the i-th camera at timestamp k before occlusion;

[0025] S24. Construct a cross-frame temporal sorting proxy task based on the preliminary image feature set, randomly arrange the features of consecutive frames, and set the original temporal order index set S. order And generate the corresponding ranking prediction target;

[0026] S25. Construct a dynamic behavior consistency verification agent task based on the preliminary image feature set, and sample continuous gait behavior segments. And the scrambled pseudo-fragments Train the encoder to distinguish between natural action segments and fake segments;

[0027] S26. Construct a local motion trajectory generation agent task based on the preliminary image feature set, and extract feature subsets of local key regions. Based on the initial position, predict the position offset within several future frames, and define a trajectory to generate a target set T. loc ;

[0028] S27. Define the comprehensive self-supervised learning loss function using S22-S26. Includes gait prediction loss Loss due to shading and reconstruction Temporal sorting loss Dynamic behavior consistency verification loss Local motion trajectory generation loss

[0029]

[0030] Where α1, α2, α3, α4, and α5 are the weight hyperparameters of their respective supervisory agent tasks;

[0031] S28. Based on a comprehensive self-supervised learning loss function Encoder E for shared semantic feature representation shared Joint optimization training ultimately yields a pre-trained set of shared semantic image features F. shared :

[0032]

[0033] in, The shared semantic feature representation of the k-th frame image from the i-th camera is the feature vector extracted by the encoder after optimization.

[0034] Optionally, S27 includes the following steps:

[0035] S271. Calculate gait prediction loss For each gait prediction training pair, the encoder's predicted gait features for the (k+1)th frame of the i-th camera are represented by shared semantic features. Gait features of the target in the (k+1)th frame of the i-th camera in the gait prediction target set The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all gait prediction samples to obtain the gait prediction loss.

[0036]

[0037] in, The shared semantic features represent the gait features predicted by the encoder for the (k+1)th frame of the i-th camera. N represents the target gait features. gait =M(K-1) is the total number of gait prediction samples;

[0038] S272. Calculate occlusion reconstruction loss That is, for each occlusion reconstruction training pair, the encoder uses shared semantic features to represent the occluded image features. Reconstructing output features Complete reconstructed target features of the i-th camera in the k-th frame of the target feature set reconstructed by occlusion The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all occluded reconstruction samples to obtain the occlusion reconstruction loss.

[0039]

[0040] in, For encoders to extract occluded image features The reconstruction output, To correspond to the complete reconstruction of target features, N mask =MK represents the total number of occluded reconstruction samples;

[0041] S273. Calculate the time-series sorting loss That is, for each temporally ordered training pair, the encoder's prediction probability of the temporal position of the out-of-order frame sequence is represented by shared semantic features. The real time position indicator for the i-th camera at frame k in the original time sequence index set The cross-entropy between samples is used as the loss value, and the average is calculated over all time-series ranked samples to obtain the time-series ranking loss.

[0042]

[0043] in, Let c be the indicator of the i-th camera's k-th frame in real time sequence, belonging to position c. N represents the corresponding probability predicted by the encoder. order =MK represents the total number of samples in the time-series ranking;

[0044] S274. Calculate the loss for dynamic behavior consistency verification. That is, for each pair of action segments, the probability that the encoder predicts the i-th action segment as a true segment is represented by shared semantic features. With corresponding label y i The binary cross-entropy between the labels is used as the loss value, corresponding to the label y. i The value is 1 for real fragments and 0 for fake fragments. The average value over all fragments is used to obtain the dynamic behavior consistency verification loss.

[0045]

[0046] Among them, y i ∈{0,1} represents the label indicating whether the i-th behavior segment is a real segment. N represents the encoder's predicted probability of consistency of behavioral segments. cons The total number of segments;

[0047] S275. Calculate the loss for generating local motion trajectories. That is, training pairs are generated for each local trajectory, and the encoder is based on local key region features to represent the shared semantic features. Predicted future position offset Offset of the true future position corresponding to the target set generated by the local motion trajectory The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all generated trajectory samples to obtain the local motion trajectory generation loss.

[0048]

[0049] in, For encoders based on local key region features Predicted displacement vector Generate a target vector for the corresponding trajectory, where δ is the prediction step size and N is the target vector. traj =M(K-δ) is the total number of trajectory generation samples.

[0050] Optionally, S3 includes the following steps:

[0051] S31. Share the pre-trained semantic image feature set F shared Input the identity recognition branch network and the behavior recognition branch network respectively;

[0052] S32. Identity Recognition Branch Network Based on Pre-trained Shared Semantic Image Feature Set F shared Gait features were extracted separately. Clothing features With posture features The three types of features are concatenated and then used to generate an identity feature vector through a feature fusion module. in, This represents the identity features of the person corresponding to the k-th frame image of the i-th camera;

[0053] S33. Action recognition branch network based on pre-trained shared semantic image feature set F shared By extracting spatiotemporal features of behavior that include temporal changes in actions through temporal modeling structures, behavioral feature vectors are output. in, This represents the behavioral feature corresponding to the k-th frame image from the i-th camera;

[0054] S34. Introduce a joint attention module for identity and behavior between the identity recognition branch network and the behavior recognition branch network, and integrate the identity feature vector... With behavioral feature vector Perform linear mappings respectively to obtain the corresponding query vectors. key vector AND value vector Calculate joint attention weights Generate an attentional representation of identity features on behavioral features, where joint attention weights are used. Used to measure the correlation strength between person identity features and behavioral features in the k-th frame image of the i-th camera;

[0055] S35. Based on joint attention weights By weighted fusion of identity feature vectors With behavioral feature vector Enhanced identity behavior feature representation The identity and behavior enhancement feature represents the combined information of identity and behavior features from the k-th frame of the i-th camera.

[0056] Optionally, S4 includes the following steps:

[0057] S41. Enhance the representation of identity behavior with features The input is a structural sparse mapper. A structural sparse regularization constraint is applied to the identity / behavior enhancement feature representation through the structural sparse mapper to obtain a sparse joint feature representation. Sparse joint feature representation preserves the key features that contribute to identity and behavior classification in the k-th frame image of the i-th camera;

[0058] S42. Representation using sparse joint features The identity recognition branch network completes the classification of personnel in the park, and the identity classifier outputs the probability vector of the identity category. And based on probability vectors Determine the identity tag of the park personnel corresponding to the k-th frame image of the i-th camera.

[0059] S43. Representation using sparse joint features The behavior classification of park personnel is performed in the behavior recognition branch network, and the behavior classifier outputs the behavior category probability vector. And based on probability vectors Determine the behavior label of the park personnel corresponding to the k-th frame image of the i-th camera.

[0060] S44. Sparse structural regularization constraints enhance the representation of identity and behavior features. The grouping sparsity constraint is specifically defined as grouping features into independent feature groups based on identity and behavioral features, applying L2 norm constraints to each group, and optimizing the structural sparsity regularization constraint term to obtain the final sparse joint feature representation. It exhibits density within each feature group and sparsity between different groups.

[0061] Optionally, S5 includes the following steps:

[0062] S51. Construct a dynamic identity and behavior graph node set V based on park personnel identity tags and park personnel behavior tags. dynamic Each node includes a park personnel identity tag, a park personnel behavior tag, a timestamp, and camera location information. The node represents the identity and behavioral status of park personnel at a specific moment.

[0063] S52. Based on the dynamic identity behavior graph node set V dynamic The temporal and spatial relationships of each node are used to generate a dynamic identity behavior graph G. dynamic In this context, the graph edges represent the continuous movement paths and behavioral evolution paths of personnel at different times or under different cameras;

[0064] S53. Based on the dynamic identity behavior graph G dynamic Real-time tracking of the evolution path of the identity and behavior of park personnel, extraction of identity and behavior matching relationship based on spatiotemporal change features between nodes, output of the consistency results of identity and behavior matching of park personnel, including consistent samples of identity and behavior matching of park personnel, slightly inconsistent samples of identity and behavior matching of park personnel, and seriously inconsistent samples of identity and behavior matching of park personnel.

[0065] S54. When the consistency result of the identity and behavior matching of park personnel indicates the presence of slightly inconsistent or seriously inconsistent samples, the anomaly detection module is triggered, combining the sparse joint feature set F. sparse With Dynamic Identity Behavior Graph G dynamic Based on the anomaly detection rules, determine the anomaly type and output anomaly alarm information:

[0066] Abnormal Identity Disguise: When park personnel are in the same or cross-camera scenarios, their identity tags show abnormal drifting, accompanied by significant shifts in their behavior tags.

[0067] Abnormal Alternative Access: Personnel from different parks exchange park personnel identity tags or park personnel identity tags do not match behavioral trajectories within a similar time window;

[0068] Follow-in anomaly: In the scenario of entering an area or passageway entrance / exit, multiple park personnel form an abnormal identity behavior following pattern within a preset time interval, and there is subsequent overlap of abnormal trajectories;

[0069] Abnormal Stay: Park personnel stay in non-preset stay areas for an extended period of time, and park personnel behavior tags show abnormal lingering or loitering behavior.

[0070] Optionally, the identity behavior matching consistency result is determined according to the following rules:

[0071] Consistent sample of park personnel identity and behavior: Park personnel maintain consistent park personnel identity labels at consecutive times, and changes in park personnel behavior labels conform to the set behavior evolution standards;

[0072] Sample of minor inconsistencies in the identity and behavior of park personnel: The identity labels of park personnel remain consistent at consecutive times, but there are abnormal changes in the behavior labels of park personnel within a preset time range, and the normal behavior pattern can be restored in the subsequent time.

[0073] Samples of serious inconsistencies in the identity and behavior of park personnel: Park personnel exhibit inconsistent identity labels at consecutive times, or abnormal changes in park personnel behavior labels persist and cannot be recovered, accompanied by abnormal identity drift or abnormal behavior deviation.

[0074] An image recognition-based park management system, used to execute an image recognition-based park management method, includes the following modules:

[0075] The video acquisition module is used to collect raw surveillance video data continuously captured by multiple cameras at the park entrances and exits, and generate a pre-processed park surveillance video dataset.

[0076] The feature extraction module is used to extract features from the preprocessed park surveillance video dataset, construct a preliminary image feature set, and define a comprehensive self-supervised learning loss function based on gait prediction, occlusion reconstruction, temporal sorting, dynamic behavior consistency verification and local motion trajectory generation agent task definition, and jointly optimize to obtain a pre-trained shared semantic image feature set.

[0077] The feature branch module includes an identity recognition branch network and a behavior recognition branch network, which are used to extract gait features, clothing features, posture features and behavior spatiotemporal features based on the pre-trained shared semantic image feature set, and output identity feature vector and behavior feature vector respectively.

[0078] The joint attention module is used to calculate joint attention weights based on identity feature vectors and behavior feature vectors, and generate identity and behavior enhanced feature representations.

[0079] The sparse mapping module is used to apply structural sparse regularization constraints to the identity and behavior enhancement feature representation, generate sparse joint feature representation, and complete the identity classification and behavior classification of park personnel based on the sparse joint feature representation, and output the park personnel identity label and park personnel behavior label respectively.

[0080] The graph construction module is used to build a dynamic identity and behavior graph based on the identity tags and behavior tags of park personnel, track the evolution path of park personnel's identity and behavior, and output the consistency results of identity and behavior matching.

[0081] The anomaly detection module is used to determine anomalies when the consistency of identity and behavior matching is abnormal. It combines the sparse joint feature set and the dynamic identity and behavior graph to output alarm information such as identity spoofing, alternative passage, tailgating, or abnormal stay.

[0082] The beneficial effects of this invention are:

[0083] (1) This invention adopts a unified encoder combined with multi-agent task self-supervised training. By introducing five agent tasks, namely gait prediction, occlusion reconstruction, cross-frame temporal sorting, dynamic behavior consistency verification and local motion trajectory generation, it can autonomously mine the deep correlation of image spatiotemporal features without relying on a large amount of manual annotation. This significantly improves the adaptability of the feature extraction module to occlusion, viewpoint changes and fine-grained behavior changes in complex entrance and exit environments.

[0084] (2) This invention introduces a joint attention module for identity and behavior between the identity recognition branch network and the behavior recognition branch network, realizing bidirectional correlation modeling of personnel identity features and behavior features. By linearly mapping identity and behavior features and calculating joint attention weights, the model can dynamically adjust the degree of attention to behavior features under different identity conditions, thereby improving the discriminability and robustness of behavior classification.

[0085] (3) In the feature mapping process, the present invention introduces a structural sparse mapper. By applying grouped sparse regularization constraints to the identity and behavior enhancement features, the decoupling and complementary expression of identity features and behavior features in the feature space are strengthened. Through sparsification, the model can highlight the key information that contributes the most to the classification globally while ensuring local density, thus achieving better performance in both identity and behavior dual classification tasks. Attached Figure Description

[0086] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0087] Figure 1 This is a flowchart of a park management method and system based on image recognition proposed in this invention;

[0088] Figure 2 This is a comparison chart of key indicators for park management between the method of this invention and traditional methods. Detailed Implementation

[0089] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0090] refer to Figures 1 to 2A campus management method based on image recognition includes the following steps:

[0091] S1. Collect raw surveillance video data continuously captured by multiple cameras at the entrance and exit of the park, construct the park surveillance video dataset, perform image preprocessing on the park surveillance video dataset, complete frame synchronization, brightness equalization, background noise suppression and target area cropping, and generate the preprocessed park surveillance video dataset.

[0092] S2. Use a unified encoder to extract features from the preprocessed park surveillance video dataset, construct a shared semantic feature representation encoder, and design gait prediction proxy task, occlusion reconstruction proxy task and cross-frame temporal sorting proxy task based on the shared semantic feature representation encoder. Use deep self-supervised learning to perform unsupervised pre-training on the shared semantic feature representation encoder to obtain a pre-trained shared semantic image feature set.

[0093] S3. Input the pre-trained shared semantic image feature set into the identity recognition branch network, extract gait features, clothing features, and posture features from the pre-trained shared semantic image feature set, and output the identity feature vector. Input the pre-trained shared semantic image feature set into the behavior recognition branch network, extract the behavior spatiotemporal features by combining the temporal modeling structure, and output the behavior feature vector. Introduce the identity and behavior joint attention module between the identity recognition branch network and the behavior recognition branch network, calculate the joint attention weight based on the identity feature vector and the behavior feature vector, and generate the identity and behavior enhanced feature representation.

[0094] S4. Apply a structural sparse mapper to the identity behavior enhancement feature representation, use sparse regularization terms to constrain the identity behavior enhancement feature representation, obtain sparse joint feature representation, use the sparse joint feature representation to complete the identity classification of park personnel in the identity recognition branch network, obtain park personnel identity labels, use the sparse joint feature representation to complete the behavior classification of park personnel in the behavior recognition branch network, obtain park personnel behavior labels.

[0095] S5. Construct dynamic identity and behavior graph nodes based on park personnel identity tags and park personnel behavior tags, and generate dynamic identity and behavior graphs according to the spatiotemporal relationship of personnel. Track the evolution path of park personnel identity and behavior in real time according to the dynamic identity and behavior graphs, detect the consistency of park personnel identity and behavior matching, and output the identity and behavior matching consistency results. When the identity and behavior matching consistency results show that there is inconsistency, trigger the anomaly detection module, combine sparse joint feature representation and dynamic identity and behavior graphs to determine abnormal behaviors such as identity spoofing, alternative passage, tailgating entry, and abnormal stay, and output abnormal alarm information to realize park management.

[0096] In this embodiment, S1 includes the following steps:

[0097] S11. Simultaneously collect continuously captured raw surveillance video data by deploying multiple camera devices at the entrances and exits of the park, set a unified sampling time window Δt, and record the collected raw surveillance video data as the park's raw surveillance video dataset;

[0098] S12. Perform image frame synchronization processing on the original surveillance video dataset of the park, set a unified reference timestamp set T, and match the frame images in the video segment of each camera with the closest reference timestamp to form a synchronized image sequence.

[0099] S13. Perform brightness equalization processing on the image sequence after frame synchronization. Adjust the original brightness histogram of each frame image to a balanced brightness distribution that meets the target standard through histogram equalization. Apply a background noise suppression algorithm to separate the foreground region and background region of each frame image based on the background modeling parameter set, retain only the foreground region, remove background noise, and obtain the background-suppressed image sequence.

[0100] S14. Perform target region cropping on the background-suppressed image sequence. Based on the personnel region cropping box detected in the k-th frame image of the i-th camera in the detection box set, extract the personnel region image according to the cropping box to generate the final preprocessed park surveillance video dataset V. final :

[0101]

[0102] in, 1≤k≤K represents the Kth frame image captured by the i-th camera after cropping the target region, and M represents the number of cameras. Let K represent the video segment captured by the i-th camera within the time window, and let K represent the number of frames captured by the i-th camera within the time window Δt.

[0103] In this embodiment, S2 includes the following steps:

[0104] S21. Transfer the preprocessed park surveillance video dataset V final The encoder is input with shared semantic features, which represent the encoder's features for each frame of the image. Feature extraction is performed to generate a preliminary image feature set F. init ;

[0105] S22. Construct a gait prediction proxy task based on the preliminary image feature set, predict the gait features of the next frame for a continuous frame feature sequence, set a gait prediction target set, and form a gait prediction training pair. Gait prediction training pairs represent the initial image features from the (k+1)th frame of the i-th camera. Given the input, predict the gait features of the (k+1)th frame.

[0106] S23. Construct an occlusion reconstruction proxy task based on the preliminary image feature set, and analyze the preliminary image features of each frame. Apply random occlusion mask m i，k Obtain occluded image features Simultaneously define the target feature set R for reconstruction. mask , forming a training program for gait prediction based on occlusion reconstruction in, This represents the complete reconstructed target feature of the initial image features of the i-th camera at timestamp k before occlusion;

[0107] S24. Construct a cross-frame temporal sorting proxy task based on the preliminary image feature set, randomly arrange the features of consecutive frames, and set the original temporal order index set S. order And generate the corresponding ranking prediction target;

[0108] S25. Construct a dynamic behavior consistency verification agent task based on the preliminary image feature set, and sample continuous gait behavior segments. And the scrambled pseudo-fragments Train the encoder to distinguish between natural action segments and fake segments;

[0109] S26. Construct a local motion trajectory generation agent task based on the preliminary image feature set, and extract feature subsets of local key regions. Based on the initial position, predict the position offset within several future frames, and define a trajectory to generate a target set T. loc ;

[0110] S27. Define the comprehensive self-supervised learning loss function using S22-S26. Includes gait prediction loss Loss due to shading and reconstruction Temporal sorting loss Dynamic behavior consistency verification loss Local motion trajectory generation loss

[0111]

[0112] Where α1, α2, α3, α4, and α5 are the weight hyperparameters of their respective supervisory agent tasks;

[0113] S28. Based on a comprehensive self-supervised learning loss function Encoder E for shared semantic feature representation shared Joint optimization training ultimately yields a pre-trained set of shared semantic image features F. shared :

[0114]

[0115] in, The shared semantic feature representation of the k-th frame image from the i-th camera is the feature vector extracted by the encoder after optimization.

[0116] In one implementation, a shared semantic feature representation encoder E shared A unified dual-stream spatiotemporal network structure is adopted: the spatial stream extracts static appearance features through four-level residual convolutional blocks, and the temporal stream extracts short-term motion features through three-dimensional convolutional blocks. The two streams are fused through cross-stream attention after each level output, and finally, a 512-dimensional frame-level semantic vector is obtained through global average pooling. Shared semantic features indicate that the encoder simultaneously receives signals from five types of proxy tasks during the pre-training phase: gait prediction, occlusion reconstruction, temporal sequencing, dynamic behavior consistency verification, and local motion trajectory generation. To ensure compatibility across tasks, a task-shared projection head is set at the encoder tail: a lightweight mapper containing two fully connected layers, layer normalization, and PReLU activation, outputting a unified 256-dimensional feature vector. In self-supervised learning loss function Under reverse optimization, the shared semantic feature representation encoder parameters are jointly updated with the five tasks, and after convergence, they are frozen into the pre-trained shared semantic image feature set F. shared The backbone network.

[0117] In this embodiment, S27 includes the following steps:

[0118] S271. Calculate gait prediction loss For each gait prediction training pair, the encoder's predicted gait features for the (k+1)th frame of the i-th camera are represented by shared semantic features. Gait features of the target in the (k+1)th frame of the i-th camera in the gait prediction target set The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all gait prediction samples to obtain the gait prediction loss.

[0119]

[0120] in, The shared semantic features represent the gait features predicted by the encoder for the (k+1)th frame of the i-th camera. N represents the target gait features. gait =M(K-1) is the total number of gait prediction samples;

[0121] S272. Calculate occlusion reconstruction loss That is, for each occlusion reconstruction training pair, the encoder uses shared semantic features to represent the occluded image features. Reconstructing output features Complete reconstructed target features of the i-th camera in the k-th frame of the target feature set reconstructed by occlusion The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all occluded reconstruction samples to obtain the occlusion reconstruction loss.

[0122]

[0123] in, For encoders to extract occluded image features The reconstruction output, To correspond to the complete reconstruction of target features, N mask =MK represents the total number of occluded reconstruction samples;

[0124] S273. Calculate the time-series sorting loss That is, for each temporally ordered training pair, the encoder's prediction probability of the temporal position of the out-of-order frame sequence is represented by shared semantic features. The real time position indicator for the i-th camera at frame k in the original time sequence index set The cross-entropy between samples is used as the loss value, and the average is calculated over all time-series ranked samples to obtain the time-series ranking loss.

[0125]

[0126] in, Let c be the indicator of the i-th camera's k-th frame in real time sequence, belonging to position c. N represents the corresponding probability predicted by the encoder. order =MK represents the total number of samples in the time-series ranking;

[0127] S274. Calculate the loss for dynamic behavior consistency verification. That is, for each pair of action segments, the probability that the encoder predicts the i-th action segment as a true segment is represented by shared semantic features. With corresponding label y i The binary cross-entropy between the labels is used as the loss value, corresponding to the label y. i The value is 1 for real fragments and 0 for fake fragments. The average value over all fragments is used to obtain the dynamic behavior consistency verification loss.

[0128]

[0129] Among them, y i ∈{0,1} represents the label indicating whether the i-th behavior segment is a real segment. N represents the encoder's predicted probability of consistency of behavioral segments. cons The total number of segments;

[0130] S275. Calculate the loss for generating local motion trajectories. That is, training pairs are generated for each local trajectory, and the encoder is based on local key region features to represent the shared semantic features. Predicted future position offset Offset of the true future position corresponding to the target set generated by the local motion trajectory The squared Euclidean distance between the samples is used as the loss value, and the average is calculated over all generated trajectory samples to obtain the local motion trajectory generation loss.

[0131]

[0132] in, For encoders based on local key region features Predicted displacement vector Generate a target vector for the corresponding trajectory, where δ is the prediction step size and N is the target vector. traj =M(K-δ) is the total number of trajectory generation samples.

[0133] In this embodiment, S3 includes the following steps:

[0134] S31. Share the pre-trained semantic image feature set F shared Input the identity recognition branch network and the behavior recognition branch network respectively;

[0135] S32. Identity Recognition Branch Network Based on Pre-trained Shared Semantic Image Feature Set F shared Gait features were extracted separately. Clothing features With posture features The three types of features are concatenated and then used to generate an identity feature vector through a feature fusion module. in, This represents the identity features of the person corresponding to the k-th frame image of the i-th camera;

[0136] S33. Action recognition branch network based on pre-trained shared semantic image feature set F shared By extracting spatiotemporal features of behavior that include temporal changes in actions through temporal modeling structures, behavioral feature vectors are output. in, This represents the behavioral feature corresponding to the k-th frame image from the i-th camera;

[0137] S34. Introduce a joint attention module for identity and behavior between the identity recognition branch network and the behavior recognition branch network, and integrate the identity feature vector... With behavioral feature vector Perform linear mappings respectively to obtain the corresponding query vectors. key vector AND value vector Calculate joint attention weights Generate an attentional representation of identity features on behavioral features, where joint attention weights are used. Used to measure the correlation strength between person identity features and behavioral features in the k-th frame image of the i-th camera;

[0138] S35. Based on joint attention weights By weighted fusion of identity feature vectors With behavioral feature vector Enhanced identity behavior feature representation The identity and behavior enhancement feature represents the combined information of identity and behavior features from the k-th frame of the i-th camera.

[0139] In one implementation, the identity recognition branch network includes:

[0140] The feature adaptation layer will share the pre-trained semantic image feature set F shared The input consists of three parallel sub-channels: the gait sub-channel uses one-dimensional convolution sliding to aggregate the lower limb movement rhythm over 10 frames; the clothing sub-channel uses adaptive channel attention to filter color and texture-sensitive channels; and the pose sub-channel uses two layers of lightweight relational convolution to focus on the neighborhood of key points on the shoulder, elbow, and knee.

[0141] The feature fusion layer concatenates the features output from the three sub-channels along their channel dimensions, followed by a 1×1 convolution to compress them to 256 dimensions and then applies BatchNorm to form a unified identity feature vector.

[0142] The classification head uses a "fully connected layer + ArcFace angular margin" structure to output an identity category probability vector. During training, cross-entropy and angle margin are used as dual constraints, and during inference, the label with the highest probability is selected.

[0143] Behavior recognition branch network

[0144] The temporal coding layer shares the pre-trained semantic image feature set F shared Stacked into a sequence along the time dimension, the input is a bidirectional gated loop unit, capturing the action dependencies within a 16-frame window before and after, and outputting a 256-dimensional temporal context vector.

[0145] The spatial enhancement layer introduces a one-time self-attention block in parallel, which supplements local action details by spatial weighting; the GRU output and the attention output are added together by residuals to obtain the behavior feature vector.

[0146] The classification head uses two fully connected layers and Softmax to generate behavior class probability vectors. Determine the behavior label based on the highest probability.

[0147] The inter-module connection ensures that frame-level features output during the inference phase are directly fed into the two branches, guaranteeing that identity and behavior share common semantic representations. The identity feature vector and behavior feature vector are then fed into the identity-behavior joint attention module to calculate the joint attention weights. Obtain identity behavior enhancement features The enhanced features are filtered for redundancy by a structural sparse mapper to generate sparse joint features. Unify the supply of identity classification headers and behavior classification headers to maintain consistent standards and dimensions in the judgment of the two tasks.

[0148] In this embodiment, S4 includes the following steps:

[0149] S41. Enhance the representation of identity behavior with features The input is a structural sparse mapper. A structural sparse regularization constraint is applied to the identity / behavior enhancement feature representation through the structural sparse mapper to obtain a sparse joint feature representation. Sparse joint feature representation preserves the key features that contribute to identity and behavior classification in the k-th frame image of the i-th camera;

[0150] S42. Representation using sparse joint features The identity recognition branch network completes the classification of personnel in the park, and the identity classifier outputs the probability vector of the identity category. And based on probability vectors Determine the identity tag of the park personnel corresponding to the k-th frame image of the i-th camera.

[0151] S43. Representation using sparse joint features The behavior classification of park personnel is performed in the behavior recognition branch network, and the behavior classifier outputs the behavior category probability vector. And based on probability vectors Determine the behavior label of the park personnel corresponding to the k-th frame image of the i-th camera.

[0152] S44. Sparse structural regularization constraints enhance the representation of identity and behavior features. The grouping sparsity constraint is specifically defined as grouping features into independent feature groups based on identity and behavioral features, applying L2 norm constraints to each group, and optimizing the structural sparsity regularization constraint term to obtain the final sparse joint feature representation. It exhibits density within each feature group and sparsity between different groups.

[0153] In this embodiment, S5 includes the following steps:

[0154] S51. Construct a dynamic identity and behavior graph node set V based on park personnel identity tags and park personnel behavior tags. dynamic Each node includes a park personnel identity tag, a park personnel behavior tag, a timestamp, and camera location information. The node represents the identity and behavioral status of park personnel at a specific moment.

[0155] S52. Based on the dynamic identity behavior graph node set V dynamic The temporal and spatial relationships of each node are used to generate a dynamic identity behavior graph G. dynamic In this context, the graph edges represent the continuous movement paths and behavioral evolution paths of personnel at different times or under different cameras;

[0156] S53. Based on the dynamic identity behavior graph G dynamic Real-time tracking of the evolution path of the identity and behavior of park personnel, extraction of identity and behavior matching relationship based on spatiotemporal change features between nodes, output of the consistency results of identity and behavior matching of park personnel, including consistent samples of identity and behavior matching of park personnel, slightly inconsistent samples of identity and behavior matching of park personnel, and seriously inconsistent samples of identity and behavior matching of park personnel.

[0157] S54. When the consistency result of the identity and behavior matching of park personnel indicates the presence of slightly inconsistent or seriously inconsistent samples, the anomaly detection module is triggered, combining the sparse joint feature set F. sparse With Dynamic Identity Behavior Graph G dynamic Based on the anomaly detection rules, determine the anomaly type and output anomaly alarm information:

[0158] Abnormal Identity Disguise: When park personnel are in the same or cross-camera scenarios, their identity tags show abnormal drifting, accompanied by significant shifts in their behavior tags.

[0159] Abnormal Alternative Access: Personnel from different parks exchange park personnel identity tags or park personnel identity tags do not match behavioral trajectories within a similar time window;

[0160] Follow-in anomaly: In the scenario of entering an area or passageway entrance / exit, multiple park personnel form an abnormal identity behavior following pattern within a preset time interval, and there is subsequent overlap of abnormal trajectories;

[0161] Abnormal Stay: Park personnel stay in non-preset stay areas for an extended period of time, and park personnel behavior tags show abnormal lingering or loitering behavior.

[0162] In this implementation, the consistency result of identity behavior matching is determined according to the following rules:

[0163] Consistent sample of park personnel identity and behavior: Park personnel maintain consistent park personnel identity labels at consecutive times, and changes in park personnel behavior labels conform to the set behavior evolution standards;

[0164] Sample of minor inconsistencies in the identity and behavior of park personnel: The identity labels of park personnel remain consistent at consecutive times, but there are abnormal changes in the behavior labels of park personnel within a preset time range, and the normal behavior pattern can be restored in the subsequent time.

[0165] Samples of serious inconsistencies in the identity and behavior of park personnel: Park personnel exhibit inconsistent identity labels at consecutive times, or abnormal changes in park personnel behavior labels persist and cannot be recovered, accompanied by abnormal identity drift or abnormal behavior deviation.

[0166] An image recognition-based park management system, used to execute an image recognition-based park management method, includes the following modules:

[0167] The video acquisition module is used to collect raw surveillance video data continuously captured by multiple cameras at the park entrances and exits, and generate a pre-processed park surveillance video dataset.

[0168] The feature extraction module is used to extract features from the preprocessed park surveillance video dataset, construct a preliminary image feature set, and define a comprehensive self-supervised learning loss function based on gait prediction, occlusion reconstruction, temporal sorting, dynamic behavior consistency verification and local motion trajectory generation agent task definition, and jointly optimize to obtain a pre-trained shared semantic image feature set.

[0169] The feature branch module includes an identity recognition branch network and a behavior recognition branch network, which are used to extract gait features, clothing features, posture features and behavior spatiotemporal features based on the pre-trained shared semantic image feature set, and output identity feature vector and behavior feature vector respectively.

[0170] The joint attention module is used to calculate joint attention weights based on identity feature vectors and behavior feature vectors, and generate identity and behavior enhanced feature representations.

[0171] The sparse mapping module is used to apply structural sparse regularization constraints to the identity and behavior enhancement feature representation, generate sparse joint feature representation, and complete the identity classification and behavior classification of park personnel based on the sparse joint feature representation, and output the park personnel identity label and park personnel behavior label respectively.

[0172] The graph construction module is used to build a dynamic identity and behavior graph based on the identity tags and behavior tags of park personnel, track the evolution path of park personnel's identity and behavior, and output the consistency results of identity and behavior matching.

[0173] The anomaly detection module is used to determine anomalies when the consistency of identity and behavior matching is abnormal. It combines the sparse joint feature set and the dynamic identity and behavior graph to output alarm information such as identity spoofing, alternative passage, tailgating, or abnormal stay.

[0174] Example 1:

[0175] To improve the security management and personnel flow monitoring at the park's entrances and exits, Company A decided to implement intelligent upgrades to its main entrances and exits. The park has eight main entrances and exits, with an average daily personnel flow of over 6,500 people. Due to its proximity to the subway station, there are frequent visitors. The traditional management model based on facial recognition and manual patrols can no longer meet the needs for efficient, accurate, and dynamic anomaly detection.

[0176] During the transformation process, Zhilian Information Technology Co., Ltd. decided to introduce the image recognition-based park management method and system of this invention, and conduct comparative tests with the original traditional single face recognition system to fully verify the feasibility and effectiveness of the method of this invention in the actual environment.

[0177] The deployment of the method of this invention begins with the data acquisition phase. Technicians installed binocular high-definition cameras at eight entrances and exits. All cameras were connected to a unified sampling time window Δt set to 3 seconds, and raw surveillance video data was collected synchronously during a 30-day test period, totaling approximately 68TB of video data. Through the image preprocessing module, the system performed frame synchronization, brightness equalization, and background noise suppression on the raw surveillance video data, ultimately generating a clear and standardized campus surveillance video dataset. The foreground separation accuracy reached 97.6%, laying a good foundation for subsequent feature extraction.

[0178] In the feature extraction stage, the unified encoder of this invention is used to perform deep feature extraction on the preprocessed dataset. Under the joint training of five self-supervised agent tasks, namely gait prediction, occlusion reconstruction, temporal sorting, dynamic behavior consistency verification and local motion trajectory generation, a pre-trained shared semantic image feature set is obtained. Actual tests show that in entrance and exit scenarios with an occlusion rate of more than 30%, the stability of the extracted features is improved by about 12.5% ​​compared with traditional CNN.

[0179] Entering the identity and behavior recognition stage, the method of this invention extracts identity features and behavior features separately through a dual-branch network, and dynamically weights and models feature associations by introducing a joint attention module for identity and behavior. In the personnel identity verification test, a total of 960 people, including 680 employees inside the park and external visitors, were selected as samples, including those wearing uniforms, hats, and backpacks, to simulate real entry and exit scenarios. The results show that the traditional method has an identity recognition accuracy of 82.3% under strong occlusion conditions, while the method of this invention has an accuracy of 91.7%, an improvement of 9.4%.

[0180] In terms of behavior recognition, the method of this invention extracts behavior patterns based on spatiotemporal features and combines them with identity features for enhanced recognition. It can accurately identify tailgating, abnormal loitering, and disguised substitution of complex behaviors. In a 30-day actual test, the system detected a total of 142 abnormal events, including 38 tailgating, 11 disguised entry, and 93 abnormal loitering. Through manual verification, the anomaly detection accuracy of the method of this invention reached 92.1%, while the anomaly detection accuracy of the original traditional method was only 81.5%, and the anomaly false negative rate was reduced from 18.5% to 7.9%.

[0181] To further verify the advantages of this invention, technicians used a comparative experimental group during the model training phase. The traditional method used 35,000 static face images as training samples and 3,500 entry / exit records as the test set; while the method of this invention used 12,000 continuous surveillance video clips as training samples, totaling approximately 380,000 frames. After training, the system underwent centralized testing at Gate 7 of the park, collecting data during peak hours (7:30 AM to 9:30 AM). The results are as follows:

[0182] Table 1 Comparison of data between the present invention and conventional methods.

[0183] project Traditional methods Method of the present invention Identity recognition accuracy 83.1％ 91.8％ Behavioral anomaly detection accuracy 78.5％ 90.6％ Improvement in recognition stability under occlusion conditions - +12.2％ Success rate of tailgating detection 74.3％ 88.5％ Average anomaly alarm response time 5.8 seconds 3.4 seconds

[0184] The data above shows that the present invention is significantly superior to traditional single recognition systems in key indicators such as joint modeling of identity and behavior, anomaly detection accuracy, occlusion robustness, and response speed.

[0185] During the application process, several incidents actually occurred. At the south gate of the park, an outsider was detected attempting to follow an employee into the park. The traditional system failed to identify the outsider in time because the outsider was following closely behind the employee and his face was obscured. However, the method of this invention successfully issued an abnormal alarm through dynamic behavior trajectory consistency analysis and gait anomaly detection. Security personnel arrived at the scene within 15 seconds to handle the situation, thus avoiding potential security risks.

[0186] For example, when the system identifies an unfamiliar person wearing a hat and clothing similar to the uniform of park employees in Area B of the East Gate of the park, the system uses the sparse mapping of identity behavior joint features of the present invention to find that the walking trajectory of the person deviates abnormally from the standard employee entry and exit habits in the behavior pattern analysis, and detects that the clothing features and gait features do not match, thus successfully marking the person as a person who has entered under a false identity and promptly intercepting the person.

[0187] Overall operational data shows that during the testing period, the method of this invention reduced the missed detection rate of abnormal events at park entrances and exits by 57% and shortened the average security response time by 41%, greatly improving the park's intelligent security level and the efficiency of personnel flow management.

[0188] In summary, the image recognition-based park management method of this invention can effectively overcome the problems of traditional methods, such as the separation of identity and behavior analysis, poor robustness of feature extraction, and insufficient accuracy of anomaly detection, in actual deployment. It significantly improves the intelligence level, response speed, and anomaly handling capability of the park management system, and has good application prospects and promotion value.

[0189] This invention employs a unified encoder combined with multi-agent task self-supervised training. By introducing five agent tasks—gait prediction, occlusion reconstruction, cross-frame temporal sorting, dynamic behavior consistency verification, and local motion trajectory generation—it autonomously mines the deep correlation of spatiotemporal features of images without relying on extensive manual annotation. This significantly improves the adaptability of the feature extraction module to occlusion, viewpoint changes, and fine-grained behavioral variations in complex entrance and exit environments.

[0190] This invention introduces a joint attention module for identity and behavior between the identity recognition branch network and the behavior recognition branch network, realizing bidirectional correlation modeling of personnel identity features and behavior features. By linearly mapping identity and behavior features and calculating joint attention weights, the model can dynamically adjust the degree of attention to behavior features under different identity conditions, thereby improving the discriminativeness and robustness of behavior classification.

[0191] This invention introduces a structural sparse mapper in the feature mapping process. By applying grouped sparse regularization constraints to the identity and behavior enhancement features, it strengthens the decoupling and complementary expression of identity features and behavior features in the feature space. Through sparsification, the model can highlight the key information that contributes the most to classification globally while ensuring local density, thereby achieving better performance in both identity and behavior dual classification tasks.

[0192] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An image recognition-based park management method, characterized by, Comprising the following steps: S1. Collecting original monitoring video data of the park entrance and performing image preprocessing to generate preprocessed park monitoring video dataset; S2. Using a unified encoder to extract features from the preprocessed park monitoring video dataset, constructing a shared semantic feature representation encoder, and based on the shared semantic feature representation encoder, designing a gait prediction proxy task, a occlusion reconstruction proxy task and a cross-frame time sequence sorting proxy task to obtain a pre-trained shared semantic image feature set; S3. Extracting features from the pre-trained shared semantic image feature set through an identity recognition branch network to output an identity feature vector, inputting the pre-trained shared semantic image feature set into a behavior recognition branch network to output a behavior feature vector, and introducing an identity and behavior joint attention module between the identity recognition branch network and the behavior recognition branch network to generate an identity and behavior enhanced feature representation; S4. Applying a structure sparse mapper to the identity and behavior enhanced feature representation to obtain a sparse joint feature representation, using the sparse joint feature representation to complete park personnel identity classification in the identity recognition branch network to obtain a park personnel identity label, and using the sparse joint feature representation to complete park personnel behavior classification in the behavior recognition branch network to obtain a park personnel behavior label; S5. Based on the park personnel identity label and the park personnel behavior label, constructing a dynamic identity and behavior graph node and generating a dynamic identity and behavior graph, tracking the identity and behavior evolution path of the park personnel in real time through the dynamic identity and behavior graph, and when the identity and behavior matching consistency result shows inconsistency, outputting an abnormal alarm information. 2.The park management method based on image recognition of claim 1, wherein, The S1 comprises the following steps: S11. Synchronously collecting continuously photographed original monitoring video data through a multi-camera device deployed at the entrance of the park, setting a unified sampling time window Δt, and collecting the obtained original monitoring video data as a park original monitoring video dataset; S12. Performing image frame synchronization processing on the park original monitoring video dataset, setting a unified reference timestamp set T to match each camera's video segment frame image to the closest reference timestamp to form a frame-synchronized image sequence; S13. Performing brightness equalization processing on the frame-synchronized image sequence, adjusting the original brightness histogram of each frame image to an equalized brightness distribution that meets the target standard through histogram equalization method, and applying a background noise suppression algorithm to separate the foreground region and the background region based on a background modeling parameter set for each frame image, only keeping the foreground region and removing the background noise to obtain a background-suppressed image sequence; S14. Perform target region cropping on the image sequence after background suppression, and crop the personnel region image according to the personnel region cropping frame in the i-th camera k-th frame image in the detection frame set to generate the final pre-processed park monitoring video dataset V final : wherein, 1≤k≤K represents the i-th camera after the target region cropping processing Kth frame image of the set, M represents the number of cameras, represents the video clip collected by the i-th camera within the time window, K represents the number of frames collected by the i-th camera within the time window Δt. 3.The park management method based on image recognition of claim 2, wherein, The S2 comprises the following steps: S21. The pre-processed park monitoring video dataset V final inputting a shared semantic feature representation encoder, and performing feature extraction on each frame of image features by the shared semantic feature representation encoder to generate a preliminary feature extraction, generating a preliminary Set of image features F init ; S22. Construct a gait prediction agent task based on the preliminary image feature set, predict the next frame gait feature for the continuous frame feature sequence, set the gait prediction target set, and form a gait prediction training pair The gait prediction training pair represents the preliminary image feature of the (k+1)th frame of the i camera The input is the preliminary image feature of the (k+1)th frame S23. Constructing an occlusion reconstruction proxy task based on the preliminary image feature set, and obtaining the complete reconstruction target feature of the preliminary image feature of each frame Applying a random occlusion mask m i,k , obtaining the occlusion image feature Simultaneously setting the reconstruction target feature set R mask , forming an occlusion reconstruction gait prediction training pair Wherein, represents the complete reconstruction target feature of the preliminary image feature before occlusion corresponding to the i-th camera at the timestamp k. S24. Construct a cross-frame time sequence ordering agent task based on the preliminary image feature set, arrange the continuous frame features in disorder, and set an original time sequence index set S order , and generate a corresponding ordering prediction target; S25. Construct a dynamic behavior consistency verification agent task based on the preliminary image feature set, and sample a continuous gait behavior segment with the scrambled pseudo segments Train the encoder to distinguish between natural action segments and fake segments; S26. Construct a local motion trajectory generation agent task based on the preliminary image feature set, and extract a local key region feature subset Based on the initial position, the position offset in the future several frames is predicted, and a trajectory generation target set T is set loc ; S27. Define a comprehensive self-supervised learning loss function by S22-S26 Inclusion of gait prediction loss Inclusion of occlusion reconstruction loss Inclusion of temporal ordering loss Inclusion of dynamic behavior consistency verification loss Inclusion of local motion trajectory generation loss Wherein, α1, α2, α3, α4, α5 are weight hyperparameters of respective supervision proxy tasks; S28. based on the integrated self-supervised learning loss function encoding the shared semantic feature representation E shared jointly optimizing training, eventually obtaining the pre-trained shared semantic image feature set F shared : wherein, is the feature vector extracted by the optimized shared semantic feature representation encoder for the i-th camera k-th frame image.

4. The park management method based on image recognition according to claim 3, characterized in that, The S27 comprises the following steps: S271. Calculate the gait prediction loss For each gait prediction training pair, the squared Euclidean distance between the i-th camera k+1 frame gait feature predicted by the shared semantic feature representation encoder and the i-th camera k+1 frame target gait feature in the gait prediction target set is taken as the loss value, and the gait prediction loss is obtained by averaging all gait prediction samples S272. Compute the occlusion reconstruction loss i.e. for each occlusion reconstruction training pair, encode the occlusion image features with the shared semantic feature representation encoder the features of the reconstruction output the Euclidean distance squared between the features of the reconstruction output and the corresponding i-th camera k-th frame complete reconstruction target feature in the occlusion reconstruction target feature set as the loss value, and average over all occlusion reconstruction samples to obtain the occlusion reconstruction loss S273. compute the temporal ordering loss i.e. for each temporal ordering training pair, the cross-entropy between the shared semantic feature representation encoder's predicted probability of the time position of the out-of-order frame sequence and the corresponding i-th camera k-th frame real time position indicator in the original time order index set is taken as the loss value, and the temporal ordering loss is obtained by averaging all temporal ordering samples S274. Calculate dynamic behavior consistency verification loss i.e. for each pair of behavior segments, the probability that the i-th behavior segment predicted by the shared semantic feature representation encoder is the real segment and the corresponding label y i between the binary cross-entropy as the loss value, the corresponding label y i is 1 when the real segment, and 0 when the fake segment, and the average of all segments is obtained Dynamic behavior consistency verification loss S275. computing a local motion trajectory generation loss i.e. for each local trajectory generation training pair, to share semantic feature representation encoder based on local key region features predicted future position offset corresponding real future position offset in the local motion trajectory generation target set the squared Euclidean distance between them as a loss value, and average all trajectory generation samples to get the local motion trajectory generation loss 5. The park management method based on image recognition according to claim 3, characterized in that, The S3 comprises the following steps: S31. share the pre-trained semantic image feature set F shared respectively input the identity recognition branch network and the behavior recognition branch network; S32. The identity recognition branch network is based on the pre-trained shared semantic image feature set F shared , respectively extracting gait features clothing features and posture features After concatenating the three types of features, the identity feature vector is generated through the feature fusion module wherein, represents the identity feature of the personnel corresponding to the kth frame of image of the ith camera; S33. The behavior recognition branch network is based on the pre-trained shared semantic image feature set F shared , extracts the behavior spatiotemporal features containing the action temporal changes through the temporal modeling structure, and outputs the behavior feature vector wherein, represents the behavior feature corresponding to the kth frame of image of the ith camera. S34. Introducing an identity-behavior joint attention module between the identity recognition branch network and the behavior recognition branch network, performing linear mapping on the identity feature vector and the behavior feature vector respectively to obtain the corresponding query vector key vector and value vector Calculate the joint attention weight Generate the attention representation of identity features to behavior features, where the joint attention weight is used to measure the correlation strength between the identity features and the behavior features of the personnel in the kth frame image of the ith camera. S35. based on the joint attention weight by weighted fusion of the identity feature vector with the behavior feature vector to obtain an enhanced identity-behavior enhanced feature representation The identity-behavior enhanced feature representation integrates joint information of identity and behavior features in the i-th camera k-th frame image.

6. The park management method based on image recognition according to claim 5, characterized in that, The S4 comprises the following steps: S41. Representing the identity-behavior enhanced feature An input structure sparse mapper, and a structure sparse constraint term is imposed on the identity-behavior enhanced feature representation by the structure sparse mapper to obtain a sparse joint feature representation The sparse joint feature representation retains key features that contribute to the classification of identity and behavior in the kth frame of the i th camera image. S42. Utilize sparse joint feature representation The park personnel identity classification is completed in the identity recognition branch network, and an identity category probability vector is output through the identity classifier And based on the probability vector The park personnel identity label corresponding to the kth frame image of the ith camera is determined S43. Utilize sparse joint feature representation The park personnel behavior classification is completed in the behavior recognition branch network, and a behavior category probability vector is output through the behavior classifier And based on the probability vector The park personnel behavior label corresponding to the kth frame image of the ith camera is determined S44. The structure sparse regularization constraint term is to enhance the feature representation of the identity behavior The group sparse constraint is specifically defined as grouping the identity feature dimensions and the behavior feature dimensions as independent features, and applying L2 norm constraint respectively. By optimizing the structure sparse regularization constraint term, the final sparse joint feature representation has density within each feature group, but sparsity between different groups.

7. The park management method based on image recognition according to claim 6, characterized in that, The S5 comprises the following steps: S51. Constructing a dynamic identity behavior graph node set V based on the park personnel identity label and the park personnel behavior label dynamic Each node includes a park personnel identity label, a park personnel behavior label, a timestamp, and camera location information. The node represents the identity and behavior state of the park personnel at a specific time. S52. A set of nodes V according to the dynamic identity behavior graph dynamic The time sequence relationship of each node is associated with the spatial position association relationship, and a dynamic identity behavior graph G is generated dynamic Wherein, the graph edge represents the continuous moving path and behavior evolution path of the personnel at different time or different cameras. S53. According to the dynamic identity behavior graph G dynamic Real-time tracking of the identity and behavior evolution path of the park personnel, extracting the identity behavior matching relationship based on the space-time change characteristics between nodes, outputting the identity behavior matching consistency results of the park personnel, including the identity behavior matching consistent sample of the park personnel, the identity behavior slightly inconsistent sample of the park personnel and the identity behavior seriously inconsistent sample of the park personnel; S54. When the park personnel identity behavior matching consistency result indicates that there is a slight inconsistency sample or a serious inconsistency sample, trigger the anomaly detection module, combine the sparse joint feature set F sparse with the dynamic identity behavior graph G dynamic , determine the anomaly type according to the anomaly detection rule, and output the anomaly alarm information: Camouflage identity anomaly: In the same camera or cross-camera scene, the park personnel identity label of the park personnel abnormally drifts and is accompanied by a significant deviation of the park personnel behavior label; Substitute access anomaly: different park personnel appear in the park personnel identity tag exchange or park personnel identity tag and behavior track mismatch in the similar time window; Tail entry anomaly: in the entrance area or channel entrance scene, multiple park personnel form an abnormal identity behavior following mode in the preset time interval, and there is an abnormal track overlap phenomenon afterwards; Abnormal stay anomaly: park personnel stay in non-pre-set stay area for a long time, and park personnel behavior label shows abnormal stay or wandering action. 8.The park management method based on image recognition of claim 7, wherein, The identity behavior matching consistency result is determined according to the following rules: Park personnel identity behavior matching consistent sample: the park personnel identity label of the park personnel at consecutive time keeps consistent, and the park personnel behavior label change conforms to the set behavior evolution standard; Park personnel identity behavior slightly inconsistent sample: the park personnel identity label of the park personnel at consecutive time keeps consistent, but the park personnel behavior label abnormally changes within the preset time range and can recover to normal behavior mode in subsequent time; Park personnel identity behavior seriously inconsistent sample: the park personnel identity label of the park personnel at consecutive time is inconsistent, or the park personnel behavior label abnormally changes and cannot be recovered, accompanied by identity abnormal drift or behavior abnormal offset.

9. An image recognition based park management system for performing an image recognition based park management method according to any one of claims 1 to 8, characterized by It includes the following modules: Video acquisition module, for collecting original monitoring video data continuously shot by multiple cameras at park entrances and exits, generating pre-processed park monitoring video data set; Feature extraction module, for feature extraction of pre-processed park monitoring video data set, constructing preliminary image feature set, defining comprehensive self-supervised learning loss function according to gait prediction, occlusion reconstruction, time sequence sorting, dynamic behavior consistency verification and local motion trajectory generation proxy tasks, and jointly optimizing to obtain pre-trained shared semantic image feature set; Feature branch module, including identity recognition branch network and behavior recognition branch network, for respectively extracting gait feature, clothing feature, posture feature and behavior spatio-temporal feature based on pre-trained shared semantic image feature set, outputting identity feature vector and behavior feature vector; Joint attention module, for calculating joint attention weight based on identity feature vector and behavior feature vector, generating identity behavior enhanced feature representation; Sparse mapping module, for applying structural sparse regularization constraint to identity behavior enhanced feature representation, generating sparse joint feature representation, and completing park personnel identity classification and behavior classification based on sparse joint feature representation, respectively outputting park personnel identity label and park personnel behavior label; Graph construction module, for constructing dynamic identity behavior graph based on park personnel identity label and park personnel behavior label, tracking park personnel identity behavior evolution path, and outputting identity behavior matching consistency result; Abnormal detection module, for when the identity behavior matching consistency is abnormal, combining sparse joint feature set and dynamic identity behavior graph for abnormal judgment, and outputting disguise identity anomaly, substitute access anomaly, tail entry anomaly or abnormal stay anomaly alarm information.

Images