Pedestrian type identification method based on monitoring video and computer readable storage medium

Abstract

The invention discloses a pedestrian type identification method based on a monitoring video and a computer readable storage medium. The method comprises the following steps: acquiring a monitoring video of a target area to construct a data set; dividing the data set into a training set and a test set; performing feature selection on the training set and the test set through a maximum correlation minimum redundancy algorithm, and determining key features for pedestrian type recognition; a Stacking integration model used for pedestrian type identification is constructed; training the Stacking integration model through the training set, and testing the test set; and outputting a corresponding predicted pedestrian type through the key features of the pedestrian in the trained defect prediction model monitoring video. According to the method, feature selection is performed through a maximum correlation minimum redundancy algorithm to determine key features for pedestrian type recognition, and pedestrian type recognition is realized through the trained Stacking integrated model, so that the accuracy of pedestrian type recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring systems and pedestrian recognition, and in particular to a method for identifying pedestrian types based on monitoring videos and a computer-readable storage medium. Background Art

[0002] The monitoring system is also called a closed-circuit television monitoring system (English: Closed-CircuitTelevision, abbreviated as CCTV). A typical monitoring system is mainly composed of five parts: front-end audio and video acquisition equipment, audio and video transmission equipment, back-end storage, control and display equipment. The back-end equipment can be further divided into central control equipment and sub-control equipment. There are many ways to construct the front-end and back-end equipment, and the connection between them (also called the transmission system) can be achieved through coaxial cable, twisted pair, optical fiber, microwave, wireless and other methods.

[0003] In crowded squares, corridors and other areas that need to be monitored, currently, equipment such as table monitors are mainly used to capture images of some pedestrians in the crowd (subject to the lens angle of view), and then perform image recognition based on various needs. Among them, identifying and classifying pedestrian behavior types (such as walkers, runners, stayers, wanderers) has the following benefits:

[0004] In terms of public safety, abnormal situations can be detected in a timely manner, such as the long stay of wanderers, which may indicate unsafe behavior or events, thus helping to prevent public safety threats such as crime and terrorist attacks. In emergency situations, such as natural disasters such as fires and earthquakes, by analyzing the movement patterns and speeds of pedestrians, people in need of assistance can be quickly located and the allocation of rescue resources can be optimized.

[0005] In terms of traffic management, it helps urban traffic managers understand pedestrian travel patterns, thereby optimizing the design and construction of infrastructure such as sidewalks and crossing facilities, and improving traffic efficiency and safety. In busy traffic areas, by identifying those who stay and wander, traffic jams caused by pedestrian congestion can be avoided, improving overall traffic flow.

[0006] In terms of business analysis, classification can reveal information such as consumers’ shopping habits and interest preferences, providing businesses with valuable market intelligence. For example, by analyzing the behavior of those who stay and those who wander, businesses can understand which products or promotions attract customers’ attention, thereby adjusting marketing strategies and improving sales performance.

[0007] However, how to design a method to realize pedestrian type recognition based on surveillance video is a technical problem that needs to be solved urgently. Summary of the invention

[0008] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a method for pedestrian type recognition based on surveillance video, determine the key features for pedestrian type recognition through feature selection by the maximum relevance minimum redundancy algorithm, and realize pedestrian type recognition through a trained Stacking integrated model, thereby improving the accuracy of pedestrian type recognition.

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] The pedestrian type recognition method based on surveillance video includes:

[0011] S1: Obtain surveillance videos of the target area to construct a dataset; divide the dataset into a training set and a test set;

[0012] S2: Perform feature selection on the training set and test set using the maximum relevance minimum redundancy algorithm to determine the key features for pedestrian type recognition;

[0013] S3: Building a Stacking ensemble model for pedestrian type recognition;

[0014] S4: Train the Stacking ensemble model using the training set and test it on the test set;

[0015] S5: The trained defect prediction model monitors the key features of pedestrians in the video and outputs the corresponding predicted pedestrian type.

[0016] Preferably, in step S1, the data set is constructed by the following steps:

[0017] S101: Decoding the surveillance video of the target area into continuous image frames;

[0018] S102: performing target detection and target tracking on the image frame, determining the pedestrian target and its moving trajectory in the image frame, and retaining the key frame containing the pedestrian target;

[0019] S103: pre-processing the key frames;

[0020] S104: labeling the preprocessed key frames by type, and adding the labeled key frames to the data set.

[0021] Preferably, in step S102, target detection and target tracking are performed through the following steps:

[0022] S1021: Performing target detection on the image frame by using the SSD algorithm to determine all pedestrian targets in the corresponding image frame;

[0023] S1022: Matching a corresponding unique labeling ID for each pedestrian target through a Sort algorithm, and tracking and generating a moving trajectory of each pedestrian target;

[0024] The Sort algorithm includes the following processing steps:

[0025] 1) Track all pedestrian targets in the current image frame through Kalman filtering, predict the state of each pedestrian target in the next image frame, and predict and generate the bounding box of each pedestrian target;

[0026] 2) The bounding box of each pedestrian target in the current image frame is matched with the bounding box of the known pedestrian target in the previous image frame by using the Hungarian algorithm. If the match is successful, the unique annotation ID and movement trajectory of the corresponding pedestrian target in the previous image frame are obtained;

[0027] 3) Match the unmatched movement track in the previous image frame with the unmatched bounding box in the current image frame through cascade matching. If the match is successful, associate the corresponding bounding box with its matched movement track;

[0028] 4) Output the unique annotation ID and updated movement trajectory of all pedestrian targets in the current image frame.

[0029] Preferably, in step S2, the original features in the training set and the test set include:

[0030] 1) Movement characteristics: movement speed, movement acceleration and movement direction;

[0031] 2) Postural characteristics: body posture and limb movements;

[0032] 3) Trajectory characteristics: movement trajectory, dwell time and wandering behavior.

[0033] Preferably, in step S2, the processing steps of the maximum relevance minimum redundancy algorithm are as follows:

[0034] S201: normalizing the feature values ​​of all features;

[0035] S202: Calculate the mutual information value between each feature and the target variable through the mutual information formula;

[0036] S203: Calculate mutual information values ​​between features using a mutual information formula;

[0037] S204: setting a preset number m of key features, and initializing an empty set as a key feature set;

[0038] S205: Select the feature with the largest mutual information with the target variable from all features and add it to the key feature set;

[0039] S206: for each remaining feature, calculate the average mutual information increment between the key feature set and the target variable after adding the feature to the key feature set, and calculate the average mutual information between the feature and the selected features in the key feature set;

[0040] S207: According to the principle of maximum relevance-minimum redundancy, select the feature that maximizes the average mutual information increment-average redundancy and add it to the key feature set;

[0041] S208: Repeat steps S206 to S207 until the number of features in the key feature set reaches a preset number m.

[0042] Preferably, in step S3, the first layer of the Stacking integrated model includes a random forest model, an XGBoost model, a gradient boosting machine model and a decision tree model, and the second layer includes a support vector machine model.

[0043] Preferably, in step S3, the hyperparameters of each model of the Stacking ensemble model are tuned by a particle swarm optimization algorithm, and the specific steps are as follows:

[0044] S301: Setting relevant parameters of the particle swarm optimization algorithm; setting the search range of the hyperparameters of the target model as the restriction conditions for the position and speed of the particles;

[0045] S302: Randomly initialize the speed, position, individual optimal solution and global optimal solution of all particles within the set search range;

[0046] S303: Calculating the fitness function value of each particle in each evolution process through the fitness function;

[0047] S304: Update the speed and position of the particle;

[0048] S305: Compare the fitness values ​​of each particle and update the individual optimal solution and the global optimal solution of each particle;

[0049] S306: Determine whether the set algorithm termination condition is met: if so, take the global optimal solution, i.e., the optimal hyperparameters of the target model, as the output result; otherwise, return to step S303.

[0050] Preferably, in step S4, the training set is input into the random forest model, XGBoost model, gradient boosting machine model and decision tree model for training according to five-fold cross validation, and the prediction results of each model are output; the prediction results of each model are integrated with the corresponding true values ​​into new training sets and new test sets, which are input into the support vector machine model, and the corresponding predicted pedestrian type is output.

[0051] Preferably, step S4 specifically includes the following steps:

[0052] S401: Divide the training set into five folds; select four folds as the training set in each training, and the remaining fold as the validation set;

[0053] S402: According to the division results in step S401, different training sets are combined to train the target model; the validation set of the corresponding group is predicted by the target model to obtain a one-fold prediction sequence validation prediction result; the test set is predicted by the target model to obtain a prediction sequence test prediction result; this cycle is repeated until all combinations are traversed, and finally five validation prediction results and five test prediction results are obtained;

[0054] S403: vertically merging the five validation prediction results in sequence into a new training feature of the prediction sequence, and averaging the five test prediction results to obtain a new test feature of the prediction sequence based on the test set;

[0055] S404: Repeat the operations from S402 to S403 for each model, and finally obtain four new training features and four new test features;

[0056] S405: horizontally merging the four new training features and the true values ​​of the endurance time corresponding to the training set in sequence to obtain a new training set;

[0057] S406: horizontally merging the four new test sets and the true values ​​of the endurance time corresponding to the test sets in sequence to obtain a new test set;

[0058] S407: Use the support vector machine model to perform training based on the new training set and the new test set and output the predicted pedestrian type.

[0059] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the method for identifying pedestrian types based on surveillance video described in the present invention is implemented.

[0060] Compared with the prior art, the pedestrian type recognition method based on surveillance video in the present invention has the following beneficial effects:

[0061] The present invention determines the key features for pedestrian type recognition by performing feature selection through the maximum relevance minimum redundancy algorithm. First, the maximum relevance minimum redundancy algorithm can minimize the redundancy between features while maintaining the maximum relevance between the features and the target variable, and select the most critical features for the classification task, that is, in pedestrian type recognition, it can screen out those features that can best distinguish different pedestrian types (such as walkers, runners, stayers and wanderers, etc.), thereby improving the accuracy of pedestrian type recognition. Secondly, by removing irrelevant or redundant features, the amount of calculation during model training can be significantly reduced, and the training speed can be accelerated; at the same time, since the number of variables that the model needs to consider is reduced, the risk of model overfitting is also reduced, making the model more stable on unseen test data. Finally, feature selection enables the model to adapt to data changes under different monitoring environments. At the same time, monitoring video data is often large and complex. The key features can be quickly and accurately screened out through the maximum relevance minimum redundancy algorithm, which can greatly shorten the time of data preprocessing and provide efficient data support for subsequent classification and recognition work.

[0062] The present invention realizes pedestrian type recognition through a trained Stacking integrated model. First, the Stacking integrated model can fully utilize the advantages of each base classifier by combining the prediction results of multiple base classifiers, thereby obtaining a higher prediction accuracy than a single classifier, and can more accurately judge the type of pedestrian in pedestrian type recognition. Secondly, since the Stacking integrated model contains multiple base classifiers, even if some of the base classifiers perform poorly under specific conditions, the overall model can still maintain a high recognition rate through the correct prediction of other base classifiers, thereby enhancing the robustness and stability of the model. Finally, pedestrian type recognition may involve features of multiple dimensions and levels, and these features may be interrelated and difficult to accurately classify individually. The Stacking integrated model can more comprehensively capture the relationship between these complex features by combining multiple base classifiers, thereby better adapting to complex classification problems; at the same time, the Stacking integrated model allows flexible addition or replacement of base classifiers to adapt to different application scenarios and data characteristics.

[0063] The present invention uses a particle swarm optimization algorithm to perform hyperparameter tuning on each model of the Stacking integrated model. First, the particle swarm algorithm is a global optimization algorithm that simulates the social interaction in the foraging behavior of bird flocks. Through information sharing and collaboration between particles, the global optimal solution is found in the solution space, so that the algorithm can more comprehensively explore the parameter space and find a parameter combination with better performance when adjusting the model hyperparameters. Secondly, during the optimization process, the particle swarm algorithm can quickly converge to a better solution through the update rules of particle speed and position. Therefore, when adjusting the model hyperparameters, a parameter combination with better performance can be found faster, saving computing resources and time. Finally, the parameters in the particle swarm algorithm (such as inertia weight, individual learning factor and social learning factor) can be adaptively adjusted according to actual conditions to better adapt to different optimization problems, so that the algorithm can maintain high efficiency and stability when facing complex hyperparameter tuning tasks.

[0064] The present invention inputs the training set into each model of the first layer according to five-fold cross-validation for training and outputting prediction results, and then integrates the prediction results of each model of the first layer with the corresponding true values ​​into a new training set and a new test set, which are input into the second layer model to output the corresponding predicted pedestrian type. First, the five-fold cross-validation divides the data set into five parts, which are used as the test set and training set for multiple training and testing in turn, so as to evaluate the performance differences of each model on different data sets, thereby judging the stability of the model. Secondly, cross-validation can reduce the model's dependence on specific training data and reduce the risk of overfitting through multiple training and testing, thereby improving the generalization ability of the model. Finally, because cross-validation provides a comprehensive evaluation of model performance, the best model selected based on the evaluation results will be able to provide more reliable and stable prediction results, thereby improving the performance of pedestrian crossing tolerance time prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to make the purpose, technical solution and advantages of the invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0066] Figure 1 This is the logical block diagram of the pedestrian type recognition method based on surveillance video. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0068] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0069] The following is a further detailed description through specific implementation methods:

[0070] Embodiment 1:

[0071] This embodiment discloses a method for identifying pedestrian types based on surveillance videos.

[0072] like Figure 1As shown, the pedestrian type recognition method based on surveillance video includes:

[0073] S1: Obtain surveillance video of the target area to construct a dataset; divide the dataset into a training set and a test set;

[0074] S2: Perform feature selection on the training set and test set through the Maximum Relevance Minimum Redundancy (mRMR) algorithm to determine the key features for pedestrian type recognition;

[0075] In this embodiment, the selected key features include moving speed, moving direction, body posture, movement trajectory, dwell time and wandering behavior.

[0076] S3: Building a Stacking ensemble model for pedestrian type recognition;

[0077] S4: Train the Stacking ensemble model using the training set and test it on the test set;

[0078] S5: The trained defect prediction model monitors the key features of pedestrians in the video (moving speed, moving direction, body posture, movement trajectory, dwell time and wandering behavior) and outputs the corresponding predicted pedestrian type.

[0079] In this embodiment, the pedestrian types include walkers, runners, stayers and wanderers.

[0080] The present invention determines the key features for pedestrian type recognition by performing feature selection through the maximum relevance minimum redundancy algorithm. First, the maximum relevance minimum redundancy algorithm can minimize the redundancy between features while maintaining the maximum relevance between the features and the target variable, and select the most critical features for the classification task, that is, in pedestrian type recognition, it can screen out those features that can best distinguish different pedestrian types (such as walkers, runners, stayers and wanderers, etc.), thereby improving the accuracy of pedestrian type recognition. Secondly, by removing irrelevant or redundant features, the amount of calculation during model training can be significantly reduced, and the training speed can be accelerated; at the same time, since the number of variables that the model needs to consider is reduced, the risk of model overfitting is also reduced, making the model more stable on unseen test data. Finally, feature selection enables the model to adapt to data changes under different monitoring environments. At the same time, monitoring video data is often large and complex. The key features can be quickly and accurately screened out through the maximum relevance minimum redundancy algorithm, which can greatly shorten the time of data preprocessing and provide efficient data support for subsequent classification and recognition work.

[0081] The present invention realizes pedestrian type recognition through a trained Stacking integrated model. First, the Stacking integrated model can fully utilize the advantages of each base classifier by combining the prediction results of multiple base classifiers, thereby obtaining a higher prediction accuracy than a single classifier, and can more accurately judge the type of pedestrian in pedestrian type recognition. Secondly, since the Stacking integrated model contains multiple base classifiers, even if some of the base classifiers perform poorly under specific conditions, the overall model can still maintain a high recognition rate through the correct prediction of other base classifiers, thereby enhancing the robustness and stability of the model. Finally, pedestrian type recognition may involve features of multiple dimensions and levels, and these features may be interrelated and difficult to accurately classify individually. The Stacking integrated model can more comprehensively capture the relationship between these complex features by combining multiple base classifiers, thereby better adapting to complex classification problems; at the same time, the Stacking integrated model allows flexible addition or replacement of base classifiers to adapt to different application scenarios and data characteristics.

[0082] In order to better introduce the technical aspects of the present invention, this embodiment is introduced through the following parts.

[0083] 1. Dataset

[0084] In this embodiment, the data set is constructed through the following steps:

[0085] S101: Decoding the surveillance video of the target area into continuous image frames;

[0086] S102: performing target detection and target tracking on the image frame, determining the pedestrian target and its moving trajectory in the image frame, and retaining the key frame containing the pedestrian target;

[0087] S103: pre-processing the key frames;

[0088] In this embodiment, the preprocessing includes image denoising, image enhancement and image binarization processing;

[0089] S104: label the key frames after preprocessing, and add the labeled key frames to the data set. 80% of the data set is used as a training set and 20% as a test set.

[0090] In this embodiment, the key frames may be annotated by manual annotation, and the annotated types include walkers, runners, stayers, and wanderers.

[0091] Specifically, target detection and target tracking are performed through the following steps:

[0092] S1021: performing target detection on the image frame by using the Single Shot MultiBox Detector (SSD) algorithm to determine all pedestrian targets in the corresponding image frame;

[0093] The processing steps of the SSD algorithm are as follows:

[0094] 1) Video input and preprocessing: The surveillance video is used as input, usually in the format of MP4, AVI, etc. The video is decoded and each frame of the image is extracted. According to the requirements of the SSD algorithm, the image is preprocessed by scaling, normalizing, etc. to meet the input size requirements of the network.

[0095] 2) Feature extraction: SSD usually uses VGG16, MobileNet, etc. as the basic network for feature extraction. These networks are pre-trained and can extract effective features in the image. Multi-scale feature map: SSD uses multiple feature maps of different levels in the basic network for object detection. These feature maps have different scales and resolutions and can capture target information of different scales in the image.

[0096] 3) Default Boxes Setting: At each position in each feature map, SSD sets multiple default boxes (also called anchor boxes). These default boxes have different aspect ratios and scales and are used to predict possible target bounding boxes. The default boxes are set based on the statistical information of the targets in the dataset to ensure that the sizes and shapes of most targets are covered.

[0097] 4) Bounding box prediction and classification: For each default box, SSD predicts its corresponding category confidence and bounding box offset. Category confidence indicates the probability that an object of a certain category exists in the default box. Bounding box offset is used to adjust the position and size of the default box to more accurately match the object boundary.

[0098] 5) Non-maximum suppression (NMS): Since multiple default boxes may predict the same object, a non-maximum suppression algorithm is needed to eliminate redundant bounding boxes. NMS retains the best bounding box based on the confidence and overlap of the bounding box, and suppresses other bounding boxes with high overlap.

[0099] 6) Result output and post-processing: After NMS processing, the remaining bounding box is output as the final detection result. The detection result can be post-processed as needed, such as bounding box smoothing, category filtering, etc.

[0100] 7) Pedestrian target detection in video: Apply the above process to each image frame in the surveillance video to achieve real-time detection of pedestrian targets.

[0101] S1022: Match each pedestrian target with a corresponding unique label ID through the Sort (Simple Online and Realtime Tracking) algorithm, and track and generate the movement trajectory of each pedestrian target.

[0102] The Sort algorithm includes the following processing steps:

[0103] 1) Track all pedestrian targets in the current image frame through Kalman filtering, predict the state of each pedestrian target in the next image frame, and predict and generate the bounding box of each pedestrian target;

[0104] 2) The bounding box of each pedestrian target in the current image frame is matched with the bounding box of the known pedestrian target in the previous image frame by using the Hungarian algorithm. If the match is successful, the unique annotation ID and movement trajectory of the corresponding pedestrian target in the previous image frame are obtained;

[0105] 3) Match the unmatched movement track in the previous image frame with the unmatched bounding box in the current image frame through cascade matching. If the match is successful, associate the corresponding bounding box with its matched movement track;

[0106] 4) Output the unique annotation ID and updated movement trajectory of all pedestrian targets in the current image frame.

[0107] 2. Feature Selection

[0108] In this embodiment, the original features in the training set and the test set include:

[0109] 1) Movement characteristics: movement speed, movement acceleration and movement direction;

[0110] In this embodiment, the moving speed can distinguish between walkers and runners, etc. The speed can be obtained by calculating the displacement of the pedestrian between consecutive frames.

[0111] The change in mobile acceleration can reflect whether the pedestrian suddenly accelerates or decelerates. It helps to identify abnormal behavior. The acceleration can be calculated by comparing the displacement changes of adjacent frames.

[0112] The moving direction can reflect the path and intention of the pedestrian. A vector can be used to represent the moving direction.

[0113] 2) Postural characteristics: body posture and limb movements;

[0114] In this embodiment, the body posture includes standing, bending, sitting, etc., which can reflect the current state of the pedestrian. The body posture and limb movements of the pedestrian can be identified by a human posture estimation algorithm (such as a method based on deep learning).

[0115] Limb movements include arm swings and leg movements, so the behavior classification can be further refined, such as distinguishing normal walking from wandering with a specific purpose. Posture can be described by extracting joint point information, such as the joint positions of the head and limbs.

[0116] 3) Trajectory characteristics: movement trajectory, dwell time and wandering behavior.

[0117] In this embodiment, the motion trajectory includes a straight line, a curve, a broken line, etc., which can reflect the behavior pattern of the pedestrian. The trajectory can be formed by recording the moving path of the pedestrian, and then the shape of the trajectory can be analyzed by using methods such as curve fitting.

[0118] The time spent in a specific area can distinguish between those who stay and those who pass by. The time a pedestrian spends in a specific area can be calculated.

[0119] Wandering behavior is repeated movement within an area without a clear purpose.

[0120] Specifically, the processing steps of the maximum relevance minimum redundancy algorithm are as follows:

[0121] S201: normalizing the feature values ​​of all features;

[0122] In this embodiment, the normalization process is to scale the feature values ​​of all features to an interval such as [0, 1] or [-1, 1].

[0123] S202: Calculate the mutual information value between each feature and the target variable (such as pedestrian type) through the mutual information formula;

[0124] In this embodiment, the target variable is set as "pedestrian" in the pedestrian type. The mutual information reflects the correlation between the feature and the target variable.

[0125] S203: Calculate mutual information values ​​between features using a mutual information formula;

[0126] In this embodiment, the mutual information value is used to evaluate the redundancy between features. A feature combination with high redundancy may contain repeated information, which is not conducive to feature selection.

[0127] S204: setting a preset number m of key features, and initializing an empty set as a key feature set;

[0128] In this embodiment, the number of key features is set to 6.

[0129] S205: Select the feature with the largest mutual information with the target variable from all features and add it to the key feature set;

[0130] S206: For each remaining feature, calculate the average mutual information increment (i.e., correlation increment) between the key feature set and the target variable after adding the feature to the key feature set, and calculate the average mutual information (i.e., redundancy) between the feature and the selected features in the key feature set;

[0131] S207: According to the principle of maximum relevance-minimum redundancy, select the feature that maximizes the average mutual information increment-average redundancy and add it to the key feature set;

[0132] S208: Repeat steps S206 to S207 until the number of features in the key feature set reaches a preset number m.

[0133] In this embodiment, the key features finally selected include moving speed, moving direction, body posture, movement trajectory, dwell time and wandering behavior.

[0134] 3. Stacking Integration Model

[0135] In this embodiment, the first layer of the Stacking integrated model includes a Random Forests (RF) model, an XGBoost model, a Gradient Boosting Machines (GBMs) model, and a Decision Trees (Decision Trees) model, and the second layer includes a Support Vector Machines (SVMs) model.

[0136] Specifically, the particle swarm optimization algorithm is used to tune the hyperparameters of each model of the Stacking ensemble model. The specific steps are as follows:

[0137] S301: Setting relevant parameters of the particle swarm optimization algorithm, including particle swarm size, maximum number of iterations, acceleration constant and inertia weight; setting the search range of hyperparameters of the target model (such as a random forest model) as a restriction condition for the position and speed of particles;

[0138] The inertia weight of the particle swarm optimization algorithm is optimized by linear reduction, that is, the inertia weight is designed to gradually decrease linearly from ω1 to ω2 with the number of iterations;

[0139] The formula is:

[0140]

[0141] S302: Randomly initialize the speed, position, individual optimal solution and global optimal solution of all particles within the set search range;

[0142] S303: Calculating the fitness function value of each particle in each evolution process through the fitness function;

[0143] In this embodiment, the fitness function takes the hyperparameters of the target model as input, performs pedestrian type recognition of the Stacking ensemble model, and returns the evaluation index of the pedestrian type recognition result as output.

[0144] S304: Update the velocity and position of the particle using the following formula;

[0145] The formula is:

[0146]

[0147] Where: is the updated particle velocity; ω is the inertia weight, which is in the range of 0.4 to 0.9; is the current particle speed; c1c2 is the learning factor; r1r2 is the random data of [0,1], used for the initial random position; is the optimal position of the current particle; is the current particle position; is the current optimal particle individual; Update the position for the particle;

[0148] S305: Compare the fitness values ​​of each particle and update the individual optimal solution and the global optimal solution of each particle;

[0149] S306: Determine whether the set algorithm termination condition is met: if so, take the global optimal solution, i.e., the optimal hyperparameters of the target model, as the output result; otherwise, return to step S303.

[0150] The present invention uses a particle swarm optimization algorithm to perform hyperparameter tuning on each model of the Stacking integrated model. First, the particle swarm algorithm is a global optimization algorithm that simulates the social interaction in the foraging behavior of bird flocks. Through information sharing and collaboration between particles, the global optimal solution is found in the solution space, so that the algorithm can more comprehensively explore the parameter space and find a parameter combination with better performance when adjusting the model hyperparameters. Secondly, during the optimization process, the particle swarm algorithm can quickly converge to a better solution through the update rules of particle speed and position. Therefore, when adjusting the model hyperparameters, a parameter combination with better performance can be found faster, saving computing resources and time. Finally, the parameters in the particle swarm algorithm (such as inertia weight, individual learning factor and social learning factor) can be adaptively adjusted according to actual conditions to better adapt to different optimization problems, so that the algorithm can maintain high efficiency and stability when facing complex hyperparameter tuning tasks.

[0151] 4. Model Training

[0152] In this embodiment, the training set is input into the random forest model, XGBoost model, gradient boosting machine model and decision tree model for training according to five-fold cross validation, and the prediction results of each model are output; the prediction results of each model are integrated with the corresponding true values ​​into new training sets and new test sets, which are input into the support vector machine model, and the corresponding predicted pedestrian type is output.

[0153] The specific steps include:

[0154] S401: Divide the training set into five folds; select four folds as the training set in each training, and the remaining fold as the validation set;

[0155] S402: According to the division results in step S401, different training sets are combined to train the target model; the validation set of the corresponding group is predicted by the target model to obtain a one-fold prediction sequence validation prediction result; the test set is predicted by the target model to obtain a prediction sequence test prediction result; this cycle is repeated until all combinations are traversed, and finally five validation prediction results and five test prediction results are obtained;

[0156] S403: vertically merging the five validation prediction results in sequence into a new training feature of the prediction sequence, and averaging the five test prediction results to obtain a new test feature of the prediction sequence based on the test set;

[0157] S404: Repeat the operations from S402 to S403 for each model, and finally obtain four new training features and four new test features;

[0158] S405: horizontally merging the four new training features and the true values ​​of the endurance time corresponding to the training set in sequence to obtain a new training set;

[0159] S406: horizontally merging the four new test sets and the true values ​​of the endurance time corresponding to the test sets in sequence to obtain a new test set;

[0160] S407: Use the support vector machine model to perform training based on the new training set and the new test set and output the predicted pedestrian type.

[0161] The present invention inputs the training set into each model of the first layer according to five-fold cross-validation for training and outputting prediction results, and then integrates the prediction results of each model of the first layer with the corresponding true values ​​into a new training set and a new test set, which are input into the second layer model to output the corresponding predicted pedestrian type. First, the five-fold cross-validation divides the data set into five parts, which are used as the test set and training set for multiple training and testing in turn, so as to evaluate the performance differences of each model on different data sets, thereby judging the stability of the model. Secondly, cross-validation can reduce the model's dependence on specific training data and reduce the risk of overfitting through multiple training and testing, thereby improving the generalization ability of the model. Finally, because cross-validation provides a comprehensive evaluation of model performance, the best model selected based on the evaluation results will be able to provide more reliable and stable prediction results, thereby improving the performance of pedestrian crossing tolerance time prediction.

[0162] Embodiment 2:

[0163] This embodiment discloses a computer-readable storage medium.

[0164] A computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed, the pedestrian type recognition method based on surveillance video described in Example 1 is implemented.

[0165] The computer program may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above method embodiments may be implemented. The computer program includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A pedestrian type recognition method based on surveillance video, characterized in that: include: S1: Obtain surveillance videos of the target area to build a dataset; Divide the dataset into training and testing sets; S2: Perform feature selection on the training set and test set using the maximum relevance minimum redundancy algorithm to determine the key features for pedestrian type recognition; S3: Building a Stacking ensemble model for pedestrian type recognition; S4: Train the Stacking ensemble model using the training set and test it on the test set; S5: The trained defect prediction model monitors the key features of pedestrians in the video and outputs the corresponding predicted pedestrian type.

2. The method for identifying pedestrian types based on surveillance video according to claim 1, characterized in that: In step S1, the data set is constructed by the following steps: S101: Decoding the surveillance video of the target area into continuous image frames; S102: performing target detection and target tracking on the image frame, determining the pedestrian target and its moving trajectory in the image frame, and retaining the key frame containing the pedestrian target; S103: pre-processing the key frames; S104: labeling the preprocessed key frames by type, and adding the labeled key frames to the data set.

3. The method for identifying pedestrian types based on surveillance video according to claim 1, characterized in that: In step S102, target detection and target tracking are performed through the following steps: S1021: Performing target detection on the image frame by using the SSD algorithm to determine all pedestrian targets in the corresponding image frame; S1022: Matching a corresponding unique labeling ID for each pedestrian target through a Sort algorithm, and tracking and generating a moving trajectory of each pedestrian target; The Sort algorithm includes the following processing steps: 1) Track all pedestrian targets in the current image frame through Kalman filtering, predict the state of each pedestrian target in the next image frame, and predict and generate the bounding box of each pedestrian target; 2) The bounding box of each pedestrian target in the current image frame is matched with the bounding box of the known pedestrian target in the previous image frame by using the Hungarian algorithm. If the match is successful, the unique annotation ID and movement trajectory of the corresponding pedestrian target in the previous image frame are obtained; 3) Match the unmatched movement track in the previous image frame with the unmatched bounding box in the current image frame through cascade matching. If the match is successful, associate the corresponding bounding box with its matched movement track; 4) Output the unique annotation ID and updated movement trajectory of all pedestrian targets in the current image frame.

4. The method for identifying pedestrian types based on surveillance video according to claim 1, characterized in that: In step S2, the original features in the training set and the test set include: 1) Movement characteristics: movement speed, movement acceleration and movement direction; 2) Postural characteristics: body posture and limb movements; 3) Trajectory characteristics: movement trajectory, dwell time and wandering behavior.

5. The method for identifying pedestrian types based on surveillance video according to claim 1, characterized in that: In step S2, the processing steps of the maximum relevance minimum redundancy algorithm are as follows: S201: normalizing the feature values ​​of all features; S202: Calculate the mutual information value between each feature and the target variable through the mutual information formula; S203: Calculate mutual information values ​​between features using a mutual information formula; S204: setting a preset number m of key features, and initializing an empty set as a key feature set; S205: Select the feature with the largest mutual information with the target variable from all features and add it to the key feature set; S206: for each remaining feature, calculate the average mutual information increment between the key feature set and the target variable after adding the feature to the key feature set, and calculate the average mutual information between the feature and the selected features in the key feature set; S207: According to the principle of maximum relevance-minimum redundancy, select the feature that maximizes the average mutual information increment-average redundancy and add it to the key feature set; S208: Repeat steps S206 to S207 until the number of features in the key feature set reaches a preset number m.

6. The method for identifying pedestrian types based on surveillance video according to claim 1, characterized in that: In step S3, the first layer of the Stacking ensemble model includes a random forest model, an XGBoost model, a gradient boosting machine model, and a decision tree model, and the second layer includes a support vector machine model.

7. The method for identifying pedestrian types based on surveillance video according to claim 6, characterized in that: In step S3, the hyperparameters of each model of the Stacking ensemble model are tuned by the particle swarm optimization algorithm. The specific steps are as follows: S301: Setting relevant parameters of the particle swarm optimization algorithm; setting the search range of the hyperparameters of the target model as the restriction conditions for the position and speed of the particles; S302: Randomly initialize the speed, position, individual optimal solution and global optimal solution of all particles within the set search range; S303: Calculating the fitness function value of each particle in each evolution process through the fitness function; S304: Update the speed and position of the particle; S305: Compare the fitness values ​​of each particle and update the individual optimal solution and the global optimal solution of each particle; S306: Determine whether the set algorithm termination condition is met: if so, take the global optimal solution, i.e., the optimal hyperparameters of the target model, as the output result; otherwise, return to step S303.

8. The method for identifying pedestrian types based on surveillance video according to claim 6, characterized in that: In step S4, the training set is input into the random forest model, XGBoost model, gradient boosting machine model and decision tree model for training according to five-fold cross validation, and the prediction results of each model are output; the prediction results of each model and the corresponding true values ​​are integrated into new training sets and new test sets, which are input into the support vector machine model, and the corresponding predicted pedestrian type is output.

9. The method for identifying pedestrian types based on surveillance video according to claim 8, characterized in that: Step S4 specifically includes the following steps: S401: Divide the training set into five folds; select four folds as the training set in each training, and the remaining fold as the validation set; S402: combining different training sets according to the division results in step S401 to train the target model; predicting the validation set of the corresponding group through the target model to obtain a one-fold prediction sequence to verify the prediction result; Use the target model to predict the test set and obtain the prediction results of the prediction sequence test; repeat this cycle until all combinations are traversed, and finally obtain five verification prediction results and five test prediction results; S403: vertically merging the five validation prediction results in sequence into a new training feature of the prediction sequence, and averaging the five test prediction results to obtain a new test feature of the prediction sequence based on the test set; S404: Repeat the operations from S402 to S403 for each model, and finally obtain four new training features and four new test features; S405: horizontally merging the four new training features and the true values ​​of the endurance time corresponding to the training set in sequence to obtain a new training set; S406: horizontally merging the four new test sets and the true values ​​of the endurance time corresponding to the test sets in sequence to obtain a new test set; S407: Use the support vector machine model to perform training based on the new training set and the new test set and output the predicted pedestrian type.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the method for pedestrian type recognition based on surveillance video as described in any one of claims 1 to 9 is implemented.

Images