A personnel trajectory tracking method and system based on video analysis

Abstract

The present application relates to the technical field of video analysis, and provides a personnel trajectory tracking method and system based on video analysis, which comprises: the system comprises a personnel comparison module, video is frame by frame decomposed from a camera in a region to be tracked, a monitoring image sequence is loaded to obtain a to-be-tracked person feature; a feature extraction node of the personnel comparison module is activated, a plurality of monitoring image sequences are subjected to person feature extraction, a plurality of groups of to-be-compared person features are generated; similarity analysis is performed to generate a plurality of groups of person similarities; a first image set with a plurality of maximum similarities and satisfying a similarity threshold is traversed and extracted, and is adjusted according to a time sequence to generate a to-be-tracked person trajectory image; and personnel tracking is performed according to the to-be-tracked person trajectory image. The technical problem that joint tracking between multiple cameras is difficult to realize in the tracking process and a personnel trajectory is not clear in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of video analytics, specifically to a method and system for tracking people's trajectories based on video analytics. Background Technology

[0002] In order to safeguard social order and protect lives and public property, the analysis and tracking of people's trajectories are required in various scenarios such as public place management, enterprise production safety, and urban traffic order maintenance.

[0003] Currently, the main method for tracking people's trajectories is to analyze video data using target tracking algorithms. However, due to the large number of datasets in the tracking field and the diverse and complex shooting scenarios, including factors such as target deformation, blurring, rotation, occlusion, exceeding the field of view, and irregular movement, video analysis is quite difficult. Furthermore, the integration capabilities of multiple video surveillance terminals are weak, making it difficult to achieve joint tracking between multiple cameras. This results in unclear trajectories, high tracking difficulty, and long tracking time.

[0004] In summary, existing personnel trajectory tracking technologies suffer from technical problems such as difficulty in achieving joint tracking between multiple cameras and unclear personnel trajectories due to the high difficulty in analyzing and processing video information. Summary of the Invention

[0005] This application provides a video analysis-based method and system for tracking people's trajectories, aiming to solve the technical problems in the prior art where it is difficult to achieve joint tracking between multiple cameras and the trajectories of people are unclear.

[0006] In view of the above problems, this application provides a method and system for tracking personnel trajectories based on video analysis.

[0007] The first aspect of this application discloses a method for tracking people's trajectories based on video analysis, applied to a video analysis-based people's trajectory tracking system. The system includes a people comparison module, comprising: frame-by-frame decomposition of video from multiple cameras in a region to be tracked, loading multiple monitoring image sequences; acquiring features of the person to be tracked, wherein the features include clothing features, facial features, and body shape features; activating feature extraction nodes in the people comparison module based on the clothing features, facial features, and body shape features; extracting person features from the multiple monitoring image sequences to generate multiple sets of person features to be compared; performing similarity analysis on the multiple sets of person features to be compared and the features of the person to be tracked, generating multiple sets of person similarities; traversing the multiple sets of person similarities, extracting multiple first image sets with the highest similarity and satisfying a similarity threshold; adjusting the first image set according to time sequence to generate a trajectory image of the person to be tracked; and tracking people based on the trajectory image of the person to be tracked.

[0008] Another aspect of this application discloses a video analysis-based personnel trajectory tracking system, comprising: an information decomposition and loading module for decomposing video from multiple cameras in a region to be tracked frame by frame and loading multiple monitoring image sequences; a tracking feature extraction module for acquiring features of the person to be tracked, wherein the features of the person to be tracked include clothing features, facial features, and body shape features; a monitoring feature extraction module for activating feature extraction nodes of a personnel comparison module based on the clothing features, facial features, and body shape features, extracting personnel features from the multiple monitoring image sequences, and generating multiple sets of personnel features to be compared; a personnel similarity analysis module for traversing the multiple sets of personnel features to be compared and the features of the person to be tracked to perform similarity analysis and generate multiple sets of personnel similarity; a similarity judgment module for traversing the multiple sets of personnel similarity and extracting a first image set with multiple maximum similarities that satisfy a similarity threshold; a trajectory adjustment and generation module for adjusting the first image set according to time sequence to generate a trajectory image of the person to be tracked; and a task execution module for tracking personnel based on the trajectory image of the person to be tracked.

[0009] A third aspect of this application discloses an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the methods described above.

[0010] The fourth aspect disclosed in this application also provides a computer-readable storage medium that stores computer instructions for causing a computer to perform the method described in any of the above steps.

[0011] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0012] This technical solution employs a method of frame-by-frame video decomposition from cameras in the area to be tracked, loading monitoring image sequences to obtain the features of the person to be tracked. The feature extraction node of the personnel comparison module is activated, and multiple sets of personnel features are extracted from multiple monitoring image sequences, resulting in multiple sets of features to be compared. Similarity analysis is performed on these multiple sets of features and the features of the person to be tracked, yielding multiple sets of similarity scores. Multiple image sets with the highest similarity scores and meeting the similarity threshold are traversed and extracted, and these image sets are adjusted according to time sequence to obtain the trajectory image of the person to be tracked. This personnel tracking solution, based on the trajectory image of the person to be tracked, achieves multi-dimensional analysis and multi-lens joint tracking by jointly analyzing data collected from multiple cameras and extracting and comparing clothing, facial, and body features, thus improving the accuracy and consistency of the tracking.

[0013] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0014] Figure 1 This application provides a possible flowchart of a video analytics-based personnel trajectory tracking method as an embodiment of the present application.

[0015] Figure 2 This application provides a flowchart illustrating the extraction of human features from multiple monitoring image sequences in a video analysis-based human trajectory tracking method.

[0016] Figure 3 This application provides a principle block diagram of a video analytics-based personnel trajectory tracking system as an embodiment;

[0017] Figure 4 This is a schematic block diagram of an electronic device provided in an embodiment of this application.

[0018] Figure labeling: Information decomposition and loading module 100, tracking feature extraction module 200, monitoring feature extraction module 300, person similarity analysis module 400, similarity judgment module 500, trajectory adjustment and generation module 600, task execution module 700. Detailed Implementation

[0019] The overall concept of the technical solution provided in this application is as follows:

[0020] This application provides a method and system for tracking people's trajectories based on video analysis. It employs a method of frame-by-frame decomposition of video from cameras in the area to be tracked, loading monitoring image sequences to obtain the features of the person to be tracked. The feature extraction node of the people comparison module is activated, and human features are extracted from multiple monitoring image sequences to obtain multiple sets of human features to be compared. Similarity analysis is performed on the multiple sets of human features to be compared and the features of the person to be tracked to obtain multiple sets of human similarity. Multiple sets of images with the highest similarity and meeting the similarity threshold are traversed and extracted, and the image sets are adjusted according to time sequence to obtain the trajectory image of the person to be tracked. This technical solution for tracking people based on the trajectory image of the person to be tracked achieves multi-dimensional analysis and multi-lens joint tracking by jointly analyzing data collected from multiple cameras and extracting and comparing human clothing, facial, and body features, thereby improving the accuracy and consistency of tracking.

[0021] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0022] Example 1

[0023] like Figure 1 As shown, this application provides a video analysis-based personnel trajectory tracking method, applied to a video analysis-based personnel trajectory tracking system. The system includes a personnel comparison module, comprising the following steps:

[0024] S10: Decompose the video from multiple cameras in the area to be tracked frame by frame and load multiple monitoring image sequences;

[0025] Specifically, with the development of positioning and communication technologies, personnel trajectory tracking systems have been widely applied in fields such as safety supervision and smart petrochemicals. These systems can analyze and compare collected video information frame by frame to obtain the movement trajectory of the target object, enabling real-time tracking. The system includes a personnel comparison module, which provides feature extraction and feature comparison functions. It analyzes and processes the input video information, extracts the required feature information, and compares it with the target feature information. This module includes, but is not limited to, a feature extraction model trained on a downsampling convolutional neural network model and a similarity analysis and evaluation model based on multiple preset similarity functions.

[0026] Video information is collected from multiple cameras in the area to be tracked, serving as the foundational data for analysis. Using an image processing library, such as OpenCV, the collected information is analyzed frame-by-frame according to different cameras, yielding an image decomposition sequence for each camera. The resulting image set comprises multiple monitoring image sequences. This achieves joint analysis and processing of data collected from multiple cameras, enabling joint tracking and improving the accuracy and consistency of the tracking process.

[0027] S20: Obtain the features of the person to be tracked, wherein the features of the person to be tracked include clothing features, facial features and body shape features;

[0028] Specifically, the features of the person to be tracked are preset features of the person to be searched and tracked. The image or video information of the target person is extracted by the convolutional layer of the personnel comparison module, including the person's clothing features, facial features, and body shape features. The clothing features include the color, shape, and distribution of clothing; the facial features include the position and shape of the eyes, ears, mouth, nose, and eyebrows, face shape, and hairstyle; the body shape features include height, weight, and body proportions.

[0029] S30: Activate the feature extraction node of the personnel comparison module according to the clothing features, facial features and body shape features of the person, extract the person features from the multiple monitoring image sequences, and generate multiple sets of person features to be compared.

[0030] Furthermore, such as Figure 2 As shown, step S30 involves activating the feature extraction node of the personnel comparison module based on the clothing features, facial features, and body shape features of the person, extracting person features from the multiple monitoring image sequences, and generating multiple sets of person features to be compared. The steps include:

[0031] S31: The feature extraction node includes at least a clothing feature extraction node, a facial feature extraction node, and a body shape feature extraction node;

[0032] S32: Activate the clothing feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of clothing features of people to be compared;

[0033] S33: Activate the facial feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of facial features of people to be compared.

[0034] S34: Activate the body shape feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of human body shape features to be compared;

[0035] S35: Add the multiple sets of clothing features, facial features, and body shape features of the persons to be compared to the multiple sets of persons to be compared to the multiple sets of persons to be compared to the multiple sets of persons to be compared to the multiple sets of persons to be compared features.

[0036] Furthermore, the clothing feature extraction node, the facial feature extraction node, and the body shape feature extraction node are obtained by training a downsampling convolutional neural network model.

[0037] Specifically, the feature extraction node includes several nodes related to the person's appearance, capable of extracting features from the person's clothing, facial features, and body shape. It includes at least a clothing feature extraction node, a facial feature extraction node, and a body shape feature extraction node.

[0038] The clothing feature extraction node, the facial feature extraction node, and the body shape feature extraction node are convolutional neural network models with feature extraction capabilities obtained by training a downsampling convolutional neural network model. The preferred model training process is as follows:

[0039] Based on big data analysis of the area to be tracked, multiple sets of surveillance video footage were collected to collect pedestrian flow data, including pedestrian clothing data, pedestrian facial data, and pedestrian body shape data. The pedestrian clothing data, pedestrian facial data, and pedestrian body shape data were divided into an 8:2 ratio, with 80% of the data used as training data and 20% used as validation data.

[0040] During training, arbitrary sets of pedestrian clothing data, pedestrian facial data, and pedestrian body shape data are used as training input data and target output information, respectively, and input into the downsampling convolutional neural network model. After forward propagation through convolutional layers, downsampling layers, and fully connected layers, the output result is obtained, which is the predicted feature extraction result, i.e., the output feature vector. The deviation between the output feature vector and the target output information is calculated. When the deviation is less than the preset deviation, the verification counter is incremented by 1; when the deviation is greater than or equal to the preset deviation, the verification counter is set to 0. The training continues by selecting transportation weather record data, ideal transportation time record data, and actual transportation time record data from the training data. When the number of verification counters is greater than or equal to the preset number, verification is performed using the verification data. The verification method is exactly the same as the training method. When the count of the verification counter is greater than or equal to the preset number, the convolutional neural network model is considered to have converged; otherwise, the training data is returned to continue training.

[0041] When activating feature extraction nodes, activate clothing feature extraction nodes, facial feature extraction nodes, and body shape feature extraction nodes separately.

[0042] After activating the clothing feature extraction node, the multiple monitoring image sequences are traversed. The multiple monitoring image sequences are used as input information, and features are extracted through the convolutional layer, downsampling layer, and fully connected layer in the clothing feature extraction node to generate multiple sets of clothing features of the people to be compared.

[0043] After activating the facial feature extraction node, the multiple monitoring image sequences are traversed. The multiple monitoring image sequences are used as input information, and features are extracted through the convolutional layer, downsampling layer, and fully connected layer in the clothing feature extraction node to generate multiple sets of facial features of the person to be compared.

[0044] After activating the body shape feature extraction node, the multiple monitoring image sequences are traversed. The multiple monitoring image sequences are used as input information, and features are extracted through the convolutional layer, downsampling layer, and fully connected layer in the clothing feature extraction node to generate multiple sets of body shape features of the people to be compared.

[0045] The obtained multiple sets of features of the person to be compared include the multiple sets of clothing features, the multiple sets of facial features, and the multiple sets of body shape features.

[0046] S40: Traverse the multiple sets of features of the person to be compared and the features of the person to be tracked, perform similarity analysis, and generate multiple sets of person similarity;

[0047] Specifically, the multiple sets of person similarity are obtained by performing distance deviation analysis on the feature vectors of the person to be tracked and the feature vectors of the multiple sets of compared persons through a similarity analysis function, resulting in multiple similarity analysis results. The multiple sets of compared persons include clothing features, facial features, and body shape features. The features of the person to be tracked include clothing features, facial features, and body shape features. This achieves similarity analysis across multiple aspects such as clothing, face, and body shape. Through static and dynamic feature analysis, it enables multi-dimensional and comprehensive analysis and matching of the person to be tracked, thereby providing comprehensive data for tracking and improving the accuracy of trajectory tracking.

[0048] S50: Traverse the multiple sets of character similarity and extract the first set of images with the maximum similarity and satisfying the similarity threshold;

[0049] Specifically, the similarity threshold is a similarity range set based on historical tracking data, which includes, but is not limited to, the mapping relationship between historical similarity data and tracking trajectory overlap rate. The multiple sets of character similarities are iterated through, sorted from high to low similarity, and image information of several characters whose features meet the similarity threshold and rank highly in the similarity ranking are extracted to form the first image set.

[0050] S60: Adjust the first image set according to the time sequence to generate an image of the trajectory of the person to be tracked;

[0051] S70: Track personnel based on the trajectory image of the person to be tracked.

[0052] Specifically, since the first image set contains a certain amount of image information and may include information about several individuals with high similarity to the person to be tracked, after adjusting the images according to their capture time sequence, a set of images of several individuals is obtained, constituting the trajectory image of the person to be tracked. This trajectory image is transmitted to the display interface of the personnel trajectory tracking system, where a human judgment is made based on the image, thereby enabling personnel tracking and positioning. This achieves the decomposition, comparison, and extraction of the target person's features, and the feature information with the highest similarity to the extracted feature information is adjusted in time sequence to obtain a more accurate and smoother trajectory image.

[0053] Furthermore, the step of traversing the multiple sets of features of the persons to be compared and the features of the persons to be tracked to perform similarity analysis and generate multiple sets of person similarity scores includes the following steps:

[0054] S41: Construct a clothing similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of clothing similarity scores;

[0055] S42: Construct a facial similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of facial similarity scores;

[0056] S43: Construct a body shape similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of body shape similarities;

[0057] S44: Store the multiple sets of clothing similarity, facial similarity, and body shape similarity of the multiple sets of characters according to the extraction source association to generate the multiple sets of character similarity.

[0058] Furthermore, the step S41, which involves constructing a clothing similarity evaluation function, iterating through the multiple sets of features of the individuals to be compared and the features of the individuals to be tracked to perform similarity analysis, and generating multiple sets of clothing similarity scores, includes:

[0059] The clothing similarity evaluation function is:

[0060]

[0061] Wherein, SIM1 represents clothing similarity, x ix represents the pixel value of the i-th point of the person to be compared. i0 The pixel value of the pixel closest to the pixel value of the i-th point after adjusting the pose of the person to be tracked according to the person to be compared is represented by k, the total number of pixels to be compared is represented by a, and the minimum pixel value deviation is represented by a.

[0062] Specifically, similarity analysis is preferably performed by constructing a similarity evaluation function. The clothing similarity evaluation function is as follows:

[0063]

[0064] Wherein, SIM1 represents clothing similarity, x i x represents the pixel value of the i-th point of the person to be compared. i0 The pixel value of the pixel closest to the pixel value of the i-th point after adjusting the pose of the person to be tracked according to the person to be compared is represented by k, the total number of pixels to be compared is represented by a, and the minimum pixel value deviation is represented by a.

[0065] Based on the above similarity evaluation function, the feature information in the multiple sets of features to be compared with the features of the person to be tracked is analyzed in sequence to obtain the clothing similarity analysis results, that is, the clothing similarity of the multiple sets of people.

[0066] The similarity evaluation function uses k to represent the total number of pixels to be compared. When analyzing the pixel value of the i-th point, the pose of the person to be tracked is adjusted based on the extracted pixel value of the i-th point, and the pixel value of the i0-th point is obtained by finding the pixel value closest to the pixel value of the i-th point. i0 Find x i and x i0 The Euclidean distance between pixels is used as the pixel deviation. When the deviation is less than or equal to the minimum pixel value deviation 'a' of similar pixels, it is counted using the count function. The data result after analyzing and comparing all pixel sites is then divided by the total number of pixels, k, to obtain the clothing similarity. The proportion of pixels whose deviation meets the minimum pixel value deviation to the total number of pixels to be compared is used. If, after adjusting the pose of the person to be tracked, no clothing image information is found at that location, the pixel deviation at that location is considered greater than 'a' and is not counted.

[0067] Furthermore, the step S42, which involves constructing a facial similarity evaluation function, traversing the multiple sets of features of the persons to be compared and the features of the persons to be tracked, and performing similarity analysis to generate multiple sets of facial similarity scores, includes:

[0068] The facial similarity evaluation function is:

[0069]

[0070] Wherein, SIM2 represents facial similarity, d(y j ,y j0 y represents the Euclidean distance between the j-th facial localization point after alignment of the facial image to be compared and the facial image to be tracked. j The coordinates of the j-th facial localization point in the facial image to be compared are represented by y. j0 The coordinates of the j-th facial positioning point in the image of the person to be tracked are represented by q, the total number of facial positioning point coordinates are represented by b, and the minimum positioning distance deviation is considered as facial similarity.

[0071] Specifically, multiple sets of features of the person to be compared and the features of the person to be tracked are input into the facial similarity evaluation function, and the similarity between each set of features of the person to be compared and the features of the person to be tracked is calculated in turn to obtain multiple sets of facial similarities.

[0072] The facial similarity function first sets multiple positioning points on the human face for analysis and comparison, such as the center, start, middle, and end points of the eyebrows, the tip of the nose, the corners and outer corners of the eyes, and the upper, middle, and lower positions of the ears as preset positioning points. During similarity analysis, the facial images of the person to be compared and the person to be tracked are first aligned, and distance analysis of the facial positioning points is performed, using d(y j ,y j0 The Euclidean distance is represented by b, which represents the minimum positioning distance deviation considered as facial similarity. When the Euclidean distance of the positioning points is less than or equal to the minimum positioning distance deviation b, the count function is used to count the points. The data results after analyzing and comparing all positioning points are then divided by the total number of facial positioning point coordinates q. The proportion of the number of positioning point coordinates that meet the minimum positioning distance deviation to the total number of facial positioning point coordinates is obtained, which is the facial similarity between a set of features of the person to be compared and the features of the person to be tracked.

[0073] Furthermore, the construction of the body shape similarity evaluation function iterates through the multiple sets of features of the people to be compared and the features of the people to be tracked to perform similarity analysis, generating multiple sets of body shape similarities; step S43 includes:

[0074] The body shape similarity evaluation function is:

[0075]

[0076] Wherein, SIM3 represents body shape similarity, H represents the total number of preset body shape similarity assessment localization points after posture overlap adjustment between the person to be compared and the person to be tracked, and d(z l ,z l0 z represents the Euclidean distance between the l-th localization point after the poses of the person to be compared and the person to be tracked are aligned. l The z-coordinate represents the coordinates of the l-th location point of the person to be compared. l0The coordinates of the l-th location point of the person to be tracked are represented, and c represents the minimum positioning distance deviation considered as similar body shape.

[0077] Specifically, multiple sets of features of the person to be compared and the features of the person to be tracked are input into the body shape similarity evaluation function, and the similarity between each set of features of the person to be compared and the features of the person to be tracked is calculated in turn to obtain multiple sets of body shape similarities.

[0078] First, positioning points need to be set for the human figure, such as the head, neck, shoulders, elbows, hands, waist, hips, knees, and ankles. H represents the total number of preset positioning points. Since the poses of the people captured in the video vary, the poses need to be corrected and adjusted to ensure they overlap. After overlap, the Euclidean distance of the positioning points is analyzed, using d(z)... l ,z l0 ) characterization, where z l The z-coordinate represents the coordinates of the l-th location point of the person to be compared. l0 This represents the coordinates of the l-th location point of the person to be tracked. When d(z) l ,z l0 When the deviation is less than or equal to the minimum positioning distance deviation c for similar body shape, the count function is used to count. The data result after analyzing and comparing all positioning points is divided by the preset total number of positioning points H for similar body shape evaluation. The proportion of positioning points whose deviation meets the minimum positioning distance deviation to the total number of positioning points is obtained, which is the body shape similarity between a set of features of the person to be compared and the features of the person to be tracked.

[0079] The extraction source refers to the corresponding person to be compared. For example, the extraction source for the clothing similarity, facial similarity, and body shape similarity of Zhang San is all Zhang San. Storing information associated with the extraction source means storing all similarity information of the person to be compared at the same address, generating the similarity score for that person. Further, by storing multiple sets of data sequentially, multiple sets of person similarities are obtained. Through the construction and calculation of multiple similarity functions, accurate similarity analysis is achieved, improving the accuracy of feature matching.

[0080] In summary, the video analysis-based personnel trajectory tracking method and system provided in this application have the following technical effects:

[0081] 1. By jointly analyzing and processing data collected from multiple cameras, and by extracting and comparing the characteristics of clothing, face, and body shape of people, multi-dimensional analysis and multi-lens joint tracking are achieved, thereby improving the accuracy and consistency of tracking.

[0082] 2. By calculating multiple similarity functions, the feature information with the highest similarity and meeting the preset similarity threshold is obtained, and time-series adjustment is performed to achieve accurate analysis of similarity and coherent arrangement of images, resulting in more accurate and smoother trajectory images, thus providing high-quality trajectory information for character tracking.

[0083] Example 2

[0084] Based on the same inventive concept as the video analysis-based personnel trajectory tracking method in the foregoing embodiments, such as Figure 3 As shown in the figure, this application provides a video analytics-based personnel trajectory tracking system, which includes:

[0085] The information decomposition and loading module 100 is used to decompose the video frame by frame from multiple cameras in the area to be tracked and load multiple monitoring image sequences.

[0086] The tracking feature extraction module 200 is used to acquire the features of the person to be tracked, wherein the features of the person to be tracked include clothing features, facial features and body shape features;

[0087] The monitoring feature extraction module 300 is used to activate the feature extraction node of the personnel comparison module based on the clothing features, facial features, and body shape features of the person, extract the person features from the multiple monitoring image sequences, and generate multiple sets of person features to be compared.

[0088] The character similarity analysis module 400 is used to traverse the multiple sets of character features to be compared and the character features to be tracked to perform similarity analysis and generate multiple sets of character similarity.

[0089] The similarity judgment module 500 is used to traverse the multiple sets of character similarities, extract multiple sets of images with the maximum similarity and satisfy the similarity threshold.

[0090] The trajectory adjustment and generation module 600 is used to adjust the first image set according to the time sequence to generate a trajectory image of the person to be tracked.

[0091] The task execution module 700 is used to track people based on the trajectory image of the person to be tracked.

[0092] Furthermore, the monitoring feature extraction module 300 performs the following steps:

[0093] The feature extraction nodes include at least clothing feature extraction nodes, facial feature extraction nodes, and body shape feature extraction nodes;

[0094] Activate the clothing feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of clothing features of people to be compared;

[0095] Activate the facial feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of facial features of people to be compared.

[0096] Activate the body shape feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of human body shape features to be compared;

[0097] The multiple sets of clothing features, facial features, and body shape features of the individuals to be compared are added to the multiple sets of features of the individuals to be compared.

[0098] Furthermore, the monitoring feature extraction module 300 performs the following steps:

[0099] The clothing feature extraction node, the facial feature extraction node, and the body shape feature extraction node are obtained by training a downsampling convolutional neural network model.

[0100] Furthermore, the steps performed by the character similarity analysis module 400 include:

[0101] Construct a clothing similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of clothing similarity scores;

[0102] Construct a facial similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of facial similarity scores.

[0103] Construct a body shape similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked to perform similarity analysis, and generate multiple sets of body shape similarities;

[0104] The similarity scores of clothing, faces, and body shapes of the multiple sets of people are stored in association according to the extraction source to generate the multiple sets of people similarity scores.

[0105] Furthermore, the steps performed by the character similarity analysis module 400 include:

[0106] The clothing similarity evaluation function is:

[0107]

[0108] Wherein, SIM1 represents clothing similarity, x i x represents the pixel value of the i-th point of the person to be compared. i0The pixel value of the pixel closest to the pixel value of the i-th point after adjusting the pose of the person to be tracked according to the person to be compared is represented by k, the total number of pixels to be compared is represented by a, and the minimum pixel value deviation is represented by a.

[0109] Furthermore, the steps performed by the character similarity analysis module 400 include:

[0110] The facial similarity evaluation function is:

[0111]

[0112] Wherein, SIM2 represents facial similarity, d(y j ,y j0 y represents the Euclidean distance between the j-th facial localization point after alignment of the facial image to be compared and the facial image to be tracked. j The coordinates of the j-th facial localization point in the facial image to be compared are represented by y. j0 The coordinates of the j-th facial positioning point in the image of the person to be tracked are represented by q, the total number of facial positioning point coordinates are represented by b, and the minimum positioning distance deviation is considered as facial similarity.

[0113] Furthermore, the steps performed by the character similarity analysis module 400 include:

[0114] The body shape similarity evaluation function is:

[0115]

[0116] Wherein, SIM3 represents body shape similarity, H represents the total number of preset body shape similarity assessment localization points after posture overlap adjustment between the person to be compared and the person to be tracked, and d(z l ,z l0 z represents the Euclidean distance between the l-th localization point after the poses of the person to be compared and the person to be tracked are aligned. l The z-coordinate represents the coordinates of the l-th location point of the person to be compared. l0 The coordinates of the l-th location point of the person to be tracked are represented, and c represents the minimum positioning distance deviation considered as similar body shape.

[0117] Example 3

[0118] An electronic device according to an embodiment of this application includes a memory and a processor. The memory is used to store non-transitory computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0119] The processor may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of this application, the processor is used to execute computer-readable instructions stored in the memory, causing the electronic device to perform all or part of the steps of a video analysis-based personnel trajectory tracking method according to the foregoing embodiments of this application.

[0120] Those skilled in the art should understand that, in order to solve the technical problem of how to achieve a good user experience, this embodiment may also include well-known structures such as communication buses and interfaces, and these well-known structures should also be included within the protection scope of this application.

[0121] like Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of this application. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0122] like Figure 4 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0123] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 4Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0124] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, all or part of the steps of a video analytics-based people trajectory tracking method are performed.

[0125] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0126] A computer-readable storage medium according to an embodiment of this application stores non-transitory computer-readable instructions. When these non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the aforementioned video analysis-based personnel trajectory tracking method are performed.

[0127] The aforementioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or portable hard drive), media with built-in rewritable non-volatile memory (e.g., memory card), and media with built-in ROM (e.g., ROM cartridge).

[0128] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0129] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0130] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0131] Additionally, as used herein, the “or” used in a list of items beginning with “at least one” indicates a separate list, such that a list of, for example, “at least one of A, B, or C” means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word “exemplary” does not imply that the described example is preferred or better than other examples.

[0132] It should also be noted that in the system and method of this application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of this application.

[0133] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0134] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0135] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for tracking personnel trajectories based on video analytics, characterized in that, An application to a video analytics-based personnel trajectory tracking system, the system including a personnel comparison module, comprising: Video from multiple cameras in the area to be tracked is decomposed frame by frame, and multiple monitoring image sequences are loaded. The features of the person to be tracked are obtained, including clothing features, facial features, and body shape features. Based on the clothing features, facial features, and body shape features of the person, the feature extraction node of the personnel comparison module is activated, and the person features are extracted from the multiple monitoring image sequences to generate multiple sets of person features to be compared. The similarity analysis is performed on the multiple sets of features of the person to be compared and the features of the person to be tracked, and multiple sets of similarity scores are generated. The system iterates through the multiple sets of character similarities and extracts a first set of images with the highest similarity that also meet a similarity threshold. The similarity threshold is a similarity range set based on historical tracking data, and the historical tracking data includes a mapping relationship between historical similarity data and the overlap rate of tracking trajectories. The first image set is adjusted according to time sequence to generate an image of the trajectory of the person to be tracked. Personnel tracking is performed based on the trajectory image of the person to be tracked; This involves iterating through the multiple sets of features of the individuals to be compared and the features of the individuals to be tracked, performing similarity analysis, and generating multiple sets of individual similarity scores, including: Construct a clothing similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of clothing similarity scores; Construct a facial similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of facial similarity scores. Construct a body shape similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked to perform similarity analysis, and generate multiple sets of body shape similarities; The multiple sets of clothing similarity, facial similarity, and body shape similarity of the characters are stored in association according to the extraction source to generate the multiple sets of character similarity. The clothing similarity evaluation function is: in, Characterizing clothing similarity The pixel value representing the i-th point of the person to be compared. The pixel value is the closest pixel value to the i-th pixel value after adjusting the pose of the person being tracked based on the person being compared. Represents the total number of pixels to be compared. It represents the smallest pixel value deviation considered as similar pixels.

2. The method as described in claim 1, characterized in that, Based on the clothing features, facial features, and body shape features of the individuals, the feature extraction node of the personnel comparison module is activated. Person features are extracted from the multiple monitored image sequences to generate multiple sets of person features to be compared, including: The feature extraction nodes include at least clothing feature extraction nodes, facial feature extraction nodes, and body shape feature extraction nodes; Activate the clothing feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of clothing features of people to be compared; Activate the facial feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of facial features of people to be compared. Activate the body shape feature extraction node, traverse the multiple monitoring image sequences to extract features, and generate multiple sets of human body shape features to be compared; The multiple sets of clothing features, facial features, and body shape features of the individuals to be compared are added to the multiple sets of features of the individuals to be compared.

3. The method as described in claim 2, characterized in that, The clothing feature extraction node, the facial feature extraction node, and the body shape feature extraction node are obtained by training a downsampling convolutional neural network model.

4. The method as described in claim 1, characterized in that, The facial similarity evaluation function is: in, Characterizing facial similarity, The Euclidean distance between the j-th facial localization point after alignment of the facial image to be compared and the facial image to be tracked is represented. The coordinates of the j-th facial localization point in the facial image of the person to be compared are represented. The coordinates of the j-th facial localization point in the image of the person to be tracked. 'b' represents the total number of facial positioning point coordinates, and 'b' represents the minimum positioning distance deviation considered as facial similarity.

5. The method as described in claim 1, characterized in that, The body shape similarity evaluation function is: in, Characterizing body shape similarity, The total number of preset body shape similarity assessment points representing the person to be compared and the person to be tracked after posture overlap adjustment. The Euclidean distance between the l-th localization point after the poses of the person to be compared and the person to be tracked are aligned is represented. The coordinates of the l-th location point of the person to be compared are represented. The coordinates of the l-th location point of the person to be tracked are represented, and c represents the minimum positioning distance deviation considered as similar body shape.

6. A personnel trajectory tracking system based on video analysis, characterized in that, The system includes a personnel comparison module, comprising: The information decomposition and loading module is used to decompose the video from multiple cameras in the area to be tracked frame by frame and load multiple monitoring image sequences. The tracking feature extraction module is used to obtain the features of the person to be tracked, wherein the features of the person to be tracked include clothing features, facial features and body shape features; The monitoring feature extraction module is used to activate the feature extraction node of the personnel comparison module based on the clothing features, facial features, and body shape features of the person, extract the person features from the multiple monitoring image sequences, and generate multiple sets of person features to be compared. The character similarity analysis module is used to traverse the multiple sets of character features to be compared and the character features to be tracked to perform similarity analysis and generate multiple sets of character similarity. The similarity judgment module is used to traverse the multiple sets of character similarities, extract a first set of images with multiple maximum similarities and satisfying the similarity threshold, wherein the similarity threshold is a similarity range set based on historical tracking data, and the historical tracking data includes the mapping relationship between historical similarity data and the overlap rate of tracking trajectories; The trajectory adjustment and generation module is used to adjust the first image set according to the time sequence to generate a trajectory image of the person to be tracked. The task execution module is used to track people based on the trajectory image of the person to be tracked; This involves iterating through the multiple sets of features of the individuals to be compared and the features of the individuals to be tracked, performing similarity analysis, and generating multiple sets of individual similarity scores, including: Construct a clothing similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of clothing similarity scores; Construct a facial similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked, perform similarity analysis, and generate multiple sets of facial similarity scores. Construct a body shape similarity evaluation function, traverse the multiple sets of features of the people to be compared and the features of the people to be tracked to perform similarity analysis, and generate multiple sets of body shape similarities; The multiple sets of clothing similarity, facial similarity, and body shape similarity of the characters are stored in association according to the extraction source to generate the multiple sets of character similarity. The clothing similarity evaluation function is: in, Characterizing clothing similarity The pixel value representing the i-th point of the person to be compared. The pixel value is the closest pixel value to the i-th pixel value after adjusting the pose of the person being tracked based on the person being compared. Represents the total number of pixels to be compared. It represents the smallest pixel value deviation considered as similar pixels.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a video analysis-based personnel trajectory tracking method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform a video analysis-based personnel trajectory tracking method as described in any one of claims 1-5.

Images