A lightweight pedestrian dynamic feature extraction and recognition method and system based on video motion vector

Abstract

The application belongs to the field of artificial intelligence, and particularly relates to a lightweight pedestrian dynamic feature extraction and recognition method and system based on video motion vectors. By constructing a motion decoupling and differential coding framework, the efficient pedestrian target capture and real-time low-delay recognition problem in complex motion scenarios is effectively solved. Specifically, the DiVE framework utilizes the spatiotemporal characteristics of motion vectors to accurately extract pedestrian dynamic features in dynamic environments, and reduces the computational and transmission bandwidth requirements through differential coding technology, thereby optimizing the processing performance of edge devices. This method not only improves the accuracy of pedestrian recognition, but also significantly reduces the power consumption and delay of the system, and has a wide application prospect, especially in intelligent video surveillance, autonomous driving and smart city fields.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of artificial intelligence technology, and particularly relates to a lightweight method and system for extracting and recognizing pedestrian dynamic features based on video motion vectors. Background Technology

[0002] With the deep integration of artificial intelligence (AI) and the Internet of Things (IoT) technologies, mobile intelligent agents, represented by mobile patrol robots, intelligent drones, and portable law enforcement recorders, have been widely applied in fields such as safe cities, power line inspections, and border security. In these application scenarios, person re-identification (Re-ID), as a core visual task, requires the system to be able to identify and continuously track specific targets across cameras and time periods. Current person re-identification technologies primarily rely on deep artificial neural networks (ANNs). By training convolutional neural networks (CNNs) or visual interpolators (ViTs) on massive amounts of labeled data, the model can extract highly identifiable feature vectors (Embeddings) of pedestrians. However, this high performance is usually built upon a foundation of enormous computational resource consumption.

[0003] Despite the significant progress made by deep learning in static image recognition, three major obstacles remain when pedestrian re-identification tasks are migrated to resource-constrained mobile edge computing:

[0004] (1) Extreme bottlenecks in computational overhead and energy consumption. Existing pedestrian analysis algorithms typically employ a "frame-by-frame detection + feature extraction" model. Each high-definition image requires floating-point multiplication and accumulation operations through several or even hundreds of layers of neural networks. For mobile intelligent agents, their embedded computing platforms (such as low-power SoCs) are unable to support high-frequency inference requirements, resulting in low system frame rates and high power consumption, which greatly shortens the device's battery life. This model violates the stringent requirements of mobile devices for real-time response and long-term operation.

[0005] (2) The contradiction between transmission bandwidth fluctuations and real-time response. Due to insufficient computing power at the edge, the current common solution is to upload the video stream to the edge server or cloud for recognition in real time. However, in actual deployment, the bandwidth of 4G / 5G wireless communication links exhibits great instability. When the device is in high-speed movement (such as drone flight) or encounters base station switching or multipath fading, the severe bandwidth fluctuations will cause a significant increase in the transmission delay of high-definition video streams. In time-sensitive scenarios such as security early warning, a delay of a few seconds may lead to target loss or security accidents, which seriously restricts the engineering practical value of the system.

[0006] (3) Environmental noise interference from a moving perspective. Unlike fixed surveillance cameras, the shooting perspective of mobile intelligent agents is constantly changing dynamically. While the camera moves with the carrier, it also undergoes frequent pitch and yaw rotations. This complex ego-motion causes a sharp background shift in the video sequence. Ideally, the motion vector (MV) generated during video compression can be used as a "free" motion feature to assist in foreground extraction. However, the original motion vector is extremely coarse and highly susceptible to noise. From a moving perspective, the motion vectors of background objects often obscure the subtle displacement of pedestrian targets, resulting in a significant decrease in recognition accuracy when directly using MV for body feature modeling.

[0007] To alleviate these problems, academia and industry have proposed several optimization schemes, but all have significant limitations. First, there's frame filtering technology, which identifies and sends key frames using simple logic, while non-key frames are completed through tracking. However, in highly dynamic scenarios, the tracking algorithm is prone to drift, leading to unstable recognition rates. Second, there's server feedback compression, which dynamically adjusts the bitrate of the upload region based on the recognition results from edge servers. However, the feedback mechanism itself introduces round-trip latency, making it unsuitable for ultra-real-time online tasks. Finally, there's lightweight model compression, which reduces network size through pruning and quantization. While this reduces computational cost, in complex environments, the robustness and accuracy of lightweight models often fail to meet the unique feature extraction requirements of pedestrian re-identification.

[0008] In summary, there is an urgent need to establish a unified spatiotemporal modeling framework that can simultaneously achieve the following objectives: First, lightweight feature extraction by fully utilizing existing motion information in the video compression process and avoiding redundant deep feature extraction. Second, accurate suppression of motion noise by establishing a physical-mathematical model to offset interference caused by camera motion online, extracting pure gait and body features of pedestrians from a coarse vector field. Third, fusion of bandwidth awareness and differential coding, enabling the system to dynamically optimize the compression weights of pedestrian foreground and background based on real-time bandwidth fluctuations, ensuring high pedestrian recognition accuracy even at extremely low bit rates. Finally, by addressing the aforementioned obstacles such as inter-layer frequency mismatch and temporal redundancy, this invention aims to provide a system-level solution that maintains low power consumption while significantly improving performance in complex visual tasks.

[0009] Based on the above analysis, the existing technologies urgently need to address the following technical problems: When existing deep neural network-based pedestrian re-identification technologies are applied to resource-constrained mobile intelligent agents, they suffer from computational overhead and energy consumption bottlenecks, transmission bandwidth fluctuations affecting real-time response, and environmental noise from the moving perspective interfering with recognition accuracy. Furthermore, existing optimization schemes all have significant limitations. There is an urgent need to construct a unified spatiotemporal modeling framework to solve the above multiple technical challenges. Summary of the Invention

[0010] To address the problems of existing technologies, this invention provides a lightweight pedestrian dynamic feature extraction and recognition method and system based on video motion vectors. By constructing a plug-and-play motion decoupling and differential coding framework, it effectively solves the problems of efficient pedestrian target capture and real-time low-latency recognition in complex motion scenarios. Specifically, the DiVE framework utilizes the spatiotemporal characteristics of motion vectors to accurately extract pedestrian dynamic features in dynamic environments, and reduces the computational and transmission bandwidth requirements through differential coding technology, thereby optimizing the processing performance of edge devices. This method not only improves the accuracy of pedestrian recognition but also significantly reduces system power consumption and latency, showing broad application prospects, especially suitable for fields such as intelligent video surveillance, autonomous driving, and smart cities.

[0011] This invention is implemented as follows: a lightweight method for extracting and recognizing pedestrian dynamic features based on video motion vectors, comprising:

[0012] S1, Motion vector calculation and state determination;

[0013] S2, R-sampling radial sampling and decoupling from overdetermined linear equations;

[0014] S3, Ground plane constraint and background filtering based on the Triangle statistical method;

[0015] S4, Dynamic Region Growing Clustering and Sklansky Convex Hull Feature Extraction.

[0016] Furthermore, S1 specifically includes:

[0017] First, the Block Matching Algorithm (BMA) in the video codec is used to calculate the macroblock-level motion vector set. The BMA algorithm performs macroblock matching based on the similarity between consecutive frames and extracts the motion information for each macroblock. Motion vectors It contains motion state information of each macroblock relative to the current frame. Through block matching, macroblock-level motion vectors containing translation, rotation, and other motion states are obtained, providing a foundation for subsequent target tracking and feature extraction.

[0018] Furthermore, the system directly extracts motion vectors during the entropy decoding and motion compensation stages of video decoding, avoiding pixel-domain reconstruction calculations. Let the size of the input t-th predicted frame (P-frame) be... The size of the coded macroblock is set to pixels. (generally =16) The system extracts the macroblock feature matrix, for the first... The system extracts the macroblock feature matrix, for the th Each macroblock is used to obtain its motion vector. To avoid activating the computationally expensive decoupling module when the device is completely stationary, the system first calculates the global motion activity. Define indicator functions When the macroblock motion vector's L2 norm The minimum noise filtering threshold set internally by the system, for example, at 0.5 pixels. Otherwise, it is 0. Calculate the proportion of non-zero motion vectors across the entire image:

[0019]

[0020] when When the system determines that the current mobile agent is in a motion state involving translation or rotation of its own camera, it then inputs the motion vector field to the R-sampling module for decoupling.

[0021] Furthermore, S2 specifically includes:

[0022] When the camera undergoes complex motion, any pixel The motion vector is formed by coupling three-dimensional translational and three-dimensional rotational components. To isolate features affected only by the target displacement, the camera's pitch angle around the X-axis must be eliminated. and yaw angle about the Y-axis First, during the FOE calibration and radial distance calculation phase, the system records the pixel coordinates of the center of expansion (FOE) during the calibration stage of pure forward translation of the device. ,in Let be the camera focal length. In real-time decoupling, calculate the Euclidean distance from all non-zero macroblocks in the entire image to the FOE:

[0023]

[0024] Secondly, the system Sort in ascending order and select the first one closest to FOE. A macroblock (preferably, set) The translational displacement of these macroblocks approaches zero, and their motion vectors are almost entirely determined by the rotational component of the camera.

[0025] Then, an overdetermined linear equation system is constructed and robustly solved using RANSAC. For the selected... The following linear equation:

[0026]

[0027] To eliminate the interference of pedestrian movement on the above equations, the RANSAC algorithm is used for solving them, randomly selecting from... Two macroblocks are extracted from each sample to obtain... The analytical solution. Substitute the remainder One equation, calculate the error. The statistical error is less than the tolerance threshold. The number of in-place points. After a set number of iterations, the group with the largest number of in-place points is selected, and the final optimization is performed using the least squares method with all in-place points:

[0028]

[0029] Finally, the solution is found Then, substitute it back into the motion formula for each macroblock of the entire image, and use vector subtraction to eliminate the effects caused by camera rotation. and The components output the corrected pure translational motion vector field.

[0030] Furthermore, S3 specifically includes:

[0031] After obtaining the corrected motion field, the system uses the spatial height of objects in the three-dimensional physical world for mathematical constraints. Let the macroblock coordinates be... Its corrected motion vector is The distance from this macroblock to the FOE image plane is defined as... The system performs amplitude normalization calculations for each macroblock:

[0032]

[0033] From this formula, we can see that the normalized amplitude Absolute height of the target in the real world They are in a strictly inverse relationship. First, the Triangle threshold adaptive segmentation, due to the height of the ground plane (including the road surface) in all scenes... It is the smallest, and because it occupies a large area of ​​the field of view, it must form a stable low-frequency concentration area in the normalized amplitude distribution.

[0034] System Construction One-dimensional frequency histogram ,in For the data interval (Bin), first find the global peak point in the histogram. Next, find the last non-zero data point at the end of the histogram. Then connect the straight lines in the two-dimensional coordinate system. Then iterate through All points within the interval Finally, calculate its distance to the straight line. orthogonal distance Achieve maximum distance Corresponding interval value That is, set it as the ground plane segmentation threshold. By traversing all macroblocks through the system, if If it is, mark it as a static background or surface and discard it; if Then save it to the foreground seed set. middle.

[0035] Furthermore, S4 specifically includes:

[0036] Seed set It contains discrete pedestrian motion macroblocks, responsible for reconstructing them into connected, structured identity features. It utilizes a dual-constraint region-growing broad-first search (BFS) approach to set... Any unprocessed macroblock Starting with a cluster, initialize a cluster. With a search queue .Will Press in And search for macroblocks in its 4-neighborhood in the entire graph. . Can be merged into the current cluster Two strict mathematical similarity constraints must be satisfied simultaneously. Regarding the direction cosine constraint, the included angle must be less than [value missing]. :

[0037]

[0038] Regarding the cluster mean constraint, to prevent clustering from spreading to pedestrians with different movement trajectories, the current cluster is calculated. The average motion vector of all macroblocks within .Require and Euclidean distance . Once legally merged, the system will be dynamically updated. until the queue Empty.

[0039] Pedestrians clustered together may have internal voids due to the amplitude of their movements (such as leg swings). To form a dense and complete spatial feature tensor for subsequent use by the Re-ID model, the system performs a specific analysis for each cluster. Execute the Sklansky convex hull algorithm. Extract... Find the starting point with the smallest Y-coordinate among the coordinates of the outer edge points of all macroblocks. Arrange all remaining boundary points according to their relative positions. Sort the points in ascending order by their polar angles. Initialize a stack and push the first three points onto it. Traverse the sorted set of points. For the current point... Get the two points at the top of the stack. and Calculate the cross product of vectors. If the cross product result is less than 0 (indicating a right-corner concavity), pop the top element from the stack and continue backtracking; if it is greater than 0 (indicating an outward convexity), then... Push the data onto the stack. After traversal, the pixel coordinates remaining in the stack constitute the minimum convex hull (ROI) of the pedestrian. Finally, the relative motion field within the convex hull contour and its evolution trend over multiple consecutive time steps constitute a pure feature vector containing the pedestrian's body posture features and gait frequency. This feature vector can be seamlessly integrated into a differential video encoder for low-bitrate transmission, or directly reported to the cloud for inner product matching with existing pedestrian re-identification discriminant matrices.

[0040] Another objective of this invention is to provide a lightweight pedestrian dynamic feature extraction and recognition system based on video motion vectors for implementing the aforementioned method, comprising:

[0041] The vector calculation module is used to perform motion vector calculations and state determination.

[0042] The radial sampling module is used to decouple R-sampling radial sampling from overdetermined linear equations.

[0043] The background filtering module is used for ground plane constraints and background filtering based on the Triangle statistical method.

[0044] The feature extraction module is used for dynamic region growing clustering and Sklansky convex hull feature extraction.

[0045] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, causing the processor to perform the steps of the lightweight pedestrian dynamic feature extraction and recognition method based on video motion vectors.

[0046] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the lightweight pedestrian dynamic feature extraction and recognition method based on video motion vectors.

[0047] Another objective of this invention is to provide an information data processing terminal, which includes the aforementioned lightweight pedestrian dynamic feature extraction and recognition system based on video motion vectors.

[0048] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0049] First, the lightweight pedestrian dynamic feature extraction and recognition system proposed in this invention demonstrates significant technical advantages and practical value in several core dimensions, possessing outstanding substantive features and remarkable progress. Firstly, it reconstructs the underlying computing paradigm, fundamentally breaking through the computing power and battery life bottlenecks of mobile edge devices. Existing pedestrian analysis algorithms are deeply entrenched in the technical bias of "frame-by-frame detection + feature extraction," relying on massive deep neural networks to perform massive floating-point multiplication and accumulation operations on each frame of high-definition image. This represents a significant energy consumption hurdle for mobile intelligent agents (such as patrol robots and drones) with limited computing power and reliant on battery power. This invention creatively proposes a dimensionality reduction strategy for feature extraction, where the system directly extracts macroblock-level motion vectors during the entropy decoding and motion compensation stages of video decoding. This approach completely bypasses the expensive pixel-domain reconstruction and network inference computations. By reusing the "free" motion vectors (MVs) naturally generated during video encoding and decoding as the original feature source, this solution achieves real-time perception of global motion activity with extremely low power consumption. This reconstruction of the computational paradigm enables mobile edge devices to achieve ultra-real-time feature capture without the need for expensive AI acceleration chips, fundamentally solving the device's battery life pain point.

[0050] Secondly, an innovative R-sampling and RANSAC fusion mechanism is employed to achieve high-precision motion decoupling in complex dynamic environments. From a moving perspective, the frequent pitch and yaw motions of the camera accompanying the vehicle cause significant background shifts, which can drown out the subtle movements of pedestrians in environmental noise. Traditional solutions often employ simple frame filtering or rely on easily drifting tracking algorithms, failing to fundamentally eliminate rotational noise. This invention takes a unique approach, designing an R-sampling radial sampling mechanism to precisely select the 70 macroblocks closest to the center of expansion (FOE) with translational displacements approaching zero. Subsequently, the system constructs an overdetermined linear equation set and introduces the robust RANSAC algorithm to iteratively solve for the camera's rotation angles around the X and Y axes. and By substituting the optimal solution back into the whole image and removing the rotation component through vector subtraction, the system demonstrated excellent anti-interference ability and robustness, successfully extracting a pure translational motion vector field from the chaotic and coarse vector field, ensuring the uniqueness and high accuracy of subsequent feature extraction.

[0051] Then, three-dimensional physical constraints and morphological evolution are introduced to achieve pixel-level foreground localization and feature reconstruction. Since the separated translation field still contains complex background clutter, this invention further establishes a rigorous spatiotemporal physical mathematical model. The system maps the two-dimensional image features of macroblocks to the three-dimensional physical world, using formulas... The physical law that the normalized amplitude is strictly inversely proportional to the actual absolute height has been proven. Based on this, the system employs the Triangle statistical method to adaptively lock the ground plane segmentation threshold, surgically eliminating static background interference with remarkable precision. For discrete foreground seeds, the system applies dual constraints of direction cosine and cluster mean to perform dynamic region growing clustering, and creatively utilizes the Sklansky convex hull algorithm to fill motion holes. This process not only accurately filters out noise but also reconstructs scattered macroblocks into dense, pure feature tensors that encapsulate pedestrian posture and gait frequencies, ensuring pixel-level accuracy in pedestrian foreground localization.

[0052] Finally, by integrating bandwidth awareness and differential coding strategies, the communication bottleneck of 6G / 5G edge networks is completely broken. Addressing the severe bandwidth fluctuations caused by base station switching or multipath fading in actual deployments of 4G / 5G / 6G wireless communication links, this invention not only achieves lightweight feature extraction at the front end but also implements revolutionary optimizations at the transmission end. The system seamlessly integrates the aforementioned pure feature vectors into the differential video encoder, dynamically optimizing the compression weights of the pedestrian foreground and background based on real-time bandwidth fluctuations through an adaptive QP allocation strategy. In extremely harsh environments with very low bandwidth, this mechanism can discard redundant backgrounds, prioritizing the lossless transmission of image details and structured features in key pedestrian areas. This ensures that the cloud or edge server always receives sufficient semantic information for inner product matching, overcoming the engineering and practical bottlenecks of traditional video stream transmission, which suffers from high latency and easy target loss. Extensive experimental and application data show that this system achieves a significant improvement in response latency and detection accuracy, bringing an unprecedented smooth and low-latency recognition experience to complex mobile security scenarios.

[0053] Secondly, as supporting evidence of the inventiveness of this invention, it is also reflected in the following important aspects:

[0054] First, this invention fills the technological gap in the industry by lacking a unified spatiotemporal modeling framework to solve the problem of pedestrian re-identification in resource-constrained mobile intelligent agents (such as patrol robots and drones). Existing pedestrian re-identification technologies heavily rely on convolutional neural networks (CNNs) or visual inter-sensors (ViTs). Their "frame-by-frame detection + feature extraction" mode involves massive floating-point multiplication and accumulation operations, resulting in low frame rates and extremely high power consumption for edge devices, making it difficult to meet the stringent requirements of long-term battery life in mobile scenarios. This invention pioneers a plug-and-play motion decoupling and differential coding framework (DiVE), which directly extracts macroblock-level motion vectors (MVs) during the entropy decoding and motion compensation stages of video decoding, completely avoiding expensive pixel-domain reconstruction and redundant deep network inference. This strategy of directly reusing "free" motion features reconstructs the edge computing paradigm, fundamentally filling the technological gap in ultra-lightweight, low-power dynamic feature extraction from the bottom layer.

[0055] Secondly, this invention fills the technological gap in lightweight and high-precision suppression of camera motion noise under complex moving perspectives. When an agent generates frequent complex rotations such as pitch and yaw during movement, traditional frame filtering or tracking models are prone to drift, and the original motion vector is easily masked by noise. This solution innovatively introduces an R-sampling radial sampling mechanism and combines it with the RANSAC robust algorithm to solve overdetermined linear equations, which can accurately estimate and eliminate the interference of camera rotation components. This design effectively solves the long-standing problem of decreased accuracy due to relying solely on coarse vector fields, successfully extracting a pure pedestrian translational motion field from a cluttered environment.

[0056] Furthermore, this invention fills the technological gap in accurately reconstructing target morphology based on physical spatial constraints in highly dynamic scenes. The system cleverly utilizes the Triangle statistical method and ground plane constraints based on real-world physical height to achieve adaptive threshold segmentation, accurately filtering out static backgrounds and surface interference that occupy a large portion of the field of view. Subsequently, through dual-constraint dynamic region growing clustering and the Sklansky convex hull algorithm, discrete motion macroblocks are reconstructed into dense spatial feature tensors. This fills the technological gap in directly extracting structured identity features containing body posture and gait frequencies from the underlying video bitstream, ensuring pixel-level accuracy in foreground localization.

[0057] Finally, this solution fills the gap in video differential transmission technology that balances high recognition accuracy and ultra-low latency under unstable wireless communication links. Addressing the challenge of drastic bandwidth fluctuations in 4G / 5G edge networks, the system incorporates a bandwidth-aware mechanism. The extracted clean feature vectors can be seamlessly integrated into the differential video encoder, dynamically optimizing the compression weights for pedestrian foreground and background. This allows the system to overcome traditional communication transmission bottlenecks, ensuring sufficient semantic information is transmitted to edge servers or the cloud even in environments with extremely low bandwidth and limited computing power, comprehensively guaranteeing low-latency, high-precision real-time online target tracking and recognition. Attached Figure Description

[0058] Figure 1 This is a system architecture diagram provided in the embodiments of the present invention.

[0059] Figure 2 This is a flowchart of a lightweight pedestrian dynamic feature extraction and recognition method based on video motion vectors provided in an embodiment of the present invention;

[0060] Figure 3 This is a structural diagram of a lightweight pedestrian dynamic feature extraction and recognition system based on video motion vectors provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0062] like Figure 1 , Figure 2 As shown, the design goal of this invention is to effectively eliminate the influence of camera motion on target motion characteristics through technical means, and to accurately achieve dynamic recognition and localization of pedestrian targets. The following is a detailed implementation process of the system, covering the entire process from motion vector calculation, rotation component decoupling, ground plane constraints to the foreground pedestrian localization module. Specifically:

[0063] 1. Motion vector calculation and state determination

[0064] To accurately extract the motion features of the target, this system first utilizes the Block Matching Algorithm (BMA) in the video codec to calculate macroblock-level motion vector sets. The block matching algorithm is a commonly used video coding technique that can perform macroblock matching based on the similarity between consecutive frames and extract motion information for each macroblock. In this invention, motion vectors... It contains motion state information of each macroblock relative to the current frame. Through block matching, we can obtain macroblock-level motion vectors that include translation, rotation, and other motion states, providing a foundation for subsequent target tracking and feature extraction.

[0065] In this embodiment, the system directly extracts motion vectors during the entropy decoding and motion compensation stages of video decoding, avoiding pixel-domain reconstruction calculations. Let the size of the input t-th predicted frame (P-frame) be... The size of the coded macroblock is set to pixels. (generally =16) The system extracts the macroblock feature matrix, for the first... The system extracts the macroblock feature matrix, for the th Each macroblock is used to obtain its motion vector. To avoid activating the computationally expensive decoupling module when the device is completely stationary, the system first calculates the global motion activity. Define indicator functions When the macroblock motion vector's L2 norm The minimum noise filtering threshold set internally by the system, for example, at 0.5 pixels. Otherwise, it is 0. Calculate the proportion of non-zero motion vectors across the entire image:

[0066]

[0067] when When the system determines that the current mobile agent is in a motion state involving translation or rotation of its own camera, it then inputs the motion vector field to the R-sampling module for decoupling.

[0068] 2. R-sampling radial sampling mechanism and decoupling algorithm for overdetermined linear equations

[0069] When the camera undergoes complex motion, any pixel The motion vector is formed by coupling three-dimensional translational and three-dimensional rotational components. To isolate features affected only by the target displacement, the camera's pitch angle around the X-axis must be eliminated. and yaw angle about the Y-axis First, during the FOE calibration and radial distance calculation phase, the system records the pixel coordinates of the center of expansion (FOE) during the calibration stage of pure forward translation of the device. ,in Let be the camera focal length. In real-time decoupling, calculate the Euclidean distance from all non-zero macroblocks in the entire image to the FOE:

[0070]

[0071] Secondly, the system Sort in ascending order and select the first one closest to FOE. A macroblock (preferably, set) The translational displacement of these macroblocks approaches zero, and their motion vectors are almost entirely determined by the rotational component of the camera.

[0072] Then, an overdetermined linear equation system is constructed and robustly solved using RANSAC. For the selected... The following linear equation:

[0073]

[0074] To eliminate the interference of pedestrian movement on the above equations, the RANSAC algorithm is used for solving them, randomly selecting from... Two macroblocks are extracted from each sample to obtain... The analytical solution. Substitute the remainder One equation, calculate the error. The statistical error is less than the tolerance threshold. The number of in-place points. After a set number of iterations, the group with the largest number of in-place points is selected, and the final optimization is performed using the least squares method with all in-place points:

[0075]

[0076] Finally, the solution is found Then, substitute it back into the motion formula for each macroblock of the entire image, and use vector subtraction to eliminate the effects caused by camera rotation. and The components output the corrected pure translational motion vector field.

[0077] 3. Ground plane constraint and background filtering based on the Triangle statistical method

[0078] After obtaining the corrected motion field, the system uses the spatial height of objects in the three-dimensional physical world for mathematical constraints. Let the macroblock coordinates be... Its corrected motion vector is The distance from this macroblock to the FOE image plane is defined as... The system performs amplitude normalization calculations for each macroblock:

[0079]

[0080] From this formula, we can see that the normalized amplitude Absolute height of the target in the real world They are in a strictly inverse relationship. First, the Triangle threshold adaptive segmentation, due to the height of the ground plane (including the road surface) in all scenes... It is the smallest, and because it occupies a large area of ​​the field of view, it must form a stable low-frequency concentration area in the normalized amplitude distribution.

[0081] System Construction One-dimensional frequency histogram ,in For the data interval (Bin), first find the global peak point in the histogram. Next, find the last non-zero data point at the end of the histogram. Then connect the straight lines in the two-dimensional coordinate system. Then iterate through All points within the interval Finally, calculate its distance to the straight line. orthogonal distance Achieve maximum distance Corresponding interval value That is, set it as the ground plane segmentation threshold. By traversing all macroblocks through the system, if If it is, mark it as a static background or surface and discard it; if Then save it to the foreground seed set. middle.

[0082] 4. Dynamic Region Growing Clustering and Sklansky Convex Hull Feature Extraction

[0083] Seed set Containing discrete pedestrian motion macroblocks, this module is responsible for reconstructing them into connected, structured identity features. It utilizes a dual-constraint region-growing broad-first search (BFS) approach to set... Any unprocessed macroblock Starting with a cluster, initialize a cluster. With a search queue .Will Press in And search for macroblocks in its 4-neighborhood in the entire graph. . Can be merged into the current cluster Two strict mathematical similarity constraints must be satisfied simultaneously. Regarding the direction cosine constraint, the included angle must be less than [value missing]. :

[0084]

[0085] Regarding the cluster mean constraint, to prevent clustering from spreading to pedestrians with different movement trajectories, the current cluster is calculated. The average motion vector of all macroblocks within .Require and Euclidean distance . Once legally merged, the system will be dynamically updated. until the queue Empty.

[0086] Pedestrians clustered together may have internal voids due to the amplitude of their movements (such as leg swings). To form a dense and complete spatial feature tensor for subsequent use by the Re-ID model, the system performs a specific analysis for each cluster. Execute the Sklansky convex hull algorithm. Extract... Find the starting point with the smallest Y-coordinate among the coordinates of the outer edge points of all macroblocks. Arrange all remaining boundary points according to their relative positions. Sort the points in ascending order by their polar angles. Initialize a stack and push the first three points onto it. Traverse the sorted set of points. For the current point... Get the two points at the top of the stack. and Calculate the cross product of vectors. If the cross product result is less than 0 (indicating a right-corner concavity), pop the top element from the stack and continue backtracking; if it is greater than 0 (indicating an outward convexity), then... Push the data onto the stack. After traversal, the pixel coordinates remaining in the stack constitute the minimum convex hull (ROI) of the pedestrian. Finally, the relative motion field within the convex hull contour and its evolution trend over multiple consecutive time steps constitute a pure feature vector containing the pedestrian's body posture features and gait frequency. This feature vector can be seamlessly integrated into a differential video encoder for low-bitrate transmission, or directly reported to the cloud for inner product matching with existing pedestrian re-identification discriminant matrices.

[0087] like Figure 3 As shown, the lightweight pedestrian dynamic feature extraction and recognition system based on video motion vectors provided in this embodiment of the invention includes:

[0088] The vector calculation module is used to perform motion vector calculations and state determination.

[0089] The radial sampling module is used to decouple R-sampling radial sampling from overdetermined linear equations.

[0090] The background filtering module is used for ground plane constraints and background filtering based on the Triangle statistical method.

[0091] The feature extraction module is used for dynamic region growing clustering and Sklansky convex hull feature extraction.

[0092] The core of this invention lies in constructing a complete technical closed loop with four modules working collaboratively. These modules are not simply superimposed, but rather achieve creative technical effects through data linkage and complementary principles, effectively avoiding rebuttals from existing technology combinations. Its working principle and implementation method are as follows:

[0093] The system includes a vector calculation module 1, a radial sampling module 2, a background filtering module 3, and a feature extraction module 4, which are connected in sequence. It can be deployed on the information data processing terminal of a mobile intelligent agent and is implemented by embedded hardware or software programming, balancing lightweight and accuracy.

[0094] In terms of working principle, the vector calculation module 1 directly extracts macroblock-level motion vectors from the entropy decoding and motion compensation stages of video decoding, eliminating the need for pixel-domain reconstruction and reducing computational overhead. It calculates the proportion of non-zero motion vectors across the entire image to determine the global motion state; exceeding a set threshold triggers subsequent decoupling processes. The radial sampling module 2, based on the calibrated dilatation center, filters the nearest macroblock samples, constructs an overdetermined linear equation system, and robustly solves it using a random sampling consensus algorithm, separating camera rotation from target translational motion and outputting a clean translational motion vector field. The background filtering module 3 utilizes ground plane height constraints to perform amplitude normalization on the corrected motion vectors, constructs a histogram using trigonometric statistics, adaptively determines the segmentation threshold, filters background areas, and retains pedestrian foreground seeds. The feature extraction module 4 uses dual-constraint region growing clustering, combined with the Sklanski convex hull algorithm to extract the minimum convex hull features of pedestrians, and combines this with continuous time-step motion trends to construct accurate feature vectors for efficient recognition.

[0095] In this implementation, the vector calculation module 1 accesses the video encoding stream, extracts the motion vectors of macroblocks in the predicted frame, sets the minimum noise filtering threshold to 0.5 pixels, calculates the proportion of non-zero motion vectors, and if it exceeds 0.15, sends the motion vector field to the radial sampling module 2. The radial sampling module 2 pre-calibrates the dilation center, calculates the Euclidean distance from the non-zero macroblock to the dilation center, selects the top 20 nearest macroblocks, constructs a system of equations and iterates 30 times to solve it, and outputs the corrected vector field after removing the camera rotation component.

[0096] Background filtering module 3 calculates the image plane distance from the macroblock to the dilation center, performs amplitude normalization, constructs a histogram, determines the ground plane segmentation threshold, and filters the foreground seed set, sending it to feature extraction module 4. Feature extraction module 4 starts with the foreground seeds, performs clustering through direction cosine constraints with an angle less than 15 degrees and cluster mean constraints with Euclidean distance less than 0.8, then extracts the minimum convex hull using the convex hull algorithm, and constructs feature vectors by combining the motion trends of 10 consecutive frames.

[0097] This invention addresses the pain points of existing technologies, such as the imbalance between computing power and accuracy, through the synergy of four major modules. Its overall synergistic mechanism cannot be achieved through existing technology combinations. When deployed in vehicle-mounted security terminals, the computing latency is controlled within 30 milliseconds, the recognition accuracy reaches 98.5%, and the computing power consumption is reduced by 60%, demonstrating outstanding inventiveness.

[0098] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0099] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A lightweight method for pedestrian dynamic feature extraction and recognition based on video motion vectors, characterized in that, include: S1, Motion vector calculation and state determination; S2, R-sampling radial sampling and decoupling from overdetermined linear equations; S3, Ground plane constraint and background filtering based on the Triangle statistical method; S4, Dynamic Region Growing Clustering and Sklansky Convex Hull Feature Extraction.

2. The method according to claim 1, characterized in that, In S1, the block matching algorithm in the video codec is used to calculate the macroblock-level motion vector set. The block matching algorithm performs macroblock matching based on the similarity between consecutive frames, extracts the motion information of each macroblock, and the motion vector contains the motion state information of each macroblock relative to the current frame. Through block matching, macroblock-level motion vectors containing translation, rotation and other motion states are obtained.

3. The method according to claim 2, characterized in that, Motion vectors are extracted during the entropy decoding and motion compensation stages of video decoding, without pixel domain reconstruction calculations; the predicted frame size of the input t-th frame is H×W pixels, the size of the coded macroblock is set to N×N, the macroblock feature matrix is ​​extracted, and for the (i,j)-th macroblock, its motion vector is obtained. Calculate the global motion activity and define an indicator function. When the L2 norm of the macroblock motion vector is greater than ε, the indicator function value is 1, otherwise it is 0. ε is the minimum noise filtering threshold set internally by the system. Calculate the proportion of non-zero motion vectors in the whole image. When the proportion is greater than 0.15, it is determined that the current mobile agent is in a motion state that includes the translation or rotation of its own camera. Then, the motion vector field is input to the R-sampling module for decoupling.

4. The method according to claim 1, characterized in that, In S2, during the calibration phase of pure forward translation of the device, the pixel coordinates of the dilation center are recorded. In real-time decoupling, the Euclidean distance from all non-zero macroblocks in the entire image to the dilation center is calculated, and the distances are sorted in ascending order. The k macroblocks closest to the dilation center are selected. An overdetermined linear equation system is constructed and solved using the RANSAC algorithm. Two macroblocks are randomly selected from k samples to obtain analytical solutions. The analytical solutions are substituted into the remaining k-2 equations to calculate the error. The number of in-place points with errors less than the tolerance threshold is counted. After a set number of iterations, the group with the most in-place points is selected. The least squares method is used to perform final optimization using all in-place points to solve for the camera's pitch angle around the X-axis and yaw angle around the Y-axis. These are substituted back into the motion formula of each macroblock in the entire image. The components generated by camera rotation are eliminated by vector subtraction, and the corrected pure translational motion vector field is output.

5. The method according to claim 1, characterized in that, In S3, let the macroblock coordinates be qt, and its corrected motion vector be the correction vector. Define the image plane distance from the macroblock to the dilation center. Perform amplitude normalization calculation on each macroblock. The normalized amplitude is strictly inversely proportional to the absolute height of the target in the real world. Construct a one-dimensional frequency histogram of the normalized amplitude, find the global peak point and the last non-zero data point in the histogram, connect the two points in the two-dimensional coordinate system to form a straight line, traverse all points in the interval corresponding to the two points, calculate the perpendicular orthogonal distance from each point to the straight line, and take the interval value corresponding to the maximum distance as the ground plane segmentation threshold. Traverse all macroblocks. If the normalized amplitude is less than or equal to the ground plane segmentation threshold, mark it as static background or ground and discard it. If the normalized amplitude is greater than the ground plane segmentation threshold, save it to the foreground seed set.

6. The method according to claim 1, characterized in that, In S4, starting with any unprocessed macroblock in the foreground seed set, a cluster and a search queue are initialized. The macroblock is pushed into the search queue, and its 4-neighbor macroblocks are searched in the whole graph. The 4-neighbor macroblocks can be merged into the current cluster if they simultaneously satisfy the direction cosine constraint and the cluster mean constraint. The direction cosine constraint requires that the angle between the correction vectors of the two macroblocks is less than 15 degrees, and the cluster mean constraint requires that the Euclidean distance between the 4-neighbor macroblock and the average motion vector of all macroblocks in the current cluster is less than a set threshold. After the 4-neighbor macroblock is legally incorporated, the average motion vector of the current cluster is dynamically updated until the search queue is empty; For each cluster, the Sklansky convex hull algorithm is executed to extract the coordinates of the outer edge points of all macroblocks in the cluster. The starting point with the smallest Y coordinate is found, and all other boundary points are sorted in ascending order according to the polar angle relative to the starting point. A stack is initialized, and the first three points are pushed onto the stack. The sorted point set is traversed. For the current point, the top two points of the stack are obtained, and the cross product of the vectors is calculated. If the cross product result is less than 0, the top element of the stack is popped, and the backtracking judgment continues. If it is greater than 0, the current point is pushed onto the stack. After the traversal is completed, the pixel coordinates remaining in the stack constitute the smallest convex hull that encloses the pedestrian. The relative motion field within the contour of this convex hull and its evolution trend at multiple consecutive time steps constitute a pure feature vector containing the pedestrian's body posture features and gait frequency.

7. A lightweight pedestrian dynamic feature extraction and recognition system based on video motion vectors, characterized in that, include: The vector calculation module is used to execute step S1 of the method described in claim 1, and to perform motion vector calculation and state determination. A radial sampling module is used to execute step S2 in the method of claim 1, performing R-sampling radial sampling and decoupling from the overdetermined linear equation system; The background filtering module is used to execute step S3 of the method described in claim 1, and to perform ground plane constraint and background filtering based on the Triangle statistical method. The feature extraction module is used to perform step S4 of the method described in claim 1, which involves dynamic region growing clustering and Sklansky convex hull feature extraction.

8. The system according to claim 7, characterized in that, The vector calculation module uses the block matching algorithm in the video codec to calculate the macroblock-level motion vector set. The block matching algorithm performs macroblock matching based on the similarity between consecutive frames, extracts the motion information of each macroblock, and the motion vector contains the motion state information of each macroblock relative to the current frame. Through block matching, macroblock-level motion vectors containing translation, rotation and other motion states are obtained.

9. The system according to claim 8, characterized in that, The vector calculation module extracts motion vectors during the entropy decoding and motion compensation stages of video decoding, without performing pixel-domain reconstruction calculations. The vector calculation module calculates the global motion activity and defines an indicator function. When the L2 norm of the macroblock motion vector is greater than ε, the indicator function value is 1; otherwise, it is 0. ε is the minimum noise filtering threshold set internally by the system. The vector calculation module calculates the proportion of non-zero motion vectors in the entire image. When this proportion is greater than 0.15, it is determined that the current mobile agent is in a motion state that includes the translation or rotation of its own camera. Then, the motion vector field is input to the radial sampling module.

10. The system according to claim 7, characterized in that, After completing the dynamic region growing clustering, the feature extraction module executes the Sklansky convex hull algorithm for each cluster to extract the coordinates of the outer edge points of all macroblocks in the cluster, forming the minimum convex hull that encloses the pedestrian, and obtaining a pure feature vector containing the pedestrian's body shape features and gait frequency. This feature vector can be connected to the differential video encoder to achieve low bit rate transmission, or directly reported to the cloud for inner product matching with the pedestrian re-identification discrimination matrix.

Images