A lightweight time sequence gait identity recognition method for intelligent inspection robots

Abstract

The application provides a lightweight time sequence gait identity recognition method for an intelligent inspection robot, comprising: collecting inspection videos by the intelligent inspection robot, detecting, continuously tracking and human contour segmentation to form a contour image sequence; inputting a 2D convolution backbone network to extract spatial features frame by frame; setting a multi-frame time sequence shift module to adopt one-way multi-frame time sequence shift or two-way multi-frame time sequence shift on the continuous multi-frame feature sequence to obtain a frame-level feature sequence; inputting a local-global dual-branch time sequence aggregation module including a locally multi-scale branch and a globally query branch arranged in parallel; and splicing the locally multi-scale features and the globally time sequence features according to the channel dimension to form gait identity features for recognition and matching. The method can reduce the end-side time sequence modeling cost, improve the stability under the condition of abnormal frames, expand the time perception range of the shallow network, take into account the local multi-scale information and the global structure, and adapt to the variable length sequence in the robot inspection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, biometric recognition, and intelligent robot visual perception technology, and more specifically, to a lightweight temporal gait identification method for intelligent inspection robots. Background Technology

[0002] Intelligent inspection robots have been gradually applied in industrial parks, power plants, chemical plants, warehouses, ports, and border areas. These robots typically perform continuous patrols along predetermined or autonomously planned routes, using onboard cameras to perceive surrounding personnel, equipment, and the environment. To achieve personnel access verification in key areas, early warning of unfamiliar individuals, and target tracking, the robots need to complete long-distance identity verification without requiring the active cooperation of the individuals being identified.

[0003] Facial, fingerprint, and iris recognition methods have high requirements for imaging distance, target cooperation, and image clarity. Gait recognition, on the other hand, analyzes the individual differences in the human body's appearance and periodic movement characteristics during walking to determine identity. It features non-contact, long-distance, and difficult-to-disguise characteristics, making it suitable for inspection robots to perceive naturally walking people during movement.

[0004] Existing gait identification methods based on pedestrian contour sequences typically first detect and track pedestrians from videos, then extract continuous human contour images, use neural networks to form gait features, and finally compare them with an identity feature database. To simultaneously model spatial appearance and temporal motion information, some methods employ 3D convolutional networks to process continuous frames. However, 3D convolution requires performing convolution operations in three dimensions: time, height, and width, resulting in a large number of parameters and computational overhead. The computing platform of an inspection robot also needs to handle navigation, obstacle avoidance, localization, communication, and equipment status analysis, leaving limited computing power, storage space, and power consumption for identification. Therefore, the heavy 3D convolutional model is not conducive to deployment on the robot's edge.

[0005] Another type of lightweight method shifts some feature channels along the time dimension to achieve information exchange between adjacent frames at a lower cost. Traditional temporal shifting typically only obtains features from the previous or next frame. When the inspection robot moves, body vibration, turning, and changes in camera viewpoint can cause human position shifts in consecutive frames; mutual occlusion between personnel, occlusion by carried objects, and foreground segmentation errors can also cause abrupt changes in the local contour of a frame. If only the features of a single adjacent frame are directly shifted into the current frame, noise in abnormal frames can easily propagate to subsequent features, and the temporal perception range obtained by shallow lightweight networks through multiple adjacent shifts is still limited.

[0006] When forming sequence-level gait features, existing methods often perform max pooling along the time dimension, that is, for each feature location, only the maximum response in the entire sequence is retained. This approach is computationally simple, but it directly compresses the time dimension, making it difficult to simultaneously retain short-term micro-movements, motion patterns across multiple gait stages, and the global structure in the complete sequence. When the distance between the robot and the pedestrian changes continuously, the pedestrian's gait speed is inconsistent, the tracking time is different, or only incomplete gait cycles are collected, fixed-length sampling or single max pooling is more likely to lose effective information.

[0007] Existing gait recognition methods in the field have achieved multi-frame interaction through multi-hop temporal shifting, but their multi-branch structure increases model complexity. Context-sensitive temporal learning methods fuse features from different time ranges through attention mechanisms, but this leads to increased parameters and computational overhead as the number of channels increases. Therefore, it is necessary to propose a gait identification method suitable for deployment on the edge of intelligent inspection robots, which should maintain the lightweight nature of the 2D convolutional backbone network, reduce feature perturbations caused by abnormally adjacent frames, and take into account both local multi-scale motion information and global temporal information of variable-length sequences. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and provide a lightweight temporal gait identity recognition method for intelligent inspection robots. This method can reduce the overhead of edge-side temporal modeling, improve stability under abnormal frame conditions, expand the temporal perception range of shallow networks, take into account both local multi-scale information and global structure, and adapt to variable length sequences in robot inspection.

[0009] To achieve the above objectives, the present invention is implemented through the following technical solution: a lightweight temporal gait identification method for intelligent inspection robots, comprising the following steps:

[0010] S1. Gait sequence acquisition: The inspection video is collected by the camera device of the intelligent inspection robot. Pedestrians in the video are detected, continuously tracked and human body contour segmented. A variable-length contour image sequence of the same pedestrian is formed in chronological order. The contour images are preprocessed.

[0011] S2, Lightweight Frame-Level Feature Extraction: Input the contour image sequence into a 2D convolutional backbone network and extract spatial features frame by frame; Before the spatial convolution of the residual units of the 2D convolutional backbone network, set up a multi-frame temporal shift module to perform unidirectional or bidirectional multi-frame temporal shift on the continuous multi-frame feature sequence to obtain the frame-level feature sequence.

[0012] S3, Local-Global Dual-Branch Temporal Aggregation: This involves combining frame-level feature sequences... The input is a local-global dual-branch temporal aggregation module. This module includes a parallel local multi-scale branch and a global query branch. The local multi-scale branch forms hierarchical features with multiple temporal receptive ranges through cascaded channel-wise temporal convolutions. These features are then processed by concatenation, channel fusion, temporal max pooling, and horizontal partitioning to form local multi-scale features. The global query branch weights and aggregates effective frames of variable-length sequences through multiple learnable temporal queries to form global temporal features of fixed length that are independent of the length of the frame-level feature sequence.

[0013] S4. Gait feature fusion: Local multi-scale features and global temporal features are concatenated according to the channel dimension to form gait identity features;

[0014] S5. Identity Matching: The distance or similarity between the gait identity features of the pedestrian to be identified and the gait features of the pre-stored registered personnel is calculated, and the identification result is output according to the preset threshold.

[0015] Preferably, in step S2, the input feature sequence of the multi-frame timing shift module is set as follows: ;in, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively.

[0016] The multi-frame timing shift module employs one of the following two methods:

[0017] Method 1: Using unidirectional multi-frame time shift: Let the number of historical frames participating in the averaging be... The proportion of channels participating in timing exchange is ;Previous frame Each channel was replaced with The average feature of the corresponding channel of the nth historical frame is used, and the features of the remaining channels remain unchanged from the current frame; for the nth historical frame... Features of the output of the frame and multi-frame timing shift module for:

[0018] ;

[0019] Method 2: Using bidirectional multi-frame timing shift: replacing a portion of the channels with The average features of the historical frames were used to replace another part of the channels. The average feature of the subsequent frames remains unchanged for the remaining channels; for the , the average feature of the subsequent frames remains unchanged for the remaining channels; Features of the output of the frame and multi-frame timing shift module for:

[0020] .

[0021] Preferably, in step S3, the frame-level feature sequence of the input local-global dual-branch temporal aggregation module is set as follows: ;in, For batch size, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively.

[0022] The local multi-scale branch includes the following steps:

[0023] The frame-level feature sequence is Divide into preserved features on an average basis along the channel dimension and features to be transformed Preserving features No concatenated temporal convolutions are performed to preserve the original spatial features; features to be transformed Passing in sequence Lightweight timing transformation of the first layer; calculate the first layer. Layer timing transformation representation And recursively calculate the output of each layer of timing transformation. ;

[0024] Retain features Initial features to be transformed and the outputs of each layer of timing transformation Stitching along the channel dimension:

[0025] ;

[0026] Channel fusion is then performed through convolution, followed by batch normalization and linear rectified activation. Max pooling is then performed along the time dimension to obtain sequence-level local multi-scale features. The local multi-scale features are horizontally partitioned and converged along the height of the human body to retain the identification information of the head and shoulders, torso, arms and legs respectively.

[0027] Preferably, the first Layer timing transformation representation for:

[0028] ;

[0029] The output of each layer of timing transformation for:

[0030] ;

[0031] No. Effective time sensing range corresponding to the layer timing transformation output for:

[0032] .

[0033] Preferably, the number of lightweight timing transformation layers .

[0034] Preferably, in step S3, the global query branch includes the following steps:

[0035] Use convolution to extract frame-level feature sequences The number of channels is determined by Compress to Then pool along the width direction of the feature map to retain By dividing the human body into horizontal regions, features are obtained. ;

[0036] Features Rearrangement as a feature This allows for independent temporal convergence of each human body horizontal partition; [Settings / configurations] Learnable time series query ;

[0037] Key features are obtained through linear mapping. Sum value characteristics ; Calculate normalized weights along the time dimension ; Using normalized weights logarithmic features Perform a weighted summation to obtain the features. ; Features Rearranged as Fixed-dimensional features.

[0038] Preferably, the key features Sum value characteristics They are respectively:

[0039] ; ;

[0040] in, and It is a trainable mapping matrix;

[0041] The normalized weights for:

[0042]

[0043] in, For temperature parameters;

[0044] The features for:

[0045] .

[0046] Preferably, the number of channels Learnable time series query quantity Number of horizontal human body sections .

[0047] A readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the lightweight temporal gait identification method for intelligent inspection robots.

[0048] A computer device includes a processor and a memory for storing processor-executable programs, wherein when the processor executes the program stored in the memory, it implements the lightweight temporal gait identification method for intelligent inspection robots.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. Reduced edge-side temporal modeling overhead: The multi-frame temporal shift module achieves cross-frame interaction through data movement, mean calculation, and channel concatenation, without adding convolutional parameters to the 2D convolutional backbone network; local multi-scale branches only use channel-wise convolution along the time direction, avoiding the execution of full 3D convolution on all channels. Therefore, this invention is suitable for sharing robot edge-side computing resources with tasks such as navigation and obstacle avoidance.

[0051] 2. Improve stability under abnormal frame conditions: Compared with directly shifting in the features of a single adjacent frame, this invention averages the corresponding channels of multiple consecutive frames before shifting, so that the single-frame abrupt changes caused by occlusion, outline loss, body shaking or viewpoint shift are suppressed in the averaging process, thereby reducing noise propagation across frames.

[0052] 3. Expand the temporal perception range of shallow networks: Multi-frame shifting directly introduces information across multiple frames; local multi-scale branches form 3-frame, 5-frame, 7-frame, and 9-frame perception ranges through 4-layer cascaded temporal convolutions, enabling shallower 2D convolutional networks to extract continuous micro-movements and cross-stage motion patterns.

[0053] 4. Balancing local multi-scale information with global structure: The local multi-scale branch preserves significant local motion responses, while the global query branch transforms variable-length sequences into multiple structured temporal representations. The two branches complement each other within a unified module, avoiding the information loss caused by single max pooling that only retains peak responses.

[0054] 5. Adapting to variable-length sequences in robot inspection: The output dimension of the global query branch is determined by the number of queries rather than the number of input frames, which can adapt to changes in the relative distance between the robot and pedestrians, tracking time, and walking speed, making it easier to use fixed-dimensional features for identity database retrieval later. Attached Figure Description

[0055] Figure 1 This is a flowchart of the lightweight temporal gait identity recognition method for intelligent inspection robots according to the present invention;

[0056] Figure 2 This is a flowchart of the gait identity feature acquisition process of the present invention;

[0057] Figure 3 This is a schematic diagram of the structure of the multi-frame timing shift module of the present invention;

[0058] Figure 4 This is a schematic diagram of the structure of the local-global dual-branch timing aggregation module of the present invention. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0060] Example 1

[0061] This embodiment presents a lightweight temporal gait identification method for intelligent inspection robots, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0062] S1. Gait sequence acquisition: The inspection video is collected by the camera device of the intelligent inspection robot. Pedestrians in the video are detected, continuously tracked and human body contour segmented, and a variable length contour image sequence of the same pedestrian is formed in chronological order.

[0063] The camera device outputs continuous images; the pedestrian detection unit determines the human body region within the image; the target tracking unit assigns the same identifier to the same pedestrian across frames; the human body contour segmentation unit generates a foreground contour image from the human body region, and the contour images are arranged into a sequence according to the tracking identifier and timestamp.

[0064] The contour image is processed by position alignment, size normalization, and numerical normalization.

[0065] In one implementation, each frame of the contour image is normalized to a single-channel image with a height of 64 pixels and a width of 44 pixels. During the training phase, the frame order is kept unchanged, and horizontal flipping, random erasure, rotation, perspective transformation, and affine transformation can be used to simulate robot turning, vibration, occlusion, and viewpoint changes. During the recognition phase, all valid frames in a tracking trajectory can be used; for online inspection, a sliding time window can also be used to retain the most recent frames.

[0066] S2. Lightweight frame-level feature extraction: Input the contour image sequence into a 2D convolutional backbone network and extract spatial features frame by frame; Before the spatial convolution of the residual units of the 2D convolutional backbone network, set a multi-frame temporal shift module to perform unidirectional multi-frame temporal shift or bidirectional multi-frame temporal shift on the continuous multi-frame feature sequence to obtain a frame-level feature sequence containing stable temporal information.

[0067] Specifically, let the input feature sequence of the multi-frame timing shift module be: ;in, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively.

[0068] The multi-frame timing shift module employs one of the following two methods:

[0069] Method 1: Using unidirectional multi-frame timing shift: such as Figure 3 As shown, let the number of historical frames participating in the average be... The proportion of channels participating in timing exchange is ;Previous frame Each channel was replaced with The average feature of the corresponding channel of the nth historical frame is used, and the features of the remaining channels remain unchanged from the current frame; for the nth historical frame... Features of the output of the frame and multi-frame timing shift module for:

[0070] ;

[0071] Averaging can reduce the impact of a single anomalous frame on the shifted-in features; taking values ​​across multiple historical frames can expand the temporal perception range of a single module; channel shifting and splicing themselves do not introduce trainable parameters, thus maintaining the parameter efficiency of the 2D convolutional backbone network.

[0072] Insufficient at the beginning of the sequence For historical frames, cyclic padding, first-frame duplication, or zero-value padding can be used. Cyclic padding is preferred, where the frame corresponding to the end of the sequence is used as the starting position for the supplementary frame, taking advantage of the periodicity of gait movement. For strictly online recognition scenarios, first-frame duplication or averaging only existing historical frames is preferred to avoid using future information.

[0073] A multi-frame temporal shift module is placed before the spatial convolution of the 2D residual units, allowing the temporally swapped features to continue undergoing 2D spatial convolution. This module is preferably placed in the lower and middle layers of the backbone network, while the temporal shift is not placed in the highest layer, to preserve the spatial semantic stability of the high-level compressed features. The preferred number of historical frames... The channel switching ratio is 2. It can be set according to the number of backbone network channels and edge computing power.

[0074] Method 2: Using bidirectional multi-frame timing shift: replacing a portion of the channels with The average features of the historical frames were used to replace another part of the channels. The average feature of the subsequent frames remains unchanged for the remaining channels; for the , the average feature of the subsequent frames remains unchanged for the remaining channels; Features of the output of the frame and multi-frame timing shift module for:

[0075] ;

[0076] The unidirectional form is suitable for low-latency and online recognition, while the bidirectional form is suitable for offline recognition after obtaining the complete sequence. The unidirectional form is preferred to avoid interference between gait phase features in opposite time directions.

[0077] S3, Local-Global Dual-Branch Timing Aggregation: (e.g., ...) Figure 4 As shown, the batch size is Frame-level feature sequences The input is a local-global dual-branch temporal aggregation module. This module includes a parallel local multi-scale branch and a global query branch. The local multi-scale branch forms hierarchical features with multiple temporal receptive ranges through cascaded channel-wise temporal convolutions. These features are then processed by concatenation, channel fusion, temporal max pooling, and horizontal partitioning to form local multi-scale features. The global query branch weights and aggregates effective frames of variable-length sequences through multiple learnable temporal queries to form a fixed number of global temporal features.

[0078] Specifically, let the frame-level feature sequence of the input local-global dual-branch temporal aggregation module be... ;in, For batch size, For the number of frames in the sequence, For the number of channels, and These represent the feature map height and width, respectively. This module simultaneously feeds the same input into both the local multi-scale branch and the global query branch, processes the data separately, and then fuses them. This module, as a whole, replaces the traditional single temporal max pooling layer.

[0079] Local multi-scale branches include the following steps:

[0080] The frame-level feature sequence is Divide into preserved features on an average basis along the channel dimension and features to be transformed Preserving features No concatenated temporal convolutions are performed to preserve the original spatial features; features to be transformed Passing in sequence Lightweight timing transformation of the first layer; Layer timing transformation representation for:

[0081] ;

[0082] The output of each layer of timing transformation is recursively calculated using the following formula. :

[0083] ;

[0084] The dimensions of the convolution kernel in the time, height, and width directions are: Information is aggregated only along the time direction. The time perception range increases by 2 frames with each layer passed. Effective time sensing range corresponding to the layer timing transformation output for:

[0085] .

[0086] The optimal number of lightweight timing transformation layers should be set. This allows for the sequential acquisition of hierarchical temporal features covering 3, 5, 7, and 9 frames. Channel rearrangement is performed after each convolutional layer to allow information exchange between different channel groups.

[0087] Retain features Initial features to be transformed and the outputs of each layer of timing transformation Stitching along the channel dimension:

[0088] ;

[0089] Then through the size The convolutional layers are fused into channels, and batch normalization and linear rectified activation are performed sequentially. Then, max pooling is performed along the time dimension to obtain sequence-level local multi-scale features. The local multi-scale features are horizontally partitioned and converged along the height direction of the human body to retain the identification information of the head and shoulders, torso, arms and legs regions respectively.

[0090] The global query branch does not directly compress all frames into a single maximum response. Instead, it aggregates multiple structured temporal representations from a variable-length sequence through a fixed number of learnable queries. The global query branch includes the following steps:

[0091] The size is The convolution will transform the frame-level feature sequence The number of channels is determined by Compress to Then pool along the width direction of the feature map to retain By dividing the human body into horizontal regions, features are obtained. ;

[0092] Features Rearrangement as a feature This allows for independent temporal convergence of each human body horizontal partition; [Settings / configurations] Learnable time series query ;

[0093] Key features are obtained through linear mapping. Sum value characteristics :

[0094] ; ;

[0095] in, and It is a trainable mapping matrix;

[0096] Calculate normalized weights along the time dimension :

[0097] ;

[0098] The activation function can be the softmax function; For temperature parameters; using normalized weights logarithmic features Perform a weighted summation to obtain the features. :

[0099] ;

[0100] Features Rearranged as Fixed-dimensional features; due to the weights along the actual frame number The calculation ensures that, regardless of the length of the input sequence, the output contains... The time-series representation is thus able to adapt to changes in sequence length and walking speed caused by the relative motion between the robot and the pedestrian.

[0101] Number of channels Learnable time series query quantity Number of horizontal human body sections The eight learnable queries correspond to the eight typical movement phases of the gait cycle, but are not forcibly bound to them. Each query adaptively focuses on a discriminative time region through training. This branch outputs 64-dimensional channel features.

[0102] S4. Gait Feature Fusion: The local multi-scale features output by the local multi-scale branch and the global temporal features output by the global query branch are concatenated according to the channel dimension to form a gait identity feature that combines local salient motion information, cross-scale motion information and global temporal structure.

[0103] In one embodiment, the local multi-scale branch outputs 512-dimensional channel features, and the global query branch outputs 64-dimensional channel features. The two branches are concatenated along the channel dimension to obtain 576-dimensional partitioned gait identity features. The fused features retain both local salient responses and motion information across different temporal ranges, as well as the global temporal structure formed by multiple queries. During training, identity classification constraints and metric learning constraints are applied to the fused features; during recognition, the distances between each level partition are combined to form the gait feature distance between the person to be identified and the registered person.

[0104] S5. Identity Matching: The distance or similarity between the gait identity features of the pedestrian to be identified and the gait features of the pre-stored registered personnel is calculated. The identification results are output according to the preset threshold, including: registered personnel identity, unknown personnel marker or candidate identity ranking. The identification results are sent to the inspection task management unit of the intelligent inspection robot to perform permission verification, unfamiliar personnel warning or target personnel tracking.

[0105] A lightweight temporal gait feature extraction unit is composed of a 2D convolutional backbone network, a multi-frame temporal shifting module, and a local-global dual-branch temporal aggregation module.

[0106] During the training of the lightweight temporal gait feature extraction unit, the training data consists of a sequence of contour images with identity labels. The contour image sequence is input into the lightweight temporal gait feature extraction unit, and training is performed using a weighted sum of triplet loss and cross-entropy loss. In a preferred embodiment, the two losses have equal weights, the triplet loss interval is set to 0.2, and the cross-entropy classification scale factor is set to 16; stochastic gradient descent is used for optimization, with an initial learning rate of 0.1 and a weight decay of 0.0005. The above values ​​are used to illustrate feasible implementation and do not constitute a limitation on the scope of protection.

[0107] During identity registration, a camera device is used to acquire one or more gait sequences of authorized personnel. These sequences are then processed through the same preprocessing and feature extraction procedures to generate registration features. Multiple registration features for the same person can be averaged or stored separately to form a gait identity feature database.

[0108] On the SUSTech1K dataset, collected by the robot and including conditions such as carrying a bag, changing clothes, holding objects, shading, nighttime, and using an umbrella, the first-order hit rate of the complete model of this invention is 83.5%, which is 7.4 percentage points higher than the 76.1% of the 2D baseline model. The complete model contains approximately 6 million parameters, with approximately 2.7 billion floating-point operations and a single inference time of 2.1 milliseconds. The compared 3D convolutional model contains approximately 14.2 million parameters, with approximately 6.1 billion floating-point operations and a single inference time of 3.6 milliseconds. The above times are model inference data under the experimental settings of the paper. The actual robot edge speed also depends on the chip, image preprocessing, and parallel task load.

[0109] This embodiment can be applied to various gait identification scenarios. The following three scenarios illustrate this:

[0110] Scenario 1: Lightweight personnel identity verification for park inspection robots:

[0111] A park inspection robot equipped with a monocular camera and an edge computing unit is used. The robot moves along the inspection route, and the camera continuously collects video. The pedestrian detection and tracking unit assigns a tracking tag to each pedestrian, and the human contour segmentation unit outputs binary contour images arranged in time and normalizes the images to 64×44 pixels. The gait feature extraction network uses a 9-layer 2D residual network. Except for the highest stage, multi-frame temporal shift modules are set in the remaining residual units; the number of historical frames is set to 2, and unidirectional shifting is used. For the start position of the sequence, offline whole-segment recognition uses cyclic padding, and online sliding window recognition uses first-frame duplication. This configuration uses the average features of 2 historical frames to replace part of the current frame's channels, while the remaining channels retain the spatial information of the current frame.

[0112] The backbone network output simultaneously enters the local-global dual-branch temporal aggregation module. The local multi-scale branch divides the preserved and transformed channels in a 1:1 ratio, setting four layers of 3×1×1 channel-wise temporal convolutions. After each layer, channel rearrangement is performed, and all hierarchical features and preserved features are concatenated before channel fusion, temporal max pooling, and 16-person horizontal partitioning. The global query branch compresses the channels to 8 dimensions, setting eight shared learnable temporal queries for each of the 16 horizontal partitions, generating 64-dimensional global temporal features. The 512-dimensional features from the local branch are concatenated with the 64-dimensional features from the global branch to form a 576-dimensional gait identity feature.

[0113] The robot compares the features to be identified with the feature database of authorized personnel in the park. If the similarity exceeds a threshold, the robot returns the personnel's identity and performs area permission verification; if the similarity does not reach the threshold, the robot marks the person as an unknown person and sends the person's location, tracking identifier, and image clips to the inspection task management unit.

[0114] Scenario 2: Outdoor inspection of power stations or chemical plant areas:

[0115] After pedestrian tracking is completed, the inspection robot uses all valid contour frames of the trajectory for recognition. Considering the numerous occlusions and viewpoint changes in the site, random erasure, rotation, perspective transformation, and affine transformation are incorporated during training. The multi-frame temporal shift module employs unidirectional 2-frame averaging and cyclic interpolation; temporal shifting is mainly performed in the lower and middle layers of the backbone network, while the highest layer does not perform temporal shifting. For robots requiring stronger representation capabilities and with sufficient end-side resources, a 22-layer 2D residual network is used, while the input-output relationship of the multi-frame temporal shift module and the local-global dual-branch temporal aggregation module remains unchanged. On the Gait3D outdoor dataset, the implementation using a deeper backbone network achieves a first-place hit rate of 75.0%.

[0116] Scenario 3: Online sliding window for personnel tracking:

[0117] This is used for inspection tasks that require on-the-go identification. A first-in, first-out sequence buffer is established for each pedestrian tracking icon, storing only the most recent few frames of contour images. The first identification is triggered when the buffer reaches a preset minimum number of frames, and the identification result is updated every few new frames.

[0118] The multi-frame temporal shift module only uses the actual historical frames preceding the current frame; the first frame is copied at the beginning of the sequence, without using cyclic padding or future frames. The global query branch calculates normalized weights for the actual frames in the current buffer, so changes in the number of frames in the buffer do not change the output feature dimension. Confidence averaging can be performed on multiple consecutive recognition results, and permission is only granted when the identity result consistently and stably reaches a threshold, thus reducing misjudgments caused by short sequences and occasional occlusion.

[0119] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A lightweight temporal gait identification method for intelligent inspection robots, characterized in that: Includes the following steps: S1. Gait sequence acquisition: The inspection video is collected by the camera device of the intelligent inspection robot. Pedestrians in the video are detected, continuously tracked and human body contour segmented. A variable-length contour image sequence of the same pedestrian is formed in chronological order. The contour images are preprocessed. S2, Lightweight Frame-Level Feature Extraction: Input the contour image sequence into a 2D convolutional backbone network and extract spatial features frame by frame; Before the spatial convolution of the residual units of the 2D convolutional backbone network, set up a multi-frame temporal shift module to perform unidirectional or bidirectional multi-frame temporal shift on the continuous multi-frame feature sequence to obtain the frame-level feature sequence. S3, Local-Global Dual-Branch Temporal Aggregation: This involves combining frame-level feature sequences... Input local-global dual-branch temporal aggregation module; The local-global dual-branch temporal aggregation module includes parallel local multi-scale branches and global query branches: The local multi-scale branches form hierarchical features with multiple time perception ranges through cascaded channel-wise temporal convolutions, and after concatenation, channel fusion, temporal max pooling and horizontal partitioning, local multi-scale features are formed; The global query branch performs weighted aggregation of effective frames of variable-length sequences through multiple learnable temporal queries to form a global temporal feature of fixed length that is independent of the length of the frame-level feature sequence. S4. Gait feature fusion: Local multi-scale features and global temporal features are concatenated according to the channel dimension to form gait identity features; S5. Identity Matching: The distance or similarity between the gait identity features of the pedestrian to be identified and the gait features of the pre-stored registered personnel is calculated, and the identification result is output according to the preset threshold.

2. The lightweight temporal gait identification method for intelligent inspection robots according to claim 1, characterized in that: In step S2, the input feature sequence of the multi-frame timing shift module is assumed to be: ;in, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively. The multi-frame timing shift module employs one of the following two methods: Method 1: Using unidirectional multi-frame time shift: Let the number of historical frames participating in the averaging be... The proportion of channels participating in timing exchange is ;Previous frame Each channel was replaced with The average feature of the corresponding channel of the nth historical frame is used, and the features of the remaining channels remain unchanged from the current frame; for the nth historical frame... Features of the output of the frame and multi-frame timing shift module for: ； Method 2: Using bidirectional multi-frame timing shift: replacing a portion of the channels with The average features of the historical frames were used to replace another part of the channels. The average feature of the subsequent frames remains unchanged for the remaining channels; for the , the average feature of the subsequent frames remains unchanged for the remaining channels; Features of the output of the frame and multi-frame timing shift module for: 。 3. The lightweight temporal gait identification method for intelligent inspection robots according to claim 1, characterized in that: In step S3, let the frame-level feature sequence of the input local-global dual-branch temporal aggregation module be... ;in, For batch size, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively. The local multi-scale branch includes the following steps: The frame-level feature sequence is Divide into preserved features on an average basis along the channel dimension and features to be transformed Preserving features No concatenated temporal convolutions are performed to preserve the original spatial features; features to be transformed Passing in sequence Lightweight timing transformation of the first layer; calculate the first layer. Layer timing transformation representation And recursively calculate the output of each layer of timing transformation. ; Retain features Initial features to be transformed and the outputs of each layer of timing transformation Stitching along the channel dimension: ； Channel fusion is then performed through convolution, followed by batch normalization and linear rectified activation. Max pooling is then performed along the time dimension to obtain sequence-level local multi-scale features. The local multi-scale features are horizontally partitioned and converged along the height of the human body to retain the identification information of the head and shoulders, torso, arms and legs respectively.

4. The lightweight temporal gait identification method for intelligent inspection robots according to claim 3, characterized in that: The first Layer timing transformation representation for: ； The output of each layer of timing transformation for: ； No. Effective time sensing range corresponding to the layer timing transformation output for: 。 5. The lightweight temporal gait identification method for intelligent inspection robots according to claim 1, characterized in that: In step S3, the input feature sequence of the multi-frame temporal shift module is assumed to be: ;in, For the number of frames in the sequence, For the number of channels, and These are the feature map height and width, respectively. The global query branch includes the following steps: Use convolution to extract frame-level feature sequences The number of channels is determined by Compress to Then pool along the width direction of the feature map to retain By dividing the human body into horizontal regions, features are obtained. ; Features Rearrangement as a feature This allows for independent temporal convergence of each human body horizontal partition; [Settings / configurations] Learnable time series query ; Key features are obtained through linear mapping. Sum value characteristics ; Calculate normalized weights along the time dimension ; Using normalized weights logarithmic features Perform a weighted summation to obtain the features. ; Features Rearranged as Fixed-dimensional features.

6. A readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the lightweight temporal gait identification method for intelligent inspection robots as described in any one of claims 1-5.

7. A computer device comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the lightweight temporal gait identification method for intelligent inspection robots as described in any one of claims 1-5.

Images