A vision tracking method for embodied intelligent robots
By employing an embodied intelligent robot visual tracking method, and utilizing an improved YOLO8v model and temporal attention mechanism, the scale changes of vehicles in continuous video frames are dynamically captured. This solves the problem of tracking failure caused by high vehicle speed during vehicle tracking, and improves the accuracy and stability of recognition and tracking.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN EGO ROBOT CO LTD
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-23
AI Technical Summary
Existing technologies can accurately identify and track people floating on the sea surface during nighttime rescue operations, but they cannot adapt to changes in image scale caused by high vehicle speeds during vehicle tracking, leading to tracking failure.
Employing an embodied intelligent robot visual tracking method, a sequence of perceptual feature maps is generated through frame-by-frame iterative processing. Combined with an improved YOLO8v model and a temporal attention mechanism, the scale changes of the target in continuous video frames are dynamically captured.
It significantly improves the accuracy of target recognition and tracking, enhances the stability of scale changes between consecutive frames, and solves the tracking failure problem caused by abrupt scale changes.
Smart Images
Figure CN122265671A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image recognition technology, specifically relating to a visual tracking method for an embodied intelligent robot. Background Technology
[0002] Image recognition technology is a key technology in the field of artificial intelligence responsible for extracting and analyzing visual information from images or videos. Through a series of core technologies, such as feature extraction, deep learning models, and object detection algorithms, image recognition technology transforms complex visual data into structured information that machines can understand, and builds a multi-dimensional recognition framework to support various application scenarios, providing accurate and efficient visual perception capabilities for fields such as intelligent robots, autonomous driving, security monitoring, and medical image analysis.
[0003] The prior art (CN119229145B) discloses a method for identifying and tracking weak targets on the sea surface using shipborne infrared vision. The method includes: using a target recognition model to identify the coordinate information and confidence level of targets in an infrared video frame sequence, classifying them into high-confidence targets, low-confidence targets, and invalid targets; performing a judgment: if the target is empty, generating a new trajectory; otherwise, generating a target trajectory prediction result; matching the high-confidence target with the target trajectory prediction result; matching the low-confidence target with the trajectory that failed the first match; updating the successfully matched and unmatched trajectories, generating a new trajectory for the high-confidence target that failed the match; writing the updated coordinate information and the new trajectory into the target's trajectory sequence, iterating until the last frame image to complete the tracking.
[0004] The aforementioned patent solves the problem of weak targets being missing and unable to be accurately identified and tracked in existing nighttime environments. When rescuing people floating on the sea surface, the sea speed is relatively low, making it easier to accurately track and avoid losing the target. However, in actual vehicle tracking, the vehicle speed is relatively high, making it impossible to adapt to the scale changes of the vehicle in consecutive frames, resulting in tracking failure. Summary of the Invention
[0005] The purpose of this invention is to address the problem that when rescuing people floating on the sea surface, the sea speed is relatively low, making it easy to accurately track the target without losing it; however, in actual vehicle tracking, the vehicle speed is relatively high, making it unable to adapt to the scale changes of the vehicle in consecutive frames of the image, leading to tracking failure. Therefore, this invention proposes an embodied intelligent robot visual tracking method.
[0006] In a first aspect of this invention, a visual tracking method for an embodied intelligent robot is first proposed, the method comprising:
[0007] Acquire video of the target object and divide the video frame by frame to obtain an initial image sequence;
[0008] The first frame image is initialized to obtain a reference feature map, and the reference feature map is added to the initial image sequence as the first image to obtain the target sequence;
[0009] The perceptual feature map sequence is obtained by iterative processing based on the current frame image and the previous frame image; the current frame image is any one of the target sequences except for the reference feature map.
[0010] The tracking results are obtained by substituting the perceptual feature map sequence into the temporal tracking model.
[0011] Optionally, the initialization process of the first frame image to obtain the reference feature map includes:
[0012] Substituting the first frame image into the image detection model yields the target detection box, which is the identifier feature box of the target object being tracked.
[0013] The image detection model is an improvement based on the YOLO8v model, specifically:
[0014] Replace all C2f modules in the backbone and neck networks with C2f_MPC modules;
[0015] In the backbone network, the output of the second layer is connected to the EdgeG module, and the EdgeG module is connected to the fifth, seventh and ninth layers respectively;
[0016] The C2f_MPC module is obtained by modifying the C2f module, specifically as follows:
[0017] Replace the BotteNeck module in the C2f module with the BotteNeck_MPC module; the specific working principle of the BotteNeck_MPC module is as follows:
[0018] The features input to the BotteNeck_MPC module are determined as input feature tensors, and convolution is performed on the input feature tensors to obtain convolutional feature tensors;
[0019] The convolutional feature tensor is fed into the MPCA module to obtain the first feature tensor;
[0020] The first feature tensor is convolved to obtain the output feature tensor.
[0021] Optionally, the working principle of the MPCA module includes:
[0022] The input to the MPCA module is determined as the first feature;
[0023] The first feature is subjected to global adaptive average pooling, horizontal adaptive average pooling, and vertical adaptive average pooling respectively to obtain the second feature, the third feature, and the fourth feature;
[0024] The fifth feature is obtained by concatenating the third and fourth features;
[0025] The fifth feature is normalized to obtain the weighted feature, and the second feature and the weighted feature are multiplied element by element to obtain the output feature.
[0026] Optionally, the operation process of the EdgeG module includes:
[0027]
[0028] in, This is the output of the EdgeG module. It is the input of the EdgeG module. , , and The symbol represents the features generated during the operation, and the superscript indicates the kernel size; F represents the operator, Conv represents standard convolution, sobel represents edge feature extraction, upsample represents upsampling, and concat represents feature concatenation.
[0029] Optionally, the perceptual feature map sequence obtained by iterative processing based on the current frame image and the previous frame image includes:
[0030] The specific steps of the iterative process are as follows:
[0031] Step 1: Perform a search region operation on the current frame image to obtain a search feature map; the search region operation includes: continuously downsampling the current frame image to obtain a downsampled feature map, and normalizing the downsampled feature map to obtain a search feature map;
[0032] Step 2: Perform feature fusion on the search feature map, the reference feature map, and the output feature map of the previous frame image to obtain the fused feature map of the current frame; normalize the fused feature map to obtain the first feature map; if the previous frame image is the first image in the target sequence, then determine the reference feature map as the output feature map;
[0033] Step 3: Substitute the first feature map into the scale-aware module for processing to obtain the second feature map;
[0034] Step four: Use the first feature map as the query feature map, and the second feature map as the key feature map and value feature map respectively; calculate the third feature map based on the query feature map, key feature map, and value feature map using the time attention formula;
[0035] Step 5: Concatenate the first feature map and the third feature map to obtain a concatenated feature map; normalize the concatenated feature map to obtain a normalized feature map; perform a multilayer perceptron on the normalized feature map to obtain a fourth feature map; segment the fourth feature map to obtain the output feature map of the current frame image.
[0036] Step 6: Obtain the output feature map of the current frame image, and then execute steps 1 to 5. The iteration terminates when the last frame image in the target sequence is reached.
[0037] Step 7: Obtain all output feature maps and integrate them to obtain a perceptual feature map sequence.
[0038] Optionally, the principle and process of the scale-aware module include:
[0039] The first feature map of the current frame image is segmented to obtain the segmented output feature map, the baseline feature map, and the search feature map of the current frame image; the feature segmentation is performed based on the sequential connection points of the fused features.
[0040]
[0041]
[0042]
[0043]
[0044] in, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module; Concat represents channel concatenation; LN represents normalization. This represents the feature output of the scale-aware module for the current frame image.
[0045] Optionally, the step of calculating the third feature map based on the query feature map, key feature map, and value feature map using the time attention formula includes:
[0046] The specific calculation expression for the time attention formula is as follows:
[0047]
[0048] in, Output features representing temporal attention Let Q represent the transpose of the key feature map, Q represent the query feature map, V represent the value feature map, and Softmax() represent the soft maximization function. Here, D represents the attention scaling factor, H represents the total number of channels, and H represents the number of heads for temporal attention.
[0049] Optionally, segmenting the fourth feature map to obtain the output feature map of the current frame image includes:
[0050] The specific principle and process of the segmentation are as follows:
[0051] The fourth feature map of the current frame image is segmented to obtain an output feature map, a baseline feature map, and a search feature map; the feature segmentation is performed based on the sequential connection points of the fused features.
[0052] The output feature map after segmenting the fourth feature map is determined as the output feature map of the next frame image.
[0053] Optionally, the step of substituting the perceptual feature map sequence into the temporal tracking model to obtain the tracking result includes:
[0054] Acquire tracking state information for each frame in the perceptual feature map sequence; the tracking state information includes the position, velocity, and acceleration of the target object being tracked;
[0055] The tracking fluctuation coefficient is calculated based on the tracking status information of the perceived feature map sequence;
[0056] The tracking result is obtained by comparing the tracking volatility coefficient with the tracking volatility threshold.
[0057] Optionally, obtaining the tracking result by comparing the tracking volatility coefficient with the tracking volatility threshold includes:
[0058] If the tracking fluctuation coefficient is greater than the tracking fluctuation threshold, then the target object in the last frame of the perception feature map sequence is determined as the tracking result;
[0059] If the tracking volatility coefficient is less than or equal to the tracking volatility threshold, an early warning will be issued by the terminal.
[0060] The beneficial effects of this invention are:
[0061] This invention proposes a visual tracking method for embodied intelligent robots. By generating a sequence of perceptual feature maps through frame-by-frame iterative processing, it can dynamically capture scale changes such as scaling and stretching of the target object in consecutive video frames, thus completely eliminating the dependence on single-frame static images. This not only significantly improves the accuracy of target recognition and tracking but also enhances the stability of scale changes between consecutive frames, effectively solving the tracking failure problem caused by abrupt scale changes. Attached Figure Description
[0062] The invention will now be further described with reference to the accompanying drawings.
[0063] Figure 1 A flowchart illustrating a visual tracking method for an embodied intelligent robot provided in an embodiment of the present invention;
[0064] Figure 2 A network structure diagram based on the YOLO8v model is provided in an embodiment of the present invention;
[0065] Figure 3 This is a network structure diagram of an image detection model provided in an embodiment of the present invention;
[0066] Figure 4 This is a flowchart of an iterative process provided in an embodiment of the present invention. Detailed Implementation
[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and B can represent: A alone, A and B simultaneously, and B alone. Furthermore, descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" can explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.
[0068] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0069] This invention provides a visual tracking method for embodied intelligent robots. See also... Figure 1 , Figure 1 A flowchart illustrating a visual tracking method for an embodied intelligent robot provided in an embodiment of the present invention. The method includes the following steps:
[0070] Acquire a video of the target object being tracked, and divide the video frame by frame to obtain an initial image sequence;
[0071] The first frame image is initialized to obtain a reference feature map, and the reference feature map is added to the initial image sequence as the first image to obtain the target sequence;
[0072] A perceptual feature map sequence is obtained by iterative processing based on the current frame image and the previous frame image; the current frame image is any one of the target sequences except for the reference feature map;
[0073] The tracking results are obtained by substituting the perceptual feature map sequence into the temporal tracking model.
[0074] The embodied intelligent robot visual tracking method provided by this invention obtains a perceptual feature map sequence through iterative processing, and realizes the scaling, stretching and other scale changes of the target object in continuous frames in real time. It completely gets rid of the dependence on static frame images, improves the accuracy of target object recognition, improves the scale stability of continuous frame images, and solves the tracking failure problem.
[0075] Specifically, the target object to be tracked can be a vehicle, etc.
[0076] In one implementation, initializing the first frame image to obtain the baseline feature map includes:
[0077] Substitute the first frame image into the image detection model to obtain the target detection box, which is the identification feature box of the target object being tracked.
[0078] See Figure 2 , Figure 2 A network structure diagram based on the YOLO8v model is provided for an embodiment of the present invention. See [link to diagram]. Figure 3 , Figure 3 This is a network structure diagram of an image detection model provided in an embodiment of the present invention. The image detection model is an improvement based on the YOLO8v model, specifically:
[0079] Replace all C2f modules in the backbone and neck networks with C2f_MPC modules;
[0080] In the backbone network, the output of the second layer is connected to the EdgeG module, and the EdgeG module is connected to the fifth, seventh and ninth layers respectively.
[0081] The C2f_MPC module is derived from the C2f module through modifications, specifically:
[0082] Replace the BotteNeck module in the C2f module with the BotteNeck_MPC module; the specific working principle of the BotteNeck_MPC module is as follows:
[0083] The features input to the BotteNeck_MPC module are determined as the input feature tensor, and the input feature tensor is convolved to obtain the convolutional feature tensor;
[0084] Substitute the convolutional feature tensor into the MPCA module to obtain the first feature tensor;
[0085] Convolve the first feature tensor to obtain the output feature tensor.
[0086] Specifically, target detection boxes are obtained by recognizing images using image detection models;
[0087] In one implementation, all C2f modules in the backbone and neck networks are replaced with C2f_MPC modules. In the C2f_MPC modules, the feature tensors are convolved and then fed into the MPCA module for processing. The MPCA module can typically capture contextual information from different dimensions (such as spatial and channel) of the feature map, enhance the response to key regions, and suppress background noise. This allows the model to focus more on the key parts of the target, thereby extracting more discriminative features in complex backgrounds, ultimately improving the localization accuracy and classification confidence of the target detection box.
[0088] In one implementation, EdgeG-enhanced edge features are connected in parallel and in a skip-like fashion to three different scale layers (layers 5, 7, and 9 of the YOLOv8 model). This means that high-resolution edge information is injected into the feature pyramids at different levels. This structure facilitates more efficient multi-scale feature fusion in the feature pyramid network, making large target detection boxes more complete and small target detection boxes less likely to be lost, thereby improving the robustness of the tracking algorithm when facing changes in target scale.
[0089] In one implementation, the working principle of the MPCA module includes:
[0090] The input to the MPCA module is determined as the first feature;
[0091] The first feature is subjected to global adaptive average pooling, horizontal adaptive average pooling, and vertical adaptive average pooling respectively to obtain the second, third, and fourth features;
[0092] The fifth feature is obtained by concatenating the third and fourth features;
[0093] The fifth feature is normalized to obtain the weighted feature, and the second feature and the weighted feature are multiplied element-wise to obtain the output feature.
[0094] In one implementation, the EdgeG module's computation process includes:
[0095]
[0096] in, This is the output of the EdgeG module. It is the input of the EdgeG module. , , and The symbol represents the features generated during the operation, and the superscript indicates the kernel size; F represents the operator, Conv represents standard convolution, sobel represents edge feature extraction, upsample represents upsampling, and concat represents feature concatenation.
[0097] For details, please refer to the image detection model performance comparison table, as shown in the table below:
[0098]
[0099] Specifically, the target detection model achieved a peak detection accuracy of 82.3% mAP@0.5, which is 1.9% higher than the overall mAP of YOLOv8-based models. More importantly, it is 0.8% higher than the current advanced models, including YOLOv12n and YOLOv13n. The higher the mAP, the more adaptable it is to complex adaptive scenarios, thus improving the detection capability of image recognition.
[0100] In one implementation, the perceptual feature map sequence is obtained by iterative processing based on the current frame image and the previous frame image, including:
[0101] See Figure 4 , Figure 4 A flowchart of an iterative process provided in an embodiment of the present invention is provided. The specific steps of the iterative process are as follows:
[0102] Step 1: Perform a search region operation on the current frame image to obtain a search feature map; the search region operation includes: continuously downsampling the current frame image to obtain a downsampled feature map, and normalizing the downsampled feature map to obtain the search feature map;
[0103] Step 2: Perform feature fusion on the search feature map, the reference feature map, and the output feature map of the previous frame to obtain the fused feature map of the current frame. Normalize the fused feature map to obtain the first feature map. If the previous frame is the first image in the target sequence, then determine the reference feature map as the output feature map.
[0104] Step 3: Substitute the first feature map into the scale-aware module for processing to obtain the second feature map;
[0105] Step 4: Use the first feature map as the query feature map, and the second feature map as the key feature map and value feature map respectively; calculate the third feature map based on the query feature map, key feature map, and value feature map using the time attention formula;
[0106] Step 5: Concatenate the first feature map and the third feature map to obtain a concatenated feature map; normalize the concatenated feature map to obtain a normalized feature map; apply a multilayer perceptron to the normalized feature map to obtain a fourth feature map; segment the fourth feature map to obtain the output feature map of the current frame image.
[0107] Step 6: Obtain the output feature map of the current frame image, and then execute steps 1 to 5. The iteration terminates when the last frame image in the target sequence is reached.
[0108] Step 7: Obtain all output feature maps and integrate them to obtain a perceptual feature map sequence.
[0109] In one implementation, cross-frame fusion of baseline template features (initial semantic anchors) and historical output features (temporal context) is performed, and a temporal attention mechanism is used to dynamically weight historical information, effectively solving the problem of ambiguity caused by feature changes in the tracked object. This allows the model to establish effective temporal dependencies in long sequences, avoiding frame-by-frame error accumulation and enhancing feature stability during occlusion and deformation. The scale-aware module captures changes in the spatial dimension, while the temporal attention module mines correlations in the temporal dimension, achieving decoupling of spatial and temporal information. Finally, nonlinear recombination using residual concatenation and a multilayer perceptron ensures that the output feature map retains both fine-grained details and incorporates high-order semantic context, thus maintaining high-precision responses even when the target undergoes drastic motion or scale changes.
[0110] In one implementation, the principle of the scale-aware module includes:
[0111] Feature segmentation is performed on the first feature map of the current frame image to obtain the segmented output feature map, the baseline feature map, and the search feature map. The feature segmentation is based on the sequential connection points of the fused features, specifically based on fixed-length slices. For example, if the separation is based strictly on the sequential connection points of the fused features: feature lengths 1-64 are split into the output feature map; feature lengths 65-128 (64 features) are split into the baseline feature map; and feature lengths 129-384 (256 features) are split into the search feature map.
[0112]
[0113]
[0114]
[0115]
[0116] in, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module; Concat represents channel concatenation; LN represents normalization. This represents the feature output of the scale-aware module for the current frame image.
[0117] In one implementation, the third feature map is calculated using a time attention formula based on the query feature map, key feature map, and value feature map, including:
[0118] The specific calculation expression for the time attention formula is as follows:
[0119]
[0120] in, Output features representing temporal attention Let Q represent the transpose of the key feature map, Q represent the query feature map, V represent the value feature map, and Softmax() represent the soft maximization function. Here, D represents the attention scaling factor, H represents the total number of channels, and H represents the number of heads for temporal attention.
[0121] In one implementation, segmenting the fourth feature map to obtain the output feature map of the current frame image includes:
[0122] The specific principle and process of the segmentation are as follows:
[0123] The fourth feature map of the current frame image is segmented to obtain an output feature map, a baseline feature map, and a search feature map; the feature segmentation is performed based on the sequential connection points of the fused features.
[0124] The output feature map after segmenting the fourth feature map is determined as the output feature map of the next frame image.
[0125] In one implementation, the tracking result obtained by substituting the perceptual feature map sequence into the temporal tracking model includes:
[0126] Acquire tracking state information for each frame in the perceptual feature map sequence; the tracking state information includes the position, velocity, and acceleration of the target object being tracked;
[0127] The tracking fluctuation coefficient is calculated based on the tracking status information of the perceived feature map sequence;
[0128] The tracking result is obtained by comparing the tracking volatility coefficient with the tracking volatility threshold.
[0129] Specifically, the formula for calculating the tracking volatility coefficient is as follows:
[0130]
[0131] in, This represents the distance difference between the i-th frame and the previous frame. This represents the speed difference between the i-th frame and the previous frame. This represents the acceleration difference between the i-th frame and the previous frame. , , Let represent the average distance, average velocity, and average acceleration, respectively, and N represent the total number of frames; where the distance difference is the straight-line distance between the coordinate position of the i-th frame and the coordinate position of the (i-1)-th frame.
[0132] Specifically, the tracking fluctuation coefficient is a quantitative assessment of the trajectory stability of the target object being tracked;
[0133] In one implementation, the tracking result is obtained by comparing the tracking volatility coefficient with the tracking volatility threshold, including:
[0134] If the tracking fluctuation coefficient is greater than the tracking fluctuation threshold, then the target object in the last frame of the perception feature map sequence is determined as the tracking result;
[0135] If the tracking volatility coefficient is less than or equal to the tracking volatility threshold, an early warning will be issued by the terminal.
[0136] Specifically, the tracking fluctuation threshold is obtained by statistical analysis of historical data and set by relevant personnel; the tracking result includes the size and location of the target object; the warning indicates that the target object may be lost.
[0137] The foregoing has described one embodiment of the present invention in detail, but this content is merely a preferred embodiment and should not be considered as limiting the scope of the present invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the scope of the claims of this invention.
Claims
1. A somatic intelligent robot visual tracking method, characterized in that, The method includes: Acquire a video of the target object being tracked, and divide the video frame by frame to obtain an initial image sequence; The first frame image is initialized to obtain a reference feature map, and the reference feature map is added to the initial image sequence as the first image to obtain the target sequence; A perceptual feature map sequence is obtained by iterative processing based on the current frame image and the previous frame image; the current frame image is any one of the target sequences except for the reference feature map; The tracking results are obtained by substituting the perceptual feature map sequence into the temporal tracking model.
2. The embodied intelligent robot visual tracking method of claim 1, wherein, The initialization process of the first frame image to obtain the reference feature map includes: Substituting the first frame image into the image detection model yields the target detection box, which is the identifier feature box of the target object being tracked. The image detection model is an improvement based on the YOLO8v model, specifically: Replace all C2f modules in the backbone and neck networks with C2f_MPC modules; In the backbone network, the output of the second layer is connected to the EdgeG module, and the EdgeG module is connected to the fifth, seventh and ninth layers respectively; The C2f_MPC module is obtained by modifying the C2f module, specifically as follows: Replace the BotteNeck module in the C2f module with the BotteNeck_MPC module; the specific working principle of the BotteNeck_MPC module is as follows: The features input to the BotteNeck_MPC module are determined as input feature tensors, and convolution is performed on the input feature tensors to obtain convolutional feature tensors; Substitute the convolutional feature tensor into the MPCA module to obtain the first feature tensor; The first feature tensor is convolved to obtain the output feature tensor.
3. The embodied intelligent robot visual tracking method of claim 2, wherein, The working principle of the MPCA module includes: The input to the MPCA module is determined as the first feature; The first feature is subjected to global adaptive average pooling, horizontal adaptive average pooling, and vertical adaptive average pooling respectively to obtain the second feature, the third feature, and the fourth feature; The fifth feature is obtained by concatenating the third and fourth features; The fifth feature is normalized to obtain the weighted feature, and the second feature and the weighted feature are multiplied element by element to obtain the output feature.
4. The visual tracking method for an embodied intelligent robot according to claim 2, characterized in that, The operation process of the EdgeG module includes: ； in, This is the output of the EdgeG module. It is the input of the EdgeG module. , , and The symbol represents the features generated during the operation, and the superscript indicates the kernel size; F represents the operator, Conv represents standard convolution, sobel represents edge feature extraction, upsample represents upsampling, and concat represents feature concatenation.
5. The visual tracking method for an embodied intelligent robot according to claim 1, characterized in that, The perceptual feature map sequence obtained by iterative processing based on the current frame image and the previous frame image includes: The specific steps of the iterative process are as follows: Step 1: Perform a search region operation on the current frame image to obtain a search feature map; the search region operation includes: continuously downsampling the current frame image to obtain a downsampled feature map, and normalizing the downsampled feature map to obtain a search feature map; Step 2: Perform feature fusion on the search feature map, the reference feature map, and the output feature map of the previous frame image to obtain the fused feature map of the current frame; normalize the fused feature map to obtain the first feature map; if the previous frame image is the first image in the target sequence, then determine the reference feature map as the output feature map; Step 3: Substitute the first feature map into the scale-aware module for processing to obtain the second feature map; Step four: Use the first feature map as the query feature map, and the second feature map as the key feature map and value feature map respectively; calculate the third feature map based on the query feature map, key feature map, and value feature map using the time attention formula; Step 5: Concatenate the first feature map and the third feature map to obtain a concatenated feature map; normalize the concatenated feature map to obtain a normalized feature map; perform multilayer perceptron processing on the normalized feature map to obtain a fourth feature map; segment the fourth feature map to obtain the output feature map of the current frame image. Step 6: Obtain the output feature map of the current frame image, and then execute steps 1 to 5. The iteration terminates when the last frame image in the target sequence is reached. Step 7: Obtain all output feature maps and integrate them to obtain a perceptual feature map sequence.
6. The visual tracking method for an embodied intelligent robot according to claim 5, characterized in that, The principle and process of the scale sensing module include: The first feature map of the current frame image is segmented to obtain the segmented output feature map, the baseline feature map, and the search feature map of the current frame image; the feature segmentation is performed based on the sequential connection points of the fused features. ； ； ； ； ； in, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module, It is Input the output of the multi-scale grouping module; Concat represents channel concatenation; LN represents normalization. This represents the feature output of the scale-aware module for the current frame image.
7. The visual tracking method for an embodied intelligent robot according to claim 5, characterized in that, The process of calculating the third feature map using the time attention formula based on the query feature map, key feature map, and value feature map includes: The specific calculation expression for the time attention formula is as follows: ； in, Output features representing temporal attention Let Q represent the transpose of the key feature map, Q represent the query feature map, V represent the value feature map, and Softmax() represent the soft maximization function. Here, D represents the attention scaling factor, H represents the total number of channels, and H represents the number of heads for temporal attention.
8. The visual tracking method for an embodied intelligent robot according to claim 5, characterized in that, The output feature map of the current frame image is obtained by segmenting the fourth feature map, including: The specific principle and process of the segmentation are as follows: The fourth feature map of the current frame image is segmented to obtain an output feature map, a baseline feature map, and a search feature map; the feature segmentation is performed based on the sequential connection points of the fused features. The output feature map after segmenting the fourth feature map is determined as the output feature map of the next frame image.
9. The visual tracking method for an embodied intelligent robot according to claim 1, characterized in that, The step of substituting the perceptual feature map sequence into the temporal tracking model to obtain the tracking result includes: Acquire tracking state information for each frame in the perceptual feature map sequence; the tracking state information includes the position, velocity, and acceleration of the target object being tracked; The tracking fluctuation coefficient is calculated based on the tracking status information of the perceived feature map sequence; The tracking result is obtained by comparing the tracking volatility coefficient with the tracking volatility threshold.
10. A visual tracking method for an embodied intelligent robot according to claim 9, characterized in that, The process of obtaining the tracking result by comparing the tracking volatility coefficient with the tracking volatility threshold includes: If the tracking fluctuation coefficient is greater than the tracking fluctuation threshold, then the target object in the last frame of the perception feature map sequence is determined as the tracking result; If the tracking volatility coefficient is less than or equal to the tracking volatility threshold, an early warning will be issued by the terminal.