Electronic device, method and computer program product for object tracking

Abstract

An electronic device, method, and computer program product for object tracking are disclosed herein. The electronic device constructs and runs an object tracking network to confirm the location information of the target object in the area image. The object tracking network includes a feature extraction backbone network, a feature fusion network, and a prediction network. The feature extraction backbone network consists of two feature extractors with shared weights. The feature fusion network includes a plurality of cross-attention layers and a channel attention module, and each cross-attention layer includes two parallel cross-attention converter modules. The object tracking network is a hybrid model that combines convolutional neural networks and Transformer architecture. By introducing the Transformer-based cross-attention mechanism and channel attention mechanism, the efficiency and accuracy of feature fusion are enhanced.

Description

Electronic devices, methods, and computer programs for object tracking This invention relates to an electronic device, method, and computer program product for object tracking. More specifically, this invention relates to an electronic device, method, and computer program product for object tracking based on a twin residual cross-attention network. Visual object tracking technology has important applications in the field of computer vision, such as autonomous driving, surveillance systems, and drones. Although deep learning technology has improved the performance of visual object tracking, real-time tracking has become a major challenge for resource-constrained edge devices as the number of model parameters and computational load increases. TransT (Transformer Tracking) is a target tracking algorithm based on the Transformer architecture. It utilizes the attention mechanism of the Transformer to model the complex relationship between the target and its surrounding environment, achieving more accurate target tracking. A key feature of TransT is its typical use of a Siamese Network structure, employing two identical networks to extract features from the target and the search region, and comparing these two features through an attention mechanism to locate the target. Because TransT uses a feature fusion network based entirely on the Transformer architecture with an attention mechanism, and employs not only a single layer of cross-attention (CFA) but also self-attention (ECA), the overall model has a very large number of parameters and computational cost, making real-time inference difficult. This problem becomes even more pronounced when this mechanism is applied to edge devices with relatively limited computing resources. In view of this, how to provide a visual target tracking solution that reduces the number of parameters and computational load compared to the TransT scheme, while improving accuracy, is a technical problem that urgently needs to be solved in the field of this invention. To address this, this invention proposes a hybrid model that integrates Convolutional Neural Networks (CNNs) and the Transformer architecture. By introducing Transformer-based cross-attention and channel attention mechanisms, this invention enhances the efficiency and accuracy of feature fusion and combines it with a depth estimation network. More specifically, to at least address the aforementioned technical problems, the present invention provides an electronic device for object tracking. The electronic device may include a storage device and a processor. The storage device can be used to store a region image, a target image, and an object tracking network. The processor is electrically connected to the storage device and can be used to construct and run the object tracking network, thereby confirming the location information of an object represented by the target image in the region image. The object tracking network may include a feature extraction backbone network, a feature fusion network, and a prediction network. The feature extraction backbone network may include two feature extractors sharing weights, and the two feature extractors can be used to extract features from the region image and the target image, respectively, thereby generating a target feature map and a region feature map. The feature fusion network may include a plurality of cross-attention layers and a channel attention module. Each cross-attention layer may include two parallel cross-attention converter modules, and in the feature fusion network, both the target feature map and the region feature map can be input into the two cross-attention converter modules. This feature fusion network can be used to perform a deep cross-correlation operation on the target feature map and the region feature map, which have been refined through multiple cross-attention layers, thereby generating a response map. Furthermore, the feature fusion network can also be used to generate a channel attention vector from the response map through the channel attention module. Additionally, the feature fusion network can perform an element-wise multiplication operation between the channel attention vector and the response map to generate a weighted response map. The prediction network can then be used to generate the location information based on the weighted response map. To at least address the aforementioned technical problems, the present invention also provides an object tracking method. This object tracking method can be executed by an electronic device and may include at least the steps of "constructing an object tracking network" and "running the object tracking network to confirm the location information of an object represented by a target image in a region image." The object tracking network may include a feature extraction backbone network, a feature fusion network, and a prediction network. The feature extraction backbone network may include two feature extractors sharing weights, and these two feature extractors can be used to extract features from the region image and the target image, respectively, thereby generating a target feature map and a region feature map. The feature fusion network may include a plurality of cross-attention layers and a channel attention module. Each cross-attention layer may include two parallel cross-attention converter modules, and in the feature fusion network, both the target feature map and the region feature map can be input into the two cross-attention converter modules. This feature fusion network can be used to perform a deep cross-correlation operation on the target feature map and the region feature map, which have been refined through multiple cross-attention layers, thereby generating a response map. Furthermore, the feature fusion network can also be used to generate a channel attention vector from the response map through the channel attention module. Additionally, the feature fusion network can perform an element-wise multiplication operation between the channel attention vector and the response map to generate a weighted response map. The prediction network can then be used to generate the location information based on the weighted response map. To at least solve the above-mentioned technical problems, the present invention also provides a computer program product that stores a program for object tracking. When the computer loads and executes the program, the above-mentioned object tracking method can be completed. This invention provides a novel, efficient, and accurate Transformer-based visual tracking network architecture, consisting of a feature extraction backbone network, a feature fusion network integrating a cross-attention transform mechanism and a channel attention mechanism, and a prediction, classification, and regression network. More specifically, this invention introduces the concept of a cross-attention mechanism, which allows the exchange of information specific to two types of features between two feature maps, thereby improving the feature representation capability extracted by the network. Furthermore, this invention also introduces channel-attention deep cross-correlation, enhancing feature representation and effectively fusing features through a simple module without reducing inference speed. Moreover, this invention integrates the cross-attention mechanism and channel-attention deep cross-correlation to achieve a feature fusion network, ultimately obtaining better prediction results while achieving real-time model performance. The foregoing provides a basic description of the present invention, including the technical problem solved by the present invention, the technical means adopted, and the technical effects achieved. The following will further illustrate various embodiments of the present invention. The following embodiments are not intended to limit the claimed invention to a particular environment, application, structure, process, or situation. In the attached drawings, elements not directly related to the claimed invention are omitted. The dimensions of the elements and the proportions between them in the attached drawings are merely illustrative and are not intended to limit the claimed invention. Unless otherwise stated, the same element symbols in the following text refer to the same elements. The terminology used herein is merely for the convenience of describing the embodiments and is not intended to limit the invention for which protection is sought. Unless otherwise clearly stated, the singular "a" or "an" shall be considered to include the plural. Terms such as "comprising," "including," and "having" are used to specifically describe the presence of features, integers, steps, operations, elements, components, and / or groups stated thereafter, but do not exclude the presence or addition of one or more other additional features, integers, steps, operations, elements, components, and / or groups, etc. The term "and / or" is used to indicate any or all combinations of one or more related enumerated items. When terms such as "first," "second," and "third" are used to describe elements, the purpose is not to limit the described elements, but merely to distinguish the described elements. Thus, for example, without departing from the spirit or scope of the invention for which protection is sought, the first element in sequence may also be named "second element." The first embodiment of the present invention is an electronic device for object tracking. Referring to FIG1, an electronic device 101 for object tracking is illustrated, which can be used to identify the position of a specific object (e.g., a person, pet, car, flying object, etc.) in an image, and then track the position of the object in a continuous image stream. In some embodiments, in addition to identifying the position of the object in each image, the electronic device 101 can also be used to physically track the object in a real field by means of movement, for example, as implemented as a service robot, robotic pet, automated vehicle, etc. The electronic device 101 may substantially include at least a processor 102 and a storage device 103. In some embodiments, the electronic device 101 may further include a camera module 104, a radio module 105, and / or a transport module 106. The processor 102 may be electrically connected to the storage device 103, the camera module 104, the radio module 105, and the transport module 106, and the camera module 104 may also be electrically connected to the storage device 103. The electrical connections between the aforementioned components may be direct (i.e., connected to each other without other components) or indirect (i.e., connected to each other through other components). Processor 102 can be a programmable special integrated circuit with computation, storage, and output / input capabilities. It can accept and process various encoded instructions to perform various logical and arithmetic operations and output corresponding results. Processor 102 can be programmed to interpret various instructions and execute various tasks or programs to complete the various processes described herein. For example, processor 102 can include: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), a Microprocessor, and / or a Microcontroller. Memory 103 can be used to store data generated by electronic device 101, data transferred from external devices, and / or data generated by the user through input / output elements (e.g., a combination of keyboard, mouse, touch panel, display, etc.). For example, storage 103 may include primary memory (also known as main memory or internal memory), and processor 102 may directly read instruction sets stored in primary memory and execute these instruction sets when needed. Storage 103 may also include secondary memory (also known as external memory or auxiliary memory), which can transfer stored data to primary memory through a data buffer. The secondary memory may be, for example, but not limited to, storage media such as hard drives and optical discs. Storage 103 may also include tertiary memory, such as pluggable storage devices or cloud hard drives. Depending on different needs, storage 103 may include primary memory, secondary memory, and / or tertiary memory. The camera module 104 may be a collection of components with image capture capabilities, and may include at least a lens, a photosensitive element, an image processing unit, and control circuitry for the aforementioned components. In some embodiments, the image processing operations performed by the image processing unit may also be performed by the processor 102. The camera module 104 may be used to capture and generate a region image 107 and a target image 108. The region image 107 may be an image of the area 109 for which object tracking is to be performed. The target image 108 may be a close-up image of the object 110 (e.g., as mentioned above: a person, pet, car, flying object, etc.) at the location to be tracked. It may be a partial image extracted from the region image 107 along with the region image 107, or it may be an image taken separately by the user. Furthermore, in embodiments with a depth estimation network (details of which will be described later), the lens used by the camera module 104 may be a general RGB lens, without the need for a more expensive RGB depth (RGBD) lens with depth measurement capabilities. The microphone module 105 can be a collection of components that have a microphone function and convert sound into a signal. It may include at least a microphone (which may be, for example, a microphone embedded inside the device or a physical microphone attached to the outside of the device) and its control circuitry. The microphone module 105 may be a directional microphone capable of recognizing the direction of sound, and the processor 102 can use the microphone module 105 to recognize specific voice commands 111 to confirm the location of the object 110 to be tracked. In the smart wake-up part, wake-up word recognition can be performed using, for example, but not limited to, the open-source speech recognition module CMU-Sphinx, which is a speech recognition module suitable for a large vocabulary and non-specific vocabulary. The transport module 106 may be a collection of elements with transport functions that cause the electronic device 101 to move, extend, and / or rotate. It may at least include mechanisms and control circuits corresponding to the aforementioned modes of activity. When the mode of activity is displacement, the transport module 106 may have a structure such as wheels, tracks, or rails. When the mode of activity is extension, the transport module 106 may have a structure such as a telescopic arm or a telescopic ladder / frame. When the mode of activity is rotation, the transport module 106 may have a structure such as a pivot, base, or turntable. Those skilled in the art will clearly understand the corresponding structures and control circuits that the transport module 106 can have based on the above-described modes of activity; therefore, they will not be described in detail here. In some embodiments, when the processor 102 recognizes a specific voice command 111 through the radio module 105 and confirms the location of the object 110 as previously described, the transport module 106 can be used to move the electronic device 101 (or at least the camera module 104 therein) so that the camera module 104 faces the location of the object 110. The transport module 106 can also be used to track the location of the object 110 in real time and follow it. Storage 103 may be used to store at least region image 107, target image 108, and object tracking network 112. Object tracking network 112 may be used to perform object tracking based on region image 107 and target image 108. Processor 102 may be used to construct and run object tracking network 112, thereby identifying the location information of object 110 represented by target image 108 in region image 107. In some embodiments, the location information may be presented as a bounding box in the image and / or as coordinate values ​​in the image. In some embodiments, storage 103 may also be used to store depth estimation network 113, and processor 102 may also be used to construct and run depth estimation network 113. Depth estimation network 113 can generally be used to estimate depth information based on region image 107. In these embodiments, processor 102 may also be used to construct and run depth estimation network 113 to obtain further details about depth beyond the position information generated by object tracking network 112, allowing electronic device 101 to make more precise judgments about the position of object 110 and / or the actual way electronic device 101 follows the movement of object 110. Referring to Figure 2, which illustrates the details of the object tracking network 112 shown in Figure 1, the architecture of the object tracking network 112 is generally divided into three parts. The first part is the feature extraction backbone network 201, which presents a Siamese network architecture. The second part is the feature fusion network 202, which is constructed by combining a cross-attention Transformer module with deep cross-correlation operations. In order to further enhance the feature representation ability of the cross-correlated response map, this invention also introduces a channel attention mechanism. The third part is the prediction network 203, which further predicts the accurate target location and size from the response map. Given that Siamese network-based and Transformer-based architectures each have their own advantages and disadvantages in visual object tracking tasks, this invention attempts to extract the advantages of both CNN-based and Transformer-based architectures. Therefore, it draws upon the basic framework of Siamese networks and the tracking method based on convolutional neural networks (CNN-Transformer-Based), i.e., TransT. More specifically, the object tracking network 112 of this invention is framed by the three main components of the TransT architecture: the feature extraction backbone network, the feature fusion network, and the prediction head module. As previously mentioned, TransT is a feature fusion network based on a Transformer architecture that fully employs attention mechanisms. It uses not only a single layer of Cross-Attention (CFA) but also a Self-Attention (ECA) mechanism, resulting in a very large number of parameters and computational overhead, making real-time inference difficult. In contrast, the architecture proposed in this invention is inspired by the Cross-Attention Transformer (CAT) architecture and introduces an efficient and accurate feature fusion network. CAT is a network architecture for one-shot object detection tasks. Given an image with a class label, the model architecture must find all objects in the input image that share the same class label. For example, given two images as input—one labeled as a horse and the image to be detected—the model can use the horse-labeled image to outline objects labeled as horses in the image to be detected, as the model's output. After repeated experiments and summarization, the inventors believed that this task idea is actually the same concept as visual object tracking, that is, searching for the most similar target object in the image to be detected from an image. Therefore, this invention refers to the Cross-Attention Transformer mechanism used in the CAT architecture to design the feature fusion network of this invention. In addition, considering the shortcomings of TransT in feature fusion network design, the object tracking network 112 of this invention also adopts a cross-correlation mechanism. Overall, the input information for the object tracking network 112 consists of two images, one of which is the target image 108 (i.e., The other image is of the area to be tracked, image 107 (i.e., First, regarding the first part mentioned above, the feature extraction backbone network 201 can respectively map the input image into feature maps with higher feature dimensions. , For example, it can be set as follows: Backbone networks are fundamental to computer vision tasks, typically used as feature extractors for input images. Common examples include AlexNet, GoogleNet, and ResNet. The feature maps generated by these networks are then fed into subsequent connected networks, mapping the input RGB feature dimensions to higher-dimensional feature maps. Most are trained on large object classification datasets, resulting in highly adaptable network model parameters for various tasks. This invention only requires minor architecture adjustments to better suit its specific task. The object tracking network 112 includes two feature extractors with shared weights, namely feature extractor 204 and feature extractor 205, which adopts a twin network-like architecture as the model feature extractor. Feature extractors 204 and 205 can be used to extract features from the region image 107 and the target image 108, respectively, thereby generating a region feature map 206 and a target feature map 207. Specifically, feature extractors 204 and 205 can be, for example, but not limited to, two ResNet50 backbone networks with shared weights, to process the two input images in parallel. ResNet50 is a variant of a residual network used to address the degradation problem in deep network training. Feature extractors 204 and 205 can extract the input information (i.e., the region image 107). ) and target image 108 ( The two images are mapped and transformed into two feature maps with higher feature dimensions, i.e. and To enhance the target's ability to represent features on the feature map, its equation can be expressed as: . Next, regarding the second part mentioned above, the feature fusion network 202 can generally include multiple cross-attention layers (cross-attention layer 208 is used as an example in Figure 2) and channel attention modules 209. The most significant (but not the only) difference between this invention and TransT lies in the architectural details of the feature fusion network 202. The feature fusion network is one of the key components of the tracking framework proposed in this invention and a major design focus for visual target tracking tasks. The main function of the feature fusion network is to effectively combine the feature information of the target image and the image of the region to be tracked, thereby improving the accuracy of target tracking. Overall, the feature fusion network can integrate features from different levels, including low-level target contour and shape information and high-level semantic information. This multi-level feature fusion enables the model to more comprehensively understand the various attributes of the target object, thereby significantly enhancing the target tracking ability and improving the robustness of the target. Therefore, in the face of challenges such as changes in target appearance, lighting, and occlusion, the fusion of multiple features can provide a more stable and reliable tracking foundation, enabling the model to better adapt to these changes and maintain stable tracking performance. As mentioned earlier, the feature fusion network 202 can have a basic Transformer architecture. However, since the input to the Transformer architecture is a one-dimensional sequence, and the feature map after previous feature extraction has too many information dimensions, the cross-attention mechanism of the feature fusion network 202 will face a huge computational burden. Therefore, before the feature fusion network 202, the processor 102 can first use a convolutional kernel of a specific size (e.g., 1×1) to compress the number of feature channels (e.g., 1024 features) (e.g., compressed to 256), and flatten the feature map in the HW space direction, thereby obtaining the dimensionality-reduced target feature map ( ) and regional feature map ( ), and input the cross-attention layer. Referring next to Figure 3, which uses the cross-attention layer 208 in Figure 2 as an example, the relevant details of the cross-attention layer are illustrated. This invention introduces a cross-attention transducer module (CATM) into the Transformer architecture. The cross-attention layer 208 may contain two parallel cross-attention transducer modules, namely cross-attention transducer module 210 and cross-attention transducer module 211. In the feature fusion network 202, both the region feature map 206 and the target feature map 207 are input to cross-attention transducer module 210 and cross-attention transducer module 211. As shown in Figure 3, each cross-attention converter module adopts a dual-stream parallel processing architecture to perform interactive feature search on the bidirectional correspondence between the target feature map 207 and the region feature map 206 previously extracted by the backbone network. This allows the two feature maps to exchange information, thereby enhancing their ability to represent similar features. In Figures 2 and 3, "q", "k", and "v" represent the feature parameters in the two feature maps. Two parallel cross-attention converter modules (e.g., cross-attention converter module 210 and cross-attention converter module 211 shown in Figure 3) can simultaneously focus on extracting useful features with high similarity between two feature vectors and can form a cross-attention layer. The cross-attention layer can be chained multiple times, for example, but not limited to, four times. This is based on the setting in the TransT model, i.e., repeating the Transformer architecture four times. The two parallel cross-attention converter modules in each cross-attention layer can perform multi-head cross-attention processing, addition and normalization, feedforward neural network, etc., on each feature parameter in the two feature maps, and finally generate the processed region feature map 212 and target feature map 213. Next, the processor 102 performs a deep cross-correlation operation on the target feature map 213 and the region feature map 212, which have been refined through the plurality of cross-attention layers, in the feature fusion network 202, thereby generating a response map 214. More specifically, the deep cross-correlation operation can be represented as using the target feature map 213 ( Each channel of the region feature map 212 is a convolutional kernel. Cross-correlation is performed on each corresponding channel of ), while " "", " "" and "" respectively represent the target template feature map and the feature map of the region to be searched after refinement by the cross-attention mechanism. The equation can be expressed as: . In the above equations, "*" represents a deep cross-correlation operation, and "R" represents the response map after deep cross-correlation. Since the subsequent prediction network predicts the precise target location and size based on the target response map, channels with high response are crucial for accurate prediction. The channel attention mechanism introduced in this invention aims to enhance channel attention in the response map, thereby improving prediction accuracy. The response map 214 can be provided to the channel attention module 209, which can generate a channel attention vector based on the response map 214 and perform element-wise multiplication of the channel attention vector with the response map 214 to weight the channel attention mechanism onto the original feature map, thereby generating a weighted response map. For a more detailed explanation, please refer to Figure 4, which illustrates the details of the channel attention module 209 in Figure 2. The channel attention module 209 can be divided into two branches to perform max-pooling and average-pooling operations on the input response map 214, respectively. These two operations generate two different feature space augmentation vectors, such as feature space augmentation vector 215 and feature space augmentation vector 216 shown in Figure 4, and these two vectors are then fed into a multi-layer perceptron (MLP). The MLP may contain a hidden layer and a sigmoid function. Finally, the channel attention module 209 integrates the two augmented feature vectors element-wise by adding them together to obtain the final result of the channel attention module 209, namely the channel attention vector 217. Next, in the feature fusion network 202, the processor 102 can perform element-wise multiplication between the channel attention vector 217 and the response map 214 to weight the channel attention mechanism onto the original feature map, thereby obtaining the weighted response map 218. Through this mechanism, more important channel features will receive attention. The equation for the mechanism can be expressed as: In the above equation, For the input feature map, The feature map is enhanced by the channel attention mechanism. This is an enhancement vector for the feature space. Regarding the third part mentioned above, please continue to refer to Figure 2. The prediction head module 203 is the final sub-network of the target tracking network architecture proposed in this invention, which consists of two branches: a classification branch 219 and a regression branch 220. Overall, the prediction head module 203 can be used to generate the final position information based on the response map 218 output by the feature fusion network 202. More specifically, each branch of the prediction network 203 can be composed of three multilayer perceptron networks (MLPs), with a hidden layer dimension of d and a ReLU activation function. To predict the final position, the classification branch 219 performs more precise foreground and background target classification on the response map 218 fused by the feature fusion network, generating a classification vector 221 of H^'W^'×2. The regression branch 220 performs more detailed regression prediction on the target center point position and size values ​​on the response map 218, generating a regression vector 222 of H^'W^'×4, where "H^'" represents the H of the target response map, "W^'" represents the W of the target response map, "2" represents the foreground and background classification labels, and "4" represents the four position and size values ​​of the target "(x, y, w, h)". Finally, the processor 102 obtains the tracking result 223 of the object tracking network 112 based on the predicted values. As previously described, in some embodiments, the storage 103 may also be used to store the depth estimation network 113, and the processor 102 may also be used to construct and run the depth estimation network 113. Referring to FIG5, details of the combination of the depth estimation network 113 and the object tracking network 112 are illustrated. The depth estimation network 113 may include an encoder 501, a non-local block (NLB) 502, and a decoder 503. The input to the depth estimation network 113 may also be the region image 107, i.e., the continuous frame image for object tracking. The object tracking network 112 and the depth estimation network 113 process the input image in parallel, ultimately generating two outputs: a target tracking result 223 and a depth map 504. In some embodiments, the processor 102 may control the movement direction of the mobile transport module 106 (e.g., a work vehicle) based on the tracking result 223, and generate control signals to control the transport module 106 to move forward or stop based on the depth map 504. This invention overcomes the bottleneck of limited local features by introducing non-local blocks into depth estimation, introducing global correlation information into the model and solving the problem of local features in traditional convolutional neural networks. This problem mainly stems from the limitation of local receptive fields in CNNs during feature extraction, as the features of a local block may not provide sufficient information for determining the true depth of that block. For example, when visually similar but spatially separated objects exist in an image, the correct depth cannot be reliably determined solely from local features. NLB, however, can globally weight and fuse features from any location in the image, providing complete global cues. Furthermore, this invention incorporates a squeeze-and-excitation (SE) module into the decoder 503 of the model to dynamically weight the importance of certain feature channels. This invention uses "Mobilenet-SSDv2" as the person detection network of the system, which is typically deployed on devices with limited hardware resources, such as mobile phones, and has high accuracy performance. For the person tracking network, this invention uses the aforementioned visual target tracking network. To improve the model's real-time tracking capability with minimal loss in accuracy performance, this invention adopts "SiamCATR-MobilenetV2," which has a highly efficient and lightweight backbone network. For the depth estimation network, this invention introduces an encoder-decoder architecture with a Non-Local Decoder-Squeeze Excitation (NL-DSE) module. The second embodiment of the present invention is an object tracking method. Referring to FIG6, an object tracking method 6 according to one or more embodiments of the present invention is illustrated. The object tracking method 6 can be executed by an electronic device (e.g., but not limited to the electronic device 101 described above) and may include steps 601 and 602. Step 601 is: constructing an object tracking network, the object tracking network including a feature extraction backbone network, a feature fusion network and a prediction network, wherein: the feature extraction backbone network includes two feature extractors with shared weights, the two feature extractors being used to extract features from a region image and a target image respectively, thereby generating a target feature map and a region feature map; the feature fusion network includes a plurality of cross-attention layers and a channel attention module, wherein each cross-attention layer includes two parallel cross-attention converter modules, and in the feature fusion network In this process, both the target feature map and the region feature map are input into the two cross-attention converter modules. The feature fusion network is used to: perform a deep cross-correlation operation on the target feature map and the region feature map after refinement by the plurality of cross-attention layers, thereby generating a response map; generate a channel attention vector based on the response map through the channel attention module; perform an element-wise multiplication operation on the channel attention vector and the response map to generate a weighted response map; and the prediction network is used to generate position information based on the weighted response map. Step 602 is to run the object tracking network to confirm the position information of the object represented by the target image in the region image. In some embodiments, the object tracking method 6 may further include the step of constructing and running a depth estimation network to generate depth information based on the target image. Furthermore, in some embodiments, the depth estimation network may include an encoder, a non-local block, and a decoder, and the decoder includes a compression-excitation module. Additionally, in some embodiments, the object tracking method 6 may further include the step of moving to follow the object represented by the target image, wherein the electronic device determines the direction of movement based on the position information and determines the depth of movement based on the depth information. In some embodiments, the object tracking method 6 may further include the following steps: continuously receiving an audio signal; identifying a preset voice command in the audio signal; after identifying the voice command, moving the electronic device so that a camera module faces a source of the voice command; and taking a picture of the source, thereby generating an image of the area and an image of the target. In some embodiments, regarding object tracking method 6, the feature fusion network may use each channel of the target feature map refined by the plurality of cross-attention layers as a convolution kernel, and perform cross-correlation operation on each corresponding channel of the region feature map refined by the plurality of cross-attention layers, thereby generating the response map after deep cross-correlation. In some embodiments, regarding object tracking method 6, the feature fusion network may perform max pooling and average pooling operations on the response map through the channel attention module, thereby generating two different feature space enhancement vectors, and then feed the two vectors into a multilayer perceptron network for enhancement. The two enhanced feature space enhancement vectors are then added element-wise to generate the channel attention vector. In some embodiments, regarding object tracking method 6, the multilayer perceptron network may include a hidden layer and a sigmoid function. In some embodiments, regarding object tracking method 6, the prediction network may include a classification branch and a regression branch, each composed of three multilayer perceptron networks. The classification branch classifies the target response map fused by the feature fusion network into foreground and background targets and generates a classification vector. The regression branch predicts the target center point position and size and generates a regression vector containing that position information. Each embodiment of the object tracking method 6 corresponds essentially to one embodiment of the electronic device 101. Therefore, those skilled in the art to which this invention pertains can fully understand and implement all corresponding embodiments of the object tracking method 6 based solely on the above description of the electronic device 101, even if not every embodiment of the object tracking method 6 has been described in detail above. The third embodiment of the present invention is a computer program product that stores a program for object tracking. When an electronic computing device (e.g., a computer) loads and executes the program, the object tracking method 6 described above for the second embodiment can be executed through a plurality of program instructions contained in the computer program product. The computer program product can be stored in a non-transitory tangible machine-readable medium, such as, but not limited to, a read-only memory (ROM), a flash memory, a magnetic disk, a portable hard drive, a magnetic tape, a networkable database, or any other storage medium known to those skilled in the art and having the same function. In summary, this invention proposes a novel, efficient, and accurate Transformer-based tracking framework. Considering speed, robustness, and accuracy in practical applications, this invention designs a concise tracking network, combining a lightweight backbone variant and a simple feature fusion mechanism to achieve a good balance between speed and accuracy. To enable object tracking using only RGB imagery, this invention combines a monocular depth estimation network with the tracking network. Experimental results show that the proposed tracker achieves the best tracking results in multiple benchmark tests, outperforming the second-best tracker by more than 5.9%. Furthermore, the tracker achieves real-time tracking on embedded systems. Finally, the object tracking network achieves 97.9% accuracy on this invention's custom dataset. The above embodiments are merely illustrative of some implementations of the present invention and to explain the technical features of the present invention, and are not intended to limit the scope and range of protection of the present invention. Any changes or equivalent arrangements that can be easily made by those skilled in the art to which this invention pertains are within the scope of the present invention, and the scope of protection of the present invention is determined by the claims. 101: Electronic device; 102: Processor; 103: Storage; 104: Camera module; 105: Radio module; 106: Transportation module; 107: Region image; 108: Target image; 109: Field; 110: Object; 111: Voice command; 112: Object tracking network; 113: Depth estimation network; 201: Feature extraction backbone network; 202: Feature fusion network; 203: Prediction network; 204, 205: Feature extractor; 206, 212: Region feature map; 207, 213: Target. Feature Map 208: Cross-Attention Layer 209: Channel Attention Module 210, 211: Cross-Attention Converter Module 214, 218: Response Map 215, 216: Feature Space Augmentation Vector 217: Channel Attention Vector 219: Classification Branch 220: Regression Branch 221: Classification Vector 222: Regression Vector 223: Tracking Result 501: Encoder 502: Non-Local Block 503: Decoder 504: Depth Map 6: Object Tracking Method 601, 602: Steps Figure 1 illustrates an electronic device for object tracking according to one or more embodiments of the present invention. Figure 2 illustrates the overall architecture of an object tracking network according to one or more embodiments of the present invention. Figure 3 illustrates how the cross-attention layer works in the object tracking network shown in Figure 2. Figure 4 illustrates how the channel attention module operates in the object tracking network shown in Figure 2. Figure 5 illustrates the overall architecture of an object tracking network and a depth estimation network according to one or more embodiments of the present invention. Figure 6 illustrates an object tracking method according to one or more embodiments of the present invention. The contents shown in Figures 1 to 6 are merely examples illustrating embodiments of the present invention and are not intended to limit the scope of protection of the present invention. 107: Regional Imagery 108: Target Image 112: Item Tracking Network 113: Depth Estimation Network 201: Feature Extraction Backbone Network 202: Feature Fusion Network 203: Predictive Networks 204, 205: Feature Extractor 206,212: Regional Feature Map 207,213: Target Feature Map 208: Cross-attention layer 209: Channel Attention Module 210, 211: Cross-attention converter module 214, 218: Response Diagram 217: Channel Attention Vector 219: Classification Branches 220: Regression Branch 221: Classification Vector 222: Regression Vector 223: Tracking Results

Claims

1. An electronic device for object tracking includes: a storage device for storing a region image, a target image, and an object tracking network; and a processor electrically connected to the storage device, the processor being configured to construct and run the object tracking network, thereby identifying location information of an object represented by the target image in the region image, the object tracking network including a feature extraction backbone network, a feature fusion network, and a prediction network, wherein: The feature extraction backbone network includes two feature extractors with shared weights, which are used to extract features from the region image and the target image, respectively, to generate a target feature map and a region feature map. The feature fusion network includes multiple cross-attention layers and a channel attention module. Each cross-attention layer includes two parallel cross-attention converter modules. In the feature fusion network, both the target feature map and the region feature map are input to the two cross-attention converter modules. The feature fusion network is used to: perform a deep cross-correlation operation on the target feature map and the region feature map after refinement by the multiple cross-attention layers to generate a response map; generate a channel attention vector from the response map through the channel attention module; perform an element-wise multiplication operation between the channel attention vector and the response map to generate a weighted response map; and the prediction network is used to generate the location information based on the weighted response map. The electronic device as claimed in claim 1, wherein... The storage device is also used to store a depth estimation network, and the processor is also used to build and run the depth estimation network to generate depth information based on the target image. The electronic device as described in claim 2, wherein... The depth estimation network includes an encoder, a nonlocal block, and a decoder, and the decoder includes a compression-excitation module. The electronic device as claimed in claim 2 further includes a transport module electrically connected to the processor, wherein: The transport module is used to move the electronic device so that it follows the object represented by the target image; The processor is also used to determine the direction of movement of the transport module based on the location information, and to determine the depth of movement of the transport module based on the depth information. The electronic device as claimed in claim 1 further includes a radio module, a camera module, and a transport module, wherein the radio module, the camera module, and the transport module are all electrically connected to the processor, and the camera module is also electrically connected to the storage device, wherein: The radio module is used to continuously receive sound signals; the processor is also used to identify a preset voice command in the sound signal; the transport module is used to move the electronic device after the processor identifies the voice command so that the camera module faces a source of the voice command; and the camera module is used to take pictures of the source, thereby generating an image of the area and an image of the target. The electronic device as claimed in claim 1, wherein... The feature fusion network uses each channel of the target feature map refined by the multiple cross-attention layers as a convolution kernel, and performs cross-correlation operation on each corresponding channel of the region feature map refined by the multiple cross-attention layers, thereby generating the response map after deep cross-correlation. The electronic device as claimed in claim 1, wherein... The feature fusion network uses the channel attention module to perform max pooling and average pooling operations on the response map to generate two different feature space enhancement vectors. These two vectors are then fed into a multilayer perceptron network for enhancement. Finally, the two enhanced feature space enhancement vectors are added element-wise to generate the channel attention vector. The electronic device as described in claim 7, wherein... This multilayer perceptron network contains a hidden layer and a sigmoid function. The electronic device as claimed in claim 1, wherein... The prediction network includes a classification branch and a regression branch, both of which are composed of three multilayer perceptron networks. Specifically: the classification branch classifies the target response map fused by the feature fusion network into foreground and background targets and generates a classification vector; and the regression branch is used to predict the position and size of the target center point and generate a regression vector containing the position information. An object tracking method, performed by an electronic device, includes the following steps: constructing an object tracking network, the object tracking network including a feature extraction backbone network, a feature fusion network, and a prediction network, wherein: The feature extraction backbone network includes two feature extractors with shared weights, which are used to extract features from a region image and a target image, respectively, thereby generating a target feature map and a region feature map. The feature fusion network includes multiple cross-attention layers and a channel attention module. Each cross-attention layer includes two parallel cross-attention converter modules. In the feature fusion network, both the target feature map and the region feature map are input to the two cross-attention converter modules. The feature fusion network is used to: perform a deep cross-correlation operation on the target feature map and the region feature map after refinement by the multiple cross-attention layers, thereby generating a response map. The channel attention module generates a channel attention vector based on the response map; and performs an element-wise multiplication operation between the channel attention vector and the response map to generate a weighted response map; the prediction network generates location information based on the weighted response map; and the object tracking network is run to confirm the location information of an object represented by the target image in the regional image. The object tracking method as described in claim 10 further includes the following steps: constructing and running a depth estimation network to generate depth information based on the target image. The object tracking method as described in claim 11, wherein... The depth estimation network includes an encoder, a nonlocal block, and a decoder, and the decoder includes a compression-excitation module. The object tracking method as described in claim 11 further includes the following steps: moving to follow the object represented by the target image, wherein the electronic device determines the direction of movement based on the position information and determines the depth of movement based on the depth information. The object tracking method as described in claim 10 further includes the following steps: continuously receiving an audio signal; identifying a preset voice command in the audio signal; after identifying the voice command, moving the electronic device so that a camera module faces a source of the voice command; and taking a picture of the source, thereby generating an image of the area and an image of the target. The object tracking method as described in claim 10, wherein... The feature fusion network uses each channel of the target feature map refined by the multiple cross-attention layers as a convolution kernel, and performs cross-correlation operation on each corresponding channel of the region feature map refined by the multiple cross-attention layers, thereby generating the response map after deep cross-correlation. The object tracking method as described in claim 10, wherein... The feature fusion network uses the channel attention module to perform max pooling and average pooling operations on the response map to generate two different feature space enhancement vectors. These two vectors are then fed into a multilayer perceptron network for enhancement. Finally, the two enhanced feature space enhancement vectors are added element-wise to generate the channel attention vector. The object tracking method as described in claim 16, wherein... This multilayer perceptron network contains a hidden layer and a sigmoid function. The object tracking method as described in claim 10, wherein... The prediction network includes a classification branch and a regression branch, both of which are composed of three multilayer perceptron networks. Specifically: the classification branch classifies the target response map fused by the feature fusion network into foreground and background targets and generates a classification vector; and the regression branch is used to predict the position and size of the target center point and generate a regression vector containing the position information. A computer program product containing a program for object tracking, which, when loaded and executed by a computer, performs the object tracking method as described in any of claims 10-18.

Images