Tracking method and device of target object, medium and equipment
By adopting a backbone network with mixed linear attention in three-dimensional target object tracking, the problem of high computing complexity in the prior art is solved, and a more efficient and real-time tracking effect is achieved.
Patent Information
- Application Number
- CN202411863549.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-04-11
Smart Images
Figure CN119810147A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to technical fields such as intelligent driving and robots, and in particular to a method, device, medium and equipment for tracking a target object. Background Art
[0002] Three-dimensional (3D) object tracking based on LiDAR plays a key role in a variety of applications such as autonomous driving and robotics. Tracking the target object quickly and accurately can effectively empower downstream planning and control tasks. In related technologies, three-dimensional target tracking is usually achieved based on a single-stream tracking framework based on Transformer. This type of framework is based on the self-attention mechanism, and the computational complexity is the second power of the input point cloud sequence length (or simply the input sequence length). The input point cloud sequence length includes the template point cloud sequence length of the target object and the point cloud sequence length of the point cloud data to be searched. This tracking method, which grows quadratically with the input sequence length, has high computational complexity and low computational efficiency, which in turn affects the running speed of the tracking algorithm, resulting in low real-time performance of the tracking task. Summary of the invention
[0003] The embodiments of the present disclosure provide a method, apparatus, medium and device for tracking a target object, which can reduce computational complexity and improve tracking efficiency and real-time performance.
[0004] A first aspect of an embodiment of the present disclosure provides a method for tracking a target object, including: determining current point cloud data to be searched and template point cloud data of the target object to be tracked; determining a first template feature corresponding to the template point cloud data at a preset perspective, and a first search feature corresponding to the current point cloud data at the preset perspective; based on the first template feature and the first search feature, determining a second template feature and a second search feature through a backbone network based on hybrid linear attention; the second search feature is a feature after the first template feature interacts with the first search feature; based on the second template feature and the second search feature, determining a tracking result of the target object.
[0005] According to a second aspect of an embodiment of the present disclosure, a device for tracking a target object is provided, comprising: a determination module for determining current point cloud data to be searched and template point cloud data of a target object to be tracked; a first processing module for determining a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle; a second processing module for determining a second template feature and a second search feature based on the first template feature and the first search feature through a backbone network based on hybrid linear attention; the second search feature is a feature obtained after the first template feature interacts with the first search feature; and a third processing module for determining a tracking result of the target object based on the second template feature and the second search feature.
[0006] According to a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the target object tracking method described in any of the above embodiments of the present disclosure.
[0007] According to a fourth aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing executable instructions of the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the target object tracking method described in any of the above embodiments of the present disclosure; or, the electronic device comprises the target object tracking device described in any of the above embodiments.
[0008] A fifth aspect of the embodiments of the present disclosure provides a computer program product. When instructions in the computer program product are executed by a processor, the target object tracking method provided by any of the above embodiments of the present disclosure is executed.
[0009] Based on the target object tracking method, device, medium and equipment provided by the above embodiments of the present disclosure, when the first template feature and the first search feature under the preset viewing angle are obtained, the second template feature corresponding to the first template feature and the second search feature corresponding to the first search feature are determined by a backbone network based on hybrid linear attention. Since the computational complexity of linear attention is linearly related to the length of the input sequence (i.e., O(N), N represents the length of the input sequence), relative to Transformer self-attention, the embodiments of the present disclosure can reduce the computational complexity from the quadratic power of the input sequence length (O(N)). 2 )) is reduced to linear (O(N)), therefore, the complexity of tracking calculations can be effectively reduced, the calculation efficiency can be improved, and the real-time performance of the tracking task can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1This is an exemplary application scenario of the target object tracking method provided by the present disclosure;
[0011] Figure 2 is a flowchart of a method for tracking a target object provided by an exemplary embodiment of the present disclosure;
[0012] Figure 3 is a flowchart of a target object tracking method provided by another exemplary embodiment of the present disclosure;
[0013] Figure 4 is a flowchart of a target object tracking method provided by yet another exemplary embodiment of the present disclosure;
[0014] Figure 5 is a flowchart of a target object tracking method provided by another exemplary embodiment of the present disclosure;
[0015] Figure 6 is a schematic diagram of the network structure of a single-flow tracking model provided by an exemplary embodiment of the present disclosure;
[0016] Figure 7 is a structural schematic diagram of a column feature encoder provided by an exemplary embodiment of the present disclosure;
[0017] Figure 8 is a schematic diagram of the structure of a hybrid linear attention network layer provided by an exemplary embodiment of the present disclosure;
[0018] Fig. 9 is a schematic diagram of the structure of a hybrid linear attention layer provided by an exemplary embodiment of the present disclosure;
[0019] Fig.10 is a schematic diagram of a gating mechanism provided by an exemplary embodiment of the present disclosure;
[0020] Fig.11 is a schematic diagram of a linear attention mechanism provided by an exemplary embodiment of the present disclosure;
[0021] Fig.12 It is a schematic diagram of the principle of the self-attention mechanism of the tracking algorithm in the related art;
[0022] Fig.13 It is a schematic diagram of the structure of a single-stream tracking framework in the related art;
[0023] Fig.14 is a schematic structural diagram of a target object tracking device provided by an exemplary embodiment of the present disclosure;
[0024] Fig.15 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0025] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.
[0026] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.
[0027] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
[0028] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
[0029] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
[0030] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
[0031] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0032] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0033] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0034] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0035] SUMMARY OF THE DISCLOSURE
[0036] In the process of implementing the present disclosure, the inventors found that in the related technologies of three-dimensional (3D) object tracking, a single-stream tracking framework based on a Transformer is usually used to implement 3D target tracking. This type of framework is implemented based on a self-attention mechanism, and the computational complexity is the quadratic power of the input point cloud sequence length (or simply the input sequence length) (which can be expressed as O(N 2 ), N represents the input sequence length), the input point cloud sequence length includes, for example, the template point cloud sequence length of the target object and the point cloud sequence length of the point cloud data to be searched. This tracking method, which grows quadratically with the input sequence length, has high computational complexity and low computational efficiency, which in turn affects the running speed of the tracking algorithm and results in low real-time performance of the tracking task.
[0037] Exemplary Overview
[0038] Figure 1 This is an exemplary application scenario of the target object tracking method provided by the present disclosure. Figure 1 As shown, in the intelligent driving scene, when the vehicle (self-vehicle) 11 is driving on the road, point cloud data can be collected in real time through the sensor 12 on the self-vehicle 11. The sensor 12 can include, for example, one or more of a laser radar and other sensors that can collect point cloud data (such as a binocular camera, an infrared camera, etc.). Using the target object tracking method of the embodiment of the present disclosure, the point cloud data collected at the current moment can be used as the current point cloud data to be searched, and the point cloud data of the target object (such as the target object 13 in the figure) detected at any historical moment can be used as the template point cloud data of the target object, and the first template feature corresponding to the template point cloud data at the preset perspective and the first search feature corresponding to the current point cloud data at the preset perspective are determined; based on the first template feature and the first search feature, the second template feature and the second search feature are determined through a backbone network based on mixed linear attention (Mixed Linear Attention, referred to as: MLA); the second search feature is the feature after the first template feature interacts with the first search feature; based on the second template feature and the second search feature, the tracking result of the target object is determined. The embodiment of the present disclosure determines the second template feature corresponding to the first template feature and the second search feature corresponding to the first search feature through a backbone network based on hybrid linear attention. Since the computational complexity of linear attention is linearly related to the length of the input sequence (i.e., O(N), where N represents the length of the input sequence), compared with Transformer self-attention, the embodiment of the present disclosure can reduce the computational complexity from the quadratic power of the input sequence length (O(N)). 2 )) is reduced to linear (O(N)), therefore, the complexity of tracking calculations can be effectively reduced, the calculation efficiency can be improved, and the real-time performance of the tracking task can be effectively improved.
[0039] The target object tracking method of the embodiment of the present disclosure can be applied to any scenario or field that requires target tracking, and is not limited to the above-mentioned intelligent driving scenario.
[0040] Exemplary Methods
[0041] Figure 2 is a flow chart of a target object tracking method provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as vehicle-mounted computing platforms (or vehicle-mounted terminals), robot terminals, such as Figure 2 As shown, the method of the embodiment of the present disclosure may include the following steps:
[0042] Step 210 , determining the current point cloud data to be searched and the template point cloud data of the target object to be tracked.
[0043] Among them, the current point cloud data to be searched (which can be called search area point cloud data, or simply search point cloud) is the search area point cloud data that may contain the point cloud of the target object. For example, in an intelligent driving scenario, the current point cloud data can be the point cloud data of the current moment collected in real time by the vehicle (self-vehicle) during driving. The target object to be tracked is a target object that is detected for the first time at a certain historical moment and whose point cloud at the current moment needs to be determined. For example, the current moment is represented as moment t, and the target object A is first detected at moment tm. Then, the target object A can be tracked at each moment after moment tm to determine the real-time status of the target object A. The goal of the tracking task is to locate the target in the search area using the prior information of the template (template point cloud, template point cloud data).
[0044] In some optional embodiments, the number of the target objects may be one or more. If the number of the target objects is multiple, the template point cloud data of each target object may be determined.
[0045] In some optional embodiments, the template point cloud data of each target object can be determined based on the historical point cloud data of the target object at historical moments. For example, the point cloud data of the target object detected for the first time can be determined as the template point cloud data of the target object, or the point cloud data of the target object tracked at any historical moment before the current moment can be used as the template point cloud data of the target object, for example, at the current moment, the point cloud data of the target object tracked at the previous moment can be used as the template point cloud data at the current moment. No specific limitation is given.
[0046] Step 220 , determining a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle.
[0047] The preset perspective may refer to a bird's-eye view (Bird's-Eye-View, abbreviated as BEV).
[0048] In some optional embodiments, the first template feature corresponding to the template point cloud data and the first search feature corresponding to the current point cloud data may be determined based on any implementable feature encoder. The feature encoder may be, for example, a column feature encoder or other types of encoders.
[0049] Step 230, based on the first template feature and the first search feature, determine the second template feature and the second search feature through a backbone network based on hybrid linear attention.
[0050] In some optional embodiments, the second search feature is a feature obtained by interacting the first template feature with the first search feature. That is, the feature information of the first template feature is transferred to the search feature through a hybrid linear attention mechanism, and is fused with the linear attention result of the first search feature, and the second search feature is obtained through feature scale transformation. The first template feature is transformed into the second template feature through feature scale transformation.
[0051] In some optional embodiments, the backbone network based on hybrid linear attention (referred to as the backbone network) may include a linear attention network layer and a gating network layer, the linear attention network layer performs a linear attention operation on the first search feature to obtain a linear attention result, the gating network layer processes the first template feature, the processed template feature is fused with the linear attention result to obtain a fused feature, and the second search feature is determined based on the fused feature. The second template feature is a template feature obtained by scaling (e.g., downsampling) the first template feature and matching the scale of the second search feature.
[0052] Step 240: Determine the tracking result of the target object based on the second template feature and the second search feature.
[0053] The tracking result of the target object may include a three-dimensional prediction box (or three-dimensional bounding box BBox) of the target object in the current point cloud data. The point cloud data of the target object may be extracted from the current point cloud data based on the three-dimensional prediction box.
[0054] In some optional embodiments, the second template feature and the second search feature can be processed by a prediction head network to obtain a tracking result of the target object. The head network can adopt any prediction head network structure suitable for three-dimensional target tracking scenarios, and the embodiments of the present disclosure are not limited thereto.
[0055] The target object tracking method provided in this embodiment determines the second template feature corresponding to the first template feature and the second search feature corresponding to the first search feature through a backbone network based on hybrid linear attention. Since the computational complexity of linear attention is linearly related to the length of the input sequence (i.e., O(N), where N represents the length of the input sequence), compared with Transformer self-attention, the computational complexity of the disclosed embodiment can be reduced from the quadratic power of the input sequence length (O(N)). 2 )) is reduced to linear (O(N)), therefore, the complexity of tracking calculations can be effectively reduced, the calculation efficiency can be improved, and the real-time performance of the tracking task can be effectively improved.
[0056] Figure 3 It is a flowchart of a target object tracking method provided by another exemplary embodiment of the present disclosure.
[0057] In some optional embodiments, in the above Figure 2 Based on the embodiment shown, Figure 3 As shown, the step 210 of determining the current point cloud data to be searched and the template point cloud data of the target object to be tracked may include:
[0058] Step 2110, determining the point cloud data collected at the current moment as the current point cloud data.
[0059] The current moment may be the real time moment of a mobile device such as a vehicle or a robot during movement. The point cloud data collected at the current moment may be the point cloud data collected at the current moment by sensors such as a laser radar, a binocular camera, and an infrared camera on the mobile device.
[0060] Step 2120 , determining template point cloud data based on the tracking results of the target object at the historical moment.
[0061] Among them, the historical moment is the moment before the current moment.
[0062] In some optional embodiments, the template point cloud data may be determined based on the tracking result of the target object at any historical moment. If the tracking result of the target object at the historical moment includes a three-dimensional frame of the target object, the point cloud data of the target object may be extracted from the point cloud data at the historical moment based on the three-dimensional frame as the template point cloud data of the target object.
[0063] In some optional embodiments, the template point cloud data of the target object can be determined based on the tracking result of the most recent historical moment in which the target object was tracked before the current moment, so as to improve the validity of the template prior information of the target object and thereby improve the accuracy of the tracking result.
[0064] It should be noted that step 2110 and step 2120 may be performed in any order.
[0065] The embodiments of the present disclosure determine template point cloud data based on the tracking results of the target object at historical moments and realize dynamic updating of the template point cloud. This can avoid the problem of insufficient utilization of the target object template prior due to incomplete point cloud of the target object detected for the first time, and help improve the effectiveness of the target object template prior information, thereby improving the accuracy of the tracking results.
[0066] Figure 4 It is a flowchart of a method for tracking a target object provided by yet another exemplary embodiment of the present disclosure.
[0067] In some optional embodiments, based on any of the above embodiments, Figure 4 As shown, step 220 of determining the first template feature corresponding to the template point cloud data at the preset viewing angle, and the first search feature corresponding to the current point cloud data at the preset viewing angle, includes:
[0068] Step 2210, encoding the template point cloud data and the current point cloud data respectively based on the column feature encoder to obtain a first template feature and a first search feature under a preset viewing angle.
[0069] Among them, the Pillar Feature Encoder can map the input point cloud data (such as template point cloud data, current point cloud data, which can be represented as a point cloud input tensor) into a pseudo image of a bird's-eye view (i.e., a preset perspective) to prepare for subsequent feature extraction.
[0070] The embodiments of the present disclosure encode the template point cloud data and the current point cloud data into regular BEV features through a column feature encoder, thereby realizing histogram-based pillar feature encoding, which can reduce or avoid the loss of height information and reduce the computational cost of the pillar encoding process, thereby providing an effective feature reference for subsequent feature extraction.
[0071] In some optional embodiments, the column feature encoder can be encapsulated as a plug-in module so that it can be easily inserted into any other application scenario based on pillar feature coding, seamlessly improve the task performance of the corresponding application scenario, and effectively avoid the problem of serious performance degradation caused by quantization of the pillar coding algorithm.
[0072] In some optional embodiments, encoding the template point cloud data and the current point cloud data based on the column feature encoder in step 2210 to obtain the first template feature and the first search feature under the preset viewing angle may include:
[0073] Based on a multi-layer perceptron, the template point cloud data and the current point cloud data are processed respectively to obtain intermediate template features and intermediate search features; the intermediate template features and the intermediate search features are pooled and diverged respectively to obtain first template features and first search features.
[0074] Among them, the Multilayer Perceptron (MLP) can achieve effective encoding and representation of template point cloud data and current point cloud data based on the updated network parameters after learning, and obtain the template features of the intermediate representation (intermediate template features for short) and the search features of the intermediate representation (intermediate search features for short). By performing pooling operations on the intermediate template features and the intermediate search features, the feature data blockage can be reduced and the subsequent calculation efficiency can be improved. By performing divergent operations on the pooled features, the pooled features can be converted into BEV features, and the first template features and the first search features from the BEV perspective can be obtained, providing an effective feature reference for subsequent feature extraction.
[0075] Figure 5 It is a flowchart of a target object tracking method provided by yet another exemplary embodiment of the present disclosure.
[0076] In some optional embodiments, based on any of the above embodiments, Figure 5 As shown, the hybrid linear attention based backbone network includes at least one level of hybrid linear attention sub-network.
[0077] Among them, the number of levels of the hybrid linear attention sub-network can be set according to actual needs, for example, it can be set to 1 level, 2 levels, 3 levels, 4 levels, etc., without specific limitation.
[0078] Determining the second template feature and the second search feature based on the first template feature and the first search feature through a hybrid linear attention based backbone network in step 230 may include:
[0079] Step 2310, for the i-th level hybrid linear attention subnetwork in the at least one level hybrid linear attention subnetwork, the i-th level template feature and the i-th level search feature are processed by the i-th level hybrid linear attention subnetwork to obtain the i+1-th level template feature and the i+1-th level search feature; the n+1-th level template feature is used as the second template feature; and the n+1-th level search feature is used as the second search feature;
[0080] Wherein, i=1, 2, ..., n, n is the total number of levels of at least one hybrid linear attention subnetwork, and n is a positive integer; the input feature scales of different levels of hybrid linear attention subnetworks are different; the first-level template feature is the first template feature, and the first-level search feature is the first search feature; the second template feature is a template feature of a different scale from the first template feature. That is, the second template feature is a template feature after the first template feature is scaled, and the scale transformation method may include downsampling, convolution, etc., which is not specifically limited. Exemplarily, the scale of the first template feature is expressed as H×W×C, and the scale of the second template feature may be H / 2×W / 2×2C or other scales, which are not specifically limited.
[0081] In some optional embodiments, each level of hybrid linear attention subnetwork may include a hybrid linear attention network layer and a downsampling layer. The downsampling layer is used to downsample the output features of the hybrid linear attention layer, change the feature scale, and provide the downsampled features to the next network layer, which may be, for example, the next level of hybrid linear attention subnetwork or prediction head network. For example, the output features of the downsampling layer of the last level of hybrid linear attention subnetwork are provided to the prediction head network.
[0082] In some optional embodiments, the hybrid linear attention subnetwork may include a position embedding layer, a layer normalization layer, a hybrid linear attention layer, a residual connection, a multi-layer perceptron, a downsampling layer, and the like.
[0083] The embodiments of the present disclosure implement deep extraction of template features and search features through at least one level of hybrid linear attention sub-network, and in the feature extraction process, the template information is passed to the search feature and fused with the search feature to obtain a second search feature. The obtained second template feature is a template feature of a different scale from the first template feature, that is, the search feature is not passed to the template feature, thereby avoiding the template information being affected by the search feature and introducing noise, and realizing asymmetric attention operation, thereby ensuring high-quality template priors and helping to improve the accuracy of tracking results.
[0084] In some optional embodiments, the i-th level template feature and the i-th level search feature are processed by the i-th level hybrid linear attention sub-network in step 2310 to obtain the i+1-th level template feature and the i+1-th level search feature, which may include:
[0085] Based on the i-th level search feature, determine the first normalized feature; based on the first normalized feature and the i-th level template feature, determine the mixed linear attention result through the mixed linear attention layer; based on the mixed linear attention result, determine the third search feature; downsample the i-th level template feature and the third search feature respectively to obtain the i+1-th level template feature and the i+1-th level search feature.
[0086] In some optional embodiments, the first normalized feature can be determined by mixing the position embedding layer and the layer normalization layer in the linear attention subnetwork.
[0087] In some optional embodiments, after obtaining the mixed linear attention result, the third search feature can be determined through a first residual connection layer, a layer normalization layer, a multi-layer perceptron, and a second residual connection layer.
[0088] In the disclosed embodiment, in each level of the hybrid linear attention sub-network, a hybrid linear attention operation is performed on the search features and template features of the current level, the template features are transferred to the search features, and the search features of the next level are obtained by downsampling, and the template features of the current level are downsampled to obtain the template features of the next level, thereby realizing asymmetric attention interaction, that is, only the template information is transferred to the search area, and the search features are avoided from being transferred to the template features, thereby avoiding the search features from introducing noise into the template features, and ensuring that the template features can maintain a high quality.
[0089] In some optional embodiments, determining a hybrid linear attention result based on the first normalized feature and the i-th level template feature through a hybrid linear attention layer may include:
[0090] Perform a linear attention operation on the first normalized feature to obtain a first attention result; perform a linear projection on the i-th level template feature to obtain a first projection result; determine the third template feature based on the first projection result and the first activation function; determine a mixed linear attention result based on the first attention result and the third template feature.
[0091] Among them, the first normalized feature is the search feature. During the attention operation, only the search feature is interacted with, and it is a linear attention interaction, which can effectively reduce the computational complexity. For the template feature, the gating mechanism is used to fuse the attention result of the search feature (i.e., the first attention result), and the template information is transferred to the search feature, effectively realizing asymmetric attention and avoiding the introduction of noise information of the search feature into the template feature, thereby avoiding adverse interference with the template prior information.
[0092] In some optional embodiments, a Hadamard product (ie, multiplication of corresponding elements) operation may be performed on the first attention result and the third template feature to integrate the third template feature into the search feature.
[0093] In some optional embodiments, performing a linear attention operation on the first normalized feature to obtain a first attention result may include:
[0094] The first normalized feature is linearly mapped through the first linear layer to obtain a first processing result; the first processing result is convolved through the convolution layer to obtain a second processing result; the second processing result is linearly mapped through the second linear layer to obtain a third processing result; the second processing result is linearly mapped through the third linear layer to obtain a fourth processing result; the second processing result, the third processing result and the fourth processing result are linearly attended to through the linear attention layer to obtain a first attention result.
[0095] Among them, the linear attention layer performs linear attention operations on the second processing result, the third processing result, and the fourth processing result based on the linear attention mechanism to obtain the first attention result. For example, the third processing result is used as the query tensor Q, the fourth processing result is used as the key tensor K, and the second processing result is used as the value tensor V. The first attention result O is obtained by the following formula:
[0096] O=Q·sim(K T V)
[0097] Where sim() represents the mapping function of the approximate softmax operation in the linear attention mechanism. For example, Q is an N×d query tensor, K is an N×d key tensor, V is an N×d value tensor, N represents the feature sequence length, and d represents the dimension of the feature. T V is d×d data. It can be seen that the computational complexity is O(N). Compared with the self-attention mechanism, it effectively reduces the computational complexity.
[0098] In some optional embodiments, Figure 6 FIG. 1 is a schematic diagram of the network structure of a single-flow tracking model provided by an exemplary embodiment of the present disclosure. Figure 6 As shown, the target object tracking method of the disclosed embodiment adopts an asymmetric single-stream tracking model based on hybrid linear attention to realize the tracking of the target object. The network structure of the single-stream tracking model includes a column feature encoder, a backbone network (i.e., a backbone network based on hybrid linear attention) and a prediction head network. The backbone network takes a 3-level (such as Stage 1, Stage 2 and Stage 3 in the figure) hybrid linear attention sub-network as an example. Each level of the hybrid linear attention sub-network includes a hybrid linear attention network layer and a downsampling layer. The feature scales of the hybrid linear attention sub-networks at each level are different. For example, if the feature scale of the input feature of Stage 1 in the figure is H×W×C, the output feature scale after downsampling is H / 2×W / 2×2C, the output feature scale of Stage 2 is H / 4×W / 4×4C, and the output feature scale of Stage 3 is H / 8×W / 8×8C. The scales of each level here are only exemplary settings and are not limited to the feature scales in the figure in actual applications.
[0099] It can be understood that the single-stream tracking model of the embodiment of the present disclosure can be obtained in advance through training. For example, the model training can be performed on the server side, and the trained tracking model can be deployed to the terminal device (such as a vehicle-mounted terminal device, a robot terminal device, etc.) to be used for real-time tracking of the target object on the terminal device. The specific training process will not be repeated here.
[0100] In some optional embodiments, Figure 7 Schematic diagram of the structure of a column feature encoder provided by an exemplary embodiment of the present disclosure. Figure 7 As shown, based on any of the above embodiments, the column feature encoder includes a multi-layer perceptron (MLP), a pooling and divergence (Pooling & Scatter) layer. The point cloud data can be template point cloud data or current point cloud data. N represents the sequence length of the point cloud data (that is, the number of points), and 3 represents the three-dimensional coordinates corresponding to each point. The point cloud data is subjected to MLP to obtain N×C intermediate features (such as intermediate template features, intermediate search features), and the intermediate features are subjected to pooling and divergence layers to obtain H×W×C BEV perspective output features (first template features, first search features).
[0101] In some optional embodiments, Figure 8 is a schematic diagram of the structure of a hybrid linear attention network layer provided by an exemplary embodiment of the present disclosure. Figure 8 As shown in the above Figure 6 Based on the embodiment shown, the hybrid linear attention network layer in the i-th level hybrid linear attention subnetwork may include a positional embedding layer, a first layer normalization layer, a hybrid linear attention (MLA) layer, a first residual connection The second normalization (LayerNorm) layer, multi-layer perceptron (MLP) and the second residual connection Ft represents the i-th level template feature, Fs represents the i-th level search feature, Fs is added to the position encoding feature (the elements of the corresponding position are added) to achieve position embedding, and the search feature after position embedding is obtained. The search feature after position embedding is layer-normalized through the first Layer Norm layer to obtain the first normalized feature. Ft and the first normalized feature are input to the MLA layer, and the mixed linear attention result is obtained through the MLA layer. The mixed linear attention result is added to the search feature after position embedding through the first residual connection to obtain the first addition result; the first addition result is layer-normalized through the second Layer Norm layer to obtain the second normalized feature; the second normalized feature is processed through MLP to obtain the processed feature, and the processed feature is added to the first addition result through the residual connection to obtain the third search feature. The output features in the figure include Ft and the third search feature. Then Ft and the third search feature are downsampled respectively to obtain the i+1-th level template feature and the i+1-th level search feature.
[0102] In some optional embodiments, Fig. 9 is a schematic diagram of the structure of a hybrid linear attention layer provided by an exemplary embodiment of the present disclosure. Fig. 9 As shown, in Figure 8 Based on the embodiment shown, Figure 8The mixed linear attention (MLA) layer in may include multiple linear layers (Linear), convolutional layers (Conv), linear attention (Linear Attention) layers, Hadamard multiplication (Hadamard Product) layers (⊙), and activation functions (σ). Among them, Hadamard multiplication is also called element-level multiplication, which is to multiply the elements in the same position of two matrices (or vectors). The activation function takes SiLu (Sigmoid LinearUnit) activation function as an example. In practical applications, it can be set to other activation functions according to actual needs. Fs' represents the first normalized feature. Fs' obtains the first processing result through the linear mapping of the first linear layer (Linear), and the first processing result obtains the second processing result through the convolutional layer (Conv); the second processing result obtains the third processing result through the second linear layer (Linear); the second processing result obtains the fourth processing result through the third linear layer (Linear); the third processing result, the fourth processing result and the second processing result are input to the Linear Attention layer, and the linear attention operation is performed to obtain the first attention result. Ft is passed through a linear layer (which can be called the fourth linear layer) to obtain the first projection result, and the first projection result is processed by the activation function (first activation function) σ to obtain the third template feature. The first attention result and the third template feature are subjected to Hadamard multiplication to obtain the operation result; the operation result is then processed by a linear layer to obtain a mixed linear attention result.
[0103] In some optional embodiments, Fig.10 FIG. 1 is a schematic diagram showing the principle of a gating mechanism provided by an exemplary embodiment of the present disclosure. Fig.10 As shown, Fig. 9 A simplified schematic diagram of the process. In the attention stage, only the search feature (first normalized feature Fs') is interacted with to obtain the attention result (first attention result). The template feature (Ft) is transformed using linear projection and activation function to obtain the transformed template feature (i.e., the third template feature). The information of the transformed template feature is then integrated into the search feature through the Hadamard product to obtain the integrated search feature Fst, thus realizing the post-gating asymmetric attention (Post-GAA) operation. The formula for post-gating asymmetric attention can be expressed as follows:
[0104] F st ←Attention(F s ′)·σ(L(F t ))
[0105] Wherein, L() represents linear projection, σ() represents SiLu activation function. Attention() represents attention operation, which is a linear attention operation in the disclosed embodiment. Before the attention operation, the search feature F s ′ has not yet undergone effective feature extraction, the search features after the attention operation show higher-level geometric and semantic information, thus enabling better alignment with the converted template features.
[0106] In some optional embodiments, Fig.11 It is a schematic diagram of the principle of a linear attention mechanism provided by an exemplary embodiment of the present disclosure. Fig.12 This is a schematic diagram of the principle of the self-attention mechanism of the tracking algorithm in the related art. Fig.11 and 12 As shown, Matmul represents matrix multiplication, Softmax represents the normalized exponential function, and for the query tensor Q with a data structure of N×d and the transpose K of the key tensor with a data structure of d×N T , N×d value tensor V, in the case of interaction through the self-attention mechanism (see Fig.12 ), self-attention needs to calculate the similarities between all query-key pairs, resulting in a complexity of O(N 2 ), relative to Fig.12 The word attention mechanism shown, Fig.11 The linear attention mechanism shown applies a carefully designed mapping function φ to Q and K respectively, i.e. Sim(Q,K)=φ(Q)φ(K) T , based on the associative law of matrix multiplication, the calculation order is changed from (φ(Q)φ(K) T )V to φ(Q)(φ(K) T V), thus effectively reducing the computational complexity from O(N 2 ) into O(N). By combining linear attention with Fig.10 The gating mechanism shown together constitutes a hybrid linear attention, which not only maintains the global modeling capability, but also realizes the asymmetry between the template and search area processing, effectively promoting the feature extraction of specific targets. The hybrid linear attention design combines the advantages of linear attention and self-attention, ensuring the tracking performance of the target object while improving the computational efficiency.
[0107] Fig.13 Schematic diagram of the structure of a single-stream tracking framework in the related art. Fig.13As shown in the figure, the single-stream tracking framework in the related art includes a local encoder, a template-aware Transformer module, a multi-scale feature fusion component, and a feature augmentation component. Among them, the local encoder encodes the template point cloud and the search area point cloud (i.e., the point cloud to be searched) and performs feature splicing to obtain a spliced feature sequence. The template-aware Transformer module includes M stacked Transformer encoders for extracting multi-scale coding features. For example, three different scale coding features can be extracted by three stacked Transformer encoders, which can be expressed as F 1,S 、F 2,S and F 3,S , the multi-scale feature fusion component fuses the encoding features of different scales, for example, the encoding feature F of the smaller scale 1,S After upsampling, 2,S Add them together to get the first fusion feature, which is then upsampled and combined with the larger scale F 3,S Add together to fuse the encoded features of different scales. The feature enhancement component is used to enhance the fused features of different scales for predicting tracking results. It can be seen that the single-stream tracking model of the related art performs symmetrical feature interaction based on the self-attention mechanism in the Transformer encoder, including information interaction from the template's prior information to the search area and from the search area's information to the template. In this process, the information interaction from the search area's information to the template may introduce noise from the search area to the template's prior information, thereby interfering with the template information and reducing the quality of the template's prior information.
[0108] The target object tracking method provided by the embodiment of the present disclosure adopts a hybrid linear asymmetric attention mechanism. On the basis of transferring the prior information of the template point cloud to the search area, the information of the search area can be prevented from being transferred to the template. Therefore, the template information is not affected by the search area, and then by using the template features as the modulation vector, the clues of the specific target in the search area are enhanced, thereby maintaining the high-quality template prior. In addition, the embodiment of the present disclosure adopts linear attention to interact with the search features, and combines the gating mechanism to transfer the template information to the search area to realize the asymmetric feature interaction between the template features and the search features. Compared with self-attention, the embodiment of the present disclosure reduces the computational complexity from O(N / 2) to O(N / 2). 2) is transformed into O(N), which effectively improves the model reasoning speed, thereby improving the real-time performance of the target tracking task. Compared with the related art, the tracking method of the embodiment of the present disclosure achieves efficient feature extraction and interaction based on a more simplified process. In addition, the embodiment of the present disclosure achieves an effective balance between efficiency and accuracy in tracking tasks that are extremely sensitive to delays through hybrid linear attention, thereby improving the accuracy and reasoning speed of point cloud target tracking.
[0109] There are also some dual-stream framework tracking methods in the related art, that is, the features of the template point cloud and the search area point cloud are extracted respectively through two backbone networks, and then the features are interacted through the interactive network. The tracking process of the dual-stream framework is complicated, and the dual-stream framework uses the point cloud of the target object detected in the first frame (first time) as the template point cloud. There may be incomplete point clouds. For example, only part of the target object is detected, resulting in inaccurate template information, which affects the accuracy of the tracking results. The embodiment of the present disclosure ensures the integrity and validity of the template information by dynamically updating the template point cloud, avoiding the problem of insufficient prior information of the target object in the first frame. In addition, the embodiment of the present disclosure is based on a single-stream tracking framework, which can greatly reduce the complexity of the tracking process compared to the dual-stream framework.
[0110] The above-mentioned embodiments of the present disclosure may be implemented separately or in any combination without conflict. The specific configuration may be based on actual needs and the present disclosure does not limit this.
[0111] Any target object tracking method provided by the embodiments of the present disclosure may be executed by any appropriate electronic device with data processing capabilities, including but not limited to: electronic devices such as terminal devices and servers. Alternatively, any target object tracking method provided by the embodiments of the present disclosure may be executed by a processor, such as the processor executing any target object tracking method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.
[0112] Exemplary Devices
[0113] Fig.14 FIG. 1 is a schematic diagram of a target object tracking device provided by an exemplary embodiment of the present disclosure. The device of this embodiment can be used to implement the corresponding method embodiment of the present disclosure, such as Fig.14 The device shown may include: a determination module 51 , a first processing module 52 , a second processing module 53 and a third processing module 54 .
[0114] The determination module 51 is used to determine the current point cloud data to be searched and the template point cloud data of the target object to be tracked.
[0115] The first processing module 52 is used to determine a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle.
[0116] The second processing module 53 is used to determine the second template feature and the second search feature based on the first template feature and the first search feature through a backbone network based on hybrid linear attention; the second search feature is the feature after the first template feature interacts with the first search feature.
[0117] The third processing module 54 is used to determine the tracking result of the target object based on the second template feature and the second search feature.
[0118] In some optional embodiments, based on the above embodiments, the determination module 51 may be specifically used for:
[0119] The point cloud data collected at the current moment is determined as the current point cloud data. Based on the tracking results of the target object at the historical moment, the template point cloud data is determined. The historical moment is the moment before the current moment.
[0120] In some optional embodiments, based on any of the above embodiments, the first processing module 52 can be specifically used to: encode the template point cloud data and the current point cloud data respectively based on the column feature encoder to obtain the first template feature and the first search feature under a preset viewing angle.
[0121] In some optional embodiments, the first processing module 52 can be specifically used to: based on a multi-layer perceptron, process the template point cloud data and the current point cloud data respectively to obtain intermediate template features and intermediate search features; perform pooling and divergence processing on the intermediate template features and the intermediate search features respectively to obtain first template features and first search features.
[0122] In some optional embodiments, based on any of the above embodiments, the hybrid linear attention based backbone network includes at least one level of hybrid linear attention sub-network.
[0123] The second processing module 53 can be specifically used for: for the i-th level mixed linear attention subnetwork in at least one level mixed linear attention subnetwork, processing the i-th level template feature and the i-th level search feature through the i-th level mixed linear attention subnetwork to obtain the i+1-th level template feature and the i+1-th level search feature; the n+1-th level template feature is used as the second template feature; the n+1-th level search feature is used as the second search feature; wherein, i=1, 2, ..., n, n is the total number of levels of at least one level mixed linear attention subnetwork, and n is a positive integer; the input feature scales of different levels of mixed linear attention subnetworks are different; the 1st level template feature is the first template feature, and the 1st level search feature is the first search feature; the second template feature is a template feature of a different scale from the first template feature.
[0124] In some optional embodiments, the second processing module 53 may be specifically used for:
[0125] Based on the i-th level search feature, determine the first normalized feature; based on the first normalized feature and the i-th level template feature, determine the mixed linear attention result through the mixed linear attention layer; based on the mixed linear attention result, determine the third search feature; downsample the i-th level template feature and the third search feature respectively to obtain the i+1-th level template feature and the i+1-th level search feature.
[0126] In some optional embodiments, the second processing module 53 may be specifically used for:
[0127] Perform a linear attention operation on the first normalized feature to obtain a first attention result; perform a linear projection on the i-th level template feature to obtain a first projection result; determine the third template feature based on the first projection result and the first activation function; determine a mixed linear attention result based on the first attention result and the third template feature.
[0128] In some optional embodiments, the second processing module 53 may be specifically used for:
[0129] The first normalized feature is linearly mapped through the first linear layer to obtain a first processing result; the first processing result is convolved through the convolution layer to obtain a second processing result; the second processing result is linearly mapped through the second linear layer to obtain a third processing result; the second processing result is linearly mapped through the third linear layer to obtain a fourth processing result; the second processing result, the third processing result and the fourth processing result are linearly attended to through the linear attention layer to obtain a first attention result.
[0130] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be found in the corresponding beneficial technical effects of the above exemplary method section, which will not be repeated here.
[0131] Exemplary Electronic Devices
[0132] Fig.15 It is a structural diagram of an electronic device provided by an embodiment of the present disclosure, including at least one processor 91 and a memory 92.
[0133] The processor 91 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.
[0134] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute one or more computer program instructions to implement the methods and / or other desired functions of the various embodiments of the present disclosure described above.
[0135] In one example, the electronic device 90 may further include: an input device 93 and an output device 94 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0136] The input device 93 may also include, for example, a touch screen, a microphone, various sensors, and the like. Sensors may include, for example, image sensors (such as cameras, cameras, etc.), laser radars, millimeter-wave radars, ultrasonic radars, positioning sensors, pressure sensors, air quality sensors, temperature sensors, and the like. Image sensors, laser radars, millimeter-wave radars, ultrasonic radars, and the like may be used to sense the surrounding environment, that is, to detect dynamic and static objects in the surrounding environment. Dynamic and static objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, and buildings, as well as dynamic objects such as surrounding vehicles, pedestrians, and cyclists. Positioning sensors are used to realize the positioning of movable devices (such as self-vehicles, robots, etc.) where electronic devices are located. Positioning sensors may include, for example, inertial measurement units (Inertial Measurement Unit, referred to as: IMU), global positioning systems (Global Positioning System, referred to as: GPS), and the like. Pressure sensors may be used to detect seat pressure. Temperature sensors may be used to detect the temperature in the vehicle cabin. Air quality sensors may be used to detect the air quality in the vehicle cabin.
[0137] The output device 94 can output various information to the outside, and may include, for example, a display, a speaker, a communication network and a remote output device connected thereto, and the like.
[0138] Of course, to simplify, Fig.15 Only some of the components related to the present disclosure in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 90 may also include any other appropriate components.
[0139] The embodiment of the present disclosure also provides a chip that can be applied to a mobile device such as a vehicle-mounted computing platform, a robot terminal, etc., for tracking a target object during the movement of the mobile device. The chip includes the target object tracking device provided by any of the above embodiments.
[0140] Exemplary computer program products and computer-readable storage media
[0141] In addition to the above methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the methods of various embodiments of the present disclosure described in the above “Exemplary Methods” section.
[0142] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0143] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps in the method of various embodiments of the present disclosure described in the above “Exemplary Method” section.
[0144] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0145] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.
[0146] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.
Claims
1. A method for tracking a target object, comprising: Determine the current point cloud data to be searched and the template point cloud data of the target object to be tracked; Determine a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle; Based on the first template feature and the first search feature, determining a second template feature and a second search feature through a hybrid linear attention-based backbone network; The second search feature is a feature obtained by interacting the first template feature with the first search feature; A tracking result of the target object is determined based on the second template feature and the second search feature.
2. The method according to claim 1, wherein: The hybrid linear attention-based backbone network includes at least one level of hybrid linear attention sub-network; The determining, based on the first template feature and the first search feature, a second template feature and a second search feature through a backbone network based on hybrid linear attention includes: For the i-th level hybrid linear attention subnetwork in the at least one level hybrid linear attention subnetwork, the i-th level template feature and the i-th level search feature are processed by the i-th level hybrid linear attention subnetwork to obtain the i+1-th level template feature and the i+1-th level search feature; Among them, i=1, 2, ..., n, n is the total number of levels of the at least one level hybrid linear attention subnetwork, and n is a positive integer; the input feature scales of the hybrid linear attention subnetworks at different levels are different; the first level template feature is the first template feature, and the first level search feature is the first search feature; the n+1th level template feature is used as the second template feature; the n+1th level search feature is used as the second search feature; the second template feature is a template feature of a different scale from the first template feature.
3. The method according to claim 2, wherein: The i-th level template feature and the i-th level search feature are processed by the i-th level hybrid linear attention sub-network to obtain the i+1-th level template feature and the i+1-th level search feature, including: Based on the i-th level search feature, determining a first normalized feature; Based on the first normalized feature and the i-th level template feature, determining a hybrid linear attention result through a hybrid linear attention layer; Determining a third search feature based on the hybrid linear attention result; Downsampling is performed on the i-th level template feature and the third search feature respectively to obtain the i+1-th level template feature and the i+1-th level search feature.
4. The method according to claim 3, wherein: The determining, based on the first normalized feature and the i-th level template feature, a hybrid linear attention result through a hybrid linear attention layer includes: Performing a linear attention operation on the first normalized feature to obtain a first attention result; Performing linear projection on the i-th level template feature to obtain a first projection result; Determine a third template feature based on the first projection result and the first activation function; Based on the first attention result and the third template feature, the hybrid linear attention result is determined.
5. The method according to claim 4, wherein: The performing a linear attention operation on the first normalized feature to obtain a first attention result includes: Performing linear mapping processing on the first normalized feature through a first linear layer to obtain a first processing result; Performing convolution processing on the first processing result through a convolution layer to obtain a second processing result; Performing linear mapping processing on the second processing result through a second linear layer to obtain a third processing result; Performing linear mapping processing on the second processing result through a third linear layer to obtain a fourth processing result; A linear attention operation is performed on the second processing result, the third processing result and the fourth processing result through a linear attention layer to obtain the first attention result.
6. The method according to any one of claims 1 to 5, wherein: The determining of a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle, comprises: The template point cloud data and the current point cloud data are respectively encoded based on a column feature encoder to obtain the first template feature and the first search feature under the preset viewing angle.
7. The method according to claim 6, wherein: The column-based feature encoder encodes the template point cloud data and the current point cloud data respectively to obtain the first template feature and the first search feature under the preset viewing angle, including: Based on a multi-layer perceptron, the template point cloud data and the current point cloud data are processed respectively to obtain intermediate template features and intermediate search features; The intermediate template feature and the intermediate search feature are respectively subjected to pooling and divergence processing to obtain the first template feature and the first search feature.
8. The method according to any one of claims 1 to 5, wherein: The step of determining the current point cloud data to be searched and the template point cloud data of the target object to be tracked includes: Determine the point cloud data collected at the current moment as the current point cloud data; The template point cloud data is determined based on the tracking result of the target object at a historical moment; the historical moment is a moment before the current moment.
9. A tracking device for a target object, comprising: A determination module, used to determine the current point cloud data to be searched and the template point cloud data of the target object to be tracked; A first processing module, used to determine a first template feature corresponding to the template point cloud data at a preset viewing angle, and a first search feature corresponding to the current point cloud data at the preset viewing angle; A second processing module, configured to determine a second template feature and a second search feature based on the first template feature and the first search feature through a hybrid linear attention based backbone network; The second search feature is a feature obtained by interacting the first template feature with the first search feature; The third processing module is used to determine the tracking result of the target object based on the second template feature and the second search feature.
10. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the method according to any one of claims 1 to 8.
11. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method according to any one of claims 1 to 8; or The electronic device comprises the device according to claim 9 above.
12. A chip comprising the device according to claim 9.