Real-time target tracking method and device based on world model
By employing a real-time target tracking method based on a world model and utilizing Transformer blocks and feature filtering mechanisms, the problem of balancing target tracking accuracy and efficiency on mobile devices is solved, achieving efficient target localization and tracking under extreme conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-23
- Publication Date
- 2026-03-10
AI Technical Summary
Existing deep learning-based target tracking methods struggle to balance accuracy and efficiency on mobile devices, especially when dealing with extreme shooting angles, motion blur, and occlusion. Their computational efficiency is low and they cannot meet real-time processing requirements.
A real-time target tracking method based on a world model is adopted. By inputting multiple consecutive frames of images into a pre-trained world model, target-related features are selected using Transformer blocks and feature filtering mechanisms. Combined with a multilayer perceptron for position prediction, the method achieves accurate target localization and efficient tracking.
While maintaining tracking accuracy, it significantly reduces computational consumption, improves execution efficiency on mobile devices, and preserves the deep understanding of the world model without requiring modification of the original weights and architecture.
Smart Images
Figure CN121639740A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual target tracking technology, and specifically to a real-time target tracking method and apparatus based on a world model. Background Technology
[0002] Currently, with the continuous development and widespread application of mobile device technology with camera capabilities, the importance of target tracking technology for these devices is increasingly prominent. Compared to conventional ground-based target tracking tasks, tracking mobile devices with camera capabilities faces more challenges in estimating and predicting target positions in continuous aerial imagery. These challenges mainly include dealing with extreme shooting angles, mitigating motion blur and image noise caused by rapid movement, and handling severe occlusion. Furthermore, limited by the battery capacity and computing resources of mobile devices with camera capabilities, the efficiency of the tracking algorithm is crucial. Only by keeping computational costs low can the algorithm be successfully deployed on mobile devices with camera capabilities, thereby improving its practicality. Therefore, while pursuing high accuracy, algorithm efficiency must also be considered to meet the unique needs of mobile device applications with camera capabilities.
[0003] In recent years, with the rapid development of artificial intelligence, deep learning-based tracking methods have achieved significant breakthroughs in accuracy and have gradually become a research hotspot. Current mainstream methods mostly adopt a single-stream structural framework, such as introducing a pre-trained Vision Transformer as the backbone network to seamlessly integrate feature extraction and fusion processes. ViT excels at global image modeling and extracting high-level semantic features, thereby achieving accurate target localization. However, its computational efficiency is usually lower than that of convolutional neural network-based backbone networks, making it difficult to meet the real-time processing requirements of mobile devices with camera capabilities. Therefore, researchers have begun to improve ViT by making it lightweight, for example, by using variant structures with fewer parameters or introducing adaptive feature selection mechanisms. However, model lightweighting often comes with a loss of accuracy. Therefore, how to achieve a better balance between accuracy and efficiency in mobile device tracking tasks with camera capabilities remains a problem that urgently needs to be solved. Summary of the Invention
[0004] The first aspect of the present invention provides a real-time target tracking method based on a world model, comprising: in response to motion of a mobile device, acquiring in real time a series of multiple frames of original images captured by the mobile device, each frame of the original image containing a target object; inputting the series of multiple frames of original images into a world model to obtain the target position of the target object in the series of original images, so as to achieve tracking of the target object; wherein the world model is obtained by pre-training based on feature selection and target localization of the series of multiple frames of training images.
[0005] In the above scheme, multiple consecutive frames of original images are input into the world model to obtain the target location of the target object in the multiple frames of original images. This includes: determining the cropped image of the first frame in the multiple consecutive frames of original images as the template image; inputting the template image into the embedding layer of the world model to obtain the first image feature of the template image; and inputting the first image feature of the template image and a preset category vector into the Transformer block of the world model to obtain the second image feature and the first category feature of the template image corresponding to multiple stages, respectively.
[0006] The above scheme involves inputting multiple consecutive frames of original images into a world model to obtain the target location of the target object in the multiple frames of original images. It also includes: determining the search image in the multiple consecutive frames of original images; inputting the search image into the embedding layer of the world model to obtain the third image feature of the search image; and inputting the third image feature of the search image and a preset category vector into the Transformer block of the world model to obtain the fourth image feature and the second category feature of the search image corresponding to multiple stages, respectively.
[0007] In the above scheme, multiple consecutive frames of original images are input into the world model to obtain the target location of the target object in the multiple frames of original images. It also includes: filtering the target image features of the search image in each stage based on the similarity calculation results between the first category feature of the template image in each stage and the fourth image feature of the search image in the corresponding stage; and determining the first position of the target object in the search image in each stage based on the spatial coordinates corresponding to the target image features in each stage.
[0008] In the above scheme, in the Transformer block, the fourth image feature of each stage is the target image feature of the previous stage.
[0009] In the above scheme, multiple consecutive frames of original images are input into the world model to obtain the target position of the target object in the multiple frames of original images. It also includes: calculating and obtaining the mean value of the target image features in multiple stages of the search image; inputting the mean value of the target image features into a preset multilayer perceptron to obtain the position offset value; and summing the first position of the target object in each stage of the search image with the position offset value to obtain the second position of the target object in each stage of the search image.
[0010] In the above scheme, multiple consecutive frames of original images are input into the world model to obtain the target position of the target object in the multiple frames of original images. It also includes: summing the second positions of the target objects in multiple stages according to the preset weights of each stage to obtain the target position of the target object in the search image.
[0011] In the above scheme, the first position of the target object in the search image at each stage is determined based on the spatial coordinates corresponding to the target image features at each stage. This includes: combining the target image features at each stage with the position encoding corresponding to the position index to determine the spatial position information of the target image features in the search image; and determining the first position of the target object in the search image at each stage based on the spatial position information of the target image features in the search image.
[0012] A second aspect of the present invention provides a real-time target tracking device based on a world model, comprising: an original image acquisition module, configured to acquire multiple consecutive frames of original images captured by the mobile device in real time in response to movement of the mobile device, wherein each frame of the original image contains a target object; and a target position acquisition module, configured to input the multiple consecutive frames of original images into the world model to acquire the target position of the target object in the multiple frames of original images, so as to achieve tracking of the target object; wherein the world model is obtained by pre-training based on feature selection and target localization of multiple consecutive frames of training images.
[0013] The technical solutions of the embodiments of the present invention have at least the following beneficial effects:
[0014] (1) This method is based on the world model to achieve real-time target tracking. It retains a fixed proportion of tokens in each stage of the Transformer block of the model, which ensures the determinism of the inference time. In addition, it determines which tokens to retain by using the global characteristics of the template image, which can ensure that features strongly related to the target are retained.
[0015] (2) This method introduces a world model to achieve target tracking without modifying the original weights and architecture of the world model, thus fully preserving the world prior knowledge contained therein. On this basis, a feature selection mechanism based on the similarity between features is also proposed to further achieve accurate target localization. This method not only maintains the world model's deep understanding of the scene, but also significantly reduces the computational resources consumed, improving execution efficiency while ensuring tracking accuracy. Attached Figure Description
[0016] Figure 1 A flowchart illustrating a real-time target tracking method based on a world model according to an embodiment of the present invention is shown schematically.
[0017] Figure 2 The diagram illustrates the processing steps of the world model for template images and search images according to an embodiment of the present invention.
[0018] Figure 3 This diagram illustrates the process of target image feature selection according to an embodiment of the present invention.
[0019] Figure 4A rough location map determined based on target image features according to an embodiment of the present invention is illustrated schematically; and
[0020] Figure 5 A schematic diagram illustrating the structure of a real-time target tracking device based on a world model according to an embodiment of the present invention is shown. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0022] Figure 1 A flowchart illustrating a real-time target tracking method based on a world model according to an embodiment of the present invention is shown.
[0023] Please refer to the following for details. Figure 1 In an embodiment of the present invention, the specific process of the real-time target tracking method based on the world model includes operations S110 to S120.
[0024] In operation S110, in response to the movement of the mobile device, multiple consecutive frames of raw images captured by the mobile device are acquired in real time, and each frame of raw image contains the target object.
[0025] For example, for mobile devices with shooting capabilities, such as drones, it is necessary to photograph the target object while in motion. After the drone performs continuous aerial photography, it acquires multiple frames of raw images in real time, and each frame of the raw image contains the target object to be photographed.
[0026] Furthermore, in order to enable mobile devices to determine the position of target objects in continuously captured images, this embodiment proposes to perform real-time target object position tracking on multiple consecutive frames of original images based on a preset world model and corresponding calculation methods, which will be described in detail below.
[0027] In operation S120, multiple consecutive frames of original images are input into the world model to obtain the target position of the target object in the multiple frames of original images, so as to realize the tracking of the target object.
[0028] In embodiments of the present invention, the world model is a model pre-trained based on feature selection and target localization of multiple consecutive training images. It should be noted that during the model training process, the parameters of the backbone network are always frozen, and only the parameters in the feature adaptor and predictor are updated, thereby ensuring that the world knowledge in the world model is not destroyed.
[0029] Specifically, in this embodiment, the backbone network of the world model includes a patch embedding layer. and N Transformer layers The Transformer blocks, consisting of N=12 layers (e.g., N=12), also include a learnable class vector, denoted as Cls, to aggregate global information from the input image. Furthermore, the 12 Transformer layers are divided into four stages (layers 1-3, 4-6, 7-9, and 10-12), denoted as... Each stage focuses on different types of image characteristics (typically from local to global, from low-level to high-level). In embodiments of the present invention, multiple Transformer layers are divided into multiple stages, thereby combining the features of multiple stages to improve tracking accuracy.
[0030] In an embodiment of the present invention, the cropped image of the first frame in a series of consecutive original images is determined as the template image.
[0031] Specifically, in target tracking tasks, a template image is required. This template image is used to determine the target object to be tracked. The initial template image is usually manually extracted from the first frame of the entire video stream. That is, the image containing the target object is cropped from the first frame of the captured multiple consecutive frames of original images and used as the template image. .
[0032] In an embodiment of the present invention, a template image is input into the embedding layer of the world model to obtain the first image feature of the template image. Then, the first image feature of the template image and a preset category vector are input into the Transformer block of the world model to obtain the second image feature and the first category feature corresponding to multiple stages of the template image, respectively.
[0033] Specifically, the template image Input to patch embedding layer The first image feature token of the template image is obtained, denoted as T. t Then T t It is concatenated with the class vector Cls and then fed into the various Transformer layers in a series of consecutive stages. Then, the feature maps of each stage are extracted. These feature maps include the second image features and the first category features, denoted as... The feature maps of different intermediate layers describe features at multiple levels, which can more comprehensively describe the characteristics of the scene in the image.
[0034] Based on the above, this example describes the feature map extraction process of the template image using a world model. The following section describes in detail how to use a world model to extract the feature map of the search image and achieve accurate localization of the target object.
[0035] In an embodiment of the present invention, a search image is determined from multiple consecutive frames of original images.
[0036] Understandably, the search image is usually each frame in the video stream, or it can be considered as a non-first frame in a series of original images. The search image needs to be searched from the search image based on the template image to determine whether the target object to be tracked exists and to accurately locate the position of the target object.
[0037] In an embodiment of the present invention, the search image is input into the embedding layer of the world model to obtain the third image feature of the search image. Then, the third image feature of the search image and a preset category vector are input into the Transformer block of the world model to obtain the fourth image feature and the second category feature corresponding to multiple stages of the search image, respectively. In the Transformer block, the fourth image feature of each stage is the target image feature of the previous stage.
[0038] Specifically, the search image I s Also input into the patch embedding layer The third image feature token of the search image is obtained, denoted as T. s Then T s The third image feature, concatenated with the category vector Cls, serves as the input to a series of multi-stage Transformer layers. This third image feature can be understood as the input object of the first stage of the Transformer layer. Furthermore, after four stages... After processing, the fourth image features corresponding to each stage of the search image are obtained respectively. Second category features .
[0039] It should be noted that, in this embodiment, within the Transformer block, the fourth image feature of each stage is the target image feature of the previous stage. Here, the target image feature is the important image feature retained after filtering the fourth image feature in each stage; this target image feature is the input object of the Transformer layer in each stage, starting from the second stage.
[0040] Understandably, when processing template images, all feature tokens (i.e., the first image feature token and the class vector Cls) are sequentially fed into each stage of each transformer block. However, processing search images is different. When processing search images, the third image feature token starts from the first stage, and after each stage, a portion of the tokens are discarded, retaining only the important parts (i.e., the target image features). These target image features are then used as input for the next stage to improve computational efficiency.
[0041] The following describes in detail the process of determining the target image features through feature filtering of the search image, and the process of accurately locating the target object based on the target image features.
[0042] Figure 2 The diagram illustrates the processing flow of the template image and the search image by the world model according to an embodiment of the present invention. Figure 3 The diagram illustrates the process of target image feature selection according to an embodiment of the present invention.
[0043] In an embodiment of the present invention, the target image features of the search image at each stage are selected based on the similarity calculation results between the first category features of the template image at each stage and the fourth image features of the search image at the corresponding stage.
[0044] Specifically, such as Figure 2 As shown, when the third image feature token of the search image is input into the Transformer block in the world model, after the first stage... Output the fourth image feature and the second category feature of the first stage, denoted as After the template image passes through the first stage, the output is the second image feature and the first category feature, denoted as... .like Figure 3 As shown, the first category features of the template image are then calculated. and the fourth image feature of the search image The cosine similarity between them is obtained. The value ranges from (-1, 1). A larger R value indicates a higher similarity between vectors. The top 50% of the maximum values in R are selected to filter out the target image features in the first stage of the search image. .
[0045] Furthermore, such as Figure 2 As shown, for the second stage, the search image uses the target image features selected in the first stage. This will be the input object for the second stage, after... The output of the fourth image feature is also the target image feature of the first stage. Furthermore, such as Figure 3 As shown, the first category features of the template image in the second stage are then calculated. and the first stage of searching for target image features The cosine similarity between them is obtained. The value range is (-1, 1). The top 50% of the maximum values in R are selected to filter out the target image features in the second stage of the search image. .
[0046] Furthermore, such as Figure 2As shown, the target image features in the third stage of the search image are calculated sequentially. and the target image features in the fourth stage Here, the target image features at each stage are a set of multiple feature tokens.
[0047] Based on the above, the feature maps output at each stage are finally extracted. It is used for predicting the location of a target.
[0048] Based on the above, it is understandable that, compared to existing technologies that concatenate the template image and the search image together and process them together in the Transformer block, this approach facilitates simultaneous feature fusion and search, but the computational complexity is [not specified]. Where n is the number of search image tokens and m is the number of template image tokens. This embodiment leverages the powerful understanding capabilities of the world model to process the two types of images separately, eliminating the feature fusion step and reducing the computational complexity to O(n). In other words, this embodiment does not require the introduction of an additional module for feature fusion. It only utilizes the similarity between the features of the template image and the features of the search image to filter and locate image feature tokens, thereby improving execution efficiency.
[0049] In an embodiment of the present invention, the first position of the target object in the search image at each stage is determined based on the spatial coordinates corresponding to the target image features at each stage.
[0050] Figure 4 The diagram schematically illustrates a rough location map determined based on target image features according to an embodiment of the present invention.
[0051] In embodiments of the present invention, the spatial location information of the target image features in the search image is determined by combining the target image features of each stage with the location encoding corresponding to the location index. Then, based on the spatial location information of the target image features in the search image, the first position of the target object in the search image of each stage is determined.
[0052] Specifically, calculate separately Each stage filters out the spatial coordinates represented by the set of tokens (i.e., target image features), which are the coordinates of the smallest bounding rectangle of the polygon formed by these tokens in 2D space (e.g., ...). Figure 4 The image shows the approximate location (i.e., the first location) P of the target object being tracked. i , recorded as .
[0053] Based on the above, such as Figure 2As shown, it can be understood that when processing the template image and the search image through the world model, for each stage in the Transformer block, the features of the search image are filtered to determine the features of the target image, and the approximate position (i.e., the first position) of the target object in the search image of each stage is output based on these features. i .
[0054] Furthermore, based on the first determined position and target image features, the target object is precisely located to achieve target object tracking.
[0055] In an embodiment of the present invention, the mean value of target image features at multiple stages of the search image is calculated and obtained. The mean value of target image features is then input into a preset multilayer perceptron to obtain a position offset value. Finally, the first position of the target object in each stage of the search image is summed with the position offset value to obtain the second position of the target object in each stage of the search image.
[0056] Specifically, the token sets (i.e., target image features) selected in each stage are averaged to obtain... Then, they are input into a multilayer perceptron (MLP) to obtain the position offset value, denoted as... Finally, the second position predicted by the feature at each stage is the sum of the first position and the offset position, which yields the second position. .
[0057] In an embodiment of the present invention, the second positions of multiple target objects in each stage are summed according to the preset weight of each stage to obtain the target position of the target object in the search image.
[0058] Specifically, the location of the target object in the search image is represented as a weighted sum of the second locations predicted at each stage.
[0059]
[0060] in, The weights are for the corresponding stages. .
[0061] It should be noted that in this embodiment, the feature tokens are gradually filtered in each stage, and the scope of the tracked target is gradually refined. Therefore, the weights corresponding to each stage gradually increase. For example, the weights of the four stages can be 0.1, 0.2, 0.3, and 0.4, respectively.
[0062] Based on the above, it is understandable that, compared to existing technologies that typically use token content to directly predict the target's location, this embodiment proposes a two-stage prediction method. It first roughly determines the target's location using the spatial position of the token, and then adjusts the location using the token content. That is, this embodiment first roughly estimates the target's location based on the spatial features of the selected image feature tokens, and then fine-tunes the rough location using the content of the image feature tokens, thereby determining the target's final location and ensuring tracking accuracy.
[0063] Unlike traditional approaches that train models from scratch or fine-tune pre-trained models for specific tasks, this embodiment of the invention introduces a world model to achieve UAV target tracking without modifying the original weights and architecture of the world model, thus fully preserving its inherent prior world knowledge. Furthermore, a feature selection mechanism based on feature similarity is proposed to further achieve precise target localization. This method not only maintains the world model's deep understanding of the scene but also significantly reduces computational resources, improving execution efficiency while ensuring tracking accuracy.
[0064] Based on the above-described real-time target tracking method based on a world model, this invention also provides a real-time target tracking device based on a world model. The following will combine... Figure 5 The device is described in detail.
[0065] Figure 5 A schematic diagram illustrating the structure of a real-time target tracking device based on a world model according to an embodiment of the present invention is shown.
[0066] like Figure 5 As shown, the real-time target tracking device 500 based on the world model in this embodiment includes an original image acquisition module 510 and a target position acquisition module 520.
[0067] The raw image acquisition module 510 is used to acquire multiple consecutive frames of raw images captured by the mobile device in real time in response to movement of the mobile device, each frame of the raw image containing the target object. In one embodiment, the raw image acquisition module 510 can be used to perform the operation S110 described above, which will not be repeated here.
[0068] The target location acquisition module 520 is used to input multiple consecutive frames of original images into the world model to obtain the target location of the target object in the multiple frames of original images, so as to achieve target object tracking. In one embodiment, the target location acquisition module 520 can be used to perform the operation S120 described above, which will not be repeated here.
[0069] According to embodiments of the present invention, any plurality of modules in the original image acquisition module 510 and the target location acquisition module 520 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of the present invention, at least one of the original image acquisition module 510 and the target location acquisition module 520 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the original image acquisition module 510 and the target location acquisition module 520 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.
[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0071] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.
[0072] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A real-time target tracking method based on a world model, characterized by, The method comprises: in response to the motion of the mobile device, acquiring in real time a plurality of continuous original images taken by the mobile device, each of the original images containing a target object; inputting the plurality of continuous original images into a world model to obtain a target position of the target object in the plurality of original images, so as to track the target object; wherein the world model is obtained by pre-training based on feature screening and target positioning of a plurality of continuous training images.
2. The real-time target tracking method based on a world model according to claim 1, characterized in that, inputting the plurality of continuous original images into the world model to obtain the target position of the target object in the plurality of original images comprises: determining a cropped image of a first frame of the plurality of continuous original images as a template image; inputting the template image into an embedding layer of the world model to obtain a first image feature of the template image; inputting the first image feature of the template image and a preset category vector into a Transformer block of the world model to obtain a second image feature and a first category feature corresponding to a plurality of stages of the template image, respectively.
3. The real-time target tracking method based on a world model according to claim 2, characterized in that, inputting the plurality of continuous original images into the world model to obtain the target position of the target object in the plurality of original images further comprises: determining a search image in the plurality of continuous original images; inputting the search image into the embedding layer of the world model to obtain a third image feature of the search image; inputting the third image feature of the search image and the preset category vector into the Transformer block of the world model to obtain a fourth image feature and a second category feature corresponding to a plurality of stages of the search image, respectively.
4. The real-time target tracking method based on a world model according to claim 2 or 3, characterized in that, inputting the plurality of continuous original images into the world model to obtain the target position of the target object in the plurality of original images further comprises: screening a target image feature of each stage of the search image according to a similarity calculation result of the first category feature of each stage of the template image and the fourth image feature of the corresponding stage of the search image; determining a first position of the target object in each stage of the search image according to a spatial coordinate corresponding to each stage of the target image feature.
5. The real-time target tracking method based on a world model according to claim 4, characterized in that, In the Transformer block, the fourth image feature of each stage is the target image feature of the previous stage.
6. The real-time target tracking method based on a world model according to claim 5, characterized in that, inputting the plurality of continuous original images into the world model to obtain the target position of the target object in the plurality of original images further comprises: calculating and obtaining a mean value of the target image features of the plurality of stages of the search image; inputting the mean value of the target image features into a preset multi-layer perceptron to obtain a position offset value; summing the first position of the target object in each stage of the search image and the position offset value to obtain a second position of the target object in each stage of the search image.
7. The real-time target tracking method based on a world model according to claim 6, characterized in that, inputting the plurality of continuous original images into the world model to obtain the target position of the target object in the plurality of original images further comprises: summing the second positions of the target object of the plurality of stages according to a preset weight of each stage to obtain the target position of the target object in the search image.
8. The real-time target tracking method based on a world model according to claim 4, characterized in that, The first position of the target object in the search image in each stage is determined according to the spatial coordinates corresponding to the target image feature of each stage, and the first position of the target object in the search image in each stage comprises: combining the target image feature of each stage with the position coding corresponding to the position index to determine the spatial position information of the target image feature in the search image; determining the first position of the target object in the search image in each stage according to the spatial position information of the target image feature in the search image.
9. A real-time target tracking apparatus based on a world model, characterized by, The device comprises: An original image acquisition module is configured to acquire continuously multiple frames of original images taken by a mobile device in real time in response to motion of the mobile device, and each frame of the original images contains a target object; A target position acquisition module is configured to input the continuously multiple frames of original images into a world model to acquire target positions of the target object in the multiple frames of original images, so as to track the target object; The world model is obtained by pre-training based on feature screening and target positioning of continuously multiple frames of training images.