A target personnel trajectory positioning method, device, equipment and medium

Abstract

This invention discloses a method, apparatus, device, and medium for locating the trajectory of a target person, belonging to the field of target person trajectory localization. The method comprises: acquiring single-modal training sample data and dual-modal training sample data; inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model; establishing a student model based on the Mamba architecture, and inputting the dual-modal training sample data into the student model for iterative training until the total loss value of the current iteration meets a first preset condition, outputting the student model of the current iteration as the target person trajectory localization model; finally, acquiring pedestrian data to be located, and inputting the pedestrian data into the target person trajectory localization model to obtain the localization result corresponding to the pedestrian data. Therefore, by implementing this invention, the problem of high cost in target person trajectory localization in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of target personnel trajectory positioning, and in particular to a method, apparatus, device, and medium for target personnel trajectory positioning. Background Technology

[0002] Target personnel trajectory localization technology has been widely applied in video surveillance, intelligent security, and trajectory analysis. Its core purpose is to quickly identify the same pedestrian across multiple cameras and time periods. In practical applications, by training the model, it can automatically extract and match the visual features of pedestrians or respond to text description queries. This allows for the rapid retrieval and association of the target's movement trajectory in other monitored areas even after the target has disappeared from one camera, effectively solving the problem of low efficiency in manual investigation and significantly improving the intelligence level and response speed of public safety monitoring.

[0003] In existing technologies, mainstream cross-modal target and person trajectory localization model training typically relies on large-scale Transformer backbone network architectures, utilizing attention mechanisms to construct robust image-text alignment models. However, this method suffers from significant cost drawbacks: the high complexity of attention computation within the Transformer architecture results in a massive number of parameters in the trained model. This high computational cost directly leads to high inference latency and large memory consumption, making it difficult to directly deploy the trained model to resource-constrained edge cameras or mobile devices, creating a technical bottleneck of "high-precision models cannot be deployed at low cost." Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for locating the trajectory of a target person, which can solve the problem of high cost in the prior art for locating the trajectory of a target person.

[0005] In a first aspect, embodiments of the present invention provide a method for locating the trajectory of a target person, including: Acquire unimodal training sample data and bimodal training sample data; wherein, each unimodal training sample in the unimodal training sample data includes a first pedestrian image, and each bimodal training sample in the bimodal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The single-modal training sample data is input into the first preset language image model and subjected to two-stage iterative training to obtain the teacher model. Each single-modal training sample in the single-modal training sample data is input into the teacher model in sequence, and the first distillation scale of each single-modal training sample on the teacher side is output. A student model is built based on the Mamba architecture, and the bimodal training sample data is input into the student model for iterative training until the total loss value of the current iteration meets the first preset condition. The student model of the current iteration is then output as the target person trajectory localization model. The total loss value of the current iteration includes distillation loss, which is determined based on each of the first distillation scales and the second distillation scale of the current iteration. Acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

[0006] This application first trains a teacher model on single-modal data and outputs a distillation scale. Leveraging the teacher model's powerful feature extraction capabilities, a high-quality knowledge base is constructed, providing accurate supervision signals for subsequent training and ensuring knowledge accuracy. Then, a student model based on the Mamba architecture is adopted. Utilizing the linear complexity of the state-space model, this overcomes the limitations of high computational complexity and large memory consumption of traditional Transformer models, significantly reducing the number of model parameters and computational resource consumption. This directly solves the technical problems of high computational cost and difficulty in deploying cross-modal target personnel trajectory localization models on edge devices in existing technologies. Finally, by introducing a total loss value including distillation loss for iterative training of the student model, the deep semantic knowledge of the teacher model in "image-text alignment" and "identity discrimination" is effectively transferred to the lightweight student model. This compensates for the insufficient long-range dependency modeling capability caused by the simplified structure of the lightweight model, thereby maintaining high-precision cross-modal retrieval performance while significantly reducing inference costs and model size, achieving a balance between low cost and high accuracy.

[0007] As a preferred example of the first aspect, the step of inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model includes: In the first stage of iterative training, the parameters of the first image encoder and the first text encoder in the first preset language image model are frozen, and each of the first pedestrian images is input into the first image encoder and the first text encoder for iterative training until the first total loss value of the current iteration meets the second preset condition. The first preset language image model of the current iteration is then output as the intermediate teacher model. In the second stage of iterative training, the parameters of the second image encoder in the intermediate teacher model are frozen and the parameters of the second text encoder in the intermediate teacher model are unfrozen. Each of the first pedestrian images is then input into the second image encoder and the second text encoder for iterative training until the second total loss value of the current iteration meets the third preset condition.

[0008] In this preferred example, a two-stage iterative training strategy is used to construct the teacher model. The first stage involves training with the encoder parameters frozen, effectively preserving the pre-trained model's powerful general image-text alignment capabilities, preventing catastrophic forgetting, and ensuring the stability of the model's basic feature extraction. The second stage employs a strategy of "freezing the image encoder and unfreezing the text encoder," fine-tuning the text description for specific domains while maintaining visual feature stability, significantly improving the model's semantic alignment accuracy in the target scene. This approach balances the model's generalization ability with its adaptability to specific tasks, ultimately generating a high-quality teacher model that provides an accurate and reliable supervisory benchmark for subsequent knowledge distillation.

[0009] As a preferred example of the first aspect, the step of sequentially inputting each unimodal training sample from the unimodal training sample data into the teacher model and outputting the first distillation scale of each unimodal training sample on the teacher side includes: The weight parameters of the teacher model are frozen, and each unimodal training sample in the unimodal training sample data is sequentially input into the teacher model for offline forward inference to obtain the teacher-side image features, teacher-side text features and teacher-side prediction probabilities corresponding to each unimodal training sample. Based on the teacher-side image features, teacher-side text features, and teacher-side prediction probabilities corresponding to each of the unimodal training samples, the first distillation scale of each of the unimodal training samples on the teacher side is obtained.

[0010] In this preferred example, freezing the weight parameters of the teacher model ensures the stability and consistency of its output, establishing a fixed supervision standard. Offline forward inference is used to acquire features and probabilities, successfully decoupling the teacher model's computation process from student training, avoiding repetitive forward propagation, and significantly reducing memory usage and computation time costs during training. The final constructed first distillation scale fully preserves the teacher model's deep knowledge at the image, text, and prediction levels, providing accurate and high-quality guidance signals for the student model, thus ensuring the effectiveness of knowledge distillation and the retrieval accuracy of the final model.

[0011] As a preferred example of the first aspect, the step of inputting the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and outputting the student model of the current iteration as the target person trajectory localization model, includes: During each iteration, the second pedestrian image and its corresponding text description are input into the image encoder and text encoder of the student model in the current iteration, respectively, to obtain the student-side image features and student-side text features; Based on the student-side image features and the student-side text features, the identity recognition loss, triplet loss, and bidirectional contrast loss are calculated, and the total loss value of the current iteration is determined based on the identity recognition loss, triplet loss, bidirectional contrast loss, and distillation loss. The parameters of the student model in the current iteration are updated based on the total loss value of the current iteration until the total loss value of the current iteration satisfies a first preset condition.

[0012] In this preferred example, image and text features are extracted using a dual encoder to construct the basic representation for cross-modal retrieval. Furthermore, identity recognition, triplet, and bidirectional contrastive losses are comprehensively calculated, providing multi-dimensional supervision from three dimensions: identity discrimination, feature distance measurement, and image-text semantic alignment. This ensures the accuracy and robustness of the model's feature extraction. Crucially, distillation loss is introduced to jointly calculate the total loss, effectively transferring the deep knowledge of the teacher model to the lightweight student model, compensating for the representational limitations caused by its structural simplification. Finally, by iteratively updating parameters until convergence, the model achieves high-precision target localization performance while reducing deployment costs.

[0013] As a preferred example of the first aspect, the distillation loss is determined based on each of the first distillation scales and the second distillation scale of the current iteration, including: Obtain the teacher-side prediction probability, teacher-side image features, and teacher-side text features corresponding to the second distillation scale of the current iteration from each of the first distillation scales, and determine the student-side prediction probability, student-side image features, and student-side text features based on the second distillation scale of the current iteration; The soft label divergence loss is calculated based on the teacher-side prediction probability and the student-side prediction probability, and the feature mean squared error loss is calculated based on the teacher-side image features, the teacher-side text features, the student-side image features, and the student-side text features. The distillation loss is determined based on the soft label divergence loss and the feature mean squared error loss.

[0014] In this preferred example, by calculating the soft-label divergence loss, the output probability distribution of the student model is made to approximate that of the teacher model, thereby effectively transferring deep semantic knowledge of identity discrimination and improving classification accuracy. Simultaneously, the feature space of the teacher and student is directly aligned using the feature mean squared error loss, ensuring fine-grained representation of text and image features and cross-modal alignment accuracy. Combining the distillation losses determined by these two methods achieves a two-level knowledge transfer from semantic distribution to feature representation, effectively compensating for the insufficient representational ability of the lightweight student model due to its simplified structure. This significantly reduces inference costs while ensuring the model's high-precision trajectory localization performance.

[0015] As a preferred example of the first aspect, determining the distillation loss based on the soft-label divergence loss and the feature mean squared error loss includes: The teacher distance matrix is ​​determined based on the teacher-side image features and the teacher-side text features, and the student distance matrix is ​​determined based on the student-side image features and the student-side text features. The relationship preservation loss is calculated based on the distance between the teacher distance matrix and the student distance matrix, and the distillation loss is determined based on the soft label divergence loss, the feature mean squared error loss, and the relationship preservation loss.

[0016] In this preferred example, the structural relationships between samples are quantified by constructing a distance matrix between teachers and students. Then, a relationship preservation loss is calculated to ensure that the student model maintains consistent sample geometric relationships with the teacher model in the feature space, effectively preserving global topological information and compensating for the shortcomings of simple feature alignment that ignores the relationships between samples. Finally, a multi-level distillation strategy integrating soft labels, feature means, and relationship loss achieves comprehensive knowledge transfer from probability distribution and feature representation to structural relationships, significantly enhancing the discriminative logic of the lightweight model and ensuring high-precision trajectory localization performance while reducing costs.

[0017] As a preferred example of the first aspect, after obtaining the location result corresponding to the pedestrian data, the method further includes: Based on the positioning results, the location and time information of the target person are determined, and the movement trajectory of the target person is generated based on the location and time information.

[0018] In this preferred example, specific location and time information are parsed from the positioning results, transforming abstract identity matching data into quantifiable spatiotemporal dimensions, thus enhancing the intuitiveness and usability of the results. Furthermore, by associating time series data with location coordinates to generate movement trajectories, the movement paths of target personnel within the monitoring network can be reconstructed. This method effectively solves the problem of difficult cross-regional information correlation in traditional manual investigations, enabling the positioning results output by the lightweight model to be directly transformed into practical value, significantly improving the trajectory analysis capabilities and response efficiency of intelligent security systems.

[0019] Secondly, the present invention provides a target personnel trajectory positioning device, comprising: a data acquisition module, a first positioning module, a second positioning module, and a third positioning module; The data acquisition module is used to acquire single-modal training sample data and dual-modal training sample data; wherein, each single-modal training sample in the single-modal training sample data includes a first pedestrian image, and each dual-modal training sample in the dual-modal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The first positioning module is used to input the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model, and to input each single-modal training sample in the single-modal training sample data into the teacher model in sequence, and output the first distillation scale of each single-modal training sample on the teacher side. The second positioning module is used to build a student model based on the Mamba architecture, and input the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and output the student model of the current iteration as the target person trajectory positioning model; wherein, the total loss value of the current iteration includes distillation loss, which is determined according to each of the first distillation scales and the second distillation scale of the current iteration; The third positioning module is used to acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

[0020] As a preferred example of the second aspect, the step of inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model includes: In the first stage of iterative training, the parameters of the first image encoder and the first text encoder in the first preset language image model are frozen, and each of the first pedestrian images is input into the first image encoder and the first text encoder for iterative training until the first total loss value of the current iteration meets the second preset condition. The first preset language image model of the current iteration is then output as the intermediate teacher model. In the second stage of iterative training, the parameters of the second image encoder in the intermediate teacher model are frozen and the parameters of the second text encoder in the intermediate teacher model are unfrozen. Each of the first pedestrian images is then input into the second image encoder and the second text encoder for iterative training until the second total loss value of the current iteration meets the third preset condition.

[0021] As a preferred example of the second aspect, the step of sequentially inputting each unimodal training sample from the unimodal training sample data into the teacher model and outputting the first distillation scale of each unimodal training sample on the teacher side includes: The weight parameters of the teacher model are frozen, and each unimodal training sample in the unimodal training sample data is sequentially input into the teacher model for offline forward inference to obtain the teacher-side image features, teacher-side text features and teacher-side prediction probabilities corresponding to each unimodal training sample. Based on the teacher-side image features, teacher-side text features, and teacher-side prediction probabilities corresponding to each of the unimodal training samples, the first distillation scale of each of the unimodal training samples on the teacher side is obtained.

[0022] As a preferred example of the second aspect, the step of inputting the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and outputting the student model of the current iteration as the target person trajectory localization model, includes: During each iteration, the second pedestrian image and its corresponding text description are input into the image encoder and text encoder of the student model in the current iteration, respectively, to obtain the student-side image features and student-side text features; Based on the student-side image features and the student-side text features, the identity recognition loss, triplet loss, and bidirectional contrast loss are calculated, and the total loss value of the current iteration is determined based on the identity recognition loss, triplet loss, bidirectional contrast loss, and distillation loss. The parameters of the student model in the current iteration are updated based on the total loss value of the current iteration until the total loss value of the current iteration satisfies a first preset condition.

[0023] As a preferred example of the second aspect, the distillation loss is determined based on each of the first distillation scales and the second distillation scale of the current iteration, including: Obtain the teacher-side prediction probability, teacher-side image features, and teacher-side text features corresponding to the second distillation scale of the current iteration from each of the first distillation scales, and determine the student-side prediction probability, student-side image features, and student-side text features based on the second distillation scale of the current iteration; The soft label divergence loss is calculated based on the teacher-side prediction probability and the student-side prediction probability, and the feature mean squared error loss is calculated based on the teacher-side image features, the teacher-side text features, the student-side image features, and the student-side text features. The distillation loss is determined based on the soft label divergence loss and the feature mean squared error loss.

[0024] As a preferred example of the second aspect, determining the distillation loss based on the soft-label divergence loss and the feature mean squared error loss includes: The teacher distance matrix is ​​determined based on the teacher-side image features and the teacher-side text features, and the student distance matrix is ​​determined based on the student-side image features and the student-side text features. The relationship preservation loss is calculated based on the distance between the teacher distance matrix and the student distance matrix, and the distillation loss is determined based on the soft label divergence loss, the feature mean squared error loss, and the relationship preservation loss.

[0025] As a preferred example of the second aspect, after obtaining the positioning result corresponding to the pedestrian data, the method further includes: Based on the positioning results, the location and time information of the target person are determined, and the movement trajectory of the target person is generated based on the location and time information.

[0026] In summary, this application first trains a teacher model on single-modal data and outputs a distillation scale. Leveraging the teacher model's powerful feature extraction capabilities, a high-quality knowledge base is constructed, providing accurate supervision signals for subsequent training and ensuring knowledge accuracy. Next, a student model based on the Mamba architecture is adopted. Utilizing the linear complexity of the state-space model, this overcomes the limitations of high computational complexity and large memory consumption of traditional Transformer models, significantly reducing the number of model parameters and computational resource consumption. This directly solves the technical problems of high computational cost and difficulty in deploying cross-modal target personnel trajectory localization models on edge devices in existing technologies. Finally, by introducing a total loss value including distillation loss for iterative training of the student model, the deep semantic knowledge of the teacher model in "image-text alignment" and "identity discrimination" is effectively transferred to the lightweight student model. This compensates for the shortcomings of the lightweight model in long-range dependency modeling capabilities due to structural simplification, thereby maintaining high-precision cross-modal retrieval performance while significantly reducing inference costs and model size, achieving a balance between low cost and high accuracy.

[0027] Another embodiment of the present invention provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the steps of the target personnel trajectory positioning method of the present invention.

[0028] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps such as the target personnel trajectory positioning method of the present invention. Attached Figure Description

[0029] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating an embodiment of a target personnel trajectory localization method provided by the present invention; Figure 2 This is a teacher model structure diagram of an embodiment of a target personnel trajectory localization method provided by the present invention; Figure 3 This is a module structure diagram of one embodiment of a target personnel trajectory positioning device provided by the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0033] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0034] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0035] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0036] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0037] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0038] Example 1 See Figure 1 To address the high cost of target personnel trajectory localization in existing technologies, an embodiment of the present invention provides a target personnel trajectory localization method, comprising: S1. Obtain unimodal training sample data and bimodal training sample data; wherein, each unimodal training sample in the unimodal training sample data includes a first pedestrian image, and each bimodal training sample in the bimodal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; S2. Input the single-modal training sample data into the first preset language image model and perform two-stage iterative training to obtain the teacher model. Then, input each single-modal training sample in the single-modal training sample data into the teacher model in sequence and output the first distillation scale of each single-modal training sample on the teacher side. As a preferred example of the first aspect, the step of inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model includes: In the first stage of iterative training, the parameters of the first image encoder and the first text encoder in the first preset language image model are frozen, and each of the first pedestrian images is input into the first image encoder and the first text encoder for iterative training until the first total loss value of the current iteration meets the second preset condition. The first preset language image model of the current iteration is then output as the intermediate teacher model. In the second stage of iterative training, the parameters of the second image encoder in the intermediate teacher model are frozen and the parameters of the second text encoder in the intermediate teacher model are unfrozen. Each of the first pedestrian images is then input into the second image encoder and the second text encoder for iterative training until the second total loss value of the current iteration meets the third preset condition.

[0039] Specifically, the step of inputting the single-modal training sample data into the first preset language image model for two-stage iterative training to obtain the teacher model can be implemented in the following preferred manner: like Figure 2 As shown, the first preset language image model adopts the CLIP-ReID model, and its image encoder ViT-B / 16 is denoted as... The text encoder is denoted as The single-modal training sample data is preprocessed and then input into... , the text "This is a photo of [ ][ ][ ][ The preprocessed text "person" is then entered into the input. ,in[ ][ ][ ][ ] are four learnable vectors used to represent pedestrian features.

[0040] In the first phase of iterative training, freeze and The output is obtained (Visual features) and (Text features), both output dimensions are the same, then the image-text contrast loss is calculated. Calculate the text-image contrast loss ,in Let represent the similarity calculation, where ... .

[0041] In the second phase of iterative training, freeze ,thaw Because in the first stage, learnable vectors [ ][ ][ ][ Image features have already been learned, so only fine-tuning is needed in the second stage. This makes it suitable for pedestrian re-identification tasks. Our output is... (Visual features) and (Text features), for Calculate a triplet loss ,in and It is the feature distance between positive and negative sample pairs. It is the interval hyperparameter. Simultaneously, for each... and The ID cross-entropy loss is calculated by using a fully connected layer to compute its logit (predicted label). ,in To predict the probability of being correct, This represents the smoothing label. Therefore, the total loss function for the second stage is... Finally, training continued until the mAP metric for market1501 reached 89.5%.

[0042] As a preferred example of the first aspect, the step of sequentially inputting each unimodal training sample from the unimodal training sample data into the teacher model and outputting the first distillation scale of each unimodal training sample on the teacher side includes: The weight parameters of the teacher model are frozen, and each unimodal training sample in the unimodal training sample data is sequentially input into the teacher model for offline forward inference to obtain the teacher-side image features, teacher-side text features and teacher-side prediction probabilities corresponding to each unimodal training sample. Based on the teacher-side image features, teacher-side text features, and teacher-side prediction probabilities corresponding to each of the unimodal training samples, the first distillation scale of each of the unimodal training samples on the teacher side is obtained.

[0043] S3. Establish a student model based on the Mamba architecture, and input the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and output the student model of the current iteration as the target person trajectory localization model; wherein, the total loss value includes distillation loss, which is determined according to each of the first distillation scales and the second distillation scale of the current iteration; As a preferred example of the first aspect, the step of inputting the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and outputting the student model of the current iteration as the target person trajectory localization model, includes: During each iteration, the second pedestrian image and its corresponding text description are input into the image encoder and text encoder of the student model in the current iteration, respectively, to obtain the student-side image features and student-side text features; Based on the student-side image features and the student-side text features, the identity recognition loss, triplet loss, and bidirectional contrast loss are calculated, and the total loss value of the current iteration is determined based on the identity recognition loss, triplet loss, bidirectional contrast loss, and distillation loss. The parameters of the student model in the current iteration are updated based on the total loss value of the current iteration until the total loss value of the current iteration satisfies a first preset condition.

[0044] As a preferred example of the first aspect, the distillation loss is determined based on each of the first distillation scales and the second distillation scale of the current iteration, including: Obtain the teacher-side prediction probability, teacher-side image features, and teacher-side text features corresponding to the second distillation scale of the current iteration from each of the first distillation scales, and determine the student-side prediction probability, student-side image features, and student-side text features based on the second distillation scale of the current iteration; The soft label divergence loss is calculated based on the teacher-side prediction probability and the student-side prediction probability, and the feature mean squared error loss is calculated based on the teacher-side image features, the teacher-side text features, the student-side image features, and the student-side text features. The distillation loss is determined based on the soft label divergence loss and the feature mean squared error loss.

[0045] As a preferred example of the first aspect, determining the distillation loss based on the soft-label divergence loss and the feature mean squared error loss includes: The teacher distance matrix is ​​determined based on the teacher-side image features and the teacher-side text features, and the student distance matrix is ​​determined based on the student-side image features and the student-side text features. The relationship preservation loss is calculated based on the distance between the teacher distance matrix and the student distance matrix, and the distillation loss is determined based on the soft label divergence loss, the feature mean squared error loss, and the relationship preservation loss.

[0046] It should be noted that the student model can use dual Mamba as the backbone network, namely Vision-Mamba on the image side and Text-Mamba on the text side, and form a unified embedding at the end through cross-fusion.

[0047] Specifically, the formula for calculating the total loss value can be as follows: ; in, This is the total loss value. For cross-entropy loss, For triple loss, For bidirectional InfoNCE contrast loss, For soft-label divergence loss, The loss is the squared error of the characteristic mean. To maintain the relationship despite losses, the initial settings are as follows: To prevent gradients of different magnitudes from dominating, GradNorm dynamic rescaling is used during training. , and .

[0048] S4. Obtain pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

[0049] As a preferred example of the first aspect, after obtaining the location result corresponding to the pedestrian data, the method further includes: Based on the positioning results, the location and time information of the target person are determined, and the movement trajectory of the target person is generated based on the location and time information.

[0050] In summary, this application first trains a teacher model on single-modal data and outputs a distillation scale. Leveraging the teacher model's powerful feature extraction capabilities, a high-quality knowledge base is constructed, providing accurate supervision signals for subsequent training and ensuring knowledge accuracy. Next, a student model based on the Mamba architecture is adopted. Utilizing the linear complexity of the state-space model, this overcomes the limitations of high computational complexity and large memory consumption of traditional Transformer models, significantly reducing the number of model parameters and computational resource consumption. This directly solves the technical problems of high computational cost and difficulty in deploying cross-modal target personnel trajectory localization models on edge devices in existing technologies. Finally, by introducing a total loss value including distillation loss for iterative training of the student model, the deep semantic knowledge of the teacher model in "image-text alignment" and "identity discrimination" is effectively transferred to the lightweight student model. This compensates for the shortcomings of the lightweight model in long-range dependency modeling capabilities due to structural simplification, thereby maintaining high-precision cross-modal retrieval performance while significantly reducing inference costs and model size, achieving a balance between low cost and high accuracy.

[0051] Secondly, the present invention provides a target personnel trajectory positioning device, comprising: a data acquisition module, a first positioning module, a second positioning module, and a third positioning module; The data acquisition module is used to acquire single-modal training sample data and dual-modal training sample data; wherein, each single-modal training sample in the single-modal training sample data includes a first pedestrian image, and each dual-modal training sample in the dual-modal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The first positioning module is used to input the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model, and to input each single-modal training sample in the single-modal training sample data into the teacher model in sequence, and output the first distillation scale of each single-modal training sample on the teacher side. The second positioning module is used to build a student model based on the Mamba architecture, and input the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and output the student model of the current iteration as the target person trajectory positioning model; wherein, the total loss value of the current iteration includes distillation loss, which is determined according to each of the first distillation scales and the second distillation scale of the current iteration; The third positioning module is used to acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

[0052] Example 2 like Figure 3 As shown, based on the above method embodiments, corresponding device embodiments are provided; One embodiment of the present invention provides a target personnel trajectory positioning device, including: a data acquisition module 31, a first positioning module 32, a second positioning module 33 and a third positioning module 34; The data acquisition module 31 is used to acquire single-modal training sample data and dual-modal training sample data; wherein, each single-modal training sample in the single-modal training sample data includes a first pedestrian image, and each dual-modal training sample in the dual-modal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The first positioning module 32 is used to input the single-modal training sample data into the first preset language image model for two-stage iterative training to obtain the teacher model, and input each single-modal training sample in the single-modal training sample data into the teacher model in sequence, and output the first distillation scale of each single-modal training sample on the teacher side. The second positioning module 33 is used to build a student model based on the Mamba architecture, and input the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and output the student model of the current iteration as the target person trajectory positioning model; wherein, the total loss value of the current iteration includes distillation loss, which is determined according to each of the first distillation scales and the second distillation scale of the current iteration; The third positioning module 34 is used to acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

[0053] In a preferred embodiment, the step of inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model includes: In the first stage of iterative training, the parameters of the first image encoder and the first text encoder in the first preset language image model are frozen, and each of the first pedestrian images is input into the first image encoder and the first text encoder for iterative training until the first total loss value of the current iteration meets the second preset condition. The first preset language image model of the current iteration is then output as the intermediate teacher model. In the second stage of iterative training, the parameters of the second image encoder in the intermediate teacher model are frozen and the parameters of the second text encoder in the intermediate teacher model are unfrozen. Each of the first pedestrian images is then input into the second image encoder and the second text encoder for iterative training until the second total loss value of the current iteration meets the third preset condition.

[0054] In a preferred embodiment, the step of sequentially inputting each unimodal training sample from the unimodal training sample data into the teacher model and outputting the first distillation scale of each unimodal training sample on the teacher side includes: The weight parameters of the teacher model are frozen, and each unimodal training sample in the unimodal training sample data is sequentially input into the teacher model for offline forward inference to obtain the teacher-side image features, teacher-side text features and teacher-side prediction probabilities corresponding to each unimodal training sample. Based on the teacher-side image features, teacher-side text features, and teacher-side prediction probabilities corresponding to each of the unimodal training samples, the first distillation scale of each of the unimodal training samples on the teacher side is obtained.

[0055] In a preferred embodiment, the step of inputting the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets a first preset condition, and outputting the student model of the current iteration as the target person trajectory localization model, includes: During each iteration, the second pedestrian image and its corresponding text description are input into the image encoder and text encoder of the student model in the current iteration, respectively, to obtain the student-side image features and student-side text features; Based on the student-side image features and the student-side text features, the identity recognition loss, triplet loss, and bidirectional contrast loss are calculated, and the total loss value of the current iteration is determined based on the identity recognition loss, triplet loss, bidirectional contrast loss, and distillation loss. The parameters of the student model in the current iteration are updated based on the total loss value of the current iteration until the total loss value of the current iteration satisfies a first preset condition.

[0056] In a preferred embodiment, the distillation loss is determined based on each of the first distillation scales and the second distillation scale of the current iteration, including: Obtain the teacher-side prediction probability, teacher-side image features, and teacher-side text features corresponding to the second distillation scale of the current iteration from each of the first distillation scales, and determine the student-side prediction probability, student-side image features, and student-side text features based on the second distillation scale of the current iteration; The soft label divergence loss is calculated based on the teacher-side prediction probability and the student-side prediction probability, and the feature mean squared error loss is calculated based on the teacher-side image features, the teacher-side text features, the student-side image features, and the student-side text features. The distillation loss is determined based on the soft label divergence loss and the feature mean squared error loss.

[0057] As a preferred embodiment, determining the distillation loss based on the soft-label divergence loss and the feature mean squared error loss includes: The teacher distance matrix is ​​determined based on the teacher-side image features and the teacher-side text features, and the student distance matrix is ​​determined based on the student-side image features and the student-side text features. The relationship preservation loss is calculated based on the distance between the teacher distance matrix and the student distance matrix, and the distillation loss is determined based on the soft label divergence loss, the feature mean squared error loss, and the relationship preservation loss.

[0058] In a preferred embodiment, after obtaining the location result corresponding to the pedestrian data, the method further includes: Based on the positioning results, the location and time information of the target person are determined, and the movement trajectory of the target person is generated based on the location and time information.

[0059] In summary, this application first trains a teacher model on single-modal data and outputs a distillation scale. Leveraging the teacher model's powerful feature extraction capabilities, a high-quality knowledge base is constructed, providing accurate supervision signals for subsequent training and ensuring knowledge accuracy. Next, a student model based on the Mamba architecture is adopted. Utilizing the linear complexity of the state-space model, this overcomes the limitations of high computational complexity and large memory consumption of traditional Transformer models, significantly reducing the number of model parameters and computational resource consumption. This directly solves the technical problems of high computational cost and difficulty in deploying cross-modal target personnel trajectory localization models on edge devices in existing technologies. Finally, by introducing a total loss value including distillation loss for iterative training of the student model, the deep semantic knowledge of the teacher model in "image-text alignment" and "identity discrimination" is effectively transferred to the lightweight student model. This compensates for the shortcomings of the lightweight model in long-range dependency modeling capabilities due to structural simplification, thereby maintaining high-precision cross-modal retrieval performance while significantly reducing inference costs and model size, achieving a balance between low cost and high accuracy.

[0060] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the target personnel trajectory positioning method provided by any of the above-described method embodiments of the present invention.

[0061] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0062] Example 3 Based on the above embodiments of the target personnel trajectory positioning method, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the target personnel trajectory positioning method of any embodiment of the present invention.

[0063] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0064] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0065] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0066] Example 4 Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the target personnel trajectory positioning method described in any of the above-described method embodiments of the present invention.

[0067] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0068] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A target person trajectory positioning method, characterized in that, include: Acquire unimodal training sample data and bimodal training sample data; wherein, each unimodal training sample in the unimodal training sample data includes a first pedestrian image, and each bimodal training sample in the bimodal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The single-modal training sample data is input into the first preset language image model and subjected to two-stage iterative training to obtain the teacher model. Each single-modal training sample in the single-modal training sample data is input into the teacher model in sequence, and the first distillation scale of each single-modal training sample on the teacher side is output. A student model is built based on the Mamba architecture, and the bimodal training sample data is input into the student model for iterative training until the total loss value of the current iteration meets the first preset condition. The student model of the current iteration is then output as the target person trajectory localization model. The total loss value includes distillation loss, which is determined based on each of the first distillation scales and the second distillation scale of the current iteration. Acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

2. The method of claim 1, wherein, The step of inputting the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model includes: In the first stage of iterative training, the parameters of the first image encoder and the first text encoder in the first preset language image model are frozen, and each of the first pedestrian images is input into the first image encoder and the first text encoder for iterative training until the first total loss value of the current iteration meets the second preset condition. The first preset language image model of the current iteration is then output as the intermediate teacher model. In the second stage of iterative training, the parameters of the second image encoder in the intermediate teacher model are frozen and the parameters of the second text encoder in the intermediate teacher model are unfrozen. Each of the first pedestrian images is then input into the second image encoder and the second text encoder for iterative training until the second total loss value of the current iteration meets the third preset condition.

3. The method of claim 1, wherein, The step of sequentially inputting each unimodal training sample from the unimodal training sample data into the teacher model and outputting the first distillation scale of each unimodal training sample on the teacher side includes: The weight parameters of the teacher model are frozen, and each unimodal training sample in the unimodal training sample data is sequentially input into the teacher model for offline forward inference to obtain the teacher-side image features, teacher-side text features and teacher-side prediction probabilities corresponding to each unimodal training sample. Based on the teacher-side image features, teacher-side text features, and teacher-side prediction probabilities corresponding to each of the unimodal training samples, the first distillation scale of each of the unimodal training samples on the teacher side is obtained.

4. The method of claim 1, wherein, The step of inputting the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and outputting the student model of the current iteration as the target person trajectory localization model, includes: During each iteration, the second pedestrian image and its corresponding text description are input into the image encoder and text encoder of the student model in the current iteration, respectively, to obtain the student-side image features and student-side text features; Based on the student-side image features and the student-side text features, the identity recognition loss, triplet loss, and bidirectional contrast loss are calculated, and the total loss value of the current iteration is determined based on the identity recognition loss, triplet loss, bidirectional contrast loss, and distillation loss. The parameters of the student model in the current iteration are updated based on the total loss value of the current iteration until the total loss value of the current iteration satisfies a first preset condition.

5. The method of claim 1, wherein, The distillation loss is determined based on each of the first distillation scales and the second distillation scale of the current iteration, including: Obtain the teacher-side prediction probability, teacher-side image features, and teacher-side text features corresponding to the second distillation scale of the current iteration from each of the first distillation scales, and determine the student-side prediction probability, student-side image features, and student-side text features based on the second distillation scale of the current iteration; The soft label divergence loss is calculated based on the teacher-side prediction probability and the student-side prediction probability, and the feature mean squared error loss is calculated based on the teacher-side image features, the teacher-side text features, the student-side image features, and the student-side text features. The distillation loss is determined based on the soft label divergence loss and the feature mean squared error loss.

6. A method of target personnel trajectory positioning as claimed in claim 5, characterized in that, The step of determining the distillation loss based on the soft-label divergence loss and the feature mean squared error loss includes: The teacher distance matrix is ​​determined based on the teacher-side image features and the teacher-side text features, and the student distance matrix is ​​determined based on the student-side image features and the student-side text features. The relationship preservation loss is calculated based on the distance between the teacher distance matrix and the student distance matrix, and the distillation loss is determined based on the soft label divergence loss, the feature mean squared error loss, and the relationship preservation loss.

7. The method of claim 1, wherein, After obtaining the location result corresponding to the pedestrian data, the process further includes: Based on the positioning results, the location and time information of the target person are determined, and the movement trajectory of the target person is generated based on the location and time information.

8. A target person trajectory positioning apparatus characterized by comprising: include: The system includes a data acquisition module, a first positioning module, a second positioning module, and a third positioning module. The data acquisition module is used to acquire single-modal training sample data and dual-modal training sample data; wherein, each single-modal training sample in the single-modal training sample data includes a first pedestrian image, and each dual-modal training sample in the dual-modal training sample data includes a second pedestrian image and a text description corresponding to the second pedestrian image; The first positioning module is used to input the single-modal training sample data into a first preset language image model for two-stage iterative training to obtain a teacher model, and to input each single-modal training sample in the single-modal training sample data into the teacher model in sequence, and output the first distillation scale of each single-modal training sample on the teacher side. The second positioning module is used to build a student model based on the Mamba architecture, and input the bimodal training sample data into the student model for iterative training until the total loss value of the current iteration meets the first preset condition, and output the student model of the current iteration as the target person trajectory positioning model; wherein, the total loss value of the current iteration includes distillation loss, which is determined according to each of the first distillation scales and the second distillation scale of the current iteration; The third positioning module is used to acquire pedestrian data to be located, and input the pedestrian data into the target person trajectory positioning model to obtain the positioning result corresponding to the pedestrian data.

9. A terminal device, comprising: The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements the target personnel trajectory localization method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the target personnel trajectory positioning method as described in any one of claims 1-7.

Images