A semantic multi-target tracking method, device and equipment for railway platforms

Abstract

This invention relates to the field of track sensing and provides a method, apparatus, and device for semantic multi-target tracking at track platforms. The method includes: acquiring a first image with a resolution lower than a first preset threshold; processing the first image with a resolution lower than the first preset threshold using a diffusion inversion method and super-resolution technology to obtain a second image with a resolution higher than a second preset threshold; removing noise from the second image using a trained noise removal model; annotating the second image with target trajectories to generate target trajectories; and performing a semantic understanding task based on the trained multi-target tracking model to generate descriptions of the target trajectories and annotations of their interaction relationships. This invention solves the problem of insufficient target tracking accuracy in low-resolution images in existing technologies and improves the accuracy and robustness of multi-target tracking in track platform scenarios.

Description

Technical Field

[0001] This invention relates to the field of track sensing technology, and in particular to a method, apparatus and equipment for semantic multi-target tracking of track platforms. Background Technology

[0002] With the rapid development of rail transit, rail stations, as important transportation hubs, face increasingly complex management and safety assurance tasks. Against this backdrop, target tracking systems at rail stations have become a crucial technical means to improve operational efficiency and ensure passenger safety. Especially in real-time monitoring, maintaining high-precision multi-target tracking in complex environments is a pressing challenge. Traditional target tracking methods often face problems such as target occlusion, blurring, and decreased motion tracking accuracy when dealing with large crowds and high-density targets, placing higher demands on rail station monitoring systems.

[0003] Currently, target tracking technologies at railway stations largely rely on image processing and deep learning methods. While some progress has been made, several bottlenecks remain in practical applications. For example, traditional super-resolution techniques, although improving image quality, place a heavy computational burden on real-time image streams and lack sufficient robustness, making it difficult to cope with changes in dynamic environments. In multi-target tracking, existing methods exhibit low accuracy and stability in complex situations such as multiple targets crossing each other, occlusion, and changes in target appearance. Especially with low-resolution images, significant loss of target detail further degrades tracking accuracy.

[0004] Therefore, existing technologies still cannot meet the demand for high-precision, multi-target tracking in the context of railway station scenarios. Summary of the Invention

[0005] This invention provides a semantic multi-target tracking method, apparatus, and equipment for railway stations, which solves the problem of insufficient target tracking accuracy in low-resolution images in the prior art and improves the accuracy and robustness of multi-target tracking in railway station scenarios.

[0006] A semantic multi-target tracking method for railway stations includes:

[0007] A first image with a resolution lower than a first preset threshold is acquired, and the first image with a resolution lower than the first preset threshold is processed based on the diffusion inversion method and super-resolution technology to obtain a second image with a resolution higher than a second preset threshold.

[0008] The noise in the second image is removed according to a trained noise removal model, which is trained based on a first training set, which includes the second image with noise.

[0009] Annotate the target trajectory in the noise-removed second image to generate the target trajectory;

[0010] Based on the trained multi-target tracking model, a semantic understanding task is performed according to the target trajectory to generate a description of the target trajectory and annotations of the interaction relationship; the trained multi-target tracking model is trained on a second training set, which includes a second image, target trajectory annotations, and annotation data for the semantic understanding task.

[0011] According to the present invention, a semantic multi-target tracking method for railway stations is provided. The method, which processes a first image with a resolution lower than a first preset threshold using a diffusion inversion method and a super-resolution technique to obtain a second image with a resolution higher than a second preset threshold, specifically includes: inputting the first image into a trained noise predictor and determining an optimal noise mapping set based on a diffusion acceleration algorithm; gradually acquiring a transition image corresponding to the first image by inverting the trajectory according to the optimal noise mapping set; inputting the transition image into a trained super-resolution model and outputting the second image.

[0012] According to the present invention, a semantic multi-target tracking method for railway stations is provided, wherein training the noise removal model based on a first training set specifically includes: acquiring the second image, injecting noise into the second image through a random noise model to generate a noisy sample image; determining a loss function, wherein the loss function is determined based on mean squared error loss, perceptual loss and generative adversarial network loss; training the noise removal model with the objective of minimizing the loss function, and outputting the trained noise removal model.

[0013] According to the present invention, a semantic multi-target tracking method for railway stations is provided. The step of annotating the target trajectory of the noise-removed second image to generate the target trajectory specifically includes: identifying each target in each frame of the noise-removed second image; generating an initial trajectory for each target based on the identified targets; determining the target features of each target in the initial trajectory based on the target coordinates and image features within the initial trajectory; fusing the target features into the target trajectory features; and generating a complete trajectory for each target based on the fused target trajectory features.

[0014] According to the present invention, a semantic multi-target tracking method for railway platforms is provided. The method, based on a trained multi-target tracking model, performs a semantic understanding task according to the target trajectory to generate descriptions and annotations of the target trajectory's interaction relationships. Specifically, the method includes: inputting the generated target trajectory into the trained multi-target tracking model; sequentially injecting target trajectory features into image features with cross-interest to generate image captions describing the entire image content; generating instance captions describing the behavior of a single target trajectory based on the target trajectory features; identifying the interaction relationship between two targets and generating labels describing the interaction state; and generating annotations describing the target trajectory and its interaction relationships based on the image captions, instance captions, and the interaction recognition labels.

[0015] According to the present invention, a semantic multi-target tracking method for railway platforms is provided. The step of training a multi-target tracking model based on a second training set specifically includes: constructing a multi-target tracking model and initializing the parameters of the multi-target tracking model; the multi-target tracking model includes a target detection module, a trajectory generation module, and a semantic understanding module; determining a multi-task loss function, the multi-task loss function being determined based on target detection loss, trajectory generation loss, and semantic understanding loss; and performing end-to-end training of the multi-target tracking model based on the second training set with the objective of minimizing the multi-task loss function, thereby obtaining a trained multi-target tracking model.

[0016] According to the present invention, a semantic multi-target tracking method for railway platforms includes the following steps: inputting the first image into a trained noise predictor and determining the optimal noise mapping set based on a diffusion acceleration algorithm. Specifically, this includes: inputting the first image with a resolution lower than a first preset threshold into the trained noise predictor; the noise predictor employing a variational autoencoder reparameterization technique to predict the Gaussian distribution parameters of the noise; constructing a noise mapping set and reducing the number of noise mappings in the noise mapping set based on a local noise prediction strategy; and further compressing the noise mapping set based on the reduced number of noise mappings using a diffusion acceleration algorithm to obtain the optimal noise mapping set.

[0017] The present invention also provides a semantic multi-target tracking device for railway platforms, comprising the following modules:

[0018] The diffusion inversion module is used to acquire a first image with a resolution lower than a first preset threshold, and process the first image with a resolution lower than the first preset threshold based on the diffusion inversion method and super-resolution technology to obtain a second image with a resolution higher than a second preset threshold.

[0019] A noise removal module is used to remove noise from the second image according to a trained noise removal model, wherein the noise removal model is trained according to a first training set, and the first training set includes the second image with noise.

[0020] The target trajectory generation module is used to annotate the second image after noise removal and generate the target trajectory.

[0021] The target tracking module is used to perform a semantic understanding task based on the target trajectory according to the trained multi-target tracking model, and generate a description of the target trajectory and annotations of the interaction relationship; the trained multi-target tracking model is trained according to a second training set, which includes a second image, target trajectory annotations and annotation data of the semantic understanding task.

[0022] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the semantic multi-target tracking method for track stations as described above.

[0023] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the semantic multi-target tracking method for track platforms as described above.

[0024] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the semantic multi-target tracking method for track stations as described above.

[0025] This invention provides a semantic multi-target tracking method, apparatus, and device for railway stations, which has the following beneficial effects: By using diffusion inversion method and super-resolution technology to process low-resolution images, the resolution and detail information of the images are significantly improved, and noise interference in the images is effectively removed by combining a noise removal model, thereby improving image quality; furthermore, through target trajectory annotation and multi-target tracking model, the accurate description of target trajectories and semantic understanding of interaction relationships are realized, ultimately solving the problem of insufficient target tracking accuracy under low-resolution images, and significantly improving the accuracy, robustness, and semantic understanding ability of multi-target tracking in railway station scenarios. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating the semantic multi-target tracking method for railway platforms provided by the present invention.

[0028] Figure 2This is a schematic diagram of the structure of the semantic multi-target tracking device for railway platforms provided by the present invention.

[0029] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] With the rapid development of rail transit, rail stations, as important transportation hubs, face increasingly complex management and safety assurance tasks. Against this backdrop, target tracking systems at rail stations have become a crucial technological means to improve operational efficiency and ensure passenger safety. Traditional target tracking methods often encounter problems such as target occlusion, blurring, and decreased motion tracking accuracy when dealing with large crowds and high-density targets, placing higher demands on rail station monitoring systems. Especially in real-time monitoring, maintaining tracking accuracy in complex environments is a challenge that urgently needs to be addressed.

[0032] Currently, target tracking technologies at railway stations largely rely on image processing and deep learning methods. While some progress has been made, several bottlenecks remain in practical applications. For example, traditional super-resolution techniques, although improving image quality, place a heavy computational burden on real-time image streams and lack sufficient robustness, making it difficult to handle changes in dynamic environments. Furthermore, in multi-target tracking, existing methods exhibit low accuracy and stability in complex situations such as multiple people crossing paths, occlusion, and changes in target appearance. Therefore, current technologies still cannot meet the demands for high-precision, multi-target tracking in railway station scenarios.

[0033] The "Semantic Multi-Target Tracking Method for Railway Stations Using a Diffusion Inversion Super-Resolution Strategy" proposed in this invention innovatively combines diffusion inversion and super-resolution techniques, overcoming several bottlenecks in existing technologies. The diffusion inversion strategy effectively improves the detail and clarity of the target image, enabling high-precision target recognition even at low resolution. Meanwhile, the super-resolution technique enhances image quality while reducing errors caused by low resolution, significantly improving tracking accuracy and anti-interference capabilities. Compared to traditional methods, this invention provides more accurate and real-time tracking in complex and dynamic environments such as railway stations, exhibiting greater adaptability and application prospects.

[0034] In summary, this invention provides a semantic multi-target tracking method for railway stations using a diffusion inversion super-resolution strategy, which effectively improves the accuracy, real-time performance, and robustness of multi-target tracking at railway stations, and overcomes the shortcomings of traditional methods in complex environments.

[0035] This invention combines diffusion inversion methods with super-resolution techniques and trains a noise predictor to significantly improve image resolution. Then, the BenSMOT tool (a tool for multi-object tracking and semantic understanding, primarily used for target trajectory annotation and semantic information generation, supporting the training and evaluation of multi-object tracking models) is used to annotate target trajectories on high-resolution images. Through a multi-object tracking model, trajectory-related semantic understanding is achieved, and the model output is optimized through end-to-end training of the multi-object tracking model. This significantly improves the quality and tracking accuracy of low-resolution images obtained in complex environments such as railway stations, providing a sound theoretical foundation for a practical application scenario.

[0036] To achieve the above objectives, the present invention adopts the following technical solution: a semantic multi-target tracking method for railway stations using a diffusion inversion super-resolution strategy, the method comprising the following sequential steps:

[0037] (1) The diffusion inversion method is used in combination with super-resolution technology to process the input low-resolution image;

[0038] (2) Train the noise predictor and finally output a high-resolution image;

[0039] (3) Use the BenSMOT tool to annotate the target trajectory on the image;

[0040] (4) Implement trajectory-related semantic understanding tasks using a multi-target tracking model;

[0041] (5) The image processing, trajectory annotation and semantic understanding modules are used for end-to-end training of the multi-target tracking model;

[0042] (6) Extract the features of each target trajectory and realize the semantic understanding of instance captions, interactive recognition and image captions through an additional prediction head.

[0043] The following is combined Figures 1-3 The embodiments of the present invention are described in detail.

[0044] Figure 1 This is a flowchart illustrating the semantic multi-target tracking method for railway platforms provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0045] S110. Obtain a first image with a resolution lower than a first preset threshold, process the first image with a resolution lower than the first preset threshold based on the diffusion inversion method and super-resolution technology, and obtain a second image with a resolution higher than a second preset threshold.

[0046] According to the present invention, a semantic multi-target tracking method for railway stations is provided. Based on the diffusion inversion method and super-resolution technology, a first image with a resolution lower than a first preset threshold is processed to obtain a second image with a resolution higher than a second preset threshold. Specifically, the method includes: inputting the first image into a trained noise predictor and determining the optimal noise mapping set based on the diffusion acceleration algorithm; gradually obtaining the corresponding transition image of the first image by inverting the trajectory according to the optimal noise mapping set; inputting the transition image into a trained super-resolution model and outputting the second image.

[0047] According to the semantic multi-target tracking method for railway platforms provided by the present invention, a first image is input into a trained noise predictor, and an optimal noise mapping set is determined based on a diffusion acceleration algorithm. Specifically, the method includes: inputting a first image with a resolution lower than a first preset threshold into a trained noise predictor; the noise predictor using the reparameterization technique of a variational autoencoder to predict the Gaussian distribution parameters of the noise; constructing a noise mapping set and reducing the number of noise mappings in the noise mapping set based on a local noise prediction strategy; and further compressing the noise mapping set based on the reduced number of noise mappings using a diffusion acceleration algorithm to obtain the optimal noise mapping set.

[0048] Specifically, low-resolution images are obtained through image acquisition systems (such as surveillance cameras, sensors, etc.); by performing multiple backpropagations (or inverse diffusion) on local areas of the image, the gradient of the image is processed, thereby enhancing the details of the image and gradually recovering the lost high-frequency information.

[0049] Specifically, a parameter is introduced as The noise predictor, denoted as Its input is a low-resolution (LR) image. and time step It employs the reparameterization technique of variational autoencoder (VAE) to predict the mean and variance parameters of a Gaussian distribution, ensuring that the output maps the desired noise. It conforms to a Gaussian distribution, where the noise mapping set Depend on A noise mapping (typically in most current diffusion models) It consists of ) and corresponds to each step of the diffusion process.

[0050] Next, a local noise prediction (PnP) strategy is used to predict noise from the intermediate time step. Start the sampling process and reduce Zhong Cong arrive The number of noise maps, i.e., the selection and will The constraint is to have a relatively high signal-to-noise ratio, encouraging the selection of a smaller one. Value, in practice, is generally considered to be Set as This corresponds to a signal-to-noise ratio threshold of 1.44 in the widely used stable diffusion.

[0051] Furthermore, to further accelerate the diffusion process, existing diffusion acceleration algorithms are integrated to further compress the noise mapping set. ,in , refers to the element index in the noise map set, and in practice it is usually... Set as This reduces the prediction burden on the noise predictor.

[0052] Given a noise mapping set and noise predictor The goal is to base the results on given low-resolution (LR) observations. Restore high-resolution (HR) images Specifically, from LR images Build the initial state And through trajectory inversion Gradually restore high-resolution images; in the context of image super-resolution (SR), due to HR images It is inaccessible during testing, therefore a noise predictor is used. Directly from LR images Build similar Formula, i.e. Thus, an optimal noise mapping is found. To perturb low-resolution images This enables the pre-trained diffusion model to... Generate the corresponding The optimal noise mapping set for reconstructing a given image after processing with a diffusion model is determined, and the high-frequency components of the image are ultimately recovered.

[0053] Subsequently, super-resolution technology is used to further enhance image quality. This involves training a neural network model to infer structural information of a high-resolution image from a low-resolution image. The formula for super-resolution reconstruction is expressed as follows: This maps each pixel of a low-resolution image to a high-resolution space, where... This refers to a high-resolution image, that is, an image obtained through super-resolution reconstruction. This indicates a low-resolution image, meaning the input image to be processed has a resolution lower than a preset threshold. The super-resolution reconstruction function is a neural network model responsible for inferring structural information of a high-resolution image from a low-resolution image. This represents the model parameters, i.e. the learnable parameters in the super-resolution reconstruction function, which are obtained through optimization during the training process.

[0054] By combining diffusion inversion and super-resolution techniques, the quality and detail recovery capabilities of low-resolution images are significantly improved. First, by introducing a noise predictor and a local noise prediction (PnP) strategy, the model can progressively recover lost high-frequency information in the image, enhancing its detail representation. Second, utilizing the reparameterization technique of a variational autoencoder (VAE) and a diffusion acceleration algorithm, the noise mapping generation process is optimized, reducing computational complexity and improving processing efficiency. Next, through trajectory inversion and super-resolution reconstruction, the model can infer structural information from high-resolution images, further improving image resolution and sharpness. Ultimately, these methods work together to effectively address the problems of detail loss and noise interference in low-resolution images, and also provide high-quality input data for subsequent target tracking and semantic understanding tasks, thereby improving the overall system performance and robustness.

[0055] S120. Remove noise from the second image using the trained noise removal model. The noise removal model is trained using a first training set, which includes the second image with noise.

[0056] According to the present invention, a semantic multi-target tracking method for railway stations is provided, which trains a noise removal model based on a first training set. Specifically, the method includes: acquiring a second image; injecting noise into the second image through a random noise model to generate a noisy sample image; determining a loss function based on mean squared error loss, perceptual loss, and generative adversarial network loss; training the noise removal model with the goal of minimizing the loss function; and outputting the trained noise removal model.

[0057] Specifically, the input consists of an unprocessed low-resolution image and a high-resolution image obtained in the previous step. A random noise model is introduced and noise is injected into the high-resolution image to generate training samples with noise.

[0058] Set the preselected time step set as Mean square error loss Perceived loss A Generative Adversarial Network (GAN) loss The total loss function is then defined as: By minimizing the loss function, the noise removal model learns how to remove noise from an image and restore the true details of the image to the greatest extent. Finally, the learned noise removal model extracts and removes noise components from the input image, outputting a high-quality, high-resolution image.

[0059] By introducing a random noise model and multiple loss functions, the training performance and image restoration quality of the noise removal model are significantly improved. First, random noise is injected into high-quality images to generate training samples, simulating noise interference in real-world scenes and enhancing the model's robustness. Second, by combining mean squared error loss, perceptual loss, and generative adversarial network (GAN) loss, the model can not only restore image details at the pixel level but also maintain the visual quality of the image at the perceptual level, while further enhancing the realism of the image using the adversarial mechanism of GAN. Finally, by minimizing the total loss function, the noise removal model can effectively learn the noise distribution and remove noise, outputting high-quality, high-resolution images. This method not only improves the image denoising capability but also provides clearer input data for subsequent target tracking and semantic understanding tasks, thereby improving the overall system performance.

[0060] S130. Annotate the target trajectory on the second image after noise removal to generate the target trajectory.

[0061] According to the present invention, a semantic multi-target tracking method for railway stations is provided, which annotates the target trajectory of a second image after noise removal to generate a target trajectory. Specifically, the method includes: identifying each target in each frame of the second image after noise removal; generating an initial trajectory for each target based on the identified target; determining the target features of each target in the initial trajectory based on the target coordinates and image features within the initial trajectory; fusing the target features into the target trajectory features; and generating a complete trajectory for each target based on the fused target trajectory features.

[0062] Specifically, for each target in the image, two different fusion modules are used to combine the features of a single object into dynamic features such as the target's spatial location, velocity, orientation, and trajectory length.

[0063] Specifically, given an N-frame image, the multi-object tracking model first uses a CNN backbone to extract their features. ,in Indicates the first The frame features are then analyzed, and an initial proposal set is generated using a proposal generator in a popular detection architecture, and then passed through a threshold. Threshold filtering is performed to retain target proposals that meet the criteria. Then, all retained target proposals are sent to the association module (using the byte association algorithm in the multi-target tracking model) to generate the initial trajectory for each target. Finally, using the target coordinates and image features within the trajectory, the target features of each target in the trajectory are obtained through Region Interest Pooling (RoI Pooling).

[0064] Next, the semantic details related to the trajectory in the image are predicted, including instance descriptions, instance interactions, and overall image descriptions. Specifically, feature fusion is first performed. Based on the features of the previously obtained image and target, an attention-based image fusion module (VFM) is used to merge image-level features into overall image features, and a concatenated trajectory fusion module (TFM) is used to integrate the target features of each frame into the target trajectory features. The image fusion module integrates the fused image features... To the Frame calculation is In the formula, CA(·) represents the cross-attention mechanism, which is used to fuse features from the previous frame. Features of the current frame Perform fusion. Combine features from N frames. The input VFM is sequentially fed into the image fusion module for image feature fusion, ultimately generating the overall fused image features. The orbit fusion module utilizes the given first... Target features of trajectory j in frame Average pooling is then used to obtain a relatively simple two-dimensional feature. ,in Dimension Then, for the first Each target trajectory is connected to its discrete features into a complete feature. Then, the trajectory features are enhanced by utilizing the self-attention mechanism (SA). .

[0065] By employing feature extraction, fusion, and trajectory generation in a multi-target tracking model, the accuracy and semantic understanding capabilities of target tracking are significantly improved. First, image features are extracted using a CNN backbone, and combined with a proposal generator and threshold filtering to select high-quality target proposals, reducing interference from noise and low-confidence targets. Next, an initial trajectory is generated using a byte association algorithm, and target features are extracted through Region of Interest Pooling (RoI Pooling) to ensure trajectory continuity and accuracy. Subsequently, an attention-based image fusion module (VFM) and a concatenated trajectory fusion module (TFM) fuse image-level features and trajectory features, enhancing the model's ability to capture dynamic target features (such as velocity, direction, and trajectory length). Finally, a self-attention mechanism (SA) is used to further optimize trajectory features, providing high-quality feature representations for subsequent semantic understanding tasks (such as instance illustration and interaction recognition). These methods work together to not only improve the accuracy and robustness of target tracking but also lay a solid foundation for semantic understanding tasks in complex scenarios.

[0066] S140. Based on the trained multi-target tracking model, perform a semantic understanding task according to the target trajectory to generate a description of the target trajectory and annotations of interaction relationships. The trained multi-target tracking model is trained on a second training set, which includes the second image, target trajectory annotations, and annotation data for the semantic understanding task.

[0067] According to the present invention, a semantic multi-target tracking method for railway stations is provided. Based on a trained multi-target tracking model, a semantic understanding task is performed according to the target trajectory to generate descriptions and annotations of interaction relationships of the target trajectory. Specifically, the method includes: inputting the generated target trajectory into the trained multi-target tracking model; sequentially injecting the target trajectory features into image features with cross-interest to generate image captions describing the entire image content; generating instance captions describing the behavior of a single target trajectory based on the target trajectory features; identifying the interaction relationship between two targets and generating labels describing the interaction state; and generating annotations of descriptions and interaction relationships of the target trajectory based on the image captions, instance captions, and interaction recognition labels.

[0068] Specifically, multi-target tracking models utilize fused image features. Trajectory features This involves implementing semantic understanding tasks related to trajectory, namely image captioning, instance captioning, and interaction recognition. Image captioning generation, in particular, involves a multi-object tracking model sequentially processing trajectory features... Inject image features with cross-interest Then, the caption results are predicted through a linear item layer and a text decoder. The example captions are generated directly based on the target trajectory features, that is, given the first... strip Trajectory characteristics Apply a linear projection layer to the target trajectory features After converting to an appropriate dimension, the designed text decoder is used to predict the annotation result for each trajectory, as shown in the formula. Interaction recognition in multi-target tracking models focuses only on the interaction between two trajectories; that is, for two trajectories... and Suppose the first image in the image sequence... The first trajectory is an active trajectory, the second... The trajectory is a passive trajectory. First, a cross-attention mechanism is used to fuse the features of the active and passive trajectories. Then, a multilayer perceptron (MLP) is used to process the fused features to predict the interaction relationships between the trajectories. ,as follows: ,in, This refers to a multi-layer perceptron, which is used to process fused features and predict the interaction relationships between trajectories.

[0069] By fusing image and trajectory features, the multi-object tracking model can efficiently complete trajectory-related semantic understanding tasks, including image caption generation, instance caption generation, and interaction recognition, thus significantly improving the model's ability to understand complex scenes. First, by injecting trajectory features into image features with a cross-attention mechanism, the model can generate global image captions, describing the scene content holistically and enhancing the model's grasp of the global semantics of the image. Second, instance caption generation is directly based on target trajectory features. Through a linear projection layer and a text decoder, the model can generate accurate behavioral descriptions for each target trajectory, providing more granular semantic information. Finally, the interaction recognition task fuses features from active and passive trajectories through a cross-attention mechanism and utilizes a multilayer perceptron (MLP) to predict the interaction relationships between trajectories, enabling the model to accurately capture dynamic interaction behaviors between targets. These functions work together to not only improve the performance of the multi-object tracking model in trajectory detection and semantic understanding tasks but also better adapt it to the application needs of complex scenes (such as railway platforms), providing strong technical support for target behavior analysis and interaction understanding in real-world scenarios.

[0070] According to the present invention, a semantic multi-target tracking method for railway stations is provided, which trains a multi-target tracking model based on a second training set. Specifically, the method includes: constructing a multi-target tracking model and initializing the parameters of the multi-target tracking model; the multi-target tracking model includes a target detection module, a trajectory generation module, and a semantic understanding module; determining a multi-task loss function, which is determined based on the target detection loss, trajectory generation loss, and semantic understanding loss; and performing end-to-end training of the multi-target tracking model based on the second training set with the goal of minimizing the multi-task loss function, to obtain a trained multi-target tracking model.

[0071] Specifically, for each frame of image, the loss for object detection is first calculated, which includes the prediction error of object position and category. After detecting objects in the entire image sequence, the position of each object on the time axis is matched using a trajectory association algorithm to form a complete trajectory. Then, for each trajectory, the loss related to instance captions, interaction recognition, and image captions is calculated. The losses of the above tasks are combined and end-to-end training of the multi-object tracking model is performed. By minimizing the total loss, the various parts of the network are optimized, thereby improving the overall performance of trajectory detection, annotation, and semantic understanding tasks.

[0072] By combining the losses from multiple tasks such as object detection, trajectory association, instance caption generation, interaction recognition, and image caption generation for end-to-end training, the overall performance of the model can be effectively optimized. This multi-task joint training approach enables the model to learn feature representations from multiple related tasks simultaneously, improving its generalization ability and robustness.

[0073] For each target trajectory in the image, similar to standard multi-object tracking (MOT) benchmarks, an axis-aligned bounding box is provided to indicate its spatial location in the image. Each trajectory is labeled with precise sentences in natural language to describe the detailed behavior of the associated target. For interaction recognition, 335 interactions (i.e., verbs, 327 of which are from WordNet) are first collected. Then, a targeted interaction label is provided for each trajectory interaction, represented by a triple of <subject, verb, object>. If the associated objects have an interaction relationship, the interaction state is described using the above triple. For the entire image, a concise text summary is generated to describe the target trajectories and scene content in the image from a global perspective. To ensure high-quality annotation, a multi-round mechanism is adopted. Each image is first manually annotated by volunteers familiar with the task and an expert researching the relevant problem. After this first round, all annotations for the image are sent to a validation team by two or more experts for review. If the initial annotations do not receive unanimous agreement from the experts, they are returned to the original annotation team for refinement using the validation team's feedback. This process is repeated for each image until the annotation of all image sequences in BenSMOT is completed.

[0074] By providing precise target trajectory annotations and rich semantic information, the training and evaluation quality of the multi-object tracking model is significantly improved. First, by providing axis-aligned bounding boxes and precise natural language descriptions for each target trajectory, the spatial location and behavioral details of the target in the image are clearly indicated, thereby enhancing the model's understanding of the target trajectory. Second, by collecting 335 interaction verbs and providing directional labels (in the form of <subject, verb, object> triples) for each trajectory interaction, the model can accurately capture the interaction relationships between targets, further improving the depth and accuracy of semantic understanding. Furthermore, generating a text summary of the global image describes the scene content from a holistic perspective, enabling the model to better understand the contextual information of complex scenes. To ensure high-quality annotation, a multi-round annotation and validation mechanism is employed to ensure the accuracy and consistency of the annotations, thus providing reliable data support for the training and evaluation of the multi-object tracking model.

[0075] The semantic multi-target tracking device for railway platforms provided by the present invention is described below. The semantic multi-target tracking device for railway platforms described below can be referred to in correspondence with the semantic multi-target tracking method for railway platforms described above.

[0076] like Figure 2 The image shows a semantic multi-target tracking device for a railway station provided by the present invention, comprising:

[0077] The diffusion inversion module 210 is used to acquire a first image with a resolution lower than a first preset threshold, process the first image with a resolution lower than the first preset threshold based on the diffusion inversion method and super-resolution technology, and obtain a second image with a resolution higher than a second preset threshold.

[0078] The noise removal module 220 is used to remove noise from the second image according to the trained noise removal model. The noise removal model is trained according to the first training set, which includes the second image with noise.

[0079] The target trajectory generation module 230 is used to annotate the second image after noise removal to generate the target trajectory.

[0080] The target tracking module 240 is used to perform a semantic understanding task based on the target trajectory according to the trained multi-target tracking model, and generate a description of the target trajectory and annotations of the interaction relationship; the trained multi-target tracking model is trained according to the second training set, which includes the second image, target trajectory annotations and annotation data of the semantic understanding task.

[0081] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a semantic multi-target tracking method for a track station. This method includes: acquiring a first image with a resolution lower than a first preset threshold; processing the first image with a resolution lower than the first preset threshold based on a diffusion inversion method and a super-resolution technique to obtain a second image with a resolution higher than a second preset threshold; removing noise from the second image according to a trained noise removal model, the noise removal model being trained based on a first training set, the first training set including the second image with noise; annotating the second image with target trajectories to generate target trajectories; and performing a semantic understanding task based on the trained multi-target tracking model according to the target trajectories to generate descriptions of the target trajectories and annotations of their interaction relationships; the trained multi-target tracking model being trained based on a second training set, the second training set including the second image, target trajectory annotations, and annotation data for the semantic understanding task.

[0082] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-target semantic tracking method for railway stations provided by the above methods. The method includes: acquiring a first image with a resolution lower than a first preset threshold; processing the first image with a resolution lower than the first preset threshold based on a diffusion inversion method and a super-resolution technique to obtain a second image with a resolution higher than a second preset threshold; removing noise from the second image according to a trained noise removal model, wherein the noise removal model is trained according to a first training set, the first training set including the second image with noise; annotating the second image with target trajectories after noise removal to generate target trajectories; performing a semantic understanding task based on the trained multi-target tracking model according to the target trajectory to generate descriptions and annotations of interaction relationships of the target trajectory; wherein the trained multi-target tracking model is trained according to a second training set, the second training set including the second image, target trajectory annotations, and annotation data of the semantic understanding task.

[0084] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the semantic multi-target tracking method for railway stations provided by the above methods. The method includes: acquiring a first image with a resolution lower than a first preset threshold; processing the first image with a resolution lower than the first preset threshold based on a diffusion inversion method and a super-resolution technique to obtain a second image with a resolution higher than a second preset threshold; removing noise from the second image according to a trained noise removal model, the noise removal model being trained according to a first training set, the first training set including the second image with noise; annotating the second image with target trajectories after noise removal to generate target trajectories; performing a semantic understanding task based on the trained multi-target tracking model according to the target trajectory to generate descriptions and annotations of interaction relationships of the target trajectory; the trained multi-target tracking model being trained according to a second training set, the second training set including the second image, target trajectory annotations, and annotation data of the semantic understanding task.

[0085] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0086] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic multi-target tracking method for railway stations, characterized in that, include: The process involves acquiring a first image with a resolution lower than a first preset threshold, processing the first image with a resolution lower than the first preset threshold using a diffusion inversion method and a super-resolution technique to obtain a second image with a resolution higher than a second preset threshold. This includes: inputting the first image into a trained noise predictor, wherein the noise predictor employs a variational autoencoder reparameterization technique to predict the Gaussian distribution parameters of the noise; determining an optimal noise mapping set based on a local noise prediction strategy and a diffusion acceleration algorithm; progressively acquiring a transition image corresponding to the first image through an inversion trajectory based on the optimal noise mapping set; and inputting the transition image into a trained super-resolution model to output the second image. The noise in the second image is removed according to a trained noise removal model, which is trained based on a first training set, which includes the second image with noise. Annotate the target trajectory in the noise-removed second image to generate the target trajectory; Based on the trained multi-target tracking model, a semantic understanding task is performed according to the target trajectory to generate a description of the target trajectory and annotations of the interaction relationship; the trained multi-target tracking model is trained on a second training set, which includes a second image, target trajectory annotations, and annotation data for the semantic understanding task.

2. The semantic multi-target tracking method for railway platforms according to claim 1, characterized in that, Training the noise removal model based on the first training set specifically includes: The second image is acquired, and noise is injected into the second image using a random noise model to generate a noisy sample image. Determine the loss function, which is based on the mean squared error loss, the perception loss, and the generative adversarial network loss; The noise removal model is trained with the goal of minimizing the loss function, and the trained noise removal model is output.

3. The semantic multi-target tracking method for railway platforms according to claim 1, characterized in that, The step of annotating the second image after noise removal to generate the target trajectory specifically includes: Identify each target in each frame of the second image after noise removal; Generate an initial trajectory for each target based on the identified targets; Based on the target coordinates and image features within the initial trajectory, determine the target features of each target in the initial trajectory; The target features are fused into the target trajectory features; Based on the fused target trajectory features, a complete trajectory for each target is generated.

4. The semantic multi-target tracking method for railway platforms according to claim 1, characterized in that, The pre-trained multi-target tracking model performs a semantic understanding task based on the target trajectory to generate descriptions of the target trajectory and annotations of interaction relationships, specifically including: The generated target trajectory is input into the trained multi-target tracking model; The target trajectory features are sequentially injected into the image features with cross-interest to generate image captions describing the entire image content; Generate instance captions describing the behavior of a single target trajectory based on the target trajectory features; Identify the interaction relationship between two targets and generate labels describing the interaction state; Based on image captions, instance captions, and interactive recognition tags, a description of the target trajectory and annotations of interactive relationships are generated.

5. The semantic multi-target tracking method for railway platforms according to claim 1, characterized in that, The step of training a multi-target tracking model based on the second training set specifically includes: A multi-target tracking model is constructed, and the parameters of the multi-target tracking model are initialized; the multi-target tracking model includes a target detection module, a trajectory generation module, and a semantic understanding module; A multi-task loss function is determined based on target detection loss, trajectory generation loss, and semantic understanding loss. With the goal of minimizing the multi-task loss function, the multi-target tracking model is trained end-to-end based on the second training set to obtain the trained multi-target tracking model.

6. The semantic multi-target tracking method for railway platforms according to claim 1, characterized in that, The step of inputting the first image into the trained noise predictor and determining the optimal noise mapping set based on the local noise prediction strategy and the diffusion acceleration algorithm specifically includes: A first image with a resolution lower than a first preset threshold is input into a trained noise predictor; a noise mapping set is constructed, and the number of noise mappings in the noise mapping set is reduced based on a local noise prediction strategy; Based on the diffusion acceleration algorithm, the noise map set is further compressed while reducing the number of noise maps, to obtain the optimal noise map set.

7. A semantic multi-target tracking device for railway platforms, characterized in that, include: The diffusion inversion module is used to acquire a first image with a resolution lower than a first preset threshold, and to process the first image with a resolution higher than a second preset threshold based on the diffusion inversion method and super-resolution technology to obtain a second image with a resolution higher than a second preset threshold. The module includes: inputting the first image into a trained noise predictor, wherein the noise predictor uses a variational autoencoder reparameterization technique to predict the Gaussian distribution parameters of the noise; determining an optimal noise mapping set based on a local noise prediction strategy and a diffusion acceleration algorithm; gradually acquiring a transition image corresponding to the first image through an inversion trajectory based on the optimal noise mapping set; and inputting the transition image into a trained super-resolution model to output the second image. A noise removal module is used to remove noise from the second image according to a trained noise removal model, wherein the noise removal model is trained according to a first training set, and the first training set includes the second image with noise. The target trajectory generation module is used to annotate the second image after noise removal and generate the target trajectory. The target tracking module is used to perform a semantic understanding task based on the target trajectory according to the trained multi-target tracking model, and generate a description of the target trajectory and annotations of the interaction relationship; the trained multi-target tracking model is trained according to a second training set, which includes a second image, target trajectory annotations and annotation data of the semantic understanding task.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the semantic multi-target tracking method for track platforms as described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the semantic multi-target tracking method for track platforms as described in any one of claims 1 to 6.

Images