Dynamic and static detection data labeling method and system based on time key association

Abstract

The invention belongs to the technical field of data processing, and relates to a dynamic and static detection data labeling method and system based on time key association. The method comprises the following steps: aligning all modal data to a unified time axis; standardized alignment is carried out through linear mapping processing, and unified feature dimensions are generated; performing feature extraction to obtain multi-modal feature data; extracting global time correlation features of the multi-modal feature data; based on an attention mechanism, extracting interaction features among the modals; and outputting a labeling result based on the interaction characteristics, and dynamically adjusting the weight and bias of an output layer according to the input interaction characteristics to update the labeling result. According to the method, through time sequence feature modeling, inter-modal association alignment and feature dynamic optimization, the problems of time synchronization and insufficient feature expression in multi-modal data labeling are solved; the method can quickly generate a high-precision and low-cost labeling result by combining static modal dynamic and unified time sequence features with cross-modal feature fusion.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular, relates to a dynamic and static detection data labeling method and system based on time key association. Background Art

[0002] In information-based detection tasks, the automatic labeling of multimodal data is a key link in improving data processing efficiency and model performance. Automated labeling mainly targets information such as the category, location, state, and behavioral characteristics of target objects in multimodal data, aiming to provide high-quality supervision data for subsequent data analysis and model training, while reducing the cost of manual intervention and improving task efficiency and accuracy. Faced with complex scenarios where dynamic modalities (such as video, audio, and sensor data) and static modalities (such as images) coexist, traditional labeling methods usually rely on manual processing or single-modality strategies, which make it difficult to effectively capture the correlation characteristics between modalities, resulting in low labeling efficiency, high cost, and unstable quality. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides a dynamic and static detection data labeling method and system based on time key association.

[0004] In a first aspect, the present invention provides a method for labeling dynamic and static detection data based on time key association, comprising: Align all modal data to a unified time axis to generate modal data of cross-modal time series; All modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension; A neural network is used to extract features from each modal data to obtain multimodal feature data; A Transformer-based time series model is used to extract global time correlation features of multimodal feature data; Extract interaction features between modalities based on attention mechanism; The annotation results are output based on the interactive features, dynamic modal weights are set, and learning weights are assigned to the features of each modality. The weights and biases of the output layer are dynamically adjusted according to the input interactive features to update the annotation results.

[0005] In a second aspect, the present invention provides a dynamic and static detection data annotation system based on time key association, including an alignment unit, an encoding and alignment unit, a first feature extraction unit, a second feature extraction unit, an interactive feature extraction unit, and an output and update unit; An alignment unit, used to align all modal data to a unified time axis and generate modal data of cross-modal time series; The encoding and alignment unit is used to encode all modal data into high-dimensional features and perform standardized alignment through linear mapping to generate a unified feature dimension; A first feature extraction unit, used to extract features from each modal data using a neural network to obtain multimodal feature data; A second feature extraction unit is used to extract global time correlation features of multimodal feature data using a Transformer-based time series model; Interaction feature extraction unit, used to extract interaction features between modalities based on the attention mechanism; The output and update unit is used to output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results.

[0006] Based on the above technical solution, the present invention can also be improved as follows.

[0007] Furthermore, before all modal data are aligned to a unified time axis, the process also includes: performing time serialization processing on static modal data, converting discrete static data into continuous time series features through time alignment, interpolation filling and feature generation; for dynamic modal data, allocating features to a unified time axis according to time windows.

[0008] Furthermore, the static data is processed in time series, including: The preset time window with missing image data is filled by interpolation method; When the image collection interval is greater than the set value, transition features are generated through generative adversarial networks or through time-aware linear interpolation, so that discrete static data is converted into continuous time series features.

[0009] Furthermore, considering the influence of time context on interpolation, we choose the nearest neighbor interpolation algorithm combined with time weight, assuming that time is , the image features obtained in the time difference ,time The most recent known image feature is , for The moment before, for After that moment, time The nearest known image feature is , for The weight of for The weight of , then: ; ; ; When the image collection interval is greater than the set value, the transition features are generated by generating adversarial networks. Let the adversarial network be ,but: ; Or generate transition features through time-aware linear interpolation, then: .

[0010] Furthermore, all modal data are aligned to a unified time axis to generate modal data of cross-modal time series, including: for dynamic modal data, the features of dynamic modal data are allocated to a unified time axis according to time windows; for static modal data, the static modal data are mapped to the time series through a time serialization method.

[0011] Furthermore, all modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension, including: For image modality data, a time-aware self-attention mechanism is introduced through convolutional neural networks or visual transformers to extract static features of the image modality; For video modality data, a separated spatiotemporal convolution and a multi-head spatiotemporal attention mechanism are used to jointly extract the temporal and spatial information of the video. For audio modality data, a time-aware self-attention mechanism is introduced through a convolutional neural network or a transformer-based network to extract the frequency domain features and time domain features of the audio modality; For sensor modal data, the time series features of the sensor modality are extracted through a one-dimensional convolutional network or a recurrent neural network.

[0012] Furthermore, a Transformer-based time series model is used to extract the global time correlation features of multimodal feature data, including: assuming that the uniformly aligned multimodal feature matrix is , is the output of the time series modeling module, represents a real number, represents the total number of time steps, represents the feature dimension, represents a time series model, then: .

[0013] Furthermore, the interaction features between the modalities are extracted based on the attention mechanism, including: The query vector representing the features of the target modality at the current time step, The key vector representing the features of the historical time step, represents the eigenvalue of the candidate mode, is the output of the time series modeling module, and the interaction characteristics between modes are , is the attention module, then: .

[0014] Furthermore, based on the interactive features, the annotation results are output, dynamic modal weights are set, learning weights are assigned to the features of each modality, and the weights and biases of the output layer are dynamically adjusted according to the input interactive features to update the annotation results, including: setting Indicates the number of modes, Represents the total number of modes, represents the time step, represents the weight of each mode, Indicates the labeling result. Indicates The fusion features of the modalities at the time step, represents the weight matrix of the output layer, represents the bias term of the output layer, Represents the time step The weight matrix of the output layer is Represents the time step The bias term of the output layer of represents the dynamic generation function of the weight matrix, The dynamic generation function of the bias term is represented. For different time steps and different input features, a dynamic weight adjustment mechanism is introduced to enable the output layer to adaptively optimize the weights and biases. Then: ; ; .

[0015] The beneficial effects of the present invention are as follows: the present invention proposes an automatic labeling method, which solves the time synchronization problem and the problem of insufficient feature expression in multimodal data labeling through time series feature modeling, inter-modal correlation alignment and dynamic feature optimization; by making static modes dynamic and unifying time series features, combined with cross-modal feature fusion, it can quickly generate high-precision and low-cost labeling results, and realize the intelligence and automation of the labeling process; the method does not require a lot of manual intervention, can adapt to the time step differences and complex dynamic relationships of different modal data, significantly improves labeling efficiency and consistency, and is suitable for scenarios such as industrial detection and medical detection that require efficient multimodal data processing, providing high-quality automatic labeling support for information detection systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a method for labeling dynamic and static detection data based on time key association provided in Example 1 of the present invention; Figure 2 This is a system block diagram of a dynamic and static detection data labeling system based on time key association provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0018] Example 1 As an example, Figure 1 As shown, in order to solve the above technical problems, this embodiment provides a dynamic and static detection data labeling method based on time key association, including: Align all modal data to a unified time axis to generate modal data of cross-modal time series; All modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension; A neural network is used to extract features from each modal data to obtain multimodal feature data; A Transformer-based time series model is used to extract global time correlation features of multimodal feature data; Extract interaction features between modalities based on attention mechanism; The annotation results are output based on the interactive features, dynamic modal weights are set, and learning weights are assigned to the features of each modality. The weights and biases of the output layer are dynamically adjusted according to the input interactive features to update the annotation results.

[0019] Optionally, before aligning all modal data to a unified time axis, the process also includes: performing time serialization processing on static modal data, converting discrete static data into continuous time series features through time alignment, interpolation filling and feature generation; for dynamic modal data, allocating features to a unified time axis according to time windows.

[0020] Optionally, perform time serialization on static data, including: The preset time window with missing image data is filled by interpolation method; When the image collection interval is greater than the set value, transition features are generated through generative adversarial networks or through time-aware linear interpolation, so that discrete static data is converted into continuous time series features.

[0021] Since the timestamps of static modal data such as pictures are often sparse and discontinuous, it is difficult to directly participate in time series modeling. This paper proposes a time serialization strategy to convert discrete static data into continuous time series features through time alignment, interpolation filling and feature generation.

[0022] Optionally, consider the impact of time context on interpolation and select the nearest neighbor interpolation algorithm combined with time weight. Suppose time is , the image features obtained in the time difference ,time The most recent known image feature is , for The moment before, for After that moment, time The nearest known image feature is , for The weight of for The weight of , then: ; ; ; When the image collection interval is greater than the set value, the transition features are generated by generating adversarial networks. Let the adversarial network be ,but: ; Or generate transition features through time-aware linear interpolation, then: .

[0023] Compared with traditional linear interpolation, the time-aware mechanism combined with the time key makes the interpolation result more robust.

[0024] Optionally, all modal data are aligned to a unified time axis to generate modal data of a cross-modal time series, including: for dynamic modal data, the features of the dynamic modal data are assigned to a unified time axis according to time windows; for static modal data, the static modal data are mapped to a time series through a time serialization method.

[0025] Realizing seamless alignment of static data and dynamic modalities in the time dimension not only solves the problem of lack of temporal continuity of static modal data, but also ensures the smoothness and authenticity of the time series through the combination of interpolation and feature generation, providing high-quality input for multimodal fusion modeling.

[0026] Multimodal detection data includes modal data such as video, image, audio, and sensor. The sampling timestamps and frequencies of different modalities are different, and it is very challenging to directly align these data. This paper proposes an association mechanism based on time keys to align all modal data to a unified time axis, generate cross-modal time series, and provide a unified time series input for subsequent annotation tasks.

[0027] Optionally, all modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension, including: For image modality data, a time-aware self-attention mechanism is introduced through convolutional neural networks or visual transformers to extract static features of the image modality; For video modality data, a separated spatiotemporal convolution and a multi-head spatiotemporal attention mechanism are used to jointly extract the temporal and spatial information of the video. For audio modality data, a time-aware self-attention mechanism is introduced through a convolutional neural network or a transformer-based network to extract the frequency domain features and time domain features of the audio modality; For sensor modal data, the time series features of the sensor modality are extracted through a one-dimensional convolutional network or a recurrent neural network.

[0028] In multimodal annotation tasks, the feature information contained in different modal data (such as pictures, videos, audio and sensors) varies greatly, so a special feature extraction network is required to efficiently encode each modality. This paper encodes all modal data into high-dimensional feature representations, and standardizes and aligns them through linear mapping to generate a unified feature dimension, providing consistent input for subsequent modeling.

[0029] As an optional implementation, for image modality data, a self-attention mechanism is introduced to enhance the global receptive field of feature extraction, so as to better capture the global context information in the image. In addition, a multi-scale feature fusion strategy is combined to extract fine-grained information and high-level semantic information in the image. For example, Represents the first pictures, represents the image features extracted by the image encoder, represents the total number of time steps, represents the feature dimension, Represents the feature matrix of the extracted image modality data, represents the picture encoder, then: .

[0030] As an optional implementation, for video modality data, a convolutional network or a spatiotemporal feature extractor based on self-attention is used to extract the spatiotemporal joint features of the video modality, which solves the problem that traditional methods are insufficient in modeling the time dimension. Optionally, a separated spatiotemporal convolution and a multi-head spatiotemporal attention mechanism are used to jointly extract the temporal information and spatial information of the video. For example, Indicates Video modality data with a step length of represents the features of the video modality data extracted by the spatiotemporal feature extractor, Represents the video encoder, represents the total number of time steps, represents the feature dimension, Represents the feature matrix of the extracted video modality data, then: .

[0031] As an optional implementation, for audio modality data, frequency domain and time domain features of the audio modality are extracted through a convolutional neural network or a transformer-based network. Based on the traditional Mel-spectrogram feature extraction, a multi-scale feature extraction module is introduced to capture the cross-band dependencies in the audio modality. In addition, the method based on the frequency domain attention mechanism is combined to improve the feature capture capability of key frequency bands. For example, Indicates Audio modal data with a step length of represents the features of the audio modality data extracted by the audio encoder, Represents the audio codec, represents the total number of time steps, represents the feature dimension, Represents the feature matrix of the extracted video modality data, then: .

[0032] As an optional implementation, for sensor modal data, the time series features of the sensor modality are extracted through a one-dimensional convolutional network or a recursive neural network. On the basis of the traditional time series modeling method, a time-aware self-attention mechanism is introduced to improve the modeling ability of long-term dependencies. At the same time, a residual connection structure is used to alleviate the problem of information loss in sensor data. For example, Indicates Sensor modal data with a step length, represents the features of the sensor modal data extracted by the sensor encoder, Indicates the sensor encoder, represents the total number of time steps, represents the feature dimension, Represents the feature matrix of the extracted video modality data, then: .

[0033] Finally, the modal features are stacked into a unified feature matrix. Let the feature matrix be , is the number of modes, then: .

[0034] The present invention eliminates the differences in feature dimensions between modalities through standardization and alignment of modal features, and significantly improves the adaptability of the annotation model to multimodal data.

[0035] Optionally, a Transformer-based time series model is used to extract the global time correlation features of the multimodal feature data, including: assuming that the uniformly aligned multimodal feature matrix is , is the output of the time series modeling module, represents a real number, represents the total number of time steps, represents the feature dimension, represents a time series model, then: .

[0036] Optionally, the interaction features between the modalities are extracted based on the attention mechanism, including: The query vector representing the features of the target modality at the current time step, The key vector representing the features of the historical time step, represents the eigenvalue of the candidate mode, is the output of the time series modeling module, and the interaction characteristics between modes are , is the attention module, then: .

[0037] Optionally, output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results, including: setting Indicates the number of modes, Represents the total number of modes, represents the time step, represents the weight of each mode, Indicates the labeling result. Indicates The fusion features of the modalities at the time step, represents the weight matrix of the output layer, represents the bias term of the output layer, Represents the time step The weight matrix of the output layer is Represents the time step The bias term of the output layer of represents the dynamic generation function of the weight matrix, The dynamic generation function of the bias term is represented. For different time steps and different input features, a dynamic weight adjustment mechanism is introduced to enable the output layer to adaptively optimize the weights and biases. Then: ; ; .

[0038] For different time steps or different input features, a dynamic weight adjustment mechanism is introduced to enable the output layer to adaptively optimize weights and biases.

[0039] The present invention captures complex temporal dynamics and modal dependencies through global temporal modeling and cross-modal interaction modeling, and can generate high-quality annotation results.

[0040] The present invention proposes an automated labeling method, which solves the time synchronization problem and insufficient feature expression problem in multimodal data labeling through time series feature modeling, inter-modal correlation alignment and dynamic feature optimization; by making static modal dynamic and unified time series features, combined with cross-modal feature fusion, it can quickly generate high-precision and low-cost labeling results, and realize the intelligence and automation of the labeling process; the method does not require a lot of manual intervention, can adapt to the time step differences and complex dynamic relationships of different modal data, significantly improves labeling efficiency and consistency, and is suitable for scenarios that require efficient multimodal data processing such as industrial detection and medical detection, providing high-quality automated labeling support for information detection systems.

[0041] Example 2 Based on the same principle as the method shown in Example 1 of the present invention, as shown in the attached Figure 2 As shown, an embodiment of the present invention also provides a dynamic and static detection data annotation system based on time key association, including an alignment unit, an encoding and alignment unit, a first feature extraction unit, a second feature extraction unit, an interactive feature extraction unit, and an output and update unit; An alignment unit, used to align all modal data to a unified time axis and generate modal data of cross-modal time series; The encoding and alignment unit is used to encode all modal data into high-dimensional features and perform standardized alignment through linear mapping to generate a unified feature dimension; A first feature extraction unit, used to extract features from each modal data using a neural network to obtain multimodal feature data; A second feature extraction unit is used to extract global time correlation features of multimodal feature data using a Transformer-based time series model; Interaction feature extraction unit, used to extract interaction features between modalities based on the attention mechanism; The output and update unit is used to output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results.

[0042] Optionally, before aligning all modal data to a unified time axis, the process also includes: performing time serialization processing on static modal data, converting discrete static data into continuous time series features through time alignment, interpolation filling and feature generation; for dynamic modal data, allocating features to a unified time axis according to time windows.

[0043] Optionally, perform time serialization on static data, including: The preset time window with missing image data is filled by interpolation method; When the image collection interval is greater than the set value, transition features are generated through generative adversarial networks or through time-aware linear interpolation, so that discrete static data is converted into continuous time series features.

[0044] Optionally, consider the impact of time context on interpolation and select the nearest neighbor interpolation algorithm combined with time weight. Suppose time is , the image features obtained in the time difference ,time The most recent known image feature is , for The moment before, for After that moment, time The nearest known image feature is , for The weight of for The weight of , then: ; ; ; When the image collection interval is greater than the set value, the transition features are generated by generating adversarial networks. Let the adversarial network be ,but: ; Or generate transition features through time-aware linear interpolation, then: .

[0045] Optionally, all modal data are aligned to a unified time axis to generate modal data of a cross-modal time series, including: for dynamic modal data, the features of the dynamic modal data are assigned to a unified time axis according to time windows; for static modal data, the static modal data are mapped to a time series through a time serialization method.

[0046] Optionally, all modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension, including: For image modality data, a time-aware self-attention mechanism is introduced through convolutional neural networks or visual transformers to extract static features of the image modality; For video modality data, a separated spatiotemporal convolution and a multi-head spatiotemporal attention mechanism are used to jointly extract the temporal and spatial information of the video. For audio modality data, a time-aware self-attention mechanism is introduced through a convolutional neural network or a transformer-based network to extract the frequency domain features and time domain features of the audio modality; For sensor modal data, the time series features of the sensor modality are extracted through a one-dimensional convolutional network or a recurrent neural network.

[0047] Optionally, a Transformer-based time series model is used to extract the global time correlation features of the multimodal feature data, including: assuming that the uniformly aligned multimodal feature matrix is , is the output of the time series modeling module, represents a real number, represents the total number of time steps, represents the feature dimension, represents a time series model, then: .

[0048] Optionally, the interaction features between the modalities are extracted based on the attention mechanism, including: The query vector representing the features of the target modality at the current time step, The key vector representing the features of the historical time step, represents the eigenvalue of the candidate mode, is the output of the time series modeling module, and the interaction characteristics between modes are , is the attention module, then: .

[0049] Optionally, output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results, including: setting Indicates the number of modes, Represents the total number of modes, represents the time step, represents the weight of each mode, Indicates the labeling result. Indicates The fusion features of the modalities at the time step, represents the weight matrix of the output layer, represents the bias term of the output layer, Represents the time step The weight matrix of the output layer is Represents the time step The bias term of the output layer of represents the dynamic generation function of the weight matrix, The dynamic generation function of the bias term is represented. For different time steps and different input features, a dynamic weight adjustment mechanism is introduced to enable the output layer to adaptively optimize the weights and biases. Then: ; ; .

[0050] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dynamic and static detection data labeling method based on time key association, characterized in that: include: Align all modal data to a unified time axis to generate modal data of cross-modal time series; All modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension; A neural network is used to extract features from each modal data to obtain multimodal feature data; A Transformer-based time series model is used to extract global time correlation features of multimodal feature data; Extract interaction features between modalities based on attention mechanism; The annotation results are output based on the interactive features, dynamic modal weights are set, and learning weights are assigned to the features of each modality. The weights and biases of the output layer are dynamically adjusted according to the input interactive features to update the annotation results.

2. According to claim 1, the method for labeling dynamic and static detection data based on time key association is characterized in that: Before aligning all modal data to a unified time axis, it also includes: time serialization of static modal data, converting discrete static data into continuous time series features through time alignment, interpolation filling and feature generation; for dynamic modal data, the features are allocated to a unified time axis according to time windows.

3. According to claim 2, the dynamic and static detection data labeling method based on time key association is characterized in that: Time serialization of static data, including: The preset time window with missing image data is filled by interpolation method; When the image collection interval is greater than the set value, transition features are generated through generative adversarial networks or through time-aware linear interpolation, so that discrete static data is converted into continuous time series features.

4. According to claim 3, the method for labeling dynamic and static detection data based on time key association is characterized in that: Considering the influence of time context on interpolation, we choose the nearest neighbor interpolation algorithm combined with time weight. Let time be , the image features obtained in the time difference ,time The most recent known image feature is , for The moment before, for After that moment, time The nearest known image feature is , for The weight of for The weight of , then: ； ； ； When the image collection interval is greater than the set value, the transition features are generated by generating adversarial networks. Let the adversarial network be ,but: ； Or generate transition features through time-aware linear interpolation, then: 。 5. According to claim 1, the method for labeling dynamic and static detection data based on time key association is characterized in that: All modal data are aligned to a unified time axis to generate modal data of cross-modal time series, including: for dynamic modal data, the features of dynamic modal data are assigned to a unified time axis according to time windows; for static modal data, the static modal data are mapped to the time series through a time serialization method.

6. The dynamic and static detection data labeling method based on time key association according to claim 1 is characterized in that: All modal data are encoded into high-dimensional features and standardized and aligned through linear mapping to generate a unified feature dimension, including: For image modality data, a time-aware self-attention mechanism is introduced through convolutional neural networks or visual transformers to extract static features of the image modality; For video modality data, a separated spatiotemporal convolution and a multi-head spatiotemporal attention mechanism are used to jointly extract the temporal and spatial information of the video. For audio modality data, a time-aware self-attention mechanism is introduced through a convolutional neural network or a transformer-based network to extract the frequency domain features and time domain features of the audio modality; For sensor modal data, the time series features of the sensor modality are extracted through a one-dimensional convolutional network or a recurrent neural network.

7. The dynamic and static detection data labeling method based on time key association according to claim 1 is characterized in that: The Transformer-based time series model is used to extract the global time correlation features of multimodal feature data, including: assuming that the uniformly aligned multimodal feature matrix is , is the output of the time series modeling module, represents a real number, represents the total number of time steps, represents the feature dimension, represents a time series model, then: 。 8. The method for labeling dynamic and static detection data based on time key association according to claim 1, characterized in that: Extract the interactive features between each modality based on the attention mechanism, including: The query vector representing the features of the target modality at the current time step, The key vector representing the features of the historical time step, represents the eigenvalue of the candidate mode, is the output of the time series modeling module, and the interaction characteristics between modes are , is the attention module, then: 。 9. The dynamic and static detection data labeling method based on time key association according to claim 1 is characterized in that: Output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results, including: Indicates the number of modes, Represents the total number of modes, represents the time step, represents the weight of each mode, Indicates the labeling result. Indicates The fusion features of the modalities at the time step, represents the weight matrix of the output layer, represents the bias term of the output layer, Represents the time step The weight matrix of the output layer is Represents the time step The bias term of the output layer of represents the dynamic generation function of the weight matrix, The dynamic generation function of the bias term is represented. For different time steps and different input features, a dynamic weight adjustment mechanism is introduced to enable the output layer to adaptively optimize the weights and biases. Then: ； ； 。 10. A dynamic and static detection data labeling system based on the dynamic and static detection data labeling method based on time key association according to claim 1, characterized in that: It includes an alignment unit, an encoding and alignment unit, a first feature extraction unit, a second feature extraction unit, an interactive feature extraction unit, and an output and update unit; An alignment unit, used to align all modal data to a unified time axis and generate modal data of cross-modal time series; The encoding and alignment unit is used to encode all modal data into high-dimensional features and perform standardized alignment through linear mapping to generate a unified feature dimension; A first feature extraction unit, used to extract features from each modal data using a neural network to obtain multimodal feature data; A second feature extraction unit is used to extract global time correlation features of multimodal feature data using a Transformer-based time series model; Interaction feature extraction unit, used to extract interaction features between modalities based on the attention mechanism; The output and update unit is used to output the annotation results based on the interactive features, set dynamic modal weights, assign learning weights to the features of each modality, and dynamically adjust the weights and biases of the output layer according to the input interactive features to update the annotation results.

Images