Training method of security event reproduction model and security event reproduction method

Abstract

The invention provides a training method of a security event reoccurrence model and a security event reoccurrence method, relates to the technical field of security event reoccurrence, and aims to perform anti-fact reasoning through the security event reoccurrence model so as to improve the accuracy of security event reoccurrence. The method comprises the steps of obtaining training data; training a security event reproduction model based on the training data; wherein the security event reproduction model comprises a video word segmentation device, a multi-modal event injection module, a potential action model and a dynamic prediction model; wherein the video word segmentation device is used for encoding a video related to a security event into a video token sequence; the multi-modal event injection module is used for encoding multi-modal data related to a security event into a multi-modal event token sequence; the potential action model is used for coding into action representation based on a video token sequence and a multi-modal event token sequence; the dynamic prediction model is used for outputting a prediction token sequence and a key point probability sequence of the security event based on the video token sequence and the action representation.

Description

Technical Field

[0001] This application relates to the field of security incident reproduction technology, and in particular to a training method for a security incident reproduction model and a security incident reproduction method. Background Technology

[0002] With increasingly stringent requirements for aviation operational safety management, incident debriefing has become a crucial step in ensuring the reliability of aviation operations and preventing the recurrence of similar risks. However, current incident debriefing work remains largely manual, relying on staff to access scattered evidence from multiple sources, including flight data recorder (FDR) data, aircraft condition monitoring system (ACMS) data, cockpit voice recorder (CVR) data, on-site surveillance videos, maintenance work orders, reporting text messages, and regulatory documents. This manual process involves aligning the timelines of various data sets and relying on expert experience for qualitative analysis and causal deduction. This manually-driven debriefing model suffers from low efficiency and high subjectivity, directly impacting the accuracy of reproducing safety incidents. Summary of the Invention

[0003] The purpose of this application is to provide a training method and a method for reproducing security incidents, which aims to improve the accuracy of security incident reproduction by performing counterfactual reasoning through the security incident reproduction model, and to enable the security incident reproduction model to locate key points in the development of security incidents.

[0004] To achieve the above objectives, the embodiments of this application provide the following technical solutions: Firstly, this application provides a training method for a security incident reproducibility model. The method includes: acquiring training data; training the security incident reproducibility model based on the training data; wherein the security incident reproducibility model includes a video segmenter, a multimodal event injection module, a latent action model, and a dynamic prediction model; wherein the video segmenter is used to encode security incident-related videos into video token sequences; the multimodal event injection module is used to encode security incident-related multimodal data into multimodal event token sequences; the latent action model is used to encode action representations based on the video token sequences and multimodal event token sequences; and the dynamic prediction model is used to output a predicted token sequence and a keypoint probability sequence for the security incident based on the video token sequence and action representations, wherein the keypoint probability sequence characterizes the probability of each time point in the security incident being a key point.

[0005] The training method for the security incident reproduction model provided in this application encodes multimodal data other than video into a multimodal event token sequence through a multimodal event injection module, thereby effectively integrating key contextual information and enabling the security incident reproduction model to more comprehensively understand the causes and background of security incidents. A latent action model generates action representations based on the video token sequence and the multimodal event token sequence, helping to strip away irrelevant details and capture the essential behavioral patterns of security incidents. Building upon this, a dynamic prediction model synchronously outputs a predicted token sequence and a key point probability sequence for the security incident based on the video token sequence and action representation, not only predicting the subsequent development of the incident but also clearly locating important turning points in the incident process. This method establishes a security incident reproduction model through a progressive process from multimodal information encoding and action representation extraction to synchronous prediction of event dynamics and key points. Based on the counterfactual reasoning of this model, it can accurately reproduce security incidents and locate the key turning points in the development of those incidents.

[0006] In some embodiments, the multimodal event injection module includes: a text encoder for encoding text data in the multimodal data related to the security event into a text token sequence; a speech encoder for encoding speech data in the multimodal data related to the security event into a speech token sequence; an image encoder for encoding image data in the multimodal data related to the security event into an image token sequence; a file encoder for encoding structured file data in the multimodal data related to the security event into a file token sequence; and a sensor data encoder for encoding sensor data in the multimodal data related to the security event into a data stream token sequence.

[0007] In some embodiments, training a security incident reconstruction model based on training data includes: training the security incident reconstruction model based on training data to obtain a training loss value; updating the weight parameters of the security incident reconstruction model based on the training loss value; wherein the training loss value is determined based on a base loss value and an additional loss value; the base loss value is used to characterize the difference between the predicted trajectory and the actual trajectory of the security incident; the additional loss value includes at least one of the following: evidence consistency loss value, time axis alignment loss value, and key point alignment loss value; wherein the evidence consistency loss value is used to characterize the difference in modality between the predicted token sequence and the evidence sequence at the time point where evidence exists for the security incident; the time axis alignment loss value is used to characterize the difference in the time dimension between the predicted token sequence and the evidence sequence for the security incident; and the key point alignment loss value is used to characterize the difference between the key point probability sequence and the pre-labeled key point time sequence for the security incident.

[0008] In some embodiments, the evidence consistency loss value satisfies the following relationship:

[0009] in, This represents the loss value in terms of evidence consistency. A set of time points in time where there is evidence of a security incident; Cross-entropy loss is used to measure the difference in video modalities. and Representing time respectively The real video token sequence and the predicted video token sequence; CTC represents the connection-temporal classification loss, used to measure the difference between speech modalities or text modalities. and Representing time respectively The actual speech modality and the corresponding predicted speech modality, or the representation of time. The true text modality and the corresponding predicted text modality; Mean squared error is used to measure the difference between numerical time-series modes. and Representing time respectively Numerical time-series modes and numerical time-series modes.

[0010] In some embodiments, the time axis alignment loss values ​​satisfy the following relationship:

[0011] in, This indicates the time axis alignment loss value; This represents the soft dynamic time warping distance calculation function, used to measure the alignment difference between two event sequences in the time dimension; This represents the event sequence extraction function; This represents a sequence of events constructed based on a sequence of evidence. This represents the event sequence constructed from the predicted token sequence.

[0012] In some embodiments, the keypoint alignment loss values ​​satisfy the following relationship:

[0013] in, This indicates the time axis alignment loss value; Represents the binary cross-entropy loss function; Indicates the time of prediction The probability of being a key point; This indicates the pre-labeled keypoint labels, at time... When located within the time window of the labeled key point Otherwise ; These are preset weighting coefficients used to balance the contributions of classification loss and temporal localization loss; A set representing keypoint indices; Indicates the predicted first A crucial moment; Indicates the relationship with the first The real time corresponding to the preset annotations of each key point.

[0014] In some embodiments, the action representation includes: a high-level action representation and a low-level action representation; the high-level action representation is used to characterize the semantic intent corresponding to the standard operating procedure; the low-level action representation is used to characterize the specific operation sequence to achieve the semantic intent.

[0015] In some embodiments, the security event reproducibility model further includes an external memory; the external memory is used to provide historical event data during the training of the security event reproducibility model.

[0016] Secondly, this application also provides a method for reproducing security incidents. This method includes: acquiring multimodal data related to the security incident; determining a predicted token sequence and a key point probability sequence of the security incident based on the multimodal data using a security incident reproduction model; wherein the security incident reproduction model is trained based on the method described in the first aspect. Based on the predicted token sequence and key point probability sequence of the security incident, the method reproduces the security incident and locates key points within it.

[0017] Thirdly, this application also provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, the electronic device enables the training method of the first aspect or the method of the second aspect described above.

[0018] Fourthly, this application also provides a computer-readable storage medium comprising: computer software instructions; when the computer software instructions are executed in an electronic device, the electronic device causes the electronic device to implement the training method of the first aspect or the method of the second aspect described above.

[0019] Fifthly, this application also provides a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to implement the training method of the first aspect or the method of the second aspect described above.

[0020] The beneficial effects of the second to fifth aspects mentioned above can be referred to the corresponding description of the first aspect, and will not be repeated here. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A schematic diagram of the framework of a training system for a security incident reproduction model provided in an embodiment of this application; Figure 2 A flowchart illustrating a training method for a security incident reproduction model provided in an embodiment of this application; Figure 3 A schematic diagram illustrating a process for training a security incident reproduction model, provided as an embodiment of this application; Figure 4 A schematic diagram of the architecture of a security event reproduction model provided in an embodiment of this application; Figure 5 A flowchart illustrating a security event reproduction method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0025] The following is a brief explanation of the technical terms used in the embodiments of this application: 1. Vector quantized variational autoencoder (VQ-VAE) is a codebook-based generative model whose architecture mainly consists of three parts: an encoder, a vector quantization module (containing the codebook), and a decoder. The encoder first maps the input data (such as image or video frames) into continuous latent feature vectors. These latent feature vectors are then fed into the vector quantization module, which uses nearest neighbor search to match the closest codebook vector in a learnable discrete codebook, thus transforming the continuous features into sparse and semantically clear discrete codes. The decoder then retrieves the corresponding vector from the codebook based on the quantized discrete codes and reconstructs data that is visually consistent with the original input.

[0026] 2. The Transformer is a deep learning model architecture based on an attention mechanism. Its core consists of a stacked encoder and decoder structure. Both the encoder and decoder contain multi-head self-attention sublayers, feedforward network sublayers, residual connections, and layer normalization modules. This model injects temporal information into the input sequence through positional encoding, uses the multi-head self-attention mechanism to compute the global dependency weights of each position in the sequence in parallel, and combines the feedforward network to achieve feature mapping. This breaks away from the sequential computation constraints of traditional recurrent or convolutional networks, improving computational efficiency and modeling capabilities.

[0027] 3. Spatiotemporal Fusion Transformer Temporal Transformer (ST-Transformer) is an improved Transformer neural network architecture for spatiotemporal sequence data. Its core uses separate spatial and temporal attention modules to model the spatial location dependency and temporal dynamic relationships of the data, respectively. It integrates two-dimensional features through strategies such as weighted fusion, concatenation, or cross-attention interaction, and can also incorporate local attention, multi-scale feature extraction, or convolution. The attention-parallel architecture optimizes computational efficiency and feature representation capability. This architecture supplements temporal information through positional encoding, combined with an encoder. Decoder stacking, multi-head attention, feedforward networks, residual connections, and layer normalization enable parallel modeling of global spatiotemporal dependencies, making it suitable for scenarios such as video analytics.

[0028] Based on the aforementioned background technology, the emergence of generative interactive environments (Genie) models has provided a new approach for aviation "safety incident analysis." The Genie model, centered on a video tokenizer, latent actions, and dynamic environment models, discretizes video frames into visual token sequences through a visual tokenizer. During the training phase, tokens are randomly masked and non-autoregressive parallel predictions are performed using a Transformer decoder. During the inference phase, token sequences are iteratively generated and video frames are reconstructed. It can also be extended to ST-Transformer, jointly processing multi-frame video and latent action embeddings to learn interactive dynamic environment models from pure video data.

[0029] However, directly applying the original Genie model framework to aviation safety incident analysis scenarios presents numerous technical bottlenecks, with the key challenge being: 1. The original Genie model relies on pure video data to carry out dynamic environmental learning, while aviation safety incident analysis needs to integrate multimodal evidence chains such as flight data recorders, cockpit voice recorders, air traffic control communications, operating regulations, and airport monitoring. The existing single-modal modeling architecture is difficult to meet the multi-source heterogeneous data fusion requirements of incident analysis.

[0030] 2. The original Genie model focuses on the generation and reconstruction of visual tokens, but lacks the ability to model the core constraints of event analysis, such as compliance of operational procedures, rationality of decision-making actions, consistency of multiple evidences, and temporal logical correlation. It cannot support the standardized assessment and causal inference of personnel behavior in security incidents.

[0031] 3. The original Genie model has difficulty integrating experience in handling typical historical security incidents, standard operating procedures, and expert knowledge strategies, resulting in a lack of domain-aware guidance in the analysis process and an inability to establish a distinguishable boundary between compliant operations and non-compliant behaviors.

[0032] In summary, these issues prevent the existing framework from accurately reconstructing and deeply analyzing complex security incidents, severely impacting the accuracy of security incident reproduction.

[0033] Based on this, the training method for the security incident reproduction model provided in this application encodes multimodal data other than video into a multimodal event token sequence through a multimodal event injection module, thereby effectively integrating key contextual information and enabling the security incident reproduction model to more comprehensively understand the causes and background of security incidents. The latent action model generates action representations based on the video token sequence and the multimodal event token sequence, which helps to strip away irrelevant details and capture the essential behavioral patterns of security incidents. On this basis, the dynamic prediction model synchronously outputs a predicted token sequence and a key point probability sequence for the security incident based on the video token sequence and action representation, not only predicting the subsequent development of the incident but also clearly locating important turning points in the incident process. This method establishes a security incident reproduction model through a progressive process from multimodal information encoding and action representation extraction to synchronous prediction of event dynamics and key points. Based on the counterfactual reasoning of this model, it can accurately reproduce security incidents and locate the key turning points in the development of the security incident. The training method for the security event reproduction model provided in this application embodiment can be applied to a model training device, which can be an electronic device such as a server, server cluster, high-performance computing platform or dedicated security analysis server. This application embodiment does not impose any restrictions on the specific form of the model training device.

[0034] In some embodiments, the model training device belongs to Figure 1 The training system for the security incident reproduction model shown. For example... Figure 1 As shown, the security incident reproduction model training system may include a security incident data acquisition device 101 and a security incident model training device 102. The security incident data acquisition device 101 is equipped with a multimodal evidence collection and preprocessing unit (for collecting and aligning surveillance video, voice communication, flight data, and structured reports). The security incident model training device 102 may be equipped with an initial video segmenter, a multimodal event injection module, a latent action model, and a dynamic prediction model. The security incident model training device 102 can establish a communication connection with the security incident data acquisition device 101 to obtain a preprocessed and aligned multimodal security incident dataset, and then perform phased or end-to-end joint training on the aforementioned models, ultimately obtaining a well-trained security incident reproduction model capable of high-fidelity reconstruction and deduction of event development based on historical evidence and action intentions.

[0035] In other embodiments, the security event model training device 102 integrates a multimodal evidence collection and preprocessing unit, a video segmenter, a multimodal event injection module, a potential action model, and a dynamic prediction model, completing the entire process from evidence collection and preprocessing to model training and optimization directly on a single device.

[0036] The security incident data acquisition device 101 and the security incident model training device 102 can be integrated on the same device (such as an integrated security analysis workstation), or the security incident data acquisition device 101 and the security incident model training device 102 can be set in separate devices.

[0037] This application also provides a method for reconstructing security incidents, which is used to dynamically reconstruct incident scenarios and perform counterfactual reasoning and strategy evaluation based on a trained security incident reconstruction model and input multimodal evidence and user interaction instructions.

[0038] Understandably, this method for reproducing security incidents can be applied to... Figure 1 The security event reproduction application device 103 shown can also be applied to the security event model training device 102 described above, and this application does not impose any restrictions on it.

[0039] The security incident reproduction application device 103 is equipped with a pre-trained security incident reproduction model. The security incident reproduction application device can be a mobile phone, augmented reality (AR) device, virtual reality (VR) device, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. This application embodiment does not impose any restrictions on the specific form of the security incident reproduction application device 103.

[0040] The training method for the security event reproduction model provided in this application will be described below with reference to specific embodiments and accompanying drawings.

[0041] Figure 2 This is a flowchart illustrating a training method for a security incident reproduction model provided in an embodiment of this application. Figure 2 As shown, the training method for this security incident reproduction model includes the following steps S201-S202: S201. Obtain training data.

[0042] The training data includes: security incident scenario data and associated evidence package data. Security incident scenario data defines the background, conditions, and focus of attention of an event, and includes scenario element data. Associated evidence package data refers to the original multimodal evidence files corresponding to the aforementioned security incident scenarios.

[0043] For example, the scenario element data includes at least one of the following: flight and airport elements, scenario elements, personnel and positions, and key point presets. For example, flight and airport elements include: flight number, aircraft type, airport / runway / taxiway, gate position, and operational phase; scenario elements include: weather, NOTAM, NAVAID status, ground traffic flow, de-icing status, and anti-icing status; personnel and positions include a list of roles such as flight crew, air traffic control, dispatch, maintenance, and ground service; key point presets define a list of key points that need to be focused on in this scenario, such as "second visual inspection before entering the taxiway" and "wind shear changeover maneuvers".

[0044] For example, the associated evidence package data includes at least one type of evidence: video evidence, audio evidence, data evidence, and text evidence. Examples include: video evidence: CCTV video, tarmac video, cockpit camera video (if available); audio evidence: CVR recordings, call recordings, or transcribed text; data evidence: FDR / ACMS data segments, E-log electronic logs, SMS / instruction / report PDFs; and text evidence: relevant regulations and standard operating procedures (SOPs).

[0045] In one possible implementation, training evidence also includes strategy cards associated with key decision points. These strategy cards can be graded by effectiveness or priority, such as A (optimal), B (acceptable), and C (flawed). Each strategy card contains standardized operational guidelines corresponding to that decision point, which may include at least one of the following: action sequence, communication script, required resources, and compliance criteria. The action sequence refers to a standard or optimized sequence of operational steps. The communication script refers to standard air-to-ground communication or internal coordination terminology. Required resources refer to the personnel, equipment, or information needed to execute the strategy. Compliance criteria are used to determine whether the action complies with regulations and standard operating procedures. Introducing strategy cards injects clear scoring criteria and operational specifications into the model, enabling it not only to predict event progression and identify critical moments but also to evaluate and optimize the quality of key decision-making.

[0046] In one possible implementation, structured logs and sensor data streams of real events can be automatically extracted from operational environments such as airline flight operations quality assurance (FOQA) systems, airport collaborative decision making (A-CDM) systems, and air traffic control (ATC) systems. These data can then be combined with anonymized official safety investigation reports and their accompanying evidence to form a high-quality training data set.

[0047] In another possible implementation, synthetic data covering various scenarios is generated in batches using high-fidelity flight simulators, air traffic control simulators, and traffic simulation systems under preset parameter settings. Experts in flight, air traffic control, dispatch, and other fields then construct structured scenario descriptions, label key points, associate policy cards, and align multimodal evidence timelines based on historical typical case libraries and standard operating procedures (SOPs), thereby constructing a training data set for model training.

[0048] In one possible implementation, the acquired training data needs to undergo preprocessing. First, the training data is cleaned and de-identified, removing sensitive information and standardizing the data format. Then, a dedicated parser and decoder transform the multimodal raw evidence (video, audio, sensor data streams, documents, etc.) into a timestamp-aligned standardized token sequence. This sequence is then precisely correlated and encapsulated with scene element data and pre-labeled keypoint time tags and policy card level tags. The final output is a standardized training data set that the model can directly train on. Each training data set contains a multimodal token sequence from a historical time window as input, and a real token sequence and keypoint label sequence from a future time window as supervision targets. For example, the dedicated parser and decoder can be different encoders and parsers within the multimodal event injection module described below.

[0049] Based on the above embodiments, the multimodal aligned training data obtained in this application can provide accurate and reliable training data for the security incident reproduction model, ensuring the accuracy and compliance of the model's predictions.

[0050] S202. Based on the training data, train the security incident reproduction model.

[0051] The security incident reproduction model includes a video segmenter, a multimodal event injection module, a potential action model, and a dynamic prediction model.

[0052] This application uses a video tokenizer to encode security event-related videos into a video token sequence.

[0053] The multimodal event injection module encodes multimodal data related to security events into a sequence of multimodal event tokens.

[0054] The latent action model is encoded into an action representation based on the video token sequence and the multimodal event token sequence.

[0055] The dynamic prediction model outputs a predicted token sequence and a key point probability sequence for the security event based on the video token sequence and the action representation, thereby achieving high-fidelity reproduction of the entire security event development process and effectively improving the determination of key moments in the security event.

[0056] In this embodiment, rigorously processed training data is input into the security event reproduction model. Through multiple rounds of iterative optimization, the security event reproduction model learns to predict the future visual development sequence based on the initial multimodal state information of the event and accurately identify the key moments for decision-making.

[0057] The following section provides a detailed introduction to the video segmenter, multimodal event injection module, latent action model, and dynamic prediction model included in the security incident reproduction model, including (1)-(4): (1) The video tokenizer is used to encode security event-related videos into a video token sequence.

[0058] In one possible implementation, the video tokenizer, based on the spatial-temporal attention mechanism and the core architecture of VQ-VAE, transforms the input security event video into a discrete sequence of video tokens.

[0059] Among them, the spatial-temporal attention mechanism is a method that uses self-attention to collaboratively model the spatial and temporal dependencies in sequential data. The spatial-temporal attention mechanism can include the ST-Transformer structure.

[0060] For example, the encoding process of this video segmenter includes two stages: In the first stage, the video segmenter first receives raw video data related to safety events as input. The encoder uses the encoder component in VQ-VAE to extract features frame by frame from the input video frame sequence. Based on this, the ST-Transformer structure in the space-time attention mechanism is adopted. At the spatial attention layer, semantic associations between various safety-critical areas are calculated within a single frame. These areas include, but are not limited to: flight instrument panel display areas, warning indicator areas, control device operation areas, and key elements of the external environment (runway markings, taxiway markings, obstacles), etc. This layer identifies the static spatial feature vectors of safety-related equipment status and spatial layout. At the temporal attention layer, the dynamic evolution patterns of specific safety elements are tracked along the time dimension, including the temporal transition patterns of warning states, the continuous execution characteristics of operational actions, and the temporal feature vectors of safety events such as the gradual changes in environmental conditions.

[0061] In the second stage, the continuous spatiotemporal feature vectors extracted in the first stage are input into the vector quantization module of VQ-VAE. This module assigns a discrete visual semantic code to each continuous vector by querying a pre-trained security visual codebook. Specifically, it calculates the distance (e.g., Euclidean distance) between the continuous vector and all vectors in the codebook and selects the closest codebook entry, using its index as the discrete video token for that frame. Finally, the entire video is transformed into a sequence of video tokens arranged in chronological order. The security visual codebook is trained from security event video data through unsupervised learning.

[0062] (2) The multimodal event injection module is used to encode multimodal data related to security events into a multimodal event token sequence.

[0063] In one possible implementation, the various encoders integrated within the multimodal event injection module, which process text, speech, and file formats, can share part or all of the encoding backbone network with the video segmenter, or be initialized using the same type of pre-trained model. This ensures that the features of different modalities have a consistent distribution in the latent space, thereby reducing the difficulty of subsequent multimodal alignment and fusion. Alternatively, each encoder can independently design its network structure and training strategy based on the specific modal data it processes (such as the temporal nature of speech, the semantic nature of text, and the numerical nature of parameter data). This application does not limit the specific network implementation of each encoder; its core function is to transform the training data of the corresponding modality into a discrete token sequence that can be processed by subsequent models.

[0064] In one possible implementation, the multimodal event injection module is also configured with an external interface for receiving new multimodal conditions input during model execution. This multimodal event injection module is configured to: receive input through the interface, call its integrated encoder to encode the input into a multimodal event token sequence in real time, and output it to the security event reproduction model to influence its subsequent inference and generation process.

[0065] (3) The latent action model is used to encode action representations based on video token sequences and multimodal event token sequences.

[0066] Among them, action representation is used to describe the hierarchical coding of the operational intentions and specific operational steps performed by participants in a security incident at a specific moment.

[0067] Operational intent refers to the high-level semantic objective corresponding to standard operating procedures, operational strategies, or decision-making logic, i.e., high-level actions. For example, high-level actions include, but are not limited to: go-around, abort takeoff, hold, runway change, diversion, adjustment of arrival and departure procedures, secondary visual confirmation or verification, initiation of de-icing / anti-icing procedures, release withdrawal or reissue, emergency declaration, and ground control handover or repetition, etc.

[0068] Specific operational steps refer to the discrete low-level operations or communication scripts that need to be executed sequentially to achieve the operational intent, i.e., low-level actions. For example, low-level actions include, but are not limited to: standard call / repeat / confirmation scripts, radio frequency selection or seat switching, subtle manipulation of aircraft thrust, speed, altitude or heading, specific sequence of light / alarm / switch positions, and ground vehicle command gestures or shuttle bus dispatch instructions, etc.

[0069] In some embodiments, action representation includes: high-level action representation and low-level action representation.

[0070] High-level action representations characterize the semantic intent corresponding to standard operating procedures. Low-level action representations characterize the specific sequence of operations used to achieve the semantic intent. In other words, high-level action representations refer to the encoding related to the aforementioned high-level actions; low-level action representations refer to the encoding related to the aforementioned low-level actions.

[0071] In one possible implementation, the potential action model includes: a low-level action branch and a high-level action branch. Specifically, based on data related to low-level actions in the video token sequence and the multimodal event token sequence, the low-level action branch is used to process and determine the low-level action representation; based on data related to high-level actions in the video token sequence and the multimodal event token sequence, the high-level action branch is used to process and determine the high-level action representation.

[0072] In one possible implementation, the latent action model is based on VQ. The VAE framework is constructed using a hierarchical quantization architecture. It receives video token sequences output by the video segmenter and multimodal event token sequences output by the multimodal event injection module as input. Internally, the model contains two action branches: a high-level branch (approximately 128 characters) and a low-level branch (approximately 32 characters), each corresponding to a discrete codebook. The high-level action codebook (approximately 128 characters) is semantically aligned with SOPs and used to encode procedural intents; the low-level action codebook (approximately 32 characters) encodes specific operational primitives. During the inference phase, the model analyzes the evolution patterns of multimodal states and outputs a hierarchical action representation in a discrete semantic space that characterizes the operational intent and control commands of security event participants.

[0073] In one possible implementation, the latent action model is semi-supervised trained based on the VQ-VAE framework. Its training objective jointly optimizes the unsupervised reconstruction loss and the supervised alignment loss. The unsupervised reconstruction loss drives the model to learn action primitives that can effectively predict future states: the model maps historical context to continuous vectors through an encoder, obtains discrete action representations through vector quantization, and then inputs them along with the historical context into the decoder to predict future states. The supervised alignment loss introduces limited expert-annotated data (such as procedural labels for critical decision-making moments), constraining the discrete primitives in the quantized codebook to semantically align with standard flight procedures and operating regulations through high-level action compliance classification loss and low-level action alignment loss with objective evidence (such as CVR, FDR), thereby ensuring that the learned action representations possess both data-driven efficiency and business logic interpretability.

[0074] (4) The dynamic prediction model is used to output the predicted token sequence and key point probability sequence of security events based on video token sequence, multimodal event token sequence and action representation. The key point probability sequence is used to characterize the probability of each time point in the security event as a key point.

[0075] In this application, the dynamic prediction model receives video token sequences, multimodal event and action token sequences as input, and outputs a multimodal prediction token sequence and a key point probability sequence for the future development of security events. The prediction token sequence is used to reconstruct or deduce the complete state evolution of the event in multiple dimensions such as vision, voice, and data with high fidelity, providing a visualized simulation scenario for accident analysis; the key point probability sequence automatically identifies the core decision moments and risk inflection points in the event development process, providing investigators with precise intervention analysis anchors. Together, they constitute a structured digital evidence chain that supports in-depth causal analysis, counterfactual deduction, and strategy optimization.

[0076] In one possible implementation, the dynamic prediction model is built upon ST-Transformer and MaskGIT decoder, outputting a predicted token sequence and a keypoint probability sequence for security events. Inputs include a multimodal token sequence (fused from a token sequence and a multimodal event token sequence), action representations inferred from a latent action model, and other relevant knowledge vectors. The dynamic prediction model, through its MaskGIT decoder, first masks the actual data corresponding to the predicted data, then sequentially predicts the visual, speech, and event compliance multimodal tokens for the next time step, collectively forming a complete discretized representation of the future security event state—the predicted token sequence. Simultaneously, this dynamic prediction model analyzes the contextual features of the generation process through a parallel time-series classification head, outputting a keypoint probability sequence identifying critical decision moments.

[0077] The ST-Transformer, acting as the encoder, performs spatiotemporal encoding on the historical multimodal token sequence to form a deep understanding of the event state. The MaskGIT decoder, on the other hand, does not predict the multimodal token sequence from left to right when generating the multimodal token sequence for each future frame. Instead, it first initializes all the positions of the tokens to be predicted as a "mask".

[0078] Based on (1)-(4), the video segmenter and the multimodal event injection module encode and fuse multi-source data into a time-aligned token sequence; the latent action model infers hierarchical action representations that conform to standard operating procedures from these tokens; and the dynamic prediction model ultimately generates a future multimodal state sequence and synchronously outputs the probability of key points. This application transforms fragmented evidence of multimodal data into coherent, compliant, and interpretable event dynamic deductions through end-to-end hierarchical modeling and generation, thereby improving the systematicness and authenticity of security event analysis.

[0079] In some embodiments, such as Figure 3 As shown, based on the training data, a security incident reproduction model is trained, including: S301. Train the security incident reproduction model based on the training data to obtain the training loss value.

[0080] The training loss value is determined based on the base loss value and the additional loss value.

[0081] In one possible implementation, the base loss value is used to characterize the difference between the predicted trajectory and the actual trajectory of a security event, and it includes at least: (1) The multimodal token prediction loss value is calculated based on the difference between the predicted token sequence output by the dynamic prediction model and the corresponding real future token sequence, and is used to drive the model to accurately generate future scenarios. As shown in Formula 1, the multimodal token prediction loss value satisfies the following relationship:

[0082] in, This represents the prediction loss value for multimodal tokens; Representing each modality of multimodality: vision Audio ,text ,other wait; It is about time Modal The token prediction results; It is the actual next token sequence; This represents cross-entropy loss, used to measure the difference between predictions and reality in multimodal data.

[0083] (2) The vector quantization loss value refers to the quantization module, which includes reconstruction loss, codebook fitting loss, and commitment loss, and is applied to the video segmenter and latent action model to stabilize the discrete latent space for learning and optimization. As shown in Equation 2, this vector quantization loss value satisfies the following relationship:

[0084] in, This refers to the loss function value of the video word segmenter. This refers to the losses incurred during reconstruction. Raw input data Output data from model reconstruction The squared Euclidean norm of the interval, also known as the mean square error, is used to guide the decoder through the quantized discrete encoding. Reconstructing the input and output The outputs should be as similar as possible to the encoder and decoder. This refers to codebook loss, which aims to improve the embedding vectors in the codebook. Output to encoder To be close enough to cover the distribution of the encoder output, This indicates stopping the gradient; here it only... Towards Move, the encoder does not update, only the codebook is updated. This refers to the commitment loss, the purpose of which is to constrain the encoder output. Avoid arbitrary drifting; ensure it stably approximates the selected codebook vector. , here express No updates, only encoder to fixed Closest, used for encoders only. In the middle, These are the weight parameters corresponding to the commitment loss, which are adjusted as the model changes.

[0085] It should be noted that the VQ algorithm used in the aforementioned video word segmenter and latent action model... The VAE architecture mainly consists of three parts: an encoder, a vector quantization module (including a learnable codebook), and a decoder. The encoder maps the input video frames to continuous features, the vector quantization module discretizes them using the codebook, and the decoder is responsible for reconstructing the video frames from the discrete codes. These three components work together to achieve efficient vision processing. Discrete representation conversion.

[0086] This application achieves comprehensive end-to-end joint training of the video segmenter, latent action model, and dynamic prediction model through collaborative optimization of the base loss value: the multimodal token prediction loss directly drives the dynamic prediction model to generate more accurate future scenes, while the vector quantization loss simultaneously stabilizes the discrete representation spaces of vision and action. This mechanism constrains the predicted development trajectory of the model output to continuously approximate the actual development trajectory, thereby systematically adjusting the internal parameters of the security event reproduction model, making the reproduction and prediction of security events more closely resemble the real situation, and ultimately providing a reliable technical foundation for the accurate analysis, assessment, and intervention of security events.

[0087] In some embodiments, the additional loss value includes at least one of the following: evidence consistency loss value, timeline alignment loss value, and keypoint alignment loss value. The following sections will explain each of these losses: (1) Evidence consistency loss value is used to characterize the difference in modality between the predicted token sequence and the evidence sequence at the time point when there is evidence for a security event.

[0088] In this application, the evidence consistency loss value is calculated by determining the target time using a preset set of evidence points and comparing the predicted token sequences for different modalities at that time with the corresponding evidence sequences. The evidence sequences are real multimodal data sequences associated with the preset evidence points. This evidence consistency loss value aims to ensure that, at the point in time where evidence exists, the generated results of the security event reproduction model must remain consistent with the original evidence across all modalities, thus avoiding the fabrication of facts.

[0089] In one possible implementation, the preset evidence points originate from a set of key moments determined by timestamping and aligning the original multimodal evidence data. For example, before training or inference, by analyzing raw evidence such as videos, voice / text recordings, and sensor time-series data related to security events, key time locations with clear factual support are manually or automatically labeled. These moments are then organized into a structured set of evidence anchors, serving as the true basis that the model must strictly adhere to at the corresponding time points. For instance, key time locations could be the time points of keyframes in videos, the start and end times of speech transcription, or the trigger times of abnormal sensor recordings.

[0090] In some embodiments, as described in Formula 3, the evidence consistency loss value satisfies the following relationship:

[0091] in, This represents the loss value in terms of the consistency of the evidence; A set of time points in time where there is evidence of a security incident; Cross-entropy loss is used to measure the difference in video modalities. and Representing time respectively The real video token sequence and the predicted video token sequence; CTC represents the connection-temporal classification loss, used to measure the difference between speech modalities or text modalities. and Representing time respectively The actual speech modality and the corresponding predicted speech modality, or the representation of time. The true text modality and the corresponding predicted text modality; Mean squared error is used to measure the difference between numerical time-series modes. and Representing time respectively Numerical time-series modes and numerical time-series modes.

[0092] For example, video modality, which is visual data or video data, can come from cockpit surveillance recordings, video streams recorded by external cameras, etc.; voice modality or text modality can be obtained through CVR or ATC call recordings, which include voice conversations between pilots and controllers and their transcribed text; numerical time-series modality mainly refers to sensor data, which can be obtained through FDR or ACMS, etc., and includes numerical sequences of flight parameters, system status, etc. that change over time.

[0093] Specifically, in this application, for each preset evidence point, the predicted output of the security incident reproduction model on modalities such as visual data, speech data, text data, and numerical time-series data needs to be compared with the corresponding real evidence sequence. Specifically, cross-entropy loss is used to measure classification consistency for the visual modality; connection-time classification loss or aligned cross-entropy loss is used for the speech or text modality to handle misalignment between labels and frames; and mean squared error or Huber loss is used to evaluate parameter bias for the numerical time-series modality. The losses of each modality are summed at each evidence point to form the total evidence consistency loss. This method ensures that the output of the security incident reproduction model at known evidence time points is strictly consistent with the facts, while in periods without evidence, the security incident reproduction model can still make reasonable inferences based on learned patterns, thus achieving a balance between fact anchoring and generative freedom.

[0094] In one possible implementation, when the evidence coverage is low (e.g., the proportion of evidence time points to the total time series length is less than a preset threshold, or the length of consecutive periods without evidence exceeds a set window), then only the known key moments are subject to evidence consistency loss as a strong constraint; non-key moments are guided by the prediction loss term to freely extrapolate the security event reproduction model.

[0095] In one possible implementation, for modal evidence such as speech with only segment-level timestamps, all speech token sequences within that time period can be aggregated and constrained using CTC loss as a whole, without the need for mandatory frame-by-frame alignment. For video evidence, only keyframes can be sampled and used for computation instead of all consecutive frames, thereby reducing the computational load. This approach enables efficient and flexible multimodal event reconstruction while ensuring the consistency of key facts. Here, segment-level timestamps refer to annotation or processing based on time periods as the smallest unit, rather than frame-by-frame or time-by-time alignment.

[0096] (2) The time axis alignment loss value is used to characterize the difference in time dimension between the predicted token sequence and the evidence sequence of the security event.

[0097] In this application, the time axis alignment loss value is constructed by mapping the multimodal evidence sequence and the generated predicted token sequence to event embedding sequences respectively, and calculating the soft dynamic time warping distance between the two. This constrains the event time sequence generated by the model to keep it aligned with the real event time sequence, avoiding order reversal or rhythm disorder.

[0098] In some embodiments, as shown in Equation 4, the time axis alignment loss value satisfies the following relationship:

[0099] in, This represents the time axis alignment loss value; This represents the soft dynamic time warping distance calculation function, used to measure the alignment difference between two event sequences in the time dimension; This represents the event sequence extraction function; This represents an event sequence constructed based on the evidence sequence; This refers to the event sequence constructed from the predicted token sequence.

[0100] Specifically, the algorithm first extracts key semantic event nodes from the evidence sequence and the generated predicted token sequence using an event sequence extraction function and encodes them into vector sequences. Then, a soft dynamic time warping distance calculation function is used to measure the difference in temporal alignment between the two event sequences. This algorithm allows for reasonable flexibility in the sequence's timeline, but strictly constrains the global order of events to remain unchanged. This timeline alignment loss value addresses the issue of ensuring that the order and rhythm of events match the actual process even when the content is identical. For example, it ensures that the causal order of "alarm first, then action" is not reversed, thereby enhancing the temporal authenticity and logical consistency of the reconstruction process.

[0101] For example, if the event sequence extracted from the evidence sequence is "alarm occurrence → call → confirmation → execution → result", while the event sequence extracted from the generated sequence is "alarm occurrence → call → execution → confirmation → result", it indicates that the generated events have a temporal reversal in the critical steps of "confirmation" and "execution". The timeline alignment loss quantifies this type of sequence error and guides the generation process to adjust the temporal dependencies between events through gradient updates, gradually approximating the true order and rhythm of the evidence sequence in the output event sequence, thereby ensuring that the reconstruction process conforms to actual causal logic.

[0102] In one possible implementation, the event sequence extraction function can be implemented using rules combined with a lightweight model. This model predefines key event types based on domain knowledge (such as "ATC instruction confirmation," "system alarm trigger," and "start execution procedure" in aviation operating procedures), and automatically detects and extracts the temporal location and semantic embedding representation of the corresponding event nodes from the multimodal input sequence by training a small classification model or sequence labeling model.

[0103] In another possible implementation, the event sequence extraction function can employ a pre-trained multimodal Transformer encoder (such as VideoBERT or its variants). This encoder directly encodes the original multimodal sequence end-to-end and automatically extracts event-level semantic vector sequences using its attention mechanism or specific output layers, without relying on explicit event type definitions or rules, thereby achieving a generalized representation of complex event structures.

[0104] In one possible implementation, It is a differentiable approximation of the classic dynamic time warping (DTW). By introducing a smoothing minimization operator to replace the non-differentiable minimization operation in the original algorithm, it makes the solution process for sequence alignment paths differentiable, thus enabling direct embedding into the end-to-end training process of neural networks and supporting gradient descent-based optimization. By adjusting the smoothing parameter in sDTW, the elasticity of the alignment process can be controlled: the larger the smoothing parameter, the wider the allowed time stretching and compression range of the alignment, and the higher the tolerance for rhythm differences; the smaller the smoothing parameter, the stricter the alignment constraint, and the closer it is to the hard alignment effect of the classic DTW.

[0105] (3) The key point alignment loss value is used to characterize the difference between the key point probability sequence of a security event and the pre-labeled key point time series.

[0106] In one possible implementation, the keypoint probability sequence is a data sequence dynamically predicted by the security event reproduction model during the temporal encoding and inference process of multimodal data input. Specifically, the security event reproduction model outputs a probability value at each time step indicating that the event belongs to a keypoint, and the probability values ​​of all time steps constitute a complete keypoint probability sequence. This sequence is aligned and optimized with the keypoint time series generated based on pre-annotation, enabling the security event reproduction model to learn to identify key event nodes at the correct time positions and providing a temporal basis for subsequent control event injection or event analysis pauses.

[0107] In one possible implementation, pre-labeled key point time series can be obtained through manual analysis and annotation by domain experts using multimodal evidence. This identifies the time windows of key decisions or state transitions in security incidents and serves as a true benchmark for training security incident reproduction models. Alternatively, key points can be identified and labeled using rules.

[0108] In some embodiments, as shown in Equation 5, the keypoint alignment loss values ​​satisfy the following relationship:

[0109] in, Represents the binary cross-entropy loss function; Indicates the time of prediction The probability of being a key point; This indicates the pre-labeled keypoint labels, at time... When located within the time window of the labeled key point Otherwise ; These are preset weighting coefficients used to balance the contributions of classification loss and temporal localization loss; A set representing keypoint indices; Indicates the predicted first A crucial moment; Indicates the relationship with the first The real time corresponding to the preset annotations of each key point.

[0110] Specifically, in this application, the keypoint alignment loss uses a binary cross-entropy function to make the predicted keypoint probability as close as possible to 1 within the actual keypoint time window and as close as possible to 0 outside the window. At the same time, a time deviation penalty term is used to further constrain the absolute error between the predicted keypoint time and the actual annotation time, thereby achieving accurate alignment of the temporal position of the keypoint while ensuring accurate identification of the keypoint's existence.

[0111] In one possible implementation, if the keypoint annotations only provide an approximate time interval without specifying precise times, all times within that interval can be considered as positive samples (i.e., corresponding to...). This method guides the probability peak to occur within a given interval using a relaxed supervision approach. If multiple key points exist, the above loss can be calculated separately for each point and then summed. Hyperparameters This design is used to balance temporal accuracy and classification confidence. In actual deployment, it can be dynamically adjusted according to task requirements or an annealing strategy can be used to gradually tighten temporal constraints. This design allows the model to learn the existence of key points with a relatively wide time window in the early stages of training, and then gradually improve the localization accuracy, ultimately achieving reliable identification and alignment at critical decision moments.

[0112] This application's keypoint alignment loss is achieved by jointly optimizing the keypoint prediction sequence output by the security event reproduction model and the expert-annotated real keypoint sequence using binary cross-entropy loss and temporal deviation penalty. This enables the security event reproduction model not only to identify the existence of key decision points but also to accurately pinpoint their occurrence time. This loss ensures that the model can automatically pause or trigger control signals at the correct time points during inference, thus providing reliable temporal anchor support for security event analysis. For example, this mechanism is equivalent to automatically adding "pause markers" to the video stream, allowing the model to learn to accurately pause at key locations where it should stop (such as "system alarm trigger" or "human intervention") to support subsequent analysis or strategy injection.

[0113] Based on the training loss framework built on S301, this design integrates a weighted fusion of the basic loss with multiple structured alignment losses (evidence consistency, timeline alignment, and keypoint alignment). This allows the security incident reconstruction model to simultaneously optimize its event generation capabilities and multi-dimensional factual constraints during training. This design enables the model not only to learn the dynamic evolution of security incidents but also to strictly align with external evidence at key time points, maintain the correct temporal logic of events, and accurately locate key decision nodes. Consequently, the output results achieve high factual consistency, strong temporal rationality, and reliable keypoint response capabilities, significantly improving the credibility and interpretability of security incident reconstruction.

[0114] S302. Update the weight parameters of the security event reproduction model based on the training loss value.

[0115] As described in S301, the training loss value is determined jointly by the base loss value and the additional loss value. As shown in Formula 6, the specific calculation formula is as follows:

[0116] in, This represents the training loss value; and The basic loss value is used to guide the model in learning the dynamic evolution of security events, thus forming a basic generative capability. for The corresponding adjustable weights; , , The additional loss values ​​represent the evidence consistency loss, timeline alignment loss, and keypoint alignment loss, respectively. , , These are the adjustable weighting coefficients corresponding to each additional loss term, used to balance the contributions of different supervision objectives to the overall training.

[0117] In one possible implementation, during training, the backpropagation algorithm is used to calculate... The gradients of each weight parameter of the model are calculated, and the parameters of the security event reproduction model are updated based on gradient descent or its variants (such as the Adam optimizer). This allows the security event reproduction model to gradually optimize its generation capabilities and satisfy various alignment constraints during iteration, ultimately obtaining a security event reproduction model with high factual consistency, temporal rationality, and key point identification capabilities.

[0118] Backpropagation and gradient descent refer to the process in each training iteration where, based on the calculated training loss value, the gradient of the loss with respect to each weight parameter of the model is calculated layer by layer through the backpropagation algorithm, and the weight parameters are updated along the opposite direction of the gradient based on the gradient descent optimization method. This allows the model to gradually reduce the loss value, optimize its generation capability and alignment performance during iteration, and finally converge to a stable and usable state for reproducing safe events.

[0119] For example, suppose , , , At this point, the total training loss function is: This configuration ensures that the model's output is strictly consistent with the facts at known time points by assigning the highest weight to the evidence consistency loss; at the same time, it introduces timeline alignment and key point alignment constraints with lower weights, gradually guiding the model to meet the requirements of temporal logic and key node positioning without excessively suppressing the model's degree of freedom in generation. This achieves a balance between factual accuracy, temporal rationality, and key decision identifiability in multi-objective optimization.

[0120] Based on steps S301-S302, this application determines the trained security event reproduction model through the following process: the security event reproduction model is iteratively trained based on training data, and in each round of training, the training loss value obtained by weighted combination of the basic loss value and the additional loss value is updated through backpropagation and gradient descent. After multiple rounds of optimization, the model gradually converges to a comprehensive goal that can simultaneously satisfy the generation quality, evidence alignment, temporal rationality and key point localization, thereby obtaining a trained security event reproduction model with high-fidelity reconstruction and credible inference capabilities.

[0121] In one implementation, event injection allows for the dynamic introduction of exogenous condition changes or human intervention adjustments at any point during the security event reproduction process. The security event reproduction model is then replanned based on the current state and the injected event, and the impact of the event is reflected in subsequent simulation steps. Each event injection can automatically generate a structured "event card" that records the trigger source, impact boundary, recommended handling measures, and optional strategies, thereby supporting multi-path simulation and intervention analysis of complex security events.

[0122] For example, dynamically introducing changes in exogenous conditions or adjustments made by human intervention can include: sudden weather changes: real-time changes in meteorological conditions such as sudden drops in visibility, wind shear warnings, and thunderstorm formation; traffic conflicts: ground operation interference such as temporary vehicles entering taxiways, changes in construction areas, and unexpected occupation of waiting points; runway events: sudden scenarios such as runway incursions, runway occupation, and conflicts between go-around and takeoff timings; communication anomalies: communication failures such as misrepeated instructions, unconfirmed critical calls, and out-of-level calls; program interruptions: operational program anomalies such as missing or overdue de-icing / anti-icing, equipment failure alarms, and shutdown of navigation aids; system failures: emergency states triggered by sudden problems in aircraft systems such as engine failures, abnormal doors, and cargo hold alarms; resource constraints: operational bottlenecks caused by temporary shortages of ground support resources (such as towing vehicles, shuttle vehicles, and de-icing stations); and rule updates: the real-time effectiveness and impact of airworthiness directives, service notices, or operational restrictions during the simulation process.

[0123] It should be noted that the various data involved in event injection can be processed and represented by the corresponding encoders in the multimodal event injection module. This module is designed with bidirectional compatibility: it can be used for unified encoding and post-analysis of input multimodal historical evidence, and it can also receive and encode newly injected event information in real time during the inference process, thereby supporting dynamic replanning. This application does not restrict the specific stage of use of the multimodal event injection module in pre-encoding or in-process injection; it can be flexibly configured according to actual application scenarios.

[0124] In one possible implementation, the encoders in the multimodal event injection module are implemented based on the Transformer architecture. This architecture dynamically calculates the semantic association weights within and across modalities of multimodal input sequences (such as speech-to-text, parametric time-series data, and regulations) through a self-attention mechanism, effectively capturing the global dependencies and causal relationships between scattered evidence in security events. This design enables the module to efficiently align and fuse heterogeneous and asynchronous multi-source evidence in a unified latent space, generating a context-coherent multimodal event token sequence, providing input conditions for subsequent high-fidelity event reconstruction by the security event reproduction model.

[0125] In some embodiments, the multimodal event injection module in step S202 above includes multiple dedicated encoders, specifically including: a text encoder, a voice encoder, an image encoder, a file encoder, and a sensor data encoder. These encoders are described in detail below (1) to (5): (1) A text encoder is used to encode text data in multimodal data related to security events into a text token sequence.

[0126] In one possible implementation, the text encoder is implemented using the DeBERTa model. DeBERTa is a pre-trained language model based on the Transformer architecture. By decoupling the attention mechanism and enhancing masked language modeling, it can deeply transform the input natural language text into semantically rich vector representations. This model is particularly suitable for understanding the complex logic and implicit intentions of normative texts in the aviation field. Its output semantic vectors can effectively support the model's accurate identification and encoding of high-level operational intentions, procedural compliance, and environmental setting requirements. The vector representation is ultimately quantized into a sequence of text tokens, which serve as multimodal conditional inputs to guide the model in scene reconstruction and deduction under semantic constraints.

[0127] (2) A speech encoder is used to encode speech data in multimodal data related to security events into a speech token sequence.

[0128] In one possible implementation, the speech encoder is implemented using the wav2vec2.0 model. wav2vec2.0 is a self-supervised learning-based speech processing model capable of directly extracting robust acoustic and linguistic features from raw audio signals, mapping spoken commands from pilots or controllers into semantic vector representations or transcribed text tokens. The output semantic vectors or text tokens are further used as event condition tokens, providing precisely time-aligned speech semantic conditional inputs to the event reproduction model.

[0129] (3) Image encoder, used to encode image data in multimodal data related to security events into a sequence of image tokens.

[0130] In one possible implementation, the image encoder is implemented using the CLIP ViT-L / 14 model. CLIP is a vision encoder pre-trained on large-scale image-text pairs. The language model's visual encoder can transform image content into vector representations highly aligned with text semantics. This model is particularly suitable for processing visual evidence with explanatory text or that needs to be associated with report text. Its output visual semantic vectors are quantized to form a sequence of image tokens, which serve as conditional inputs consistent with the text token semantic space, supporting accurate visual reconstruction of security incident scenarios and cross-modal evidence association analysis.

[0131] (4) A file encoder is used to encode structured file data in multimodal data related to security events into a sequence of file tokens.

[0132] In one possible implementation, the document encoder employs a pre-trained model (such as Longformer) designed for long text and table structures. This model, by combining local window attention and global attention mechanisms, can efficiently process long documents or semi-structured tables such as weather bulletins, operational configurations, and flight notices, extracting key fields, parameters, and relationships. The extracted structured information is mapped into semantically rich vector representations and ultimately quantized into a sequence of document tokens. This sequence serves as a highly reliable structured conditional input, directly used to initialize or correct scene state parameters and constrain the generation of compliant actions in the potential action model.

[0133] (5) Sensor data encoder, used to encode sensor data in multimodal data related to security events into a data stream token sequence.

[0134] In one possible implementation, a sensor data encoder is used to encode sensor data related to safety events (such as time-series data streams derived from FDR and ACMS) into a data stream token sequence. The sensor data encoder is implemented using a neural network model (such as ST-Transformer) designed for continuous time series. This model uses a temporal attention mechanism to jointly model multi-channel real-time signals such as wind speed, engine speed, and airborne alarms, capturing their long-term dependencies and abnormal evolution patterns, and outputting a physically meaningful temporal embedding representation. This temporal embedding, after quantization, forms a data stream token sequence, which serves as the most objective input to the system's physical state conditions, directly constraining the generation of low-level actions in the potential action model and driving the dynamic prediction model to perform simulations and deductions that conform to physical laws.

[0135] In one possible implementation, the multimodal event injection module further includes a video encoder. Exemplarily, the video encoder uses DINOv2 as its backbone network. This model, pre-trained on large-scale unlabeled data, is capable of extracting highly generalized and semantically rich frame-level visual features. For video evidence related to safety events (such as cockpit monitoring and runway surveillance footage), the video encoder first extracts features frame by frame, accurately capturing key visual elements such as dashboard alarm status, runway markings outside the cockpit, and personnel gestures. Then, through superimposed temporal modeling layers (such as spatiotemporal attention mechanisms), these frame features are fused across frames to construct a continuous video-level representation containing the dynamic evolution of the event. This representation is further mapped into a discrete sequence of visual tokens through a vector quantization module, thereby providing the multimodal event injection module with a visual thread of the safety event that is semantically and temporally aligned with evidence such as voice and data, faithful to the original visual facts.

[0136] It should be noted that the implementation of this video encoder can share or reuse the network structure and parameters with the encoder in the aforementioned video segmenter, or it can be designed and trained independently as a separate module. This application does not limit its specific network implementation. In some embodiments, the security event reproduction model further includes: an external memory; the external memory is used to provide historical event data during the training of the security event reproduction model.

[0137] In one possible implementation, an external memory stores historical security event information in a structured format, typically indexed and managed according to dimensions such as "event type—timeline—multimodal evidence—response measures—event outcome." During the training phase, the external memory can provide auxiliary samples or features to enhance the model's generalization ability to similar events. During the inference phase, the model can query the external memory to obtain patterns and response strategies of relevant historical events, serving as reference constraints for the current generation process, thereby improving the rationality and interpretability of the reconstruction results.

[0138] In one possible implementation, the dynamic prediction model generates a multimodal prediction token sequence for the next time step based on historical multimodal token sequences, potential actions, and memory vectors retrieved from an external memory bank. This enables continuous scenario reproduction and dynamic strategy deduction of security events. By integrating temporal context, action planning signals, and historical experience, the model ensures the continuity of event development while supporting real-time deduction and evaluation of multiple possible paths.

[0139] In summary, the architecture diagram of a security event reproduction model provided in this application embodiment is as follows: Figure 4The process involves taking video data related to the security incident as input, which is then converted into a video token sequence representing the flight environment by a video segmenter. The potential action model includes low-level and high-level action branches, with the high-level branch outputting a high-level action codebook and the low-level branch outputting a low-level action codebook. Finally, based on the video token sequence, the high-level action codebook, and the low-level action codebook, a dynamic prediction model outputs a predicted frame for the recurrence of the security incident.

[0140] This also includes a controller that influences the dynamic prediction model based on an external memory bank.

[0141] This includes text encoders for inputting text commands, speech encoders for inputting speech commands, image encoders for inputting images, video encoders for inputting videos, structured file encoders for inputting files, and sensor data encoders for inputting sensor data. The corresponding data encoded by the text encoder, speech encoder, image encoder, video encoder, and file encoder will affect the higher-level action branches and dynamic prediction models, while the data processed by the sensor data encoder will affect the lower-level action branches and dynamic prediction models.

[0142] It should be noted that the aforementioned encoders collectively constitute the multimodal event injection module. This module's function is to uniformly encode input data from different modalities into a latent representation with an aligned structure to the video frame sequence, serving as the initial input for the security event reproduction model. Whether it's the known multimodal conditions serving as the model's starting point or external event signals injected midway, all are processed by the corresponding encoders within this module, ensuring the effective integration of multimodal information and its consistency with the overall model input.

[0143] This application also provides a method for reproducing security incidents, applicable to... Figure 1 The security event reproduction application device shown is equipped with the security event reproduction model provided in the above embodiments. For example... Figure 5 As shown, the method for reproducing this security incident includes: S501. Acquire multimodal data related to security incidents; In one possible implementation, acquiring multimodal data related to safety incidents refers to collecting and integrating temporally correlated visual, audio, text, document, and sensor data from various data sources. Specifically, this may include surveillance video clips, cockpit voice recordings, flight parameter time-series records, operation log text, and manually annotated key event time points. After time alignment and format standardization, this data constitutes a multimodal input sequence for safety incident reconstruction and analysis. This multimodal data can be processed using a video segmenter and a multimodal event injection module within the safety incident reproduction model.

[0144] S502. Based on multimodal data related to security events, determine the predicted token sequence and key point probability sequence of security events through a security event reproduction model.

[0145] The security incident reproduction model is trained using the training method provided in the above embodiments.

[0146] In one possible implementation, based on multimodal data related to security events, a predicted token sequence and a key point probability sequence of security events are generated through a trained security event reproducibility model. The security event reproducibility model is trained using the method of jointly optimizing the basic loss, evidence consistency, time axis alignment, and key point alignment as described in the previous embodiment, and has the ability to reconstruct multimodal time series data with high fidelity and identify key nodes.

[0147] In one possible implementation, the security event reproduction model supports event injection, allowing the dynamic introduction of external event changes or human intervention strategies at any point during the simulation. Specifically, when an injected event occurs, the model, based on the currently generated predicted token sequence, key point probability sequence, and historical event patterns in an external memory, immediately replans the subsequent state evolution path and updates the multimodal output to reflect the event's impact. This method enables the simulation of various real-world disturbances such as sudden weather conditions, equipment failures, communication anomalies, or program interruptions, thereby achieving multi-path, interactive simulation and strategy evaluation of security events, enhancing the model's adaptability and decision support capabilities in complex dynamic scenarios.

[0148] S503, based on the prediction of security event token sequences and key point probability sequences, to reproduce security events and locate key points in security events.

[0149] In one possible implementation, the predicted token sequence is decoded in chronological order into corresponding multimodal outputs (such as visual images, voice statements, parameter curves, etc.) to reconstruct the continuous development process of the security event.

[0150] In one possible implementation, decoding the predicted token sequence into multimodal output can be achieved as follows: visual tokens are reconstructed into a video frame sequence using a visual decoder (such as a ViT decoding layer or a generative adversarial network); speech / text tokens generate corresponding speech waveforms or natural language descriptions using a vocoder or text synthesis model; and numerical time-series tokens are converted into continuous sensor parameter curves using a regression decoder. The decoding processes for each modality are synchronized on the timeline, and are ultimately fused to generate a coherent multimodal event sequence, thereby reconstructing the complete dynamic development process of the security event.

[0151] In one possible implementation, the security incident reproduction model supports key point pauses. When the probability of a key point output by the model during the simulation exceeds a preset threshold, a pause is automatically triggered at that moment, generating a structured prompt message containing the current event status, context analysis, and recommended handling options. This mechanism allows analysts to intervene at key decision points to manually confirm, adjust strategies, or inject new events, after which the model continues simulation based on the updated status. Key point pauses can be triggered based on preset event types or dynamically determined according to the probability sequence output by the model in real time, thereby achieving controllable interruption and interactive analysis of the simulation process.

[0152] For example, in the closed-loop interactive simulation of "user input → safety event reproduction model processing → scenario and evidence response → indicator changes and prompts", users can dynamically intervene in the event process through various means such as inserting meteorological conditions, editing operation notices, setting runway status, injecting alarms, adjusting taxiing strategies, triggering go-around decisions, and interacting with virtual characters. The safety event reproduction model responds to input in real time and updates the presentation of multimodal scenarios and evidence, and synchronously outputs changes in key operation indicators and decision prompts. Finally, it can generate a multi-strategy comparison report with one click, realizing interactive analysis, multi-path simulation, and result evaluation of the evolution process of safety events.

[0153] In some embodiments, the security incident reproduction method further includes evaluating the security incident reproduction, including risk score, expected loss, regulatory / SOP compliance, latency, resource consumption, and evidence coverage.

[0154] In one possible implementation, the risk score is calculated strictly based on the "event layer" token sequence generated by the dynamic prediction model. This sequence is generated autoregressively by the model and includes discrete semantic labels such as [Event: Wind Shear Warning], [Status: Abnormal Airspeed Decrease], and [Compliance: Below Minimum Safe Altitude]. A built-in quantitative dictionary, bound to the company's risk management framework, assigns severity weights and time decay coefficients to each event label. By identifying consecutively occurring risk labels in the projection sequence, real-time weighting and time integration are performed, ultimately outputting a dynamic risk curve spanning the entire process and its cumulative total. For example, a 10-second "Wind Shear Warning" (weight 0.8) and a momentary "Overspeed Warning" (weight 0.5) will be accumulated to a risk score of (0.8 × 10) + (0.5 × 1) = 8.5. This score directly and objectively reflects the overall risk exposure level of the projection path.

[0155] In one possible implementation, the expected loss is calculated based on the projected final state (defined by the event-layer endpoint labels generated by the model, such as [Outcome: Successful Landing], [Outcome: Aborted Takeoff]) and its associated probability vector. This probability vector is output synchronously by the dynamic prediction model when generating multiple outcome branches. A loss function table is preset, associating resource costs, time costs, and safety costs for each outcome (e.g., the loss for "returning to base" is fuel + flight cancellation costs, and the loss for "serious accident" is a preset extremely high value). The expected loss is calculated using the formula Σ(outcome probability × outcome loss). For example, if the model projects a 70% probability of "returning to base" (loss of 100,000 yuan) and a 30% probability of "diversion" (loss of 150,000 yuan), then the expected loss is 0.7 × 10 + 0.3 × 15 = 115,000 yuan. This indicator provides decision-makers with an expected economic loss based on probability statistics.

[0156] In one possible implementation, regulatory / SOP compliance is calculated based on hierarchical action representations output by a potential action model. High-level action representations (e.g., [High-level action: Perform go-around procedure]) and low-level action representations (e.g., [Low-level action: Thrust handle to TO / GA], [Low-level action: Report to tower]) are input into a rules engine. This engine embeds a state machine and logical constraints extracted from company SOPs and regulations (e.g., "Perform go-around procedure" must include "Thrust handle to TO / GA" and must precede "Report to tower"). Each action sequence is checked against all constraints, and a compliance score is ultimately output, for example: number of compliant action steps / total number of action steps × 100%. If a necessary checklist confirmation step is omitted during the simulation, the compliance score is reduced accordingly, directly quantifying the procedural compliance of the operation.

[0157] In one possible implementation, the latency metric is obtained by analyzing a timestamped sequence of events generated by a parsing model and comparing it with a standard time template. Key task nodes (such as the time interval between "alarm triggering" and "first corrective action") are extracted from the event sequence and compared with the maximum response time specified in the Standard Operating Procedure (SOP) (such as "wind shear alarm should respond within 3 seconds"). Latency is calculated as actual time elapsed minus standard time elapsed, and all time-out tasks are summed or averaged. For example, if the standard response time is 3 seconds and the actual time elapsed in the simulation is 5 seconds, then this latency is 2 seconds. This metric directly measures the shortcomings in the timeliness of the handling process.

[0158] In one possible implementation, resource consumption is quantified based on a time-series token sequence generated by a dynamic prediction model. This sequence contains consecutive parameter tokens, such as [parameter: fuel flow rate] and [parameter: engine cycle count]. Consumption is calculated by numerically integrating key parameter curves (e.g., total fuel consumption = Σ fuel flow rate × time step) or by counting events (e.g., brake usage count = number of occurrences of the [event: braking action] token). For example, in a 10-minute coasting simulation, the fuel flow rate integral result is 120 kg, and the braking action count is 8. These data are derived directly from the generated physical parameters and reflect the operational economy of the strategy.

[0159] In one possible implementation, evidence coverage is calculated by comparing the generated token sequence with the evidence anchors aligned with the multimodal event injection module. At each known evidence time point (e.g., the moment of the "Mayday" call in CVR, the frame of the engine parameter mutation in FDR), it is checked whether the corresponding modal token generated by the model is semantically and numerically consistent with the original evidence (e.g., whether the generated voice token contains "Mayday," and whether the generated parameter token reflects the same mutation). The coverage calculation formula is the number of matched evidence anchors / the total number of evidence anchors. For example, if the deduction perfectly recreates 9 out of 10 evidence anchors, the evidence coverage is 90%. This metric is the core basis for evaluating the credibility and factual fidelity of the deduction results.

[0160] In some embodiments, the security incident reproduction method further includes outputting a report on the reproduction of the security incident.

[0161] In one possible implementation, the report outputs a structured analysis report automatically generated based on the simulation results and evaluation indicators of a security incident reproduction model. The report includes at least the following parts: an event timeline reconstruction graph, a key decision point matrix comparison table, a multi-dimensional quantitative evaluation summary table, and a dual-track playback link for evidence and generation. The key decision point matrix comparison table clearly displays, in tabular form, the changes in core indicators such as subsequent risk scores, compliance, and latency resulting from different handling strategies (including historical real strategies and injected counterfactual strategies) at each identified key point. The multi-dimensional quantitative evaluation summary table comprehensively ranks all simulation paths and highlights the optimal strategy in terms of risk, compliance, and efficiency. The report supports interactive drill-down; users can directly jump to the corresponding dual-track playback scenario for detailed review by clicking on any cell in the matrix. Finally, the report is output in a standardized, auditable format and supports importing into an enterprise security management system as a structured case for knowledge accumulation.

[0162] This application, through the continuous review and dynamic simulation capabilities of a safety incident reproduction model, can fully reproduce the entire process of a safety incident from occurrence and development to handling. It supports multiple rounds of interactive strategy injection and plan adjustment at key nodes, thereby continuously optimizing the handling path and accumulating decision-making experience. The standardized response strategy library formed based on this process can not only be used for reviewing and analyzing similar incidents and preventing risks, but also serve as aviation safety training and simulation materials. It provides highly realistic, interactive, and immersive safety education scenarios for personnel in various positions, including pilots, maintenance personnel, dispatchers, control tower staff, and ground service personnel, enhancing their collaborative handling capabilities and safety risk awareness across the entire chain.

[0163] This application also provides an electronic device, which can be the aforementioned... Figure 1 A training device for security incident models or a device for reproducing security incidents. Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device includes a processor 601 and a memory 602; the memory 602 stores instructions executable by the processor 601; when the processor 601 is configured to execute the instructions, the electronic device implements the method described in the foregoing method embodiments.

[0164] This application also provides a computer-readable storage medium storing computer program instructions thereon; when the computer program instructions are executed by a computer, the computer causes the computer to implement the methods described in the foregoing embodiments. The computer may be an electronic device, a network device, or a manager. The computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0165] This application also provides a computer program product that, when run on a computer, causes the computer to execute the relevant method steps described in the above method embodiments.

[0166] The electronic devices, computer-readable storage media, or computer program products provided in this application are all used to perform the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0171] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0172] In the description of the embodiments of this application, specific features, structures, materials or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0173] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for a security incident reproduction model, characterized in that, The method includes: Obtain training data; Based on the training data, the security event reproduction model is trained; wherein, the security event reproduction model includes a video segmenter, a multimodal event injection module, a latent action model, and a dynamic prediction model; The video segmenter is used to encode security event-related videos into video token sequences. The multimodal event injection module is used to encode multimodal data related to security events into a multimodal event token sequence; The latent action model is used to encode action representations based on the video token sequence and the multimodal event token sequence; The dynamic prediction model is used to output a predicted token sequence and a key point probability sequence for the security event based on the video token sequence, the multimodal event token sequence, and the action representation. The key point probability sequence is used to characterize the probability of each time point in the security event being a key point.

2. The method according to claim 1, characterized in that, The multimodal event injection module includes: A text encoder is used to encode text data in the multimodal data related to the security event into a text token sequence; A voice encoder is used to encode voice data in the multimodal data related to the security event into a voice token sequence; An image encoder is used to encode image data in the multimodal data related to the security event into an image token sequence; A file encoder is used to encode structured file data in the multimodal data related to the security event into a sequence of file tokens; A sensor data encoder is used to encode sensor data in the multimodal data related to the security event into a data stream token sequence.

3. The method according to claim 1, characterized in that, The step of training the security incident reproduction model based on the training data includes: The security incident reproduction model is trained based on the training data to obtain the training loss value; The weight parameters of the security event reproduction model are updated based on the training loss value; The training loss value is determined based on the base loss value and the additional loss value; the base loss value is used to characterize the difference between the predicted development trajectory and the actual development trajectory of the security event. The additional loss value includes at least one of the following: evidence consistency loss value, time axis alignment loss value, and key point alignment loss value; wherein, the evidence consistency loss value is used to characterize the modal difference between the predicted token sequence and the evidence sequence of the security event at the time point where evidence is available; the time axis alignment loss value is used to characterize the temporal difference between the predicted token sequence and the evidence sequence of the security event; and the key point alignment loss value is used to characterize the difference between the key point probability sequence of the security event and the pre-labeled key point time sequence.

4. The method according to claim 3, characterized in that, The loss value for evidence consistency satisfies the following relationship: in, This represents the loss value in terms of the consistency of the evidence; A set of time points in time where there is evidence of a security incident; Cross-entropy loss is used to measure the difference in video modalities. and Representing time respectively The real video token sequence and the predicted video token sequence; CTC represents the connection-temporal classification loss, used to measure the difference between speech modalities or text modalities. and Representing time respectively The actual speech modality and the corresponding predicted speech modality, or the representation of time. The true text modality and the corresponding predicted text modality; Mean squared error is used to measure the difference between numerical time-series modes. and Representing time respectively Numerical time-series modes and numerical time-series modes.

5. The method according to claim 3, characterized in that, The time axis alignment loss value satisfies the following relationship: in, This represents the time axis alignment loss value; This represents the soft dynamic time warping distance calculation function, used to measure the alignment difference between two event sequences in the time dimension; This represents the event sequence extraction function; This represents an event sequence constructed based on the evidence sequence; This refers to the event sequence constructed from the predicted token sequence.

6. The method according to claim 3, characterized in that, The keypoint alignment loss values ​​satisfy the following relationship: in, This represents the time axis alignment loss value; Represents the binary cross-entropy loss function; Indicates the time of prediction The probability of being a key point; This indicates the pre-labeled keypoint labels, at time... When located within the time window of the labeled key point Otherwise ; These are preset weighting coefficients used to balance the contributions of classification loss and temporal localization loss; A set representing keypoint indices; Indicates the predicted first A crucial moment; Indicates the relationship with the first The real time corresponding to the preset annotations of each key point.

7. The method according to claim 1, characterized in that, The action representation includes: a high-level action representation and a low-level action representation; the high-level action representation is used to characterize the semantic intent corresponding to the standard operating procedure; the low-level action representation is used to characterize the specific operation sequence to achieve the semantic intent.

8. The method according to claim 1, characterized in that, The security event reproduction model further includes an external memory; the external memory is used to provide historical event data during the training of the security event reproduction model.

9. A method for reproducing a security incident, characterized in that, The method includes: Acquire multimodal data related to security incidents; Based on the multimodal data related to the security incident, the predicted token sequence and key point probability sequence of the security incident are determined by the security incident reproduction model; wherein, the security incident reproduction model is trained based on the training method as described in any one of claims 1-8; Based on the predicted token sequence and key point probability sequence of the security event, the security event is reproduced and the key points in the security event are located.

10. An electronic device, characterized in that, The electronic device includes: a processor and a memory; The memory stores instructions that the processor can execute; When the processor is configured to execute the instructions, the electronic device implements the training method as described in any one of claims 1-8 or the method as described in claim 9.

Images