A deep learning-based air combat real-time situation comment generation method and system
By constructing a deep learning-based comment text sequence generation network model, the problem of insufficient real-time air combat situation commentary was solved, enabling rapid and accurate automatic commentary generation of air combat situation, supporting real-time decision-making by ground commanders.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU EBOYLAMP ELECTRONICS CO LTD
- Filing Date
- 2024-03-14
- Publication Date
- 2026-07-24
AI Technical Summary
The existing technology for generating air combat situation commentary lacks real-time capability, resulting in human commentary lagging behind the air combat situation and failing to provide ground commanders with timely and accurate air combat insights.
A deep learning-based comment text sequence generation network model is adopted. By collecting air combat structured situation data and human text comment data, a comment text sequence generation network model is constructed. The generation network is trained using deep learning algorithms to achieve automatic conversion of real-time situation feature vectors into human comment content.
It enables real-time generation of air combat situation commentary, improving generation speed and accuracy, and helping ground commanders to more accurately understand and respond to air combat situations, and make timely command decisions.
Smart Images
Figure CN118193605B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of air combat situation commentary generation, specifically to a method and system for generating real-time air combat situation commentary. Background Technology
[0002] In many exercises, the real-time air combat situation refers to the combat situation of the opposing sides in a designated airspace. During the exercise, in order to understand and grasp the air combat situation, ground commanders will collect, integrate and analyze relevant information of air targets to monitor and identify the actions of the opposing sides.
[0003] Real-time air combat situations are typically characterized by their high real-time nature, complexity, and frequent changes. Situation commentary generated from real-time air combat situations can provide ground commanders with perspectives, insights, and recommendations on air combat, thereby helping commanders and decision-makers better understand and respond to air combat situations.
[0004] In current technologies, air combat situation assessments are typically generated by ground-based personnel, requiring reference to various parameters transmitted from the battlefield information transmission system. However, in air combat, units move at extremely high speeds, resulting in delays in the transmitted parameters. Furthermore, the vast amount of parameter information required for manual assessment often lags significantly behind the current air combat situation, rendering the assessments ineffective for command and control. Summary of the Invention
[0005] To address the non-real-time nature of existing manual assessments of air combat situations, the present invention aims to provide a deep learning-based method for generating real-time air combat situation reports. This method boasts excellent generation speed and can assist ground commanders in enhancing their understanding of real-time air combat situations.
[0006] To achieve the above objectives, the technical solution adopted by this invention is as follows: a method for generating real-time air combat situation commentary based on deep learning, characterized in that the method mainly includes:
[0007] S1. Steps for obtaining the comment model;
[0008] S11. Collect structured air combat situation data and human-written commentary data; select key event features from the structured situation data, perform serialization processing, and generate situation feature vectors; preprocess the human-written commentary data to generate human-written commentary content.
[0009] S12. Construct a deep learning-based comment text sequence generation network model, with the situation feature vector as the input and the human comment content as the output.
[0010] S13. Establish the correspondence between the situation feature vector and the content of the manual comment to form a data pair; input the data pair into the comment text sequence to generate a network model and complete the training;
[0011] S2. Steps for generating situational commentary;
[0012] S21. Acquire real-time air combat structured situation data and perform serialization processing to obtain real-time situation feature vectors.
[0013] S22. Input the real-time situation feature vector into the trained comment text sequence generation network model to generate real-time human comment content corresponding to the current situation.
[0014] Therefore, the comment model acquisition step is performed before the actual real-time acquisition of human commentary content. In actual exercises and confrontations, the situation commentary generation step can be performed directly. The comment model acquisition step requires first collecting training data, namely, structured air combat situation data and human text commentary data.
[0015] Structured air combat situation data refers to a series of specific data parameters related to air combat obtained through battlefield information transmission systems or airborne and ground-based radars; human-written commentary data refers to concise but precise verbal commentary; a large amount of the above data is collected and prepared for analysis and screening. At this point, a large amount of historical structured air combat situation data is selected for training.
[0016] First, the large amount of historical air combat structured situational data acquired is screened, unnecessary situational data is removed, and suitable feature data is selected and serialized. Suitable feature data is mainly divided into three categories. For example, in one model, these are aircraft features, missile features, and key event features. These features are serialized, each feature is assigned a unique identifier and flag bit, and after normalization, they are all within the range [0,1], which facilitates model convergence.
[0017] Next, a deep learning-based comment text sequence generation network model is constructed. This model uses a decoder framework and includes multiple decoding layers. It maps the feature vectors obtained after data serialization to the content of human comments, forming data pairs. In other words, it establishes a correspondence between the feature vectors obtained after data serialization and the content of human comments, forming data pairs (x...). i ,y i ). Where x i Let y be the eigenvector. i This represents the human-generated comment content corresponding to this feature vector. Multiple data pairs (x...) i ,y iThe dataset is used to input comment text sequences to generate a network model, which is then trained and saved.
[0018] In actual combat applications, battlefield information transmission systems or air-based and ground-based radars will acquire a large amount of real-time structured air combat situational data. By serializing the above-mentioned real-time structured air combat situational data, real-time situational feature vectors can be obtained. By inputting the above-mentioned real-time situational feature vectors into a trained comment text sequence generation network model, real-time human comment content corresponding to the current situation can be automatically generated. The generation speed is extremely fast and can guarantee the real-time requirements.
[0019] By combining real-time structured air combat situational data with the aforementioned comment text sequence generation network model, corresponding human commentary content can be generated extremely quickly. The generation speed and accuracy far exceed those of ground-based human commentary. As a result, ground commanders can more accurately grasp the real-time air combat situation and make the most timely and correct command decisions.
[0020] As a preferred embodiment of the present invention, the serialization process in step S11 is as follows:
[0021] Statistics of all collected raw structured situational data (OBS) org The maximum and minimum values of each dimension are used to form the eigenvalue vector obs. max and the eigenminimum vector obs min The normalization preprocessing method is as follows:
[0022]
[0023] Among them, obs org This is the original structured situational data; obs nor The status feature vector after normalization preprocessing; obs max The eigenvalue vector; obs min This is the vector of minimum eigenvalues.
[0024] Therefore, for the normalized preprocessed situation feature vector obs nor All data will be between [0,1]. Training with normalized data can make the model converge faster.
[0025] As a preferred embodiment of the present invention, step S13 further includes a sub-step, a dataset generation step, specifically:
[0026] Determine whether the number of data pairs meets the training requirements. If it does, organize the data pairs into a dataset. If not, repeat the generation of data pairs until the number requirement is met, and then organize the data pairs into a dataset.
[0027] Therefore, since deep learning-based comment text sequence generation network models require a large number of data pairs for training, it is necessary to organize the data pairs that meet the quantity requirements into datasets and then input these datasets into the comment text sequence generation network model. Each dataset contains a large number of feature vectors and the corresponding human comment content for each feature vector. The comment text sequence generation network model is trained using these datasets, and the larger the number of data pairs in the dataset, the better the training effect and the higher the accuracy of the network model's output.
[0028] As a preferred embodiment of the present invention, the method further includes a dataset expansion step, specifically:
[0029] The original dataset is fed into the generative model of the GAN network, and some noise is randomly generated and input into the generative model to obtain several output fake data. The obtained fake data and the original dataset are combined to obtain a new dataset that has been expanded by adding fake samples to the original dataset.
[0030] Therefore, GAN (Generative Adversarial Networks) is a generative model trained using adversarial mechanisms. Its core idea is to learn the distribution of data through the interaction between a generator and a discriminator. The generator aims to generate new samples similar to real data, while the discriminator aims to distinguish real data from fake data generated by the generator. In GANs, the model is trained adversarially to make the distribution of generated data as close as possible to the distribution of real data. Specifically, the generator network receives a dataset as input and attempts to generate data similar to the real dataset. The discriminator network's task is to distinguish between the real dataset and the dataset generated by the generator network. In this way, the discriminator network also learns and improves during training, thereby driving the generator network to generate datasets that are closer to reality.
[0031] After the dataset expansion step, the accuracy of real-time human commentary generated by this commentary text sequence generation network model can be further enhanced. On the basis of the extremely fast generation speed of real-time air combat situation commentary, higher accuracy can be guaranteed, enabling ground commanders to grasp the air combat situation more accurately. In addition, the fake data samples generated by generative adversarial network (GAN) can be added to the dataset for training, which can effectively improve the model's generalization ability.
[0032] As a preferred embodiment of the present invention, the data in the dataset is input into the comment text sequence generation network model in batches, the difference between the output results and the corresponding human comment content is compared, and the parameters of the comment text sequence generation network model are updated according to the gradient backpropagation algorithm.
[0033] Therefore, the comment text sequence generation network model needs to be updated based on the output during training. Backpropagation is a commonly used training algorithm in neural networks, used to adjust the network weights based on given input and output, gradually reducing the network's output error so that the network can better approximate the desired output. When using the gradient backpropagation algorithm, it calculates the gradient of the loss function with respect to each parameter, and then updates the parameter values in the opposite direction of the gradient to minimize the loss function. After updating the parameters of the comment text sequence generation network model using the gradient backpropagation algorithm, the output will be closer to the content of the corresponding human comment, thus ensuring the reliability and accuracy of subsequent real-time human comment generation.
[0034] A deep learning-based real-time air combat situation commentary generation system, characterized in that the system comprises:
[0035] The training data collection module is used to collect structured air combat situational data and human-written commentary data;
[0036] The situation feature vector generation module is used to select key event features from the structured situation data, perform serialization processing, and generate situation feature vectors.
[0037] The preprocessing module for human commentary is used to extract the core of sentences from the human text commentary data and remove modifiers to obtain the processed human text commentary data.
[0038] The network model training module is used to construct a comment text sequence generation network model and generate data pairs of situational feature vectors and human comment content; the comment text sequence generation network model is trained using the data pairs.
[0039] The real-time situation feature vector generation module is used to acquire real-time air combat structured situation data and perform serialization processing to obtain real-time situation feature vectors.
[0040] The real-time human comment content acquisition module is used to input the real-time situation feature vector into the trained comment text sequence generation network model to obtain the real-time human comment content corresponding to the current situation.
[0041] Therefore, the training data collection module will first collect structured air combat situational awareness data and human-written commentary data. Structured air combat situational awareness data refers to a series of specific data parameters related to air combat obtained through battlefield information transmission systems or airborne and ground-based radars; human-written commentary data refers to concise but precise verbal commentary. A large amount of this data will be collected for analysis and screening. At this stage, a large amount of historical structured air combat situational awareness data will be selected for training.
[0042] Next, the situation feature vector generation module selects key event features from the structured situation data, performs serialization processing, and generates situation feature vectors. Specifically, it filters a large amount of historical air combat structured situation data, removes unnecessary situation data, selects suitable feature data, and performs serialization processing. Suitable feature data is mainly divided into three categories. For example, in one model, these are aircraft features, missile features, and key event features. These features are serialized, each feature is assigned a unique identifier and flag bit, and after normalization processing, they are all within the range [0,1], which facilitates model convergence.
[0043] The preprocessing module for human commentary will extract the core of sentences from the human text commentary data and remove modifiers to obtain the processed human text commentary data.
[0044] The network model training module is used to construct a comment text sequence generation network model. It employs a decoder-based network model, including multiple decoding layers (e.g., Masked Self-Attension layers and Feed Forward Nerual Network layers). This model maps the sequenced data's feature vectors to the human comment content, forming data pairs. In other words, it establishes a correspondence between the sequenced feature vectors and the human comment content, creating data pairs (x...). i ,y i ). Where x i Let y be the eigenvector. i This represents the human-generated comment content corresponding to this feature vector. Multiple data pairs (x...) i ,y i The dataset is used to input comment text sequences to generate a network model, which is then trained and the trained model is retained.
[0045] In practical applications, the real-time situation feature vector generation module acquires real-time structured air combat situation data and performs serialization processing to obtain a real-time situation feature vector. The real-time human commentary content acquisition module inputs the real-time situation feature vector into the trained commentary text sequence generation network model to obtain the corresponding real-time human commentary content under the current situation. Its generation speed is extremely fast, ensuring real-time performance.
[0046] As a preferred embodiment of the present invention, the situation feature vector generation module includes a serialization processing unit, which is used to statistically analyze all collected raw structured situation data (obs). org The maximum and minimum values of each dimension are used to form the eigenvalue vector obs. maxand the eigenminimum vector obs min .
[0047] Therefore, the normalization preprocessing method is as follows:
[0048]
[0049] Among them, obs org This is the original structured situational data; obs nor The status feature vector after normalization preprocessing; obs max The eigenvalue vector; obs min This is the vector of minimum eigenvalues.
[0050] The normalized preprocessed situation feature vector obs nor All data will be between [0,1]. Training with normalized data can make the model converge faster.
[0051] As a preferred embodiment of the present invention, the network model training module includes a dataset generation unit, which is used to determine whether the number of data pairs meets the training requirements. If it does, the data pairs are organized into a dataset; if not, the data pairs are repeatedly generated until the number requirement is met, and then the data pairs are organized into a dataset.
[0052] Therefore, since deep learning-based comment text sequence generation network models require a large number of data pairs for training, it is necessary to organize the data pairs that meet the quantity requirements into datasets and then input these datasets into the comment text sequence generation network model. Each dataset contains a large number of feature vectors and the corresponding human comment content for each feature vector. The comment text sequence generation network model is trained using these datasets, and the larger the number of data pairs in the dataset, the better the training effect and the higher the accuracy of the network model's output.
[0053] The dataset generation unit can continuously generate new data pairs and input these newly generated data pairs into the comment text sequence generation network model, thereby training the network model and improving the reliability of the model output.
[0054] In summary, the present invention has the following beneficial effects:
[0055] 1. By constructing and training a deep learning-based network model for generating comment text sequences, real-time human commentary can be automatically obtained upon inputting a real-time situational feature vector. This model boasts advantages such as requiring no human intervention, rapid response, and high accuracy, enabling it to meet the real-time air combat situation requirements in realistic air combat scenarios. This helps commanders enhance their understanding of the air combat situation and issue correct commands.
[0056] 2. When training the comment text sequence generation network model, a generative model based on GAN network is introduced. Random noise can be input into the generative model to obtain fake output data. The fake data obtained is then combined with the original dataset to obtain a new dataset, thereby expanding the dataset, increasing the number of training datasets, and thus enhancing the accuracy of the model. Attached Figure Description
[0057] Figure 1 This is a flowchart of the method for generating real-time air combat situational commentary;
[0058] Figure 2 This is a system block diagram of the air combat real-time situation commentary generation system;
[0059] Figure 3 A flowchart of the model training process;
[0060] Figure 4 Generate a flowchart for comments;
[0061] Figure 5 A diagram illustrating how to expand the dataset for a GAN network;
[0062] Figure 6 Network structure diagram;
[0063] Figure 7 This is a schematic diagram of data serialization processing. Detailed Implementation
[0064] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. Any person may implement the present disclosure in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
[0065] Example 1 illustrates the steps for obtaining a comment model, specifically for generating a network model to obtain a trained comment text sequence.
[0066] like Figure 1 and Figure 3 As shown, the initial step involves collecting structured air combat situational awareness data and human-written commentary data. Structured air combat situational awareness data refers to a series of specific data parameters related to air combat obtained through battlefield information transmission systems or airborne and ground-based radars; human-written commentary data refers to concise but precise verbal commentary. A large amount of this data is collected for analysis and screening. At this stage, a large amount of historical structured air combat situational awareness data is selected for training.
[0067] Next, key event features are selected from the structured situational data and processed sequentially to generate situational feature vectors. For example, such as... Figure 7 As shown, suitable features are selected from the acquired structured air combat situation data. Each feature corresponds to one dimension in the feature vector, which contains three types of features:
[0068] a) Aircraft characteristics: number of all aircraft, red and blue teams, survival status, relative distance, relative angle, altitude difference, speed, radar status, and jamming status;
[0069] b) Missile characteristics: all missile serial numbers, availability status, flight time, speed, relative distance between the missile and the aircraft, relative angle between the missile and the aircraft, flight time, speed, and open-eye status;
[0070] c) Key event characteristics: aircraft launches missiles, shoots down enemy aircraft, is shot down by enemy aircraft, launches flares, successfully evades missiles, locks onto the enemy, is locked onto by the enemy.
[0071] The serialization process converts feature attributes into decimal numbers. For each aircraft or missile, a unique identifier is assigned at the beginning of its sequence, followed by a flag indicating whether the aircraft or missile is alive. Other indicators are normalized by first statistically analyzing all collected raw feature data (obs). org The maximum and minimum values of each dimension are used to form the eigenvalue vector obs. max and the eigenminimum vector obs min The normalization preprocessing method is as follows:
[0072]
[0073] After processing, all indicators are between [0,1]. For example, the "relative distance 70000" in the missile characteristics will become 0.7 after normalization.
[0074] Next, the data of manually written comments is extracted. By extracting the main body of the sentences and removing redundant modifiers, the content of the manually written comments is generated.
[0075] Next, a deep learning-based comment text sequence generation network model is constructed. This model uses a decoder framework and includes multiple decoding layers, such as... Figure 6As shown, it includes a Masked Self-Attension layer and a Feed Forward Nerual Network layer. It maps the situational feature vectors obtained after data serialization to the content of human comments, forming data pairs. That is, it establishes a correspondence between the feature vectors obtained after data serialization and the content of human comments, forming data pairs (x...). i ,y i ). Where x i Let y be the eigenvector. i This refers to the human-generated commentary corresponding to this feature vector.
[0076] Next, it will be determined whether the number of data pairs meets the training requirements. If it does, then multiple data pairs (x) will be... i ,y i The dataset generated is input into the comment text sequence to generate a network model, and the network model is trained; if the desired result is not achieved, the original dataset is fed into the generative model of the GAN network, such as... Figure 5 As shown, some noise is randomly generated and input into the generation model to obtain several output fake data. The obtained fake data is combined with the original dataset to obtain a new dataset that has been expanded by adding fake samples to the original dataset. Then, the new dataset is input into the comment text sequence generation network model to train the network model.
[0077] Furthermore, after outputting the results, the difference between the output and the corresponding human comment is compared, and the parameters of the comment text sequence generation network model are updated according to the gradient backpropagation algorithm. Then, it is determined whether the model training is complete. If not, the parameters are updated again until training is finished, and the model is saved.
[0078] At this point, the trained comment text sequence generation network model was obtained.
[0079] Example 2 is an illustration of the situation commentary generation steps, namely, to obtain real-time human commentary content corresponding to the current situation.
[0080] like Figure 1 and Figure 4 As shown, firstly, real-time air combat structured situational data is acquired. Then, the same steps are followed to filter and sequence the data to obtain real-time situational feature vectors. At this point, the real-time situational feature vectors are input into the previously acquired and trained comment text sequence generation network model. The comment text sequence generation network model will automatically generate real-time human comment content. Then, it is determined whether the inference has ended. If it has not ended, the above process is repeated. If it has ended, the comment generation process is terminated.
[0081] In summary, by combining real-time structured air combat situational data with the aforementioned commentary text sequence generation network model, corresponding human commentary content can be generated extremely quickly. The generation speed and accuracy far exceed those of ground-based human commentary. As a result, ground commanders can more accurately grasp the real-time air combat situation and make the most timely and correct command decisions.
[0082] Several embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or technological improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A method for generating real-time air combat situational awareness based on deep learning, characterized in that, This method mainly includes: S1. Steps for obtaining the comment model; S11. Collect structured air combat situation data and human-written commentary data; select key event features from the structured situation data, perform serialization processing, and generate situation feature vectors; preprocess the human-written commentary data to generate human-written commentary content. S12. Construct a deep learning-based comment text sequence generation network model, wherein the comment text sequence generation network model is constructed using a decoding framework, including a multi-layer decoding network, and the multi-layer decoding network includes a Masked Self-Attension layer and a Feed Forward Nerual Network layer; the model input is the situation feature vector, and the output is the human comment content; S13. Establish the correspondence between the situation feature vector and the content of the manual comment to form a data pair; input the data pair into the comment text sequence to generate a network model and complete the training; S2. Steps for generating situational commentary; S21. Acquire real-time air combat structured situation data and perform serialization processing to obtain real-time situation feature vectors. S22. Input the real-time situation feature vector into the trained comment text sequence generation network model to generate real-time human comment content corresponding to the current situation.
2. The method for generating real-time air combat situation commentary according to claim 1, characterized in that, In step S11, the serialization process is as follows: Statistical analysis of all collected raw structured situational data The maximum and minimum values of each dimension are used to construct an eigenvalue vector from all the maximum and minimum values. and the eigenminimum vector The normalization preprocessing method is as follows: Equation (1) in, This is the original structured situational data; This is the situation feature vector after normalization preprocessing; The eigenvalue vector; This is the vector of minimum eigenvalues.
3. The method for generating real-time air combat situation commentary according to claim 1, characterized in that, Step S13 also includes a sub-step, the dataset generation step, which is as follows: Determine whether the number of data pairs meets the training requirements. If it does, organize the data pairs into a dataset. If not, repeat the generation of data pairs until the number requirement is met, and then organize the data pairs into a dataset.
4. The method for generating real-time air combat situation commentary according to claim 3, characterized in that, The method also includes a dataset augmentation step, specifically: The original dataset is fed into the generative model of the GAN network, and some noise is randomly generated and input into the generative model to obtain several output fake data. The obtained fake data and the original dataset are combined to obtain a new dataset that has been expanded by adding fake samples to the original dataset.
5. The method for generating real-time air combat situation commentary according to claim 4, characterized in that, The data in the dataset is input into the comment text sequence generation network model in batches. The difference between the output results and the corresponding human comment content is compared, and the parameters of the comment text sequence generation network model are updated according to the gradient backpropagation algorithm.
6. A deep learning-based real-time air combat situation commentary generation system, characterized in that, The system includes: The training data collection module is used to collect structured air combat situational data and human-written commentary data; The situation feature vector generation module is used to select key event features from the structured situation data, perform serialization processing, and generate situation feature vectors. The preprocessing module for human commentary is used to extract the core of sentences from the human text commentary data and remove modifiers to obtain the processed human text commentary data. The network model training module is used to construct a comment text sequence generation network model and generate data pairs of situational feature vectors and human comment content; the comment text sequence generation network model is trained using the data pairs; wherein, the comment text sequence generation network model is constructed using a decoding framework, which includes a multi-layer decoding network, and the multi-layer decoding network includes a Masked Self-Attension layer and a Feed Forward Nerual Network layer; The real-time situation feature vector generation module is used to acquire real-time air combat structured situation data and perform serialization processing to obtain real-time situation feature vectors. The real-time human comment content acquisition module is used to input the real-time situation feature vector into the trained comment text sequence generation network model to obtain the real-time human comment content corresponding to the current situation.
7. The air combat real-time situation commentary generation system according to claim 6, characterized in that, The situation feature vector generation module includes a serialization processing unit, which is used to statistically analyze all collected raw structured situation data. The maximum and minimum values of each dimension are used to construct an eigenvalue vector from all the maximum and minimum values. and the eigenminimum vector The normalization preprocessing method is as follows: Equation (1) in, This is the original structured situational data; This is the situation feature vector after normalization preprocessing; The eigenvalue vector; This is the vector of minimum eigenvalues.
8. The air combat real-time situation commentary generation system according to claim 7, characterized in that, The network model training module includes a dataset generation unit, which is used to determine whether the number of data pairs meets the training requirements. If it does, the data pairs are organized into a dataset; if not, the data pairs are generated repeatedly until the number requirement is met, and then the data pairs are organized into a dataset.