Method and device for extracting key graphic and text information of radar simulation countermeasure system
Through deep learning and natural language processing technology, combined with image processing methods, text and image information in radar system are extracted and fused, and the problem of information extraction and fusion of radar system in electronic confrontation scenarios is solved, achieving efficient and accurate key information Extract and real-time requirements.
Patent Information
- Application Number
- CN202411516874.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-14
AI Technical Summary
Existing radar systems are difficult to effectively extract and integrate a large amount of radar data in electronic confrontation scenarios, resulting in difficulty in extracting key information, difficulty in taking into account real-time and accuracy, and challenges in the fusion of multi-source information.
Deep learning method combined with natural language processing and image processing technology, extract text features through sequence-to-sequence mode and attention mechanism, use the Mixer layer to process image modal information, and fuse text and image features through L2 distance and cross entropy loss functions. Generate key graphic information.
The comprehensive and accurate extraction of text and picture information in the radar simulation system is realized, avoiding information omissions or misjudgment, improving information extraction efficiency, meeting real-time requirements, reducing manual intervention, and improving the intelligence of the system and flexibility.
Smart Images
Figure CN119445593A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image and text information extraction, and in particular to a method and device for extracting key image and text information of a radar simulation countermeasure system. Background Art
[0002] Radar is a radio wave detection device. Its full name comes from the abbreviation of "Radio Detection and Ranging". In electronic warfare, radar plays a vital role. The main function of radar is to detect, track and locate the target by emitting electromagnetic waves to a specific area and analyzing the electromagnetic waves reflected by the target. In the context of electronic warfare, radar is not only a key tool for information collection, but also the main battlefield for electronic interference and anti-interference.
[0003] On the one hand, radar reconnaissance is a key link in electronic warfare. By using reconnaissance equipment to obtain enemy radar signals and through research and analysis with advanced equipment, the location of the radar can be determined and the information hidden in the radar signal can be analyzed. This information is crucial for understanding the enemy's dynamics, intentions and deployment, and helps us make correct strategic decisions.
[0004] On the other hand, radar is also the main target of interference and anti-interference in electronic countermeasures. The enemy may use electronic interference technology to try to paralyze our radar system, thereby destroying our ability to perceive the battlefield situation. Therefore, we need to develop effective anti-interference technology to ensure that the radar system can work normally and provide accurate intelligence information.
[0005] Due to the particularity of the scanning method, the traditional radar display can only display the image information of the radar echo once. The digital display system can be divided into: character display system, graphic image display system and situation display system according to its display content.
[0006] In general electronic warfare scenarios, the radar system processes radar information in the following three aspects: (1) Detecting the target echo from the radar receiver output and determining the target’s presence; (2) Measure and record the target’s distance, angle, speed and other information; (3) Carry out follow-up processing on the target based on the admitted target information.
[0007] When the number of targets and the speed of movement do not change dramatically, the radar images are very similar and the differences between adjacent frames are small. At the same time, key information needs to be generated by integrating data from multiple radars. The following difficulties exist in extracting accurate key information: 1. Information extraction and integration: The radar situation summary needs to accurately and comprehensively reflect the key elements such as target information, motion trajectory, speed, direction, etc. detected by the radar system. Radar systems often generate a large amount of raw data. How to effectively extract useful information from this data and integrate it concisely and clearly is a challenging task.
[0008] 2. Real-time and accuracy: Radar situation key information needs to be generated in real time so that decision makers can understand the battlefield situation in a timely manner and make corresponding decisions. However, there is often a contradiction between real-time and accuracy. In the pursuit of real-time, some accuracy may be sacrificed; and excessive pursuit of accuracy may result in the generation speed not meeting actual needs. How to find a balance between real-time and accuracy is a key issue that needs to be solved in the generation of radar situation display and control key information.
[0009] 3. Multi-source information fusion: In practical applications, radar situation display and control key information often requires the fusion of data from multiple radar systems, sensors or other information sources. The data format, accuracy and reliability between different information sources may be different. How to effectively fuse this information to generate accurate and comprehensive situation information is also a challenging task.
[0010] Therefore, it is urgent to improve this shortcoming. The present invention studies and improves the existing technology and its shortcomings, and provides a method and device for extracting key graphic information of a radar simulation countermeasure system. Summary of the invention
[0011] The purpose of the present invention is to provide a method and device for extracting key graphic information of a radar simulation countermeasure system to solve the problems raised in the above-mentioned background technology.
[0012] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, a method for extracting key graphic information of a radar simulation countermeasure system is provided, comprising the following steps: S1, pre-processing stage: Text modality preprocessing: Using natural language processing technology, the text is converted into tokens according to the preset vocabulary, and tokens can be any of words, phrases, punctuation marks or special characters; Image modality preprocessing: Use a unified strategy to convert images of modality information into a unified format and width and height; S2. Text feature extraction: Using deep learning methods, sequence-to-sequence mode, and attention distribution mechanism to completely omit the OOV processing part in the traditional mode (unlike general NLP tasks, the content output by the console is limited, and the text is a limited set. There is no problem of huge vocabulary and OOV. In other words, there will be no words that do not exist in the vocabulary. Removing the part that usually handles OOV problems in NLP can simplify and speed up the extraction of key information); The encoder-decoder structure is used. After the input sentence is converted into tokens, it is input into the encoder one by one in sequence, and a hidden state sequence is generated internally. At each time step t, the decoder receives the output of the encoder and generates a decoding state ,attention The calculation formula is as follows: in, are all parameters of a trainable neural network. is the encoder hidden state, is the decoder status, is a vector, represents each component, tanh is the activation function, , It is a normalization method in machine learning and is defined as: in, is the output value of the i-th node, and C is the number of output nodes; Context vector The generation formula is as follows: Through the context vector The context information is encoded, and the value of the t time step is the sum of the encoder state and the attention. After that, it passes through two fully connected layers to generate a vocabulary-based output. The formula is as follows: in, Represents a vector concatenation operation, V and b are both trainable parameters, is the distribution of all words based on the vocabulary, is the context vector, is the decoder state; The output probability of word w is P(w) = (w) , and the loss function for word w at time step t is defined as follows: The loss function for the t time steps of all sequences is: S3. Modal information extraction of images: The input image is first split into S patches (a patch refers to a part, the default configuration is S=9, which can be adjusted according to needs), with no overlapping parts between each patch, and then the different channels (channels, color images are generally RGB or RGBA) of each patch form an input matrix of the number of patch x channel; processed by the Mixer layer, the Mixer layer includes MLP and data normalization layer; The processing steps of the data normalization layer are: is the input, is the output, and the mean of the input sequence is , the variance is , It is a very small quantity to prevent the denominator from being 0, and then normalized by the following formula: Then perform scaling mapping: in, is the scaling factor, which is a constant parameter to be adjusted and used to control the final output ; S4. Modal information fusion of text and pictures: By integrating the features of the image and text, the image that best matches the current text description is selected as the final key image output; Through steps S2 and S3, for a piece of text output and the related picture, the feature encoding of the text is used To represent, the feature encoding of the image is used To express, L2 distance is used to calculate the correlation of image and text features, and the distance expression of image and text features is obtained: in, Refers to the L2 distance, also called the Euclidean distance; For two vectors with p components , the distance is defined as above formula; The loss function of the image part uses cross entropy: Among them, it means The distribution probability of the picture. If there are n different pictures, appearing randomly, then the probability of each picture distribution is ; The overall loss function of the model, combined with the formula The loss function of the text part is defined as: in, is a predefined constant parameter.
[0013] Furthermore, the modalities of the radar simulation confrontation system include text and pictures. The text modal information exists in text form and is input and output through a text editor and a command line interface. It contains a variety of important information such as radar parameter settings, system status descriptions, command instructions, etc. The picture modal information is mainly reflected in the radar simulation confrontation system as radar scanning images, maps, and combat situation diagrams to intuitively display key information such as the target position, shape, and motion trajectory detected by the radar.
[0014] Furthermore, in step S2, based on the premise that the content of the scene description is relatively simple, a fixed vocabulary is used, and the encoding of each word uses a fixed value. No pre-trained model or complex structure such as BERT is used, and the situation where the same word may have different encodings under different semantics is not considered.
[0015] Furthermore, in step S2, both the encoder and the decoder adopt a single-layer bidirectional LSTM structural unit. In a specific implementation, the dimensions of the input and output are configurable parameters, which are configured according to the specific scenario.
[0016] Furthermore, the attention The value of is the probability distribution of the original text in the vocabulary, and the greater the attention, the higher the importance.
[0017] Furthermore, the data normalization layer pulls the input distribution of the neuron to the value interval after being mapped by the nonlinear function to the standard normal distribution interval with a mean of 0 and a variance of 1, so that the input value of the nonlinear transformation function falls into the area where the activation function is more sensitive to increase the gradient.
[0018] Furthermore, in step S3, the dimensions of the input and output of each Mixer layer are consistent. The Mixer layer input is S patches, and each patch outputs a C-dimensional feature vector, where C is a constant. For a two-dimensional input, the output dimension is (S×C).
[0019] In the second aspect, a device for extracting key graphic and text information of a radar simulation confrontation system is provided, which is applied to the key graphic and text information extraction method of a radar simulation confrontation system as mentioned above. The device includes: a memory, a processor, and computer program instructions stored on the memory and executable on the processor. When the processor executes the computer program instructions, the key graphic and text information extraction method of the radar simulation confrontation system as mentioned above is implemented.
[0020] In a third aspect, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method for extracting key graphic information of the radar simulation countermeasure system as described above.
[0021] The present invention provides a method and device for extracting key graphic information of a radar simulation countermeasure system, which has the following beneficial effects: The present invention combines text key information extraction and image feature extraction, and can simultaneously process text and image information in a radar simulation confrontation system, thereby extracting more comprehensive and accurate key information. Compared with an extraction method that relies solely on a single information source (such as text only or image only), the problem of information omission or misjudgment is effectively avoided; and for text key information extraction and image feature extraction, an efficient algorithm and process framework are designed, which can automatically analyze, understand and process image and text information to quickly and accurately extract, significantly improving the efficiency of information extraction and meeting the high real-time requirements of the radar simulation confrontation system, so as to quickly respond and make decisions in a rapidly changing battlefield environment; and a method for fusing text and image information is provided to further reduce manual intervention, improve the degree of automation, and enable the radar simulation confrontation system to have stronger intelligence. In addition, the present invention has certain flexibility and scalability, and can be adjusted and optimized according to actual needs, so as to adjust the parameters and strategies of text key information extraction and image feature extraction according to different radar simulation confrontation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A processing flow chart of a method for extracting key graphic information of a radar simulation countermeasure system according to the present invention; Figure 2 A logical structure diagram of text key information extraction of a method for extracting key graphic and text information of a radar simulation countermeasure system according to the present invention; Figure 3 It is a picture feature extraction flow chart of a method for extracting key graphic information of a radar simulation countermeasure system of the present invention; Figure 4 It is a structural diagram of the Mixer layer of a method for extracting key graphic information of a radar simulation countermeasure system of the present invention; Figure 5 This is a structural diagram of the MLP of a method for extracting key graphic information of a radar simulation countermeasure system according to the present invention. DETAILED DESCRIPTION
[0023] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0024] like Figure 1-Figure 5 As shown, a method for extracting key graphic information of a radar simulation countermeasure system includes the following steps: S1, pre-processing stage: Text modality preprocessing: Using natural language processing technology, the text is converted into tokens according to the preset vocabulary, and tokens can be any of words, phrases, punctuation marks or special characters; Image modality preprocessing: Use a unified strategy to convert images of modality information into a unified format and width and height; The modalities of the radar simulation confrontation system include text and pictures. The text modal information exists in the form of text and is input and output through a text editor and a command line interface. It contains a variety of important information such as radar parameter settings, system status descriptions, and command instructions. The picture modal information in the radar simulation confrontation system is mainly reflected in radar scanning images, maps, and combat situation diagrams to intuitively display key information such as the position, shape, and motion trajectory of the target detected by the radar; S2. Text feature extraction: Using deep learning methods, sequence-to-sequence mode, and attention distribution mechanism to completely omit the OOV processing part in the traditional mode (unlike general NLP tasks, the content output by the console is limited, and the text is a limited set. There is no problem of huge vocabulary and OOV. In other words, there will be no words that do not exist in the vocabulary. Removing the part that usually handles OOV problems in NLP can simplify and speed up the extraction of key information); Based on the premise that the content of the scene description is relatively simple, a fixed vocabulary is used, and the encoding of each word uses a fixed value. No pre-trained model or complex structure such as BERT is used, and the situation that the same word may have different encodings under different semantics is not considered; The encoder-decoder structure is adopted. Both the encoder and the decoder use a single-layer bidirectional LSTM structure unit. In the specific implementation, the input and output dimensions are configurable parameters. According to the specific scenario configuration, after the input sentence is converted into a token, it is input into the encoder one by one in sequence, and a hidden state sequence is generated internally. At each time step t, the decoder receives the output of the encoder and generates a decoding state ,attention The calculation formula is as follows: in, are all parameters of a trainable neural network. is the encoder hidden state, is the decoder status, is a vector, represents each component, tanh is the activation function, , It is a normalization method in machine learning and is defined as: in, is the output value of the i-th node, and C is the number of output nodes; attention The value of is the probability distribution of the original text in the vocabulary, and the greater the attention, the higher the importance; the context vector The generation formula is as follows: Through the context vector The context information is encoded, and the value of the t time step is the sum of the encoder state and the attention. After that, it passes through two fully connected layers to generate a vocabulary-based output. The formula is as follows: in, Represents a vector concatenation operation, V and b are both trainable parameters, is the distribution of all words based on the vocabulary, is the context vector, is the decoder state; The output probability of word w is P(w) = (w) , and the loss function for word w at time step t is defined as follows: The loss function for the t time steps of all sequences is: S3. Modal information extraction of images: The input image is first split into S patches (a patch refers to a part, the default configuration is S=9, which can be adjusted according to needs), and there is no overlap between each patch. Then the different channels (channels, color images are generally RGB or RGBA) of each patch constitute an input matrix of patch x channel; through the processing of the Mixer layer, the structure of the Mixer layer is as follows Figure 4 As shown, the Mixer layer includes MLP and data normalization layer. The structure of MLP is as follows Figure 5 As shown; Among them, GELU is an activation function, the full name is "Gaussian Error Linear Unit", (the website https: / / blog.csdn.net / kkxi123456 / article / details / 122694916 has a detailed description), this case is just used, no further principle and detailed description are given; Data normalization layer: pulls the input distribution of the neuron to the value interval after the nonlinear function mapping to the standard normal distribution interval with a mean of 0 and a variance of 1, so that the input value of the nonlinear transformation function falls into the area where the activation function is more sensitive, so as to increase the gradient and avoid the problem of gradient disappearance. The processing steps are as follows: is the input, is the output, and the mean of the input sequence is , the variance is , It is a very small quantity to prevent the denominator from being 0, and then normalized by the following formula: Then perform scaling mapping: in, is the scaling factor, which is a constant parameter to be adjusted and used to control the final output ; The input and output dimensions of each Mixer layer are consistent, so that the network has better scalability. The Mixer layer input is S patches, and each patch outputs a C-dimensional vector of features, where C is a constant. For a two-dimensional input, the output dimension is (S×C). S4. Modal information fusion of text and pictures: By integrating the features of the image and text, the image that best matches the current text description is selected as the final key image output; Through steps S2 and S3, for a piece of text output and the related picture, the feature encoding of the text is used To represent, the feature encoding of the image is used To express, L2 distance is used to calculate the correlation of image and text features, and the distance expression of image and text features is obtained: in, Refers to the L2 distance, also called the Euclidean distance; For two vectors with p components , the distance is defined as above formula; The loss function of the image part uses cross entropy: Among them, it means The distribution probability of the picture. If there are n different pictures, appearing randomly, then the probability of each picture distribution is ; The overall loss function of the model, combined with the formula The loss function of the text part is defined as: in, is a predefined constant parameter.
[0025] A device for extracting key graphic information of a radar simulation confrontation system is applied to the above-mentioned method for extracting key graphic information of a radar simulation confrontation system. The device includes: a memory, a processor, and computer program instructions stored in the memory and executable on the processor. When the processor executes the computer program instructions, the above-mentioned method for extracting key graphic information of a radar simulation confrontation system is implemented.
[0026] A computer-readable storage medium stores computer-executable instructions, which are used to implement the above-mentioned method for extracting key graphic information of a radar simulation countermeasure system when executed by a processor.
[0027] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.
Claims
1. A method for extracting key graphic information of a radar simulation countermeasure system, characterized in that: The following steps are involved: S1, pre-processing stage: Text modality preprocessing: Using natural language processing technology, the text is converted into tokens according to the preset vocabulary, and tokens can be any of words, phrases, punctuation marks or special characters; Image modality preprocessing: Use a unified strategy to convert images of modality information into a unified format and width and height; S2. Text feature extraction: It uses deep learning methods, sequence-to-sequence model, and attention distribution mechanism to completely omit the OOV processing part in the traditional model; The encoder-decoder structure is used. After the input sentence is converted into tokens, it is input into the encoder one by one in sequence, and a hidden state sequence is generated internally. At each time step t, the decoder receives the output of the encoder and generates a decoding state ,attention The calculation formula is as follows: in, are all parameters of a trainable neural network. is the encoder hidden state, is the decoder status, is a vector, represents each component, tanh is the activation function, , It is a normalization method in machine learning and is defined as: in, is the output value of the i-th node, and C is the number of output nodes; Context vector The generation formula is as follows: Through the context vector The context information is encoded, and the value of the t time step is the sum of the encoder state and the attention. After that, it passes through two fully connected layers to generate a vocabulary-based output. The formula is as follows: in, Represents a vector concatenation operation, V and b are both trainable parameters, is the distribution of all words based on the vocabulary, is the context vector, is the decoder state; The output probability of word w is P(w) = (w) , and the loss function for word w at time step t is defined as follows: The loss function for the t time steps of all sequences is: S3. Modal information extraction of images: The input image is first split into S patches, with no overlapping parts between each patch, and then the different channels of each patch (channels, color images are generally RGB or RGBA) form an input matrix of the number of patch x channel; processed by the Mixer layer, which includes MLP and data normalization layer; The processing steps of the data normalization layer are: is the input, is the output, and the mean of the input sequence is , the variance is , It is a very small quantity to prevent the denominator from being 0, and then normalized by the following formula: Then perform scaling mapping: in, is the scaling factor, which is a constant parameter to be adjusted and used to control the final output ; S4. Modal information fusion of text and pictures: By integrating the features of the image and text, the image that best matches the current text description is selected as the final key image output; Through steps S2 and S3, for a piece of text output and the related picture, the feature encoding of the text is used To represent, the feature encoding of the image is used To express, L2 distance is used to calculate the correlation of image and text features, and the distance expression of image and text features is obtained: in, Refers to the L2 distance, also called the Euclidean distance; For two vectors with p components , the distance is defined as above formula; The loss function of the image part uses cross entropy: Among them, it means The distribution probability of the picture. If there are n different pictures, appearing randomly, then the probability of each picture distribution is ; The overall loss function of the model, combined with the formula The loss function of the text part is defined as: in, is a predefined constant parameter.
2. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: The modalities of the radar simulation confrontation system include text and pictures. The text modal information exists in text form and is input and output through a text editor and a command line interface. It includes radar parameter settings, system status descriptions, and command instruction information. The picture modal information is reflected in the radar simulation confrontation system as radar scanning images, maps, and combat situation diagrams to intuitively display key information detected by the radar, including target position, shape, and motion trajectory.
3. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: In step S2, based on the premise that the content of the scene description is relatively simple, a fixed vocabulary is used, and the encoding of each word uses a fixed value. No pre-trained model or complex structure such as BERT is used, and the situation where the same word may have different encodings under different semantics is not considered.
4. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: In step S2, both the encoder and the decoder adopt a single-layer bidirectional LSTM structural unit. In the specific implementation, the dimensions of the input and output are configurable parameters and are configured according to the specific scenario.
5. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: The attention The value of is the probability distribution of the original text in the vocabulary, and the greater the attention, the higher the importance.
6. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: The data normalization layer pulls the input distribution of the neuron to the value interval after being mapped by the nonlinear function to the standard normal distribution interval with a mean of 0 and a variance of 1, so that the input value of the nonlinear transformation function falls into the area where the activation function is more sensitive to increase the gradient.
7. The method for extracting key graphic information of a radar simulation countermeasure system according to claim 1, characterized in that: In step S3, the dimensions of the input and output of each Mixer layer are consistent. The input of the Mixer layer is S patches, and the feature output of each patch is a C-dimensional vector, where C is a constant. For a two-dimensional input, the output dimension is (S×C).
8. A device for extracting key graphic information of a radar simulation confrontation system, applied to the method for extracting key graphic information of a radar simulation confrontation system as claimed in any one of claims 1 to 7, characterized in that: The device includes: a memory, a processor, and computer program instructions stored in the memory and executable on the processor. When the processor executes the computer program instructions, the method for extracting key graphic information of a radar simulation countermeasure system as described in any one of claims 1 to 7 above is implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method for extracting key graphic information of a radar simulation countermeasure system as described in any one of claims 1 to 7.