Joint Operational Target System Data Service System Based on Large Model

Abstract

The present invention relates to the field of computer information technology, and in particular to a large-scale model-based joint combat target system data service system. The system includes a target system data processing module, a combat target feature fusion module, a combat relationship knowledge graph construction module, and a combat system service warning module. The system can obtain joint combat target system data and perform format classification and target cleaning processing to obtain joint combat target system standard data; perform feature fusion mapping on the joint combat target system standard data to obtain a combat target system fusion feature vector representation; perform entity recognition and relationship extraction on the combat target system fusion feature vector representation, and simultaneously perform relationship combination weight graph construction, combat service query and situation warning analysis to generate combat situation warning information and provide corresponding joint combat response strategy recommendation solutions. The present invention can use large-scale model technology to innovate the data processing and combat service model of the joint combat target system.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology, and in particular to a large-model-based joint combat target system data service system. Background Art

[0002] In modern joint operations, target system data is crucial. It encompasses a wide range of information, including enemy military facilities, combat force deployment, and strategic resource distribution. It serves as a crucial basis for developing operational plans, deploying combat resources, and conducting combat operations. Furthermore, this data comes from a wide range of sources, is presented in diverse formats, and changes frequently. With the development of large-scale modeling technology, it has demonstrated powerful capabilities in areas such as natural language processing, image recognition, and data analysis. However, traditional combat data processing methods struggle to cope with the massive, complex, and heterogeneous nature of combat target system data. Data integration, analysis, and mining are inefficient, making it difficult to provide timely and accurate valuable information for operational decision-making. Furthermore, it is difficult to quickly generate reasonable combat plans and action recommendations based on target system data, thereby reducing the efficiency of operational decision-making. Summary of the Invention

[0003] Based on this, it is necessary for the present invention to provide a joint combat target system data service system based on a large model to solve at least one of the above technical problems.

[0004] To achieve the above objectives, a large-scale model-based joint combat target system data service system includes the following modules:

[0005] The target system data processing module is used to obtain the joint combat target system data corresponding to different channels, and classify the joint combat target system data corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data and target video data; and perform target cleaning processing on the joint combat target data corresponding to different target data formats to obtain the joint combat target system standard data;

[0006] The combat target feature fusion module is used to embed text features of target text data within the standard data of the joint combat target system to generate combat target text features; based on the convolutional neural network and the large model of visual Transformer, target visual features are recognized on the target image data and target video data within the standard data of the joint combat target system to generate combat target visual features; and feature fusion mapping is performed on the combat target text features and combat target visual features to obtain the fusion feature vector representation of the combat target system;

[0007] The operational relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fusion feature vector representation of the operational target system to obtain the operational target entities and the joint operational relationships between each operational target entity. Based on the joint operational relationships between each operational target entity, the relationship combination weight graph between each operational target entity is constructed to generate a joint operational target system knowledge graph.

[0008] The combat system service warning module is used to obtain joint combat query requirement statements, and based on the joint combat query requirement statements, conduct combat service queries and situation warning analysis on the joint combat target system knowledge graph to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations.

[0009] Furthermore, the target system data processing module includes the following functions:

[0010] Obtain joint combat target system data corresponding to different channels;

[0011] By classifying and dividing the joint combat target system data corresponding to different channels according to the target data format, the joint combat target data corresponding to different target data formats are obtained, including target text data, target image data and target video data;

[0012] Information entropy evaluation is performed on the joint combat target data corresponding to different target data formats, so as to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and for the target image data and target video data, the pixel information entropy corresponding to the local area is calculated by analyzing the distribution and change of the pixel values, so as to obtain the target data information entropy corresponding to different target data formats;

[0013] Based on the target data information entropy corresponding to different target data formats, the corresponding joint combat target data is noise-assistedly located and cross-format data cleaning is performed to locate the existing text semantic errors and local noise areas corresponding to images and videos based on the abnormally high parts of the corresponding information entropy. Based on the text semantic errors mentioned in the target text data, the corresponding parts are searched in the image and video data for consistency check. At the same time, the complementarity between different target data formats is used to supplement the missing information to obtain cross-format cleaned data of joint combat targets.

[0014] By combining joint combat rules, combat logic correction and verification are performed between different target data formats in the cross-format cleansing data of joint combat targets. The joint combat rules stipulate the relationship between joint combat targets and the logical constraints of the sequence of actions. According to the logical constraints, it is checked whether the target actions described in the target text data comply with the combat rules, and whether the status and position of the target in the image and video data are consistent with the text description, so as to obtain the standard data of the joint combat target system.

[0015] Furthermore, the different channels specifically include satellite reconnaissance systems, drone monitoring platforms, ground sensor networks and combat intelligence databases.

[0016] Furthermore, the combat target feature fusion module includes the following functions:

[0017] Based on the pre-trained language model, target text data in the joint combat target system standard data is embedded with text features to generate combat target text features;

[0018] Perform feature pre-analysis on target image data and target video data within the standard data of the joint combat target system. For target image data, use color space conversion based on optical principles to convert images from RGB space to HSV space and CIELAB space, analyze the hue distribution and color contrast characteristics of the images under different color spaces, and use edge detection to extract the target edge contour features corresponding to the images. Target video data is decomposed into continuous frame images to analyze the time series relationship between frames, and the optical flow information between adjacent frames is calculated to determine the target's motion trend and speed, thereby obtaining multimodal pre-analysis features of the joint combat target.

[0019] Based on the multimodal pre-analysis features of joint combat targets and combined with convolutional neural networks and visual transformers, a large model is constructed to generate a large model of convolutional neural networks and visual transformers;

[0020] Based on the convolutional neural network and visual transformer large model, the target image data and target video data in the standard data of the joint combat target system are used to identify target visual features. In the convolutional neural network branch, convolution operations at different layers are used to obtain features from small-scale details to large-scale global features. In the visual transformer branch, the size of the input block is adjusted to obtain the changes in the motion state of the joint combat target at different time scales, thereby generating visual features of the combat target, including target shape, position, and combat status features.

[0021] The text features and visual features of the combat targets are subjected to feature fusion mapping to map the text features and visual features of the combat targets into the same feature space, and the corresponding weights are determined according to the intrinsic correlation between the text features and visual features of the combat targets to perform feature fusion and obtain the fusion feature vector representation of the combat target system.

[0022] Furthermore, the text feature embedding of target text data in the joint combat target system standard data based on the pre-trained language large model includes:

[0023] Conduct an in-depth analysis of the pre-trained language model at the architectural level, using graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths within the language model, and obtain the pre-trained model architecture analysis results;

[0024] Based on the analysis results of the pre-trained large model architecture and the introduction of a pre-processing strategy based on military domain knowledge, the target text data within the standard data of the joint combat target system is pre-processed and enhanced. The semantic relationships between target words are expanded using the military semantic network to obtain the pre-processed and enhanced target text data.

[0025] Based on the pre-trained language model, the pre-processed and enhanced target text data is mapped to the model. The target text data is divided into target description, action intention, and combat environment parts according to the logical structure corresponding to the military action. Each part is mapped to the feature space area corresponding to the language model to generate the target text-model mapping rules.

[0026] The preprocessed and enhanced target text data is input into the pre-trained language model according to the target text-model mapping rules for text feature extraction, and the text feature embedding is performed using the text encoder corresponding to the CLIP model to generate combat target text features.

[0027] Furthermore, the large model of the convolutional neural network and visual transformer is specifically composed of an input layer and a convolutional neural network branch and a visual transformer branch connected by an intermediate connection layer, wherein the multimodal pre-analysis features of the joint combat target are input in parallel into the convolutional neural network branch and the visual transformer branch through the input layer for model training, so that the corresponding 5x5 convolution layer, 3x3 convolution layer and 1x1 pooling layer are designed in the convolutional neural network branch to extract the local detail features corresponding to the image and video frames, and atrous convolution is used to expand the receptive field to obtain the corresponding combat target context information, and in the visual transformer branch, the image blocks and video frame blocks are converted into sequences to capture the global dependency between different time positions through the self-attention mechanism, and a cross-branch interaction module is designed in the intermediate connection layer to allow the features between the convolutional neural network branch and the visual transformer branch to interact and fuse.

[0028] Furthermore, the operational relationship knowledge graph construction module includes the following functions:

[0029] Perform combat target entity recognition on the fusion feature vector representation of the combat target system to obtain combat target entities, including combat troops, weapons and equipment, and military facilities;

[0030] Conduct combat attribute analysis on each combat target entity to obtain the corresponding combat attributes of each combat target entity;

[0031] Based on the combat attributes corresponding to each combat target entity, the combat knowledge relationship between each combat target entity is extracted to obtain the joint combat relationship between each combat target entity, including affiliation, command relationship, collaborative combat relationship, support relationship, attack and defense relationship, and threat relationship;

[0032] Based on the joint combat relationship between each combat target entity, a relationship combination weight graph is constructed between each combat target entity to generate a knowledge graph of the joint combat target system.

[0033] Furthermore, the construction of a relational weight graph between each combat target entity based on the joint combat relationship between each combat target entity includes:

[0034] Based on the joint combat relationship between each combat target entity, the number of potential relationship connections between each combat target entity is determined to determine the total number of entities that have a joint combat relationship between any two combat target entities, and the number of potential relationship connections between each combat target entity is obtained;

[0035] Based on the potential relationship connection number between each combat target entity, the relationship combination degree between each combat target entity is calculated to obtain the combat relationship combination degree between each combat target entity;

[0036] Based on the degree of combat relationship integration between various combat target entities, a relationship integration weight graph is constructed between the combat target entities and the joint combat relationships between various combat target entities, with the combat target entities as nodes, the joint combat relationships between various combat target entities as edges, and the degree of combat relationship integration as the edge relationship integration weight between various combat target entities, so as to generate a knowledge graph of the joint combat target system.

[0037] Furthermore, the combat system service warning module includes the following functions:

[0038] Obtain joint combat query requirements raised by combatants in natural language;

[0039] Perform word segmentation and entity extraction on the joint operations query requirement statement to obtain the joint operations query statement entity;

[0040] Based on the joint combat query statement entity and combined with the text encoder corresponding to the CLIP model, the text semantic embedding of the joint combat query requirement statement is performed to generate the joint combat query text semantic embedding;

[0041] Based on the semantic embedding of the joint combat query text, a combat service matching query is performed on the corresponding joint combat target in the knowledge graph of the joint combat target system to calculate the semantic matching degree between the semantic embedding of the joint combat query text and the joint combat target. Based on the semantic matching degree, the combat service plan corresponding to the joint combat target is retrieved and determined, including combat missions, combat action trends, and combat intentions.

[0042] Obtain the corresponding combat resources, time limits and geographical environment constraints, and conduct combat situation warning analysis on the combat service plan corresponding to the joint combat target based on the combat resources, time limits and geographical environment constraints, so as to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations based on the combat situation warning information.

[0043] Beneficial effects of the present invention:

[0044] The large-scale model-based joint combat target system data service system proposed in the present invention generally consists of a target system data processing module, a combat target feature fusion module, a combat relationship knowledge graph construction module, and a combat system service warning module. Compared with the existing technology, the beneficial effect of the present application lies in that by acquiring joint combat target system data from different channels, this process is like building an information bridge, bringing together key information scattered in various corners. By classifying and dividing according to the target data format, it is possible to clearly distinguish target text data, target image data, and target video data. This classification method lays a solid foundation for subsequent data processing, allowing different types of data to be optimized in their respective adaptive processes. Target cleaning processing of data in different formats is a key link in removing data impurities and improving data quality. In actual combat scenarios, data in different formats may contain errors, duplications, or incompleteness. If these problematic data are not processed, they will seriously affect the accuracy of subsequent analysis. In this way, standard data for the joint combat target system can be obtained, ensuring data consistency, accuracy, and completeness, avoiding misjudgments and errors caused by data quality issues, and thus improving the accuracy and reliability of combat command. Secondly, by extracting and fusing features from the cleaned standard data of the joint combat target system, and embedding text features into target text data, this process converts lengthy and complex text information into a vector form that is easy for computers to understand and process. These text features can accurately capture key information in the text, such as the nature, location, and mission description of the combat target. At the same time, a large model based on convolutional neural networks and visual transformers is used to identify target visual features from target image data and target video data, greatly improving the ability to understand image and video information. Convolutional neural networks excel at extracting local features in images, while visual transformers can analyze images and videos from a global perspective. The combination of the two can comprehensively and accurately identify visual features such as the target's shape, color, and motion trajectory, contributing to a deeper understanding of the internal structure and interrelationships of the combat target system, thereby providing valuable information for combat decision-making in a timely and accurate manner.Then, by deeply exploring the fused feature vector representation of the combat target system, the combat target entities and the joint combat relationships between various combat target entities can be accurately extracted from the fused feature vectors. Entity recognition is like marking key nodes in a complex information jungle, clarifying the specific target objects involved in the operation, such as different military units, weapons and equipment, etc. Relationship extraction is like building a bridge connecting these nodes, revealing the interactions and connections between various combat target entities, such as affiliation, command relationship, collaborative combat relationship, support relationship, attack and defense relationship, and threat relationship. This knowledge graph can intuitively display the overall picture of the combat target system, enabling combat commanders to clearly see the relationship context between various combat targets, quickly understand the combat situation, and provide strong knowledge support for the formulation of scientific and reasonable combat plans, thereby improving the efficiency and accuracy of combat decision-making. Finally, after obtaining joint operational query requirements, the system conducts operational service queries and situational warning analysis based on the knowledge graph of the joint operational target system, enabling rapid and accurate responses to the information needs of operational commanders. The operational service query function acts like an intelligent operational information search engine. Based on the query requirements input by commanders, it can quickly locate and extract relevant operational target information, operational relationships, and historical operational experience data from the knowledge graph. Furthermore, by applying advanced data analysis algorithms and models, it monitors and predicts operational situations in real time, enabling timely detection of potential threats and risks and issuing early warning signals. For example, when an abnormal change in the deployment of enemy combat forces is detected, and this change is similar to a historical attack action pattern, the system can quickly issue an early warning, alerting commanders to prepare for response. This greatly improves the efficiency and response speed of operational decision support, enabling commanders to make correct decisions quickly in complex and changing operational environments, thereby providing a strong guarantee for achieving joint operational objectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0046] Figure 1 This is a module diagram of the data service system for the joint combat target system based on the large model of the present invention;

[0047] Figure 2 for Figure 1 Functional flow diagram of the data processing module of the target system;

[0048] Figure 3 for Figure 1 Schematic diagram of the functional flow of the combat target feature fusion module. DETAILED DESCRIPTION

[0049] The following is a clear and complete description of the technical system of the present invention in conjunction with the accompanying drawings. It is obvious that the embodiments described are part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts are within the scope of protection of the present invention.

[0050] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0051] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0052] To achieve this, please refer to Figures 1 to 3 The present invention provides a joint combat target system data service system based on a large model, the system comprising the following modules:

[0053] The target system data processing module is used to obtain the joint combat target system data corresponding to different channels, and classify the joint combat target system data corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data and target video data; and perform target cleaning processing on the joint combat target data corresponding to different target data formats to obtain the joint combat target system standard data;

[0054] The combat target feature fusion module is used to embed text features of target text data within the standard data of the joint combat target system to generate combat target text features; based on the convolutional neural network and the large model of visual Transformer, target visual features are recognized on the target image data and target video data within the standard data of the joint combat target system to generate combat target visual features; and feature fusion mapping is performed on the combat target text features and combat target visual features to obtain the fusion feature vector representation of the combat target system;

[0055] The operational relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fusion feature vector representation of the operational target system to obtain the operational target entities and the joint operational relationships between each operational target entity. Based on the joint operational relationships between each operational target entity, the relationship combination weight graph between each operational target entity is constructed to generate a joint operational target system knowledge graph.

[0056] The combat system service warning module is used to obtain joint combat query requirement statements, and based on the joint combat query requirement statements, conduct combat service queries and situation warning analysis on the joint combat target system knowledge graph to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations.

[0057] In the embodiment of the present invention, please refer to Figure 1 FIG. 1 is a schematic diagram of the modules of the joint combat target system data service system based on the large model of the present invention. In this example, the joint combat target system data service system based on the large model includes the following modules:

[0058] S1: Target system data processing module, used to obtain joint combat target system data corresponding to different channels, and classify the joint combat target system data corresponding to different channels according to the target data format to obtain joint combat target data corresponding to different target data formats, including target text data, target image data, and target video data; and perform target cleaning processing on the joint combat target data corresponding to different target data formats to obtain joint combat target system standard data;

[0059] In an embodiment of the present invention, joint combat target system data is obtained from different channels such as satellite reconnaissance systems, unmanned aerial vehicle monitoring platforms, ground sensor networks, and combat intelligence databases, and format classification is performed by combining file extension recognition and data content feature analysis. Files ending with .txt, .doc, etc. are determined to be target text data; files ending with .jpg, .png, etc. are target image data; files ending with .mp4, .avi, etc. are target video data. For files without clear extensions, binary structure and features are analyzed for classification. In the target cleaning processing stage, the text information entropy of each sentence or paragraph is calculated for the target text data, and semantic errors are located and corrected based on information entropy anomalies; for target image and video data, the pixel value distribution and changes are analyzed to calculate the local area pixel information entropy, and the noise area is located and removed. At the same time, cross-format data consistency check and information complementation are performed, and combat logic correction and verification are performed in combination with joint combat rules, and finally standard data of the joint combat target system is obtained.

[0060] S2: Combat target feature fusion module, used to embed text features of target text data within the joint combat target system standard data to generate combat target text features; based on the convolutional neural network and the visual Transformer large model, target visual feature recognition is performed on the target image data and target video data within the joint combat target system standard data to generate combat target visual features; the combat target text features and combat target visual features are feature fused and mapped to obtain the combat target system fusion feature vector representation;

[0061] In an embodiment of the present invention, a pre-trained language model such as BERT is used to embed text features for target text data. The text is first segmented and the word units are input into the model. The embedding layer maps the word units into vectors, and the vector representation is updated through multiple encoder layers. Finally, the combat target text features are obtained by average pooling. For target image and video data, they are input into a large model consisting of a convolutional neural network branch connected by an input layer and an intermediate connection layer, and a visual Transformer branch. The convolutional neural network branch extracts local and contextual features through a 5x5 convolution layer, a 3x3 convolution layer, a 1x1 pooling layer, and a hole convolution; the visual Transformer branch converts image and video frame blocks into sequences, captures global dependencies through a self-attention mechanism, and the cross-branch interaction module of the intermediate connection layer enables feature interaction and fusion, thereby obtaining combat target visual features. The combat target text features and visual features are mapped to the same feature space by using a linear transformation matrix, and feature fusion is performed based on weights determined according to cosine similarity, etc., to finally obtain a fused feature vector representation of the combat target system.

[0062] S3: Operational relationship knowledge graph construction module, used to perform entity recognition and relationship extraction on the fusion feature vector representation of the operational target system to obtain the operational target entities and the joint operational relationships between each operational target entity. Based on the joint operational relationships between each operational target entity, the relationship combination weight graph between each operational target entity is constructed to generate a joint operational target system knowledge graph.

[0063] In an embodiment of the present invention, a deep learning-based named entity recognition model, such as a bidirectional long short-term memory (Bi-LSTM) network combined with a conditional random field (CRF), is used to perform entity recognition on the fused feature vector representation of the combat target system. Combat target entities such as combat troops, weapons and equipment, and military facilities are obtained. A rule-based and machine learning approach is then used to extract relationships. Rules based on combat attributes are formulated to determine relationships such as affiliation and command. A graph convolutional network (GCN) model is used to determine relationships such as coordination and support. A relational weighted graph is constructed based on a graph database, with combat target entities as nodes and joint combat relationships as edges. Edge-relationship weights are assigned based on the degree of relationship integration. The knowledge graph for the joint combat target system is ultimately generated using a Neo4j database for storage and management.

[0064] S4: Combat system service warning module, which is used to obtain joint combat query requirement statements, and conduct combat service query and situation warning analysis on the joint combat target system knowledge graph based on the joint combat query requirement statements, so as to generate combat situation warning information corresponding to the joint combat target service and provide corresponding joint combat response strategy recommendations.

[0065] In an embodiment of the present invention, voice signals from combatants are collected at a sampling rate of 16,000 times per second by voice acquisition equipment at a joint operations command center. These signals are then converted into joint operations query statements using a voice recognition algorithm. Alternatively, query statements are directly obtained through keyboard input. The query statement is then segmented and entity extracted to obtain query entities. This is then combined with a text encoder using the CLIP model to perform text semantic embedding. The semantic match between the semantic embedding of the joint operations query text and the joint operations objectives in the knowledge graph of the joint operations objective system is calculated, and combat service plans are retrieved based on the match. Simultaneously, combat resource, time limit, and geographic environmental constraint information is obtained from the combat resource database, combat planning system, and geographic information system. This information is combined with the combat service plan to perform combat situation warning analysis, such as warnings for resource shortages, time pressure, and environmental risks. Predefined response strategy templates are then invoked based on the warning information and adjusted and optimized based on the specific combat situation, ultimately generating a joint operations response strategy recommendation.

[0066] Furthermore, the target system data processing module includes the following functions:

[0067] Obtain joint combat target system data corresponding to different channels;

[0068] By classifying and dividing the joint combat target system data corresponding to different channels according to the target data format, the joint combat target data corresponding to different target data formats are obtained, including target text data, target image data and target video data;

[0069] Information entropy evaluation is performed on the joint combat target data corresponding to different target data formats, so as to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and for the target image data and target video data, the pixel information entropy corresponding to the local area is calculated by analyzing the distribution and change of the pixel values, so as to obtain the target data information entropy corresponding to different target data formats;

[0070] Based on the target data information entropy corresponding to different target data formats, the corresponding joint combat target data is noise-assistedly located and cross-format data cleaning is performed to locate the existing text semantic errors and local noise areas corresponding to images and videos based on the abnormally high parts of the corresponding information entropy. Based on the text semantic errors mentioned in the target text data, the corresponding parts are searched in the image and video data for consistency check. At the same time, the complementarity between different target data formats is used to supplement the missing information to obtain cross-format cleaned data of joint combat targets.

[0071] By combining joint combat rules, combat logic correction and verification are performed between different target data formats in the cross-format cleansing data of joint combat targets. The joint combat rules stipulate the relationship between joint combat targets and the logical constraints of the sequence of actions. According to the logical constraints, it is checked whether the target actions described in the target text data comply with the combat rules, and whether the status and position of the target in the image and video data are consistent with the text description, so as to obtain the standard data of the joint combat target system.

[0072] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Schematic diagram of the functional flow of the target system data processing module. In this embodiment, the target system data processing module includes the following functions:

[0073] S11: Obtain joint combat target system data corresponding to different channels;

[0074] In an embodiment of the present invention, high-resolution satellite image data and related geographic positioning information are obtained from a satellite reconnaissance system through a dedicated data interface at a fixed transmission rate per second. For the UAV monitoring platform, with the help of a wireless communication module, images and videos taken during the flight and environmental data collected by sensors are received in real time. The ground sensor network uses wired or wireless transmission methods to summarize and transmit the monitored target motion trajectory, electromagnetic signals and other data. The combat intelligence database extracts historical combat intelligence, target information and other data through a database query interface according to preset query conditions, and preliminarily integrates the data obtained from these different channels to build a basic set of joint combat target system data, and finally obtains the joint combat target system data.

[0075] S12: Classifying and dividing the joint combat target system data corresponding to different channels according to target data formats to obtain joint combat target data corresponding to different target data formats, including target text data, target image data, and target video data;

[0076] In an embodiment of the present invention, data is format-classified by combining file extension identification with data content feature analysis. Files ending in text formats such as .txt and .doc are directly identified as target text data. Files ending in image formats such as .jpg and .png are identified as target image data. Files ending in video formats such as .mp4 and .avi are classified as target video data. For files without clear extensions, the binary structure and features of the data are analyzed. If the data contains text encoding information, it is classified as target text data; if the data exhibits the characteristics of an image pixel matrix, it is determined to be target image data; if the data has the characteristics of a video frame sequence, it is classified as target video data. Ultimately, the data of the joint combat target system from different channels is accurately divided into target text data, target image data, and target video data categories.

[0077] S13: performing information entropy evaluation on the joint combat target data corresponding to different target data formats, so as to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and calculate the pixel information entropy corresponding to the local area by analyzing the distribution and change of the pixel values ​​for the target image data and the target video data, so as to obtain the target data information entropy corresponding to the different target data formats;

[0078] In the embodiment of the present invention, the target text data is first segmented, the frequency of each word is counted, and the information entropy calculation formula is used to calculate the frequency of each word. ,in is a word The probability of occurrence is calculated, and the text information entropy of each sentence or paragraph is calculated. For the target image data, the image is divided into multiple local areas of fixed size, and the distribution of pixel values ​​in each area is counted. The pixel information entropy of the local area is also calculated according to the above information entropy formula. For the target video data, the video is decomposed into continuous frame images, and the pixel information entropy of the local area is calculated for each frame according to the method of processing the target image data. The change of the pixel information entropy of the local area between frames is analyzed, and finally the target data information entropy corresponding to different target data formats is obtained.

[0079] S14: Based on the target data information entropy corresponding to different target data formats, noise-assisted positioning is performed on the corresponding joint combat target data and cross-format data cleaning is performed to locate the existing text semantic errors and local noise areas corresponding to the image and video according to the abnormally high parts of the corresponding information entropy, and the corresponding parts are searched in the image and video data according to the text semantic errors mentioned in the target text data for consistency check. At the same time, the complementarity between different target data formats is used to supplement the missing information to obtain cross-format cleaned data of the joint combat target;

[0080] In an embodiment of the present invention, by setting a normal threshold range of information entropy, when the information entropy of a sentence or paragraph in the target text data exceeds the threshold, it indicates that there is a text semantic error, and the error is further located through a semantic analysis algorithm. For target images and video data, when the pixel information entropy of a local area is abnormally high, the area is determined to be a noise area. After the semantic error of the target text data is found, the corresponding part is searched in the image and video data according to the target features, position and other information described in the text to check whether there are contradictions or inconsistencies. If the target text data lacks certain key information, such as the appearance features of the target, relevant information can be extracted from the image and video data for supplementation; conversely, if the identification information of a target in the image or video data is missing, it can be searched and supplemented from the target text data. After such processing, cross-format cleaned data of joint combat targets is obtained.

[0081] S15: By combining the joint combat rules, the combat logic correction and verification between different target data formats in the cross-format cleaning data of the joint combat targets are carried out. The joint combat rules stipulate the relationship between joint combat targets and the logical constraints of the sequence of actions. According to the logical constraints, it is checked whether the target actions described in the target text data comply with the combat rules, and whether the status and position of the target in the image and video data are consistent with the text description, so as to obtain the standard data of the joint combat target system.

[0082] In an embodiment of the present invention, the joint combat rules are stored and managed in the form of logical expressions. For target text data, the target actions described therein are parsed and compared with the logical constraints in the joint combat rules. For example, the rules stipulate that a certain combat action must be performed after another action. If the order described in the text does not match, corrections are made. For image and video data, target recognition and tracking algorithms are used to determine the state and position of the target. This information is compared with the target state and position described in the target text data. If there is any inconsistency, corrections are made based on more reliable information sources. For example, if the image clearly shows that the position of the target is inconsistent with the text description, the image information shall prevail. After comprehensive combat logic correction and verification, standard data of the joint combat target system that conforms to the joint combat rules is finally obtained.

[0083] Furthermore, the different channels specifically include satellite reconnaissance systems, drone monitoring platforms, ground sensor networks and combat intelligence databases.

[0084] Furthermore, the combat target feature fusion module includes the following functions:

[0085] Based on the pre-trained language model, target text data in the joint combat target system standard data is embedded with text features to generate combat target text features;

[0086] Perform feature pre-analysis on target image data and target video data within the standard data of the joint combat target system. For target image data, use color space conversion based on optical principles to convert images from RGB space to HSV space and CIELAB space, analyze the hue distribution and color contrast characteristics of the images under different color spaces, and use edge detection to extract the target edge contour features corresponding to the images. Target video data is decomposed into continuous frame images to analyze the time series relationship between frames, and the optical flow information between adjacent frames is calculated to determine the target's motion trend and speed, thereby obtaining multimodal pre-analysis features of the joint combat target.

[0087] Based on the multimodal pre-analysis features of joint combat targets and combined with convolutional neural networks and visual transformers, a large model is constructed to generate a large model of convolutional neural networks and visual transformers;

[0088] Based on the convolutional neural network and visual transformer large model, the target image data and target video data in the standard data of the joint combat target system are used to identify target visual features. In the convolutional neural network branch, convolution operations at different layers are used to obtain features from small-scale details to large-scale global features. In the visual transformer branch, the size of the input block is adjusted to obtain the changes in the motion state of the joint combat target at different time scales, thereby generating visual features of the combat target, including target shape, position, and combat status features.

[0089] The text features and visual features of the combat targets are subjected to feature fusion mapping to map the text features and visual features of the combat targets into the same feature space, and the corresponding weights are determined according to the intrinsic correlation between the text features and visual features of the combat targets to perform feature fusion and obtain the fusion feature vector representation of the combat target system.

[0090] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Schematic diagram of the functional flow of the combat target feature fusion module in this embodiment. The combat target feature fusion module includes the following functions:

[0091] S21: Based on the pre-trained language model, target text data in the joint combat target system standard data is embedded with text features to generate combat target text features;

[0092] In an embodiment of the present invention, a large language model pre-trained on large-scale general corpus is selected, such as BERT based on the Transformer architecture, and the target text data in the standard data of the joint combat target system is segmented to obtain a series of word units. These word units are input into the pre-trained language model. The embedding layer of the model maps each word unit to a low-dimensional vector representation. As the word unit is passed through multiple encoder layers of the model, each word unit will continuously update its vector representation according to the context information. The vector representations of all word units are averaged and pooled to obtain a vector of fixed length. This vector is the combat target text feature, which contains the semantic information of the target text data, and finally generates the combat target text feature.

[0093] S22: Perform feature pre-analysis on target image data and target video data within the standard data of the joint combat target system. For target image data, use color space conversion based on optical principles to convert the image from RGB space to HSV space and CIELAB space, analyze the hue distribution and color contrast characteristics of the image under different color spaces, and use edge detection to extract the target edge contour features corresponding to the image. Target video data is decomposed into continuous frame images to analyze the time series relationship between frames, and calculate the optical flow information between adjacent frames to determine the target's motion trend and speed, thereby obtaining multimodal pre-analysis features of the joint combat target.

[0094] In an embodiment of the present invention, a color space conversion algorithm is used for target image data to convert the image from RGB space to HSV space and CIELAB space based on optical principles. In the HSV space, the distribution of pixels of different hues is statistically analyzed, and the saturation and brightness range of the color are calculated to analyze the hue distribution and color contrast characteristics. In the CIELAB space, relevant statistical analysis is also performed, and the Canny edge detection algorithm is used to extract the edge contour features of the target in the image. For target video data, the video is decomposed into continuous frame images according to the frame rate. By comparing the changes in pixels in adjacent frame images, the Lucas-Kanade optical flow algorithm is used to calculate the optical flow information, and the motion trend and speed of the target between frames are determined. At the same time, the time series relationship between frames is analyzed, such as the time points when the target appears and disappears, and the features of these images and videos are integrated to finally obtain the multimodal pre-analysis features of the joint combat target.

[0095] S23: Based on the multimodal pre-analysis features of joint combat targets, a large model is constructed by combining convolutional neural networks and visual transformers to generate a large model of convolutional neural networks and visual transformers;

[0096] In an embodiment of the present invention, a large model of a convolutional neural network and a visual transformer is constructed, whose input layer receives multimodal pre-parsed features of the joint combat target. In the convolutional neural network branch, a 5x5 convolution layer is designed to extract large-scale local features, a 3x3 convolution layer further refines the local features, and a 1x1 pooling layer performs feature dimensionality reduction. At the same time, a dilated convolution is used to expand the receptive field by setting different dilation rates, thereby obtaining contextual information about the combat target. In the visual transformer branch, images and video frames are divided into fixed-size blocks, which are converted into sequence inputs. The self-attention mechanism is used to calculate the correlation between elements at different positions in the sequence, capturing the global dependency between different time positions. A cross-branch interaction module is designed in the intermediate connection layer, such as using feature splicing and attention fusion to allow the features of the convolutional neural network branch and the visual transformer branch to interact, enabling the model to comprehensively utilize local and global information. The model is trained using a stochastic gradient descent algorithm, and the network parameters are continuously adjusted to ultimately generate a large model of the convolutional neural network and the visual transformer.

[0097] S24: Based on the convolutional neural network and visual transformer large model, target visual feature recognition is performed on target image data and target video data in the standard data of the joint combat target system. In the convolutional neural network branch, convolution operations at different layers are used to obtain small-scale detail features to large-scale global features. In the visual transformer branch, the target motion state changes of the joint combat target at different time scales are obtained by adjusting the size of the input block to generate combat target visual features, including target shape, position and combat state features.

[0098] In an embodiment of the present invention, target image data and target video data from the standard data of the joint combat target system are input into a large model of a convolutional neural network and a visual transformer. In the convolutional neural network branch, the low-level 5x5 convolutional layer and the 3x3 convolutional layer extract small-scale detail features of the image and video frames, such as the texture and edges of the target. As the convolutional layer deepens, the size of the feature map gradually decreases, and the extracted features also transition from local details to global features. In the visual transformer branch, by adjusting the size of the input block, such as splitting a large image block or video frame block into smaller blocks, the motion state changes of the joint combat target can be observed at different time scales. Combining the outputs of the convolutional neural network branch and the visual transformer branch, the shape and position of the target are determined through post-processing algorithms such as non-maximum suppression. At the same time, based on the target's motion state and contextual information, the target's combat state, such as attack or defense, is judged, and ultimately the visual features of the combat target are generated.

[0099] S25: Perform feature fusion mapping on the combat target text features and the combat target visual features to map the combat target text features and the combat target visual features into the same feature space, and determine the corresponding weights for feature fusion according to the intrinsic correlation between the combat target text features and the combat target visual features to obtain the combat target system fusion feature vector representation.

[0100] In an embodiment of the present invention, a linear transformation matrix is ​​used to map the text features and visual features of combat targets to the same feature space, so as to analyze the intrinsic correlation between the two feature vectors by calculating indicators such as cosine similarity and mutual information between the two feature vectors. Based on these correlation indicators, an adaptive weight allocation algorithm is used to determine the weights of the text features and visual features of the combat targets. For example, if the text features and visual features have high consistency in describing the position information of the target, then they are given correspondingly higher weights during fusion. The weighted text features and visual features are added together to obtain a new feature vector. This vector is the fused feature vector representation of the combat target system, which integrates information from both textual and visual modalities and can more comprehensively describe the joint combat target, ultimately obtaining the fused feature vector representation of the combat target system.

[0101] Furthermore, the text feature embedding of target text data in the joint combat target system standard data based on the pre-trained language large model includes:

[0102] Conduct an in-depth analysis of the pre-trained language model at the architectural level, using graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths within the language model, and obtain the pre-trained model architecture analysis results;

[0103] In an embodiment of the present invention, the pre-trained language model is regarded as a complex graph structure, in which the neurons of each layer are regarded as nodes of the graph, and the connections between layers and neurons are regarded as edges. First, the layer information is extracted from the code structure of the model, the number of layers of the model and the number of neurons in each layer are clarified, and the adjacency matrix in graph theory is used to represent the inter-layer connection relationship. The elements in the matrix represent whether there is a connection between the two layers and the strength of the connection. For the neuron interaction pattern, the information transmission rules between neurons are determined by analyzing the type and parameters of the activation function. The path search algorithm in the network analysis method, such as breadth-first search, is used to track the flow path of information within the model, and the input-output relationship of each layer, the activation status of neurons and the order of information transmission are recorded in detail. Finally, the pre-trained large model architecture analysis result is obtained, which contains detailed information on the inter-layer connection relationship, neuron interaction pattern and information flow path.

[0104] Preferably, based on the analysis results of the pre-trained large model architecture and by introducing a pre-processing strategy based on military domain knowledge, the target text data in the standard data of the joint combat target system is pre-processed and enhanced, so as to expand the semantic relationship between the target words by using the military semantic network, thereby obtaining the target text data after pre-processing and enhancement;

[0105] In an embodiment of the present invention, by analyzing the results of the pre-trained large model architecture, the model's feature preferences and processing capabilities for the input data are understood, and by introducing a pre-processing strategy based on military domain knowledge, a military semantic network is first established. The network contains a large number of military terms, concepts, and semantic relationships between them, such as "tank" and "armored forces" have a subordinate relationship, and "air strike" and "air supremacy" have an associated relationship. For the target text data in the standard data of the joint combat target system, word segmentation processing is performed to split the text into individual words. Then, the relevant semantic information of each word is searched in the military semantic network, and the semantic relationship between the target words is expanded. For example, if "infantry" is mentioned in the text, it can be associated with related information such as "infantry weapons" and "infantry tactics" through the military semantic network, and these expanded information is added to the target text data. After such processing, the target text data after pre-processing and enhancement is finally obtained, which contains richer military semantic information.

[0106] Preferably, text-to-model mapping is performed on the pre-processed and enhanced target text data based on the pre-trained language model, so as to divide the target text data into target description, action intention, and combat environment parts according to the logical structure corresponding to the military action, and each part is mapped to the feature space area corresponding to the language model to generate target text-to-model mapping rules;

[0107] In an embodiment of the present invention, by analyzing the results of the pre-trained large model architecture, the feature space distribution of the model and the functions of different areas are understood. The target text data after pre-processing and enhancement is divided according to the logical structure of the military action. A rule-based method is used to divide the text into three parts: target description, action intention and combat environment according to the keywords and grammatical structure in the text. For example, the part containing words such as "capture" and "destroy" may belong to the action intention; the part describing the combat location, weather and other information belongs to the combat environment. Then, the sensitivity of different layers and neurons in the pre-trained language large model to different types of information is analyzed, and the target description, action intention and combat environment parts are respectively mapped to the feature space areas in the model that are most sensitive to this type of information. Through multiple experiments and adjustments, the accurate mapping position of each part in the model feature space is determined, and finally a target text-model mapping rule is generated, which clarifies the correspondence between each part of the text and the model feature space area.

[0108] Preferably, the target text data after preprocessing and enhancement is input into the pre-trained language model according to the target text-model mapping rules for text feature extraction, and the text feature embedding is performed using the text encoder corresponding to the CLIP model to generate combat target text features.

[0109] In an embodiment of the present invention, the target text data after preprocessing and enhancement is input into the corresponding feature space area of ​​the pre-trained language model according to the target text-model mapping rules, and the target description, action intention and combat environment parts are respectively input. Inside the model, the data is processed layer by layer, and each neuron calculates the input according to its own weight and activation function, and finally outputs the feature representation of the text. Then, these feature representations are input into the text encoder corresponding to the CLIP model. The text encoder of the CLIP model is based on the Transformer architecture and includes multiple encoder layers. In each encoder layer, the correlation between text features is captured through the multi-head self-attention mechanism, and the semantic information of the text is further extracted. After processing by the encoder, the text feature is converted into a fixed-length vector representation. This vector representation is the combat target text feature, which contains rich semantic information of the target text data and can be used for subsequent joint combat target system data service tasks, and finally generates the combat target text feature.

[0110] Furthermore, the large model of the convolutional neural network and visual transformer is specifically composed of an input layer and a convolutional neural network branch and a visual transformer branch connected by an intermediate connection layer, wherein the multimodal pre-analysis features of the joint combat target are input in parallel into the convolutional neural network branch and the visual transformer branch through the input layer for model training, so that the corresponding 5x5 convolution layer, 3x3 convolution layer and 1x1 pooling layer are designed in the convolutional neural network branch to extract the local detail features corresponding to the image and video frames, and atrous convolution is used to expand the receptive field to obtain the corresponding combat target context information, and in the visual transformer branch, the image blocks and video frame blocks are converted into sequences to capture the global dependency between different time positions through the self-attention mechanism, and a cross-branch interaction module is designed in the intermediate connection layer to allow the features between the convolutional neural network branch and the visual transformer branch to interact and fuse.

[0111] Furthermore, the operational relationship knowledge graph construction module includes the following functions:

[0112] Perform combat target entity recognition on the fusion feature vector representation of the combat target system to obtain combat target entities, including combat troops, weapons and equipment, and military facilities;

[0113] In an embodiment of the present invention, a named entity recognition model based on deep learning is adopted to process the fusion feature vector representation of the combat target system. The model is based on a bidirectional long short-term memory network (Bi-LSTM) combined with a conditional random field (CRF) and is pre-trained on a large-scale military text dataset. First, the fusion feature vector of the combat target system is input into the Bi-LSTM layer, which can capture the contextual information in the sequence data and learn the semantic features of each position in the feature vector. Then, the output of the Bi-LSTM is passed to the CRF layer. The CRF layer predicts a label for each position based on the contextual information and predefined label constraints. The labels include categories such as combat forces, weapons and equipment, and military facilities. Through the forward propagation and prediction of the model, the combat target entities in the fusion feature vector representation of the combat target system are finally identified and classified into the categories of combat forces, weapons and equipment, and military facilities.

[0114] Preferably, performing combat attribute analysis on each combat target entity to obtain the combat attribute corresponding to each combat target entity;

[0115] In an embodiment of the present invention, a combat attribute analysis system containing multiple rules and templates is constructed for the identified combat target entities. For combat forces, their organizational information, including personnel numbers, arms composition, etc., is obtained from the military database. At the same time, their combat capabilities, such as attack capabilities and defense capabilities, are analyzed. These capabilities can be quantified through historical combat data and training results. For weapons and equipment, their technical parameters, such as range, accuracy, rate of fire, etc., as well as usage conditions, such as environmental adaptability and maintenance requirements, are analyzed. For military facilities, their geographical location, protection level, carrying capacity and other attributes are examined. The relevant information of each combat target entity is input into the combat attribute analysis system. The system analyzes and extracts according to preset rules and templates, and finally obtains the combat attributes corresponding to each combat target entity.

[0116] Preferably, the combat knowledge relationships between the various combat target entities are extracted based on the combat attributes corresponding to the various combat target entities, so as to obtain the joint combat relationships between the various combat target entities, including the affiliation relationship, command relationship, cooperative combat relationship, support relationship, attack and defense relationship, and threat relationship;

[0117] In an embodiment of the present invention, combat knowledge relationships are extracted by using a method that combines rule-based and machine learning methods. First, a series of rules based on combat attributes are formulated. For example, if the personnel and equipment of a combat unit belong to the organizational system of another combat unit, it is determined that there is a subordinate relationship between them; if a combat unit is clearly designated to command another combat unit in the combat plan, there is a command relationship. For some relationships that are difficult to directly judge through rules, such as collaborative combat relationships, support relationships, attack and defense relationships, and threat relationships, a machine learning model is used for judgment. A graph convolutional network (GCN) model is used to represent combat target entities and their combat attributes as nodes and features of a graph. By learning the feature interactions and topological structures between nodes, the relationship type between nodes is predicted. Each combat target entity and its combat attributes are input into the rule system and the machine learning model. After comprehensive judgment and reasoning, the joint combat relationship between each combat target entity is finally obtained.

[0118] Preferably, a relational weight graph is constructed between each combat target entity based on the joint combat relationship between each combat target entity to generate a knowledge graph of the joint combat target system.

[0119] In an embodiment of the present invention, a relationship combination weight graph is constructed based on a graph database, each combat target entity is regarded as a node in the graph, and the joint combat relationship between each combat target entity is regarded as an edge. For each joint combat relationship, a corresponding relationship combination weight is assigned according to the degree of combat relationship combination between its various combat target entities. For example, the weights of affiliation and command relationships are relatively high because they reflect stronger constraints and control relationships; while the weights of collaborative combat relationships and support relationships are adjusted according to specific combat tasks and cooperation frequencies. By using a graph database management system such as Neo4j, the nodes, edges and relationship combination weights are stored and managed. Through the connection of nodes and edges, all combat target entities and the joint combat relationships between them are organized into a graph structure, and finally a knowledge graph of the joint combat target system is generated.

[0120] Furthermore, the construction of a relational weight graph between each combat target entity based on the joint combat relationship between each combat target entity includes:

[0121] Based on the joint combat relationship between each combat target entity, the number of potential relationship connections between each combat target entity is determined to determine the total number of entities that have a joint combat relationship between any two combat target entities, and the number of potential relationship connections between each combat target entity is obtained;

[0122] In an embodiment of the present invention, various combat target entities and the clear joint combat relationships between them are extracted from relevant data records of joint operations. These relationships can be affiliation, command relationship, collaborative combat relationship, support relationship, attack and defense relationship, and threat relationship, etc. These combat target entities are represented as nodes in a graph, and the joint combat relationships are represented as edges between the nodes. An initial combat relationship graph is constructed. For any two combat target entities, the entire combat relationship graph is traversed to find entities that have a joint combat relationship with both the first combat target entity and the second combat target entity. A breadth-first search algorithm is used to start the search from the first combat target entity. The number of entities encountered during the search that are also connected to the second combat target entity is recorded. This operation is performed on all pairs of combat target entities, and finally the number of potential relationship connections between each combat target entity is obtained.

[0123] Preferably, the relationship binding degree between each combat target entity is calculated based on the number of potential relationship connections between each combat target entity to obtain the combat relationship binding degree between each combat target entity;

[0124] In an embodiment of the present invention, the relationship binding degree is quantitatively calculated by combining the potential relationship connection numbers between each combat target entity to calculate the corresponding combat relationship binding degree, and finally the combat relationship binding degree between each combat target entity is obtained. In addition, any relationship binding association method in this field can be used to replace the relationship binding degree calculation process.

[0125] Preferably, a relational combination weight graph is constructed between the combat target entities and the joint combat relationships between the combat target entities based on the combat relationship combination degree between the combat target entities, so that the combat target entities are used as nodes, the joint combat relationships between the combat target entities are used as edges, and the combat relationship combination degree is used as the edge relationship combination weight between the combat target entities to generate a joint combat target system knowledge graph.

[0126] In an embodiment of the present invention, by taking the combat target entity as the basis, each combat target entity is regarded as a node in the graph, and the joint combat relationship between any two combat target entities is represented as an edge connecting the two nodes, and according to the combat relationship binding degree between each combat target entity calculated previously, it is assigned to the corresponding edge as the relationship binding weight of the edge, and a graph database management system is used to store and manage this graph structure, such as the Neo4j database. In the process of constructing the graph, it is ensured that each node and edge has a unique identifier to facilitate subsequent query and analysis. By organizing and representing all combat target entities and joint combat relationships in this way, a knowledge graph of the joint combat target system is finally generated.

[0127] Furthermore, the combat system service warning module includes the following functions:

[0128] Obtain joint combat query requirements raised by combatants in natural language;

[0129] In an embodiment of the present invention, by setting up special voice acquisition equipment and text input terminals in the joint operations command center, combatants can submit joint operations query requirements through voice or keyboard input. When combatants use voice input, the voice acquisition equipment samples the voice signal at a sampling rate of 16,000 times per second and converts it into a digital audio signal. Subsequently, advanced voice recognition algorithms, such as an end-to-end voice recognition model based on deep learning, are used to accurately convert the digital audio signal into a text-based joint operations query requirement statement. If the combatants use keyboard input, the system directly obtains the input text content as the query requirement statement.

[0130] Preferably, word segmentation and entity extraction are performed on the joint combat query requirement statement to obtain the joint combat query statement entity;

[0131] In an embodiment of the present invention, a word segmentation algorithm based on a combination of rules and machine learning is used to process joint combat query demand statements. The rule part accurately segments common words and fixed collocations in the sentence based on professional dictionaries and grammatical rules in the military field. The machine learning part uses a bidirectional long short-term memory network (Bi-LSTM) combined with a conditional random field (CRF) model trained with a large amount of military text data to further optimize the word segmentation results. After word segmentation is completed, named entity recognition (NER) technology is used for entity extraction. Based on the pre-trained BERT model, it is fine-tuned on the military field dataset to identify entities such as combat units, combat locations, combat time, and combat tasks in the sentence, and finally obtain the joint combat query statement entities.

[0132] Preferably, the text semantic embedding of the joint combat query requirement statement is performed based on the joint combat query statement entity and in combination with the text encoder corresponding to the CLIP model to generate the joint combat query text semantic embedding;

[0133] In an embodiment of the present invention, a joint combat query sentence entity is taken as input and fed into the text encoder of the CLIP model. The text encoder of the CLIP model is based on the Transformer architecture and is composed of multiple encoder layers stacked together. In each encoder layer, a multi-head self-attention mechanism calculation is first performed to capture the semantic information in the sentence by calculating the correlation between words at different positions. Then, a feedforward neural network calculation is performed to perform a nonlinear transformation on the output of the attention mechanism. After processing by multiple encoder layers, a semantic representation vector of each word is obtained. Finally, these word vectors are average pooled and integrated into a fixed-length vector as the semantic embedding of the joint combat query text, thereby finally generating the semantic embedding of the joint combat query text.

[0134] Preferably, a combat service matching query is performed on the corresponding joint combat target in the knowledge graph of the joint combat target system based on the semantic embedding of the joint combat query text, so as to calculate the semantic matching degree between the semantic embedding of the joint combat query text and the joint combat target, and the combat service plan corresponding to the joint combat target is retrieved and determined based on the semantic matching degree, including combat missions, combat action trends and combat intentions;

[0135] In an embodiment of the present invention, the knowledge graph of the joint combat target system stores a large number of joint combat targets and related information. For each joint combat target, the same method as that for generating the semantic embedding of the query text is used to convert its description information into a semantic embedding vector. The cosine similarity between the semantic embedding vector of the joint combat query text and the semantic embedding vector of each joint combat target in the knowledge graph is calculated to measure their semantic matching degree. A similarity threshold is set, such as 0.7, and joint combat targets with a similarity greater than the threshold are screened out. For the screened joint combat targets, their corresponding combat tasks, combat action trends, combat intentions and other information are retrieved from the knowledge graph to form a combat service plan, and finally the combat service plan corresponding to the joint combat target is determined.

[0136] Preferably, the corresponding combat resources, time limits and geographical environment constraints are obtained, and based on the combat resources, time limits and geographical environment constraints, a combat situation warning analysis is performed on the combat service plan corresponding to the joint combat target to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendation plans based on the combat situation warning information.

[0137] In an embodiment of the present invention, currently available combat resource information, including the number of weapons and equipment, the number of personnel, etc., is obtained from a combat resource database; time limit information, such as the start time and end time of combat, is obtained from a combat planning system; and geographical environment constraint information, such as topography, meteorological conditions, etc., is obtained from a geographic information system. This information is comprehensively analyzed with the combat service plan. For example, it is determined whether the combat resources can meet the needs of the combat mission based on the assessment. If the resources are insufficient, a resource shortage warning is issued; it is determined whether the combat operation can be completed on time based on the time limit. If the time is tight, a time pressure warning is issued; the impact of geographical environment constraints on combat operations is analyzed based on the geographical environment constraints. For example, if severe weather may affect air combat operations, a corresponding environmental risk warning is issued. Based on these warning information, a predefined response strategy template is called, and adjustments and optimizations are made in combination with the specific combat situation to ultimately generate a joint combat response strategy recommendation plan.

[0138] The present invention is therefore intended to be illustrative and non-restrictive in all respects, with the scope of the invention being defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the application documents are intended to be embraced therein.

[0139] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A joint combat target system data service system based on a large model, characterized by: Includes the following modules: The target system data processing module is used to obtain the joint combat target system data corresponding to different channels and classify the joint combat target system data corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data and target video data; Perform target cleaning on joint combat target data corresponding to different target data formats to obtain standard data of the joint combat target system; The combat target feature fusion module is used to embed text features of target text data within the standard data of the joint combat target system to generate combat target text features; based on the convolutional neural network and the large model of visual Transformer, the target visual features of the target image data and target video data within the standard data of the joint combat target system are recognized to generate combat target visual features; Perform feature fusion mapping on the combat target text features and the combat target visual features to obtain the fusion feature vector representation of the combat target system; The operational relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fusion feature vector representation of the operational target system to obtain the operational target entities and the joint operational relationships between the various operational target entities; based on the joint operational relationships between the various operational target entities, a relationship combination weight graph is constructed between the various operational target entities to generate a joint operational target system knowledge graph; wherein, the relationship combination weight graph construction includes: Based on the joint combat relationship between each combat target entity, the number of potential relationship connections between each combat target entity is determined to determine the total number of entities that have a joint combat relationship between any two combat target entities, and the number of potential relationship connections between each combat target entity is obtained; Based on the potential relationship connection number between each combat target entity, the relationship combination degree between each combat target entity is calculated to obtain the combat relationship combination degree between each combat target entity; Based on the operational relationship combination degree between each operational target entity, a relational combination weight graph is constructed between the operational target entity and the joint operational relationship between each operational target entity, with the operational target entity as the node, the joint operational relationship between each operational target entity as the edge, and the operational relationship combination degree as the edge relationship combination weight between each operational target entity, so as to generate a joint operational target system knowledge graph; The combat system service warning module is used to obtain joint combat query requirement statements, and based on the joint combat query requirement statements, conduct combat service queries and situation warning analysis on the joint combat target system knowledge graph to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations.

2. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The target system data processing module includes the following functions: Obtain joint combat target system data corresponding to different channels; By classifying and dividing the joint combat target system data corresponding to different channels according to the target data format, the joint combat target data corresponding to different target data formats are obtained, including target text data, target image data and target video data; Information entropy evaluation is performed on the joint combat target data corresponding to different target data formats, so as to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and for the target image data and target video data, the pixel information entropy corresponding to the local area is calculated by analyzing the distribution and change of the pixel values, so as to obtain the target data information entropy corresponding to different target data formats; Based on the target data information entropy corresponding to different target data formats, the corresponding joint combat target data is noise-assistedly located and cross-format data cleaning is performed to locate the existing text semantic errors and local noise areas corresponding to images and videos based on the abnormally high parts of the corresponding information entropy. Based on the text semantic errors mentioned in the target text data, the corresponding parts are searched in the image and video data for consistency check. At the same time, the complementarity between different target data formats is used to supplement the missing information to obtain cross-format cleaned data of joint combat targets. By combining joint combat rules, combat logic correction and verification are performed between different target data formats in the cross-format cleansing data of joint combat targets. The joint combat rules stipulate the relationship between joint combat targets and the logical constraints of the sequence of actions. According to the logical constraints, it is checked whether the target actions described in the target text data comply with the combat rules, and whether the status and position of the target in the image and video data are consistent with the text description, so as to obtain the standard data of the joint combat target system.

3. The large model-based joint combat target system data service system according to claim 2 is characterized in that: The different channels specifically include satellite reconnaissance systems, drone monitoring platforms, ground sensor networks and combat intelligence databases.

4. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The combat target feature fusion module Features include: Based on the pre-trained language model, target text data in the joint combat target system standard data is embedded with text features to generate combat target text features; Perform feature pre-analysis on target image data and target video data within the standard data of the joint combat target system. For target image data, use color space conversion based on optical principles to convert images from RGB space to HSV space and CIELAB space, analyze the hue distribution and color contrast characteristics of the images under different color spaces, and use edge detection to extract the target edge contour features corresponding to the images. Target video data is decomposed into continuous frame images to analyze the time series relationship between frames, and the optical flow information between adjacent frames is calculated to determine the target's motion trend and speed, thereby obtaining multimodal pre-analysis features of the joint combat target. Based on the multimodal pre-analysis features of joint combat targets and combined with convolutional neural networks and visual transformers, a large model is constructed to generate a large model of convolutional neural networks and visual transformers; Based on the convolutional neural network and visual transformer large model, the target image data and target video data in the standard data of the joint combat target system are used to identify target visual features. In the convolutional neural network branch, convolution operations at different layers are used to obtain features from small-scale details to large-scale global features. In the visual transformer branch, the size of the input block is adjusted to obtain the changes in the motion state of the joint combat target at different time scales, thereby generating visual features of the combat target, including target shape, position, and combat status features. The text features and visual features of the combat targets are subjected to feature fusion mapping to map the text features and visual features of the combat targets into the same feature space, and the corresponding weights are determined according to the intrinsic correlation between the text features and visual features of the combat targets to perform feature fusion and obtain the fusion feature vector representation of the combat target system.

5. The large model-based joint combat target system data service system according to claim 4 is characterized in that: The text feature embedding of target text data in the joint combat target system standard data based on the pre-trained language large model includes: Conduct an in-depth analysis of the pre-trained language model at the architectural level, using graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths within the language model, and obtain the pre-trained model architecture analysis results; Based on the analysis results of the pre-trained large model architecture and the introduction of a pre-processing strategy based on military domain knowledge, the target text data within the standard data of the joint combat target system is pre-processed and enhanced. The semantic relationships between target words are expanded using the military semantic network to obtain the pre-processed and enhanced target text data. Based on the pre-trained language model, the pre-processed and enhanced target text data is mapped to the model. The target text data is divided into target description, action intention, and combat environment parts according to the logical structure corresponding to the military action. Each part is mapped to the feature space area corresponding to the language model to generate the target text-model mapping rules. The preprocessed and enhanced target text data is input into the pre-trained language model according to the target text-model mapping rules for text feature extraction, and the text feature embedding is performed using the text encoder corresponding to the CLIP model to generate combat target text features.

6. The large model-based joint combat target system data service system according to claim 4 is characterized in that: The large model of the convolutional neural network and visual transformer is specifically composed of an input layer and a convolutional neural network branch and a visual transformer branch connected by an intermediate connection layer. The multimodal pre-analysis features of the joint combat target are input in parallel into the convolutional neural network branch and the visual transformer branch through the input layer for model training, so that corresponding 5x5 convolution layers, 3x3 convolution layers and 1x1 pooling layers are designed in the convolutional neural network branch to extract local detail features corresponding to images and video frames. At the same time, void convolution is used to expand the receptive field to obtain the corresponding combat target context information. In the visual transformer branch, image blocks and video frame blocks are converted into sequences to capture the global dependency between different time positions through the self-attention mechanism, and a cross-branch interaction module is designed in the intermediate connection layer to allow the features between the convolutional neural network branch and the visual transformer branch to interact and fuse.

7. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The operational relationship knowledge graph construction module includes the following functions: Perform combat target entity recognition on the fusion feature vector representation of the combat target system to obtain combat target entities, including combat troops, weapons and equipment, and military facilities; Conduct combat attribute analysis on each combat target entity to obtain the corresponding combat attributes of each combat target entity; Based on the combat attributes corresponding to each combat target entity, the combat knowledge relationship between each combat target entity is extracted to obtain the joint combat relationship between each combat target entity, including affiliation, command relationship, collaborative combat relationship, support relationship, attack and defense relationship, and threat relationship; Based on the joint combat relationship between each combat target entity, a relationship combination weight graph is constructed between each combat target entity to generate a knowledge graph of the joint combat target system.

8. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The combat system service warning module includes the following functions: Obtain joint combat query requirements raised by combatants in natural language; Perform word segmentation and entity extraction on the joint operations query requirement statement to obtain the joint operations query statement entity; Based on the joint combat query statement entity and combined with the text encoder corresponding to the CLIP model, the text semantic embedding of the joint combat query requirement statement is performed to generate the joint combat query text semantic embedding; Based on the semantic embedding of the joint combat query text, a combat service matching query is performed on the corresponding joint combat target in the knowledge graph of the joint combat target system to calculate the semantic matching degree between the semantic embedding of the joint combat query text and the joint combat target. Based on the semantic matching degree, the combat service plan corresponding to the joint combat target is retrieved and determined, including combat missions, combat action trends, and combat intentions. Obtain the corresponding combat resources, time limits and geographical environment constraints, and conduct combat situation warning analysis on the combat service plan corresponding to the joint combat target based on the combat resources, time limits and geographical environment constraints, so as to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations based on the combat situation warning information.

Images