Joint combat target system data service system based on large model

Abstract

The invention relates to the technical field of computer information, in particular to a joint combat target system data service system based on a large model. The system comprises a target system data processing module, a combat target feature fusion module, a combat relation knowledge graph construction module and a combat system service early warning module, and can obtain joint combat target system data and perform format classification division and target cleaning processing to obtain joint combat target system standard data; performing feature fusion mapping on the standard data of the joint combat target system to obtain combat target system fusion feature vector representation; and performing entity identification and relation extraction on the combat target system fusion feature vector representation, performing relation combination weight graph construction and combat service query and situation early warning analysis, generating combat situation early warning information, and providing a corresponding joint combat coping strategy suggestion scheme. According to the invention, the data processing and combat service mode of the joint combat target system can be innovated by using a large model technology.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology, and in particular to a data service system for a joint operation target system based on a large model. Background Art

[0002] In a modern joint operation environment, target system data is extremely crucial. It covers various aspects of information such as enemy military facilities, combat force deployments, and strategic resource distributions, and is an important basis for formulating combat plans, allocating combat resources, and implementing combat operations. Moreover, the data sources are extensive, the formats are diverse, and the data changes frequently. With the development of large model technology, it has demonstrated powerful capabilities in the fields of natural language processing, image recognition, data analysis, etc. However, traditional combat data processing methods are difficult to handle the massive, complex, and heterogeneous combat target system data, resulting in low efficiency in data integration, analysis, and mining. It is unable to provide valuable information for combat decision-making in a timely and accurate manner, and is difficult to quickly generate reasonable combat plans and action suggestions based on the target system data, thus reducing the combat decision-making efficiency. Summary of the Invention

[0003] Based on this, it is necessary for the present invention to provide a data service system for a joint operation target system based on a large model to solve at least one of the above technical problems.

[0004] To achieve the above object, a data service system for a joint operation target system based on a large model includes the following modules:

[0005] A target system data processing module, configured to obtain joint operation target system data corresponding to different channels, and classify and divide the joint operation target system data corresponding to different channels according to the target data format to obtain joint operation target data corresponding to different target data formats, including target text data, target image data, and target video data; perform target cleaning processing on the joint operation target data corresponding to different target data formats to obtain standard data for the joint operation target system;

[0006] A combat target feature fusion module, configured to perform text feature embedding on the target text data in the standard data of the joint operation target system to generate combat target text features; perform target visual feature recognition on the target image data and target video data in the standard data of the joint operation target system based on a large model of a convolutional neural network and a Vision Transformer to generate combat target visual features; perform feature fusion mapping on the combat target text features and the combat target visual features to obtain a feature vector representation of the integrated features of the combat target system;

[0007] The combat relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the integrated feature vector representation of the combat target system to obtain combat target entities and the joint combat relationships between each combat target entity; construct a relationship combination weight graph between each combat target entity based on the joint combat relationships between each combat target entity to generate a knowledge graph of the joint combat target system;

[0008] The combat system service warning module is used to obtain the joint combat query requirement statement, and perform combat service query and situation warning analysis on the knowledge graph of the joint combat target system based on the joint combat query requirement statement to generate the combat situation warning information corresponding to the joint combat target service, and provide the corresponding joint combat response strategy suggestion plan.

[0009] Furthermore, the target system data processing module includes the following functions:

[0010] Obtain the joint combat target system data corresponding to different channels;

[0011] Classify and divide the joint combat target system data corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data, and target video data;

[0012] Perform information entropy evaluation on the joint combat target data corresponding to different target data formats to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and calculate the pixel information entropy corresponding to the local area for the target image data and target video data by analyzing the distribution and change of the pixel values to obtain the target data information entropy corresponding to different target data formats;

[0013] Perform noise-assisted positioning on the corresponding joint combat target data based on the target data information entropy corresponding to different target data formats and perform cross-format data cleaning, to locate the existing text semantic errors and the local noise areas corresponding to the images and videos according to the abnormally high parts of the corresponding information entropy, and find the corresponding parts in the image and video data according to the text semantic errors mentioned in the target text data for consistency checking, and at the same time use the complementarity between different target data formats to supplement the missing information to obtain the cross-format cleaning data of the joint combat target;

[0014] By combining the joint operation rules, combat logic correction and verification are carried out between different target data formats within the cross-format cleaning data of joint operation targets. The joint operation rules stipulate the relationships between joint operation targets and the logical constraints of the sequence of actions, so as to check whether the target actions described in the target text data conform to the operation rules according to the logical constraints, and determine whether the states and positions of the targets in the image and video data are consistent with the text descriptions, so as to obtain the standard data of the joint operation target system.

[0015] Further, the different channels specifically include a satellite reconnaissance system, a drone monitoring platform, a ground sensor network, and a combat intelligence database.

[0016] Further, the combat target feature fusion module includes the following functions:

[0017] Perform text feature embedding on the target text data within the standard data of the joint operation target system based on a pre-trained large language model to generate combat target text features;

[0018] Perform feature pre-analysis on the target image data and target video data within the standard data of the joint operation target system. For the target image data, use color space conversion based on optical principles to convert the image from the RGB space to the HSV space and the CIELAB space, analyze the hue distribution and color contrast features corresponding to the image in different color spaces, and use edge detection to extract the target edge contour features corresponding to the image. For the target video data, decompose it into continuous frame images to analyze the time series relationship between frames, and calculate the optical flow information between adjacent frames to determine the movement trend and speed of the target, and obtain the multi-modal pre-analysis features of the joint operation target;

[0019] Based on the multi-modal pre-analysis features of the joint operation target and combined with a convolutional neural network and a Vision Transformer, a large model fusion is constructed to generate a large model of the convolutional neural network and the Vision Transformer;

[0020] Based on the large model of the convolutional neural network and the Vision Transformer, perform target visual feature recognition on the target image data and target video data within the standard data of the joint operation target system. In the convolutional neural network branch, obtain small-scale detailed features to large-scale global features through convolutional operations of different layers, and in the Vision Transformer branch, obtain the changes in the target motion state of the joint operation target at different time scales by adjusting the size of the input block, so as to generate combat target visual features, including target shape, position, and combat state features;

[0021] Perform feature fusion mapping on the combat target text features and combat target visual features to map the combat target text features and combat target visual features into the same feature space, and determine the corresponding weights for feature fusion according to the internal correlation between the combat target text features and combat target visual features to obtain the fused feature vector representation of the combat target system.

[0022] Furthermore, the text feature embedding of the target text data in the joint combat target system standard data based on the pre-trained large language model includes:

[0023] Conduct in-depth analysis of the architecture of the pre-trained large language model to use graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths inside the large language model, and obtain the architecture analysis results of the pre-trained large model;

[0024] Based on the architecture analysis results of the pre-trained large model and introduce a preprocessing strategy based on military domain knowledge to enhance the text preprocessing of the target text data in the joint combat target system standard data, so as to use the military semantic network to expand the semantic relationships between target vocabulary and obtain the target text data after preprocessing enhancement;

[0025] Based on the pre-trained large language model, perform text-model mapping on the target text data after preprocessing enhancement to divide the target text data into target description, action intention, and combat environment parts according to the logical structure corresponding to military operations, and map each part to the corresponding feature space area of the large language model to generate target text-model mapping rules;

[0026] Input the target text data after preprocessing enhancement into the pre-trained large language model according to the target text-model mapping rules for text feature extraction, and use the text encoder corresponding to the CLIP model for text feature embedding to generate combat target text features.

[0027] Furthermore, the large models of the convolutional neural network and the Vision Transformer are specifically composed of an input layer, a convolutional neural network branch, and a Vision Transformer branch connected by an intermediate connection layer. Among them, the multi-modal pre-parsed features of the joint operation target are respectively and parallelly input into the convolutional neural network branch and the Vision Transformer branch through the input layer for model training. In the convolutional neural network branch, corresponding 5x5 convolutional layers, 3x3 convolutional layers, and 1x1 pooling layers are designed to extract the local detail features corresponding to the images and video frames. At the same time, dilated convolution is used to expand the receptive field to obtain the corresponding operation target context information. In the Vision Transformer branch, the image patches and video frame patches are transformed into sequences to capture the global dependencies between different time positions through the self-attention mechanism. And a cross-branch interaction module is designed in the intermediate connection layer to enable the features between the convolutional neural network branch and the Vision Transformer branch to interact and fuse.

[0028] Furthermore, the combat relationship knowledge graph construction module includes the following functions:

[0029] Perform combat target entity recognition on the combat target system fusion feature vector representation to obtain combat target entities, including combat troops, weapons and equipment, and military facilities;

[0030] Perform combat attribute analysis on each combat target entity to obtain the combat attributes corresponding to each combat target entity;

[0031] Extract combat knowledge relationships between each combat target entity based on the combat attributes corresponding to each combat target entity to obtain the joint combat relationships between each combat target entity, including subordination relationships, command relationships, cooperative combat relationships, support relationships, offensive and defensive relationships, and threat relationships;

[0032] Construct a relationship combination weight graph between each combat target entity based on the joint combat relationships between each combat target entity to generate a joint combat target system knowledge graph.

[0033] Furthermore, the construction of the relationship combination weight graph between each combat target entity based on the joint combat relationships between each combat target entity includes:

[0034] Determine the number of potential relationship connections between each combat target entity based on the joint combat relationships between each combat target entity to determine the total number of entities that still have a joint combat relationship with any two combat target entities, and obtain the number of potential relationship connections between each combat target entity;

[0035] Calculate the relationship combination degree between each combat target entity by using the combat relationship combination metric formula based on the number of potential relationship connections between each combat target entity, so as to obtain the combat relationship combination degree between each combat target entity;

[0036] Construct a relationship combination weight graph between the combat target entity and the joint combat relationships between each combat target entity based on the combat relationship combination degree between each combat target entity, so as to use the combat target entity as a node, use the joint combat relationships between each combat target entity as an edge, and use the combat relationship combination degree as the edge relationship combination weight value between each combat target entity, so as to generate a joint combat target system knowledge graph.

[0037] Further, the combat relationship combination metric formula is specifically:

[0038]

[0039] In the formula, R ij is the combat relationship combination degree between the i-th combat target entity T i and the j-th combat target entity T j , N is the total number of combat target entities, Sim(T i , T j ) is the entity similarity between the i-th combat target entity T i and the j-th combat target entity T j , α ij is the entity similarity weight between the i-th combat target entity T i and the j-th combat target entity T j , exp is the exponential function, k ij is the number of potential relationship connections between the i-th combat target entity T i and the j-th combat target entity T j , d ij is the combat distance between the i-th combat target entity T i and the j-th combat target entity T j .

[0040] Further, the combat system service warning module includes the following functions:

[0041] Obtain the joint combat query requirement statement proposed by combat personnel through natural language;

[0042] Perform word segmentation and entity extraction on the joint combat query requirement statement to obtain the joint combat query statement entity;

[0043] Perform text semantic embedding on the joint operation query requirement statement based on the joint operation query statement entity and in combination with the text encoder corresponding to the CLIP model to generate the text semantic embedding of the joint operation query;

[0044] Perform an operation service matching query on the corresponding joint operation objectives within the joint operation objective system knowledge graph based on the text semantic embedding of the joint operation query, calculate the semantic matching degree between the text semantic embedding of the joint operation query and the joint operation objectives, and retrieve and determine the operation service plan corresponding to the joint operation objectives according to the semantic matching degree, including operation tasks, operation action trends, and operation intentions;

[0045] Obtain the corresponding operation resources, time limits, and geographical environment constraints, and perform an operation situation early warning analysis on the operation service plan corresponding to the joint operation objectives based on the operation resources, time limits, and geographical environment constraints to generate the operation situation early warning information corresponding to the joint operation objective service, and provide the corresponding joint operation response strategy suggestion plan according to the operation situation early warning information.

[0046] The beneficial effects of the present invention:

[0047] The data service system for the joint operation target system based on large models proposed by the present invention is generally composed of a target system data processing module, a combat target feature fusion module, a combat relationship knowledge graph construction module, and a combat system service warning module. Compared with the prior art, the beneficial effect of this application lies in obtaining the data of the joint operation target system from different channels. This process is like building an information bridge to gather the key information scattered in every corner. By classifying according to the target data format, it is possible to clearly distinguish target text data, target image data, and target video data. This classification method lays a solid foundation for subsequent data processing work, enabling different types of data to be optimized in their respective adapted processes. And by performing target cleaning on different formats of data, it is a key link to remove data impurities and improve data quality. In actual combat scenarios, there are problems such as errors, duplicates, or incompleteness in different formats of data. If these problematic data are not processed, it will seriously affect the accuracy of subsequent analysis. In this way, the standard data of the joint operation target system can be obtained, ensuring the consistency, accuracy, and integrity of the data, avoiding misjudgments and mistakes caused by data quality problems, and thus improving the accuracy and reliability of combat command. Secondly, through feature extraction and fusion of the cleaned standard data of the joint operation target system, for the target text data, text feature embedding is performed to generate combat target text features. This process can transform long and complex text information into a vector form that is easy for computers to understand and process. These text features can accurately capture the key information in the text, such as the nature, location, task description, etc. of the combat target. At the same time, using large models based on convolutional neural networks and vision transformers to perform target visual feature recognition on target image data and target video data greatly improves the ability to understand image and video information. Convolutional neural networks are good at extracting local features in images, while vision transformers can analyze images and videos from a global perspective. The combination of the two can comprehensively and accurately identify visual features such as the shape, color, and movement trajectory of the target, helping to more deeply understand the internal structure and interrelationships of the combat target system, and thus being able to provide valuable information for combat decision-making in a timely and accurate manner.Then, by deeply mining the fused feature vector representation of the combat target system, the combat target entities and the joint combat relationships between the various combat target entities can be accurately extracted from the fused feature vectors. Entity recognition is like marking key nodes in a complex information jungle, clarifying the specific target objects involved in the operation, such as different military units, weapons and equipment, etc. Relationship extraction is like building a bridge connecting these nodes, revealing the interactions and connections between various combat target entities, such as affiliation, command relationship, collaborative combat relationship, support relationship, attack and defense relationship, and threat relationship. The knowledge graph can intuitively display the overall picture of the combat target system, allowing combat commanders to clearly see the relationship between various combat targets and quickly understand the combat situation, providing strong knowledge support for the formulation of scientific and reasonable combat plans, thereby improving the efficiency and accuracy of combat decision-making. Finally, after obtaining the joint combat query demand statement, the combat service query and situation warning analysis are carried out based on the knowledge graph of the joint combat target system, which can quickly and accurately respond to the information needs of combat commanders. The combat service query function is like an intelligent combat information search engine. According to the query needs input by the commander, it can quickly locate and extract relevant combat target information, combat relations, and historical combat experience data in the knowledge graph, and through the use of advanced data analysis algorithms and models, the combat situation is monitored and predicted in real time, and potential threats and risks can be discovered in time, and warning signals can be issued in advance. For example, when it is found that the deployment of enemy combat forces has changed abnormally, and this change is similar to the historical attack action pattern, the system can quickly issue an early warning to remind the commander to prepare for the response, which greatly improves the efficiency and response speed of combat decision support, enabling commanders to quickly make correct decisions in a complex and changing combat environment, thereby providing a strong guarantee for achieving joint combat goals. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0049] Figure 1 It is a module schematic diagram of the data service system of the joint combat target system based on the large model of the present invention;

[0050] Figure 2 for Figure 1 Functional flow diagram of data processing module in the target system;

[0051] Figure 3 for Figure 1 Schematic diagram of the functional flow of the combat target feature fusion module. DETAILED DESCRIPTION

[0052] The technical system of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0053] In addition, the accompanying drawings are only schematic diagrams of the present invention and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0054] It should be understood that although terms such as "first" and "second" may be used here to describe each unit, these units should not be limited by these terms. These terms are only used to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, the first unit may be called the second unit, and similarly the second unit may be called the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed associated items.

[0055] To achieve the above object, please refer to Figures 1 to 3 , the present invention provides a joint operation target system data service system based on a large model. The system includes the following modules:

[0056] The target system data processing module is used to obtain the joint operation target system data corresponding to different channels, and classify and divide the joint operation target system data corresponding to different channels according to the target data format to obtain the joint operation target data corresponding to different target data formats, including target text data, target image data, and target video data; perform target cleaning processing on the joint operation target data corresponding to different target data formats to obtain the standard data of the joint operation target system;

[0057] The combat target feature fusion module is used to perform text feature embedding on the target text data in the standard data of the joint combat target system to generate combat target text features; perform target visual feature recognition on the target image data and target video data in the standard data of the joint combat target system based on a large model of convolutional neural network and vision Transformer to generate combat target visual features; perform feature fusion mapping on the combat target text features and combat target visual features to obtain a fused feature vector representation of the combat target system;

[0058] The combat relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fused feature vector representation of the combat target system to obtain combat target entities and the joint combat relationships between each combat target entity; construct a relationship combination weight graph between each combat target entity based on the joint combat relationships between each combat target entity to generate a knowledge graph of the joint combat target system;

[0059] The combat system service warning module is used to obtain a joint combat query requirement statement, and perform combat service query and situation warning analysis on the knowledge graph of the joint combat target system based on the joint combat query requirement statement to generate combat situation warning information corresponding to the joint combat target service, and provide a corresponding joint combat response strategy suggestion plan.

[0060] In the embodiments of the present invention, please refer to Figure 1 As shown, it is a schematic diagram of the modules of the joint combat target system data service system based on a large model of the present invention. In this example, the joint combat target system data service system based on a large model includes the following modules:

[0061] S1: The target system data processing module is used to obtain the joint combat target system data corresponding to different channels, and classify and divide the joint combat target system data corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data, and target video data; perform target cleaning processing on the joint combat target data corresponding to different target data formats to obtain the standard data of the joint combat target system;

[0062] In the embodiments of the present invention, data of the joint operation target system is obtained from different channels such as satellite reconnaissance systems, unmanned aerial vehicle monitoring platforms, ground sensor networks, and combat intelligence databases, and format classification is performed by combining file extension recognition and data content feature analysis. Files ending with.txt,.doc, etc. are determined as target text data; those ending with.jpg,.png, etc. are target image data; those ending with.mp4,.avi, etc. are target video data. For files without a clear extension, the binary structure and features are analyzed for classification. In the target cleaning and processing stage, for the target text data, the text information entropy of each sentence or paragraph is calculated, and semantic errors are located and corrected based on abnormal information entropy; for the target image and video data, the pixel value distribution and changes are analyzed to calculate the local region pixel information entropy, the noise regions are located and removed. At the same time, cross-format data consistency checking and information complementation are carried out, and the combat logic correction and verification are combined with the joint operation rules to finally obtain the standard data of the joint operation target system.

[0063] S2: The combat target feature fusion module is used to perform text feature embedding on the target text data in the standard data of the joint operation target system to generate combat target text features; based on large models of convolutional neural networks and vision transformers, target visual feature recognition is performed on the target image data and target video data in the standard data of the joint operation target system to generate combat target visual features; the combat target text features and combat target visual features are subjected to feature fusion mapping to obtain a fused feature vector representation of the combat target system;

[0064] In the embodiments of the present invention, for the target text data, a pre-trained large language model such as BERT is used for text feature embedding. First, the text is tokenized, the tokens are input into the model, the embedding layer maps the tokens to vectors, and the vector representations are updated through multiple encoder layers, and finally average pooling is performed to obtain combat target text features. For the target image and video data, they are input into a large model composed of a convolutional neural network branch and a vision transformer branch connected by an input layer and an intermediate connection layer. The convolutional neural network branch extracts local and context features through 5x5 convolutional layers, 3x3 convolutional layers, 1x1 pooling layers, and dilated convolutions; the vision transformer branch converts the image and video frame patches into sequences and captures global dependencies through self-attention mechanisms. The cross-branch interaction module in the intermediate connection layer enables feature interaction and fusion, thereby obtaining combat target visual features, and the combat target text features and visual features are mapped to the same feature space using a linear transformation matrix, and weights are determined according to cosine similarity, etc. for feature fusion, and finally a fused feature vector representation of the combat target system is obtained.

[0065] S3: The combat relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fusion feature vector representation of the combat target system to obtain combat target entities and the joint combat relationships between various combat target entities; construct a relationship combination weight graph between various combat target entities based on the joint combat relationships between various combat target entities to generate a joint combat target system knowledge graph;

[0066] In the embodiment of the present invention, by adopting a named entity recognition model based on deep learning, such as a bidirectional long short-term memory network (Bi-LSTM) combined with a conditional random field (CRF), entity recognition is performed on the fusion feature vector representation of the combat target system to obtain combat target entities such as combat troops, weapons and equipment, and military facilities. A method combining rules and machine learning is used for relationship extraction. Rules based on combat attributes are formulated to judge relationships such as subordination and command, and a graph convolutional network (GCN) model is used to judge relationships such as cooperation and support. A relationship combination weight graph is constructed based on a graph database, with combat target entities as nodes and joint combat relationships as edges. Edge relationship combination weights are assigned according to the relationship combination degree and stored and managed using a Neo4j database, and finally a joint combat target system knowledge graph is generated.

[0067] S4: The combat system service warning module is used to obtain a joint combat query requirement statement, and perform combat service query and situation warning analysis on the joint combat target system knowledge graph based on the joint combat query requirement statement to generate combat situation warning information corresponding to the joint combat target service and provide a corresponding joint combat response strategy suggestion plan.

[0068] In the embodiment of the present invention, in the joint combat command center, the voice signals of combat personnel are collected by a voice acquisition device at a sampling rate of 16,000 times per second and converted into a joint combat query requirement statement through a voice recognition algorithm; or a query statement is directly obtained through keyboard input. The query requirement statement is segmented and entity-extracted to obtain query statement entities, and text semantic embedding is performed in combination with the text encoder of the CLIP model. The semantic matching degree between the joint combat query text semantic embedding and the semantics of the joint combat targets in the joint combat target system knowledge graph is calculated, and the combat service plan is retrieved according to the matching degree. At the same time, combat resource, time limit, and geographical environment constraint information are obtained from the combat resource database, combat plan system, and geographical information system, and combat situation warning analysis is performed in combination with the combat service plan, such as warnings of resource shortage, time urgency, environmental risks, etc. Based on the warning information, a predefined response strategy template is called and adjusted and optimized in combination with the specific combat situation, and finally a joint combat response strategy suggestion plan is generated.

[0069] Further, the target system data processing module includes the following functions:

[0070] Obtain the combined operation target system data corresponding to different channels;

[0071] Classify and divide the combined operation target system data corresponding to different channels according to the target data format to obtain the combined operation target data corresponding to different target data formats, including target text data, target image data, and target video data;

[0072] Evaluate the information entropy of the combined operation target data corresponding to different target data formats, calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and calculate the pixel information entropy corresponding to the local area for the target image data and target video data by analyzing the distribution and change of the corresponding pixel values, so as to obtain the target data information entropy corresponding to different target data formats;

[0073] Perform noise-assisted positioning and execute cross-format data cleaning on the corresponding combined operation target data based on the target data information entropy corresponding to different target data formats, locate the existing text semantic errors and the local noise areas corresponding to the image and video according to the abnormally high part of the corresponding information entropy, and find the corresponding parts in the image and video data according to the text semantic errors mentioned in the target text data for consistency checking, and at the same time use the complementarity between different target data formats to supplement the missing information to obtain the combined operation target cross-format cleaning data;

[0074] Perform combat logic correction and verification between different target data formats in the combined operation target cross-format cleaning data by combining the combined operation rules. The combined operation rules stipulate the relationships between combined operation targets and the logical constraints of the sequence of actions, so as to check whether the target actions described in the target text data conform to the combat rules according to the logical constraints, and determine whether the status and position of the targets in the image and video data are consistent with the text description, so as to obtain the standard data of the combined operation target system.

[0075] As an embodiment of the present invention, refer to Figure 2 shown in Figure 1 is the functional flow diagram of the target system data processing module in

[0076] S11: Obtain the combined operation target system data corresponding to different channels;

[0077] In an embodiment of the present invention, high-resolution satellite image data and related geolocation information are obtained from a satellite reconnaissance system through a dedicated data interface at a fixed transmission rate per second. For a UAV monitoring platform, images, videos captured during flight, and environmental data collected by sensors are received in real time through a wireless communication module. A ground sensor network aggregates and transmits data such as monitored target movement trajectories and electromagnetic signals using wired or wireless transmission methods. A combat intelligence database extracts data such as historical combat intelligence and target information based on preset query conditions through a database query interface, and preliminarily integrates the data obtained from these different channels to construct a basic set of data for a joint combat target system, and finally obtains the data for the joint combat target system.

[0078] S12: Classify and divide the data for the joint combat target system corresponding to different channels according to the target data format to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data, and target video data;

[0079] In an embodiment of the present invention, the data is classified by combining file extension recognition and data content feature analysis. For files ending with text formats such as.txt and.doc, they are directly determined as target text data. For files ending with image formats such as.jpg and.png, they are determined as target image data. Files ending with video formats such as.mp4 and.avi are classified as target video data. For files without a clear extension, analyze the binary structure and features of their data. If the data contains text encoding information, it is classified as target text data; if the data exhibits the characteristics of an image pixel matrix, it is determined as target image data; if the data has the characteristics of a video frame sequence, it is classified as target video data. Finally, the data for the joint combat target system from different channels is accurately classified into the categories of target text data, target image data, and target video data.

[0080] S13: Evaluate the information entropy of the joint combat target data corresponding to different target data formats, calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and calculate the pixel information entropy corresponding to the local area for the target image data and target video data by analyzing the distribution and changes of the corresponding pixel values to obtain the target data information entropy corresponding to different target data formats;

[0081] In an embodiment of the present invention, for the target text data, first perform word segmentation on the text, count the occurrence frequency of each word, and according to the information entropy calculation formula where p(x i ) is the word x iCalculate the text information entropy of each sentence or paragraph based on the probability of occurrence. For the target image data, divide the image into multiple local regions of a fixed size, and count the distribution of pixel values in each region. Similarly, calculate the pixel information entropy of the local region according to the above information entropy formula. For the target video data, decompose the video into consecutive frame images, calculate the pixel information entropy of the local region for each frame image in the same way as processing the target image data, and then analyze the change of the pixel information entropy of the local region between frames. Finally, obtain the target data information entropy corresponding to different target data formats.

[0082] S14: Perform noise-assisted positioning on the corresponding joint operation target data based on the target data information entropy corresponding to different target data formats and execute cross-format data cleaning, so as to locate the existing text semantic errors and the corresponding local noise regions of the image and video according to the abnormally high part of the corresponding information entropy, and find the corresponding parts in the image and video data according to the text semantic errors mentioned in the target text data for consistency checking. At the same time, utilize the complementarity between different target data formats to supplement the missing information, so as to obtain the cross-format cleaning data of the joint operation target.

[0083] In the embodiment of the present invention, by setting the normal threshold range of the information entropy, when the information entropy of a certain sentence or paragraph in the target text data exceeds the threshold, it indicates that there is a text semantic error, and the location of the error is further located through the semantic analysis algorithm. For the target image and video data, when the pixel information entropy of the local region is abnormally high, it is determined that this region is a noise region. After discovering the semantic error in the target text data, according to the information such as the target features and positions described in the text, find the corresponding parts in the image and video data to check whether there are contradictions or inconsistencies. If some key information is missing in the target text data, such as the appearance features of the target, relevant information can be extracted from the image and video data for supplementation; on the contrary, if the identification information of a certain target in the image or video data is missing, it can be found and supplemented from the target text data. After such processing, the cross-format cleaning data of the joint operation target is obtained.

[0084] S15: Perform combat logic correction verification between different target data formats in the cross-format cleaning data of the joint operation target by combining the joint operation rules. The joint operation rules stipulate the relationships between joint operation targets and the logical constraints of the sequence of actions, so as to check whether the target actions described in the target text data conform to the combat rules according to the logical constraints, and determine whether the states and positions of the targets in the image and video data are consistent with the text description, so as to obtain the standard data of the joint operation target system.

[0085] In an embodiment of the present invention, by storing and managing the joint operation rules in the form of logical expressions, for the target text data, the target actions described therein are parsed and compared with the logical constraints in the joint operation rules. For example, if the rule stipulates that a certain combat action must be carried out after another action, and the order described in the text does not match, it is corrected. For image and video data, target recognition and tracking algorithms are used to determine the status and location of the target, and this information is compared with the target status and location described in the target text data. If there is an inconsistency, it is corrected according to a more reliable information source. For example, if the image clearly shows that the location of the target does not match the text description, the image information shall prevail. After comprehensive combat logic correction and verification, the standard data of the joint operation target system that conforms to the joint operation rules is finally obtained.

[0086] Further, the different channels specifically include a satellite reconnaissance system, a drone monitoring platform, a ground sensor network, and a combat intelligence database.

[0087] Further, the combat target feature fusion module includes the following functions:

[0088] Perform text feature embedding on the target text data in the standard data of the joint operation target system based on a pre-trained large language model to generate combat target text features;

[0089] Perform feature pre-analysis on the target image data and target video data in the standard data of the joint operation target system. For the target image data, use color space conversion based on optical principles to convert the image from the RGB space to the HSV space and the CIELAB space, analyze the hue distribution and color contrast features corresponding to the image in different color spaces, and use edge detection to extract the target edge contour features corresponding to the image. For the target video data, decompose it into consecutive frame images to analyze the time series relationship between frames, and calculate the optical flow information between adjacent frames to determine the movement trend and speed of the target, and obtain the multi-modal pre-analysis features of the joint operation target;

[0090] Based on the multi-modal pre-analysis features of the joint operation target and combined with a convolutional neural network and a Vision Transformer, perform large model fusion construction to generate a large model of the convolutional neural network and the Vision Transformer;

[0091] A large model based on a convolutional neural network and a vision Transformer performs target visual feature recognition on target image data and target video data within the standard data of the joint operation target system, so as to obtain small-scale detailed features to large-scale global features through convolutional operations of different layers in the convolutional neural network branch, and obtain the changes in the target motion state of the joint operation target at different time scales by adjusting the size of the input block in the vision Transformer branch, so as to generate combat target visual features, including target shape, position, and combat state features;

[0092] Perform feature fusion mapping on the combat target text features and the combat target visual features, so as to map the combat target text features and the combat target visual features into the same feature space, and determine the corresponding weights for feature fusion according to the internal association between the combat target text features and the combat target visual features, and obtain the fused feature vector representation of the combat target system.

[0093] As an embodiment of the present invention, refer to Figure 3 shown, for Figure 1 the functional flow diagram of the combat target feature fusion module in

[0094] S21: Perform text feature embedding on the target text data in the standard data of the joint operation target system based on a pre-trained large language model to generate combat target text features;

[0095] In the embodiment of the present invention, by selecting a large language model pre-trained on a large-scale general corpus, such as BERT based on the Transformer architecture, the target text data in the standard data of the joint operation target system is tokenized to obtain a series of tokens. These tokens are input into the pre-trained large language model, and the embedding layer of the model will map each token into a low-dimensional vector representation. As the tokens are passed through multiple encoder layers of the model, each token will continuously update its vector representation according to the context information. Perform average pooling operation on the vector representations of all tokens to obtain a fixed-length vector, which is the combat target text feature. It contains the semantic information of the target text data and finally generates the combat target text feature.

[0096] S22: Pre-parse the feature of the target image data and target video data in the standard data of the joint operation target system. For the target image data, use the color space conversion based on the optical principle to convert the image from the RGB space to the HSV space and the CIELAB space, analyze the hue distribution and color contrast characteristics corresponding to the image in different color spaces, and use edge detection to extract the target edge contour feature corresponding to the image. For the target video data, decompose it into consecutive frame images, analyze the time series relationship between frames, and calculate the optical flow information between adjacent frames to determine the movement trend and speed of the target, obtaining the multi-modal pre-parse features of the joint operation target;

[0097] In the embodiment of the present invention, for the target image data, use the color space conversion algorithm to convert the image from the RGB space to the HSV space and the CIELAB space according to the optical principle. In the HSV space, count the pixel distribution of different hues, calculate the saturation and brightness ranges of the color to analyze the hue distribution and color contrast characteristics. In the CIELAB space, perform relevant statistical analysis in the same way, and use the Canny edge detection algorithm to extract the edge contour feature of the target in the image. For the target video data, decompose the video into consecutive frame images according to the frame rate. By comparing the changes of pixels in adjacent frame images, use the Lucas-Kanade optical flow algorithm to calculate the optical flow information, determine the movement trend and speed of the target between frames. At the same time, analyze the time series relationship between frames, such as the time points when the target appears and disappears, etc. Integrate the features of these images and videos to finally obtain the multi-modal pre-parse features of the joint operation target.

[0098] S23: Based on the multi-modal pre-parse features of the joint operation target and combined with the convolutional neural network and the Vision Transformer, conduct large model fusion construction to generate the large models of the convolutional neural network and the Vision Transformer;

[0099] In an embodiment of the present invention, by constructing a large model of a convolutional neural network and a Vision Transformer, its input layer receives multi-modal pre-parsed features of joint operation targets. In the convolutional neural network branch, a 5x5 convolutional layer is designed to extract local features at a larger scale, a 3x3 convolutional layer further refines the local features, and a 1x1 pooling layer is used for feature dimensionality reduction. At the same time, dilated convolution is adopted, and the receptive field is expanded by setting different dilation rates to obtain the context information of the operation target. In the Vision Transformer branch, the image and video frames are segmented into blocks of a fixed size, these blocks are transformed into sequence inputs, and through the self-attention mechanism, the correlation between elements at different positions in the sequence is calculated to capture the global dependencies between different time positions. In the intermediate connection layer, a cross-branch interaction module is designed, for example, in the way of feature concatenation and attention fusion, to enable the features of the convolutional neural network branch and the Vision Transformer branch to interact, so that the model can comprehensively utilize local and global information. The model is trained using the stochastic gradient descent algorithm, continuously adjusting the network parameters, and finally generating a large model of a convolutional neural network and a Vision Transformer.

[0100] S24: Based on the large model of the convolutional neural network and the Vision Transformer, perform target visual feature recognition on the target image data and target video data in the standard data of the joint operation target system, so as to obtain small-scale detailed features to large-scale global features through convolutional operations of different layers in the convolutional neural network branch, and obtain the changes in the target motion state of the joint operation target at different time scales by adjusting the size of the input blocks in the Vision Transformer branch, so as to generate the visual features of the operation target, including target shape, position, and combat state features;

[0101] In an embodiment of the present invention, by inputting the target image data and target video data in the standard data of the joint operation target system into the large model of the convolutional neural network and the Vision Transformer, in the convolutional neural network branch, the low-level 5x5 convolutional layer and 3x3 convolutional layer extract small-scale detailed features of the image and video frames, such as the texture and edges of the target. As the convolutional layer deepens, the size of the feature map gradually decreases, and the extracted features transition from local details to global features. In the Vision Transformer branch, by adjusting the size of the input blocks, such as splitting large image blocks or video frame blocks into smaller blocks, the changes in the motion state of the joint operation target can be observed at different time scales. Combining the outputs of the convolutional neural network branch and the Vision Transformer branch, through post-processing algorithms such as non-maximum suppression, the shape and position of the target are determined. At the same time, according to the motion state and context information of the target, the combat state of the target, such as attack, defense, etc., is judged, and finally the visual features of the operation target are generated.

[0102] S25: Feature fusion mapping is performed on the combat target text features and the combat target visual features to map the combat target text features and the combat target visual features into the same feature space, and corresponding weights are determined according to the internal association between the combat target text features and the combat target visual features for feature fusion, obtaining a fused feature vector representation of the combat target system.

[0103] In the embodiment of the present invention, the combat target text features and the combat target visual features are mapped into the same feature space by using a linear transformation matrix, so as to analyze their internal association by calculating indexes such as cosine similarity and mutual information between the two feature vectors. According to these association indexes, an adaptive weight assignment algorithm is used to determine the weights of the combat target text features and the combat target visual features. For example, if the text features and the visual features have a high consistency in describing the position information of the target, then relatively high weights are given to both during fusion. The weighted text features and visual features are added together to obtain a new feature vector, which is the fused feature vector representation of the combat target system. It integrates the information of both text and visual modalities and can describe the joint combat target more comprehensively, finally obtaining the fused feature vector representation of the combat target system.

[0104] Further, the text feature embedding of the target text data in the joint combat target system standard data based on the pre-trained large language model includes:

[0105] Perform in-depth analysis at the architecture level of the pre-trained large language model to use graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths inside the large language model, obtaining the architecture analysis result of the pre-trained large model;

[0106] In the embodiment of the present invention, the pre-trained large language model is regarded as a complex graph structure, where the neurons of each layer are used as the nodes of the graph, and the inter-layer connections and the connections between neurons are used as the edges. First, the layer information is extracted from the code structure of the model to clarify the number of layers of the model and the number of neurons in each layer. The inter-layer connection relationship is represented by using the adjacency matrix in graph theory, and the elements in the matrix represent whether there is a connection between two layers and the strength of the connection. For the neuron interaction pattern, by analyzing the type and parameters of the activation function, the information transfer rule between neurons is determined, and the path search algorithm in network analysis methods, such as breadth-first search, is used to trace the information flow path inside the model, and the input-output relationship of each layer, the activation situation of neurons, and the sequence of information transfer are recorded in detail. Finally, the architecture analysis result of the pre-trained large model is obtained, which contains the detailed information of the inter-layer connection relationship, neuron interaction pattern, and information flow path.

[0107] Preferably, based on the parsing results of the pre-trained large model architecture and introducing a preprocessing strategy based on military domain knowledge, text preprocessing enhancement is performed on the target text data within the standard data of the joint operation target system, so as to utilize the military semantic network to expand the semantic relationships between target vocabulary, and obtain the target text data after preprocessing enhancement;

[0108] In the embodiment of the present invention, according to the parsing results of the pre-trained large model architecture, the feature preferences and processing capabilities of the model for input data are understood, so as to introduce a preprocessing strategy based on military domain knowledge. First, a military semantic network is established, which contains a large number of military terms, concepts, and their semantic relationships. For example, there is a subordination relationship between "tank" and "armored force", and an association relationship between "air raid" and "air superiority". For the target text data within the standard data of the joint operation target system, word segmentation is performed to split the text into individual words. Then, relevant semantic information of each word is searched in the military semantic network to expand the semantic relationships between target vocabulary. For example, if the text mentions "infantry", relevant information such as "infantry weapons" and "infantry tactics" can be associated through the military semantic network, and these expanded information are added to the target text data. After such processing, the target text data after preprocessing enhancement is finally obtained, which contains richer military semantic information.

[0109] Preferably, based on the pre-trained language large model, text-model mapping is performed on the target text data after preprocessing enhancement, so as to divide the target text data into target description, action intention, and operational environment parts according to the logical structure corresponding to military operations, and map each part to the corresponding feature space area of the language large model to generate target text-model mapping rules;

[0110] In the embodiment of the present invention, according to the parsing results of the pre-trained large model architecture, the feature space distribution and functions of different regions of the model are understood. For the target text data after preprocessing enhancement, it is divided according to the logical structure of military operations. Using a rule-based method, according to the keywords and grammatical structures in the text, the text is divided into three parts: target description, action intention, and operational environment. For example, the part containing words such as "capture" and "destroy" may belong to the action intention; the part describing information such as the combat location and weather belongs to the operational environment. Then, the sensitivity of different layers and neurons in the pre-trained language large model to different types of information is analyzed, and the target description, action intention, and operational environment parts are respectively mapped to the feature space areas in the model that are most sensitive to this type of information. Through multiple experiments and adjustments, the accurate mapping positions of each part in the model feature space are determined, and finally target text-model mapping rules are generated, which clarify the corresponding relationship between each part of the text and the model feature space area.

[0111] Preferably, the pre - processed and enhanced target text data is input into a pre - trained large - language model according to the target text - model mapping rule for text feature extraction, and the text encoder corresponding to the CLIP model is used for text feature embedding to generate combat target text features.

[0112] In the embodiment of the present invention, by inputting the pre - processed and enhanced target text data into the corresponding feature space regions of the pre - trained large - language model according to the target text - model mapping rule, the target description, action intention, and combat environment parts are respectively input. Inside the model, the data undergoes layer - by - layer processing. Each neuron calculates the input according to its own weight and activation function, and finally outputs the feature representation of the text. Then, these feature representations are input into the text encoder corresponding to the CLIP model. The text encoder of the CLIP model is based on the Transformer architecture and contains multiple encoder layers. In each encoder layer, the multi - head self - attention mechanism is used to capture the correlation between text features to further extract the semantic information of the text. After the processing of the encoder, the text features are converted into a fixed - length vector representation, which is the combat target text feature. It contains rich semantic information of the target text data and can be used for subsequent joint combat target system data service tasks, and finally generates combat target text features.

[0113] Furthermore, the large models of the convolutional neural network and the Vision Transformer are specifically composed of an input layer and a convolutional neural network branch and a Vision Transformer branch connected by an intermediate connection layer. Among them, the multi - modal pre - parsed features of the joint combat target are respectively and parallelly input into the convolutional neural network branch and the Vision Transformer branch through the input layer for model training. In the convolutional neural network branch, corresponding 5x5 convolutional layers, 3x3 convolutional layers, and 1x1 pooling layers are designed to extract the local detailed features corresponding to images and video frames. At the same time, dilated convolution is used to expand the receptive field to obtain the corresponding combat target context information. In the Vision Transformer branch, image patches and video frame patches are converted into sequences to capture the global dependencies between different time positions through the self - attention mechanism, and a cross - branch interaction module is designed in the intermediate connection layer to enable the interaction and fusion of features between the convolutional neural network branch and the Vision Transformer branch.

[0114] Furthermore, the combat relationship knowledge graph construction module includes the following functions:

[0115] Perform combat target entity recognition on the combat target system fusion feature vector representation to obtain combat target entities, including combat troops, weapons and equipment, and military facilities;

[0116] In an embodiment of the present invention, a named entity recognition model based on deep learning is used to process the fusion feature vector representation of the combat target system. The model is based on a bidirectional long short-term memory network (Bi-LSTM) combined with a conditional random field (CRF), and is pre-trained on a large-scale military text dataset. First, the fusion feature vector of the combat target system is input into the Bi-LSTM layer, which can capture the context information in the sequence data and learn the semantic features of each position in the feature vector. Then, the output of the Bi-LSTM is passed to the CRF layer, which predicts the label for each position according to the context information and predefined label constraints. The labels include categories such as combat troops, weaponry, and military facilities. Through the forward propagation and prediction of the model, the combat target entities in the fusion feature vector representation of the combat target system are finally identified and classified into the categories of combat troops, weaponry, and military facilities respectively.

[0117] Preferably, a combat attribute analysis is performed on each combat target entity to obtain the combat attributes corresponding to each combat target entity;

[0118] In an embodiment of the present invention, for the identified combat target entities, a combat attribute analysis system including multiple rules and templates is constructed. For combat troops, their establishment information, including the number of personnel, arms composition, etc., is obtained from the military database, and at the same time, their combat capabilities, such as attack capabilities and defense capabilities, are analyzed. These capabilities can be quantified through historical combat data and training results. For weaponry, its technical parameters, such as range, accuracy, and firing rate, as well as usage conditions, such as environmental adaptability and maintenance requirements, are analyzed. For military facilities, attributes such as their geographical location, protection level, and bearing capacity are investigated. The relevant information of each combat target entity is input into the combat attribute analysis system, and the system analyzes and extracts according to the preset rules and templates, and finally obtains the combat attributes corresponding to each combat target entity.

[0119] Preferably, based on the combat attributes corresponding to each combat target entity, an extraction of combat knowledge relationships is performed between each combat target entity to obtain the joint combat relationships between each combat target entity, including subordination relationships, command relationships, cooperative combat relationships, support relationships, attack and defense relationships, and threat relationships;

[0120] In the embodiments of the present invention, by using a method that combines rules and machine learning for extracting combat knowledge relationships. First, a series of rules based on combat attributes are formulated. For example, if the personnel and equipment of a combat unit belong to the establishment system of another combat unit, it is determined that there is a subordination relationship between them; if a combat unit is clearly designated in a combat plan to command another combat unit, there is a command relationship. For some relationships that are difficult to directly judge by rules, such as cooperative combat relationships, support relationships, attack and defense relationships, and threat relationships, a machine learning model is used for judgment. The graph convolutional network (GCN) model is adopted to represent combat target entities and their combat attributes as nodes and features of a graph. By learning the feature interactions and topological structures between nodes, the relationship types between nodes are predicted. The various combat target entities and their combat attributes are input into the rule system and the machine learning model. After comprehensive judgment and reasoning, the joint combat relationships between the various combat target entities are finally obtained.

[0121] Preferably, based on the joint combat relationships between the various combat target entities, a relationship combination weight graph is constructed between the various combat target entities to generate a knowledge graph of the joint combat target system.

[0122] In the embodiments of the present invention, a relationship combination weight graph is constructed based on a graph database. Each combat target entity is used as a node in the graph, and the joint combat relationships between the various combat target entities are used as edges. For each type of joint combat relationship, a corresponding relationship combination weight value is assigned according to the combat relationship combination degree between the various combat target entities. For example, the weight values of the subordination relationship and the command relationship are relatively high because they reflect a strong constraint and control relationship; while the weight values of the cooperative combat relationship and the support relationship are adjusted according to the specific combat tasks and cooperation frequencies. And by using a graph database management system, such as Neo4j, the nodes, edges, and relationship combination weight values are stored and managed. Through the connection of nodes and edges, all combat target entities and their joint combat relationships are organized into a graph structure, and finally a knowledge graph of the joint combat target system is generated.

[0123] Further, the construction of the relationship combination weight graph between the various combat target entities based on the joint combat relationships between the various combat target entities includes:

[0124] Determining the number of potential relationship connections between the various combat target entities based on the joint combat relationships between the various combat target entities to determine the total number of entities that still have a joint combat relationship with any two combat target entities, and obtaining the number of potential relationship connections between the various combat target entities;

[0125] In the embodiments of the present invention, by extracting each combat target entity and the clear combined operation relationships between them from the relevant data records of combined operations, these relationships can be subordination relationships, command relationships, coordinated operation relationships, support relationships, attack-defense relationships, threat relationships, etc. Represent these combat target entities as nodes in a graph, and represent the combined operation relationships as edges between the nodes to construct an initial combat relationship graph. For any two combat target entities, traverse the entire combat relationship graph to find those entities that have both a combined operation relationship with the first combat target entity and a combined operation relationship with the second combat target entity. Use the breadth-first search algorithm to start searching from the first combat target entity, record the number of entities that are also connected to the second combat target entity encountered during the search process, and perform such operations on all pairs of combat target entities. Finally, obtain the potential relationship connection numbers between each combat target entity.

[0126] Preferably, based on the potential relationship connection numbers between each combat target entity, use the combat relationship combination metric formula to calculate the relationship combination degree between each combat target entity to obtain the combat relationship combination degree between each combat target entity;

[0127] In the embodiments of the present invention, a suitable combat relationship combination metric formula is constructed by combining the total number of combat target entities, the entity similarity between each combat target entity, the entity similarity weight, the potential relationship connection number, the combat distance, and relevant parameters to perform the quantitative calculation of the relationship combination degree, so as to calculate the corresponding combat relationship combination degree, and finally obtain the combat relationship combination degree between each combat target entity. In addition, this combat relationship combination metric formula can also use any relationship combination association method in the art to replace the process of calculating the relationship combination degree, and is not limited to this combat relationship combination metric formula.

[0128] Preferably, based on the combat relationship combination degree between each combat target entity, construct a relationship combination weight graph between the combat target entity and the combined operation relationships between each combat target entity, use the combat target entity as a node, use the combined operation relationships between each combat target entity as an edge, and use the combat relationship combination degree as the edge relationship combination weight value between each combat target entity to generate a combined operation target system knowledge graph.

[0129] In the embodiment of the present invention, based on the combat target entities, each combat target entity is regarded as a node in the graph. For the joint combat relationship existing between any two combat target entities, it is represented as an edge connecting these two nodes, and according to the combat relationship combination degree calculated previously between each combat target entity, it is assigned to the corresponding edge as the relationship combination weight of the edge. A graph database management system is used to store and manage this graph structure, such as the Neo4j database. During the process of constructing the graph, it is ensured that each node and edge has a unique identifier for convenient subsequent query and analysis. By organizing and representing all combat target entities and joint combat relationships in this way, a knowledge graph of the joint combat target system is finally generated.

[0130] Further, the specific formula for measuring the combat relationship combination is as follows:

[0131]

[0132] In the formula, R ij is the combat relationship combination degree between the i-th combat target entity T i and the j-th combat target entity T j , N is the total number of combat target entities, Sim(T i , T j ) is the entity similarity between the i-th combat target entity T i and the j-th combat target entity T j , α ij is the entity similarity weight between the i-th combat target entity T i and the j-th combat target entity T j , exp is the exponential function, k ij is the potential relationship connection number between the i-th combat target entity T i and the j-th combat target entity T j , d ij is the combat distance between the i-th combat target entity T i and the j-th combat target entity T j .

[0133] The present invention obtains a combat relationship combination measurement formula through using a specific mathematical model and verification for calculating the relationship combination degree between each combat target entity. This formula fully considers the combat relationship combination degree R i between the i-th combat target entity T j and the j-th combat target entity T ij , the total number N of combat target entities, and the entity similarity between the i-th combat target entity T i and the j-th combat target entity T jThe entity similarity Sim(T i , T j ), the i-th combat target entity T i and the j-th combat target entity T j The entity similarity weight α ij , the exponential function exp, the i-th combat target entity T i and the j-th combat target entity T j The potential relationship connection number k ij , the i-th combat target entity T i and the j-th combat target entity T j The combat distance d ij , according to the i-th combat target entity T i and the j-th combat target entity T j The combat relationship combination degree R ij The mutual correlation relationship between R and the above parameters constitutes a functional relationship This formula can realize the calculation process of the relationship combination degree between each combat target entity. At the same time, by combining factors in different dimensions (such as similarity, weight, distance, etc.), it can quantify the relationship between different combat targets. This quantification enables the commander to clearly identify which combat targets have a closer relationship, which helps to formulate a more reasonable combat plan. By calculating the combat relationship combination degree between each combat target entity, the most concerned targets in the joint combat process can be effectively identified, which helps to concentrate resources and strength, optimize the military force deployment and tactical configuration, and thus improve the combat effectiveness. The combat environment is often dynamically changing. Using this formula, the relationship combination degree between combat targets can be updated in real time. The commander can adjust the combat plan and targets at any time according to the latest intelligence and battlefield situation, enhancing the ability to flexibly respond to various emergencies. Through a comprehensive analysis of the combat relationship, it can promote the coordinated combat among all parties in the joint combat. For example, by identifying the key joint combat targets, the relevant troops can maintain tacit understanding during the combat process, implement joint offensives or coordinated coverings, and improve the overall combat effect. In addition, this combat relationship combination degree formula takes into account multiple input factors (similarity, weight, distance, etc.), which provides a multi-dimensional perspective for analysis. Different parameters can be selected for adjustment according to different military situations to better serve specific combat needs.

[0134] Furthermore, the combat system service warning module includes the following functions:

[0135] Obtain the joint combat query requirement statement proposed by combat personnel through natural language;

[0136] In the embodiments of the present invention, by setting up dedicated voice collection devices and text input terminals in the joint operations command center, combatants can submit joint operations query requirements through voice or keyboard input. When combatants use voice input, the voice collection device samples the voice signal at a sampling rate of 16,000 times per second, converts it into a digital audio signal, and then uses advanced voice recognition algorithms, such as end-to-end voice recognition models based on deep learning, to accurately convert the digital audio signal into a text-form joint operations query requirement statement. If combatants use keyboard input, the system directly obtains the input text content as the query requirement statement.

[0137] Preferably, word segmentation and entity extraction are performed on the joint operations query requirement statement to obtain joint operations query statement entities;

[0138] In the embodiments of the present invention, by using a word segmentation algorithm that combines rules and machine learning to process the joint operations query requirement statement, the rule part accurately segments common words and fixed collocations in the statement based on professional dictionaries and grammar rules in the military field. The machine learning part uses a bidirectional long short-term memory network (Bi-LSTM) combined with a conditional random field (CRF) model trained with a large amount of military text data to further optimize the word segmentation result. After word segmentation, named entity recognition (NER) technology is used for entity extraction. Based on the pre-trained BERT model, fine-tuning is performed on the military field dataset to identify entities such as combat units, combat locations, combat times, and combat tasks in the statement, and finally joint operations query statement entities are obtained.

[0139] Preferably, based on the joint operations query statement entity and combined with the text encoder of the CLIP model, text semantic embedding is performed on the joint operations query requirement statement to generate joint operations query text semantic embedding;

[0140] In the embodiments of the present invention, by taking the joint operations query statement entity as input and feeding it into the text encoder of the CLIP model, the text encoder of the CLIP model is based on the Transformer architecture and consists of multiple encoder layers stacked. In each encoder layer, first, multi-head self-attention mechanism calculation is performed to capture semantic information in the statement by calculating the correlation between words at different positions, and then feed-forward neural network calculation is performed to perform a non-linear transformation on the output of the attention mechanism. After processing by multiple encoder layers, the semantic representation vectors of each word are obtained. Finally, average pooling operation is performed on these word vectors to integrate them into a fixed-length vector as the joint operations query text semantic embedding, and finally joint operations query text semantic embedding is generated.

[0141] Preferably, based on the semantic embedding of the joint operation query text, perform an operation service matching query on the corresponding joint operation objectives in the knowledge graph of the joint operation objective system, calculate the semantic matching degree between the semantic embedding of the joint operation query text and the joint operation objectives, and retrieve and determine the operation service plan corresponding to the joint operation objectives according to the semantic matching degree, including operation tasks, operation action trends, and operation intentions;

[0142] In the embodiment of the present invention, the knowledge graph of the joint operation objective system stores a large amount of joint operation objectives and their related information. For each joint operation objective, use the same method as generating the semantic embedding of the query text to convert its description information into a semantic embedding vector. By calculating the cosine similarity between the semantic embedding vector of the joint operation query text and the semantic embedding vectors of each joint operation objective in the knowledge graph, measure their semantic matching degree, set a similarity threshold, such as 0.7, and screen out the joint operation objectives with a similarity greater than the threshold. For the screened joint operation objectives, retrieve information such as their corresponding operation tasks, operation action trends, and operation intentions from the knowledge graph to form an operation service plan, and finally determine the operation service plan corresponding to the joint operation objectives.

[0143] Preferably, obtain the corresponding operation resources, time limits, and geographical environment constraints, and perform an operation situation early warning analysis on the operation service plan corresponding to the joint operation objectives based on the operation resources, time limits, and geographical environment constraints, so as to generate the operation situation early warning information corresponding to the joint operation objective service, and provide the corresponding joint operation response strategy suggestion plan according to the operation situation early warning information.

[0144] In the embodiment of the present invention, obtain the currently available operation resource information from the operation resource database, including the quantity of weapons and equipment, the number of personnel, etc.; obtain the time limit information from the operation plan system, such as the start time and end time of the operation, etc.; obtain the geographical environment constraint information from the geographical information system, such as terrain and landform, meteorological conditions, etc., and comprehensively analyze this information with the operation service plan. For example, evaluate whether the operation resources can meet the requirements of the operation tasks according to the operation resources, and issue a resource shortage warning if the resources are insufficient; judge whether the operation actions can be completed on time according to the time limit, and issue a time urgency warning if the time is tight; analyze the impact of the geographical environment constraint on the operation actions, such as bad weather may affect the air operation actions, and issue the corresponding environmental risk warning. Based on these warning information, call the predefined response strategy template, adjust and optimize it in combination with the specific operation situation, and finally generate the joint operation response strategy suggestion plan.

[0145] Therefore, from any perspective, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Thus, all changes falling within the meaning and scope of the equivalent elements of the application documents are intended to be encompassed within the present invention.

[0146] The above description is only a specific implementation manner of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features invented herein.

Claims

1. A joint combat target system data service system based on a large model, characterized in that: Includes the following modules: The target system data processing module is used to obtain the joint combat target system data corresponding to different channels, and classify the joint combat target system data corresponding to different channels according to the target data format, so as to obtain the joint combat target data corresponding to different target data formats, including target text data, target image data and target video data; Perform target cleaning on joint combat target data corresponding to different target data formats to obtain standard data of joint combat target system; The combat target feature fusion module is used to embed text features of target text data in the standard data of the joint combat target system to generate combat target text features; based on the convolutional neural network and the large model of the visual Transformer, the target visual feature recognition is performed on the target image data and target video data in the standard data of the joint combat target system to generate combat target visual features; Perform feature fusion mapping on the combat target text features and the combat target visual features to obtain the fusion feature vector representation of the combat target system; The operational relationship knowledge graph construction module is used to perform entity recognition and relationship extraction on the fusion feature vector representation of the operational target system to obtain the operational target entity and the joint operational relationship between each operational target entity; based on the joint operational relationship between each operational target entity, the relationship combination weight graph between each operational target entity is constructed to generate a knowledge graph of the joint operational target system; The combat system service warning module is used to obtain joint combat query demand statements, and based on the joint combat query demand statements, conduct combat service queries and situation warning analysis on the joint combat target system knowledge graph to generate combat situation warning information corresponding to the joint combat target service, and provide corresponding joint combat response strategy recommendations.

2. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The target system data processing module includes the following functions: Obtain joint combat target system data corresponding to different channels; By classifying and dividing the joint combat target system data corresponding to different channels according to the target data format, the joint combat target data corresponding to different target data formats are obtained, including target text data, target image data and target video data; Information entropy evaluation is performed on the joint combat target data corresponding to different target data formats, so as to calculate the text information entropy corresponding to each sentence or paragraph for the target text data, and the pixel information entropy corresponding to the local area is calculated by analyzing the distribution and change of the pixel value for the target image data and the target video data, so as to obtain the target data information entropy corresponding to different target data formats; Based on the target data information entropy corresponding to different target data formats, the corresponding joint combat target data is noise-assistedly located and cross-format data cleaning is performed to locate the existing text semantic errors and the local noise areas corresponding to the images and videos according to the abnormally high parts of the corresponding information entropy, and the corresponding parts are searched in the image and video data according to the text semantic errors mentioned in the target text data for consistency check, and the complementarity between different target data formats is used to supplement the missing information to obtain the cross-format cleaned data of the joint combat target; By combining the joint combat rules, the combat logic correction and verification are performed between different target data formats in the cross-format cleaning data of the joint combat targets. The joint combat rules stipulate the relationship between joint combat targets and the logical constraints of the sequence of actions. According to the logical constraints, it is checked whether the target actions described in the target text data comply with the combat rules, and it is determined whether the status and position of the target in the image and video data are consistent with the text description, so as to obtain the standard data of the joint combat target system.

3. The large model-based joint combat target system data service system according to claim 2 is characterized in that: The different channels specifically include satellite reconnaissance systems, drone monitoring platforms, ground sensor networks and combat intelligence databases.

4. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The combat target feature fusion module Features include: Based on the pre-trained language model, text features are embedded into the target text data in the standard data of the joint combat target system to generate combat target text features; The target image data and target video data in the standard data of the joint combat target system are pre-analyzed to convert the target image data from RGB space to HSV space and CIELAB space using color space conversion based on optical principles, analyze the hue distribution and color contrast characteristics of the images under different color spaces, and use edge detection to extract the target edge contour features corresponding to the image. The target video data is decomposed into continuous frame images to analyze the time series relationship between frames, and the optical flow information between adjacent frames is calculated to determine the target's motion trend and speed, so as to obtain the multi-modal pre-analysis features of the joint combat target. Based on the multimodal pre-analysis features of joint combat targets and combined with convolutional neural networks and visual transformers, a large model is constructed to generate a large model of convolutional neural networks and visual transformers; Based on the convolutional neural network and the large model of the visual transformer, the target image data and the target video data in the standard data of the joint combat target system are used to identify the target visual features, so as to obtain the small-scale detail features to the large-scale global features through the convolution operations of different layers in the convolutional neural network branch, and to obtain the target motion state changes of the joint combat target at different time scales by adjusting the size of the input block in the visual transformer branch, so as to generate the visual features of the combat target, including the target shape, position and combat state features; The text features and visual features of the combat targets are fused and mapped to the same feature space, and the corresponding weights are determined according to the intrinsic correlation between the text features and the visual features of the combat targets to perform feature fusion and obtain the fused feature vector representation of the combat target system.

5. The large model-based joint combat target system data service system according to claim 4 is characterized in that: The text feature embedding of target text data in the standard data of the joint combat target system based on the pre-trained language large model includes: Conduct an in-depth analysis of the pre-trained language model at the architectural level, and use graph theory and network analysis methods to sort out the corresponding inter-layer connection relationships, neuron interaction patterns, and information flow paths within the language model to obtain the pre-trained model architecture analysis results; Based on the analysis results of the pre-trained large model architecture and the introduction of a pre-processing strategy based on military domain knowledge, the target text data in the standard data of the joint combat target system is pre-processed and enhanced, so as to use the military semantic network to expand the semantic relationship between target words and obtain the target text data after pre-processing and enhancement; Based on the pre-trained language model, the pre-processed and enhanced target text data is mapped from text to model, so as to divide the target text data into target description, action intention and combat environment parts according to the logical structure corresponding to the military action, and each part is mapped to the feature space area corresponding to the language model to generate the target text-model mapping rules; The preprocessed and enhanced target text data is input into the pre-trained language model according to the target text-model mapping rules for text feature extraction, and the text feature embedding is performed using the text encoder corresponding to the CLIP model to generate combat target text features.

6. The large model-based joint combat target system data service system according to claim 4 is characterized in that: The large model of the convolutional neural network and the visual transformer specifically consists of an input layer and a convolutional neural network branch and a visual transformer branch connected by an intermediate connection layer, wherein the multimodal pre-analysis features of the joint combat target are respectively input in parallel into the convolutional neural network branch and the visual transformer branch through the input layer for model training, so that the corresponding 5x5 convolutional layer, 3x3 convolutional layer and 1x1 pooling layer are designed in the convolutional neural network branch to extract the local detail features corresponding to the image and video frame, and the hollow convolution is used to expand the receptive field to obtain the corresponding combat target context information, and the image blocks and video frame blocks are converted into sequences in the visual transformer branch to capture the global dependency between different time positions through the self-attention mechanism, and a cross-branch interaction module is designed in the intermediate connection layer to allow the features between the convolutional neural network branch and the visual transformer branch to interact and fuse.

7. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The operational relationship knowledge graph construction module includes the following functions: Perform combat target entity recognition on the combat target system fusion feature vector representation to obtain combat target entities, including combat troops, weapons and equipment, and military facilities; Conduct combat attribute analysis on each combat target entity to obtain the combat attributes corresponding to each combat target entity; Based on the combat attributes corresponding to each combat target entity, the combat knowledge relationship between each combat target entity is extracted to obtain the joint combat relationship between each combat target entity, including affiliation, command relationship, coordinated combat relationship, support relationship, attack and defense relationship, and threat relationship; Based on the joint combat relationship between various combat target entities, a relationship combination weight graph is constructed between various combat target entities to generate a knowledge graph of the joint combat target system.

8. The large model-based joint combat target system data service system according to claim 7 is characterized in that: The construction of the relationship combination weight graph between various combat target entities based on the joint combat relationship between various combat target entities includes: Based on the joint combat relationship between each combat target entity, the number of potential relationship connections between each combat target entity is determined to determine the total number of entities that still have a joint combat relationship between any two combat target entities, and the number of potential relationship connections between each combat target entity is obtained; Based on the potential relationship connection number between each combat target entity, the combat relationship combination measurement formula is used to calculate the relationship combination degree between each combat target entity to obtain the combat relationship combination degree between each combat target entity; Based on the degree of operational relationship integration between various operational target entities, a relational integration weight graph is constructed between the operational target entities and the joint operational relationships between various operational target entities, with the operational target entities as nodes, the joint operational relationships between various operational target entities as edges, and the operational relationship integration degree as the edge relationship integration weight between various operational target entities, so as to generate a knowledge graph of the joint operational target system.

9. The large model-based joint combat target system data service system according to claim 8 is characterized in that: The operational relationship combined measurement formula is specifically: In the formula, R ij is the i-th combat target entity T i With the jth combat target entity T j The degree of operational relationship between them, N is the total number of combat target entities, Sim(T i ,T j ) is the i-th combat target entity T i With the jth combat target entity T j The entity similarity between ij is the i-th combat target entity T i With the jth combat target entity T j The entity similarity weight between them, exp is the exponential function, k ij is the i-th combat target entity T i With the jth combat target entity T j The number of potential relationship connections between ij is the i-th combat target entity T i With the jth combat target entity T j The combat distance between them.

10. The large model-based joint combat target system data service system according to claim 1 is characterized in that: The combat system service warning module includes the following functions: Obtain joint combat query requirement statements raised by combatants in natural language; Perform word segmentation and entity extraction on the joint operations query requirement statement to obtain the joint operations query statement entity; Based on the joint combat query statement entity and combined with the text encoder corresponding to the CLIP model, the text semantic embedding of the joint combat query requirement statement is performed to generate the joint combat query text semantic embedding; Based on the semantic embedding of the joint combat query text, a combat service matching query is performed on the corresponding joint combat target in the knowledge graph of the joint combat target system to calculate the semantic matching degree between the semantic embedding of the joint combat query text and the joint combat target, and the combat service plan corresponding to the joint combat target is retrieved and determined according to the semantic matching degree, including combat missions, combat action trends and combat intentions; Obtain the corresponding combat resources, time limits and geographical environment constraints, and conduct combat situation warning analysis on the combat service plan corresponding to the joint combat objectives based on the combat resources, time limits and geographical environment constraints, so as to generate combat situation warning information corresponding to the joint combat objective service, and provide corresponding joint combat response strategy recommendation plans based on the combat situation warning information.

Images