A method, device and storage medium for evaluating the effectiveness of a combat system architecture

Abstract

The present invention provides a method, device and storage medium for evaluating the effectiveness of a combat system architecture, which belongs to the field of simulation technology. The method includes: importing a combat mission effectiveness evaluation index data set, and performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain multiple target capability index data, and using all target capability index data and combat mission effectiveness evaluation index data sets as data sets to be processed; performing predictive analysis on the data sets to be processed to obtain a combat system architecture effectiveness evaluation result. The present invention can greatly improve the training speed and prediction accuracy, and does not require human participation in the evaluation process, thus realizing automated effectiveness evaluation, overcoming the shortcomings of many human factors, strong subjectivity, long time consumption and high cost in the evaluation process.

Description

Technical Field

[0001] The present invention mainly relates to the field of simulation technology, and in particular to a combat system architecture effectiveness evaluation method, device and storage medium. Background Art

[0002] At present, system architecture combat is an important theoretical basis for the training, exercises and operations of major military powers in the world. System architecture combat is to use the Internet to connect the command and combat system, intelligence collection system and weapon equipment system into a whole to form an efficient and unified combat system. Convert information advantage into combat action advantage. At present, in the task combat system architecture of the Air Force system, commanders directly score combat effectiveness based on combat experience. It is essentially an experience-driven effectiveness evaluation method with many human factors, strong subjectivity, long time consumption and high cost. With the application of high-tech in the military field, modern warfare has shown unprecedented complexity, the war situation is changing rapidly, and the battlefield data is growing massively, requiring higher speed and accuracy in effectiveness evaluation. The experience-driven method can no longer meet the requirements of modern warfare. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method, device and storage medium for evaluating the effectiveness of a combat system architecture in view of the deficiencies in the prior art.

[0004] The technical solution of the present invention to solve the above technical problem is as follows: A method for evaluating the effectiveness of a combat system architecture comprises the following steps:

[0005] Importing a combat mission effectiveness evaluation index data set, and performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and using all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed;

[0006] Perform predictive analysis on the data set to be processed to obtain an evaluation result of the effectiveness of the combat system architecture.

[0007] Another technical solution of the present invention to solve the above technical problem is as follows: a combat system architecture effectiveness evaluation device, comprising:

[0008] A data set partitioning module is used to import a combat mission effectiveness evaluation index data set, and perform data set partitioning analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and use all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed;

[0009] The evaluation result acquisition module is used to perform predictive analysis on the data set to be processed to obtain an evaluation result of the effectiveness of the combat system architecture.

[0010] Another technical solution of the present invention to solve the above-mentioned technical problem is as follows: A combat architecture effectiveness evaluation device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the combat architecture effectiveness evaluation method as described above is implemented.

[0011] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the combat architecture effectiveness evaluation method as described above is implemented.

[0012] The beneficial effects of the present invention are as follows: a plurality of target capability indicator data are obtained by data set division and analysis of a combat mission effectiveness evaluation indicator data set, and all target capability indicator data and a combat mission effectiveness evaluation indicator data set are used as a data set to be processed, and a combat system architecture effectiveness evaluation result is obtained by predictive analysis of the data set to be processed, which can greatly improve the training speed and prediction accuracy, and does not require human participation in the evaluation process, thereby realizing automated effectiveness evaluation and overcoming the shortcomings of many human factors, strong subjectivity, long time consumption and high cost in the evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flowchart of a method for evaluating the effectiveness of a combat system architecture provided by an embodiment of the present invention;

[0014] Figure 2 A diagram of the division process of a combat mission effectiveness evaluation index data set provided by an embodiment of the present invention;

[0015] Figure 3 A schematic diagram of a process for performing predictive analysis on a data set to be processed provided by an embodiment of the present invention;

[0016] Figure 4 A module block diagram of a combat system architecture effectiveness evaluation device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0018] Figure 1 A flowchart of a method for evaluating the effectiveness of a combat system architecture provided in an embodiment of the present invention.

[0019] like Figure 1 As shown, a method for evaluating the effectiveness of a combat system architecture comprises the following steps:

[0020] Importing a combat mission effectiveness evaluation index data set, and performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and using all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed;

[0021] Perform predictive analysis on the data set to be processed to obtain an evaluation result of the effectiveness of the combat system architecture.

[0022] It should be understood that the purpose of analyzing the data set division of the combat mission effectiveness evaluation index data set is to construct an evaluation index system.

[0023] In the above embodiment, multiple target capability indicator data are obtained by dividing and analyzing the data set of the combat mission effectiveness evaluation index data set, and all the target capability indicator data and the combat mission effectiveness evaluation index data set are used as the data set to be processed. The combat system architecture effectiveness evaluation result is obtained by predictive analysis of the data set to be processed, which can greatly improve the training speed and prediction accuracy. No human participation is required in the evaluation process, and automated effectiveness evaluation is realized, overcoming the shortcomings of many human factors, strong subjectivity, long time consumption and high cost in the evaluation process.

[0024] Optionally, as an embodiment of the present invention, before importing the combat mission effectiveness evaluation index data set, the method further includes:

[0025] Import combat mission information, and model the combat mission information based on the DoDAF combat architecture model to obtain an overview and summary information diagram, a high-level combat concept diagram, a capability concept view, and a capability decomposition view, and obtain a combat mission effectiveness evaluation indicator data set through the overview and summary information diagram, the high-level combat concept diagram, the capability concept view, and the capability decomposition view.

[0026] It should be understood that the DoDAF operational architecture model is a system architecture specification developed by the U.S. Department of Defense based on the C4ISR architecture framework. The principles and applicability of its architecture have been extended to all joint capability domains, not just the C4ISR field. It not only defines the method of describing the architecture, but also provides a general guide for understanding, using, comparing, integrating and interoperating the architecture - the guiding principles and data model specifications for developing and describing the architecture, and gradually becomes the modeling standard for architecture design and requirements analysis description. DoDAF version 2.02 is the top-level, overall framework and conceptual model for architecture development. It embodies the "data-centric" system modeling idea and emphasizes the use of architecture data to support analysis and decision-making. A visual form view model for architecture description is defined in this framework. The view model includes All Viewpoint (AV), Capability Viewpoint (CV), Data and Information Viewpoint (DIV), Operational Viewpoint (OV), Project Viewpoint (PV), Services Viewpoint (SvcV), Standards Viewpoint (StdV) and System Viewpoint (SV). Each view consists of a series of selected architecture data, which are visualized in dashboard, fusion, text, synthesis or diagram form, representing the data collected and information derived during the development process.

[0027] It should be understood that the combat mission effectiveness evaluation indicator data set is extracted manually from the overview and summary information diagram, the high-level combat concept diagram, the capability concept view, and the capability decomposition view.

[0028] Specifically, according to the assigned combat mission (i.e., the combat mission information) and the DoDAF model (i.e., the DoDAF combat architecture model), the modeling overview and summary information AV-1 diagram (i.e., the overview and summary information diagram) is constructed. According to the text description formed by the air combat concept modeling, the combat scenario task is described in detail by focusing on the specific contents of the comprehensive combat elements such as the combat area, combat environment, combat time, threat target, and mission tasks generated by combat decisions.

[0029] According to the assigned combat mission (i.e. the combat mission information) and the DoDAF model (i.e. the DoDAF combat architecture model), the high-level combat concept diagram OV-1 (i.e. the high-level combat concept diagram) is modeled. Based on the combat concept modeling results, the overall background description of AV-1 combat and the outline of the architecture are the overall design of the air combat system under the air combat concept.

[0030] Based on the assigned combat mission (i.e., the combat mission information) and the DoDAF model (i.e., the DoDAF combat architecture model), the capability concept view CV-1 diagram (i.e., the capability concept view) is modeled to describe the strategic background and high-level scope of the system combat capability.

[0031] According to the assigned combat mission (i.e., the combat mission information) and the DoDAF model (i.e., the DoDAF combat architecture model), a capability decomposition view CV-2 diagram (i.e., the capability decomposition view) is modeled. Based on the capability scope, the capability is decomposed layer by layer to obtain capability elements, i.e., the underlying sub-capabilities, and a capability structure tree is established to describe the hierarchy and composition relationship of the capability.

[0032] Specifically, the present invention selects fighters and their combat systems as objects, and designs an air combat system structure to realize the construction of an air combat system task model by studying the combat aircraft support of fighters, single-machine penetration into the depth of the enemy's front line, using the attack advantages brought by stealth, supersonic speed and other capabilities, destroying the enemy escort formation while carrying out long-range air-to-air missile strikes on the enemy's early warning aircraft, and realizing the construction of an air combat system task model. First, the AV-1 model (i.e., the overview and summary information diagram) is mainly based on the text description formed by the concept modeling of air combat combat missions, and by emphasizing the specific contents of the comprehensive combat elements such as the mission tasks generated by the combat area, combat environment, combat time, threat targets and combat decisions, the combat assumption tasks are described in detail, such as breaking through the enemy's borders, including breaking through the enemy's border description, purpose, combat mode and task completion standard. Then the OV-1 model (i.e., the advanced combat concept diagram) gives the solution of the air combat assumption from the task execution level with a high-level graphic, including the interaction of each combat unit of the air combat system and the information data between each unit. After our early warning system captures enemy intelligence, the ground command center and the early warning aircraft conduct coordinated mission planning and task allocation; the early warning aircraft obtains command and control of the fighter jets, conducts situational awareness, route planning and penetration guidance; after breaking through the enemy's borders, the fighter jets identify, track, and lock onto the enemy early warning aircraft and escort formations to conduct beyond-visual-range strikes and other tasks.

[0033] With the help of air combat mission concept analysis, AV-1 model (i.e., the overview and summary information diagram) and OV-1 model (i.e., the advanced combat concept diagram), the CV-1 model (i.e., the capability concept view) is established, and the system architecture life cycle is divided into seven system development stages: ground preparation, takeoff formation, guidance flight, airspace penetration, detection and identification, over-the-horizon attack and return. The capability requirements of each stage are summarized and summarized, and eight secondary capabilities of mission planning, command and control, communication, target perception, fire strike, defense, support and mobility are obtained, and the attribution relationship between the stages and capabilities is established. Then, the CV-2 model (i.e., the capability decomposition view) is used to decompose the capabilities layer by layer from the scope of capabilities, and the capability elements, i.e., the underlying sub-capabilities, are obtained. A capability structure tree is established to describe the hierarchy and composition relationship of the capabilities. Taking the target perception capability as an example, the raid fighter can search for targets independently or jointly, establish target tracking based on the search signal, and meet the situation requirements. Therefore, the first-level target perception combat capability is subdivided into three second-level capabilities: threat warning, target identification, and target positioning, and then target identification is divided into three third-level capabilities: visible light identification, radar identification, and infrared identification. This provides capability indicator selection for building an evaluation indicator system in subsequent steps.

[0034] In the above embodiment, the combat mission information is modeled based on the DoDAF combat architecture model to obtain a combat mission effectiveness evaluation indicator data set, which provides the selection of capability indicators for subsequent data processing, realizes automated effectiveness evaluation, and overcomes the shortcomings of many human factors, strong subjectivity, long time consumption and high cost in the evaluation process.

[0035] Optionally, as an embodiment of the present invention, Figures 1 to 2 As shown, the process of performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain multiple target capability index data includes:

[0036] Performing a first division on the combat mission effectiveness evaluation indicator data set to obtain a plurality of first capability indicator data;

[0037] Numbering the plurality of first capability indicator data according to the order of the first division to obtain a first capability indicator number corresponding to each of the first capability indicator data;

[0038] Performing a second division on each of the first capability indicator data to obtain a plurality of second capability indicator data corresponding to each of the first capability indicator data;

[0039] Performing a third division on each of the second capability indicator data respectively to obtain a plurality of third capability indicator data corresponding to each of the second capability indicator data;

[0040] Renumbering the plurality of third capability indicator data according to the order of the third division to obtain a second capability indicator number corresponding to each of the third capability indicator data;

[0041] Normalization analysis is performed on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the third capability indicator data to obtain target capability indicator data corresponding to each of the third capability indicator data.

[0042] It should be understood that Figure 2 As shown, first, a hierarchical evaluation index set V is established, and the effectiveness layer has only one indicator, the overall effectiveness E of the air combat mission (i.e., the combat mission effectiveness evaluation index data set), and the value of E represents the overall effectiveness of the air combat mission. Next, the capability layer divides the overall effectiveness of the system (i.e., the combat mission effectiveness evaluation index data set) into three unrelated capabilities (i.e., the first capability indicator data), including detection and tracking capability, killing capability, and survivability. Then, the indicator layer matches the various capabilities of the capability layer with the corresponding indicators in the indicator set, including mission planning capability, command and control capability, communication capability, target perception capability, firepower strike capability, defense capability, comprehensive support capability, and mobility capability (i.e., multiple second capability indicator data). Finally, all qualitative or quantitative indicators in the indicator set V (i.e., multiple third capability indicator data) are normalized and quantified to obtain the corresponding capability values ​​(i.e., the target capability indicator data).

[0043] Specifically, the hierarchical analysis method is used to establish an evaluation index system diagram, which is divided into four levels. According to the principle of bottom-up and hierarchical integration, starting from the bottom-level quantitative indicators, the calculation model is integrated upward layer by layer to finally obtain the overall capability of the air combat mission (i.e., the combat mission effectiveness evaluation index data set). According to the air combat mission effectiveness evaluation index system, a hierarchical evaluation index set V is established, in which the effectiveness layer has only one indicator, the overall effectiveness of the air combat mission E (i.e., the combat mission effectiveness evaluation index data set), and the value of E (i.e., the combat mission effectiveness evaluation index data set) represents the overall effectiveness of the air combat mission; the capability layer divides the overall effectiveness of the system into three unrelated capabilities Ai (i=1, 2, 3) (i.e., the first capability indicator data), and E (i.e., the combat mission effectiveness evaluation index data set) can be calculated by Ai (i.e., the first capability indicator data), that is, E=f E (A1, A2, A3), where f E A i(i.e., the first capability indicator data) to E (i.e., the combat mission effectiveness evaluation indicator data set); the indicator layer corresponds the various capabilities of the capability layer to the corresponding indicators in the indicator set, and any capability A i (i.e., the first capability indicator data) can be calculated from one or more indicators, i.e., A i =f i (v i,1 ,v i,2 ,…,v i,j ), where v i,j (i=1,2,3)(j=1,2,…,N i ) is the influence ability A i performance index (i.e., the second capability index data), N i For and ability A i The number of relevant performance indicators, f i v i,j To A i The mapping function of .

[0044] In the above embodiment, the data set division and analysis of the combat mission effectiveness evaluation index data set obtains multiple target capability index data, which can obtain more accurate data, provide guarantee for subsequent data processing, and greatly improve the training speed and prediction accuracy.

[0045] Optionally, as an embodiment of the present invention, the process of performing normalization analysis on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the third capability indicator data to obtain the target capability indicator data corresponding to each of the third capability indicator data includes:

[0046] Determine whether each of the first capability indicator number and the second capability indicator number meets a condition, wherein the condition includes that the first capability indicator number is equal to any one of a plurality of preset first capability indicator numbers, and the second capability indicator number is equal to any one of a plurality of preset second capability indicator numbers;

[0047] If the condition is satisfied, the third capability indicator data is normalized and calculated by the first formula to obtain fifth capability indicator data corresponding to the third capability indicator data, wherein the first formula is:

[0048]

[0049] Among them, v′ k is the fifth capability indicator data, v k is the third capability indicator data, as the indicator rating value, and taking the fifth capability indicator data as the target capability indicator data;

[0050] If not, the third capability indicator data is normalized and calculated by the second formula to obtain sixth capability indicator data corresponding to the third capability indicator data, wherein the second formula is:

[0051]

[0052] Among them, v″ k is the sixth capability indicator data, v k is the third capability indicator data, is the indicator rated value, and the sixth capability indicator data is used as the target capability indicator data.

[0053] Preferably, the preset first capability indicator number may be 1 or 2, and the preset second capability indicator number may be 1 or 2 or 3.

[0054] It should be understood that before the performance evaluation, all qualitative or quantitative indicators in the indicator set V (i.e., the plurality of third capability indicator data) need to be normalized and quantified, that is, any indicator v in V (i.e., the plurality of third capability indicator data) is selected. k (k=1,2,…,K) (i.e., the third capability indicator data), if v k (i.e. the third capability indicator data) is a quantitative indicator, and the indicator v k (i.e. the third capability indicator data) and its corresponding capability A i (i.e., the change relationship of the first capability indicator data).

[0055] Specifically, if with v k (i.e., the third capability indicator data) increases, A i (i.e., the first capability indicator data) also increases accordingly (i.e., the first capability indicator number is equal to any one of a plurality of preset first capability indicator numbers, and the second capability indicator number is equal to any one of a plurality of preset second capability indicator numbers), then v k The normalized value (i.e., the fifth capability indicator data) is as follows:

[0056]

[0057] in, Indicators required to meet the communication requirements for air combat missionsv k Rated value;

[0058] If with v k (i.e., the third capability indicator data) increases, A i(i.e., the first capability indicator data) remains unchanged or decreases accordingly (i.e., the first capability indicator number is not equal to any one of a plurality of preset first capability indicator numbers, or the second capability indicator number is not equal to any one of a plurality of preset second capability indicator numbers), then:

[0059]

[0060] If v k (i.e. the third capability indicator data) is a qualitative indicator. k (i.e. the third capability indicator data) is fuzzy processed to obtain v k The level of the third capability indicator data in the evaluation set is mapped to a value between 0 and 1, which is v k (i.e. the third capability index data) normalized and quantized index value v′ k .

[0061] In the above embodiment, the target capability indicator data is obtained by normalizing and analyzing each third capability indicator data, the second capability indicator number and the first capability indicator number respectively, and targeted processing can be performed according to different data, thereby improving the accuracy of the data, providing guarantee for subsequent data processing, and greatly improving the training speed and prediction accuracy.

[0062] Optionally, as an embodiment of the present invention, Figure 1 and 3 As shown, the process of performing predictive analysis on the data set to be processed to obtain the combat system architecture effectiveness evaluation result includes:

[0063] Dividing the data set to be processed according to a preset ratio to obtain a training set, a validation set and a test set;

[0064] Dividing the training set according to a preset number of divisions to obtain a plurality of training subsets;

[0065] Constructing a stochastic gradient descent model, importing the validation set, the test set, and the plurality of training subsets into the stochastic gradient descent model, and training the stochastic gradient descent model using a five-fold cross validation method to obtain a first prediction value and a second prediction value corresponding to each of the training subsets;

[0066] Constructing a support vector regression model, importing the validation set, the test set, and the plurality of training subsets into the support vector regression model, and training the support vector regression model using a five-fold cross validation method to obtain a third prediction value and a fourth prediction value corresponding to each of the training subsets;

[0067] Constructing a random forest regression model, importing the validation set, the test set, and the plurality of training subsets into the random forest regression model, and training the random forest regression model using a five-fold cross validation method to obtain a fifth prediction value and a sixth prediction value corresponding to each of the training subsets;

[0068] Constructing a linear regression model, and importing the plurality of the second prediction values, the plurality of the fourth prediction values, and the plurality of the sixth prediction values ​​into the linear regression model for training to obtain a trained model;

[0069] The first predicted value, the third predicted value and the fifth predicted value are predicted by the trained model to obtain a combat system architecture effectiveness evaluation result.

[0070] Preferably, the preset ratio may be 6:2:2, and the preset number of divisions may be 5.

[0071] It should be understood that the stochastic gradient descent model, namely the stochastic gradient descent (SGD) model, is an optimization algorithm that uses an iterative method to solve the objective function to obtain the optimal solution. It uses gradient iteration to find the local optimal solution based on the least squares cost function. The corresponding update formula is:

[0072]

[0073] In terms of training speed, the stochastic gradient descent method only uses one sample to iterate at a time, so the training speed is very fast, while the batch gradient descent method cannot train satisfactorily when the sample size is large. In terms of accuracy, the stochastic gradient descent method uses only one sample to determine the gradient direction, which may result in a non-optimal solution. In terms of convergence speed, the stochastic gradient descent method iterates one sample at a time, resulting in a large change in the iteration direction, and cannot quickly converge to the local optimal solution.

[0074] Specifically, the support vector regression model, namely support vector regression (SVR), is an implementation of support vector machine (SVM). Similar to the traditional regression model, its basic idea is to solve an optimal hyperplane so that the error from all sample points to the hyperplane is minimized. The SVR model has certain advantages in solving small sample high-dimensional problems. When constructing the SVR model, the selection of different kernel functions will affect the accuracy of the model. Generally, there are two ways to select the kernel function: one is to select the kernel function based on the prior knowledge when there is a certain prior knowledge of the characteristics of the data set; the other is to try different kernel functions and choose the kernel function with the smallest error. When solving different problems, the characteristics of each data set are quite different. Therefore, this paper adopts the second method to establish the SVR model with linear kernel function, polynomial kernel function, RBF kernel function and Sigmoid kernel function respectively, select the kernel function with the best R index, and establish the SVR primitive model.

[0075] It should be understood that the random forest regression model, i.e., random forest regression (RF), is an integrated learning technique that aggregates the results of multiple decorrelated decision trees collected in the forest and outputs classification results. Each decision tree of the extremely random tree is constructed from the original training sample. At each test node, each tree has a random sample with k features in the sample. Each decision tree must select the best feature from these feature sets and then split the data according to some mathematical indicators (generally the Gini index). This random feature sample leads to the generation of multiple unrelated decision trees. In the process of building the forest, for each feature, the normalized total reduction of the mathematical indicator (such as using the Gini index) used to split the feature decision is calculated. This value is called the importance of the Gini factor. After the Gini importance is arranged in descending order, the first k features can be selected as needed.

[0076] It should be understood that the linear regression model is linear regression, which is a statistical analysis method that uses regression analysis in mathematical statistics to determine the quantitative relationship between two or more variables that are interdependent. It is widely used. Its expression is y=w'x+e, where e is the error that follows a normal distribution with a mean of 0.

[0077] It should be understood that the five-fold cross validation method is a cross validation method, which is to first divide the data set into k mutually exclusive subsets of similar size, and each subset keeps the consistency of data distribution as much as possible, that is, it is obtained from the data set through stratified sampling. Then, each time the union of k-1 subsets is used as the training set, and the remaining subset is used as the test set, so that k groups of training sets / test sets can be obtained, so that k training and testing can be performed, and the mean of these k test results is finally returned. Obviously, the stability and fidelity of the evaluation results of the cross validation method depend to a large extent on the value of k. To emphasize this point, the cross validation method is usually referred to as "k-fold cross validation". In the present invention, the k value is set to 5, so it is a five-fold cross validation. Cross validation method; cross validation method can also be understood as a method of randomly extracting a part of the data from a set of measured data to establish a model, and using the remaining data to test this model. For example, in the artificial neural network method, a part of the data is randomly extracted from a set of measured data to train the network and obtain weight coefficients. Predictions are made based on the obtained weight coefficients, and the difference between the predicted value and the experimental value is used as an evaluation parameter to verify the established network model. This part of the data is one kth of the training set, which is the k-fold cross-validation. The k value taken by the present invention is 5.

[0078] It should be understood that the five-fold cross validation method is a cross validation method with a k value of 5, which can be understood as:

[0079] 1. When training a model, if the hyperparameters have been pre-specified, the model trained by k-cross validation is just a model of the same structure with different parameters trained by different data. Some articles pre-specify hyperparameters, and using k-cross validation can only simply indicate that the accuracy of the model is like this under this set of hyperparameters, and it cannot indicate that the current set of hyperparameters is better than other ones. 2. The usage of k-cross validation is to perform k-cross validation training models for the n sets of hyperparameters you want to try, and then compare the n average errors obtained by the k-cross validation method under the n sets of hyperparameters, and then select the set of hyperparameters with smaller errors. 3. When doing experiments, for example, to compare the effects of different learning algorithms, you must first use the k-cross validation method to determine a set of better hyperparameters for different algorithms, and then compare their accuracies on the test set.

[0080] It should be understood that the steps of the five-fold cross validation method, i.e., the cross validation method with a k value of 5, are:

[0081] 1. Randomly divide the entire dataset into K “layers”

[0082] 2. For each portion:

[0083] 1). Use this set as the test set and the rest as the training set; (use the K-1 layers to train the model, and then use the Kth layer for verification)

[0084] 2). Get the model on the training set;

[0085] 3). Get the generated error on the test set, so that there is a prediction result for each piece of data; (record the error obtained from each prediction result)

[0086] 3. The average of the k recorded errors is called the cross-validation error. It can be used as a measure of model performance.

[0087] 4. Choose the model with the smallest error.

[0088] It should be understood that the secondary learner greatly affects the generalization ability of Stacking ensemble learning. The input features of the secondary learner, that is, the output features of the primary primitive model, are very close to the true labels of the data. Using a simple model as the secondary learner can reduce the risk of overfitting.

[0089] It should be understood that linear regression (i.e., the linear regression model) is used as the algorithm of the secondary learner. The linear regression model is the most basic regression model with low model complexity. It fits the data set through continuous straight lines, planes, or hyperplanes to minimize the residual of the training data set, and can efficiently solve the problem of linear correlation characteristics between data. Therefore, the linear regression model is selected to establish a secondary learner model, and the above three primitive models (i.e., the stochastic gradient descent model, the support vector regression model, and the random forest regression model) are combined.

[0090] It should be understood that, first, the selection and construction of the primitive model needs to consider the accuracy and diversity of each model at the same time, mainly using the meta-models such as the stochastic gradient descent (SGD) model, the support vector regression (SVR) model, and the random forest regression (RF) model; secondly, the linear regression model is selected to establish the secondary learner model, and the above three primitive models (i.e., the stochastic gradient descent model, the support vector regression model, and the random forest regression model) are combined; finally, all the trained base models are used to predict the entire training set, and the first layer is used as the base classifier of the Stacking algorithm by training multiple classifiers, and the output of the second layer is used as the input of the meta-classifier of the second layer to train the Stacking model. Similarly, the prediction process must first form a new test set (i.e., the first predicted value, the third predicted value, and the fifth predicted value) through the prediction of all base models, and finally the test set (i.e., the first predicted value, the third predicted value, and the fifth predicted value) is predicted.

[0091] It should be understood that the data set (ie, the data set to be processed) contains a total of 2000 samples, each of which has 22 measured network indicators, including 8 indicators of detection and tracking capability, 6 indicators of killing capability, and 8 indicators of survivability. The training data is organized in a common format.

[0092] Specifically, Figure 3 As shown, in the first step, the data set (i.e., the data set to be processed) is divided into a training set, a validation set, and a test set in a ratio of 6:2:2. The training set is divided into 5 parts (i.e., multiple training subsets), namely T1, T2, T3, T4, and T5.

[0093] In the second step, we use the validation sets T1, T2, T3, T4, and T5 in turn, and the remaining four sets as training sets, and perform 5-fold cross validation to train the model; then we perform predictions on the test set. We get the 5 prediction values ​​P trained by the SGD model. 1i (i=1, 2, 3, 4, 5) (i.e., the second predicted value) and a predicted value B1 on the test set (i.e., the first predicted value), merge these five vertically overlapping copies to obtain a new training set A1, repeat the second step to train the SVR model and the RF model to obtain a new training set A2 (i.e., multiple fourth predicted values), A3 (i.e., multiple sixth predicted values) and predicted values ​​B2 (i.e., the third predicted value) and B3 (i.e., the fifth predicted value) on the test set.

[0094] In the third step, the predicted values ​​obtained after the three base models are trained are used as three new features (A1, A2, A3), and the LR model is used for training to establish the LR model (i.e., the trained model).

[0095] The fourth step is to use the trained LR model (i.e., the trained model) to predict the three new prediction values ​​(B1, B2, B3) (i.e., the first prediction value, the third prediction value, and the fifth prediction value) constructed by the prediction values ​​of the three base models on the test set, and obtain the final prediction effectiveness evaluation value (i.e., the effectiveness evaluation result of the combat system architecture).

[0096] In the above embodiment, the predictive analysis of the data set to be processed obtains the combat system architecture effectiveness evaluation result, reduces the risk of overfitting and the complexity of the model, minimizes the residual of the training data set, and can efficiently solve the problem of linear correlation characteristics between data.

[0097] Optionally, as another embodiment of the present invention, a combat system architecture effectiveness evaluation method of the present invention targets new features of the air force system mission combat system architecture and existing problems in the research, takes the air force system mission combat system as the research object, adopts DoDAF standardization to describe the air force system mission combat system and its combat command process, and introduces an integrated learning method to construct an evaluation model. The evaluation process does not require human participation, and related parameters can be automatically optimized, which greatly improves the training speed and prediction accuracy, and ultimately realizes automatic performance evaluation.

[0098] Figure 4 A module block diagram of a combat system architecture effectiveness evaluation device provided by an embodiment of the present invention.

[0099] Optionally, as another embodiment of the present invention, Figure 4 As shown, a combat system structure effectiveness evaluation device comprises:

[0100] A data set partitioning module is used to import a combat mission effectiveness evaluation index data set, and perform data set partitioning analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and use all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed;

[0101] The evaluation result acquisition module is used to perform predictive analysis on the data set to be processed to obtain an evaluation result of the effectiveness of the combat system architecture.

[0102] Optionally, as an embodiment of the present invention, a modeling and analysis module is further included before the data set division module, and the modeling and analysis module is specifically used to:

[0103] Import combat mission information, and model the combat mission information based on the DoDAF combat architecture model to obtain an overview and summary information diagram, a high-level combat concept diagram, a capability concept view, and a capability decomposition view, and obtain a combat mission effectiveness evaluation indicator data set through the overview and summary information diagram, the high-level combat concept diagram, the capability concept view, and the capability decomposition view.

[0104] Optionally, as an embodiment of the present invention, the data set partitioning module is specifically used for:

[0105] Performing a first division on the combat mission effectiveness evaluation indicator data set to obtain a plurality of first capability indicator data;

[0106] Numbering the plurality of first capability indicator data according to the order of the first division to obtain a first capability indicator number corresponding to each of the first capability indicator data;

[0107] Performing a second division on each of the first capability indicator data to obtain a plurality of second capability indicator data corresponding to each of the first capability indicator data;

[0108] Performing a third division on each of the second capability indicator data respectively to obtain a plurality of third capability indicator data corresponding to each of the second capability indicator data;

[0109] Renumbering the plurality of third capability indicator data according to the order of the third division to obtain a second capability indicator number corresponding to each of the third capability indicator data;

[0110] Normalization analysis is performed on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the third capability indicator data to obtain target capability indicator data corresponding to each of the third capability indicator data.

[0111] Optionally, another embodiment of the present invention provides a combat architecture effectiveness evaluation device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the combat architecture effectiveness evaluation method described above is implemented. The device may be a computer or other device.

[0112] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the combat architecture effectiveness evaluation method as described above is implemented.

[0113] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0118] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating the effectiveness of a combat system architecture, characterized in that: The steps include: Importing a combat mission effectiveness evaluation index data set, and performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and using all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed; Performing predictive analysis on the data set to be processed to obtain an operational system architecture effectiveness evaluation result; The process of performing data set division analysis on the combat mission effectiveness evaluation index data set to obtain multiple target capability index data includes: Performing a first division on the combat mission effectiveness evaluation indicator data set to obtain a plurality of first capability indicator data; Numbering the plurality of first capability indicator data according to the order of the first division to obtain a first capability indicator number corresponding to each of the first capability indicator data; Performing a second division on each of the first capability indicator data to obtain a plurality of second capability indicator data corresponding to each of the first capability indicator data; Performing a third division on each of the second capability indicator data respectively to obtain a plurality of third capability indicator data corresponding to each of the second capability indicator data; Renumbering the plurality of third capability indicator data according to the order of the third division to obtain a second capability indicator number corresponding to each of the third capability indicator data; Performing normalization analysis on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the first capability indicator data, respectively, to obtain target capability indicator data corresponding to each of the third capability indicator data; The process of performing normalization analysis on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the first capability indicator data to obtain the target capability indicator data corresponding to each of the third capability indicator data comprises: Determine whether each of the first capability indicator number and the second capability indicator number meets a condition, wherein the condition includes that the first capability indicator number is equal to any one of a plurality of preset first capability indicator numbers, and the second capability indicator number is equal to any one of a plurality of preset second capability indicator numbers; If the condition is satisfied, the third capability indicator data is normalized and calculated by the first formula to obtain fifth capability indicator data corresponding to the third capability indicator data, wherein the first formula is: Among them, v′ k is the fifth capability indicator data, v k is the third capability indicator data, as the indicator rating value, and taking the fifth capability indicator data as the target capability indicator data; If not, the third capability indicator data is normalized and calculated by the second formula to obtain sixth capability indicator data corresponding to the third capability indicator data, wherein the second formula is: Among them, v″ k is the sixth capability indicator data, v k is the third capability indicator data, as the indicator rating value, and taking the sixth capability indicator data as the target capability indicator data; The process of performing predictive analysis on the data set to be processed to obtain the combat system architecture effectiveness evaluation result comprises: Dividing the data set to be processed according to a preset ratio to obtain a training set, a validation set and a test set; Dividing the training set according to a preset number of divisions to obtain a plurality of training subsets; Constructing a stochastic gradient descent model, importing the validation set, the test set, and the plurality of training subsets into the stochastic gradient descent model, and training the stochastic gradient descent model using a five-fold cross validation method to obtain a first prediction value and a second prediction value corresponding to each of the training subsets; Constructing a support vector regression model, importing the validation set, the test set, and the plurality of training subsets into the support vector regression model, and training the support vector regression model using a five-fold cross validation method to obtain a third prediction value and a fourth prediction value corresponding to each of the training subsets; Constructing a random forest regression model, importing the validation set, the test set, and the plurality of training subsets into the random forest regression model, and training the random forest regression model using a five-fold cross validation method to obtain a fifth prediction value and a sixth prediction value corresponding to each of the training subsets; Constructing a linear regression model, and importing the plurality of the second prediction values, the plurality of the fourth prediction values, and the plurality of the sixth prediction values ​​into the linear regression model for training to obtain a trained model; The first prediction value, the third prediction value and the fifth prediction value are predicted by the trained model to obtain a combat system architecture effectiveness evaluation result.

2. The method for evaluating the effectiveness of a combat system architecture according to claim 1, characterized in that: Before importing the combat mission effectiveness evaluation indicator dataset, it also includes: Import combat mission information, and model the combat mission information based on the DoDAF combat architecture model to obtain an overview and summary information diagram, a high-level combat concept diagram, a capability concept view, and a capability decomposition view, and obtain a combat mission effectiveness evaluation indicator data set through the overview and summary information diagram, the high-level combat concept diagram, the capability concept view, and the capability decomposition view.

3. A combat system structure effectiveness evaluation device, characterized in that: include: A data set partitioning module is used to import a combat mission effectiveness evaluation index data set, and perform data set partitioning analysis on the combat mission effectiveness evaluation index data set to obtain a plurality of target capability index data, and use all the target capability index data and the combat mission effectiveness evaluation index data set as a data set to be processed; An evaluation result acquisition module is used to perform predictive analysis on the data set to be processed to obtain an evaluation result of the effectiveness of the combat system architecture; The data set partitioning module is specifically used for: Performing a first division on the combat mission effectiveness evaluation indicator data set to obtain a plurality of first capability indicator data; Numbering the plurality of first capability indicator data according to the order of the first division to obtain a first capability indicator number corresponding to each of the first capability indicator data; Performing a second division on each of the first capability indicator data to obtain a plurality of second capability indicator data corresponding to each of the first capability indicator data; Performing a third division on each of the second capability indicator data respectively to obtain a plurality of third capability indicator data corresponding to each of the second capability indicator data; Renumbering the plurality of third capability indicator data according to the order of the third division to obtain a second capability indicator number corresponding to each of the third capability indicator data; Performing normalization analysis on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the first capability indicator data, respectively, to obtain target capability indicator data corresponding to each of the third capability indicator data; In the data set partitioning module, respectively performing normalization analysis on each of the third capability indicator data, the second capability indicator number corresponding to each of the third capability indicator data, and the first capability indicator number corresponding to each of the first capability indicator data to obtain the target capability indicator data corresponding to each of the third capability indicator data includes: Determine whether each of the first capability indicator number and the second capability indicator number meets a condition, wherein the condition includes that the first capability indicator number is equal to any one of a plurality of preset first capability indicator numbers, and the second capability indicator number is equal to any one of a plurality of preset second capability indicator numbers; If the condition is satisfied, the third capability indicator data is normalized and calculated by the first formula to obtain fifth capability indicator data corresponding to the third capability indicator data, wherein the first formula is: Among them, v′ k is the fifth capability indicator data, v k is the third capability indicator data, as the indicator rating value, and taking the fifth capability indicator data as the target capability indicator data; If not, the third capability indicator data is normalized and calculated by the second formula to obtain sixth capability indicator data corresponding to the third capability indicator data, wherein the second formula is: Among them, v″ k is the sixth capability indicator data, v k is the third capability indicator data, as the indicator rating value, and taking the sixth capability indicator data as the target capability indicator data; The evaluation result obtaining module is specifically used for: Dividing the data set to be processed according to a preset ratio to obtain a training set, a validation set and a test set; Dividing the training set according to a preset number of divisions to obtain a plurality of training subsets; Constructing a stochastic gradient descent model, importing the validation set, the test set, and the plurality of training subsets into the stochastic gradient descent model, and training the stochastic gradient descent model using a five-fold cross validation method to obtain a first prediction value and a second prediction value corresponding to each of the training subsets; Constructing a support vector regression model, importing the validation set, the test set, and the plurality of training subsets into the support vector regression model, and training the support vector regression model using a five-fold cross validation method to obtain a third prediction value and a fourth prediction value corresponding to each of the training subsets; Constructing a random forest regression model, importing the validation set, the test set, and the plurality of training subsets into the random forest regression model, and training the random forest regression model using a five-fold cross validation method to obtain a fifth prediction value and a sixth prediction value corresponding to each of the training subsets; Constructing a linear regression model, and importing the plurality of the second prediction values, the plurality of the fourth prediction values, and the plurality of the sixth prediction values ​​into the linear regression model for training to obtain a trained model; The first prediction value, the third prediction value and the fifth prediction value are predicted by the trained model to obtain a combat system architecture effectiveness evaluation result.

4. The combat system architecture effectiveness evaluation device according to claim 3, characterized in that: A modeling and analysis module is also included before the data set division module, and the modeling and analysis module is specifically used to: Import combat mission information, and model the combat mission information based on the DoDAF combat architecture model to obtain an overview and summary information diagram, a high-level combat concept diagram, a capability concept view, and a capability decomposition view, and obtain a combat mission effectiveness evaluation indicator data set through the overview and summary information diagram, the high-level combat concept diagram, the capability concept view, and the capability decomposition view.

5. A combat architecture effectiveness evaluation system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the combat architecture effectiveness evaluation method according to any one of claims 1 to 2 is implemented.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the combat architecture effectiveness evaluation method according to any one of claims 1 to 2 is implemented.

Images