A Compatible Multi-Model Output Fusion Method Based on Set Operations

Abstract

The present invention discloses a compatible multi-model output fusion method based on set operations, and the method comprises the following steps: (1) Divide the data set. Randomly divide the data into a training set D T and a test set D V in a ratio of 4:1; (2) Construct sub-models. Select more than two machine learning methods and use the training data set D T to construct S sub-models; (3) Calculate weights. Calculate the weights according to the accuracy of the models and the similarity between the data and the models; (4) Multi-model output fusion and verification. According to the weights calculated in step (3) and the accuracy of each model, fuse the outputs of multiple models and verify on the test set. This method is highly compatible because the proposed method is applicable to any baseline method, different percentages of focus data, and even does not require model accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-model fusion, and specifically refers to a compatible multi-model output fusion method based on set operations. Background Art

[0002] For the modeling of complex systems, machine learning has become a widely used method. The model is trained using a training data set and then verified on a test data set. For example, a backpropagation neural network (BPNN) is used for complex spatio-temporal dynamic prediction, and a radial basis function (RBF) neural network is used for non-linear system modeling, etc. However, in most practical cases, the selected machine learning method is mainly based on availability rather than applicability to actual conditions. Considering that each machine learning method has its advantages and disadvantages, the performance of complex system modeling largely depends on the respective machine learning methods. An intuitive solution is to use multiple machine learning methods to build multiple models and then fuse them to produce the best results, namely multi-model fusion.

[0003] Traditional multi-model fusion is mainly divided into two types, namely (1) multi-model fusion fuses multiple sub-models into a unified model for generating an output for a given test input, and (2) multi-model output fusion fuses the multiple outputs of multiple models with a given test input. For the former multi-model fusion, the conventional methods are Bagging and Boosting in ensemble learning. For the latter multi-model output fusion, there are weight-based methods and non-weight-based methods when each model is attached with physical meanings. In comparison, multi-model fusion produces a unified model, while multi-model output fusion does not. In addition, multi-model fusion usually only uses one machine learning method as a baseline method to build multiple models. For example, a fuzzy neural network is used in Bagging, and Gaussian process regression is used in Boosting. Multi-model output fusion does not have such a requirement. We can combine the outputs of BPNN, RBF, etc. or variants of a single machine learning model to produce a more superior result. That is, multi-model output fusion can essentially solve the challenges caused by potentially poor machine learning methods.

[0004] At the level of multi-model output fusion, since no single model can guarantee the best results on all data, even a poor model may produce better results in some aspects of the data, and a better model does not necessarily produce better results on all data. That is, a poor model can make up for the deficiencies of a better model. How to use the limited outputs of poor models to make up for some poor outputs of better models is a challenging task. Traditionally, linear weights are used in multi-model output fusion. The advantage of linear weights is the convenience of acquisition and implementation.

[0005] However, there are the following three main disadvantages in assigning linear weights to the outputs generated by multiple models. First, the calculation of linear weights lacks a high degree of traceability. Second, the current methods of assigning weights are usually one-time, which means that the weights are the same for each set of data rather than being case-specific. Finally, linearly weighted combination of the outputs generated from multiple models produces intermediate values between the best and worst results of the original multiple models, which contradicts the goal of multi-model fusion, i.e., to produce the best results on the original multiple models. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a compatible multi-model output fusion method based on set operations. This method is highly compatible because the proposed method is applicable to any baseline method, different percentages of focus data, and even does not require model accuracy.

[0007] To solve the above technical problems, the technical solution of the present invention is as follows:

[0008] A compatible multi-model output fusion method based on set operations, the method comprising the following steps:

[0009] (1) Divide the data set. Randomly divide the data into a training set D T and a test set D V ;

[0010] (2) Construct sub-models. Select more than two machine learning methods and use the training data set D T to construct S sub-models;

[0011] (3) Calculate weights. Calculate the weights according to the accuracy of the models and the similarity between the data and the models;

[0012] (4) Multi-model output fusion and verification. According to the weights calculated in step (3) and the accuracy of each model, fuse the outputs of the multiple models and verify on the test set.

[0013] Preferably, step (3) is specifically:

[0014] (3.1) Calculate the accuracy of the s-th sub-model according to its result, and calculate the mean absolute percentage error MAPE s :

[0015]

[0016] where N is the size of the training data set, y n is the original output of the n-th group of training data, is the predicted output of the n-th group of training data in the s-th model;

[0017] The mean absolute percentage error MAPEs Used to calculate the accuracy r of the sub-model s :

[0018] r s = 1 - MAPE s (2)

[0019] (3.2) Determine the focus data list of the s-th model according to set operations;

[0020] Reorder the training data in ascending order according to the average error generated by the s-th model to form a new ordered list L s ;

[0021] L s = {d 1,s ,…,d p,s ,…,d P,s}} (3)

[0022] where e(d 1,s ) < … < e(d p,s ) < … < e(d P,s );

[0023] Select the initial focus data list with smaller errors related to the s-th model from L s

[0024] Determine the shared data set regarding the s-th model according to set operations

[0025]

[0026] where j ∈ S, s ≠ j;

[0027] Determine the final focus data list Λ s ;

[0028]

[0029] (3.3) Form a set of core data from the focus data list Λ s ;

[0030] Calculate the similarity between the core data of the m-th input factor regarding the s-th model and the q-th group of data

[0031]

[0032] where, and are the normalized versions of the m-th input factor of the q-th group of data and the core data respectively; ​

[0033] (3.4) Calculate the similarity between the q-th group of data and the s-th model according to the Euclidean distance

[0034]

[0035] wherein, Calculated from formula (5), θ m represents the weight of the m-th input factor. If there is no information on calculating the weight, it can be assumed that θ m = 1,

[0036] (3.5) Calculate the output allocation weight w of the s-th model with respect to the q-th group of data according to the model accuracy r s calculated in (3.1) and the similarity calculated in (3.4); q,s ;

[0037]

[0038] In addition, the allocation weight should be normalized, that is

[0039] Preferably, step (4) is specifically:

[0040] (4.1) Determine the multi-model output fusion for the q-th group of data:

[0041]

[0042] wherein, represents the output of the model with the highest accuracy;

[0043] (4.2) For the test data set containing Q groups of data, calculate the mean absolute error for verification:

[0044]

[0045] wherein, and respectively represent the actual output and the predicted output of the q-th group of test data.

[0046] The present invention has the following characteristics and beneficial effects:

[0047] Adopting the above technical solutions, (1) a process for calculating weights by comprehensively considering the accuracy and similarity of models is proposed. (2) The similarity is calculated based on set operations rather than the commonly used distance-based methods. (3) The proposed method is highly compatible because the proposed method is applicable to any baseline method, different percentages of focus data, and even does not require model accuracy.

[0048] The actual case of the overall reconnaissance ability evaluation of the unmanned aerial vehicle (UAV) swarm was studied. The sub-models were constructed using BPNN and RBF respectively for multi-model output fusion and a large number of comparative validations were carried out. The conclusions are as follows: (1) The result of the proposed multi-model output fusion method is better than each single model; (2) The result of the proposed method is better than equal weights, no model accuracy, and different percentages of the focus data list; (3) The proposed method has high compatibility under different multi-model output fusion conditions. Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0050] Figure 1 It is the overall framework diagram of the present invention.

[0051] Figure 2 It is the explanatory diagram of the weight calculation of the present invention.

[0052] Figure 3 It is the result diagram of the similarity, assigned weight, final output, and respective errors of the test data set of the present invention.

[0053] Figure 4 It is the explanatory diagram of the output fusion process of the present invention. Detailed Embodiments

[0054] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0055] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "a plurality" is two or more.

[0056] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0057] The present invention provides a compatible multi-model output fusion method based on set operations. First, the original data set is divided into a training set and a test set according to 4:1. Then, multiple sub-models are constructed with a backpropagation neural network (BPNN) and a radial basis function (RBF) neural network as benchmark models, the weights of each sub-model are calculated, and finally the outputs of the multi-model are fused and verified. The research results show that the result of the multi-model output fusion is better than that of each single model, and this advantage is also reflected in the comparative experimental results with methods such as equal weights, no model accuracy, and different percentages of focused data lists.

[0058] A compatible multi-model output fusion method based on set operations proposed in this embodiment includes four steps, and the overall framework of the method is as Figure 1 shown:

[0059] (1) Divide the data set. Randomly divide the data into a training set D T and a test set D V .

[0060] Specifically, in this embodiment, through a series of on-site nuclear simulation tests, a total of 500 groups of UAV cluster deployment data are collected. Randomly select 400 groups as the training data set, and the remaining 100 groups as the test data set. Each group of data contains 5 inputs (x1 : Detection coverage, x 2 : Detection altitude, x 3 : Navigation time, x 4 : Detection accuracy, x 5 : Recognition time) and an output y, i.e., the overall detection ability. Table 1 gives the upper bounds (ub) and lower bounds (lb) of the input-output parameters.

[0061] Table 1 Upper and lower bounds of the input feature parameters

[0062]

[0063] (2) Construct sub-models. Select more than two machine learning methods and use the training dataset D T to construct S sub-models.

[0064] In this example, two machine learning methods are used, namely the backpropagation neural network (BPNN) and the radial basis function (RBF). The parameter settings of both are as follows:

[0065] For BPNN, it is implemented using nntool in Matlab. The number of layers is 2, the number of nodes is [3, 3], the training function is trainlm, and the transfer function is tansig.

[0066] For RBF, its initial function is initlay, and the transfer functions are radbas and purelin.

[0067] (3) Calculate weights. Calculate the weights according to the accuracy of the model and the similarity between the data and the model.

[0068] Specifically, as Figure 2 shown,

[0069] (3.1), first calculate the MAPE of the two models:

[0070]

[0071] Furthermore, calculate the accuracy of the two models:

[0072]

[0073] where r s ∈[0, 1].

[0074] (3.2), according to the errors generated by the two models, sort the training dataset respectively to form an ordered number table L s ={d 1,s ,…, d p,s ,…, d P,s}, and their errors are in accordance with e(d 1,s)<…<e(d p,s )<…<e(d P,s Arrange in the order of, in this example, take L 1 and L 2 The first 10% of as the initial focus data list of the two models and Then the shared dataset If using and Construct three models, and the shared dataset of each model is different. For example In this example Finally, identify the focus data list Λ s , for example Indicates that the shared dataset needs to be excluded from the initial focus data list , otherwise it may cause great interference to the calculation of the similarity .

[0075] (3.3), first use the clustering method to form a set of core data from the focus data list Λ s , and then calculate the similarity between the core data of the m-th input factor of the two models and the q-th group of data It should be noted that the input factors and the core data should be normalized. Finally

[0076] (3.4), calculate the similarity regarding M attributes, where θ m Represents the weight of the m-th input factor. Usually, if there is no information on calculating the weight, it can be assumed that the weight θ m = 1, and the calculated

[0077] (3.5), calculate the output allocation weight w q,s of the s-th model regarding the q-th group of data. Similarly, w q,s ∈[0,1], because r s ∈[0,1] and In addition, the weights should be normalized, that is

[0078] (4) Multi-model output fusion and verification. According to the weights calculated in step (3) and the accuracies of each model, fuse the outputs of the multi-models and verify on the test set.

[0079] (4.1) Determine the multi-model output fusion regarding the q-th group of data:

[0080]

[0081] Among them, Represents the output of the model with the highest precision;

[0082] (4.2) For the test data set containing Q groups of data, calculate the mean absolute error MAE for verification:

[0083]

[0084] where, and respectively represent the actual output and predicted output of the q-th group of test data.

[0085] Specifically, as Figure 3 shown:

[0086] where (a) shows the similarity between 100 groups of test data and BPNN and RBF, and (b) shows the weights assigned to BPNN and RBF. Among the 100 groups of test data, 74 groups meet condition 1 of formula (9), that is, the output of the model with the highest precision has the highest weight among the S weights at this time, it is directly used as the fusion output For the remaining 26 groups of data that meet condition 2 of formula (9), the outputs of the fused BPNN and RBF are used as the final output. The final output and their respective errors are shown in (c) and (d). The MAEs of BPNN, RBF, and the method in this paper are 0.1335, 0.1530, and 0.1092 respectively. It can be seen that compared with directly using BPNN and RBF as the benchmark methods, the method proposed in this paper can reduce the MAE by 28.63% (=(0.1335 - 0.1092) / 0.1335) and 18.20% (=(0.1335 - 0.1092) / 0.1335) respectively.

[0087] Furthermore, in this embodiment, two groups of test data, namely the 16th group and the 59th group of data, will be used as examples to demonstrate the detailed fusion process to illustrate the proposed method.

[0088] 1. Calculate the precision of the two models:

[0089]

[0090] 2. Calculate the similarity between the test data and the two models. Specifically, the similarity between the 16th group of data and BPNN / RBF is 0.7368 / 0.5852, and the similarity between the 59th group of data and BPNN / RBF is 0.6616 / 0.7024.

[0091] 3. Calculate the weights between the test data and the two models:

[0092]

[0093] Further normalize the weights:

[0094]

[0095] 4. Perform multi-model output fusion, Figure 4 The fusion process of the 59th group of data is shown:

[0096] For the 16th group of data, there is w 16,BPNN > w 16,RBF， Considering that there is already r BPNN > r RBF , indicating that the prediction of the model BPNN for the 16th group of data is better. Therefore, no fusion is required, and the output of BPNN is the final fusion output. Specifically, the original output is The predicted output of BPNN is The error is e 16 = e 16,BPNN = 0.0155 (abs(2.0385 - 2.0230)). In contrast, the predicted output of RBF is The error is e 16,RBF = 0.0688 (abs(2.0918 - 2.0230)), which is much higher than e 16,BPNN = 0.0155. If their outputs are fused, then At this time, the error is e 16 = 0.0389 (abs(2.0619 - 2.0230)).

[0097] For the 59th group of data, there is w 59,BPNN < w 59,RBF , although r BPNN > r RBF , but according to the conditions, the outputs of BPNN and RBF need to be fused to generate the final output. Specifically, the original output is The predicted outputs of BPNN and RBF are respectively and The fused output of the two is y 59 = 0.4882×1.7250 + 0.5118×1.4870 = 1.6032, and the error is e 59 = 0.0273 (abs(1.6032 - 1.5759)), which is smaller than e 59,BPNN = 0.1491 and e 59,RBF = 0.0889.

[0098] The embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, without departing from the principle and spirit of the present invention, various changes, modifications, substitutions, and variations to these embodiments including components still fall within the protection scope of the present invention.

Claims

1. A compatible multi-model output fusion method based on set operations, characterized in that, the method is applied to the overall reconnaissance ability evaluation of an unmanned aerial vehicle swarm, and includes the following steps: S1. Collect a dataset and randomly divide the dataset into a training set D T and a test set D V , where each group of data in the dataset contains 5 inputs and one output. The 5 outputs include: detection coverage, detection altitude, navigation time, detection accuracy, and recognition time. The output is the overall detection ability; S2. Arbitrarily select at least two machine learning models, and use the training set D T to train the machine learning models, thereby obtaining S sub-models; S3. Calculate weights according to the accuracy of the sub-models and the similarity between the data input to the sub-models and the sub-models. Calculate the output allocation weight w of the s-th sub-model with respect to the q-th group of data q,s ; Among them, r s is the accuracy of the s-th sub-model, and is the similarity between the input data and the sub-model; The accuracy of the sub-model is calculated by the mean absolute percentage error MAPE s and is obtained by calculation; The similarity between the input data and the sub-model is calculated by the Euclidean distance algorithm, and the expression is as follows: where M is the total number of input factors in the dataset, is the similarity between the core data of the m-th input factor of the s-th model and the q-th group of data, and θ m represents the weight of the m-th input factor; The method for obtaining the core data used to calculate the similarity is: (1) Determine the focus data list of the s-th sub-model according to set operations; (2) Form a set of core data from the focus data list Λ according to the clustering algorithm s ; The method for obtaining the focus data list is: Re - sort the training data in ascending order according to the average error generated by the s - th model to form a new ordered list L s : L s = {d 1,s , …, d p,s , …, d P,s} where, e(d 1,s ) < … < e(d p,s ) < … < e(d P,s ); Select from L s a list of initial focus data with smaller errors related to the s-th model Determine the shared data set for the s-th model according to set operations where j ∈ S, s ≠ j; Determine the final list of focus data Λ s ; S4. According to the weights and accuracies of the various sub-models calculated in step S3, fuse the outputs of the S sub-models and verify on the test set.

2. The compatible multi-model output fusion method based on set operations according to claim 1, characterized in that, The training set D T and the test set D V are randomly divided according to a ratio of 4:

1.

3. The compatible multi-model output fusion method based on set operations according to claim 1, characterized in that, the two machine learning models selected in step S2 are a backpropagation neural network and a radial basis function.

4. The compatible multi-model output fusion method based on set operations according to claim 1, characterized in that, the method for calculating the accuracy of the sub-model is: Calculate its accuracy based on the results of the s-th sub-model, and calculate the mean absolute percentage error MAPE of the sub-model s : where N is the size of the training data set, and y n is the original output of the n-th group of training data, and is the predicted output of the n-th group of training data in the s-th model; According to the Mean Absolute Percentage Error (MAPE) s calculate the accuracy r of the sub-model s : r s = 1 - MAPE s .

5. The compatible multi-model output fusion method based on set operations according to claim 1, characterized in that, In the step S3, calculate the similarity between the core data of the m-th input factor for the s-th model and the q-th group of data The expression is as follows: Among them, and are the normalized versions of the m-th input factor of the q-th group of data and the core data in the dataset, respectively.

6. The compatible multi-model output fusion method based on set operations according to claim 1, characterized in that, In the step S3, it also includes normalizing the assigned weights, that is 7. The compatible multi-model output fusion method based on set operations according to claim 6, characterized in that, the fusion method in step S4 is: Determine the multi-model output fusion for the q-th group of data: Among them, represents the output of the model with the highest precision.

8. The compatible multi-model output fusion method based on set operations according to claim 7, characterized in that, the verification method in step S4 is: For a test data set containing Q groups of data, calculate the mean absolute error for verification: Among them, and respectively represent the actual output and the predicted output of the q-th group of test data.

Images