An intelligent system evaluation method based on the credibility of multi-model cross-identification data

Abstract

The present invention discloses an intelligent system evaluation method based on the credibility of multi-model cross-identification data. The method first collects training sets and test sets through field and simulation tests. Then, a sub-model is constructed and randomly sampled from the training set to generate S sub-training sets, respectively trained S The proposed method uses a sub-model to predict the training set, generating and ranking multiple errors. A clustering algorithm is then used to identify initial untrustworthy data. The initial untrustworthy datasets for all sub-models are then counted and frequency calculated. The final untrustworthy dataset is then determined based on the frequency and removed from the training set. The remaining data after removal is considered trustworthy data. Finally, a tabular denoising diffusion probability model is used to enhance the trustworthy data. The enhanced dataset is used to train new sub-models, which are then validated using a test set. This method can accurately identify data credibility and demonstrates high compatibility across different machine learning methods.

Description

Technical Field

[0001] The present invention relates to the field of intelligent system evaluation, and in particular to an intelligent system evaluation method based on the credibility of multi-model cross-identification data. Background Art

[0002] Unmanned swarm systems, as highly autonomous and collaborative intelligent systems, are rapidly becoming a key tool in a variety of fields. These systems, composed of multiple drones, are capable of completing complex tasks through collaborative operations and information sharing, demonstrating a high degree of autonomy and flexibility. For example, in logistics and distribution, unmanned swarms can achieve multi-point delivery and route optimization; in environmental monitoring, they can cover wider areas and provide more comprehensive data; and in emergency rescue, they can respond quickly, conduct multi-point reconnaissance, and deliver supplies.

[0003] In an unmanned swarm system, each drone is equipped with multiple sensors, collecting vast amounts of data, including flight status parameters, environmental information, and target identification data. This data is not only massive in volume but also comes from diverse sources, increasing the complexity of data management and analysis. Traditional methods for detecting abnormal data (such as setting normal ranges and using statistical analysis) typically only identify abnormalities in the data itself, but fail to assess how well the data matches the system's input-output relationships. Consequently, these methods often struggle to effectively identify untrustworthy data in unmanned swarm systems.

[0004] Furthermore, the complexity and diversity of data in unmanned swarm systems require evaluation methods that comprehensively measure data credibility from multiple perspectives. However, different machine learning methods perform differently in different scenarios, making it difficult for a single method to comprehensively assess data credibility. Therefore, an evaluation method that integrates the strengths of multiple machine learning methods is needed. This method, through multi-model cross-validation, can comprehensively assess data credibility and eliminate untrustworthy data.

[0005] Even after eliminating untrustworthy data through multi-model cross-validation, problems such as insufficient data and uneven data distribution may still exist, impacting the model's generalization and prediction accuracy. This is particularly true in unmanned swarm systems, where trusted data is often concentrated in specific tasks or environments, resulting in a lack of sufficient training data for models facing new tasks or complex scenarios. Summary of the Invention

[0006] To address the above challenges, this paper proposes an intelligent system assessment method based on multi-model cross-identification of data credibility, which identifies data credibility through multiple machine learning methods. To address the first challenge, this method generates multiple errors through a single machine learning model to preliminarily assess data credibility. For the second challenge, if data is identified as unreliable by multiple models, reflecting a relatively high error or average error of the data, it is classified as unreliable data. Conversely, if it is identified as reliable by multiple methods, indicating a low error or average error, the data is considered reliable. For the third challenge, the trusted data is expanded through a data augmentation algorithm, thereby increasing the diversity and quantity of the data.

[0007] In order to solve the above technical problems, the technical solution of the present invention is:

[0008] An intelligent system evaluation method based on the credibility of multi-model cross-identification data includes the following steps:

[0009] (1) Through a series of field and simulation tests, a total of N groups of drone cluster deployment data D containing M input features were collected. P groups were randomly selected as training sets, and the remaining Q groups were used as test sets.

[0010] (2) Construct sub-models. Use the generated sub-datasets to train S sub-models respectively, and use the trained sub-models to predict the training set and generate multiple errors.

[0011] The sub-models include back propagation neural network (BPNN) and Gaussian process regression (GPR) models. Random sampling is performed from the training set to generate S sub-training sets.

[0012] (3) Initial data credibility assessment. Sort the errors obtained in step (2), and then use the clustering algorithm to identify the initial unreliable data. . Count the initial untrusted data sets of all sub-models and generate frequencies ;

[0013] (4) Final confirmation of untrusted dataset. Confirm the final untrusted dataset And remove them from the training set, and the remaining data after removal is the credible data;

[0014] (5) Data enhancement and model reconstruction. The tabular denoising diffusion probability model (TabDDPM) is used to enhance the credible data and obtain the enhanced dataset.

[0015] (6) Comprehensive evaluation of intelligent systems. The enhanced dataset is used to train new sub-models, output evaluation results, and improve the generalization ability and prediction accuracy of the intelligent system evaluation model. The reconstructed model is validated using the test set, and the average error is calculated to evaluate the impact of data credibility on the model's prediction performance.

[0016] Preferably, step (3) is:

[0017] (3.1) According to The absolute error of the sub-model is calculated based on the results of the sub-model (model type is BPNN and GPR) :

[0018] (1)

[0019] in, is the original output of the p-th group of training data, is the predicted output of the p-th group of training data in the s-th model;

[0020] (3.2) Determine the initial untrusted data of the sth model:

[0021] According to the calculation results in formula (1), the absolute errors of the s-th model results are re-sorted in ascending order to form a new ordered list :

[0022] (2)

[0023] in, .

[0024] Use the density-based clustering algorithm (DBSCAN) to cluster the ordered list Cluster the data related to the sth model, set the error threshold, and identify the cluster of samples with the most errors higher than the error threshold. The data in the identified cluster is used as the initial untrustworthy data set for the sub-model ;

[0025] (3.3) Further confirm the credibility of the data:

[0026] Count the initial untrustworthy data sets identified by all sub-models in (3.2) and generate the frequency .in Indicates how many models identify the p-th group of data in the training set as untrustworthy.

[0027] Preferably, step (4) is specifically:

[0028] (4.1) Confirm the final untrusted dataset:

[0029] According to the set threshold from Confirm the untrusted data identified by the nth model :

[0030] (3)

[0031] According to the set operation, the untrusted data identified by different models are Merge to generate the final untrusted dataset:

[0032] (4)

[0033] in, represents the final untrusted dataset after the final merger, and N represents the number of machine learning methods adopted;

[0034] (4.2) Eliminate untrustworthy data:

[0035] The final untrustworthy data generated in (4.1) is removed from the training set, and the remaining data forms the trustworthy dataset.

[0036] Preferably, step (5) is specifically:

[0037] (5.1) Comprehensive scoring modeling:

[0038] A scoring system is constructed based on the input attributes of the dataset and the domain knowledge of the unmanned swarm system to guide data enhancement and generate data that conforms to reality. The capability score Y is defined as a multi-parameter weighted function:

[0039] (5)

[0040] in, Represents the i-th input parameter of the trusted data, is the normalization function of the i-th input parameter, is the weight coefficient of the i-th input parameter, determined by expert experience or statistical methods.

[0041] (5.2) Forward diffusion:

[0042] For the i-th credible data ( For trusted data Each input in the output Add noise gradually so that it approaches the noise distribution after T steps. Different inputs in the, because each input has different importance to the output, use different noise amplitudes (standard deviation ) is disturbed. The new input of step t It can be expressed as:

[0043] (6)

[0044] in, is the diffusion step size, which is different for different input settings. Value, new input This is the enhanced data obtained.

[0045] (5.3) Reverse denoising:

[0046] For the augmented data obtained after T steps in (5.2), the pre-trained conditional diffusion model is used to gradually remove noise to restore the true distribution of the data. While gradually removing noise, in order to ensure that the generated data meets the comprehensive score constraint, the comprehensive score Y in (5.1) is injected into the conditional diffusion model as a condition to ensure that the generated data meets the comprehensive score constraint. It is consistent with the expected score. The objective function of the denoising process can be expressed as:

[0047] (7)

[0048] in, is the forward diffusion in (5.2) The noise added by the wheel, is the noise estimated by the conditional diffusion model in the tth round of reverse denoising, It is a scoring function for calculating the quality of the data generated by each denoising step, and the calculation method is the same as same, is a hyperparameter used to balance denoising accuracy and score matching. For trusted data The ability score is obtained according to formula (5).

[0049] Preferably, step (6) is specifically:

[0050] Add the enhanced data after denoising in step (5) to the trusted dataset to form the final training dataset ,use Train a new model, make predictions on the test set containing Q groups of data, and calculate the average error between the predicted values and the true values to evaluate the prediction performance of the model.

[0051] The present invention has the following characteristics and beneficial effects:

[0052] Using the above technical solution, (1) a data credibility-based intelligent system evaluation method is proposed, which identifies data credibility through multiple machine learning methods. (2) Data should have a certain degree of credibility rather than being completely trustworthy for all data. (3) The proposed method is highly compatible and can be applied to different machine learning methods, and the effectiveness of the method is not necessarily proportional to the number of methods used.

[0053] In a real-world case study of unmanned aerial vehicle (UAV) capability assessment, this study used BPNN and GPR to construct sub-models for credibility data identification, and conducted in-depth and extensive comparative validation. The study yielded the following important conclusions: (1) The proposed method can accurately identify data credibility; (2) The proposed method outperforms the method that directly uses the original training set for modeling; and (3) The proposed method shows a high degree of compatibility among models constructed using different machine learning methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] To illustrate the embodiments of the present invention or the technical solutions in the prior art, it is necessary to briefly introduce the relevant drawings. The drawings shown below are only part of the embodiments of the present invention. For those skilled in the art, even without creative work, they can understand the technical content corresponding to the other drawings based on these drawings.

[0055] Figure 1 It is the overall method framework diagram of the present invention;

[0056] Figure 2 is an illustration of the present invention's identification of an initial untrusted data set;

[0057] Figure 3 is an illustration of the present invention performing a clustering operation on unreliable data in a training set;

[0058] Figure 4 It is the result diagram of the individual errors and error changes of the test data set of the present invention;

[0059] Figure 5 It is an explanatory diagram of the error comparison under different model combinations in the present invention. DETAILED DESCRIPTION

[0060] In conjunction with the accompanying drawings, a method for evaluating the credibility of intelligent systems based on multi-model cross-recognition data is further described, comprising the following steps:

[0061] Step 1: Through field and simulation tests, collect drone cluster deployment data containing M input features and N groups, randomly select P groups as training sets, and the rest as test sets.

[0062] Step 2: Build a sub-model, randomly sample from the training set to generate S sub-training sets, train S sub-models respectively, use the trained sub-models to predict the training set and generate multiple errors.

[0063] Step 3: Sort the errors, use the clustering algorithm to identify the initial untrustworthy data, count the initial untrustworthy data sets of all sub-models, and generate frequencies.

[0064] Step 4: Confirm the final untrustworthy data set based on the frequency and remove it from the training set. The remaining data after removal is the trustworthy data.

[0065] Step 5: Use the tabular denoising diffusion probability model to enhance the credible data and obtain the enhanced data set.

[0066] Step 6: Train a new sub-model based on the enhanced dataset, output the evaluation results, and verify them using the test set.

[0067] Example:

[0068] The present invention provides an intelligent system assessment method based on multi-model cross-identification data credibility. First, the training set is randomly sampled 100 times, with 90% of the data being sampled each time. Then, 100 sub-models are constructed using a back-propagation neural network (BPNN) and Gaussian process regression (GPR). The MAE generated by each sub-model on the training set is then ranked, and a clustering algorithm is used to obtain an initial untrustworthy dataset from the ordered sequence. The initial results of the 100 sub-models are then aggregated to generate a final untrustworthy dataset. Finally, the untrustworthy dataset is eliminated. The dataset after the untrustworthy data is enhanced using the TabDDPM model, and a comprehensive scoring mechanism is introduced to perform real-time quality assessment on the generated new data to ensure that the generated new data is realistic. A new model is trained using the enhanced dataset and validated on a test set to evaluate predictive performance. Research results show that modeling using the dataset without the untrustworthy data is superior to modeling using the dataset without the untrustworthy data.

[0069] This embodiment proposes an intelligent system evaluation method based on multi-model cross-recognition data credibility, which includes four steps. The overall framework of the method is as follows: Figure 1 As shown:

[0070] (1) Divide the data set. Randomly sample 100 times from the training set, each time extracting 360 groups of data, accounting for 90% of the total training set data.

[0071] Specifically, in this embodiment, through a series of field and simulation tests, a total of 500 sets of drone cluster deployment data were collected, 400 sets were randomly selected as training data sets, and the remaining 100 sets were used as test data sets. Each set of data contains 5 inputs ( :Investigation coverage, :Reconnaissance height, : Navigation time, :Detection accuracy, : Identification time) with an output (Overall detection capability) Table 1 gives the upper (ub) and lower (lb) limits of the input and output parameters.

[0072] Table 1 Upper and lower limits of input characteristic parameters

[0073]

[0074] (2) Construct sub-models. Select two machine learning methods to construct 100 sub-models, use these sub-models for the new training set after sampling, and obtain the error of the p-th group of training data for the s-th model. :

[0075]

[0076] in, is the original output of the p-th group of training data, is the predicted output of the p-th set of training data in the s-th model.

[0077] This example uses two machine learning methods, namely back propagation neural network (BPNN) and Gaussian process regression (GPR). The parameter settings for both are as follows:

[0078] For BPNN, it is implemented using the nntool tool in MATLAB. The number of layers is 3, the number of neurons is 4, the maximum number of iterations is 1000, and the transfer function is trainlm.

[0079] For GPR, the fitrgp tool in MATLAB is used, the fitting method is exact, the prediction method is exact, and the explicit basis function is constant.

[0080] (3) Initial credibility data identification. Specifically, Figure 2 shown.

[0081] (3.1) Sort the errors generated by BPNN and GPR in step (2) respectively to obtain an ordered list and .

[0082] (3.2) Use the DBSCAN clustering algorithm, set the parameters MinPts to 0.0005 and epsilon to 10. and Perform clustering to form an initial untrusted data set and . Figure 3 The errors and clustering results of GPR and BPNN are shown.

[0083] (3.3) Summarize the clustering results of the 100 sub-models of BPNN and GPR respectively, and calculate the frequency of each set of training data being identified as unreliable data by BPNN and GPR.

[0084] (4.1) Select data with a frequency greater than or equal to 50% as the final untrusted data set for BPNN and GPR respectively and . It contains 23 training set data, accounting for 5.75% of the entire training set, and the remaining 377 sets of training data are determined as credible data by BPNN; It contains 28 training set data, accounting for 7% of the entire training set, and the remaining 372 sets of training data are determined as credible data by GPR.

[0085] (4.2) Generate synthetic untrusted datasets for BPNN and GPR using set operations: ,in, Contains 20 training data, accounting for 5% of the training set.

[0086] (4.3) Eliminate from the original training set Finally, a trusted data set is formed.

[0087] (5) Perform data enhancement on the trusted dataset. Select a set of data from the trusted dataset To illustrate the data enhancement process in the present invention.

[0088] (5.1) Establish a comprehensive scoring standard. The comprehensive scoring standard adopted in this plan is a standardized evaluation system based on the experience of experts in the UAV field and actual operational needs. Spatial parameters are normalized using min-max (such as reconnaissance coverage and reconnaissance altitude), and time parameters are normalized using inverse normalization, that is, the smaller the value, the greater the contribution (such as navigation time and recognition time). The comprehensive scoring formula is defined as: .

[0089] (5.2) Forward diffusion. For the original credible data Add Gaussian noise, and adjust the noise amplitude according to the importance of the parameters. Add small noise to the key parameters of the unmanned swarm system (accuracy, recognition time, navigation time), and set the noise standard deviation ; Add medium noise to the secondary parameters (coverage, height) of the unmanned cluster system and set the noise standard deviation ; Add large noise to the parameter (navigation time) that has relatively small impact on the unmanned cluster system and set the noise standard deviation The noise of wheel T is recorded as The following table 2 is the credible data Results of 100 rounds of forward diffusion.

[0090] Table 2 Forward diffusion results

[0091]

[0092] (5.3) Reverse denoising. The pure noise data generated by (5.2) is gradually restored to structured data that meets the comprehensive score constraints. The current credible sample is calculated by (5.1). The target comprehensive score is The denoising process is implemented by multiple rounds of UNet network recursive prediction, and each round introduces a comprehensive score The UNet network encoder uses three layers of 1D convolution with the number of channels [128, 256, 512], the activation function is ReLU, and time and score encoding are concatenated after each layer; the decoder uses three layers of transposed convolution with the number of channels [512, 256, 128], connected to the corresponding layer of the encoder using skip connections; the final output channel is 5, corresponding to the noise component of each feature variable. The objective function of the tth round of the inverse denoising model is defined as:

[0093]

[0094] in, represents the noise predicted by the model during the t-th round of denoising, Represents the comprehensive score of the input parameters after the tth round of denoising.

[0095] Table 3 below is credible data Results of 100 rounds of reverse denoising.

[0096] Table 3 Reverse denoising results

[0097]

[0098] (6) Use the enhanced credible dataset to train a new model. For the test set containing Q group data, calculate the mean average error (MAE) to verify and evaluate the prediction performance.

[0099] Specifically, such as Figure 4 As shown in Figures 2 and 3, (a) and (b) show the change in absolute error obtained using BPNN and GPR before and after the proposed method is applied. Taking six test data sets as an example—namely, the 4th, 7th, 22nd, 46th, 62nd, and 93rd data points—the MAE for all data points shows a consistent downward trend. Figures 3 and 4 show that the absolute error decreases overall after the proposed method is applied. The absolute error of BPNN and GPR decreased for 63 and 80 data points, respectively.

[0100] Before using our method, the MAEs for BPNN and GPR were 0.0039 and 0.0048, respectively. After using our method, the MAEs dropped to 0.0033 and 0.0038, respectively. This shows that compared to using BPNN and GPR directly as baselines, our method can reduce the MAEs by 15.38% (=(0.0039-0.0033) / 0.0039) and 20.83% (=(0.0048-0.0038) / 0.0048), respectively.

[0101] To further validate the effectiveness of this method, in addition to the previously mentioned BPNN and GPR, two other machine learning methods—the radial basis function (RBF) and the support vector machine (SVM)—were also used. The RBF expansion coefficient was set to 2.21E06, the number of neurons to 3, and the mean square error target to 1.3E-02. The SVM kernel constraint (BoxConstraint) was set to 177.81, the kernel scale (kernelScale) to 1.3187, and the tolerance interval (Epsilon) to 0.0159. This combination of four machine learning methods was used to validate this method.

[0102] Figure 5 The following figure compares the number of untrusted data identified under different combinations:

[0103] (1) The data with lower confidence levels identified by the BPNN, GPR, and RBF models showed similarities. Specifically, the number of BPNN+GPR models decreased from 23 / 28 to 20, the number of BPNN+RBF models decreased from 23 / 19 to 17, the number of GPR+RBF models decreased from 28 / 19 to 19, and the number of BPNN+GPR+RBF models decreased from 23 / 28 / 19 to 14.

[0104] (2) Compared with the other three models, the SVM model has a significant difference in identifying data with lower credibility. In the two-model combination, SVM+BPNN identified 4 groups of data, SVM+GPR identified 5 groups of data, and SVM+RBF identified 5 groups of data, which are much smaller than the data identified by the BPNN, GOR, and RBF combinations. For the three-model combination, SVM+BPNN+GPR identified 4 groups, SVM+BPNN+RBF identified 4 groups, and SVM+GPR+RBF identified 5 groups, which are also much smaller than the groups identified by the BPNN+GPR+RBF combination.

[0105] Table 4 below is a comparative analysis of the statistical results of various method combinations in the present invention, including MAE on the test set and comparison of directly using BPNN, GPR, RBF and SVM.

[0106] Table 4 Comparison of statistical results under various method combinations in the present invention

[0107]

[0108] The following conclusions can be drawn from Table 4:

[0109] (1) Among all the combinations, the BPNN+GPR combination achieved the best results. After using this method, the MAEs of BPNN and GPR on the test dataset were 0.0033 and 0.0038, respectively. In other words, compared with directly using BPNN and GPR, the present invention reduced the MAEs of BPNN and GPR on the test dataset by 15.38% (= (0.0039-0.0033) / 0.0039) and 20.83% (= (0.0048-0.0038) / 0.0048), respectively. By using this method, the MAEs of the BPNN+RBF, GPR+RBF, and BPNN+GPR+RBF combinations also decreased, with decreases ranging from 7.27% to 12.5%.

[0110] (2) The introduction of SVM affects the performance of the combination. In BPNN+SVM, the MAE of BPNN decreased by 5.13%, while the MAE of SVM increased by 5.65%. In GPR+SVM, the MAE of GPR decreased by 2.08%, while the MAE of SVM increased by 14.25%. Similarly, in RBF+SVM, the MAE of RBF decreased by 3.64%, while the MAE of SVM increased by 14.25%. This trend continues in the three-model combination and the four-model combination. The MAE of SVM continues to rise in all combinations, indicating that it has a negative impact on the overall performance. Except for SVM, the MAE of other models also decreased, but the range of decrease was between [2.08%, 5.13%], which is smaller than the range of [7.69%, 12.5%] in conclusion (1).

[0111] (3) The effectiveness of the proposed method is not necessarily proportional to the number of methods used. Specifically, in the BPNN+GPR combination, the MAE of BPNN decreased by 15.38%, and the MAE of GPR decreased by 20.83%. Similarly, in the BPNN+RBF combination, the MAE of BPNN decreased by 7.69%, and the MAE of RBF decreased by 9.09%. In contrast, the MAE reduction of the BPNN+GPR+RBF combination was smaller, with 10.26% for BPNN, 10.42% for GPR, and 7.27% for RBF.

[0112] To further validate the effectiveness of this method, we tested it with three other machine learning methods: random forest (RF), gradient boosted regression tree (GBRT), and confidence rule base (BRB). Table 5 shows the parameter settings and MAE of the three methods. As shown in Table 5, RF and GBRT produce similar results, both outperforming BRB. The poor performance of BRB may be due to insufficient training data.

[0113] Table 5 Parameter settings and MAE of the three machine learning methods in this invention

[0114]

[0115] Based on the above verification results, the following conclusions can be drawn:

[0116] (1) The results show that the combination of BPNN and GPR performs best. After using the method proposed in this paper, the MAE of BPNN on the test set is reduced by 15.38%, and the MAE of GPR is reduced by 20.83%. If the MAE requirement is relaxed, all combinations of BPNN, GPR, and RBF are acceptable.

[0117] (2) Low-precision models are not suitable for combination. In all combinations containing the lowest-precision SVM, the MAE of the SVM increases, while the MAE of other models except the SVM decreases very little. For example, when GPR is combined with SVM, its MAE decreases by only 2.08%. When SVM is combined with BPNN, the MAE of SVM increases by 5.64%.

[0118] (3) The effectiveness of a method is not necessarily proportional to the number of methods used. For example, in the BPNN+GPR+RBF combination, the MAE of BPNN is reduced by 10.26%, which is lower than the 15.38% of BPNN+GPR. The MAE of GPR is reduced by 10.42%, which is lower than the 20.83% of BPNN+GPR. The MAE of RBF is reduced by 7.27%, which is lower than the 9.09% of BPNN+GPR.

[0119] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes, modifications, substitutions, and variations of the embodiments and components thereof made by those skilled in the art without violating the core principles and spirit of the present invention should be deemed to fall within the scope of protection of the present invention.

Claims

1. An intelligent system evaluation method based on multi-model cross-identification data credibility, characterized in that: The following steps are involved: Step 1: Through field and simulation tests, collect N groups of drone swarm deployment data containing M input features, randomly select P groups as training sets, and the rest as test sets; Step 2: Construct a sub-model, randomly sample S sub-training sets from the training set, train S sub-models respectively, use the trained sub-models to predict the training set and generate multiple errors; Step 3: Sort the errors, use the clustering algorithm to identify the initial untrustworthy data, count the initial untrustworthy data sets of all sub-models, and generate the frequency; Step 4: Confirm the final untrustworthy data set based on the frequency and remove it from the training set. The remaining data after removal is the trustworthy data. Step 5: Use the tabular denoising diffusion probability model to enhance the credible data and obtain the enhanced data set. The specific implementation process is as follows: Step 5.1: Build a scoring system and define the ability score Y as a multi-parameter weighted function: Among them, x i Represents the i-th input parameter of the trusted data, f i (x i ) is the normalization function of the i-th input parameter, α i is the weight coefficient of the i-th input parameter; Step 5.2, y i is the trusted data d i The output of the i-th trusted data d i ={x i,1 ,x i,2 ,...,x i,j ,y i }, each input x in i,j Gradually add noise, for d i Different inputs in the t step are disturbed by different noise amplitudes σ. Expressed as: Among them, β t,j is the diffusion step length, the new input This is the enhanced data obtained; Step 5.3: For the enhanced data obtained after T steps in step 5.2, use the pre-trained conditional diffusion model to gradually denoise and restore the true distribution of the data. While gradually removing the noise, the comprehensive score Y is injected into the conditional diffusion model as a condition. The objective function of the denoising process is expressed as: L=(ε-ε(x t ,t)) 2 +λ((f(x0′)-Y) 2 ) Where ε is the noise added in the Ttth round of forward diffusion in step 5.2, ε(x t ,t) is the noise estimated by the conditional diffusion model in the tth round of reverse denoising, f(x) is the scoring function for calculating the quality of the data generated by each denoising step, and the calculation method is the same as Y, where Y is the credible data d i The obtained ability score, λ is a hyperparameter; Step 6: Train a new sub-model based on the enhanced dataset, output the evaluation results, and verify them using the test set.

2. The intelligent system evaluation method based on multi-model cross-recognition data credibility according to claim 1 is characterized in that: The sub-models described in step 2 include a back propagation neural network (BPNN) and a Gaussian process regression (GPR) model.

3. The intelligent system evaluation method based on multi-model cross-recognition data credibility according to claim 2 is characterized in that: The specific implementation process of step 3 is as follows: Step 3.1, calculate the absolute error of the sub-model based on the result of the s-th sub-model; Step 3.2: Determine the initial unreliable data for the sth model: Reorder the absolute errors of the sth model results in ascending order to form an ordered list. Use a density-based clustering algorithm to cluster the data related to the s-th model in the ordered list, set an error threshold, and identify the cluster containing the most samples with errors higher than the error threshold; use the data in the identified cluster as the initial untrusted data set for the sub-model; Step 3.3: Count the initial untrustworthy data sets identified by all sub-models and generate the frequency f = {f1,...,f p ,...,f P }, f p How many models identify the p-th group of data in the training set as untrustworthy? 4. The intelligent system evaluation method based on multi-model cross-recognition data credibility according to claim 3 is characterized in that: The specific implementation process of step 4 is as follows: Step 4.1, set the threshold δ, and confirm that the nth model recognizes f from f P Untrustworthy data Λ greater than or equal to the threshold δ n , the untrustworthy data Λ identified by different models n Merge to generate the final untrusted data set; Step 4.2: Remove the final untrustworthy data from the training set, and the remaining data form a trustworthy data set.

5. The intelligent system evaluation method based on multi-model cross-recognition data credibility according to claim 4 is characterized in that: The specific implementation of step 6 is: adding the enhanced data after denoising in step 5 to the trusted dataset to form the final training dataset D final , using D final Train the new sub-model and make predictions using the test set.

Images