Verification methods, systems, and storage media for digital twin trust assessment methods

Abstract

This invention discloses a verification method, system, and storage medium for a digital twin credibility assessment method, belonging to the field of modeling and simulation technology. The method includes: constructing an experimental platform; calibrating the credibility level of the experimental platform to obtain an optimized experimental platform; verifying the accuracy and universality of the assessment method based on the optimized experimental platform; verifying the assessment method based on the accuracy and universality results; obtaining and outputting the verification results. This invention ensures full element coverage with as few experimental platforms as possible, reduces redundancy, lowers the cost of the verification scheme, and allows for fine-tuning of the model for specific experimental platforms, taking into account the impact of the on-site environment on the experiment.

Description

Technical Field

[0001] This invention relates to the field of modeling and simulation technology, and more specifically to a verification method, system, and storage medium for digital twin credibility assessment methods. Background Technology

[0002] The reliability assessment methods for digital twins have complex indicator systems, and the specific assessment forms vary depending on the physical entity. Unlike traditional assessment methods, digital twin reliability assessment methods are dynamic and require attention to the evolution of the digital twin. The assessment methods themselves exhibit high complexity; therefore, experimental verification of the effectiveness of digital twin assessment methods is necessary.

[0003] However, existing methods for evaluating the reliability of digital twin systems lack efficient verification methods, which cannot ensure the accuracy of the evaluation method itself, nor can they ensure that the evaluation method itself has good generalization, and therefore cannot be widely applied to other fields.

[0004] Therefore, how to provide a verification method, system, and storage medium for a digital twin trust assessment method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a verification method, system, and storage medium for digital twin trust assessment. By considering orthogonal decomposition across different dimensions, it constructs a minimum number of experimental platforms for digital twin trust assessment, along with a combination method, reducing the number of experimental platforms and thus significantly reducing the consumption of human and material resources. Simultaneously, the experimental trust level is obtained through the evolution process of the test model. Based on the experimental trust level, the initial trust level is calibrated to ensure the accuracy of the verification method.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] On one hand, the present invention provides a verification method for a digital twin credibility assessment method, comprising:

[0008] S100: Obtain the evaluation method indicators, obtain the corresponding domain, perform orthogonal decomposition on the domain, and construct an experimental platform based on the orthogonal decomposition results;

[0009] S200: Calibrate the reliability level of the experimental platform to obtain the optimized experimental platform;

[0010] S300: Verify the accuracy of the evaluation method based on the optimized experimental platform;

[0011] S400: Verify the universality of the evaluation method based on the optimized experimental platform;

[0012] S500: Validate the evaluation method based on the accuracy results and the universality results, obtain the validation results, and output them.

[0013] Preferably, S100 further includes:

[0014] Let N be the number of neighborhoods after orthogonal decomposition, and select a typical device t in the i-th neighborhood. ij Where ij represents the j-th device in the i-th domain, and the price of the device is denoted as p. ij ;

[0015] Let the total number of verification devices required be M (M≤N), then the total cost of the verification devices is P. sum The calculation formula is:

[0016]

[0017] Construct an experimental platform based on the total cost of the verification equipment.

[0018] Preferably, S200 includes:

[0019] S210: Test the experimental platform and obtain the corresponding experimental reliability level;

[0020] S220: Obtain the initial trust level, set a threshold, and determine whether the initial trust level and the experimental trust level are less than the threshold;

[0021] S230: If the initial confidence level and the experimental confidence level are less than the threshold, then the average value of the initial confidence level and the experimental confidence level is calculated, and the confidence level of the experimental platform is adjusted according to the average value to obtain the optimized experimental platform.

[0022] Preferably, S300:

[0023] S310: Obtain the digital twin model, execute the corresponding evaluation method, and calculate the credibility level;

[0024] S320: Sort the confidence levels of the calibrated experimental platforms and the calculated confidence levels respectively, determine whether the sorting results are consistent, and judge the accuracy of the evaluation method based on the consistency results.

[0025] Preferably, S400:

[0026] S410: Let the experimental platform be {A, B, C, ..., INF};

[0027] Where A, B, C, and INF represent different physical objects;

[0028] S420: Constructing a two-dimensional array for a digital twin system:

[0029] [[a1, a2, a3, a4], [b1, b2, b3, b4], ..., [inf1, inf2, inf3, inf4]]:

[0030] Where [a1, a2, a3, a4], [b1, b2, b3, b4], and [inf1, inf2, inf3, inf4] represent different digital twin model queues;

[0031] S430: Obtain the set of accuracy results for the evaluation results {Y} a Y b Y c , ..., Y inf};

[0032] Among them, Y a Y b Y c Y inf These represent the accuracy rates of different evaluation results:

[0033] S440: Calculate the accuracy set {Y} a Y b Y c , ..., Y inf The variance between} is used to determine the universality of the evaluation method.

[0034] On the other hand, the present invention provides a verification system for a digital twin trustworthiness assessment method, comprising:

[0035] A construction module is used to obtain the indicators of the evaluation method, obtain the corresponding domain, perform orthogonal decomposition on the domain, and construct an experimental platform based on the orthogonal decomposition results;

[0036] An optimization module, connected to the construction module, is used to calibrate the reliability level of the experimental platform to obtain an optimized experimental platform.

[0037] The first verification module, connected to the construction module, is used to verify the accuracy of the evaluation method based on the optimized experimental platform.

[0038] The second verification module, connected to the construction module, is used to verify the universality of the evaluation method based on the optimized experimental platform.

[0039] The output module, connected to the first verification module and the second verification module, is used to verify the evaluation method based on the accuracy result and the universality result, obtain the verification result, and output it.

[0040] In another aspect, the present invention provides a computer-readable storage medium storing computer-readable instructions, characterized in that the computer-readable instructions, when executed by a processor, implement the steps of the verification method described above.

[0041] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a verification method, system, and storage medium for digital twin trust assessment, which has the following beneficial effects:

[0042] (1) Compared with the prior art, the verification method of the present invention can significantly reduce costs. By performing multidisciplinary and multi-domain orthogonalization or near-orthogonalization, and selecting experimental platforms based on the orthogonalization results, the method ensures that as few experimental platforms as possible can achieve full element coverage, reducing redundancy and lowering the cost of the verification scheme.

[0043] (2) Compared with the prior art, the verification method of the present invention is more reasonable. By calibrating the confidence level of the model, not only is the initial confidence level of the model maintained and the performance of the model in the long-term use process fully considered, but also the model is fine-tuned for the specific experimental platform by testing the evolution process of the model, taking into account the influence of the field environment on the experiment. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 A flowchart illustrating the verification method provided by this invention;

[0046] Figure 2 This is a schematic diagram of the construction process of the experimental platform provided in an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of the reliability level calibration process of the experimental platform provided in this embodiment of the invention;

[0048] Figure 4 A schematic diagram illustrating the accuracy verification and evaluation method provided in this embodiment of the invention;

[0049] Figure 5 A schematic diagram illustrating the general applicability of the verification and evaluation method provided in the embodiments of the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of the verification system provided in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] See appendix Figure 1 As shown in the figure, an embodiment of the present invention discloses a verification method for a digital twin trustworthiness assessment method, comprising:

[0053] S100: Obtain the evaluation method's metrics, identify the corresponding domain, perform orthogonal decomposition on the domain, and construct an experimental platform based on the orthogonal decomposition results;

[0054] S200: Calibrate the reliability level of the experimental platform to obtain the optimized experimental platform;

[0055] S300: Verify the accuracy of the evaluation method based on the optimized experimental platform;

[0056] S400: Verify the universality of the evaluation method based on the optimized experimental platform;

[0057] S500: Validate the evaluation method based on the accuracy and universality results, obtain the validation results, and output them.

[0058] See appendix Figure 2 As shown, in one specific embodiment, S100 specifically includes:

[0059] Domain orthogonal decomposition minimizes the number of experimental platforms. Digital twin evaluation methods involve a vast and complex indicator system spanning multiple disciplines and domains. A single experimental platform cannot cover all indicators or achieve complete verification; therefore, multiple platforms are needed to jointly complete verification testing. However, building such platforms is costly, time-consuming, and labor-intensive. By using orthogonal or near-orthogonal decomposition across multiple disciplines and domains, followed by optimal combination, the number of experimental platforms can be minimized, thereby reducing costs.

[0060] The specific steps of orthogonal decomposition of the domain are as follows:

[0061] The evaluation method involves a vast system of indicators spanning one or more disciplines within electromechanical, hydraulic, pneumatic, and thermal engineering. Testing each indicator individually requires numerous experimental platforms, significantly increasing costs and consuming considerable time and effort. As shown in Table 1, the multidisciplinary and multi-domain factors involved in the reliable evaluation method are orthogonalized or approximately orthogonalized. Experimental platforms are selected based on the domain orthogonalization results to ensure that as few experimental platforms as possible achieve full element coverage, reducing redundancy and lowering the cost of the verification scheme.

[0062] Table 1. Orthogonal decomposition of the domain

[0063]

[0064] The specific steps for analyzing typical instruments and equipment are as follows:

[0065] Select typical instruments and equipment from various disciplines. Some fields are relatively abstract and have a wide range of applications, such as geometry, numerical analysis, and modeling. Most equipment in these fields can verify the accuracy of their specifications; that is, these fields are not sensitive to equipment, and any equipment will suffice. Other fields have distinct characteristics, such as sensing and measurement, communication, fluid dynamics, and materials science. Verifying specifications in these fields requires selecting appropriate equipment. For sensing and measurement, cameras, thermometers, Hall effect sensors, and voltage / current sensors are typical instruments with corresponding attributes. For communication, data terminals, service switching systems, and routing equipment are typical instruments with corresponding attributes. In fluid dynamics, wind tunnels are typical instruments with corresponding attributes. In materials science, structural steel and plastic instruments are typical instruments, possessing object-specific attributes. Therefore, the selection of typical instruments and equipment is specifically as follows: Figure 2 As shown.

[0066] Table 2 Typical Instrument and Equipment Selection

[0067]

[0068] In a specific embodiment, the steps for optimizing the combination and constructing the experimental platform are as follows:

[0069] Let N be the number of neighborhoods after orthogonal decomposition, and select typical equipment t in the i-th neighborhood. ij (The subscript ij indicates that it is the j-th device in the i-th domain), and the price of this device is denoted as p. ijSince a single device can encompass multiple domains, the required number of devices can be less than N. For example, a robotic arm integrates voltage and current sensors, pressure sensors, encoders, and cameras, making it a typical instrument in the sensing and measurement field. Furthermore, the robotic arm exchanges data with the control board, the host computer, and even has a network interface, making it a typical device in the communication field as well. Choosing robotic arms across multiple domains can potentially reduce the number of experimental platforms and thus lower verification costs. Let M (M≤N) be the total number of verification devices required, then the total cost P of the verification equipment is... sum It can be written as:

[0070]

[0071] More specifically, this becomes an optimization problem: selecting suitable equipment so that P sum Minimize as much as possible. Use the following selection strategy to reduce P. sum

[0072] Choose the largest common sub-device from the following categories: {geometry, numerical analysis, sensing and measurement, communication, fluid dynamics, materials science, ..., modeling}. Since the robotic arm involves not only general fields like geometry, numerical analysis, and modeling / simulation, but also sensors, measurement, and communication, it can be selected as the largest common sub-device.

[0073] Determine if the price of the largest sub-device exceeds the threshold. If it does not, proceed to step c. If the price of the largest sub-device exceeds the threshold, proceed to step a, and reselect, this time choosing the second largest common sub-device.

[0074] Remove the already involved domains to obtain the remaining domain set {material domain, ..., fluid domain}. If the remaining domain set is not empty, iterate through a and b to continue selecting a suitable device; if the remaining domain set is empty, proceed to d.

[0075] End. A selection of equipment / instruments has been obtained. (In this example, the selected options are a robotic arm, a 3D printer, ..., and an aircraft engine.)

[0076] See appendix Figure 3 As shown, in one specific embodiment, the trust level calibration of the S200 experimental platform specifically includes:

[0077] Through preliminary experiments, the model's evolution mechanism and input-output relationship are assessed to determine their correctness, and the model's reliability meets requirements. The experimental reliability level is obtained by comparing the correlation and similarity between the digital twin system's state and the actual state, as well as the time required for evolution. A weighted average of the experimental reliability level and the initial reliability level is calculated as the reference reliability level for the current digital twin system.

[0078] The specific steps for obtaining the experiment's confidence level include:

[0079] The current state and operational data of the physical object are acquired through sensor data collection and manual data collection. A portion of this data is then extracted as test data, and the other portion as evolution data. The evolution data is input into a digital twin system, which evolves according to its internal mechanisms to obtain the next state. The test data is used to calculate and predict the state of the physical object, obtaining the actual state at the next moment. The difference between the evolved state and the actual state is then compared; a larger difference indicates a lower experimental reliability level for the digital twin model, and a smaller difference indicates a higher experimental reliability level.

[0080] Specifically, for the robotic arm, the current control commands and voltage / current information are collected. The control commands are used as test data, and the voltage / current as evolution data. This data is input into the robotic arm's twin model, causing the electromagnetic and kinematic models to evolve. The motor model drives the entire robotic arm model's evolution, recording the evolutionary state Q1 at that moment. Based on the control commands and the driver program, the actual state Q2 of the robotic arm can be determined. The difference between these two states is compared: ΔQ = |Q1 - Q2|. A larger ΔQ indicates a lower experimental reliability, while a smaller ΔQ indicates a higher experimental reliability. For the robotic arm, the evolutionary state Q can be approximately equivalent to its position (x, y, z) and orientation (rotation matrix R).

[0081] Specific methods for adjusting the reliability level of the experimental platform include:

[0082] The initial confidence level is provided by the model library. When the difference between the obtained experimental confidence level and the initial confidence level exceeds a threshold, the confidence level of the experimental platform needs to be adjusted. First, the accuracy and operating mechanism of the unit models are adjusted to change their confidence level. Then, the confidence level of the combined models is changed by adjusting the topology, interfaces, and combination methods, thereby modifying the confidence level of the experimental platform. The modified experimental platform needs to undergo expert review, combined with prior knowledge and other methods, to obtain a new initial confidence level.

[0083] Specifically, for the robotic arm, several models are first selected from the database. (The models selected from the model library inherently possess an initial confidence level. However, this initial confidence level may deviate due to different scenarios.) Then, the experimental confidence level is obtained to acquire the experimental confidence level for each robotic arm model. For each robotic arm model, if the difference between the experimental confidence level and the initial confidence level exceeds a threshold, calibration is required. For example, by adjusting the DH parameters or changing the motor control model, the robotic arm model is modified to ensure that the model conforms to the current verification scenario.

[0084] The specific steps to obtain the calibration's trust level include:

[0085] The process of obtaining and adjusting the experimental platform's confidence level is repeated until the difference between the experimental confidence level and the initial confidence level is less than a threshold. At this point, the average of the initial confidence level and the experimental confidence level is calculated as the final calibration confidence level.

[0086] Specifically, for a given robotic arm model, assuming a threshold of 5%, when the initial confidence level is 80% and the experimental confidence level is 82%, the calibration confidence level is (80% + 82%) / 2 = 81%.

[0087] See appendix Figure 4 As shown, in one specific embodiment, the specific steps for S300 to verify the accuracy of the evaluation method include:

[0088] To objectively verify the accuracy of the evaluation method, it is necessary to evaluate different digital twin systems of the same physical entity within the same lifecycle. A multi-set experimental structure is employed for mutual verification, ensuring the objectivity and robustness of the verification process. Among the multiple experimental schemes, suitable model groups are selected from the model library, and evaluation metrics related to the physical entity and digital twin models are extracted, considering the metric coverage of the evaluation method. The accuracy of the tested reliable evaluation method is determined by combining the reasonableness of the result ranking and the time consumption of the evaluation method.

[0089] Specifically, selecting a suitable model from the model library includes:

[0090] Select a set of digital twin models from the model library and obtain their initial confidence levels from the model library. After the second step of calibration, obtain their calibration confidence levels.

[0091] Specifically, the evaluation methods include:

[0092] Digital twin models exhibit evolutionary characteristics, requiring the control of a single variable when implementing evaluation methods. This involves performing reliable evaluation methods on different digital twin models at the same time, in the same state, and with the same physical object, to obtain their calculated reliability values. By controlling for a single variable, unnecessary interference is reduced, ensuring the persuasiveness of the verification results.

[0093] Specifically, the comparative evaluation results include:

[0094] The calibration confidence levels are ranked, and then the calculated confidence levels are ranked. The consistency of the ranking results is compared, and the value of the evaluation method is judged based on trends. By comparing the calibration confidence levels and calculated confidence levels of the twin models to see if they match, and the calculation accuracy, a quantitative analysis is conducted to determine if there is any bias in the evaluation method.

[0095] More specifically, for the robotic arm, let n models be selected from the model library, and let y be the computational confidence level of the i-th model. i (Based on the evaluation algorithm), the calibration confidence level is [value missing]. (This can be obtained from step S200).

[0096] Let the threshold be σ, then in

[0097] See appendix Figure 5 As shown, in a specific embodiment, the universality of the S400 verification and evaluation method specifically includes the following steps:

[0098] The reliability assessment methods for digital twins are often comprehensive and complex, with a certain degree of generalization. A single experimental platform cannot verify the universality of the method under test; multiple different experimental platforms are needed to verify the generalization of the method under test. Let the experimental platforms be {A, B, C, ..., INF}. Using the above method, a two-dimensional array of the digital twin system [[a1, a2, a3, a4], [b1, b2, b3, b4], ..., [inf1, inf2, inf3, inf4]] is constructed. Similar to the verification using a single platform, the accuracy of the evaluation result {Y} is obtained. a Y b Y c , ..., Y inf}, and then calculate the accuracy set {Y}. a Y b Y c , ..., Y inf The variance between}. The larger the variance, the worse the generalizability of the method under test; the smaller the variance, the better the generalizability of the method under test.

[0099] More specifically, for robotic arms, it's impossible to verify the accuracy of the evaluation method in the fluid and materials domains. Therefore, it's necessary to construct 3D printer and engine testing platforms. A model sequence can be selected from the model library. Following step S300, the accuracy rate of the evaluation method on the robotic arm testing platform, 3D printer testing platform, and engine testing platform can be calculated separately. Due to differences in equipment, the accuracy rate will fluctuate. For example, the accuracy rate might be 95% on the robotic arm testing platform, 90% on the 3D printer, and 94% on the engine testing platform.

[0100] Specifically,

[0101]

[0102] Universality = -ln(variance) = -ln(0.046%) = 7.68428

[0103] Specifically, select multiple physical objects with differences, including:

[0104] By combining the orthogonal experimental platform built with S100, specific hardware devices were selected to obtain a set of physical objects with significant differences.

[0105] Specifically, the evaluation methods include:

[0106] Similar to S300, the accuracy of the evaluation method is calculated. An accuracy rate is obtained for each digital twin model queue.

[0107] Specifically, verifying the universality of the evaluation method includes:

[0108] Calculate the variance between the accuracy rates obtained by the evaluation methods. The larger the variance, the worse the generalizability; the smaller the variance, the better the generalizability.

[0109] In one specific embodiment, step S500 specifically includes:

[0110] By combining the accuracy and universality of S300 and S400, we can conclude whether the evaluation method is correct and efficient.

[0111] More specifically, when the accuracy calculated according to S300 is greater than 90%, the evaluation method can be considered correct within the verification domain. When the universality calculated according to S400 is greater than 7, the evaluation method is considered applicable in most domains, exhibiting good universality. Finally, the digital twin trustworthiness evaluation method is validated based on a combination of accuracy and universality results, and the validation results are output.

[0112] See appendix Figure 6As shown in the figure, an embodiment of the present invention discloses a verification system for a digital twin trustworthiness assessment method, comprising:

[0113] The module is used to obtain the metrics of the evaluation method, obtain the corresponding domain, perform orthogonal decomposition on the domain, and build an experimental platform based on the orthogonal decomposition results.

[0114] The optimization module, connected to the construction module, is used to calibrate the reliability level of the experimental platform and obtain the optimized experimental platform.

[0115] The first verification module, connected to the construction module, is used to verify the accuracy of the evaluation method based on the optimized experimental platform.

[0116] The second verification module, connected to the construction module, is used to verify the universality of the evaluation method based on the optimized experimental platform.

[0117] The output module, connected to the first verification module and the second verification module, is used to verify the evaluation method based on the accuracy and universality results, obtain the verification results, and output them.

[0118] In one specific embodiment, the present invention discloses a computer-readable storage medium storing computer-readable instructions, characterized in that the computer-readable instructions, when executed by a processor, implement the steps of the verification method described above.

[0119] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a verification method, system, and storage medium for digital twin trust assessment, which has the following beneficial effects:

[0120] (1) Compared with the prior art, the verification method of the present invention can significantly reduce costs. By performing multidisciplinary and multi-domain orthogonalization or near-orthogonalization, and selecting experimental platforms based on the orthogonalization results, the method ensures that as few experimental platforms as possible can achieve full element coverage, reducing redundancy and lowering the cost of the verification scheme.

[0121] (2) Compared with the prior art, the verification method of the present invention is more reasonable. By calibrating the confidence level of the model, not only is the initial confidence level of the model maintained and the performance of the model in the long-term use process fully considered, but also the model is fine-tuned for the specific experimental platform by testing the evolution process of the model, taking into account the influence of the field environment on the experiment.

[0122] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0123] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A verification method for a digital twin trust evaluation method, characterized in that Including: S100: Obtain the metrics of the evaluation method, obtain the corresponding domain, perform orthogonal decomposition on the domain, and construct an experimental platform according to the orthogonal decomposition result, including: Let the number of domains after orthogonal decomposition be , and select a typical device in the th domain . Among them, indicates that it is the th device in the th domain. Denote the price of the device as ; Let the total number of required verification devices be , , then the total cost of the verification devices is calculated by the formula: Construct an experimental platform based on the total cost of the verification device; S200: Calibrate the credibility level of the experimental platform to obtain an optimized experimental platform, including: S210: Test the experimental platform to obtain the corresponding experimental credibility level; S220: Obtain the initial credibility level and set a threshold, and judge whether the initial credibility level and the experimental credibility level are less than the threshold; S230: If the initial credibility level and the experimental credibility level are less than the threshold, calculate the average value of the initial credibility level and the experimental credibility level, and adjust the credibility level of the experimental platform according to the average value to obtain an optimized experimental platform; S300: Verify the accuracy of the evaluation method based on the optimized experimental platform, including: S310: Obtain a digital twin model, execute the corresponding evaluation method, and calculate the credibility level; S320: Sort the credibility levels of the calibrated experimental platform and the calculated credibility levels respectively, judge whether the sorting results are consistent, and judge the accuracy of the evaluation method according to the consistency results; S400: Verify the universality of the evaluation method based on the optimized experimental platform, including: S410: Set the experimental platform as ; Where A, B, C, and INF respectively represent different physical objects; S420: Construct a two-dimensional array of the digital twin system as: ； Where [a1, a2, a3, a4], [b1, b2, b3, b4], [inf1, inf2, inf3, inf4] respectively represent different digital twin model queues; S430: Obtain the set of correct rates of evaluation results ; Among them, Y c , inf , b , Y b , Y c , Y inf respectively represent the correct rates of different evaluation results: S440: Calculate the variance among the sets of correct rates and determine the universality of the evaluation method based on the variance result; Between them, according to the variance result, judge the universality of the evaluation method; S500: Verify the evaluation method according to the results of the accuracy and the universality, obtain the verification result and output it.

2. A verification system for a digital twin trust evaluation method using the verification method of the digital twin trust evaluation method described in claim 1, characterized in that, Including: A construction module, used to obtain the metrics of the evaluation method, obtain the corresponding domain, perform orthogonal decomposition on the domain, and construct an experimental platform according to the orthogonal decomposition result; An optimization module, connected to the construction module, used to calibrate the credibility level of the experimental platform to obtain an optimized experimental platform; A first verification module, connected to the construction module, used to verify the accuracy of the evaluation method based on the optimized experimental platform; A second verification module, connected to the construction module, used to verify the universality of the evaluation method based on the optimized experimental platform; An output module, connected to the first verification module and the second verification module, used to verify the evaluation method according to the results of the accuracy and the universality, obtain the verification result and output it.

3. A computer-readable storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by a processor, the steps of the verification method according to claim 1 are implemented.

Images