Trustworthiness evaluation method, device, system and storage medium

Abstract

The application discloses a trustworthiness evaluation method and device, an intelligent network system and a storage medium. The method is applied to the intelligent network system and includes the following steps: acquiring application information corresponding to a network function device to be evaluated; determining at least one evaluation index corresponding to the application information and at least one evaluation sub-index corresponding to each evaluation index in the at least one evaluation index based on the application information; determining a first evaluation parameter of the trustworthiness of the network function device according to each evaluation index and each evaluation sub-index in the at least one evaluation sub-index; and the first evaluation parameter is used for a trustor with authorization permission to determine whether to authorize the related function of the network function device.

Description

Technical Field

[0001] This application relates to the field of communication network technology, and in particular to a method, apparatus, intelligent network system, and storage medium for evaluating the degree of trustworthiness. Background Technology

[0002] With the development of network systems and the application of technologies such as artificial intelligence in communication networks, the application, management, and operation of autonomous networks will bring the following concerns or problems: Should operators trust autonomous networks? How can operators be convinced to trust autonomous networks and confidently entrust operational authority to the network itself? If operators are willing to trust autonomous networks, to what extent should they trust them? How to assess the credibility of autonomous networks? What key factors are needed to assess their credibility? How to quantify and evaluate the credibility of autonomous networks? Currently, there are no effective solutions. Summary of the Invention

[0003] To address the related technical issues, embodiments of this application provide a method, apparatus, intelligent network system, and storage medium for assessing trustworthiness.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a method for assessing trustworthiness in intelligent network systems, the method comprising:

[0006] Obtain application information corresponding to the network functional devices to be evaluated;

[0007] Based on the application information, at least one evaluation indicator corresponding to the application information is determined, and at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator;

[0008] A first evaluation parameter is used to determine the trustworthiness of the network function device based on each evaluation index and each of the at least one evaluation sub-index; the first evaluation parameter is used by an authorized trustee to determine whether to authorize the relevant functions of the network function device.

[0009] In the above scheme, obtaining the application information corresponding to the network functional device to be evaluated includes:

[0010] Obtain the assessment trigger instruction for the degree of trustworthiness;

[0011] The evaluation parameters of the network functional device are determined according to the evaluation trigger command;

[0012] The application information corresponding to the network functional device is determined based on the evaluation parameters.

[0013] In the above scheme, the application information includes application scenario information and / or application use case information; determining the application information corresponding to the network functional device based on the evaluation parameters includes:

[0014] The application scenario information and / or application use case information are determined based on the evaluation parameters.

[0015] In the above scheme, determining at least one evaluation indicator corresponding to the application information based on the application information, and at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator, includes:

[0016] Based on the application scenario information and / or application use case information, at least one evaluation index corresponding to the application information is determined, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0017] In the above scheme, the evaluation indicators include at least one of the following: accuracy, stability, controllability, reproducibility, interpretability, and adaptability;

[0018] When the evaluation index includes accuracy, the evaluation sub-index includes at least one of the following: reproducibility, accuracy, timeliness, and effectiveness;

[0019] When stability is included in the evaluation index, the evaluation sub-index includes at least one of the following: interruption rate, accident rate, maturity, and susceptibility to degradation;

[0020] When the assessment indicator includes controllability, the assessment sub-indicator includes at least one of the following: predictability rate, regulatory compliance, and regulatory compliance rate;

[0021] When the evaluation metric includes resilience, the evaluation sub-metric includes at least one of the following: backup rate, fallback rate, and resettable rate;

[0022] When interpretability is included in the evaluation metrics, the evaluation sub-metrics include at least one of the following: transparency, translatability, comprehensibility, accuracy of explanation, completeness of explanation, and reproducibility of explanation;

[0023] When the evaluation index includes adaptability, the evaluation sub-index includes at least one of the following: flexibility, adaptability.

[0024] In the above scheme, the first evaluation parameter for determining the trustworthiness of the network functional device based on each evaluation index and each of the at least one evaluation sub-index includes:

[0025] Obtain the first weight coefficient corresponding to each evaluation sub-indicator and the second weight coefficient corresponding to each evaluation indicator;

[0026] The first evaluation parameter for determining the trustworthiness of the network functional device is based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index.

[0027] In the above scheme, the first evaluation parameter for determining the trustworthiness of the network functional device based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index includes:

[0028] The second evaluation parameter for each indicator is determined based on the first weight coefficient corresponding to each evaluation sub-indicator and the evaluation parameter corresponding to each sub-evaluation indicator;

[0029] The first evaluation parameter determines the trustworthiness of the network functional device based on the second evaluation parameter corresponding to each indicator and the second weighting coefficient corresponding to each evaluation indicator.

[0030] The method in the above scheme further includes:

[0031] A strategy for determining the level of trustworthiness based on the first evaluation parameter; the strategy is used by the intelligent network system to grant different levels of authorization to the network functional devices.

[0032] The method in the above scheme further includes:

[0033] Determine whether the value of the first evaluation parameter is greater than or equal to a preset threshold;

[0034] If the value of the first evaluation parameter is greater than or equal to the preset threshold, the autonomous network in the intelligent network system is determined to be trusted.

[0035] If the value of the first evaluation parameter is less than the preset threshold, it is determined that the autonomous network in the intelligent network system is not trusted.

[0036] This application also provides a trustworthiness assessment device, comprising:

[0037] The acquisition unit is used to acquire application information corresponding to the network functional device to be evaluated.

[0038] The first determining unit is configured to determine at least one evaluation index corresponding to the application information based on the application information, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0039] The second determining unit is configured to determine a first evaluation parameter for the trustworthiness of the network functional device based on each evaluation index and each of the at least one evaluation sub-index; the first evaluation parameter is used by the authorized trustee to determine whether to authorize the relevant functions of the network functional device.

[0040] This application also provides an intelligent network system, including:

[0041] Memory, used to store executable instructions;

[0042] A processor, when executing executable instructions stored in the memory, implements any step of the method described above.

[0043] This application also provides a computer-readable storage medium storing executable instructions for implementing any step of the method described above when executed by a processor.

[0044] This application provides a method, apparatus, intelligent network system, and storage medium for assessing the trustworthiness of a network functional device. The method, applied to an intelligent network system, includes: acquiring application information corresponding to a network functional device to be assessed; determining at least one assessment indicator corresponding to the application information, and at least one assessment sub-indicator corresponding to each of the at least one assessment indicator; and determining a first assessment parameter for the trustworthiness of the network functional device based on each assessment indicator and each of the at least one assessment sub-indicator. The first assessment parameter is used by an authorized trustee to determine whether to authorize the relevant functions of the network functional device. By employing the scheme of this application embodiment, at least one assessment indicator corresponding to the application information and at least one assessment sub-indicator corresponding to each of the at least one assessment indicator are determined; and a first assessment parameter for the trustworthiness of the network functional device is determined based on each assessment indicator and each of the at least one assessment sub-indicator, thus achieving a quantitative assessment and objective representation and display of the trustworthiness of the network functional device. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating a method for assessing the degree of trustworthiness provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram illustrating the quantitative assessment of the trustworthiness of intelligent communication network technologies in the embodiments of this application.

[0047] Figure 3This is a schematic diagram illustrating the quantitative evaluation process of the trustworthiness of intelligent communication network technology in an embodiment of this application.

[0048] Figure 4 This is a schematic diagram of a trustworthiness assessment device according to an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of a hardware entity structure of an intelligent network system in an embodiment of this application. Detailed Implementation

[0050] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0051] With the in-depth research and widespread application of artificial intelligence technologies such as machine intelligence, the term "trust" has come into the view of technical personnel and its importance is constantly increasing. Universities, research institutions and companies around the world are investing more and more resources in the "trust" and "credibility" of intelligent technologies.

[0052] As a decision-making behavior, "trust" is influenced by past experience and related predictions about the future. The issue of "trust" in automated systems has long been a subject of psychological research. However, artificial intelligence (AI) presents a unique challenge to user "trust." AI users must "trust" AI; that is, AI must be "trustworthy," transforming the interaction between the user and the system into a relationship. "Trustworthiness" itself is a mechanism for reducing complexity; the less we understand about the technology, the more important "trust" or "trustworthiness" becomes.

[0053] In related technologies, the term "trust" is defined as: a measurable / quantifiable indicator that reflects historically accumulated trust / confidence levels and expected future trust / confidence levels.

[0054] With the development of network systems and the application of technologies such as artificial intelligence in communication networks, the following concerns or issues will arise regarding the application, management, and operation of Autonomous Networks (ANs): Should operators trust ANs? How can operators be convinced to trust ANs and confidently delegate operational authority to the network itself? If operators are willing to trust ANs, to what extent should they trust them? How can the credibility of ANs be assessed? What key factors should be used to assess their credibility? How can the credibility of ANs be quantified and evaluated?

[0055] If the above issues are not properly addressed, a series of problems and challenges will arise: If network operators lack confidence in autonomous networks (especially under pressure from network quality assessments and market competition), they will tend to adopt traditional solutions (based on manual or tool-assisted methods) for network management, preventing autonomous networks from gaining real-world application opportunities. Furthermore, from the perspective of autonomous networks, the lack of sufficient application opportunities in real-world networks deprives them of opportunities for learning, evolution, and improvement. This is detrimental to operators, solution providers, and even the entire industry's development towards network intelligence.

[0056] As machine intelligence technologies such as Artificial Intelligence (AI) are increasingly applied in network operations, the resulting instability, unreliability, and poor interpretability in the acquisition and use of software, algorithms, and data have become significant obstacles that communication operation and maintenance personnel cannot ignore in the process of using, authorizing autonomous networks, and upgrading autonomous networks themselves.

[0057] Therefore, it is necessary to conduct research on the trustworthiness assessment technology of autonomous networks as soon as possible, and to quantify and objectively represent and present the assessment results of the trustworthiness of intelligent communication network technologies. This will promote the development and evolution of autonomous networks, foster a unified understanding and assessment technology of autonomous network trustworthiness assessment in the industry, and guide the industry to determine the goals and evolution direction of autonomous networks in the process of intelligent development based on network systems with different assessment results.

[0058] Currently, in the field of intelligent communication network technology or autonomous networks, there are no relevant technologies and methods for quantitatively assessing the trustworthiness of related intelligent technologies or intelligent execution processes. This application proposes a quantitative assessment method and specific process for the trustworthiness of intelligent communication network technology, offering related solutions and methods to quantitatively assess the trustworthiness of intelligent communication network technology and thus positively promote its development and subsequent evolution.

[0059] Based on this, embodiments of this application provide a method for assessing trustworthiness, applied to an intelligent network system. The functions implemented by this method can be achieved by a processor in the intelligent network system calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the intelligent network system includes at least a processor and a storage medium. As an example, the intelligent network system may include mobile phones, computers, terminals, information transceivers, tablet devices, personal digital assistants, etc.

[0060] Figure 1This application provides a schematic diagram of a trustworthiness assessment method flow according to an embodiment of the present application; as shown Figure 1 As shown, this method is applied to an intelligent network system, and the method includes:

[0061] Step 101: Obtain the application information corresponding to the network functional device to be evaluated;

[0062] Step 102: Based on the application information, determine at least one evaluation indicator corresponding to the application information, and at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator;

[0063] Step 103: Determine a first evaluation parameter for the trustworthiness of the network functional device based on each evaluation index and each evaluation sub-index of the at least one evaluation sub-index; the first evaluation parameter is used by the authorized trustee to determine whether to authorize the relevant functions of the network functional device.

[0064] It should be noted that the intelligent network system can be determined according to the actual situation, and is not limited here. As an example, the intelligent network system can be an intelligent communication network system, which may include communication networks and autonomous networks.

[0065] In step 101, the network functional device can be determined according to the actual situation. The network functional device can be hardware or software, and there is no limitation here. As an example, the network functional device can be a network functional module or a network functional entity.

[0066] In one embodiment, obtaining the application information corresponding to the network functional device to be evaluated includes:

[0067] Obtain the assessment trigger instruction for the degree of trustworthiness;

[0068] The evaluation parameters of the network functional device are determined according to the evaluation trigger command;

[0069] The application information corresponding to the network functional device is determined based on the evaluation parameters.

[0070] It should be noted that the evaluation trigger command can be triggered manually or automatically by the intelligent network system based on the command. In addition, the command trigger can be temporary or periodic, and there is no limitation here.

[0071] The evaluation parameters are the relevant parameters that need to be configured for testing the network functional devices, and are not limited here. As an example, the evaluation parameters can be parameters related to the application scenario and / or parameters related to the application use case.

[0072] In one embodiment, the application information includes application scenario information and / or application use case information; determining the application information corresponding to the network functional device based on the evaluation parameters includes:

[0073] The application scenario information and / or application use case information are determined based on the evaluation parameters.

[0074] It should be noted that the evaluation parameters can be parameters related to the application scenario and / or parameters related to the application use case.

[0075] Determining the application information corresponding to the network functional device based on the evaluation parameters can be done by determining the application scenario information and / or application use case information corresponding to the network functional device based on parameters related to the application scenario and / or parameters related to the application use case.

[0076] In practical applications, after the trustworthiness assessment is triggered, instructions need to be sent to the network function module or network function entity being assessed to start the corresponding assessment process or work. When the trustworthiness assessment process begins, the scenarios or use cases being assessed need to be identified to determine the application scenarios or corresponding use cases of the network function module or network function entity being assessed.

[0077] In one embodiment, determining at least one evaluation metric corresponding to the application information based on the application information, and at least one evaluation sub-metric corresponding to each of the at least one evaluation metric, includes:

[0078] Based on the application scenario information and / or application use case information, at least one evaluation index corresponding to the application information is determined, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0079] It should be noted that the evaluation indicators and sub-indicators can be determined according to actual circumstances, and are not limited here. As an example, determining at least one evaluation indicator corresponding to the application information based on the application scenario information and / or application use case information can be used to obtain preset evaluation indicators; determining at least one evaluation indicator corresponding to the application information from the preset indicators based on the application scenario information and / or application use case information; determining at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator based on the application scenario information and / or application use case information can be used to obtain preset evaluation sub-indicators corresponding to each evaluation indicator; determining at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator from the preset evaluation sub-indicators corresponding to each evaluation indicator based on the application scenario information and / or application use case information. Here, both the preset evaluation indicators and preset evaluation sub-indicators can be limited according to actual circumstances, and are not limited here. As an example, the preset evaluation indicators may include accuracy, stability, controllability, reproducibility, interpretability, and adaptability; when accuracy is included as a preset evaluation indicator, the preset evaluation sub-indicators may include reproducibility rate, accuracy rate, timeliness, and effectiveness; when stability is included as a preset evaluation indicator, the preset evaluation sub-indicators may include interruption rate, accident rate, maturity, and susceptibility to deterioration; when controllability is included as a preset evaluation indicator, the preset evaluation sub-indicators may include predictability rate, regulatory compliance, and regulatory compliance rate; when reproducibility is included as a preset evaluation indicator, the preset evaluation sub-indicators may include backup rate, fallback rate, and resettable rate; when interpretability is included as a preset evaluation indicator, the preset evaluation sub-indicators may include transparency, translatability, comprehensibility, accuracy of explanation, completeness of explanation, and reproducibility of explanation.

[0080] If the preset evaluation index includes adaptability, the preset evaluation sub-index may include flexibility and adaptability.

[0081] In one embodiment, the evaluation metrics include at least one of the following: accuracy, stability, controllability, reproducibility, interpretability, and adaptability;

[0082] When the evaluation index includes accuracy, the evaluation sub-index includes at least one of the following: reproducibility, accuracy, timeliness, and effectiveness;

[0083] When stability is included in the evaluation index, the evaluation sub-index includes at least one of the following: interruption rate, accident rate, maturity, and susceptibility to degradation;

[0084] When the assessment indicator includes controllability, the assessment sub-indicator includes at least one of the following: predictability rate, regulatory compliance, and regulatory compliance rate;

[0085] When the evaluation metric includes resilience, the evaluation sub-metric includes at least one of the following: backup rate, fallback rate, and resettable rate;

[0086] When interpretability is included in the evaluation metrics, the evaluation sub-metrics include at least one of the following: transparency, translatability, comprehensibility, accuracy of explanation, completeness of explanation, and reproducibility of explanation;

[0087] When the evaluation index includes adaptability, the evaluation sub-index includes at least one of the following: flexibility, adaptability.

[0088] It should be noted that the accuracy refers to the output of relevant parameters involved in the application information; the stability refers to the information of relevant intelligent subsystems in the intelligent network system included in the application information; the controllability refers to the use modules and / or units that do not require human operation included in the application information; the resilience refers to the information of hardware and / or software other than the intelligent network system that may have an impact included in the application information; the interpretability refers to the principle or process information of some black boxes and / or intelligent algorithms in the intelligent network system included in the application information; and the adaptability refers to the information of the network functional device to be evaluated in response to external changes included in the application information.

[0089] In one embodiment, the first evaluation parameter for determining the trustworthiness of the network functional device based on each evaluation metric and each of the at least one evaluation sub-metric includes:

[0090] Obtain the first weight coefficient corresponding to each evaluation sub-indicator and the second weight coefficient corresponding to each evaluation indicator;

[0091] The first evaluation parameter for determining the trustworthiness of the network functional device is based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index.

[0092] It should be noted that both the first weighting coefficient and the second weighting coefficient can be determined according to the actual situation, and are not limited here. As an example, both the first weighting coefficient and the second weighting coefficient can be determined by preset evaluators based on the actual evaluation scenario or use case; on the other hand, they can also be determined by preset algorithms; the preset algorithms can be determined according to the actual situation.

[0093] In one embodiment, the first evaluation parameter for determining the trustworthiness of the network functional device based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index includes:

[0094] The second evaluation parameter for each indicator is determined based on the first weight coefficient corresponding to each evaluation sub-indicator and the evaluation parameter corresponding to each sub-evaluation indicator;

[0095] The first evaluation parameter determines the trustworthiness of the network functional device based on the second evaluation parameter corresponding to each indicator and the second weighting coefficient corresponding to each evaluation indicator.

[0096] It should be noted that determining the second evaluation parameter corresponding to each indicator based on the first weight coefficient corresponding to each evaluation sub-indicator and the evaluation parameter corresponding to each sub-evaluation indicator can be achieved by performing a preset operation on the first weight coefficient corresponding to each evaluation sub-indicator and the evaluation parameter corresponding to each sub-evaluation indicator to determine the second evaluation parameter corresponding to each indicator. The preset operation can be determined according to the actual situation and is not limited here. As an example, the preset operation can be a multiplication operation.

[0097] The first evaluation parameter for determining the trustworthiness of the network functional device based on the second evaluation parameter corresponding to each indicator and the second weighting coefficient corresponding to each evaluation indicator can be a preset operation performed on the second evaluation parameter corresponding to each indicator and the second weighting coefficient corresponding to each evaluation indicator to determine the first evaluation parameter for the trustworthiness of the network functional device; wherein, the preset operation can be determined according to the actual situation and is not limited here. As an example, the preset operation can be a multiplication operation.

[0098] In one embodiment, the method further includes:

[0099] Determine whether the value of the first evaluation parameter is greater than or equal to a preset threshold;

[0100] If the value of the first evaluation parameter is greater than or equal to the preset threshold, the autonomous network in the intelligent network system is determined to be trusted.

[0101] If the value of the first evaluation parameter is less than the preset threshold, it is determined that the autonomous network in the intelligent network system is not trusted.

[0102] It should be noted that the preset threshold can be determined according to the actual situation, and is not limited here. As an example, the preset threshold can be 50% or 0.5%.

[0103] The level of trust in the autonomous network within the intelligent network system can be categorized into general trust, high trust, and complete trust; general trust can also be referred to as tending trust. Generally, the higher the value of the first evaluation parameter, the higher the level of trust. In practical applications, when the autonomous network in the intelligent network system is generally trusted, the network functional device to be evaluated can be used in specific time periods and / or specific scenarios; when the autonomous network in the intelligent network system is highly trusted, the network functional device to be evaluated can be used in some time periods and / or some scenarios; when the autonomous network in the intelligent network system is completely trusted, the network functional device to be evaluated can be used in all time periods and / or all scenarios.

[0104] The distrust level of the autonomous network in the intelligent network system can be categorized into general distrust, high distrust, and complete distrust; general distrust can also be referred to as tending towards distrust. Generally, the smaller the value of the first evaluation parameter, the higher the degree of distrust. In practical applications, when the autonomous network in the intelligent network system is generally distrusted, the autonomous network needs to be optimized and / or improved; when the autonomous network in the intelligent network system is highly distrusted, the autonomous network should undergo substantial optimization; and when the autonomous network in the intelligent network system is completely distrusted, the autonomous network should be reconstructed.

[0105] The scheme adopted in this application embodiment determines at least one evaluation index corresponding to the application information of the network functional device to be evaluated, and at least one evaluation sub-index corresponding to each of the at least one evaluation index; and determines a first evaluation parameter of the trustworthiness of the network functional device based on each evaluation index and each of the at least one evaluation sub-index, thereby realizing the quantitative evaluation of the trustworthiness of the network functional device, as well as its objective representation and display.

[0106] To better understand, this example illustrates a method for assessing the trustworthiness of intelligent communication network technologies, specifically a quantitative assessment method for the trustworthiness of such technologies. Figure 2 This is a schematic diagram illustrating the quantitative assessment of the trustworthiness of intelligent communication network technologies in the embodiments of this application. Figure 3 This is a schematic diagram illustrating the quantitative evaluation process of the trustworthiness of intelligent communication network technology in this application embodiment; the following can be combined with Figure 2 and Figure 3 To understand.

[0107] To better understand the embodiments of this application, the relevant terms involved in these embodiments are defined as follows:

[0108] Trustworthiness level: The objective and quantifiable degree to which application modules or functional entities related to intelligent communication network technologies can be trusted; that is, the trustworthiness level itself should be an assessable, quantifiable, and intuitively readable constant. Evaluation indicators: In the trustworthiness level evaluation process, indicator parameters need to be considered and determined in the first stage based on the specific scenarios or use cases of trustworthiness level evaluation. Once determined, these parameters will allow for the further selection of more refined evaluation parameters (evaluation sub-indicators) in subsequent evaluations, as well as the subsequent development and deepening of the evaluation work.

[0109] Evaluation sub-indicators: In the process of trustworthiness assessment, after the evaluation indicators are confirmed according to the specific scenario or use case, each evaluation indicator needs to determine its own evaluation sub-indicators according to the specific scenario and use case, so that the evaluation of the evaluation indicators can be decomposed in a more granular dimension, making the evaluation results more objective and accurate.

[0110] The trustee: An entity with authorization authority (including but not limited to network operation and maintenance personnel, management personnel, etc.). The trustee may decide whether to authorize relevant functions for intelligent technology application modules and functional entities based on the degree of trustworthiness of the network intelligent technology.

[0111] The following details each step of the trustworthiness quantitative assessment process. This can be combined with... Figure 3 To understand.

[0112] Step 1: The trustworthiness assessment can be triggered manually or by command. Command-based triggers can be temporary or periodic. The trustworthiness assessment is a necessary step to obtain the trustworthiness assessment results.

[0113] The second step is to issue instructions to the network function module or network function entity being evaluated after the trustworthiness assessment is triggered, so as to start the corresponding assessment process or work.

[0114] Step 3: At the start of the trust assessment process, it is necessary to identify the scenarios or use cases to be assessed, and determine the application scenarios or corresponding use cases of the network function modules or network function entities to be assessed, so that the corresponding assessment indicators and assessment sub-indicators can be determined according to the corresponding scenarios or use cases in subsequent processes.

[0115] Step 4: After identifying the scenarios or use cases for credibility assessment, the subsequent assessment indicators can be determined / selected based on the information of the scenarios or use cases.

[0116] Step 5: During the trustworthiness assessment process, the corresponding assessment metrics need to be determined / selected based on the established scenario or use case information. Since different application scenarios or use cases have different requirements for trustworthiness assessment metrics and sub-metrics, determining the corresponding assessment metrics and their sub-metrics based on the application scenario or use case is a necessary step and process. This application suggests selecting from the following "Assessment Metrics": accuracy, stability, controllability, reproducibility, interpretability, and adaptability.

[0117] Step 6: After determining the evaluation indicators, the corresponding evaluation sub-indicators need to be selected for each evaluation indicator determined in Step 5, based on the determined scenario information or use case information.

[0118] Step 7: Selection of evaluation sub-indicators. Based on the scenario or use case information determined during the trustworthiness assessment process, the evaluation sub-indicators for each evaluation indicator in Step 5 need to be determined / selected. In practical applications, since different application scenarios or use cases have different requirements for trustworthiness evaluation indicators and corresponding evaluation sub-indicators, determining the corresponding evaluation indicators and their evaluation sub-indicators based on the application scenario or use case is a necessary step and process. The selection and correspondence of evaluation sub-indicators can be referred to in Table 1, which shows the correspondence between evaluation indicators and evaluation sub-indicators, as shown in Table 1 below.

[0119]

[0120] Table 1

[0121] Step 8: After selecting / determining the evaluation sub-indicators based on the scenario information or use case information, it is necessary to conduct corresponding evaluations and assessments on all selected / determined evaluation sub-indicators.

[0122] Step 9: Sub-indicator evaluation is a crucial step in trustworthiness assessment. First, it's necessary to determine whether the sub-indicator evaluation will be conducted manually or automatically through an evaluation system, based on the actual situation. Then, each sub-indicator identified / selected in Step 7 is evaluated individually to derive relevant quantitative values. In practical applications, the quantitative values ​​of the sub-indicators and the range of values ​​for the evaluation indicators can be set / defined according to the actual situation, such as [0,1], [0,100], etc. However, all indicators and sub-indicators in the same evaluation process should maintain the same numerical range and other mathematical expressions.

[0123] Step 10: In the sub-indicator evaluation phase, all selected / determined evaluation tasks / processes should be completed, and the evaluation results of all selected / determined evaluation sub-indicators should be obtained.

[0124] Step 11: After the evaluation of the sub-indicators is completed, all evaluation sub-indicators should be quantified in the same way and with the same numerical range. Then, they will be used as weighted variables for subsequent evaluation indicators.

[0125] Step 12: After the evaluation of the sub-indicators is completed, a weighted calculation needs to be performed on each selected / determined evaluation indicator. In practical applications, the weighting coefficients in the weighting calculation process of the evaluation indicators can be determined manually, or they can be determined by a corresponding algorithm.

[0126] Step 13: The evaluation results of the evaluation sub-indicators are used as calculation parameters for the evaluation indicators in the weighted calculation of the evaluation sub-indicators. The weight parameters of each evaluation indicator should be determined by the evaluator or algorithm according to the actual evaluation scenario or use case. The calculation method is as follows: Formula (1):

[0127]

[0128] Step 14: After the evaluation indicators in Step 13 are calculated, the trustworthiness of all evaluation indicator results must be calculated.

[0129] Step 15: The calculation of trustworthiness requires determining the weight of each evaluation indicator based on the actual scenario or use case. The calculation method is as follows: Formula (2):

[0130]

[0131] Step 16: The weighted calculation result of the trustworthiness level will be given to the trustee for subsequent authorization and other related decisions and actions.

[0132] Step 17: For the trustee, the degree of trustworthiness will serve as an important reference for subsequent decisions (such as whether to authorize).

[0133] Step 18: Similar to Step 17, for the trustee, credibility will serve as an important reference for subsequent decisions (such as whether to authorize).

[0134] Step 19: The trustee needs to authorize the execution of subsequent intelligent technologies to varying degrees based on the assessment results of the trustworthiness level, the reference range of credibility of relevant scenarios or use cases, and the trustee's professional judgment.

[0135] In step nineteen, which can be understood as a detailed description of trustworthiness judgment / decision, based on the degree of trustworthiness and combined with its own professional knowledge, the trustee needs to make relevant judgments and decisions on whether the assessed object can be authorized to perform or to a certain extent authorized to perform, so that the intelligent technology can subsequently run / execute relevant operations or instructions.

[0136] In practical applications, it is recommended that the results of the trustworthiness degree be in the same range as the quantitative values ​​of the evaluation sub-indicators and the values ​​of the evaluation indicators, such as [0,1], [0,100], etc. For the same evaluation process, all evaluation indicators, evaluation sub-indicators, and trustworthiness degree results can maintain the same numerical range and other mathematical expressions.

[0137] Step 20: Trust assessment itself can be a one-time event, or it can be repeated multiple times or periodically. The specific frequency, period, and number of assessments can be set / configured according to the specific scenario or use case, the needs of the trustee, etc.

[0138] This application proposes a quantification of the trustworthiness of intelligent technologies and puts forward corresponding evaluation processes and methods. Compared with other methods, this application is more feasible and objective, and can effectively solve the current problems of quantifying and objectifying the "trust" of intelligent technologies. It provides an objective basis for relevant decisions and authorizations and has good practical commercial value and potential for standardization.

[0139] To implement the method of the embodiments of this application, the embodiments of this application also provide a trustworthiness assessment device 400, applied to an intelligent network system. Figure 4 This is a schematic diagram of a trustworthiness assessment device according to an embodiment of this application; as shown. Figure 4 As shown, it includes:

[0140] Acquisition unit 401 is used to acquire application information corresponding to the network functional device to be evaluated;

[0141] The first determining unit 402 is used to determine at least one evaluation index corresponding to the application information based on the application information, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0142] The second determining unit 403 is used to determine a first evaluation parameter of the trustworthiness of the network function device based on each evaluation index and each evaluation sub-index of the at least one evaluation sub-index; the first evaluation parameter is used by the authorized trustee to determine whether to authorize the relevant functions of the network function device.

[0143] In one embodiment, the acquisition unit 401 is further configured to acquire the assessment trigger instruction for the trustworthiness level; determine the evaluation parameters of the network functional device based on the assessment trigger instruction; and determine the application information corresponding to the network functional device based on the evaluation parameters.

[0144] In one embodiment, the application information includes application scenario information and / or application use case information; the acquisition unit 401 is further configured to determine the application scenario information and / or application use case information based on the evaluation parameters.

[0145] In one embodiment, the first determining unit 402 is further configured to determine at least one evaluation index corresponding to the application information based on the application scenario information and / or application use case information, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0146] In one embodiment, the application information includes application scenario information and / or application use case information; the acquisition unit 401 is further configured to determine the application scenario information and / or application use case information based on the evaluation parameters.

[0147] In one embodiment, the first determining unit 402 is further configured to determine at least one evaluation index corresponding to the application information based on the application scenario information and / or application use case information, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

[0148] Here, in one embodiment, the evaluation metrics include at least one of the following: accuracy, stability, controllability, reproducibility, interpretability, and adaptability;

[0149] When the evaluation index includes accuracy, the evaluation sub-index includes at least one of the following: reproducibility, accuracy, timeliness, and effectiveness;

[0150] When stability is included in the evaluation index, the evaluation sub-index includes at least one of the following: interruption rate, accident rate, maturity, and susceptibility to degradation;

[0151] When the assessment indicator includes controllability, the assessment sub-indicator includes at least one of the following: predictability rate, regulatory compliance, and regulatory compliance rate;

[0152] When the evaluation metric includes resilience, the evaluation sub-metric includes at least one of the following: backup rate, fallback rate, and resettable rate;

[0153] When interpretability is included in the evaluation metrics, the evaluation sub-metrics include at least one of the following: transparency, translatability, comprehensibility, accuracy of explanation, completeness of explanation, and reproducibility of explanation;

[0154] When the evaluation index includes adaptability, the evaluation sub-index includes at least one of the following: flexibility, adaptability.

[0155] In one embodiment, the second determining unit 403 is further configured to obtain a first weight coefficient corresponding to each evaluation sub-index and a second weight coefficient corresponding to each evaluation index; and to determine a first evaluation parameter for the trustworthiness of the network functional device based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index.

[0156] In one embodiment, the second determining unit 403 is further configured to determine a second evaluation parameter corresponding to each indicator based on a first weight coefficient corresponding to each evaluation sub-indicator and an evaluation parameter corresponding to each sub-evaluation indicator; and to determine a first evaluation parameter for the trustworthiness of the network functional device based on the second evaluation parameter corresponding to each indicator and a second weight coefficient corresponding to each evaluation indicator.

[0157] In one embodiment, the determining unit 403 is further configured to determine a strategy for the degree of trustworthiness based on the first evaluation parameters; the strategy is used by the intelligent network system to grant different levels of authorization to the network functional devices.

[0158] In one embodiment, the determining unit 403 is further configured to determine whether the value of the first evaluation parameter is greater than or equal to a preset threshold; if the value of the first evaluation parameter is greater than or equal to the preset threshold, determine that the autonomous network in the intelligent network system is trusted; if the value of the first evaluation parameter is less than the preset threshold, determine that the autonomous network in the intelligent network system is not trusted.

[0159] It should be noted that the trustworthiness assessment device provided in the above embodiments is only illustrated by the division of the above program modules when assessing trustworthiness. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the trustworthiness assessment device and the trustworthiness assessment method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0160] Based on the hardware implementation of the above program modules, this application embodiment also provides an intelligent network system, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps in the trustworthiness assessment method provided in the above embodiments.

[0161] Correspondingly, embodiments of this application provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps in the trustworthiness assessment method provided in the above embodiments.

[0162] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0163] It should be noted that, Figure 5 This is a schematic diagram of a hardware entity structure of an intelligent network system in an embodiment of this application, such as... Figure 5 As shown, the hardware entity of the intelligent network system 500 includes a processor 501 and a memory 503. Optionally, the intelligent network system 500 may also include a communication interface 502.

[0164] It is understood that memory 503 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 503 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0165] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 501 or by instructions in the form of software. The processor 501 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 501 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 503. The processor 501 reads the information in the memory 503 and combines its hardware to complete the steps of the aforementioned method.

[0166] In an exemplary embodiment, the device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0167] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0168] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0169] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0170] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0171] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0172] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for assessing the degree of trustworthiness, characterized in that, Applied to intelligent network systems, the method includes: The command to obtain an assessment of trustworthiness is triggered. The evaluation parameters of the network functional device are determined according to the evaluation trigger command; The application information corresponding to the network functional device to be evaluated is determined based on the evaluation parameters; the evaluation parameters are parameters related to the application scenario and / or parameters related to the application use case; the application information includes application scenario information and / or application use case information. Based on the application information, at least one evaluation index corresponding to the application information is determined, and at least one evaluation sub-index corresponding to each of the at least one evaluation index; the evaluation index includes at least one of the following: accuracy, stability, controllability, reproducibility, interpretability, and adaptability; A first evaluation parameter is used to determine the trustworthiness of the network function device based on each evaluation index and each of the at least one evaluation sub-index; the first evaluation parameter is used by an authorized trustee to determine whether to authorize the relevant functions of the network function device.

2. The method according to claim 1, characterized in that, The step of determining the application information corresponding to the network functional device based on the evaluation parameters includes: The application scenario information and / or application use case information are determined based on the evaluation parameters.

3. The method according to claim 2, characterized in that, Based on the application information, at least one evaluation indicator corresponding to the application information is determined, and at least one evaluation sub-indicator corresponding to each of the at least one evaluation indicator, including: Based on the application scenario information and / or application use case information, at least one evaluation index corresponding to the application information is determined, and at least one evaluation sub-index corresponding to each of the at least one evaluation index.

4. The method according to claim 1, characterized in that, When the evaluation index includes accuracy, the evaluation sub-index includes at least one of the following: reproducibility, accuracy, timeliness, and effectiveness; When stability is included in the evaluation index, the evaluation sub-index includes at least one of the following: interruption rate, accident rate, maturity, and susceptibility to degradation; When the assessment indicator includes controllability, the assessment sub-indicator includes at least one of the following: predictability rate, regulatory compliance, and regulatory compliance rate; When the evaluation metric includes resilience, the evaluation sub-metric includes at least one of the following: backup rate, fallback rate, and resettable rate; When interpretability is included in the evaluation metrics, the evaluation sub-metrics include at least one of the following: transparency, translatability, comprehensibility, accuracy of explanation, completeness of explanation, and reproducibility of explanation; When the evaluation index includes adaptability, the evaluation sub-index includes at least one of the following: flexibility, adaptability.

5. The method according to claim 1, characterized in that, The first evaluation parameter for determining the trustworthiness of the network functional device based on each evaluation index and each of the at least one evaluation sub-index includes: Obtain the first weight coefficient corresponding to each evaluation sub-indicator and the second weight coefficient corresponding to each evaluation indicator; The first evaluation parameter for determining the trustworthiness of the network functional device is based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index.

6. The method according to claim 5, characterized in that, The first evaluation parameter for determining the trustworthiness of the network functional device based on the first weight coefficient corresponding to each evaluation sub-index, the second weight coefficient corresponding to each evaluation index, and each sub-evaluation index includes: The second evaluation parameter for each indicator is determined based on the first weight coefficient corresponding to each evaluation sub-indicator and the evaluation parameter corresponding to each sub-evaluation indicator; The first evaluation parameter determines the trustworthiness of the network functional device based on the second evaluation parameter corresponding to each indicator and the second weighting coefficient corresponding to each evaluation indicator.

7. The method according to claim 1, characterized in that, The method further includes: A strategy for determining the level of trustworthiness based on the first evaluation parameter; the strategy is used by the intelligent network system to grant different levels of authorization to the network functional devices.

8. The method according to any one of claims 1-6, characterized in that, The method further includes: Determine whether the value of the first evaluation parameter is greater than or equal to a preset threshold; If the value of the first evaluation parameter is greater than or equal to the preset threshold, the autonomous network in the intelligent network system is determined to be trusted. If the value of the first evaluation parameter is less than the preset threshold, it is determined that the autonomous network in the intelligent network system is not trusted.

9. A trustworthiness assessment device, characterized in that, Applied to intelligent network systems, including: An acquisition unit is configured to acquire an assessment trigger instruction for trustworthiness; determine the evaluation parameters of the network functional device based on the assessment trigger instruction; determine the application information corresponding to the network functional device to be evaluated based on the evaluation parameters; the evaluation parameters are application scenario-related parameters and / or application use case-related parameters; the application information includes application scenario information and / or application use case information. The first determining unit is configured to determine at least one evaluation index corresponding to the application information based on the application information, and at least one evaluation sub-index corresponding to each of the at least one evaluation index; the evaluation index includes at least one of the following: accuracy, stability, controllability, reproducibility, interpretability, and adaptability; The second determining unit is configured to determine a first evaluation parameter for the trustworthiness of the network functional device based on each evaluation index and each of the at least one evaluation sub-index; the first evaluation parameter is used by the authorized trustee to determine whether to authorize the relevant functions of the network functional device.

10. An intelligent network system, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the trustworthiness assessment method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, It stores executable instructions for implementing, when executed by a processor, the method for assessing the degree of trustworthiness as described in any one of claims 1 to 8.

Images