A method and system for quantitative evaluation of knowledge systems

Abstract

The application provides a quantitative evaluation method for a knowledge system, which comprises evaluation of a primary index and a secondary index, the secondary index comprising a representation category, a knowledge granularity, a knowledge connotation, a knowledge entropy before fusion, a knowledge entropy after fusion, a decay factor, a system throughput, a reasoning real-time performance, a business diversity and a business effectiveness; the primary index comprising a representation capability, an understandability, an easy operability and a business support capability, the representation capability being calculated based on the representation category, the knowledge granularity and the knowledge connotation, and providing a selection basis for a knowledge representation method of a representation layer of the knowledge system, the understandability being calculated based on the knowledge entropy before fusion and the knowledge entropy after fusion, and providing a selection basis for a knowledge density of the representation layer of the knowledge system, the easy operability being calculated based on the decay factor and the system throughput, and providing a parameter configuration basis for a management layer of the knowledge system, and the business support capability being calculated based on the reasoning real-time performance, the business diversity and the business effectiveness, and providing a parameter configuration basis for an application layer of the knowledge system.

Description

Technical Field

[0001] This invention relates to the field of evaluation technology, and more specifically, to a method and system for quantitative evaluation of knowledge systems. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technologies such as knowledge graphs and deep learning, the intelligent decision-making capabilities of knowledge systems have been continuously improved, leading to their widespread application in systems such as communication networks, urban power, and intelligent transportation. Knowledge systems provide intelligent decision-making capabilities for various scenarios through domain expert knowledge and typically include multiple functional modules such as knowledge bases, computing engines, application services, and performance monitoring. Among these, knowledge system evaluation technology provides a unified metric for system performance and is crucial for the autonomous optimization of knowledge systems.

[0003] Current knowledge system evaluation techniques often measure only a single aspect of the system, resulting in poor comprehensiveness of evaluation indicators. Furthermore, existing evaluation methods frequently include numerous qualitative indicators, making them susceptible to the subjective influence of evaluators. They also have long evaluation cycles, making it difficult to provide unified quantitative standards for different knowledge systems and reducing the efficiency of feedback and optimization. Summary of the Invention

[0004] To address the problems of poor comprehensiveness and insufficient quantification of existing knowledge system evaluation indicators, this invention proposes a quantitative evaluation method and system for knowledge systems.

[0005] According to one aspect of the present invention, a quantitative evaluation method for a knowledge system is provided, the quantitative evaluation method comprising:

[0006] The evaluation criteria for the knowledge system include secondary indicators such as representation category, knowledge granularity, knowledge connotation, knowledge entropy before fusion, knowledge entropy after fusion, decay factor, system throughput, inference real-time performance, business diversity, and business effectiveness; and

[0007] The primary indicators for evaluating a knowledge system include representation capability, understandability, ease of use, and business support capability.

[0008] Among them, the characterization category E t for:

[0009]

[0010] Where |·| is the cardinality of the set, and δ(n) is a binary function of n, with a value of 1 when the knowledge representation method supports the representation of n, and 0 otherwise. t For knowledge category set The elements in

[0011] Knowledge Granularity E g for:

[0012]

[0013] Where, n g For knowledge granularity set The elements in For n g The i-th necessary component of granularity, N g For n g The number of necessary constituent elements for granularity.

[0014] Knowledge Content E r for:

[0015]

[0016] The capability Ec is represented as:

[0017] Ec = π t E t +π g E g +π r E r , where π t , π g , π r Representing category E t Knowledge Granularity E g and the connotation of knowledge E r The weight,

[0018] Fusion of pre-knowledge entropy for:

[0019]

[0020] in, This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist Equivalence classes of the upper partition

[0021] Knowledge entropy after fusion for:

[0022]

[0023] in, It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist Equivalence classes of the partition,

[0024] The comprehensibility Re is:

[0025]

[0026] The system throughput TP is:

[0027]

[0028] in, The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. T represents the average number of requests for operation p. p Let p be the average response time.

[0029] Attenuation factor γ e for:

[0030]

[0031] Among them, TP m The maximum throughput of the knowledge system is determined through multiple measurements; ∈ represents the tolerance factor.

[0032] Ease of use (Em) is:

[0033]

[0034] Real-time reasoning S c for:

[0035]

[0036] Where t is the actual inference time, t tol For reference reasoning time,

[0037] Business Diversity s for:

[0038]

[0039] Where N represents the actual types of business support, N tol For reference regarding business types,

[0040] Business effectiveness S e for:

[0041]

[0042] in, Perform the total number of times for business i. The number of times a business transaction was successfully completed.

[0043] Business support capability Sa is:

[0044] Sa = π c S c +π s S s +π e S e

[0045] Where, π c , π s , π e S represents the real-time performance of inference. c Business Diversity s and business effectiveness S e The weight.

[0046] The knowledge system comprises a presentation layer, a management layer, and an application layer. The presentation capability Ec provides the basis for configuring knowledge density and knowledge representation methods for the presentation layer of the knowledge system. The understandability Re provides the basis for configuring knowledge density for the presentation layer of the knowledge system. The operability Em provides the basis for configuring the number of virtual nodes, backups, and reporting frequency for the management layer of the knowledge system. The business support capability Sa provides the basis for configuring the inference frequency and inference chain length for the application layer of the knowledge system.

[0047] weight π t , π g , π r and π c , π s , π e Calculated using the entropy method.

[0048] According to another aspect of the present invention, a quantitative evaluation system for a knowledge system is provided, the quantitative evaluation system comprising:

[0049] The primary indicator evaluation module includes a representation capability evaluation module, a comprehensibility evaluation module, an operability evaluation module, and a business support capability evaluation module; and

[0050] The secondary indicator evaluation module includes a representation category evaluation module, a knowledge granularity evaluation module, a knowledge connotation evaluation module, a pre-fusion knowledge entropy evaluation module, a post-fusion knowledge entropy evaluation module, a decay factor evaluation module, a system throughput evaluation module, an inference real-time evaluation module, a business diversity evaluation module, and a business effectiveness evaluation module.

[0051] Among them, the representation capability assessment module is used to assess representation capability (Ec), the comprehensibility assessment module is used to assess comprehensibility (Re), the operability assessment module is used to assess operability (Em), and the business support capability assessment module is used to assess business support capability (Sa).

[0052] The characterization category evaluation module is used to evaluate characterization category E. tThe knowledge granularity assessment module is used to assess the knowledge granularity E. g The knowledge content assessment module is used to assess the knowledge content E. r The pre-fusion knowledge entropy assessment module is used to assess the pre-fusion knowledge entropy. The merged knowledge entropy evaluation module is used to evaluate the merged knowledge entropy. The attenuation factor evaluation module is used to evaluate the attenuation factor γ. e The system throughput evaluation module is used to evaluate system throughput (TP), and the inference real-time performance evaluation module is used to evaluate inference real-time performance (S). c The business diversity assessment module is used to assess business diversity S s The business effectiveness assessment module is used to assess business effectiveness.

[0053] Among them, the characterization category E t for:

[0054]

[0055] Where |·| is the cardinality of the set, and δ(n) is a binary function of n, with a value of 1 when the knowledge representation method supports the representation of n, and 0 otherwise. t For knowledge category set The elements in

[0056] Knowledge Granularity E g for:

[0057]

[0058] Where, n g For knowledge granularity set The elements in For n g The i-th necessary component of granularity, N g For n g The number of necessary constituent elements for granularity.

[0059] Knowledge Content E r for:

[0060]

[0061] The capability Ec is represented as:

[0062] Ec = π t E t +π g E g +π r E r , where π t , π g , π r Representing category Et Knowledge Granularity E g and the connotation of knowledge E r The weight,

[0063] Fusion of pre-knowledge entropy for:

[0064]

[0065] in, This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist Equivalence classes of the upper partition

[0066] Knowledge entropy after fusion for:

[0067]

[0068] in, It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist Equivalence classes of the partition,

[0069] The comprehensibility Re is:

[0070]

[0071] The system throughput TP is:

[0072]

[0073] in, The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. T represents the average number of requests for operation p. p Let p be the average response time.

[0074] Attenuation factor γ e for:

[0075]

[0076] Among them, TP m The maximum throughput of the knowledge system is determined through multiple measurements; ∈ represents the tolerance factor.

[0077] Ease of use (Em) is:

[0078]

[0079] Real-time reasoning S c for:

[0080]

[0081] Where t is the actual inference time, t tol For reference reasoning time,

[0082] Business Diversity s for:

[0083]

[0084] Where N represents the actual types of business support, N tol For reference regarding business types,

[0085] Business effectiveness S e for:

[0086]

[0087] in, Perform the total number of times for business i. The number of times a business transaction was successfully completed.

[0088] Business support capability Sa is:

[0089] Sa = π c S c +π s S s +π e S e

[0090] Where, π c , π s , π e S represents the real-time performance of inference. c Business Diversity s and business effectiveness S e The weight.

[0091] According to another aspect of the present invention, a computer program product encoded on a computer storage medium is provided, comprising instructions that, when executed by one or more computers, cause one or more computers to perform the above-described method. Attached Figure Description

[0092] Figure 1 A block diagram of a knowledge system quantitative evaluation system according to an embodiment of the present invention is shown;

[0093] Figure 2A flowchart illustrating a quantitative evaluation method for a knowledge system according to an embodiment of the present invention is shown;

[0094] Figure 3 It is a curve showing how the comprehensibility index changes with the increase in knowledge density;

[0095] Figure 4 The relationship between system throughput (logarithm) and operability index under different attenuation factors;

[0096] Figure 5 It is the curve showing how the operability index changes with the parameters of the system management layer; and

[0097] Figure 6 It is a curve showing how business support capability indicators change with application layer parameters. Detailed Implementation

[0098] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0099] Figure 1 A block diagram of a knowledge system quantitative evaluation system according to an embodiment of the present invention is shown.

[0100] The knowledge system serves as the evaluation object of the quantitative evaluation system for knowledge systems. Based on functionality, the knowledge system is divided into three layers from bottom to top: the presentation layer, the management layer, and the application layer. The presentation layer uses knowledge representation methods to organize multi-source heterogeneous data into a unified form, storing it in the knowledge base at each data source node. The management layer is responsible for regularly updating and merging the multi-point distributed knowledge base and responding to user query requests. The application layer adapts relevant knowledge to user business needs, utilizes knowledge for autonomous decision-making, and outputs system configuration strategies.

[0101] The quantitative evaluation system for knowledge systems of the present invention includes a primary indicator evaluation module and a secondary indicator evaluation module.

[0102] The primary indicator evaluation modules of the quantitative evaluation system for the knowledge system include a representation capability evaluation module, a comprehensibility evaluation module, an operability evaluation module, and a business support capability evaluation module. Specifically, the representation capability evaluation module assesses representation capability, the comprehensibility evaluation module assesses comprehensibility, the operability evaluation module assesses operability, and the business support capability evaluation module assesses business support capability. Representation capability and comprehensibility are evaluation indicators for the representation layer of the knowledge system, enabling a quantitative assessment of the capabilities of single or multiple knowledge representation technologies employed at the representation layer; operability and business support capability provide quantitative standards for the management and application layers of the knowledge system, respectively.

[0103] The secondary indicator evaluation modules of the quantitative evaluation system for the knowledge system include a representation category evaluation module, a knowledge granularity evaluation module, a knowledge connotation evaluation module, a pre-fusion knowledge entropy evaluation module, a post-fusion knowledge entropy evaluation module, a decay factor evaluation module, a system throughput evaluation module, a reasoning real-time evaluation module, a business diversity evaluation module, and a business effectiveness evaluation module.

[0104] The representation capability assessment module calculates the representation capability Ec of the knowledge representation method, providing a basis for selecting knowledge density and knowledge representation methods for the representation layer of the knowledge system.

[0105] The representational capability Ec of the representation layer of a knowledge system can be determined by the representation category E. t Knowledge Granularity E g With knowledge connotation E r A total of three secondary indicators are used for comprehensive measurement, and the expression is as follows:

[0106] Ec = π t E t +π g E g +π r E r

[0107] Where, π t , π g , π r The weight is the corresponding secondary indicator; the larger the weight, the more important the secondary indicator.

[0108] The representation category E is evaluated by the representation category evaluation module of the secondary indicator evaluation module. t The knowledge granularity E is evaluated by the knowledge granularity evaluation module. g The knowledge content E is evaluated by the knowledge content assessment module. r .

[0109] Characterization category E t It is a standardized measure of the types of knowledge that can be represented by the knowledge representation methods of a knowledge system, and its expression is:

[0110]

[0111] Where |·| is the cardinality of the set, and δ(n) is a binary function of n, with a value of 1 when the knowledge representation method supports the representation of n, and 0 otherwise. t For knowledge category set The elements in the text. Preferably, the types of knowledge include declarative knowledge and procedural knowledge, that is... Declarative knowledge is an objective description of the internal state and external environment of a system, while procedural knowledge is a process-oriented and logical expression of events inside and outside the system.

[0112] If the knowledge graph used by the knowledge system supports the representation of declarative and procedural knowledge, and the superscript (n) represents the actual evaluation value of the indicator, then

[0113] Knowledge Granularity E g It measures the ability of knowledge representation methods in a knowledge system to express multi-granularity knowledge, and its expression is:

[0114]

[0115] Where, n g For knowledge granularity set The elements in For n g The i-th necessary component of granularity, N g For n g The number of necessary components for granularity. Preferably, based on the degree of abstraction, knowledge granularity is divided into instance knowledge, conceptual knowledge, event knowledge, and principle knowledge, i.e. Instance knowledge refers to specific phenomena occurring during the operation of an actual system, representing the finest-grained unit of knowledge. The essential components of instance knowledge are instance object identifier, instance object attribute key, instance object attribute value, and relationships between instance objects. Conceptual knowledge is an abstract description of the essential attributes of instance objects, formed by summarizing and generalizing instance knowledge. The essential components of conceptual knowledge are concept identifier, concept connotation, and concept extension. Connotation refers to the meaning of a concept, constituted by the unique attributes of the objective object reflected by the concept; extension refers to the objects or scope to which the concept applies, constituted by the objective object referred to by the concept. Event knowledge reflects behaviors occurring in an actual system, formed by the participation of one or more roles in a specific time and space, and possessing specific action characteristics. The essential components of event knowledge are time, space, instance object, and behavior. Principle knowledge describes the regular logical relationships between events, obtained through the analysis and summarization of event knowledge. Principle knowledge does not depend on specific contexts or instance subjects, but reflects the underlying principles and rules, representing the coarsest-grained unit of knowledge. The essential components of principle knowledge are principle identifier, event, and logical relationship.

[0116] If the knowledge graph used by the knowledge system supports the representation of instance knowledge, conceptual knowledge, event knowledge, and logical knowledge to varying degrees. Specifically, for instance knowledge, it supports representing instance object identifiers, instance object attribute keys, instance object attribute values, and relationships between instance objects; for conceptual knowledge, it supports representing concept identifiers and concept extensions, but not concept connotations; for event knowledge, it supports representing instance objects and behaviors, but not time and space; for logical knowledge, it supports representing events and logical relationships, but not logical identifiers. Therefore,

[0117] Knowledge Content E r It is a measure of the ability of a knowledge system's knowledge representation method to express knowledge with multiple connotations, expressed as:

[0118]

[0119] Where, n r For the collection of knowledge content The elements in. Preferably, knowledge is divided into semantic knowledge and mathematical knowledge from the perspective of connotation, that is... Semantic knowledge consists of objective facts described by language and is presented by a vocabulary system and a semantic system, while mathematical knowledge is described by a digital logic model between concepts.

[0120] If the knowledge graph used by the knowledge system supports the representation of semantic knowledge but not the representation of mathematical knowledge, then

[0121] The weights are calculated using the entropy method. Therefore, the representational power of a knowledge graph is Ec (n) =0.756.

[0122] Using the same methodology, various knowledge representation methods were evaluated. Data tables and time series diagrams only support single-category, single-content knowledge representation, with representation capabilities of 0.424 and 0.439, respectively. The representation method combining data tables and production rules can further represent procedural knowledge, supporting knowledge representation at the event and instance granularity, achieving a representation capability of 0.680. The comparison shows that the representation capability of the knowledge system's representation layer is higher than other representation methods, making it a preferred option.

[0123] The understandability assessment module calculates the understandability Re of the knowledge representation method, providing a basis for selecting the knowledge density of the knowledge system representation layer.

[0124] The understandability (Re) of the representation layer of a knowledge system is a measure of how well a node in the system understands the knowledge assembled by other nodes. It is measured by the change in knowledge entropy before and after knowledge fusion, and is expressed as:

[0125]

[0126] in, and These represent the knowledge entropy before and after fusion, respectively.

[0127] Assessed by the pre-fusion knowledge entropy assessment module, The evaluation is conducted by the merged knowledge entropy evaluation module.

[0128] Preferably, knowledge entropy uses the entropy reduction introduced by knowledge into the system as a measure of the amount of knowledge, reflecting the ability of reasoning knowledge, event knowledge, and conceptual knowledge to distinguish instance knowledge.

[0129] Fusion of pre-knowledge entropy Calculated using the following formula:

[0130]

[0131] in, This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist The equivalence classes of the partition.

[0132] Knowledge entropy after fusion Calculated using the following formula:

[0133]

[0134] in, It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist The equivalence classes of the partition.

[0135] Knowledge entropy and knowledge density Regarding knowledge density, it is defined as the average degree of knowledge in instances. The knowledge entropy is measured before and after knowledge fusion for different nodes with different densities of knowledge, yielding... Figure 3 The comprehensibility index is shown. As the knowledge density increment increases, the knowledge entropy reduction obtained by the node after knowledge fusion continuously increases, the comprehensibility of the node for the network knowledge graph improves, and reaches 1 at the incremental saturation density point.

[0136] The operability assessment module measures the performance of the knowledge system management layer under different parameter configurations, evaluates operability (Em), and provides a basis for parameter configuration for the management layer.

[0137] Ease of operation (Em) is a quantitative evaluation indicator of the capabilities of the knowledge system's management layer. It verifies the efficiency of concurrent operations such as building, integrating, storing, and querying within the knowledge system. The expression is:

[0138]

[0139] Where, γ e TP is the attenuation factor, and TP is the system throughput, defined as the number of concurrent requests processed by the system per unit time.

[0140] System throughput TP is evaluated by the system throughput evaluation module, using the following formula:

[0141]

[0142] in, The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. T represents the average number of requests for operation p. p Let p be the average response time of operation p.

[0143] Attenuation factor γ e Evaluated by the attenuation factor evaluation module.

[0144] Attenuation factor γ e Calculated using the following formula:

[0145]

[0146] Among them, TP m The maximum throughput of the knowledge system is determined through multiple measurements; ∈ is the tolerance factor, with an optimal value range of 10. -4 ~10 -5 When the system throughput reaches its maximum value TP m At this point, the operability index reaches its maximum value of 1-∈. When performing multiple measurements and evaluations on the same system, ∈ is fixed to the same value.

[0147] Figure 4 For ∈ = 9.08 × 10 -5 The relationship between the knowledge system throughput (logarithm) and operability index under different attenuation factors is shown, where the four curve colors represent four attenuation factor values. It can be seen that when the tolerance factor is fixed, the attenuation factor is inversely proportional to the system's maximum throughput.

[0148] The parameter settings of the knowledge system's management layer will affect the operability index. Figure 5 This paper examines the relationship between three parameters—the number of virtual nodes, the number of backups, and the reporting frequency—and the operability index. It shows that as the number of virtual nodes, backups, and reporting frequency increase, the system's operability initially rises gradually, then gradually declines after exceeding the optimal parameter point. This is because, when the number of virtual nodes, backups, and reporting frequency increase from lower values, the knowledge storage distribution is relatively uniform, and concurrent operations remain within the system's processing capacity. However, after the number of virtual nodes, backups, and reporting frequency exceed the optimal point, excessively frequent knowledge updates and fusion operations place a significant burden on the system, gradually reducing the system's query response speed, and thus decreasing operability. The operability index helps determine the optimal number of virtual nodes, backups, and reporting frequency in the management layer.

[0149] The business support capability assessment module measures the performance of the knowledge system application layer under different parameter configurations, evaluates the business support capability Sa, and provides a basis for parameter configuration for the application layer.

[0150] Business support capability (Sa) is a quantitative evaluation indicator of the application layer capabilities of a knowledge system, verifying whether the knowledge system can quickly and effectively support actual business operations. Sa is composed of inference real-time performance (S). c Business Diversity s Business effectiveness S e A total of three secondary indicators are used for comprehensive measurement.

[0151] Sa = π c S c +π s S s +π e S e

[0152] Where, π c , π s , π e The weight is the corresponding secondary indicator; the larger the weight, the more important the secondary indicator.

[0153] Real-time reasoning S c Assessed by the inference real-time assessment module, business diversity S s Business diversity assessment module, business effectiveness S e Evaluated by the business effectiveness assessment module.

[0154] Real-time reasoning S c Measured by inference response time, the actual inference time is t, and the reference inference time is ti. tol ,S c Calculated using the following formula:

[0155]

[0156] Business Diversity s The number of tasks supported is quantified by the number of actual business support types, and the number of reference business types is N. tol ,S s Calculated using the following formula:

[0157]

[0158] Business effectiveness S e The probability of business completion is measured by the total number of times business i is performed. The number of times the business was successfully completed was S e Calculated using the following formula:

[0159]

[0160] Application layer parameter configurations will affect business support capability metrics. Figure 6 This section presents business support capability indicators for different inference frequencies and inference chain lengths. Inference frequency is defined as the number of inference executions within 10 seconds. It can be seen that when the inference frequency is greater than 2, the business support capability with an inference chain length of 3 is higher than that with lengths of 1 and 2. This is because with an inference chain length of 3, nodes can make decisions using reasoning knowledge, resulting in higher task effectiveness. As the inference frequency increases, the business support capability first rises and then falls. This is because when the inference frequency is too high, the computational overhead per unit time continuously increases, the inference response time becomes longer, and the real-time performance of inference decreases. On the other hand, due to the excessive number of decisions made per unit time, there is an "overreaction" to environmental changes, slightly decreasing the task completion probability and slightly reducing business effectiveness. Through these business support capability indicators, the optimal inference frequency and inference chain length for the inference mechanism in the application layer can be determined.

[0161] For a specific evaluation object, actual operational data of the knowledge system can be collected, and the weights of its various secondary indicators can be calculated to obtain a comprehensive evaluation result. By collecting operational data and calculating evaluation indicators multiple times under different system configurations, curves showing the changes of each primary indicator with the key parameters of the knowledge system can be obtained, providing a reference standard for knowledge system parameter configuration schemes. Preferably, the weights of the secondary indicators are calculated using the entropy method.

[0162] Compared with existing technologies, the method of this invention can provide a unified quantitative standard for the comprehensive capabilities of the representation layer, management layer and application layer of a knowledge system, and provide a reference for the optimal parameter configuration of the knowledge system.

[0163] Figure 2 A flowchart illustrating a quantitative evaluation method for a knowledge system according to an embodiment of the present invention is shown.

[0164] First, in step S201, secondary indicators are evaluated, including representation category, knowledge granularity, knowledge connotation, knowledge entropy before fusion, knowledge entropy after fusion, decay factor, system throughput, inference real-time performance, business diversity, and business effectiveness.

[0165] Next, in step S202, the primary indicators are evaluated based on the secondary indicators. The primary indicators include representation ability, understandability, operability, and business support capability.

[0166] Specifically, the representation category E in the secondary indicators t It is a standardized measure of the types of knowledge that can be represented by the knowledge representation methods of a knowledge system, and its expression is:

[0167]

[0168] Where |·| is the cardinality of the set, and δ(n) is a binary function of n, with a value of 1 when the knowledge representation method supports the representation of n, and 0 otherwise. t For knowledge category set The elements in the text. Preferably, the types of knowledge include declarative knowledge and procedural knowledge, that is... Declarative knowledge is an objective description of the internal state and external environment of a system, while procedural knowledge is a process-oriented and logical expression of events inside and outside the system.

[0169] If the knowledge graph used by the knowledge system supports the representation of declarative and procedural knowledge, and the superscript (n) represents the actual evaluation value of the indicator, then

[0170] Knowledge Granularity E g It measures the ability of knowledge representation methods in a knowledge system to express multi-granularity knowledge, and its expression is:

[0171]

[0172] Where, n g For knowledge granularity set The elements in For n g The i-th necessary component of granularity, N g For n g The number of necessary components for granularity. Preferably, based on the degree of abstraction, knowledge granularity is divided into instance knowledge, conceptual knowledge, event knowledge, and principle knowledge, i.e. Instance knowledge refers to specific phenomena occurring during the operation of an actual system, representing the finest-grained unit of knowledge. The essential components of instance knowledge are instance object identifier, instance object attribute key, instance object attribute value, and relationships between instance objects. Conceptual knowledge is an abstract description of the essential attributes of instance objects, formed by summarizing and generalizing instance knowledge. The essential components of conceptual knowledge are concept identifier, concept connotation, and concept extension. Connotation refers to the meaning of a concept, constituted by the unique attributes of the objective object reflected by the concept; extension refers to the objects or scope to which the concept applies, constituted by the objective object referred to by the concept. Event knowledge reflects behaviors occurring in an actual system, formed by the participation of one or more roles in a specific time and space, and possessing specific action characteristics. The essential components of event knowledge are time, space, instance object, and behavior. Principle knowledge describes the regular logical relationships between events, obtained through the analysis and summarization of event knowledge. Principle knowledge does not depend on specific contexts or instance subjects, but reflects the underlying principles and rules, representing the coarsest-grained unit of knowledge. The essential components of principle knowledge are principle identifier, event, and logical relationship.

[0173] If the knowledge graph used by the knowledge system supports the representation of instance knowledge, conceptual knowledge, event knowledge, and logical knowledge to varying degrees. Specifically, for instance knowledge, it supports representing instance object identifiers, instance object attribute keys, instance object attribute values, and relationships between instance objects; for conceptual knowledge, it supports representing concept identifiers and concept extensions, but not concept connotations; for event knowledge, it supports representing instance objects and behaviors, but not time and space; for logical knowledge, it supports representing events and logical relationships, but not logical identifiers. Therefore,

[0174] Knowledge Content E r It is a measure of the ability of a knowledge system's knowledge representation method to express knowledge with multiple connotations, expressed as:

[0175]

[0176] Where, n r For the collection of knowledge content The elements in. Preferably, knowledge is divided into semantic knowledge and mathematical knowledge from the perspective of connotation, that is... Semantic knowledge consists of objective facts described by language and is presented by a vocabulary system and a semantic system, while mathematical knowledge is described by a digital logic model between concepts.

[0177] If the knowledge graph used by the knowledge system supports the representation of semantic knowledge but not the representation of mathematical knowledge, then

[0178] The representation capability Ec in the primary indicators provides a basis for selecting knowledge representation methods for the representation layer of a knowledge system.

[0179] The representational capability Ec in the primary indicator can be derived from the representational category E in the secondary indicator. t Knowledge Granularity E g With knowledge connotation E r To synthesize the metrics, the expression is:

[0180] Ec = π t E t +π g E g +π r E r

[0181] Where, π t , π g , π r The weight is the corresponding secondary indicator; the larger the weight, the more important the secondary indicator.

[0182] The weights can be calculated using the entropy method. Therefore, it represents the ability Ec (n) =0.756.

[0183] Using the same methodology, various knowledge representation methods were evaluated. Data tables and time series diagrams only support single-category, single-content knowledge representation, with representation capabilities of 0.424 and 0.439, respectively. The representation method combining data tables and production rules can further represent procedural knowledge, supporting knowledge representation at the event and instance granularity, achieving a representation capability of 0.680. The comparison shows that the representation capability of the knowledge system's representation layer is higher than other representation methods, making it a preferred option.

[0184] Preferably, knowledge entropy uses the entropy reduction introduced by knowledge into the system as a measure of the amount of knowledge, reflecting the ability of reasoning knowledge, event knowledge, and conceptual knowledge to distinguish instance knowledge.

[0185] Pre-fusion knowledge entropy in secondary indicators Calculated using the following formula:

[0186]

[0187] in, This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist The equivalence classes of the partition.

[0188] Secondary indicators

[0189] in, It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist The equivalence classes of the partition.

[0190] The understandability (Re) among the primary indicators serves as the basis for selecting the knowledge density of the knowledge system's representation layer.

[0191] The first-level metric, *Re*, measures the degree to which a node in a knowledge system understands the knowledge assembled by other nodes. It is measured by the change in knowledge entropy before and after fusion, a second-level metric. The expression is:

[0192]

[0193] The system throughput (TP) in the secondary metric is defined as the number of concurrent requests processed by the system per unit of time.

[0194]

[0195] in, The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. T represents the average number of requests for operation p. p Let p be the average response time of operation p.

[0196] Attenuation factor γ in secondary indicators e Calculated using the following formula:

[0197]

[0198] Among them, TP m The maximum throughput of the knowledge system is determined through multiple measurements; ∈ is the tolerance factor, with an optimal value range of 10. -4 ~10 -5 When the system throughput reaches its maximum value TP m At this point, the operability index reaches its maximum value of 1-∈. When performing multiple measurements and evaluations on the same system, ∈ is fixed to the same value.

[0199] The ease of operation (Em) in the primary indicator provides management with a basis for parameter configuration. The configurable management parameters include the number of virtual nodes, the number of backups, and the reporting frequency.

[0200] Ease of operation (Em) is a quantitative evaluation indicator of the capabilities of the knowledge system's management layer. It verifies the efficiency of concurrent operations such as building, integrating, storing, and querying within the knowledge system, and is measured through a decay factor γ in the secondary indicator.e The system throughput TP is obtained, and the expression is:

[0201]

[0202] Inference real-time performance (S) in the secondary indicator c Measured by inference response time, the actual inference time is t, and the reference inference time is ti. tol ,S c Calculated using the following formula:

[0203]

[0204] Business Diversity s The number of tasks supported is quantified by the number of actual business support types, and the number of reference business types is N. tol ,S s Calculated using the following formula:

[0205]

[0206] Business effectiveness S e The probability of business completion is measured by the total number of times business i is performed. The number of times the business was successfully completed was S e Calculated using the following formula:

[0207]

[0208] The business support capability Sa in the first-level indicator provides a basis for parameter configuration for the application layer. The configurable application layer parameters include inference chain length, inference frequency, etc.

[0209] Business support capability (Sa) is a quantitative evaluation indicator of the application layer capabilities of a knowledge system, verifying whether the knowledge system can quickly and effectively support actual business operations. Business support capability (Sa) is composed of the secondary indicator, specifically reasoning real-time performance (S). c Business Diversity s Business effectiveness S e To measure comprehensively.

[0210] Sa = π c S c +π s S s +π e S e

[0211] Where, π c , π s , π e This represents the weight of the corresponding secondary indicator; a larger weight indicates a higher importance for that secondary indicator. Weight π c , πs , π e It can be obtained by the entropy method.

[0212] In step S203, the parameters of the knowledge system are configured based on the primary indicators.

[0213] Specifically, based on representation capability Ec, the knowledge density and knowledge representation method of the representation layer of the knowledge system can be configured; based on understandability Re, the knowledge density of the representation layer of the knowledge system can be configured; based on ease of operation Em, the number of virtual nodes, backups, and reporting frequency of the management layer of the knowledge system can be configured; and based on business support capability Sa, the reasoning frequency and reasoning chain length of the application layer of the knowledge system can be configured.

[0214] As mentioned above, the primary indicators calculated based on the secondary indicators can provide a reference standard for the configuration scheme of knowledge system parameters, thereby optimizing the parameters of the knowledge system and achieving autonomous optimization of the knowledge system.

[0215] The method according to an embodiment of the present invention can be written as a computer program and can be implemented in a general-purpose digital computer that executes the program using a computer-readable recording medium.

[0216] In this state, the medium can continuously store programs that can be executed by a computer, or it can temporarily store programs for execution or download. Furthermore, the medium can be various recording or storage devices that combine single or multiple hardware components, not limited to media directly connected to a computer system, and can exist in a distributed manner on a network. Examples of media include magnetic storage media (such as floppy disks or hard disks) configured to store program instructions, optical recording media (such as CD-ROMs or DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, flash memory, etc. Additionally, other examples of media can include: application stores for distributing applications, sites for providing or distributing various other software, and recording or storage media managed at a server.

[0217] Although the invention has been specifically shown and described with reference to preferred embodiments using specific terminology, the embodiments and terminology should be considered in a descriptive sense only and not for limiting purposes. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for quantitative evaluation and operational parameter optimization of a distributed knowledge system, the distributed knowledge system comprising a presentation layer, a management layer, and an application layer, wherein the method is automatically executed by a computer during the operation of the distributed knowledge system, characterized in that, The method includes: Step S1, Parameter Acquisition Steps: During the operation of the distributed knowledge system, the computer collects the operating parameters of the presentation layer, management layer and application layer in real time. The operating parameters include at least system throughput, average response time, inference time and number of business executions. Step S2, Secondary Indicator Calculation Steps: Based on the aforementioned operating parameters, the computer calculates the representation category, knowledge granularity, knowledge connotation, knowledge entropy before fusion, knowledge entropy after fusion, attenuation factor, system throughput, inference real-time performance, business diversity, and business effectiveness. Step S3, Calculation steps for primary indicators: The computer calculates the representation capability, understandability, operability, and business support capability based on the aforementioned secondary indicators, wherein: The representation capability is calculated based on representation category, knowledge granularity, and knowledge connotation, and is used to represent the representation layer's ability to express multiple types and granularities of knowledge. The comprehensibility is calculated based on the knowledge entropy before and after fusion, and is used to characterize the degree to which the representation layer nodes understand the knowledge of other nodes; The ease of operation is calculated based on the attenuation factor and system throughput, and is used to characterize the processing performance of the management layer under concurrent construction, fusion, storage and query operations; The business support capability is calculated based on inference real-time performance, business diversity, and business effectiveness, and is used to characterize the application layer's ability to support actual business tasks. Step S4, Automatic Parameter Adjustment Step: Based on the calculation results of the primary indicators, the computer automatically adjusts the system operating parameters during the operation of the distributed knowledge system, wherein: Based on the aforementioned representation capabilities and comprehensibility, the knowledge density and knowledge representation methods in the representation layer are automatically adjusted; Based on the ease of operation, the number of virtual nodes, backups, and reporting frequency in the management layer are automatically adjusted to ensure that the system throughput and response time meet preset performance conditions. Based on the aforementioned business support capabilities, the inference frequency and inference chain length in the application layer are automatically adjusted to improve inference real-time performance and business execution success rate. Step S5, Closed-loop optimization steps: After the automatic parameter adjustment is completed, the computer continues to collect updated operating parameters to form a closed-loop performance optimization process for the distributed knowledge system. Among them, the characterization category for: in, For the cardinality of the set, for A binary function, when the knowledge representation method supports the understanding of... When represented by , the function value is 1; otherwise, it is 0. For knowledge category set The elements in knowledge granularity for: in, For knowledge granularity set The elements in for The first particle size A necessary component element for The number of necessary constituent elements for granularity. Knowledge connotation for: in, For the collection of knowledge content The elements in Representational ability for: ,in, Representation categories Knowledge Granularity and the connotation of knowledge The weight, Fusion of pre-knowledge entropy for: in, , This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist Equivalence classes of the partition, Knowledge entropy after fusion for: in, , It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist Equivalence classes of the partition, Understandability for: System throughput for: in, , The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. Let p be the average number of requests. Let p be the average response time. Attenuation factor for: in, The maximum throughput of the knowledge system was determined through multiple measurements. Tolerance factor, Ease of use for: Real-time reasoning for: in, For actual reasoning time, For reference reasoning time, Business Diversity for: in, For the types of actual business support, For reference regarding business types, Business effectiveness for: in, Perform the total number of times for business i. The number of times a business transaction was successfully completed. Business support capabilities for: in, Real-time reasoning Business Diversity and business effectiveness The weight.

2. The method for quantitative evaluation and operational parameter optimization of a distributed knowledge system according to claim 1, characterized in that, The distributed knowledge system includes a presentation layer, a management layer, and an application layer. The system operating parameters in step S4 are automatically adjusted by the computer during the operation of the distributed knowledge system, enabling parameter optimization without interrupting its operation. The representational capability and understandability are used to drive the automatic adjustment of knowledge density and knowledge representation methods in the representation layer; The ease of operation is used to drive the automatic adjustment of the number of virtual nodes, backups, and reporting frequency in the management layer; The business support capabilities are used to drive the automatic adjustment of inference frequency and inference chain length in the application layer.

3. The method according to claim 2, characterized in that, In step S3, the weights of representation capability, understandability, operability, and business support capability are automatically calculated by the computer based on the collected operating parameters using the entropy method, in order to eliminate the interference of manual weighting on the optimization process of system operating parameters.

4. A quantitative evaluation and operational parameter optimization system for a distributed knowledge system, the system running on a computer and automatically performing parameter acquisition, index calculation, and parameter adjustment during the operation of the distributed knowledge system, characterized in that... The system includes: The runtime parameter acquisition module is used to automatically collect system throughput, average response time, inference time, and number of business executions during system operation. The secondary indicator calculation module is used to calculate the representation category, knowledge granularity, knowledge connotation, knowledge entropy before fusion, knowledge entropy after fusion, decay factor, system throughput, inference real-time performance, business diversity, and business effectiveness based on the collected operating parameters. The primary indicator calculation module is used to calculate the representation capability, understandability, ease of operation, and business support capability based on the secondary indicators. The automatic parameter adjustment module is used to automatically adjust the system operating parameters based on the calculation results of the primary indicators during the operation of the distributed knowledge system, so as to achieve closed-loop optimization of system performance. Among them, the characterization category for: in, For the cardinality of the set, for A binary function, when the knowledge representation method supports the understanding of... When represented by , the function value is 1; otherwise, it is 0. For knowledge category set The elements in knowledge granularity for: in, For knowledge granularity set The elements in for The first particle size A necessary component element for The number of necessary constituent elements for granularity. Knowledge connotation for: in, For the collection of knowledge content The elements in Representational ability for: ,in, Representation categories Knowledge Granularity and the connotation of knowledge The weight, Fusion of pre-knowledge entropy for: in, , This is a collection of prior knowledge of principles, events, and concepts. The set of instance knowledge before fusion For set exist Equivalence classes of the partition, Knowledge entropy after fusion for: in, , It is a collection of integrated knowledge of principles, events, and concepts. For the fused set of instance knowledge, For set exist Equivalence classes of the partition, Understandability for: System throughput for: in, , The number of concurrent requests to operate on p. The number of concurrent nodes operating on p. Let p be the average number of requests. Let p be the average response time. Attenuation factor for: in, The maximum throughput of the knowledge system was determined through multiple measurements. Tolerance factor, Ease of use for: Real-time reasoning for: in, For actual reasoning time, For reference reasoning time, Business Diversity for: in, For the types of actual business support, For reference regarding business types, Business effectiveness for: in, Perform the total number of times for business i. The number of times a business transaction was successfully completed. Business support capabilities for: in, Real-time reasoning Business Diversity and business effectiveness The weight.

5. The system according to claim 4, characterized in that, The automatic parameter adjustment module is configured as follows: Based on representation capabilities and understandability, the knowledge density and knowledge representation methods in the representation layer are automatically adjusted; Based on ease of operation, the number of virtual nodes, backups, and reporting frequency in the management layer are automatically adjusted to improve system throughput and reduce response time. Based on business support capabilities, the inference frequency and inference chain length in the application layer are automatically adjusted to improve inference real-time performance and business execution success rate.

6. The system according to claim 4 or 5, characterized in that, In the primary indicator calculation module, the weight of each primary indicator is automatically determined by the computer based on the collected system operation data using the entropy method.

7. A computer program product encoded on a computer-readable storage medium, the computer program product comprising instructions that, when executed by one or more computers, cause the one or more computers to perform the method of any one of claims 1 to 3, comprising: Automatically collect operational parameters during the operation of a distributed knowledge system; The system automatically calculates secondary and primary indicators based on the collected operating parameters. The system automatically adjusts its operating parameters during operation and forms a closed-loop performance optimization process through continuous data collection and adjustment.

Images