Knowledge authentication for AI-supported decision-making systems

Abstract

The examples described herein provide a knowledge authentication procedure for AI-supported decision-making. The procedure involves receiving knowledge from a subject matter expert and authenticating that knowledge based on structured decision-making information. It further includes generating an efficiency index for individual uses of the knowledge, where the efficiency index measures the sequence and efficiency of decisions and approaches taken to solve a problem. Finally, the procedure includes storing the authenticated knowledge and the efficiency index, where the authenticated knowledge represents know-how that can be distributed with a degree of certainty regarding its usability.The process further includes generating, using a trained machine learning model, a response to a user request using authenticated knowledge and the efficiency index.

Description

background

[0001] The subject of the disclosure relates to computer systems based on artificial intelligence and, in particular, to knowledge authentication for decision-making systems supported by artificial intelligence (AI).

[0002] Decision-making is the process of selecting the best course of action from various options to achieve specific goals. In industrial environments (e.g., the automotive, aerospace, and / or similar industries), decision-making is critical because decisions directly impact efficiency, performance, and overall success. Effective decision-making ensures that resources are used optimally, processes run smoothly, and potential problems are proactively addressed or avoided altogether. This is beneficial for maintaining operational stability, meeting production targets, and remaining competitive in a dynamic market. Sound decision-making supports long-term growth and sustainability, making it a cornerstone of successful industrial management.

[0003] Expert knowledge in decision-making processes within industrial environments such as the automotive or aerospace industries involves a specialized understanding of complex systems, production processes, and industry-specific challenges in order to optimize operations and workflows. This expertise often includes in-depth familiarity with manufacturing technologies, supply chain logistics, product designs, and quality control standards.

[0004] Experts use this knowledge to analyze data, identify potential risks, and implement strategies that increase efficiency, reduce costs, and maintain high safety and quality standards. Furthermore, experts often integrate advanced tools such as predictive analytics, artificial intelligence, and automation to support real-time decision-making, ensuring the continuous improvement of production processes in a competitive and regulated environment. It may be desirable to authenticate knowledge for AI-supported decision-making systems, ensuring that the knowledge provided is accurate, reliable, and originates from valid sources. Summary

[0005] In one embodiment, a knowledge authentication method is provided for AI-supported decision-making. The method includes receiving knowledge from a subject matter expert and authenticating that knowledge based on structured decision-making information. The method further includes generating an efficiency index for individual uses of the knowledge, the efficiency index measuring the sequence and efficiency of decisions and approaches taken to solve a problem. The method also includes storing the authenticated knowledge and the efficiency index, the authenticated knowledge representing know-how that can be distributed with a degree of certainty regarding its usability.The process further includes generating, using a trained machine learning model, a response to a user request using authenticated knowledge and the efficiency index.

[0006] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include knowledge that contains at least one opinion, approach to the problem, or recommendation.

[0007] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the fact that the knowledge is implicit knowledge, wherein the trained machine learning model generates the response using the authenticated knowledge and the efficiency index.

[0008] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the efficiency index being one of a plurality of efficiency indices for solving the problem.

[0009] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include classifying the multitude of efficiency indices to solve the problem.

[0010] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the classification being based at least partially on a confidence in each of the multitude of efficiency indices that solves the problem.

[0011] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include generating the efficiency index using a random walk model.

[0012] In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include calculating the efficiency index using the following equation: ∑0m∏0n(DnPAn)m where m is a number of branches, n is a total number of nodes per branch, and PA represents a P-index or A-index value for a given n and m.

[0013] In another embodiment, a processing system is provided. The processing system comprises a memory containing computer-readable instructions or commands and a processing device for executing the computer-readable commands, wherein the computer-readable commands control the processing system to perform knowledge authentication operations for AI-supported decision-making. The operations include receiving knowledge from a subject matter expert. The operations further include authenticating the knowledge based on structured decision-making information. The operations also include generating an efficiency index for individual uses of the knowledge, wherein the efficiency index measures the sequence and efficiency of decisions and approaches taken to solve a problem.The operations further include storing authenticated knowledge and the efficiency index, where the authenticated knowledge represents know-how that can be distributed with a degree of certainty regarding its usability. The operations also include generating, using a trained machine learning model, a response to a user request using the authenticated knowledge and the efficiency index.

[0014] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include knowledge of at least one opinion, approach to the problem, or recommendation.

[0015] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include the fact that the knowledge is implicit knowledge, wherein the trained machine learning model generates the response using the authenticated knowledge and the efficiency index.

[0016] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include the efficiency index being one of a multitude of efficiency indices for solving the problem.

[0017] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include operations that further include classifying the multitude of efficiency indices to solve the problem.

[0018] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include that the classification is based at least partially on a confidence of each of the multitude of efficiency indices that solves the problem.

[0019] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include generating the efficiency index using a random walk model.

[0020] In addition to one or more of the features described herein, or as an alternative, further embodiments of the processing system may include the calculation of the efficiency index using the following equation: ∑0m∏0n(DnPAn)m where m is a number of branches, n is a total number of nodes per branch, and PA represents a P-index or A-index value for a given n and m.

[0021] In another embodiment, a computer program product is provided. The computer program product comprises a computer-readable storage medium containing program instructions, wherein the program instructions are executable by at least one processor to cause the at least one processor to perform operations for knowledge authentication for AI-supported decision-making. The operations include receiving knowledge from a subject matter expert. The operations further include authenticating the knowledge based on structured decision-making information. The operations further include generating an efficiency index for individual uses of the knowledge, wherein the efficiency index measures the sequence and efficiency of decisions and approaches taken to solve a problem.The operations further include storing authenticated knowledge and the efficiency index, where the authenticated knowledge represents know-how that can be distributed with a degree of certainty regarding its usability. The operations also include generating, using a trained machine learning model, a response to a user request using the authenticated knowledge and the efficiency index.

[0022] In addition to one or more of the features described herein, or as an alternative, further embodiments of the computer program product may include that the efficiency index is one of a plurality of efficiency indices for solving the problem, wherein the operations further include a ranking of the plurality of efficiency indices for solving the problem, and wherein the ranking is based at least partially on a confidence of each of the plurality of efficiency indices that solves the problem.

[0023] In addition to one or more of the features described herein, or as an alternative, further embodiments of the computer program product may include the generation of the efficiency index using a random walk model.

[0024] In addition to one or more of the features described herein, or as an alternative, further embodiments of the computer program product may include the calculation of the efficiency index using the following equation: ∑0m∏0n(DnPAn)m where m is a number of branches, n is a total number of nodes per branch, and PA represents a P-index or A-index value for a given n and m.

[0025] The above features and advantages, as well as other features and advantages of the disclosure, are readily apparent from the following detailed description when considered in conjunction with the accompanying drawings. Brief description of the drawings

[0026] Other features, advantages, and details appear only as examples in the following detailed description, which refers to the drawings in which: Fig. 1. A block diagram of a knowledge authentication system for artificial intelligence (AI) supported decision-making systems according to one or more embodiments is illustrated; Fig. 2. A block diagram of a knowledge authentication system for AI-supported decision-making systems is illustrated according to one or more embodiments; Fig.3. A map illustrating structured decision-making for goal-oriented problem-solving according to one or more embodiments; Fig. 4. A flowchart of a knowledge authentication procedure for AI-supported decision-making according to one or more embodiments is illustrated; and Fig. Figure 5 illustrates a block diagram of a knowledge authentication system for AI-supported decision-making systems according to one or more embodiments. Detailed description

[0027] The following description is by its nature merely exemplary and is not intended to limit the present disclosure, application, or uses. It should be understood that throughout the drawings, corresponding reference numerals denote identical or corresponding parts and features. As used herein, the term "module" refers to a processing circuit that may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (common, dedicated, or group), memory executing one or more software or firmware programs, a combinational logic circuit, and / or other suitable components providing the described functionality.

[0028] One or more embodiments described herein relate to knowledge authentication for decision-making systems supported by artificial intelligence (AI).

[0029] Technological systems frequently face challenges related to the quality of their data input, commonly referred to as the "garbage in, garbage out" problem. This problem arises when the data fed into these systems lacks the necessary depth and context, leading to suboptimal performance and unreliable results or outputs. Existing approaches to decision-making systems primarily focus on explicit knowledge, which includes easily documentable information such as facts and commands. However, these approaches often overlook implicit knowledge (e.g., the knowledge of a seasoned individual or expert), which encompasses the nuanced understanding and experience gained through real-world interactions and problem-solving.

[0030] Implicit knowledge, also known as intangible "know-how," plays a role in decision-making processes, particularly in goal-oriented problem-solving tasks. Implicit knowledge is the knowledge or understanding gained through real-world experience. For example, implicit knowledge is the "know-how" that explains the procedure used to complete a task, which is derived from context and experience. Traditional approaches to capturing this type of knowledge are either intrusive or fail to systematically document the decision-making process. As a result, the systems lack comprehensive data to generate accurate and context-relevant answers.

[0031] In knowledge management systems, ensuring the accuracy and reliability of knowledge used in decision-making processes can be challenging. The proliferation of large language models (LLMs) and synthetic information has facilitated access to vast amounts of data. However, the lack of systematic validation mechanisms can lead to inefficiencies and the dissemination of unvalidated information. This problem is particularly pronounced when the reliability of knowledge provided by automated systems or individuals remains uncertain.

[0032] Existing solutions often fail to authenticate the knowledge provided by subject matter experts (SMEs) and automated systems, leading to potential "hallucinations" or misinformation that can negatively impact task completion and decision-making processes. The lack of a structured approach to validate and measure the effectiveness of knowledge used in problem-solving further exacerbates these challenges. There is a need for a systematic approach to authenticating knowledge and ensuring its usability and reliability across different organizational contexts.

[0033] One or more embodiments described herein address these shortcomings by providing a comprehensive and systematic approach to authenticating and measuring the effectiveness of knowledge used in problem-solving and decision-making processes. One or more embodiments employ an algorithm to systematically authenticate knowledge, generating an efficiency index that measures the sequence and efficiency of decisions and approaches taken to solve a problem. One or more embodiments store validated knowledge in a database along with the efficiency metric, ensuring that the distributed knowledge is valid and ranked based on its proven usability.

[0034] By implementing this approach, one or more implementations enable intelligent data analysis strategies applicable to explicit and implicit knowledge across individuals, functional groups, and the entire organization. One or more implementations integrate a random walk model to measure the efficiency of the problem-solving process, evaluating how efficient the decision paths are and how the impact of external stimuli, such as input from subject matter experts, converges toward an effective solution. This structured approach provides a reliable repository of actionable know-how, improving the accuracy and reliability of the knowledge used in decision-making processes.

[0035] Goal-oriented problem-solving involves a decision-making process ("Why") that, based on explicit and implicit knowledge ("How"), determines the tasks to be performed to achieve a goal ("What"). Knowledge differs from information. Information can be considered data that has been organized or processed in a way that adds context or meaning. For example, information includes raw facts and figures that have been structured but not yet interpreted or fully understood. Knowledge goes a step beyond information in that it encompasses the understanding, interpretation, and application of information. Knowledge is information that has been processed by the human mind through learning, experience, and / or instruction. For example, when data is received, it can be processed to generate information.The processing can take documentation templates and formats and apply the data to them to generate, for example, design or construction requirements, guidelines, and / or standards. This information can then be used by a human user, employing their experience and intuition, to derive knowledge. This knowledge is acquired through learning and can be shared with others.

[0036] Fig. Figure 1 illustrates a block diagram of a System 100 for knowledge authentication for artificial intelligence (AI)-supported decision-making systems according to one or more embodiments. The System 100 can be implemented wholly or partially using, for example, the Processing System 500. Fig. 5 or another suitable system or device.

[0037] An SME 101 interacts with the System 100 to provide implicit knowledge through various user interactions. The SME 101's actions and decisions during specific tasks are captured and documented to generate captured implicit knowledge 116. These interactions can include, for example, typing, speaking, keyboard sequences, click / touch events, and videos, which are processed to extract meaningful insights. The SME 101 plays a role in improving the quality of data input for AI systems by providing a nuanced understanding and experience gained through real-life problem-solving.

[0038] The captured implicit knowledge 116 refers to the knowledge representations extracted from the raw data collected by the SME 101. This knowledge is systematically documented and categorized to create structured knowledge representations. The captured implicit knowledge 116 encompasses individual actions and decisions taken during specific tasks, providing insights for AI-supported decision-making. This knowledge is stored in an implicit knowledge base of a knowledge management system 122, which is used by AI systems (e.g., the LLM agent 120) to enhance decision-making capabilities.

[0039] The LLM agent 120 is an AI system that uses large language models to access and utilize the structured knowledge representations stored in the knowledge base. The LLM agent 120 generates responses to user queries (also referred to as "prompts") based on the captured implicit knowledge 116, thereby improving the AI ​​system's decision-making capabilities. The LLM agent 120 interacts with the knowledge management system 122 to retrieve context and provide accurate and relevant responses, thus improving the quality of data input and enhancing decision-making processes.

[0040] Knowledge Management System 122 is responsible for organizing and storing documented implicit knowledge. This system categorizes knowledge to create structured representations that are accessible and usable by AI systems. Knowledge Management System 122 interacts with the implicit knowledge base and the LLM agent 120 to provide a comprehensive knowledge management solution that supports AI-assisted decision-making. Knowledge Management System 122 ensures that captured knowledge is systematically documented and categorized, making it available for future use.

[0041] The knowledge management system 122 can also utilize explicit knowledge 104, which comprises easily documentable information such as facts, commands, and policies. This explicit knowledge 104 is stored in various databases and repositories, such as a database 104a for calibration guidelines, production code repositories 104b, configuration management tools / procedures 104c, functional SharePoint documents 104d, tool policies and processes, standards 104e, internal social networks 104f, and / or the like, including combinations and / or multiples thereof. This explicit knowledge 104 is used in conjunction with the captured implicit knowledge 116 to provide a comprehensive knowledge management solution.According to one or more embodiments, the explicit knowledge 104 is organized and stored in a structured format, making the explicit knowledge 104 accessible to AI systems to improve their decision-making capabilities.

[0042] Database 104a for calibration guidelines stores guidelines and procedures related to calibration processes. Production code repositories 104b store code and scripts related to production processes. Configuration management tools / procedures 104c store tools and procedures related to configuration management. Functional SharePoint documents 104d store documents related to specific functions and tasks. Tool guidelines, processes, and standards 104e store guidelines, processes, and standards related to various tools and procedures. Internal social networks 104f store information and knowledge shared within the internal social networks.

[0043] A prompt, or decision-support prompt for problem-solving 102, involves a user requesting information to solve a problem. A user can generate a prompt asking how to solve a problem, and the LLM agent 120 uses the prompt and the authenticated knowledge, as further described herein, to generate a decision-support response to solve the problem identified in the user's prompt. Non-restrictive examples of decision-support for problem-solving 102 include determining which signals to measure, determining which signals to connect, determining which calibrations to modify, determining how to modify a system to meet certain requirements, and / or the like, including combinations and / or multiples thereof.

[0044] According to one or more embodiments, knowledge authentication 103 is performed by the SME 101 using an input prompt to support decision-making for problem-solving 102. Knowledge authentication 103 (also referred to as cognitive validation) authenticates the implicit knowledge (e.g., the captured implicit knowledge 116) provided by the subject matter expert (SME) (e.g., the SME 101) and measures the SME's problem-solving process. Authenticating the knowledge ensures that the provided knowledge is accurate, reliable, and derived from validated sources. To perform knowledge authentication 103, the knowledge management system 122 (or another suitable system or device) determines an efficiency index that measures the sequence and efficiency of decisions and approaches taken to solve a problem.Knowledge Authentication 103 aims to reduce and / or eliminate the spread of false information, create a reliable repository of usable know-how, improve decision-making processes through validated and classified knowledge, and / or the like, including combinations and / or multiples thereof. Knowledge Authentication 103 is applied in relation to... Fig. 2-4 further described.

[0045] Fig. Figure 2 illustrates in particular a block diagram of a system 200 for knowledge authentication for AI-supported decision-making systems according to one or more embodiments. The system 200 can be implemented wholly or partially using, for example, the processing system 500 of Fig. 5 or another suitable system or device.

[0046] The SME 101 provides knowledge 201, which may be an opinion, an approach to a problem, a recommendation, an insight and / or the like, including combinations and / or multiples thereof, for an individual contributor problem-solving process 202.

[0047] The individual contributor problem-solving process 202 involves applying knowledge 201 to solve a specific problem or to make a decision during problem-solving. That is, the SME 101 uses knowledge 201 to solve a problem (e.g., the individual contributor problem-solving process 202). A problem may, for example, have several decision points to be made and / or sub-problems to be addressed. The SME 101 applies knowledge 201 during the individual contributor problem-solving process 202 to make decisions, address problems / sub-problems, implement tasks, etc. The individual contributor problem-solving process 202 captures structured decision-making 203, which defines the framework for how the SME 101 applied knowledge 201 during the individual contributor problem-solving process 202.

[0048] Systematic knowledge authentication 204 uses structured decision-making 203 to authenticate knowledge 201. That is, systematic knowledge authentication 204 performs cognitive validation to authenticate knowledge 201, ensuring its validity and reliability. Systematic knowledge authentication 204 receives a problem in a descriptive manner (e.g., what the actual problem to be solved is) and how many approaches or possible solutions are evaluated. According to one or more embodiments, such information can be received as metadata. Systematic knowledge authentication 204 can include cross-referencing with existing validated knowledge, an assessment of the credibility of the SME, and verification of the consistency of the information provided. Systematic knowledge authentication 204 generates an efficiency index 205 for knowledge 201.The Efficiency Index 205 measures the sequence and efficiency of decisions and approaches taken to solve a problem. According to one or more embodiments, the Efficiency Index 205 quantifies the process, including the number and sequence of decisions, the approaches proposed and evaluated, the failed approaches, and the problem decomposition, thus providing a structured method for evaluating the effectiveness of problem-solving processes. In some embodiments, the Efficiency Index 205 can identify sources of knowledge based on their applicability and usefulness to multiple users. This index provides a measurable, objective metric for validating knowledge 201, ensuring that knowledge 201 is not only correct but also practically useful in various contexts. Systematic Knowledge Authentication 204 and the Efficiency Index 205 are discussed herein with reference to [reference to be added]. Fig.3 described in more detail.

[0049] According to one or more embodiments, systematic knowledge authentication 204 employs a random walk approach to generate the efficiency index 205. Random walk is a model used in computer modeling to demonstrate how a decision is made. The random walk can use evidence, specific parameters that define how informative received stimuli are, and how quickly these stimuli are received to converge toward a decision. The random walk is used to measure how effectively external stimuli (e.g., the knowledge 201 of SME 101) help converge toward a decision that leads to an effective solution (e.g., an efficiency index with a desired confidence level (e.g., greater than a threshold, such as 75% confidence, 80% confidence, 95% confidence, etc.)).

[0050] With continued reference to Fig.2. Once the efficiency index 205 is generated, the knowledge 201 is stored as validated knowledge, along with the efficiency index 205, in a database 206 for validated knowledge and efficiency metrics. Knowledge that leads to a solution to a problem is validated knowledge, while knowledge that does not lead to a solution to the problem is not considered validated knowledge but can be used in various forms for other purposes (e.g., for metrics). The database 206 for validated knowledge and efficiency metrics serves as a reliable repository for validated knowledge that can be distributed with a degree of certainty about its usability. The database 206 for validated knowledge and efficiency metrics ensures that the validated knowledge is readily accessible for future use and can be reliably referenced in future decision-making processes.

[0051] According to one or more embodiments, the Database 206 for validated knowledge and efficiency metrics can rank efficiency indices such that a higher-ranked efficiency index indicates greater confidence in solving a problem than a lower-ranked efficiency index. That is, there can be multiple efficiency indices for solving a problem, and these efficiency indices can be ranked based on confidence in solving the problem.

[0052] The validated knowledge and efficiency indices stored in the database 206 for validated knowledge and efficiency metrics are then used in an intelligent data analysis 207, which applies advanced data analysis techniques to the validated knowledge. This enables the extraction of valuable insights and patterns that can serve as information for decision-making processes. These insights and patterns derived from the intelligent data analysis 207 are applied in the business analysis 208, which effectively uses the validated knowledge and efficiency indices to improve business analysis processes across an organization, thereby enhancing overall decision-making and problem-solving capabilities.

[0053] Fig.Figure 3 illustrates a structured decision-making process for goal-oriented problem-solving according to one or more embodiments. The map provides a visual representation of sequences of decisions, approaches, and problem decompositions that accompany solving an overall problem P1 and achieving a solution P1S1.

[0054] The process begins with identifying the overall problem P1 that needs to be solved. This problem serves as the starting point for the decision-making and problem-solving process. The ultimate goal is to derive the solution P1S1.

[0055] SME 101 is presented with three approaches, P1A1, P1A2, and P1A3, to solve the overall problem P1. The number of approaches indicates how many options SME 101 has considered. SME 101 can choose one approach: P1A1, P1A2, or P1A3. The different approaches are different ways of solving a problem, and the different choices are decisions that SME 101 can make or makes in the decision-making process. In the example of Fig. 3. SME 101 decides to evaluate approach P1A2 first. Since approach P1A2 yields the solution P1S1, approaches P1A1 and P1A3 are not evaluated; however, these approaches could be considered in other embodiments, such as if approach P1A2 does not yield a solution.

[0056] In this example, SME 101 selects approach P1A2, at which point the overall problem P1 is reduced to two problems P11 and P12 (which can be considered subproblems of the overall problem P1) through problem reduction. As in Fig. As shown in Figure 3, SME 101 first addresses problem P11 and considers approach P11A1 as a possible solution. No further problem reduction is performed, as P11 is considered a minimal problem to be solved according to approach P11A1, and the solution for P11 is P11S1. The time t used to solve problem P11 is recorded, and an efficiency index is given for the time P11 is solved, which is 1 / 3.

[0057] Returning to the problem reduction of P1A2, chart 300 continues with a focus on problem P12. Problem P12 is approached using the approach P12A1, which is considered a possible solution. The problem reduction is performed as shown, reducing P12 into two further problems, P121 and P122. For problem P121, the approach P121A1 is used, leading to the solution P121S1. For problem P122, the approach P122A1 is chosen, resulting in the solution P122S1. The times t used to solve problems P121 and P122 are recorded. The solutions P121S1 and P122S1 are combined to obtain the solution P12S1 for problem P12.

[0058] The solutions P11S1 and P12S1 are then combined to generate the solution P1S1 for the problem P1.

[0059] The solutions P11S1 and P12S1 are used to calculate the efficiency index 205 for the solution P1S1 as follows.

[0060] Cognitive indices (knowledge indices) are defined as follows: P is a know-how index that is 1 / (total number of problem reductions (TP)), therefore P = 1 / TP; A is an experience index that is 1 / (total number of approaches (TA)), therefore A = 1 / TA; and D is an intuition index that is 1 / (numerical order of decisions made (ND)), therefore P = 1 / ND.

[0061] Problem-solving nodes for decision-making are defined as follows: DP is a decision problem node DP = D n P n , where n is the number of nodes; and DA a decision approach node DA = D n A n , where n is the number of nodes.

[0062] The efficiency index 205 (DPAE_index) can be calculated using the following equation: DPAE_index=∑0m∏0n(DnPAn)m where m is a total number of branches, n is a total number of nodes per branch, and PA represents the P-index or A-index value for a given n and m.

[0063] The calculation of the efficiency index 205 (DPAE_index) is described below. For problem P1, whose problem-solving decision-making process is shown in the following table, the efficiency index 205 (DPAE_index) is calculated as follows, where m = 3, n1 = 4, n2 = 6, and n3 = 6. Branch[m] Nodes [n] node Decision [D] Problem [P] Approach[A] Node Index Branch Index DPAEIndex 1 1 P1A2 1 1 / 3 1 / 3 1 2 P11 1 1 / 2 1 / 2 1 3 P11A1 1 1 1 1 4 P11 1 1 1 0,167 2 1 P1A2 1 1 / 3 1 / 3 2 2 P12 1 / 2 1 / 2 1 / 4 2 3 P12A1 1 1 1 2 4 P121 1 1 / 2 2 5 P121A1 1 1 1 2 6 P121 1 1 1 0,042 3 1 P1A2 1 1 / 3 1 / 3 3 2 P12 1 / 2 1 / 2 1 / 4 3 3 P12A1 1 1 1 3 4 P122 1 / 2 1 / 2 1 / 4 3 5 P122A1 1 1 1 3 6 P122 1 1 1 0,021 0,229

[0064] Fig. Figure 4 is a flowchart of a knowledge authentication method 400 for AI-supported decision-making according to one or more embodiments. The method 400 can be implemented using any suitable system or device. For example, the method 400 can be implemented using the processing system 500 of Fig.5 and / or another suitable system or device. Method 400 is now implemented with reference to Fig. 1, Fig. 2 and / or 3 are described, but it is not restricted in this way.

[0065] In Block 402, Procedure 400 begins with the receipt of knowledge from SME 101. This knowledge may include insights, recommendations, opinions, advice, and / or the like, including combinations and / or multiples thereof, that are relevant to specific tasks or decision-making processes.

[0066] In Block 404, the received knowledge is authenticated based on structured decision-making information. This includes evaluating the source and content of the knowledge to ensure its validity and reliability. The authentication process may involve cross-referencing to existing validated knowledge, assessing the credibility of the SME101, and / or verifying the consistency of the provided knowledge.

[0067] Block 406 generates an efficiency index for individual uses of the knowledge. This index measures the sequence and efficiency of decisions and approaches taken to solve a problem. It provides a measurable, objective metric to validate the knowledge received from SME 101, ensuring that the knowledge is not only accurate but also practically useful in various contexts.

[0068] Block 408 stores the authenticated knowledge and the efficiency index, similar to how it is stored in knowledge management system 122. The authenticated knowledge represents know-how that can be distributed with a degree of certainty regarding its usability, as indicated by the efficiency index. Storing the authenticated knowledge and the efficiency index in a database ensures that the knowledge is readily accessible for future use by various users and can be reliably referenced in decision-making processes.

[0069] In Block 410, a trained machine learning model (e.g., using LLM agent 120) is used to generate a response to a user request (e.g., a prompt) using authenticated knowledge and the efficiency index. This step effectively leverages the validated and ranked knowledge to provide accurate and reliable responses to user requests, thereby improving the responses throughout the entire decision-making process by the AI-assisted system (e.g., LLM agent 120) by ensuring that the information provided is both valid and useful.

[0070] Additional processes can also be included, and it should be understood that the in Fig.The processes described in section 4 are for illustrative purposes only, and it should be understood that other processes can be added, or existing processes can be removed, modified, or rearranged without deviating from the scope of this disclosure. It should also be understood that the processes described in section 4 are for illustrative purposes only and are not intended to be taken literally. Fig. The processes shown in section 4 can be implemented as programmatic instructions stored on a non-transitory, computer-readable storage medium, which, when executed by means of a processor (e.g., the processor(s) 521 of Fig. 5) of a computer system (e.g., the processing system 500 of Fig. 5) be executed, cause the processor to perform the processes described herein.

[0071] It is understood that one or more of the embodiments described herein may be implemented in conjunction with any other type of computer environment known today or developed later. Fig.Figure 5, for example, presents a block diagram of a processing system 500 for implementing the techniques described herein. According to one or more embodiments described herein, the processing system 500 is an example of a cloud computing node in a cloud computing environment. In examples, the processing system 500 has one or more central processing units (also referred to as "processors" or "processing resources" or "processing devices") 521a, 521b, 521c, etc. (collectively or generally referred to as processor(s) 521 and / or processing device(s)). In aspects of this disclosure, each processor 521 may contain a reduced instruction set computer (RISC) microprocessor. The processors 521 are coupled via a system bus 533 to a system memory 522 and / or various other components.The system memory 522 can include one or more temporary and / or permanent storage devices, such as random access memory (RAM) 523, read-only memory (ROM) 524, and / or the like, including combinations and / or multiples thereof. The system bus 533 can include a basic input / output system (BIOS) that controls certain basic functions of the processing system 500.

[0072] Furthermore, an input / output (I / O) adapter 527 and a network adapter 526 are shown, which are coupled to the system bus 533. The I / O adapter 527 can be a SCSI (Small Computer System Interface) adapter that communicates with a hard disk 535 and / or a storage device 536 or any other similar component. The I / O adapter 527, the hard disk 535, and the storage device 536 are collectively referred to herein as mass storage 534. An operating system 540 for execution on the processing system 500 can be stored in the mass storage 534. The network adapter 526 connects the system bus 533 to an external network 538, which enables the processing system 500 to communicate with other such systems.

[0073] A display (e.g., a display monitor) 539 is connected to the system bus 533 via a display adapter 532, which may include a graphics adapter to improve the performance of graphics-intensive applications and a video controller. In one aspect of this disclosure, the adapters 526, 527, and / or 532 may be connected to one or more I / O buses that are connected to the system bus 533 via an intermediate bus bridge (not shown). Suitable I / O buses for connecting peripheral devices such as disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI). Additional input / output devices are connected to the system bus 533 via a user interface adapter 528 and the display adapter 532, as shown.A keyboard 529, a mouse 530 and a speaker 531 can be connected to the system bus 533 via the user interface adapter 528, which may contain, for example, a super I / O chip that integrates several device adapters into a single integrated circuit.

[0074] In some aspects of the present disclosure, the processing system 500 includes a graphics processing unit (GPU) 537. The graphics processing unit 537 is a specialized electronic circuit designed to manipulate and modify memory to accelerate the creation of images in a picture or frame buffer intended for output to a display. In general, the graphics processing unit 537 is very efficient at manipulating computer graphics and image processing and has a highly parallel structure, which makes it more effective than general-purpose CPUs for algorithms that involve the parallel processing of large blocks of data.

[0075] As configured herein, the processing system 500 thus has a processing capability in the form of processors 521, a storage capability comprising the system memory 522 and the mass storage device 534, input devices such as the keyboard 529 and the mouse 530, and an output capability comprising the loudspeaker 531 and the display 539. In some aspects of this disclosure, a portion of the system memory 522 and the mass storage device 534 jointly store the operating system 540 in order to coordinate the functions of the various components represented in the processing system 500.

[0076] The terms "a / an / an" do not denote a quantity restriction, but rather indicate the presence of at least one of the elements being referred to. The term "or" means "and / or" unless the context clearly indicates otherwise. A reference in the entire description to "an aspect" means that a particular element (e.g., a feature, a structure, a step, or a property) described in connection with that aspect is contained in at least one aspect described herein and may or may not be present in other aspects. Furthermore, it is understood that the described elements in the various aspects can be combined in any suitable way.

[0077] When it is stated that an element, such as a layer, film, area, or substrate, is located "on" another element, it can be located directly on top of the other element, or there can be elements in between. Conversely, when it is stated that an element is located "directly on" another element, there are no elements in between.

[0078] Unless otherwise specified herein, all test standards or norms are the latest applicable norm as of the filing date of this application or, if priority is claimed, as of the filing date of the earliest priority application in which the test standard appears. Unless otherwise defined, the technical and scientific terms used herein have the same meaning as they are generally understood by a person skilled in the art in the field to which this disclosure belongs.

[0079] Although the above disclosure has been described with reference to exemplary embodiments, it is understood by the person skilled in the art that various modifications can be made and equivalent elements can be substituted without altering the scope of the disclosure. Furthermore, many modifications can be made to adapt a particular situation or material to the teachings of the disclosure without altering its essential scope. Therefore, the present disclosure is not intended to be limited to the specific embodiments disclosed, but rather to encompass all embodiments that fall within its scope.

Claims

[1] Computer-implemented knowledge authentication method for AI-supported decision-making, comprising: Receiving knowledge from a subject matter expert; Authenticating knowledge based on structured decision-making information; Generating an efficiency index for individual uses of knowledge, where the efficiency index measures a sequence and efficiency of decisions and approaches taken to solve a problem; Storing authenticated knowledge and the efficiency index, where the authenticated knowledge represents know-how that can be distributed with a degree of certainty about its usability; and Generate, using a trained machine learning model, a response to a user request using authenticated knowledge and the efficiency index. [2] Computer-implemented method according to claim 1, wherein the knowledge comprises at least one opinion, approach to the problem or recommendation. [3] Computer-implemented method according to claim 1, wherein the knowledge is implicit knowledge, wherein the trained machine learning model generates the response using the authenticated knowledge and the efficiency index. [4] Computer-implemented method according to claim 1, wherein the efficiency index is one of a plurality of efficiency indices for solving the problem. [5] Computer-implemented method according to claim 4, further comprising classifying the plurality of efficiency indices to solve the problem. [6] Computer-implemented method according to claim 5, wherein the classification is based at least partially on a confidence of each of the plurality of efficiency indices that solves the problem. [7] Computer-implemented method according to claim 1, wherein the generation of the efficiency index is carried out using a random walk model. [8] Computer-implemented method according to claim 1, wherein the efficiency index is calculated using the following equation: ∑0m∏0n(DnPAn)m where m is a number of branches, n is a total number of nodes per branch, and PA represents a P-index or A-index value for a given n and m. [9] Processing system, including: a memory containing computer-readable instructions; and a processing device for executing the computer-readable instructions, wherein the computer-readable instructions control the processing system to perform knowledge authentication operations for AI-assisted decision-making, the operations comprising: Receiving knowledge from a subject matter expert; Authenticating knowledge based on structured decision-making information; Generating an efficiency index for individual uses of knowledge, where the efficiency index measures a sequence and efficiency of decisions and approaches taken to solve a problem; Storing authenticated knowledge and the efficiency index, where the authenticated knowledge represents know-how that can be distributed with a degree of certainty about its usability; and Generate, using a trained machine learning model, a response to a user request using authenticated knowledge and the efficiency index. [10] Processing system according to claim 9, wherein the knowledge comprises at least one opinion, one approach to the problem or one recommendation.

Images