Information processing system and information processing method for evaluating the integrity of decision-making processes

Abstract

We provide an information processing system that evaluates the integrity of decision-making processes. [Solution] The information processing system calculates multiple attribute values ​​for each information element included in the decision-making structure data related to the decision-making process, evaluating the information quality of the information element from multiple different perspectives. For each information element included in the decision-making structure data, the system integrates these multiple attribute values ​​through a predetermined calculation to calculate a local score corresponding to the information element. Based on the decision-making structure data and the local score, the system calculates a validity score indicating the validity of the decision-making process, a transparency index indicating the transparency of the decision-making process, and an accountability score indicating the accountability of the decision-making process. The system then performs an integration process on the validity score, transparency index, and accountability score based on a set of weight coefficients to calculate and output an integrated integrity index indicating the integrity of the decision-making process.

Description

[Technical Field]

[0001] This invention generally relates to techniques for evaluating the integrity of decision-making processes. [Background technology]

[0002] In recent years, with the advancement of artificial intelligence technology, AI (Artificial Intelligence) is increasingly involved in important decision-making processes such as medical diagnosis, financial review, policy making, and corporate management. In this context, when AI and humans collaborate in decision-making, it is necessary to objectively and quantitatively demonstrate the basis, logical consistency, transparency, and accountability of the judgment. In particular, technologies that evaluate the validity and neutrality of decisions and enable third-party verification are becoming increasingly important.

[0003] Patent Document 1 discloses acquiring a dataset of graph-structured data containing one or more subgraphs, determining the correlation between analysis of the dataset of graph-structured data using graph-explainable artificial intelligence (GXAI) technology and analysis using multiple graph analysis algorithms, generating a ranked list of multiple graph analysis algorithms based on the correlation, determining a general characteristic indicating the similarity between one or more subgraphs of the dataset of graph-structured data, assigning a threshold number of graph analysis algorithms to the general characteristic based on the ranked list of multiple graph analysis algorithms, generating an assignment table containing the general characteristic and the threshold number of graph analysis algorithms, and displaying the table within a graphical user interface (GUI) that visualizes the assignment table. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2024-169352 [Overview of the project] [Problems that the invention aims to solve]

[0005] The technology described in Patent Document 1 is a technology that assists in the selection of an algorithm by performing correlation analysis between graph-explainable artificial intelligence technology and multiple graph analysis algorithms. Patent Document 1 does not show a configuration that focuses on comprehensively evaluating the validity, transparency, and accountability of the entire decision-making process. Therefore, there is room for improvement in terms of comprehensively and quantitatively evaluating the integrity of the entire decision-making process and grasping it in a balanced manner from the perspectives of validity, transparency, and accountability.

[0006] This invention was made based on the above, and proposes an information processing system, etc., for evaluating the integrity of the decision-making process. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides an information processing system for evaluating the integrity of a decision-making process, comprising: an attribute calculation unit that calculates multiple attribute values ​​for each information element included in decision-making structure data relating to the decision-making process, which evaluate the information quality of the information element from multiple different viewpoints; a local score calculation unit that integrates the multiple attribute values ​​for each information element included in the decision-making structure data by a predetermined calculation and calculates a local score corresponding to the information element; an integrity index calculation unit that calculates a validity score indicating the validity of the decision-making process, a transparency index indicating the transparency of the decision-making process, and an accountability score indicating the accountability of the decision-making process based on the decision-making structure data and the local score calculated by the local score calculation unit; an integrated index calculation unit that performs integration processing on the validity score of the decision-making process, the transparency index of the decision-making process, and the accountability score of the decision-making process based on a plurality of pre-set weight coefficients and calculates an integrated integrity index indicating the integrity of the decision-making process; and an output unit that outputs the integrated integrity index of the decision-making process. [Effects of the Invention]

[0008] According to the present invention, an integrated integrity index indicating the integrity of a decision-making process can be output. Problems, means, and effects not described above will be clarified by the following description of embodiments. [Brief Description of the Drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of an information processing apparatus according to the first embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of decision structure data according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a processing procedure for evaluating the integrity of a decision-making process according to the first embodiment. [Figure 5] FIG. 5 is a diagram illustrating an example of attribute value calculation processing according to the first embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of local score calculation processing according to the first embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of validity score calculation processing according to the first embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of transparency index calculation processing according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of accountability score calculation processing according to the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of integrated integrity index calculation and improvement proposal processing according to the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of differential update processing according to the first embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of learning processing according to the first embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a display screen according to the first embodiment. [Mode for Carrying Out the Invention]

[0010] (I) First Embodiment The configurations, processing procedures, and other elements disclosed below are provided for describing the embodiments of the present invention, and are not intended to limit the present invention. The description based on the drawings is aimed at facilitating understanding, and omissions or simplifications may be made as necessary with respect to the shapes, arrangements, functions and the like of the elements. The present invention is not limited to the disclosed one or more embodiments, and can be implemented by functionally or structurally equivalent means, or other means that achieve technically similar objects, within the range that can be understood by those skilled in the art from the entire present specification. Unless otherwise specified, each component in the present specification is construed as including "at least one". Further, words described in the singular form include the plural form, and words described in the plural form include the singular form. Furthermore, the terms used in the present specification are not limited to specific meanings as long as they are clearly defined by the context, and should be appropriately interpreted by those skilled in the art. For example, the expression "comprising" is construed as meaning that it is not limited to the listed items.

[0011] This embodiment describes an information processing system that evaluates the integrity of a decision-making process based on decision-making structure data related to the decision-making process. Data related to the decision-making process refers to a collection of information recorded about a case that is the subject of a decision. Data related to the decision-making process includes at least text data, numerical data, and metadata that can serve as candidates for claim information, evidence information, counter-evidence information, and source information, as described below. Data related to the decision-making process may include, for example, meeting minutes, email and chat histories, workflow history in business systems, approval logs, reports stored in document management systems, evaluation sheets, and generated documents output by external systems. Data related to the decision-making process may be data input from a client terminal or data provided from an external system. Furthermore, data related to the decision-making process may be structured data or unstructured data. Integrity refers to the overall soundness and appropriateness of the decision-making process from the perspective of validity, transparency, accountability, and other governance (organizational operation, organizational management, adherence to organizational rules, adherence to ethics, and compliance with laws and regulations, etc.) aspects. Integrity is interpreted as a concept that can be evaluated quantitatively or qualitatively using various indicators defined to cover some or all of these aspects. The validity score, transparency index, and accountability score are examples of components for evaluating integrity and are not the only components of integrity; other components may be included, for example.

[0012] The following will be a detailed explanation using the drawings. In this specification, elements that are identical or functionally similar to the components shown in the drawings are denoted by the same reference numerals. The use of reference numerals in this specification is not limited to the specific embodiments shown, but is also applicable to various modifications including at least one component.

[0013] Figure 1 shows an example of an information processing system 100.

[0014] In the information processing system 100, the information processing device 110, the client terminal 120, and the external system 130 are connected via the network 101 in a way that allows them to communicate with each other.

[0015] Network 101 is a communication network for data communication between the information processing device 110, the client terminal 120, and the external system 130. Network 101 may consist of one or more types of networks, such as a wired LAN, a wireless LAN, a wide-area communication network, the Internet, or a public telephone network. Network 101 may use a known communication protocol such as TCP / IP as its protocol.

[0016] The information processing device 110 is a device that performs various processes to evaluate the integrity of the decision-making process. The information processing device 110 may receive data related to the decision-making process, such as meeting record data and document data, transmitted from the client terminal 120. The information processing device 110 may also acquire supplementary information about the decision-making process using data such as business logs and workflow history acquired from the external system 130. Based on the acquired data, the information processing device 110 generates decision-making structure data, calculates attribute values ​​and local scores for each information element included in the decision-making structure data, and performs processing to calculate a validity score, transparency index, accountability score, and integrated integrity index.

[0017] The client terminal 120 is a device that provides an interface between the information processing device 110 and the user (client). The client terminal 120 may be, for example, a desktop computer installed in an office, a portable notebook computer, a tablet terminal, or a smartphone. The client terminal 120 transmits data such as documents related to the decision-making process, meeting minutes, and judgment results entered by the user to the information processing device 110. The client terminal 120 receives the integrated integrity index, validity score, transparency index, accountability score, and improvement suggestion information transmitted from the information processing device 110 and presents it to the user in the form of graphs or text displays.

[0018] The external system 130 is a business system that the information processing device 110 can access. The external system 130 may be, for example, a database server that stores statistical data used for decision-making, a workflow management system that manages approval routes, an audit log management system that stores log information, or a server device that provides text generation services using an LLM (Large-Scale Language Model). In response to requests from the information processing device 110, the external system 130 provides information such as source information, author information, creation date and time information, approval history information, or candidate statements for integrity improvement proposals related to the decision-making process. By using the information obtained from the external system 130, the information processing device 110 enriches the metadata associated with the nodes and edges of the decision-making structure data, improves the accuracy of transparency indicators and accountability scores, and enriches the content of improvement proposal information presented to users.

[0019] The information processing system 100, with the configuration described above, evaluates the integrity of the decision-making process based on information aggregated from multiple client terminals 120 and an external system 130, and provides the evaluation results and improvement suggestions to the client terminals 120.

[0020] Figure 2 shows an example of the information processing device 110.

[0021] The information processing device 110 comprises a processor 210, a storage device 220, and an interface device 230. The processor 210 is a device that performs data arithmetic and control processing. The storage device 220 is a device for storing programs, data, etc. Programs are read and executed by the processor. The interface device 230 is a device that sends and receives information with a network or other devices, or a device that performs information input and output between a user and the device, etc. The functions of the information processing device 110 (generation unit 221, attribute calculation unit 222, local score calculation unit 223, integrity index calculation unit 224, integrated index calculation unit 225, output unit 226, update unit 227, parameter learning unit 228, etc.) are realized by software, hardware, or a combination thereof. The functions of the information processing device 110 can be realized, for example, by software execution on a general-purpose computer, hardware implementation using dedicated circuits, or distributed processing utilizing the cloud, virtual machines, etc. Some or all of the functions of the information processing device 110 may be shared by one or more devices.

[0022] The generation unit 221 generates decision structure data, which includes claim information, evidence information, counter-evidence information, and source information, as well as relational information indicating their support, counter-evidence, and source relationships, based on data related to the decision-making process received from the client terminal 120 or the external system 130. For example, the generation unit 221 analyzes text data from meeting records or workflow history data and stores nodes corresponding to claims, evidence nodes supporting the claims, counter-evidence nodes refuting the claims, and source nodes corresponding to the evidence and counter-evidence, as a directed graph structure in the storage device 220.

[0023] The attribute calculation unit 222 calculates attribute values ​​for each information element included in the decision-making structure data, evaluating the information quality from multiple perspectives. For example, the attribute calculation unit 222 calculates at least one of the following: the degree of bias indicating the degree of deviation from the statistical distribution of clusters; the consistency deviation indicating the degree of content inconsistency with other information elements; the degree of causal consistency indicating the degree of consistency of causal relationships; the degree of ethical deviation indicating the magnitude of ethical deviations; freshness which decays over time; and the reliability based on the information source, creator, and verification status.

[0024] The local score calculation unit 223 integrates multiple attribute values ​​calculated by the attribute calculation unit 222 using predetermined calculations and calculates a local score for each information element. For example, the local score calculation unit 223 weights confidence and causal consistency as elements that give a positive contribution, and bias, consistency deviation and ethical deviation as elements that give a negative contribution, normalizes the median value to a predetermined range using a nonlinear function, and then weights it according to freshness to obtain a local score in the range of 0 to 1.

[0025] The integrity index calculation unit 224 calculates the validity score, transparency index, and accountability score of the decision-making process based on relational information of the decision-making structure data and local scores calculated by the local score calculation unit 223. For example, the integrity index calculation unit 224 searches for logical paths using supporting and disproving relationships and calculates the validity score while adjusting the contributions of supporting and disproving paths. The integrity index calculation unit 224 further calculates the transparency index based on the disclosure status of metadata associated with information elements, the presence or absence of explanations of relational information, the presence or absence of explicit reproduction procedures, and the density of the network structure, and calculates the accountability score based on the degree of satisfaction of explanation requirements, the success rate of reproduction attempts, and the balance of approval paths.

[0026] The integrated index calculation unit 225 performs an integrated process on the validity score, transparency index, and accountability score calculated by the integrity index calculation unit 224 based on a set number of weight coefficients, and calculates an integrated integrity index for the decision-making process. The integrated index calculation unit 225 obtains the integrated integrity index as a continuous value between 0 and 1, for example, by combining a linear combination of each index with nonlinear normalization.

[0027] The output unit 226 outputs the integrated integrity index calculated by the integrated index calculation unit 225 and various scores calculated by the integrity indicator calculation unit 224 to the client terminal 120 via the interface device 230. The output unit 226 displays, for example, a graph showing the relationship between claims, evidence, counterarguments and sources, a numerical display of the integrated integrity index, a list of values ​​for each indicator, and a proposal for improving integrity.

[0028] The update unit 227 controls the update process of the integrated integrity index when attribute values ​​or local scores are updated for some information elements included in the decision structure data. Based on the sensitivity coefficient calculated by the integrated index calculation unit 225, the update unit 227 estimates the changes in the validity score, transparency index, and accountability score, and selects whether to perform a recalculation targeting all information elements or a differential update process, depending on the magnitude of the error.

[0029] The parameter learning unit 228 updates the parameter set, such as weight coefficients, used in the local score calculation unit 223 and the integrated index calculation unit 225, based on a training dataset that includes audited decision cases and expert evaluation scores. The parameter learning unit 228 iteratively updates the parameter set to minimize an objective function that includes an error index based on the difference between the integrated integrity index and the evaluation score, and a regularization term based on the size of the parameter set, and finalizes the parameter set when the performance on the validation data satisfies predetermined conditions.

[0030] With the above configuration, the information processing device 110 efficiently performs the following processes: comprehensively evaluate the information elements included in the decision-making process, calculate an integrated integrity index that combines validity, transparency, and accountability, and present the results and improvement suggestions to the user.

[0031] Figure 3 shows an example of decision-making structure data 300 generated by the information processing device 110.

[0032] The decision-making structure data 300 has a structure in which various information elements in the decision-making process are represented as nodes, and the logical relationships between information elements are represented as directed edges.

[0033] The decision structure data 300 includes an assertion node 310, a justification node 320, a source node 330, and a rebuttal node 340. A directed edge 350 indicates a support relationship from the justification node 320 to the assertion node 310. A directed edge 360 ​​indicates a source relationship from the source node 330 to the justification node 320 or the rebuttal node 340. A directed edge 370 indicates a rebuttal relationship from the rebuttal node 340 to the assertion node 310.

[0034] As an example of a decision-making process, we can consider a scenario where we decide whether or not to launch a new service into the market. In this case, for example, the claim node 310 shows informational elements corresponding to the conclusion that "the service should be launched." One of the rationale nodes 320 could represent the explanation that "demand is expected based on market research findings," while another rationale node 320 could represent the explanation that "revenue forecasts meet the target." The source node 330 represents research reports, statistical data, etc., that serve as the basis for each piece of rationale information, and the directed edge 360 ​​indicates which source information is supported by which document. The counter-argument node 340 represents informational elements corresponding to opposing opinions, such as "the supply system is unstable and there is a risk that service quality cannot be maintained," and the directed edge 370 indicates the relationship that provides counter-argument to the claim node 310. One of the source nodes 330 could represent a risk assessment report, etc., corresponding to the counter-argument node 340.

[0035] The information processing device 110 uses the decision structure data 300 to calculate attribute values ​​and local scores for each information element of the claim node 310, evidence node 320, source node 330, and rebuttal node 340. Furthermore, it can extract support paths and rebuttal paths based on the support relationships, source relationships, and rebuttal relationships represented by the directed edges 350, 360, and 370. By aggregating these paths, the information processing device 110 calculates the validity score, transparency index, and accountability score of the decision process and determines the integrated integrity index.

[0036] The number of nodes and the depth of the hierarchy included in the decision-making structure data 300 are not limited to the example shown in Figure 3. The information processing device 110 may also hold decision-making structure data 300 that represent a more complex decision-making process, such as a structure including multiple claim nodes 310, a structure including multiple stages of evidence nodes 320 and rebuttal nodes 340, or a structure in which multiple source nodes 330 are connected to a single evidence node 320.

[0037] Next, the processing procedures in the information processing device 110 will be described with reference to Figures 4 to 12. Figure 4 shows the basic processing flow when the information processing device 110 evaluates the integrity of the decision-making process. Figures 5 to 10 show the detailed procedures for the attribute value calculation process, local score calculation process, validity score calculation process, transparency index calculation process, accountability score calculation process, and integrated integrity index calculation and improvement suggestion process shown in Figure 4. Figure 11 shows a flowchart of the differential update process that efficiently updates the integrated integrity index calculated once in Figure 4, and Figure 12 shows a flowchart of the learning process that determines the weight coefficients and other parameters used in Figures 4 and 11 through learning.

[0038] Figure 4 shows an example of the processing procedure when the information processing device 110 evaluates the integrity of a decision-making process. When the information processing device 110 receives a request to evaluate the integrity of a decision-making process from a client terminal 120 or an external system 130, it starts the processing flow shown in Figure 4. The information processing device 110 may also receive an identifier that identifies the decision-making process to be evaluated, along with the conditions for acquiring the input data to be used for the evaluation.

[0039] In step S401, the information processing device 110 acquires data related to the decision-making process. The information processing device 110 receives meeting records, decision-making reports, and business system log data from the client terminal 120. The information processing device 110 also acquires external information such as statistical data and market research reports from the external system 130. The information processing device 110 formats the acquired data into a format usable in subsequent processing and stores it as text information and structured information that explains the decision-making process.

[0040] In step S402, the information processing device 110 generates decision structure data based on data relating to the decision-making process. The information processing device 110 divides the text and speech logs included in the data relating to the decision-making process into sentence-level or utterance-level information elements using natural language processing techniques, morphological analysis, dependency parsing, machine learning-based classification models, and other known techniques. Based on the description content and metadata of each information element, the information processing device 110 uses techniques such as classification models or rule-based judgment processing to classify information elements that indicate conclusions as claim information, information elements that indicate reasons or evidence supporting the claim information as evidence information, information elements that indicate opposing opinions or risks to the claim information as counter-evidence information, and information elements that indicate information sources such as statistical data, reports, and records from external databases as source information.

[0041] The information processing device 110 estimates, based on the descriptions of claim information, evidence information, counter-evidence information, and source information, reference descriptions, and correspondence between identifiers, using techniques such as relation estimation models or rule-based extraction processes, support relationships indicating which evidence information supports which claim information, counter-evidence relationships indicating which counter-evidence information disputes which claim information, and source relationships indicating which source information corresponds to which evidence information or counter-evidence information.

[0042] The information processing device 110 registers information elements corresponding to claim information, evidence information, counter-evidence information, and source information as nodes, and registers support relationships, counter-evidence relationships, and source relationships as directed edges connecting the nodes, thereby representing the decision-making structure data as a directed graph structure. Regardless of the type of analysis technique used, the information processing device 110 can generate decision-making structure data with the structure shown in Figure 3.

[0043] In step S403, the information processing device 110 performs attribute value calculation processing for each node and edge information element included in the decision structure data generated in step S402. The information processing device 110 treats each node and edge registered in the decision structure data as an information element, processes the data related to the decision process one by one, and assigns multiple attribute values ​​to each information element. The attribute value calculation processing will be described later with reference to Figure 5.

[0044] In step S404, the information processing device 110 performs a local score calculation process for each information element included in the decision-making structure data, based on the decision-making structure data generated in step S402 and the attribute values ​​assigned to each information element in step S403. The information processing device 110 calculates the local score for each information element using the multiple attribute values ​​assigned to each information element as input. The local score calculation process will be described later with reference to Figure 6.

[0045] In step S405, the information processing device 110 performs a validity score calculation process for the decision-making process based on the decision-making structure data generated in step S402 and the local scores calculated for each information element in step S404. The information processing device 110 takes the relational information of the decision-making structure data and the local scores of each information element as input and calculates a validity score indicating the validity of the decision-making process. The validity score calculation process will be described later with reference to Figure 7.

[0046] In step S406, the information processing device 110 performs a transparency index calculation process for the decision-making process based on the metadata and relational information associated with each information element included in the decision-making structure data generated in step S402. The information processing device 110 takes as input an index indicating the disclosure status of metadata such as source information, creation date and time, creator information, etc. associated with each information element, an index indicating the explanation status regarding support relationships, counter-evidence relationships, and source relationships between information elements, an index indicating the reproducibility for replication by a third party, and an index indicating the structural complexity of the decision-making structure data, and calculates a transparency index indicating the transparency of the decision-making process. The transparency index calculation process by the information processing device 110 will be described later with reference to Figure 8.

[0047] In step S407, the information processing device 110 performs an accountability score calculation process for the decision-making process based on the decision-making structure data generated in step S402, the metadata and relational information associated with each information element included in the decision-making structure data, and the explanation log and reproduction trial log related to the decision-making process. The information processing device 110 calculates an accountability score indicating the accountability of the decision-making process, using an indicator showing the degree of sufficiency of the explanation, an indicator showing reproducibility, and an indicator showing the balance of the approval route as input. The accountability score calculation process by the information processing device 110 will be described later with reference to Figure 9.

[0048] In step S408, the information processing device 110 calculates the integrated integrity index of the decision-making process and performs improvement suggestion processing based on the validity score calculated in step S405, the transparency index calculated in step S406, and the accountability score calculated in step S407. The information processing device 110 takes the validity score, the transparency index, the accountability score, and various pre-set thresholds as input to calculate the integrated integrity index of the decision-making process, extracts indicators where the integrity level is insufficient, generates improvement candidates, and outputs improvement suggestions. The calculation of the integrated integrity index and improvement suggestion processing by the information processing device 110 will be described later with reference to Figure 10.

[0049] Figure 5 shows an example of the attribute value calculation process.

[0050] In step S501, the information processing device 110 selects one information element from among the information elements included in the decision-making structure data for which an attribute value will be calculated. The information processing device 110 may select the unprocessed information elements in order, or it may select them based on priority according to importance or type.

[0051] In step S502, the information processing device 110 calculates the degree of bias for the information element selected in step S501. Before or at the start of the attribute value calculation process, the information processing device 110 performs clustering on multiple information elements included in the decision-making structure data and estimates statistical distribution parameters, including the mean and variance of the values ​​in each cluster. Then, in step S502, the information processing device 110 compares the statistical distribution parameters of the cluster to which the selected information element belongs with the value corresponding to the selected information element and determines the degree of bias based on indicators such as the distance from the cluster center and the outlier score. The information processing device 110 may use known methods such as the K-means method, hierarchical clustering, or Gaussian mixture models for the clustering process and statistical distribution estimation process, and may use a Z-score assuming a normal distribution or a score based on robust statistics as the formula for calculating the degree of bias. The information processing device 110 interprets information elements with a larger degree of bias as information elements that deviate more greatly from the statistical distribution, and uses the degree of bias as an attribute value that gives a negative contribution in the subsequent local score calculation process.

[0052] For example, the information processing device 110 may calculate the degree of bias bi using the following formula, where xi is the value corresponding to the selected information element, μcluster is the mean value of the cluster to which it belongs, and σcluster is the standard deviation. bi = |μcluster-xi| / σcluster

[0053] In step S503, the information processing device 110 calculates consistency deviations for the information elements selected in step S501. Before or at the start of the attribute value calculation process, the information processing device 110 converts the content of each information element included in the decision-making structure data into a vector representation using a natural language processing model or the like, and stores it in the storage device 220. Then, in step S503, the information processing device 110 calculates the similarity between the vector of the selected information element and the vectors of other information elements using cosine similarity or the like, and determines consistency deviations based on the deviation amount compared with the mean or maximum value of the similarity distribution, or negative similarity for information elements whose semantic content is judged to be in conflict. The information processing device 110 may generate vector representations using known techniques such as document embedding models, word embedding models, or sentence feature extraction models, and may use known measures such as cosine similarity, Euclidean distance, or dot product to calculate similarity. The information processing device 110 interprets information elements with larger consistency deviations as having a greater degree of inconsistency in content with other information elements, and uses the consistency deviation as an attribute value that gives a negative contribution in the subsequent local score calculation process.

[0054] For example, the information processing device 110 may use the following formula to calculate the consistency deviation ci, where ei is the vector representation corresponding to the selected information element and ej is the vector representation of a representative comparison target. ci = 1 - cos(sim(ei,ej))

[0055] In step S504, the information processing device 110 calculates the degree of causal consistency for the information elements selected in step S501. Before or at the start of the attribute value calculation process, the information processing device 110 constructs a causal relationship graph that shows the presence and direction of causal relationships between information elements based on the relationship information of the claim information, evidence information, and counter-evidence information included in the decision structure data. Then, in step S504, the information processing device 110 identifies whether the selected information element is located in a position corresponding to an assumption, a position corresponding to a conclusion, or a position corresponding to an intermediate mediating element in the causal relationship graph, and compares the presence and direction of causal relationships between the selected information element and other information elements with the correspondence between assumptions and conclusions assumed in the decision-making process.

[0056] The information processing device 110 quantifies the degree of agreement with the causal structure by, for example, giving a high evaluation when the relationship from supporting information to assertion information is consistent with the direction of causality from premise to conclusion, and giving a low evaluation when there is a reverse causal relationship from assertion information to supporting information, or when counter-evidence information is connected to logically unrelated nodes, thereby determining the degree of consistency of the causal relationship as the causal consistency score. In calculating the causal consistency score, the information processing device 110 may use known techniques such as causal graph models, structural equation models, or rule-based inference processing, or it may store assumed causal structures as rule sets for each business domain in the storage device 220 and define the causal consistency score based on the degree of agreement with those rule sets. The information processing device 110 interprets information elements with a high causal consistency score as information elements that play a role consistent with the logical structure of the entire decision-making process, and uses the causal consistency score as an attribute value that gives a positive contribution in the subsequent local score calculation process.

[0057] For example, the information processing device 110 may calculate the causal consistency score gi using the following formula. gi=Corr(Xcause,Ydecision) Here, Corr() represents a correlation coefficient such as Pearson's product-moment correlation coefficient, Xcause represents a causal variable indicating the presence or strength of an information element, and Ydecision represents a variable indicating the decision result. The information processing device 110 may define the degree of causal consistency using other correlation indicators such as Spearman's rank correlation coefficient or a score based on a causal inference model instead of a correlation coefficient.

[0058] In step S505, the information processing device 110 calculates the degree of ethical deviation for the information element selected in step S501. The degree of ethical deviation is an indicator of the extent to which the content of the selected information element deviates from ethical norms. Before or at the start of the attribute value calculation process, the information processing device 110 defines several types of ethical violations, such as discrimination, harassment, conflict of interest, and privacy violations, based on applicable ethical norms, guidelines, laws, and internal regulations, and sets a weight indicating the importance of each type of ethical violation. Then, in step S505, the information processing device 110 takes the text content of the selected information element as input and estimates the probability of each type of ethical violation occurring using known technologies such as violation detection models, rule-based inspection processes, and keyword matching processes. The information processing device 110 combines the estimated probability of occurrence of each type of ethical violation with the weight set for each type and calculates the degree of ethical deviation, which indicates the magnitude of the ethical deviation, using a calculation formula such as a weighted sum. The information processing device 110 interprets information elements with a high degree of ethical deviation as being more significantly deviating from ethical norms, and uses the degree of ethical deviation as an attribute value that gives a negative contribution in the subsequent local score calculation process.

[0059] For example, the information processing device 110 may define a weight vk indicating importance and a probability pk of violation occurrence for each type k of ethical violation, and calculate the degree of ethical deviation ui using the following formula. ui = Σk(vk × pk)

[0060] In step S506, the information processing device 110 calculates the freshness of the information element selected in step S501. Freshness is an indicator of how new the information related to the selected information element is relative to the current time. Before or at the start of the attribute value calculation process, the information processing device 110 associates the acquisition time or update time with each information element included in the decision-making structure data and stores it in the storage device 220. Then, in step S506, the information processing device 110 calculates the time difference between the acquisition time or update time associated with the selected information element and the current time, and determines the freshness based on a decay function that takes the time difference as input. The information processing device 110 may use a function form as the decay function in which the value decreases monotonically as the time difference increases, or it may use known function forms such as an exponential decay function, a function with half-life as a parameter, or a multi-step weighting function that multiplies by different coefficients for each interval. The information processing device 110 sets the freshness value range to be between 0 and 1, assigning a freshness value close to 1 to information elements with a small time difference and an freshness value close to 0 to information elements with a large time difference. The information processing device 110 interprets that information elements with a higher freshness value are more relevant to current decision-making, and uses freshness as an attribute value for weighting local scores in the subsequent local score calculation process.

[0061] For example, the information processing device 110 may calculate the freshness fi using the following formula, where Δt is the difference between the time information related to the selected information element and the current time, and λ is the decay rate. fi = exp(-λΔt) The information processing device 110 may, for example, set λ such that the half-life is 90 days.

[0062] In step S507, the information processing device 110 calculates the reliability score for the information element selected in step S501. The reliability score is an indicator of how reliable the information source and creation process related to the selected information element are. Before or at the start of the attribute value calculation process, the information processing device 110 defines a basic score for each information source, an evaluation score for each creator, and an evaluation score according to the verification history, based on the type of information source, the attributes and track record of the information creator, and the history of peer review and approval by third parties, and sets weights for each factor and stores them in the storage device 220. Then, in step S507, the information processing device 110 obtains metadata such as the type of information source, creator, and verification history associated with the selected information element, and calculates the reliability score by linearly combining the basic score of the information source, the evaluation score of the creator, and the score based on third-party verification according to the set weight coefficients. In calculating the reliability score, the information processing device 110 may use a scoring model based on a simple weighted sum, or it may use a machine learning model to estimate the reliability score from multiple factors. The information processing device 110 normalizes the confidence value range to be between 0 and 1, interprets information elements with higher confidence values ​​as more reliable, and uses the confidence value as an attribute value that gives a positive contribution in the subsequent local score calculation process.

[0063] For example, the information processing device 110 may use SSource as the source source score, Sauthor as the creator score, and Sverify as the third-party verification score, and calculate the confidence score ri using the following formula. ri=w1×Ssource+w2×Sauthor+w3×Sverify Here, w1, w2, and w3 represent the weight coefficients. Alternatively, we can assume w1 + w2 + w3 = 1.

[0064] In step S508, the information processing device 110 determines whether there are any remaining information elements for which the bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability have not been calculated. The information processing device 110 manages a processed flag or the number of processed items for the set of information elements included in the decision-making structure data, and determines whether there are any unprocessed information elements by checking whether there are any information elements for which the processed flag has not been set, or whether the number of processed items has reached the total number of information elements. If the information processing device 110 determines that there are unprocessed information elements, it returns to step S501, selects a new information element, and repeats the processing from step S502 to step S507. If the information processing device 110 determines that the bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability have been calculated for all information elements, it terminates the attribute value calculation process and proceeds to the next processing step shown in Figure 4.

[0065] The information processing device 110 uses each of the calculation formulas described for attribute value calculation processing as an example, and may adopt other calculation formulas that are suitable for the purpose and operating conditions as needed.

[0066] Figure 6 shows an example of the local score calculation process. In the local score calculation process, the information processing device 110 calculates a local score for each information element, taking the degree of bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability calculated by the attribute value calculation process as input.

[0067] In step S601, the information processing device 110 selects one information element from among the information elements included in the decision structure data to calculate a local score for. The information processing device 110 may scan the list of information elements stored in the decision structure data in order, or it may select information elements in ascending order of a pre-assigned identifier.

[0068] In step S602, the information processing device 110 retrieves the attribute values ​​associated with the selected information element. The information processing device 110 refers to the calculation results of the attribute values ​​in the storage device 220 and reads out the values ​​for bias bi, consistency deviation ci, causal consistency gi, ethical deviation ui, freshness fi, and reliability ri. For example, the information processing device 110 may search a table structure that manages the calculation results of attribute values ​​stored in the storage device 220 using the identifier of the information element as the key, and expand each attribute value into a work area in memory.

[0069] In step S603, the information processing device 110 calculates a linear combination value based on the acquired attribute values. The information processing device 110 treats the confidence level ri and causal consistency level gi as positive contributing elements, and the bias level bi, consistency deviation ci, and ethical deviation level ui as negative contributing elements, and obtains the intermediate value vi by multiplying each by the coefficient set for it and then adding and subtracting them.

[0070] For example, the information processing device 110 may calculate the intermediate value vi using the following formula. vi=α1×ri+α2×gi-β1×bi-β2×ci-β3×ui Here, ri represents confidence, gi represents causal consistency, bi represents bias, ci represents consistency deviation, and ui represents ethical deviation, while α1, α2, β1, β2, and β3 are coefficients indicating the contribution of each attribute value. The information processing device 110 may impose constraints such that the coefficients α1, α2, β1, β2, and β3 are greater than 0 and less than 1, and α1 + α2 + β1 + β2 + β3 = 1, and optimize the coefficients through a learning process using audited decision cases.

[0071] In step S604, the information processing device 110 applies a nonlinear normalization process to the intermediate value vi. The information processing device 110 maps the value to a range of 0 to 1 using the logistic function σ and calculates the normalized value s'i.

[0072] For example, the information processing device 110 may calculate the normalized value s'i using the following formula. s'i = σ(vi) = 1 / (1 + exp(-vi)) Here, exp() represents the exponential function. The information processing device 110 may perform normalization using another monotonically increasing nonlinear function instead of the logistic function.

[0073] In step S605, the information processing device 110 weights the normalized value s'i to reflect the freshness fi. The information processing device 110 defines the freshness fi as a value in the range of 0 to 1 and calculates the local score si by multiplying the normalized value s'i by the freshness fi.

[0074] For example, the information processing device 110 may calculate the local score si using the following formula. si = fi × sdali The information processing device 110 may set its parameters such that s'i is reflected almost directly for recent information elements with freshness fi close to 1, and the local score si is suppressed to a small value for older information elements with freshness fi close to 0.

[0075] In step S606, the information processing device 110 determines whether there are any remaining information elements for which the local score calculation process is incomplete. If the information processing device 110 determines that there are information elements for which local scores have not been recorded, it returns to step S601, selects another information element, and repeats the processes from step S602 to step S605. If the information processing device 110 determines that the calculation of local scores si has been completed for all information elements, it terminates the local score calculation process and supplies the local scores to subsequent processes such as the validity score calculation process shown in Figure 4.

[0076] The information processing device 110 uses the calculation formulas described for local score calculation as examples, but may adopt other calculation formulas or learning methods that are suitable for evaluation purposes and operating conditions as needed.

[0077] Figure 7 shows an example of the validity score calculation process. In the validity score calculation process, the information processing device 110 calculates a validity score that indicates the validity of the decision-making process based on the decision-making structure data.

[0078] In step S701, the information processing device 110 identifies the claim node corresponding to the decision conclusion from among the nodes included in the decision structure data. Before or at the start of processing, the information processing device 110 may receive node identifiers that are candidate conclusions as input from the user or from an external system, and register those nodes as claim nodes. If no candidate conclusion is provided, the information processing device 110 refers to the attribute information of each node stored in the decision structure data, searches for nodes that have been assigned a type label indicating a conclusion, a type label indicating a decision result, or a flag indicating final approval, and extracts those nodes as candidate claim nodes. If there are multiple candidate claim nodes, the information processing device 110 analyzes the input / output patterns of edges indicating support relationships, selects nodes that are supported by other nodes but do not support other nodes as top candidates, and decides one of the top candidates as the claim node. As a process for identifying claim nodes, the information processing device 110 may use a rule-based search process based on node attributes or a search algorithm that utilizes a graph structure.

[0079] In step S702, the information processing device 110 enumerates logical paths ending at claim nodes. Based on relational information indicating support relationships, refutation relationships, and source relationships stored in the decision structure data, the information processing device 110 performs a depth-first search on the directed graph and sequentially extracts paths to reach claim nodes. If the same node is revisited during the search, the information processing device 110 detects it as a cycle and excludes paths containing cycles from the enumeration. If the path length exceeds a predetermined maximum length, the information processing device 110 terminates the search and enumerates logical paths of a realistic length.

[0080] In step S703, the information processing device 110 calculates a path score for each of the enumerated paths. The information processing device 110 reads the local score of each node obtained by the local score calculation process shown in Figure 6 and aggregates the local scores of the nodes included in the path. The information processing device 110 may use, for example, a simple average of the local scores as the path score, or it may use a weighted average weighted according to the type of information element, such as giving greater weight to the underlying information, as the path score.

[0081] For example, the information processing device 110 may calculate the route score qk using the following formula, where Nk is the number of nodes included in the route pk, and skj is the local score of the j-th node from the beginning. qk = (1 / Nk) × (sk1 + sk2 + ... + skNk)

[0082] In step S704, the information processing device 110 classifies each path into supporting paths and refuting paths. The information processing device 110 refers to the relationship information contained in the paths and treats paths that do not contain any refuting relationships as supporting paths that support the asserted information. The information processing device 110 treats paths that contain at least one refuting relationship as refuting paths that show refutation to the asserted information, and manages the two separately.

[0083] In step S705, the information processing device 110 weights the path scores according to the length of each path and the information quality of the information elements contained in each path. The information processing device 110 sets weighting coefficients that give relatively large weights to short paths with few nodes and paths that contain many nodes with high local scores, and small weights to paths with an extremely large number of nodes and paths that contain many nodes with low local scores. For example, the information processing device 110 defines the weights by combining a function that decays with respect to the path length and a coefficient that is proportional to the average local score, and obtains the weighted path score by multiplying it by the path score.

[0084] For example, the information processing device 110 may calculate the path weight wk using the following formula, where Lk is the length of path pk and qk is the path score calculated in step S703. wk = α × qk × {1 / (1+Lk)} Here, α represents a positive coefficient that adjusts the overall scale. The information processing device 110 may also calculate the weighted path score q'k as q'k = wk × qk.

[0085] In step S706, the information processing device 110 adjusts the contribution of the refutation path. The information processing device 110 separates the weighted score of the supporting path from the weighted score of the refutation path, and controls the magnitude of the deduction from the validity score by multiplying the score of the refutation path by a coefficient between 0 and 1 (inclusive). The information processing device 110 may store the coefficient in the storage device 220 as a parameter set by the user or determined by pre-training, and adjust the influence of the refutation information according to the field and application.

[0086] For example, the information processing device 110 may calculate the value Sn' of the counter-evidence path after adjusting its contribution using the following formula, where Sp is the sum of the weighted scores of the supporting paths and Sn is the sum of the weighted scores of the counter-evidence paths. Sn′=κ×Sn Here, κ represents a coefficient between 0 and 1.

[0087] In step S707, the information processing device 110 aggregates the weighted scores of the supporting and refuting paths to calculate a validity score for the decision-making process. The information processing device 110 obtains an intermediate value by performing calculations such as subtracting the sum of the coefficient-adjusted scores of the refuting paths from the sum of the scores of the supporting paths, and applies a normalization process using a monotonically increasing function to obtain a validity score SV that is normalized to a range of 0 to 1. If SV is close to 1, the information processing device 110 determines that the asserted information is sufficiently supported by high-quality evidence and that the influence of the refuting information is small in the decision-making process. If SV is close to 0, the information processing device 110 determines that the supporting information is insufficient or that the influence of the refuting information is large in the decision-making process.

[0088] For example, the information processing device 110 may calculate the validity score SV using the following formula, based on the value Sp which represents the overall contribution of the supporting path and the value Sn' which is adjusted for the contribution of the counter-evidence path obtained in step S706. SV = σ(γ1 × Sp - γ2 × Sn′) Here, γ1 and γ2 represent positive coefficients, and σ represents the logistic function. For example, the information processing device 110 may use the function given by σ(x) = 1 / (1 + exp(-x)) for any real number x as the logistic function.

[0089] The information processing device 110 uses each calculation formula described for calculating the validity score as an example, and may adopt a different calculation formula that is suitable for the purpose and operating conditions as needed. For example, the information processing device 110 may calculate the path score w(p) using the following formula, based on the local score si of the node at position i in the node sequence constituting the path p and the coefficient κi→i+1 indicating the strength of the edge from node i to node i+1.

number

number

[0090] Figure 8 shows an example of the transparency index calculation process.

[0091] In step S801, the information processing device 110 calculates the disclosure rate for the information elements included in the decision-making structure data. The information processing device 110 determines whether metadata such as the creator's name, creation date and time, data set used, and calculation procedure used is attached to each information element. The information processing device 110 considers information elements for which all required items are present as "disclosed," and calculates the disclosure rate by dividing the number of disclosed information elements by the total number of information elements. The information processing device 110 expresses the extent to which the sources of the information used in the decision-making process are clear as a numerical value, for example, by including in the determination conditions whether the source of the supporting information and the source of the counter-evidence information are clearly indicated.

[0092] For example, the information processing device 110 has Nopen as the number of disclosed information elements and Ntotal as the total number of information elements, and the disclosure rate φ ー It can also be calculated using the following formula. φ ー =Nopen / Ntotal Here, Nopen represents the number of information elements to which all of the specified metadata, such as the creator's name, creation date and time, data set used, and calculation procedure used, is attached, while Ntotal represents the total number of information elements included in the decision structure data.

[0093] In step S802, the information processing device 110 calculates the relationship explanation rate for the relationship information included in the decision structure data. The information processing device 110 determines whether explanatory text, rule names, judgment criteria, etc., are recorded for each edge indicating a support relationship, a refutation relationship, and a source relationship. The information processing device 110 calculates the relationship explanation rate, which indicates the extent to which the meaning of the relationship is explicitly stated, by dividing the number of edges with explanations by the total number of edges. The information processing device 110 highly values ​​edges with explanations such as "supported because there is a statistically significant difference" or "refuted because the sample size is insufficient," and calculates the relationship explanation rate to be low when there are many edges without explanations.

[0094] For example, the information processing device 110 has Nexp as the number of explained edges and Nedge as the total number of edges, and the relational explanation rate ψ ー It can also be calculated using the following formula. ψ ー =Nexp / Nedge Here, Nexp represents the number of edges that show support, falsification, and source relationships, and that are accompanied by explanatory information such as explanations of reasons, rule names, and judgment criteria, while Nedge represents the total number of edges included in the decision structure data.

[0095] In step S803, the information processing device 110 calculates the recall rate, which indicates the reproducibility of the decision-making process. The information processing device 110 evaluates whether the processing procedure is clearly defined for each information element and related information to the extent that a third party can reach the same result by following the same procedure. For example, the information processing device 110 considers information elements that record the data extraction conditions, aggregation methods, and statistical model parameter settings used as "reproducible," and calculates the recall rate based on the proportion of reproducible information elements and the proportion of reproducible paths. The information processing device 110 may also recalculate by comparing with audited past cases and use the degree of agreement between the original decision result and the recalculated result as a score.

[0096] For example, the information processing device 110 determines the number of paths that are determined to be reproducible as Nrep, the total number of paths to be evaluated as Npath, and the recall rate ρ ー It can also be calculated using the following formula. ρ ー =Nrep / Npath Here, Nrep records the data extraction conditions, aggregation methods, and statistical model parameter settings used, and represents the number of paths that the information processing device 110 has determined can lead to the same result if a third party follows the same procedure. Npath represents the total number of paths that reach the claim node.

[0097] In step S804, the information processing device 110 calculates a density penalty index (complexity) to suppress information overcrowding caused by the complexity of the decision-making structure data. The information processing device 110 analyzes the distribution of the number of nodes, the number of edges, the average number of branches, and the path length, and evaluates whether it exceeds a level where it becomes difficult for ordinary users to visually track the graph structure. The information processing device 110 defines the density penalty index to be large when the average degree is large or when there are many extremely long paths, and it plays a role in deducting points from the transparency evaluation for decision-making structure data with an excessively complex structure.

[0098] For example, the information processing device 110 may calculate the node density ratio Dnode using the following formula, where Nnode is the number of nodes included in the decision-making structure data and Nref is the number of reference nodes. Dnode=Nnode / Nref Furthermore, for example, the information processing device 110 may define the density penalty index H(Gd) by the following formula. H(Gd) = 1 - Dnode Here, Nref is a baseline value corresponding to the upper limit of scale that users consider understandable. It may be set so that the node density ratio Dnode increases and the density penalty index H(Gd) decreases as the number of nodes approaches the baseline value.

[0099] In step S805, the information processing apparatus 110 integrates the disclosure rate, relationship explanation rate, recall rate, and density penalty index to calculate a transparency indicator indicating the transparency of a decision-making process. The information processing apparatus 110 takes the disclosure rate, relationship explanation rate, and recall rate as elements that make positive contributions, and takes the density penalty index as an element that makes a negative contribution. An intermediate value is obtained by multiplying each of these by a respectively set weighting coefficient, then performing addition and subtraction, and a normalization process using a monotonically increasing function is applied to obtain a transparency indicator normalized to a range of 0 to 1 inclusive. When the transparency indicator is close to 1, the information processing apparatus 110 determines that the information used for decision-making, the relationships between said information, and the reproduction procedure are sufficiently disclosed, and the decision-making process is structurally easy to understand. When the transparency indicator is close to 0, the information processing apparatus 110 determines that the information disclosure and relationship explanation are insufficient, or the decision-making process has a complex structure and is difficult to trace.

[0100] For example, the information processing apparatus 110 may set the disclosure rate φ ー , the relationship explanation rate ψ ー , the recall rate ρ ー and the density penalty index H(Gd) to integrate and calculate the transparency indicator T(d). The information processing apparatus 110 may calculate the intermediate value zT, for example, by the following formula. zT=α1×φ ー +α2×ψ ー +α3×ρ ー -α4×H(Gd) Here, α1, α2, and α3 represent weighting coefficients indicating the magnitude of contribution of the disclosure rate, relationship explanation rate, and recall rate, respectively, and α4 represents a weighting coefficient indicating the magnitude of contribution of the density penalty index. Then, the information processing apparatus 110 may apply the logistic function σ as a monotonically increasing function to the calculated zT, and calculate the transparency indicator T(d) by the following formula. T(d)=σ(zT)=1 / (1+exp(-zT / To)) Here, To represents a temperature parameter, which may be set to To=1, for example. The information processing apparatus 110 may optimize the coefficients α1 to α4 by machine learning based on label information of audited decision-making cases.

[0101] The information processing device 110 uses the calculation methods and integration methods for each indicator described in the transparency indicator calculation process as an example, but may adopt other calculation methods or weighting methods that are suitable for the required level and operating conditions of each field as needed.

[0102] Figure 9 shows an example of the accountability score calculation process.

[0103] In step S901, the information processing device 110 calculates the degree of explanation satisfaction. The information processing device 110 defines explanation items as perspectives that should be explained to the decision-making process, such as objectives, prerequisites, evaluation criteria, status of alternative considerations, and risk assessment results. The information processing device 110 analyzes the recorded explanation documents and decision-making structure data and determines whether a description exists for each explanation item and whether the specificity and quantity of the description meet predetermined standards. The information processing device 110 aggregates the degree of achievement for each explanation item and calculates the degree of explanation satisfaction based on the proportion of items covered and the weight assigned to each item. For example, the information processing device 110 significantly reduces the degree of explanation satisfaction if a high-importance item is missing, and sets a high value for the degree of explanation satisfaction if all important items are specifically explained.

[0104] For example, the information processing device 110 sets the index value ek to 1 if the criteria are met for each explanatory item k, and sets the index value ek to 0 if the criteria are not met, and multiplies it by a weight wk that indicates the importance of item k. The information processing device 110 may also calculate the explanatory satisfaction E(d) using the following formula. E(d)=〔Σk(wk×ek)〕 / 〔Σkwk〕 Here, E(d) represents the degree of explanatory satisfaction for the decision data d, wk represents the importance of explanatory item k, and ek is a value that is 1 if item k is sufficiently explained, and 0 otherwise. The information processing device 110 is configured, for example, to set a large wk for items with high importance, and to make E(d) smaller if important items are missing.

[0105] In step S902, the information processing device 110 calculates the success rate of reproduction trials. The information processing device 110 collects the results of a third party re-executing the decision-making process according to the procedures recorded in the explanatory document and decision-making structure data, and determines whether or not it matches the original decision result. The information processing device 110 calculates the success rate of reproduction trials by dividing the number of cases that matched for multiple reproduction trials by the total number of trials. For example, the information processing device 110 uses the percentage of cases in which an auditor reached the same conclusion when evaluating using the same data set and the same rule set as the success rate of reproduction trials, and evaluates the success rate of reproduction trials low if the procedure description is insufficient and the results cannot be reproduced even when the procedure is followed.

[0106] For example, the information processing device 110 may calculate the success rate of reproducible trials R(d) using the following formula, where Nsuccess is the number of matching trials and Ntrial is the total number of trials. R(d) = Nsuccess / Ntrial Here, Nsuccess and Ntrial are both positive integers, and 0 ≤ R(d) ≤ 1. The information processing device 110 records the percentage of auditors who reached the same conclusion when evaluating the same data set and the same rule set as Nsuccess / Ntrial, and evaluates R(d) to be small if the procedure description is insufficient and reproducibility is low.

[0107] In step S903, the information processing device 110 calculates the approval path balance index (degree of responsibility distribution). The information processing device 110 refers to log information and decision-making structure data that show the approval workflow related to the decision-making process and identifies which departments, positions, and areas of expertise were involved in the decision-making. The information processing device 110 calculates the distribution of roles involved in the approval and evaluates whether the approval is excessively biased towards a particular department or authority level. For example, the information processing device 110 sets the approval path balance index high when multiple stakeholders participate in the approval in a balanced manner, and low when the approval is completed by only a few individuals or when there is a large bias in expertise. The information processing device 110 may also calculate statistics such as variance and entropy from the number of types of roles involved in the approval and the number of approvals for each role, and define the approval path balance index based on these.

[0108] For example, the information processing device 110 may calculate the approval route balance index D(d) using the following formula, where M is the number of personnel involved in the approval process and L is the length of the approval route. D(d)=(M-1) / (L+1) Here, M represents the number of personnel involved in the approval process, and L represents the number of approval steps included in the approval route. The information processing device 110 may be configured to consider a distributed, flat approval route when M is large and L is short, and to make D(d) large. Alternatively, when M is small and L is long, it may be considered a route where approvals are concentrated among a few personnel, and the device may evaluate it to make D(d) small.

[0109] In step S904, the information processing device 110 integrates the explanation sufficiency, the success rate of reproduction trials, and the approval path balance index to calculate an accountability score. The information processing device 110 treats the explanation sufficiency and the success rate of reproduction trials as the primary positive contributors to accountability, and the approval path balance index as a complementary positive contributor indicating the fairness of the explanation and the degree of multifaceted verification. The information processing device 110 obtains an intermediate value by multiplying each index by a weight coefficient set and adding them, and obtains an accountability score normalized to a range of 0 to 1 by applying a normalization process using a monotonically increasing function. If the accountability score is close to 1, the information processing device 110 determines that the necessary explanation items are sufficiently covered, the results are stable even in reproduction trials by third parties, and the approval process is structured without bias. If the accountability score is close to 0, the information processing device 110 determines that the decision-making process has insufficient explanations, low reproducibility, or bias in the approval path.

[0110] For example, the information processing device 110 may calculate the accountability score A(d) using the following formula. A(d)=σ(η1×E(d)+η2×R(d)+η3×D(d)) Here, η1, η2, and η3 represent weight coefficients, which are set within a range greater than 0 and less than or equal to 1. The information processing device 110 may optimize η1, η2, and η3 by machine learning based on audited decision data, or it may set them manually according to the policies of the field or organization. The information processing device 110 normalizes the range of A(d) to 0 or greater and less than or equal to 1 by using the logistic function σ(x) = 1 / (1 + exp(-x)). If the accountability score A(d) is close to 1, the information processing device 110 determines that the decision-making process adequately covers all necessary explanation items, the results are stable in reproduction trials by third parties, and the approval process is unbiased. If the accountability score A(d) is close to 0, the information processing device 110 determines that the decision-making process has insufficient explanations, low reproducibility, or bias in the approval path.

[0111] The information processing device 110 uses the calculation methods and weighting coefficients of each indicator used in the accountability score calculation process as an example, and may adopt different sets of explanation items, evaluation procedures for reproduction trials, and definitions of approval path balance indicators depending on the purpose of use and the characteristics of the target business.

[0112] Figure 10 shows an example of the integrated integrity index calculation and improvement suggestion process.

[0113] In step S1001, the information processing device 110 acquires indicators to be used for calculating the integrated integrity index and proposing improvements. The information processing device 110 refers to the storage device 220 and reads indicator values ​​defined for each unit of the decision-making process, such as the validity score, transparency index, and accountability score. The information processing device 110 may also acquire auxiliary indicators useful for identifying areas for improvement, such as local score statistics and aggregated attribute values, as needed.

[0114] In step S1002, the information processing device 110 calculates an integrated integrity index based on the acquired indicators. The information processing device 110 treats the validity score, transparency index, and accountability score as elements that give a positive contribution, and calculates the median value by multiplying each by a weight coefficient set for it and adding them together. The information processing device 110 may, for example, apply a monotonically increasing function such as the logistic function to the median value to calculate an integrated integrity index with a range of 0 or more and 1 or less. If the integrated integrity index is close to 1, the information processing device 110 evaluates that the overall integrity level of the decision-making process is high, and if the integrated integrity index is close to 0, it evaluates that there is a problem in one of the aspects.

[0115] For example, the information processing device 110 may calculate the integrated integrity index I(d) using the following formula. I(d)=μ1×SV(d)+μ2×T(d)+μ3×A(d) Here, SV(d) represents the validity score for decision-making process d, T(d) represents the transparency index for decision-making process d, and A(d) represents the accountability score for decision-making process d. μ1, μ2, and μ3 represent weighting coefficients corresponding to validity, transparency, and accountability, respectively, and are non-negative values ​​that may be normalized, for example, such that μ1 + μ2 + μ3 = 1. The information processing device 110 may read and use values ​​optimized by a learning process performed in advance using audited decision cases and expert evaluations as the weighting coefficients μ1, μ2, and μ3 from the storage device 220.

[0116] Furthermore, in step S1002, the information processing device 110 may perform a normalization process to map the intermediate value I(d) to a range of 0 to 1. For example, the information processing device 110 may calculate the normalized value I'(d) using a logistic function with temperature parameter T in the following formula. I'(d) = 1 / (1 + exp(-I(d) / T)) Here, T represents a temperature parameter, and may be set to, for example, T=1. The information processing device 110 may record I'(d) as the integrated integrity index, or it may use I(d) as an index value and I'(d) as a normalized value for display.

[0117] In step S1003, the information processing device 110 determines whether the integrated integrity index and each component index are below the set threshold, and identifies the index that is below the threshold. The information processing device 110 reads, for example, the baseline values ​​for the validity score, the transparency index, and the accountability score from the storage device 220 and compares them with the respective index values. The information processing device 110 extracts the index that is below the baseline value as a weak area and identifies which aspect of integrity is lacking.

[0118] Furthermore, in step S1003, the information processing device 110 may also perform a comparison with the respective threshold values ​​for the validity score SV(d), the transparency index T(d), and the accountability score A(d), in addition to the integrated integrity index I′(d). For example, the information processing device 110 may set the threshold τSV for the validity score, the threshold τT for the transparency index, and the threshold τA for the accountability score to 0.75, and identify indices where SV(d) < τSV, T(d) < τT, or A(d) < τA as indices with insufficient integrity levels. If the integrated integrity index I′(d) is smaller than a predetermined threshold τI, the information processing device 110 may perform a weak area extraction process assuming that at least one constituent index is below the threshold.

[0119] In step S1004, the information processing device 110 analyzes the relationship between indicators below a threshold and local scores and attribute values, and extracts candidates for integrity improvement. For example, if the transparency indicator is low, the information processing device 110 identifies nodes and edges with low disclosure rates and relationship explanation rates, and extracts areas where additional explanatory text or explicit source information is needed as improvement candidates. If the validity score is low, the information processing device 110 searches for evidence nodes and counter-evidence nodes with particularly low local scores, and extracts areas where scrutiny of information or additional verification is desirable as improvement candidates. If the accountability score is low, the information processing device 110 may detect areas where explanatory items are missing or where approval routes are unevenly distributed, and extract candidates for supplementing explanatory items or reviewing the approval flow.

[0120] For example, the information processing device 110 may extract integrity improvement candidates using the contribution of each information element to the integrated integrity index. The information processing device 110 may evaluate the impact of changes in the local score si on the validity score SV, the transparency index T, and the accountability score A, and calculate the contribution Ci corresponding to information element i using, for example, the following formula. Ci=μ1×(∂SV / ∂si)+μ2×(∂T / ∂si)+μ3×(∂A / ∂si) Here, ∂SV / ∂si, ∂T / ∂si, and ∂A / ∂si represent partial derivatives or sensitivity approximations indicating the sensitivity of the validity score SV, transparency index T, and accountability score A to the local score si, respectively. The information processing device 110 may preferentially extract information elements whose absolute value of contribution Ci is greater than or equal to a predetermined threshold as candidates that have a large effect on improving the integrated integrity index, and use them as targets for improvement proposal generation in step S1005.

[0121] In step S1005, the information processing device 110 generates improvement proposals based on the integrity improvement candidates extracted in step S1004. First, the information processing device 110 refers to the type label and related indicator values ​​assigned to each improvement candidate and classifies whether the improvement target belongs to validity, transparency, or accountability. Depending on the classification result, the information processing device 110 refers to a template selection rule that uses the type of improvement target, the role of the target node, and the affected indicator values ​​as input items, and determines the document template to be used. The information processing device 110 then fills in the slots in the selected template with the information element names obtained from the improvement candidates, related attribute values, current indicator values, and expected target levels, and generates improvement proposal text in natural language.

[0122] The information processing device 110 may, when generating improvement proposals, call a generative AI such as a natural language generation model using the selected template and structured data to be embedded in the slots as input, and obtain the text output by the generative AI as a candidate proposal. The information processing device 110 may then examine the obtained candidate proposals based on the template for consistency of terminology, presence or absence of prohibited words, and upper limit of sentence length, and may adopt them as improvement proposals after applying rule-based correction processing as necessary.

[0123] For example, for an improvement candidate in which the absence of source information is detected as a factor in the decline of the transparency index, the information processing device 110 may generate a proposal statement such as "Record [metadata item name] for [node name] and clearly indicate the source" by inserting the name of the target node and the name of the missing metadata item. For an improvement candidate in which evidence nodes with low local scores are extracted as factors in the decline of the validity score, the information processing device 110 may generate a statement recommending the collection of additional data, verification through alternative channels, and the implementation of expert reviews. For an improvement candidate in which bias in the approval channel is detected as a factor in the decline of the accountability score, the information processing device 110 may generate an approval flow revision proposal by inserting the names of departments or roles that are not involved. For each improvement proposal generated, the information processing device 110 calculates a priority score based on the expected amount of improvement of the relevant index and the estimated work cost, sorts the proposals in descending order of priority score, and outputs them, thereby supporting the user in efficiently planning integrity improvement work.

[0124] For example, the information processing device 110 may calculate a priority score Pj using the following formula, based on the expected improvement amount ΔIj and the estimated work cost Cj associated with each improvement proposal j. Pj = α × ΔIj - β × Cj Here, Pj represents the priority score of improvement proposal j, ΔIj represents the expected increase in the integrated integrity index if improvement proposal j is implemented, Cj represents the work cost required to implement improvement proposal j, and α and β are weighting coefficients that adjust the balance between the expected improvement and the work cost. The information processing device 110 may estimate ΔIj using the sensitivity Ci of the integrated integrity index to the local score si, which is obtained based on additional information, or it may determine α and β based on statistical learning or expert evaluation. The information processing device 110 sorts the improvement proposals in descending order of the calculated priority score Pj and presents them to the user in order of priority, thereby supporting the user in efficiently planning integrity improvement work.

[0125] In step S1006, the information processing device 110 outputs the integrated integrity index and the generated improvement suggestions. The information processing device 110 sends screen data including the index values ​​and improvement suggestions to the client terminal 120, displaying it in a list or graph format for users auditing the decision-making process. The information processing device 110 may also record the trend of the integrated integrity index and the history of adopted improvement suggestions, and store them in the storage device 220 for use in subsequent differential update processing and learning processing.

[0126] The information processing device 110 uses the calculation formulas described for calculating the integrated integrity index and suggesting improvements as examples, but may adopt other calculation formulas, normalization functions, or contribution definitions that are suitable for the purpose and operating conditions as needed.

[0127] The information processing device 110 calculates the validity score SV, transparency index T, accountability score A, and integrated integrity index I for each decision-making process by executing the processes shown in Figures 4 to 10. The information processing device 110 stores intermediate data such as decision-making structure data, local score si for each node, path information, and weight coefficients in the storage device 220, in association with these index values. In the differential update process shown in Figure 11, the information processing device 110 uses the index values ​​and intermediate data stored in the storage device 220 as reference values.

[0128] Figure 11 shows an example of a differential update process that efficiently updates the integrated integrity index.

[0129] The information processing device 110 starts the differential update process shown in Figure 11 when it detects an update to an information element included in the decision-making process, after the integrated integrity index has been calculated once. For example, the information processing device 110 monitors events such as editing operations of explanatory text and supporting information by the user, reacquisition of data from the external system 130, and updates of attribute values ​​associated with the learning process, and when any of these events occur, it determines that an updated information element exists and starts the differential update process shown in Figure 11.

[0130] The information processing device 110 may also set up a periodic execution process to review the integrated integrity index at predetermined time intervals, separate from the differential update process. The information processing device 110 may automatically start the differential update process shown in Figure 11 as part of the periodic execution process, or it may start a full recalculation process using the processing flow shown in Figure 4 as needed. Furthermore, the information processing device 110 may be configured to manually start the differential update process shown in Figure 11 when it receives instruction input from an auditor or user.

[0131] In step S1101, the information processing device 110 acquires the information elements that have been updated from among the information elements included in the decision-making process. The information processing device 110 monitors the user's editing history, data reacquired from the external system 130, and the results of attribute value updates due to the learning process, and extracts the nodes whose content has been changed and the edges indicating the relationships between nodes as updated information elements. In order to identify the state before the update, the information processing device 110 may acquire the attribute values ​​and local scores of the old version corresponding to the updated information elements from the storage device 220.

[0132] In step S1102, the information processing device 110 recalculates the local score for the updated information element. The information processing device 110 applies the same procedure as the attribute value calculation process shown in Figure 5, but limited to the updated information element, to recalculate the bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability. The information processing device 110 takes the recalculated attribute values ​​as input and applies the same procedure as the local score calculation process shown in Figure 6 to obtain the updated local score si'. The information processing device 110 calculates the difference Δsi = si' - si between the updated local score si held in the storage device 220 and the updated local score si, and temporarily holds it in the storage device 220 so that it can be used in the subsequent differential update process.

[0133] In step S1103, the information processing device 110 estimates the amount of change in each index value based on the change in the local score. The information processing device 110 refers to the path information that was generated when the validity score calculation process shown in Figure 7 was executed during the reference calculation and is stored in the storage device 220, and identifies the path that includes the update information element. For each identified path, the information processing device 110 calculates the effect that the local score difference Δsi of each node included in the path has on the path score, and estimates the amount of change in the validity score ΔSV as a first-order approximation by aggregating the amount of change in the path score for paths that include the update information element. For example, the information processing device 110 may calculate the amount of change in the validity score ΔSV using the following formula. ΔSV ≈ Σi((∂SV / ∂si) × Δsi) Here, si represents the local score, Δsi represents the difference in local scores, and ∂SV / ∂si indicates the sensitivity of the validity score SV to the local score si. The information processing device 110 may estimate the sensitivity ∂SV / ∂si by a pre-training process using audited decision cases and store it in the storage device 220.

[0134] Similarly, for the transparency index T and accountability score A, the information processing device 110 refers to the list of information elements to be evaluated and the weight coefficients recorded when the processing shown in Figures 8 and 9 was performed during the baseline calculation, and re-evaluates the parts associated with the updated information elements. The information processing device 110 estimates the change in the transparency index ΔT and the change in the accountability score ΔA by first-order approximation using the sensitivity ∂T / ∂si and ∂A / ∂si to the local score difference Δsi. The information processing device 110 reduces the computational load by assuming that the change in indices not involving the updated information elements is 0 and excluding them from the differential update.

[0135] Furthermore, the information processing device 110 may calculate a contribution coefficient Ci for each updated information element in order to comprehensively evaluate the degree to which changes in local scores si have an impact on the integrated integrity index. The information processing device 110 may calculate the contribution coefficient Ci using, for example, the following formula. Ci=μ1×(∂SV / ∂si)+μ2×(∂T / ∂si)+μ3×(∂A / ∂si) Here, SV represents the validity score, T represents the transparency index, A represents the accountability score, and μ1, μ2, and μ3 represent weight coefficients set according to the importance of each index. The information processing device 110 may significantly reduce the computational cost compared to recalculating all nodes by estimating μ1, μ2, and μ3 through a learning process using audited decision cases, and evaluating the change in the integrated integrity index using the contribution coefficient Ci and local score difference Δsi for a small number of local scores si that have been updated.

[0136] In step S1104, the information processing device 110 performs a differential update of the integrated integrity index using the estimated index changes. The information processing device 110 reads the old integrated integrity index I stored in the storage device 220 and reads the weight coefficients μ1, μ2, and μ3 that were set for use in the integrated integrity index calculation process shown in Figure 10 in the base calculation of the integrated integrity index. The information processing device 110 takes the change in the validity score ΔSV, the change in the transparency index ΔT, and the change in the accountability score ΔA as input and calculates the change in the integrated integrity index ΔI using the formula ΔI = μ1 × ΔSV + μ2 × ΔT + μ3 × ΔA. Based on the old integrated integrity index I and the change in the integrated integrity index ΔI, the information processing device 110 calculates the new integrated integrity index I as I new = I old + ΔI. The information processing device 110 performs a clipping process that sets the integrated integrity index I to 0 if it falls below 0, and sets it to 1 if it exceeds 1, and saves the updated integrated integrity index I to the storage device 220.

[0137] In step S1105, the information processing device 110 estimates an error to evaluate the accuracy of the integrated integrity index obtained by the differential update. The information processing device 110 may select a portion or representative sample of the logical path containing the updated information elements, recalculate the local scores of the information elements included in the selected logical path, and recalculate the validity score, transparency index, accountability score, and integrated integrity index using the already calculated local scores for the information elements included in other logical paths that do not contain the updated information elements. The information processing device 110 compares the integrated integrity index I Verification obtained by the recalculation with the integrated integrity index I New obtained by the differential update in step S1104, and calculates the error of the integrated integrity index from the difference between the two. The information processing device 110 uses either the absolute error, squared error, or relative error as the error index for the integrated integrity index, and determines the reliability of the differential update result by comparing the error index with a pre-set tolerance threshold. If the error index exceeds the tolerance threshold, the device may perform a full recalculation in step S1106 using the processing flow shown in Figure 4.

[0138] The information processing device 110 may calculate the difference between the integrated integrity index obtained by differential update and the integrated integrity index recalculated locally as the error index E. For example, the information processing device 110 may define E as the absolute value of the difference between the differential update result and the recalculation result, and determine whether E is less than or equal to the upper error limit ε. For example, the information processing device 110 may set the upper error limit ε to 0.05 and determine that the accuracy of the differential update result is within an acceptable range if E ≤ ε. The information processing device 110 may decide whether to perform a full recalculation of the integrated integrity index based on whether the error index E exceeds the recalculation threshold τ. For example, the information processing device 110 may set the recalculation threshold τ to 0.1 and, if it determines that E > τ, perform a recalculation of various indices for the entire decision-making process using the processing flow shown in Figure 4. The information processing device 110 may define multiple operating modes depending on the value of the error index E, such as adopting the differential update result as is if E is less than or equal to ε, and displaying a warning to the user if E exceeds ε and is less than or equal to τ.

[0139] In step S1106, the information processing device 110 determines whether a complete recalculation of the integrated integrity index is necessary based on the estimation error, and executes the recalculation process if necessary. If the information processing device 110 determines that the estimation error is below the acceptable error threshold, it adopts the new integrated integrity index obtained by the differential update as the final value and terminates the differential update process. If the information processing device 110 determines that the estimation error exceeds the acceptable error threshold, it restarts the processing flow shown in Figure 4 and recalculates the integrated integrity index by sequentially executing the attribute value calculation process, local score calculation process, validity score calculation process, transparency index calculation process, accountability score calculation process, and integrated integrity index calculation process for the entire decision-making process. When the recalculation process is completed, the information processing device 110 saves the updated integrated integrity index and related indicators to the storage device 220 and terminates the differential update process shown in Figure 11.

[0140] Figure 12 shows an example of a learning process in which various parameters used to calculate the integrated integrity index are determined through learning.

[0141] In step S1201, the information processing device 110 acquires training data to be used for learning. The information processing device 110 reads out the calculation results of decision structure data, attribute values ​​of each node, local scores, validity scores, transparency indicators, accountability scores, and integrated integrity index for multiple decision processes stored in the storage device 220. The information processing device 110 further acquires evaluation labels and target integrity values ​​assigned by auditors or experts to each decision process as training data, and constructs a training dataset. For example, the information processing device 110 may associate a target value close to 1 with cases that have been certified as having a high level of integrity, and a target value close to 0 with cases that have been certified as having problems.

[0142] In step S1202, the information processing device 110 initializes the parameters to be learned. The information processing device 110 defines a parameter vector θ whose elements are all the parameters to be learned, such as the weight coefficients used for calculating local scores, the temperature parameter T of the logistic function, the decay rate used for calculating freshness, the weight coefficients used for calculating the integrated integrity index, and the weights used for calculating the contribution coefficient in the differential update process.

[0143] For example, the information processing device 110 may set initial values ​​for the weight vector μ included in the parameter vector θ such that each component is greater than or equal to 0 and the sum of all components is 1 (μj≧0, Σjμj=1). The information processing device 110 sets each component of the parameter vector θ using a uniform random number, a normally distributed random number, or a predetermined initial value, and determines the initial value of the parameter vector θ at the start of learning.

[0144] In step S1203, the information processing device 110 calculates an integrated integrity index for each training data using the current parameter vector θ. For each decision-making process di obtained in step S1201, the information processing device 110 sequentially applies the attribute value calculation process shown in Figure 5, the local score calculation process shown in Figure 6, the index calculation processes shown in Figures 7 to 9, and the integrated integrity index calculation process shown in Figure 10 to obtain an integrated integrity index DDIM(di;θ) corresponding to each case di. To speed up learning, the information processing device 110 may reuse already calculated intermediate results and recalculate parts affected by parameter changes.

[0145] In step S1204, the information processing device 110 calculates the learning objective function. For each decision-making process, the information processing device 110 compares the integrated integrity index I with the target integrity value Y assigned to the training data and evaluates the difference as the loss. The information processing device 110 may use a known loss function, such as squared error, absolute error, or logistic loss, to determine the loss value for each case, and calculate the average or sum over all cases as the objective function value J. In order to suppress excessive parameter values, the information processing device 110 may add a regularization term based on the parameter norm to the objective function to obtain parameters with high generalization performance.

[0146] For example, the information processing device 110 calculates the learning objective function. The information processing device 110 may define the objective function J(θ) for n training data di(i=1,2,…,n) based on the error between the integrated integrity index DDIM(di;θ) and the teacher label Li, and a parameter regularization term, as follows: J(θ)=(1 / n)Σi[DDIM(di;θ)-Li] 2 +λ∥θ∥ 2 Here, λ is a coefficient indicating the strength of L2 regularization, and may be set to, for example, λ = 0.01. The information processing device 110 calculates the regularization term λ∥θ∥ as the average of the squared errors of each case. 2 By adding this, the integrated integrity index is brought as close as possible to the evaluation label Li, while controlling the parameters to prevent them from becoming excessively large.

[0147] In step S1205, the information processing device 110 updates the parameters in a direction that minimizes the objective function. The information processing device 110 obtains the gradient vector by partially differentiating the objective function J for each parameter and updates the parameter vector using an optimization method such as gradient descent, stochastic gradient descent, or quasi-Newton's method. The information processing device 110 sets the update width, which corresponds to the learning rate, as a control parameter, and may perform clipping to prevent divergence if the learning rate is too large, or apply scheduling processing to improve the convergence speed if the learning rate is too small.

[0148] For example, the information processing device 110 may obtain the gradient vector ∂J / ∂θ by partially differentiating J(θ) with respect to each parameter component, and update the parameters according to the following equation using an optimization method such as gradient descent or stochastic gradient descent. θ′ = θ - η × (∂J / ∂θ) Here, η represents the learning rate. The information processing device 110 may prevent divergence by clipping the gradient vector if the learning rate η is too large, or apply a scheduling process that gradually reduces η according to the number of epochs if η is too small. The information processing device 110 normalizes the weight vector μ so that it remains non-negative and its sum is 1 after updating.

[0149] In step S1206, the information processing device 110 verifies the validity of the updated parameters. The information processing device 110 divides the training data into training data and validation data, and, for example, uses a 5-fold cross-validation method to update the parameters on the training data for each division. Then, it calculates the integrated integrity index DDIM(di;θ) on the validation data and finds the validation mean of the objective function J(θ). The information processing device 110 determines whether J(θ) in the validation data has decreased compared to the previous iteration, and if it has not decreased, it may adjust the learning rate η or the regularization coefficient λ.

[0150] In step S1207, the information processing device 110 performs convergence determination of the learning process and parameter determination. The information processing device 110 determines that the learning has converged if the amount of change in the objective function J(θ) over multiple consecutive iterations is less than a predetermined threshold, or if the number of iterations reaches the upper limit. If convergence is determined, the information processing device 110 saves the latest parameter vector θ to the storage device 220 as a determined parameter to be used in the integrated integrity index calculation process and differential update process. If convergence is not determined, the information processing device 110 returns to step S1203 and continues the learning process by repeatedly recalculating the integrated integrity index and re-evaluating the objective function using the updated parameter vector θ. After parameter determination, the information processing device 110 may calculate the integrated integrity index on an independent evaluation dataset not used for learning and verify the generalization performance of the learning results offline.

[0151] The information processing device 110 is described as an example of the calculation formulas used in the learning process, the calculation methods for each indicator, and the setting of weight coefficients, but is not limited to these. Depending on the purpose of use and the characteristics of the target business, the information processing device 110 may change the types of explanation items targeted by the learning process, the definitions of evaluation indicators that constitute each component of the validity score, transparency indicator, and accountability score, and various evaluation methods, including the evaluation procedure for reproduction trials.

[0152] Figure 13 shows an example of a display screen 1300 that displays an example of the integrated integrity index and improvement suggestions output by the information processing device 110 to the client terminal 120.

[0153] The display screen 1300 is configured to allow users to intuitively understand the status of the decision-making process by presenting the structure of the decision-making process, the numerical value of the integrated integrity index, and the improvement suggestion text.

[0154] The left-hand area of ​​the display screen 1300 functions as a graph display area that visualizes the decision structure of the decision-making process. Based on the decision-making structure data received from the information processing device 110, the client terminal 120 draws multiple nodes representing "claims," ​​"evidence," "counter-evidence," and "sources" as circular shapes. The client terminal 120 draws arrows from evidence nodes to claim nodes as "Supports," arrows from counter-evidence nodes to claim nodes as "Rebuts," and arrows from source nodes to evidence nodes as "Source." By looking at the graph display area, users can grasp at a glance which information elements support the claims, which information elements refute the claims, and which information elements are sources for the evidence.

[0155] The upper right area of ​​the display screen 1300 functions as the integrated integrity index display area. The client terminal 120 receives the calculation result of the integrated integrity index transmitted from the information processing device 110 and displays the value along with label information such as "Integrated Integrity Index" and "DDIM". In Figure 13, the integrated integrity index is displayed as "0.84" as an example, but the client terminal 120 displays a different value depending on the content and update status of the decision-making process. By looking at the integrated integrity index display area, users can quantitatively grasp the integrity level of the entire decision-making process.

[0156] The lower right area of ​​the display screen 1300 functions as an improvement suggestion display area. The information processing device 110 sends the improvement suggestion text generated by the integrated integrity index calculation and improvement suggestion processing shown in Figure 10 to the client terminal 120. The client terminal 120 displays the received improvement suggestion text in the improvement suggestion display area. In Figure 13, the improvement suggestion display area displays, as an example, the suggestion text "Please clearly indicate the source of the counter-evidence." By referring to the suggestion text displayed in the improvement suggestion display area, the user can specifically understand which information elements need to be modified and how to modify them to improve the integrated integrity index.

[0157] The information processing device 110 uses the configuration and display content of each area on the display screen 1300 as an example, and the arrangement of nodes, whether or not labels are displayed, the display format of the integrated integrity index, and the level of detail in the improvement suggestion statement may be changed depending on the purpose of use and user attributes.

[0158] This embodiment enables integrated evaluation of decision-making through collaboration between humans and artificial intelligence (AI), which has not been achieved with conventional artificial intelligence technologies or governance, risk, and compliance management systems that focus on explainability. The technology according to this embodiment builds a technological foundation that can be applied in all fields where accountability is required, such as the corporate, administrative, medical, and financial sectors.

[0159] In this embodiment, the information processing system 100 calculates local scores from multiple types of attribute values, such as bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability, and derives a validity score, transparency index, and accountability score based on the local scores. Since the information processing system 100 mechanically obtains attribute values ​​using statistical methods, natural language processing methods, and metadata analysis methods, it can quantify the quality of each information element included in the decision-making process without relying on human intervention. As a result, the information processing system 100 can evaluate the validity, transparency, and accountability of the decision-making process in a comparable manner as objective numerical values.

[0160] The information processing system 100 activates differential update processing when it detects an updated information element, estimates the change in local scores and each index value for the updated information element, and updates the integrated integrity index differentially. Since the information processing system 100 uses the sensitivity and contribution coefficients of each index value to changes in local scores to determine the change using a first-order approximation, it can significantly reduce the amount of computation compared to recalculating the entire decision-making process. Furthermore, the information processing system 100 evaluates the difference between the differential update result and the partial exact recalculation result as an error index, and performs a full recalculation if the error exceeds a threshold, thereby automatically adjusting the balance between evaluation accuracy and processing efficiency. As a result, the information processing system 100 can maintain the integrated integrity index in near real-time even in environments where the decision-making process is frequently updated.

[0161] The information processing system 100 has a configuration that uses audited decision-making cases as training data and automatically updates weights for local score calculation, parameters for freshness calculation, weights for integrated integrity index calculation, and weights for differential update processing based on an objective function. The information processing system 100 optimizes the parameters to minimize the difference between the integrated integrity index and the target integrity value assigned by experts as loss, so that the evaluation results are adjusted to be consistent with the experts' judgment criteria. As a result, the information processing system 100 can build an integrity evaluation model that is tailored to the regulatory requirements and risk tolerance of each implementing organization, and can continuously improve the evaluation accuracy through operation.

[0162] The information processing system 100 outputs a display screen 1300 to the client terminal 120, which includes a graph display of the decision structure, a numerical display of the integrated integrity index, and a display of improvement suggestions. The information processing system 100 visually displays the relationships between claim nodes, evidence nodes, counter-argument nodes, and source nodes, and simultaneously presents the integrated integrity index and improvement suggestions to the right, allowing users to intuitively understand which parts need to be corrected. Specifically, users can refer to the messages presented as improvement suggestions, select the corresponding nodes in the graph display area, and perform operations such as adding explanatory text and source information. As a result, the information processing system 100 can clearly present problems in the decision-making process even to users without specialized knowledge, and promote interactive operation toward improving integrity.

[0163] (II) Addendum The identifiers of components described herein (e.g., prefixes and symbols such as "First" and "Second") are for convenience only and do not limit the number, order, function, arrangement, etc., of the components. The same identifier may refer to different components in different embodiments, and one component may also perform the function of another component. Therefore, the identifiers of components described herein are not intended to limit the technical scope, functional scope, or scope of rights of the components, and each component should be interpreted flexibly according to the context of its embodiment.

[0164] In this specification, "interface device" means a component that may include one or more interface devices. Such interface devices may include, but are not limited to, I / O (Input / Output) interface devices, communication interface devices, or combinations thereof. For example, an I / O interface device may be configured to function as a user interface and may include at least one input device (e.g., a keyboard, a pointing device) and / or an output device (e.g., a display). These I / O interface devices may be configured to have communication functions that allow connection to remote computing devices, in which case the I / O interface device can also operate as a communication interface device. Furthermore, the communication interface device may include identical communication means (e.g., multiple NICs (Network Interface Cards)) or a combination of different types of communication means (e.g., a NIC and an HBA (Host Bus Adapter)). This enables a flexible configuration that ensures connectivity with heterogeneous systems. Interface devices configured in this way are not limited to a specific hardware configuration and can accommodate future technological advancements and diversification of embodiments.

[0165] In this specification, “storage device” means a component that may include at least one storage device. Depending on the intended use and system configuration, such storage devices may be classified, for example, into “memory” which temporarily holds data during operation and “persistent storage device” which retains data even after power is lost. Memory can function as a temporary storage medium accessible by the processor and may include volatile, non-volatile, or a combination thereof memory devices. Specifically, examples include, but are not limited to, volatile memory such as DRAM (Dynamic Random Access Memory) and SRAM (Static RAM), and non-volatile memory such as MRAM (Magnetoresistive RAM) and ReRAM (Resistive RAM). Persistent storage devices are components intended for long-term data storage and include devices using non-volatile storage media. Specifically, these may include HDD (Hard Disk Drive), SSD (Solid State Drive), NVMe (Non-Volatile Memory Express) drives, etc. Next-generation storage technologies such as phase-change memory may also be included as examples of storage device configurations. A storage device configured in this way is not limited by the type or architecture of the storage medium, and can accommodate future technological advancements and diversification of implementations.

[0166] In this specification, "processor" means a component that may include an arithmetic unit or circuit capable of performing at least one processing function. Depending on the application, such a processor may include, for example, a microprocessor device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or it may include implementations using dedicated circuits such as an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit). A processor may consist of a single core, a multi-core, or a single processor core. A processor may be configured to implement processing functions using a computer program, directly implemented using hardware circuits, or a hybrid configuration combining these. When processing is performed by a program, the processor may perform processing in cooperation with other components such as memory devices or interface devices. In this specification, a particular function may be described as a "part," but such a function can be realized by a program executed by the processor, an implementation using circuits, or a combination thereof. Therefore, such a function can be considered to be at least a part of the processor. The program may be supplied from an external program source. Examples of program sources include, but are not limited to, network-connected program distribution servers and computer-readable non-temporary storage media. Processors configured in this way are not limited to specific hardware configurations or implementation forms, and can adapt to future technological advancements and diversification of implementations.

[0167] In this specification, "system" means a set of components that may include at least one computing resource. Such a system may consist of one or more physical computers (dedicated hardware, on-premises servers, etc.) or virtualized computing resources (cloud infrastructure, virtual machines, containers, etc.). A system may include, but is not limited to, cloud computing systems, cluster configurations, serverless environments, etc. A system may be configured within a single device or in a configuration in which multiple computing resources cooperate via a network. Each component may be logically integrated or physically separated. Such a system may also include components such as processors, storage devices, and interface devices, and these components may be implemented as physical devices or realized as virtual configurations. A system configured in this way is not limited to a specific hardware configuration, implementation form, deployment form, etc., and can accommodate future technological advancements and diversification of embodiments.

[0168] The embodiments described above have, for example, the following features.

[0169] (1) An information processing system (e.g., information processing system 100) for evaluating the integrity of a decision-making process, comprising: an attribute calculation unit (e.g., attribute calculation unit 222, information processing device 110, see Figure 5) that calculates multiple attribute values ​​(e.g., attribute values ​​for bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability) for each information element (each node included in the decision-making structure data 300) included in the decision-making structure data (e.g., decision-making structure data 300) related to the decision-making process, for each information element, for each information element, for each information element, for each information element, for each information element, for each information element, for each local score calculation unit (e.g., local score si) that integrates the multiple attribute values ​​by predetermined calculations and calculates a local score (e.g., local score si) corresponding to the information element; and a validity score (e.g., The system comprises: an integrity index calculation unit (e.g., integrity index calculation unit 224, information processing device 110, see Figures 7 to 9) that calculates a validity score (SV), a transparency index (e.g., transparency index T) indicating the transparency of the decision-making process, and an accountability score (e.g., accountability score A) indicating the accountability of the decision-making process; an integrated index calculation unit (e.g., integrated index calculation unit 225, information processing device 110, see Figure 10) that performs integrated processing on the validity score, the transparency index, and the accountability score of the decision-making process based on a plurality of pre-set weighting coefficients (e.g., weighting coefficients μ1, μ2, and μ3 for calculating the integrated integrity index) indicating the integrity of the decision-making process; and an output unit (e.g., output unit 226, information processing device 110, see Figures 10 and 13) that outputs the integrated integrity index of the decision-making process.

[0170] In the above configuration, for example, the information processing system evaluates each information element included in the decision-making structure data as multiple attribute values ​​and local scores, and outputs it as an integrated integrity index via a validity score, transparency index, and accountability score. With this configuration, for example, users can comprehensively and objectively grasp the integrity of the decision-making process as a single indicator called the integrated integrity index. Furthermore, in the above configuration, for example, the information processing system calculates the integrated integrity index after independently calculating the validity score, transparency index, and accountability score. With this configuration, for example, users can identify weaknesses not only in the overall level of integrity but also in which aspects of validity, transparency, and accountability exist, and clarify areas that should be prioritized for improvement in audits and management decisions. Also, in the above configuration, for example, the information processing system outputs the integrated integrity index for each case. With this configuration, for example, users can compare the integrity levels of decision-making processes across different cases and time series, and use it as a governance indicator for continuous monitoring and quantitative tracking of improvement effects.

[0171] (2) The attribute calculation unit (for example, attribute calculation unit 222, information processing device 110, see Figure 5) performs the following processes for each information element (for example, claim node 310, evidence node 320, source node 330, and rebuttal node 340 included in the decision structure data (for example, decision structure data 300, see Figure 3)): (i) calculate a degree of bias (for example, a degree of bias bi calculated by the attribute calculation unit 222) indicating the degree of deviation of the information element from the distribution based on its statistical properties (see, for example, step S502 in Figure 5); (ii) calculate a consistency deviation (for example, a consistency deviation ci calculated by the attribute calculation unit 222) indicating the degree of inconsistency between the content of the information element and other information elements based on semantic similarity (see, for example, step S503 in Figure 5); and (iii) calculate the relationship between the information element and other information elements based on a causal relationship model. (iv) A process to calculate a degree of causal consistency (e.g., a degree of causal consistency gi calculated by the attribute calculation unit 222) indicating the degree of consistency of the causal relationship (see, for example, step S504 in Figure 5), (iv) A process to calculate a degree of ethical deviation (e.g., a degree of ethical deviation ui calculated by the attribute calculation unit 222) indicating the magnitude of the ethical deviation of the information element based on the risk of ethical violation (see, for example, step S505 in Figure 5), (v) A process to calculate a degree of freshness (e.g., a degree of freshness fi calculated by the attribute calculation unit 222) indicating the newness of the information over time (see, for example, step S506 in Figure 5), and (vi) A process to calculate a degree of reliability (e.g., a degree of reliability ri calculated by the attribute calculation unit 222) indicating the degree of reliability of the information element based on the reliability of the information source, creator and verification status by third parties (see, for example, step S507 in Figure 5).

[0172] In the above configuration, for example, the attribute calculation unit calculates attribute values ​​for each information element included in the decision-making structure data by performing at least one of the following processes: bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability. With this configuration, for example, users can grasp the quality of information elements from multiple perspectives, including statistical properties, semantic consistency, causal structure, ethical aspects, temporal freshness, and reliability of the information source. Furthermore, in the above configuration, for example, the information processing system generates attribute values ​​based on causal relationship models and ethical violation risk, in addition to evaluations based on statistical properties and semantic similarity. With this configuration, for example, users can automatically and quantitatively detect breakdowns in causal relationships and ethical problems that were difficult to detect with conventional rule-based evaluations, thereby improving the accuracy of integrity evaluations. Also, in the above configuration, for example, the information processing system is configured to allow users to select and execute necessary processes from a list of enumerated processes. With this configuration, for example, users can combine and use the necessary attribute values ​​according to the target business and regulatory requirements, and can flexibly design an integrity evaluation that fits the domain while conserving computational resources.

[0173] (3) The local score calculation unit, for each information element included in the decision structure data (e.g., decision structure data 300), treats the confidence level (e.g., confidence level ri) and causal consistency level (e.g., causal consistency level gi) of the information element as positive contributing elements, and the bias level (e.g., bias level bi), consistency deviation (e.g., consistency deviation ci), and ethical deviation level (e.g., ethical deviation level ui) of the information element as negative contributing elements. It integrates the attribute values ​​of confidence level, causal consistency level, bias level, consistency deviation, and ethical deviation level according to predetermined rules, and calculates the local score (e.g., local score si) of the information element based on a value weighted according to the freshness (e.g., freshness fi) of the integrated value (see, for example, Figure 6).

[0174] In the above configuration, for example, the local score calculation unit treats reliability and causal consistency as positive contributing factors, and bias, consistency deviation, and ethical deviation as negative contributing factors. These attribute values ​​are integrated according to predetermined rules, and the integrated value is weighted according to freshness to calculate the local score. With such a configuration, for example, users can obtain integrity evaluation results that highly value causally valid, reliable, and new information elements, and automatically suppress the influence of statistically anomalous, inconsistent, and ethically risky information elements, as well as outdated information elements. Furthermore, in the above configuration, for example, the information processing system stably calculates the local score using predetermined rules and weighting based on freshness when integrating multiple attribute values. With such a configuration, for example, users can calculate validity scores, transparency indicators, and accountability scores based on local scores that reflect the actual impact on integrity while appropriately mitigating the influence of information elements with extreme attribute values, thereby achieving a highly reliable integrity evaluation for the entire decision-making process.

[0175] (4) The decision-making structure data described above includes claim information (e.g., an information element corresponding to claim node 310), evidence information (e.g., an information element corresponding to evidence node 320), rebuttal information (e.g., an information element corresponding to rebuttal node 340), and source information (e.g., an information element corresponding to source node 330) that constitute the decision-making process, as well as relational information (e.g., relational information represented by directed edges 350, 360, and 370, see Figure 3) that shows the support relationships, rebuttal relationships, and source relationships of the claim information, evidence information, rebuttal information, and source information, and the integrity index calculation unit calculates a plurality of logical paths leading to the claim information (e.g., the steps in Figure 7) based on the relational information of the decision-making structure data. The system extracts paths leading to the claim node 310 extracted in S702, obtains a local score (e.g., local score si) calculated by the local score calculation unit for each information element included in each logical path, calculates a score for each path (e.g., path score qk), distinguishes between supporting paths that support the claim information and counter-supporting paths that provide counter-evidence to the claim information based on the relational information (see, for example, step S704 in Figure 7), integrates the scores of the supporting paths and the counter-supporting paths based on predetermined rules (see, for example, steps S705 and S706 in Figure 7), and calculates a validity score for the decision-making process (e.g., validity score SV, see step S707 in Figure 7) based on the integrated value.

[0176] In the above configuration, for example, the integrity index calculation unit extracts multiple logical paths leading to the claim information based on relational information representing the support relationships, rebuttal relationships, and source relationships of the claim information, evidence information, rebuttal information, and source information. It obtains a local score for each information element included in each logical path, calculates a score for each path, and integrates the support paths and rebuttal paths after distinguishing between them. With this configuration, for example, the user can obtain a validity score that reflects the balance and logical structure of the support and rebuttal to the claim information. Furthermore, in the above configuration, for example, the information processing system aggregates local scores on a path-by-path basis and integrates the support paths and rebuttal paths according to predetermined rules, rather than simply counting items or applying simple weighting. With this configuration, for example, the user can achieve a sophisticated validity evaluation that simultaneously considers the quality and quantity of evidence and the quality and quantity of rebuttal.

[0177] (5) The above decision-making structure data includes claim information (e.g., an information element corresponding to claim node 310), evidence information (e.g., an information element corresponding to evidence node 320), rebuttal information (e.g., an information element corresponding to rebuttal node 340), and source information (e.g., an information element corresponding to source node 330) that constitute the above decision-making process, as well as relational information (e.g., relational information represented by directed edges 350, 360, and 370) that shows the support, rebuttal, and source relationships of the claim information, evidence information, rebuttal information, and source information, and the above relational information. The integrity index calculation unit includes an index (e.g., disclosure rate φ) that shows the disclosure status of metadata, including source information, creation date and time, and creator information, associated with each information element included in the decision-making structure data. ー ), an index that shows the explanation status of the semantic content or reasoning related to the supporting relationships, disproving relationships, and source relationships between the information elements included in the above decision-making structure data (for example, relationship explanation rate ψ ー ), an indicator showing the reproducibility of replication by a third party (e.g., recall rate ρ) ーA transparency index (e.g., transparency index T(d)) is calculated based on at least one of the following: the data structure and an index indicating the structural complexity of the decision-making structure data (e.g., density penalty index H(Gd)).

[0178] In the above configuration, for example, the integrity index calculation unit calculates a transparency index based on at least one of the following indicators: the level of disclosure of metadata, the level of explanation regarding supporting relationships, counter-refutation relationships, and source relationships, the reproducibility for replication by third parties, and the structural complexity. With such a configuration, for example, users can quantitatively evaluate the level of information disclosure, including not only the results of the decision-making process but also the state of preparation of data sources, explanatory documents, and reproduction procedures. Furthermore, in the above configuration, for example, the information processing system directly reflects insufficient disclosure of metadata or lack of explanation of related information in the transparency index. With such a configuration, for example, users can clearly understand which information elements or relationships have a concentration of explanation gaps, and rationally determine the priorities for document and log preparation. Also, in the above configuration, for example, the information processing system treats the structural complexity of decision-making structure data as one element of the transparency index. With such a configuration, for example, users can automatically detect decision-making processes with excessively branched path structures or bloated node structures, and promote simplification to a structure that is easy for users to understand.

[0179] (6) The integrity index calculation unit calculates an accountability score for the decision-making process (e.g., accountability score A(d)) based on at least one of the following: an index indicating the comprehensiveness of the explanation regarding the decision-making process (e.g., explanation satisfaction E(d)), an index indicating the reproducibility of the decision-making process for replication by a third party (e.g., success rate of replication trials R(d)), and an index indicating the composition of stakeholders and the path structure of the approval process in the decision-making process (e.g., approval path balance index D(d)).

[0180] In the above configuration, for example, the integrity index calculation unit calculates an accountability score based on at least one of the following indicators: the comprehensiveness of the explanation, the reproducibility of the results for third-party replication, and the composition of stakeholders and routing structure of the approval process. With this configuration, for example, users can comprehensively evaluate the decision-making process, including the extent to which matters that need to be explained are explained, whether the same conclusion is reached through re-execution based on the explanation, and the configuration in which the approval process is operated. Furthermore, in the above configuration, for example, the information processing system incorporates the composition of stakeholders and routing structure of the approval process into the accountability score. With this configuration, for example, users can automatically detect decision-making processes where approval is concentrated in specific departments or authority levels, or decision-making processes that lack verification by diverse stakeholders, thereby promoting the correction of approval flows with governance problems. In addition, in the above configuration, for example, the information processing system integrates the comprehensiveness and reproducibility of explanation items and the structure of the approval process into a single index. With this configuration, for example, users can quantitatively evaluate the effectiveness of accountability, which cannot be judged solely by the presence or absence of formal explanatory documents, thereby improving the reliability of audits and external explanations.

[0181] (7) The integrity index calculation unit uses predetermined thresholds (e.g., threshold τSV for the validity score, threshold τT for the transparency score, and threshold τA for the accountability score) for the decision-making process to determine the validity score (e.g., validity score SV(d)), the transparency index (e.g., transparency index T(d)) and the accountability score (e.g., accountability score A(d)) of the decision-making process, respectively, to extract information elements (e.g., each node included in the decision-making structure data 300) or logical paths containing information elements (e.g., multiple logical paths leading to claim information) that contribute to calculating index values ​​below the thresholds as integrity improvement candidates (e.g., integrity improvement candidates). The output unit then outputs improvement suggestion information (e.g., improvement suggestion statement) for the integrity improvement candidates extracted by the integrity index calculation unit.

[0182] In the above configuration, for example, the integrity index calculation unit sets thresholds for each validity score, transparency index, and accountability score, and extracts information elements or logical paths that contributed to the calculation of index values ​​below the threshold as candidates for integrity improvement. With this configuration, for example, users can automatically identify the causes of integrity declines that were difficult to grasp from the numerical value of the integrated integrity index alone. Furthermore, in the above configuration, for example, the information processing system outputs improvement suggestion information for the integrity improvement candidates. With this configuration, for example, even if users do not have specialized statistical or governance knowledge, they can specifically understand which information elements to modify and in what direction to improve integrity, thereby reducing the burden of integrity improvement work. In addition, in the above configuration, for example, the information processing system supports integrity improvement by combining threshold setting, improvement candidate extraction, and improvement suggestion information output. With this configuration, for example, users can not only check the evaluation results but also take concrete actions to improve integrity, and efficiently promote the continuous improvement of the decision-making process throughout the organization.

[0183] (8) The above information processing system includes an update unit (e.g., update unit 227, information processing device 110, see Figure 11) that performs update processing of the integrated integrity index of the decision-making process (e.g., integrated integrity index I(d) or normalized value I'(d), see Figure 10), and the integrated index calculation unit calculates a sensitivity coefficient (e.g., sensitivity coefficient Ci) for estimating the change in the validity score (e.g., validity score SV(d) or validity score SV), the transparency index (e.g., transparency index T(d) or transparency index T), and the accountability score (e.g., accountability score A(d) or accountability score A) of the decision-making process when the local score (e.g., local score si) of each information element included in the decision-making structure data (e.g., decision-making structure data 300) changes, based on the rate of change of each index with respect to the local score (e.g., ∂SV / ∂si, ∂T / ∂si, and ∂A / ∂si), and the update unit includes If, for some of the multiple information elements (e.g., multiple nodes included in the decision structure data 300), the multiple attribute values ​​of the information element (e.g., bias bi, consistency deviation ci, causal consistency gi, ethical deviation ui, freshness fi, and confidence ri) or the local score of the information element are updated, the local score of the updated information element is recalculated, and a differential update of the integrated integrity index of the decision process is performed based on the sensitivity coefficient of the information element (e.g., a differential update of the integrated integrity index I, see Figure 11), the error resulting from the differential update (e.g., error index E) is estimated, and if the error is less than or equal to a predetermined error upper limit (e.g., error upper limit ε), the result of the differential update is adopted, and if the error exceeds a predetermined threshold (e.g., recalculation threshold τ), the integrated integrity index is recalculated for the entire decision structure data (e.g., recalculation based on the processing flow shown in Figure 4), and the integrated integrity index is updated.

[0184] In the above configuration, for example, the integrated index calculation unit calculates sensitivity coefficients based on the rate of change of the validity score, transparency index, and accountability score for the local score of each information element, and the update unit performs differential updates of the integrated integrity index based on the local score and sensitivity coefficient of the updated information element. With this configuration, for example, users can update the integrated integrity index with high computational efficiency in response to partial data updates without having to recalculate the entire decision structure data. Furthermore, in the above configuration, for example, the information processing system estimates the error caused by the differential update, adopts the differential update result if the error is below the error limit, and recalculates the integrated integrity index for the entire decision structure data if the error exceeds the threshold. With this configuration, for example, users can autonomously optimize the balance between reducing computational load and ensuring evaluation accuracy. Also, in the above configuration, for example, the information processing system performs continuous differential update processing by the update unit. With this configuration, for example, users can maintain the integrated integrity index in near real-time even in operating environments where information is frequently added or modified, and realize risk management based on the latest integrity assessment.

[0185] (9) The above information processing system includes a parameter learning unit (e.g., parameter learning unit 228, information processing device 110, see Figure 12) that learns a set of model parameters (e.g., a set of parameters such as weight coefficients included in the parameter vector θ, decay rate, and temperature parameter T) that include the multiple weight coefficients and the weight coefficients used in the predetermined calculation by the local score calculation unit. The parameter learning unit sets the set of model parameters based on a learning dataset (e.g., a learning dataset obtained in step S1201 of Figure 12) that includes multiple audited decision cases (e.g., learning data di) and expert evaluation scores for the decision cases (e.g., teacher labels Li or target integrity values). For each audited decision case, it sets an error index (e.g., loss term in the objective function J(θ)) based on the difference between the integrated integrity index of the audited decision case calculated by the integrated index calculation unit (e.g., integrated integrity index I(di) or normalized value I′(di)) and the evaluation score of the audited decision case, and a regularization term (e.g., L2 regularization term λ∥θ∥) based on the magnitude of the set of parameters. 2 The parameter set is updated to minimize an objective function (for example, objective function J(θ), see step S1204 in Figure 12) that includes ), and the integrated index calculation unit calculates the integrated integrity index of the decision-making process using the parameter set of the model updated by the parameter learning unit.

[0186] In the above configuration, for example, the parameter learning unit updates the model's parameter set based on a learning dataset that includes multiple audited decision-making cases and expert evaluation scores, and the integrated index calculation unit calculates an integrated integrity index using the updated parameter set. With this configuration, for example, users can evaluate the integrity of their decision-making processes based on an integrity evaluation model that is consistent with expert judgment criteria. Furthermore, in the above configuration, for example, the information processing system updates the parameter set to minimize an objective function that includes an error index based on the difference between the integrated integrity index and the evaluation score, and a regularization term based on the size of the parameter set. With this configuration, for example, users can use a model in which parameters such as weight coefficients and decay rates are objectively optimized while suppressing overfitting, ensuring stable evaluation performance across different organizations and industries. In addition, in the above configuration, for example, the information processing system is configured to accumulate audited decision-making cases during operation, allowing the parameter learning unit to continuously retrain the model. With this configuration, for example, users can automatically update the integrity evaluation model in response to changes in the regulatory environment and risk tolerance, maintaining a high level of validity and adaptability of the evaluation in long-term operation.

[0187] (10) The above information processing system includes a generation unit (e.g., generation unit 221, information processing device 110, see Figures 2 and 4) that generates decision structure data for the decision-making process (e.g., decision structure data 300, see step S402 in Figures 3 and 4) based on data relating to the decision-making process, and the generation unit generates, as decision structure data for the decision-making process, claim information (e.g., information elements corresponding to claim nodes 310), justification information (e.g., information elements corresponding to justification nodes 320), and counter-evidence information (e.g., counter-evidence). A directed graph structure (for example, the directed graph structure shown in Figure 3) is generated, which includes multiple nodes (for example, claim node 310, evidence node 320, source node 330 and counter-evidence node 340) corresponding to information elements corresponding to evidence node 340 and source information (for example, information elements corresponding to source node 330), and edges (for example, directed edge 350, directed edge 360 ​​and directed edge 370) that connect the nodes and correspond to the support relationships, counter-evidence relationships, and source relationships of the claim information, evidence information, counter-evidence information, and source information.

[0188] In the above configuration, for example, the generation unit generates a directed graph structure based on data related to the decision-making process, including multiple nodes corresponding to claim information, evidence information, counter-evidence information, and source information, and edges corresponding to their support relationships, counter-evidence relationships, and source relationships. With such a configuration, for example, users can structure and understand unstructured data such as meeting minutes and reports as decision-making structure data suitable for integrity evaluation.

[0189] The decision-making process is interpreted as a concept that includes human decision-making, machine learning model decision-making, and collaborative decision-making between humans and machine learning models. The decision-making process is interpreted as a concept that includes not only the process of arriving at a single conclusion, but also the process of comparing and considering multiple alternatives and continuously updated operational processes. The decision-making process is interpreted as a concept that includes judgment processing in areas such as business decisions, operational processes, risk management, review, approval, recommendation, forecasting, and control.

[0190] Decision structure data is interpreted as a concept that includes data structures representing information related to the decision-making process in one or more forms, such as graph, table, tree, rule set, or time-series log format. Decision structure data may be implemented not only as a graph structure representing claim information, evidence information, counter-evidence information, and source information as nodes, but also as a data structure that includes the correspondence between inputs and outputs in the decision-making process, features, and the internal state of the model. Decision structure data is interpreted as a concept that includes data structures held in memory, data structures stored in persistent storage devices, and data structures retrieved from external services via a network.

[0191] Information elements are interpreted as concepts that include nodes, edges, attribute values, metadata, log records, and intermediate representations of models contained in decision-making structure data. Information elements are interpreted as concepts that include text information, numerical information, categorical information, image information, audio information, time-series information, and combinations thereof.

[0192] An attribute value is interpreted as a concept that includes one or more quantitative or qualitative evaluation values ​​for an information element. An attribute value can be interpreted as a single real number, as well as a concept that includes vectors, matrices, probability distributions, intervals, label sequences, and structures combining multiple types of values. Attribute values ​​may employ configurations that calculate them by means of arithmetic operations, logical operations, statistical estimation, machine learning model estimation, rule-based reasoning, and manual evaluation input.

[0193] Bias is interpreted as an indicator of how far information elements deviate from the distribution. Bias can be interpreted as an indicator that includes simple differences, standardized scores, stochastic outlier indices, information-theoretic quantities, and anomaly scores from machine learning models. Bias can be interpreted as a concept that includes not only bias assessments in a single dimension, but also complex bias assessments in a multidimensional feature space.

[0194] The consistency deviation is interpreted as an indicator of the degree of inconsistency between the content of one information element and other information elements. The consistency deviation may be calculated not only based on semantic similarity, but also as an evaluation of inconsistency that includes logical consistency, statistical consistency, and consistency with domain rules. The consistency deviation is interpreted as a concept that includes not only evaluations based on comparisons of pairs of information elements, but also consistency evaluations for sets of multiple information elements.

[0195] Causal consistency is interpreted as an index indicating the degree of consistency of causal relationships between information elements based on a causal relationship model. Causal relationship models are interpreted as concepts including structural equation models, graphical models, causal Bayesian networks, time-delay models, and rule-based causal models. Causal consistency may employ a configuration that calculates it as an overall causal validity index, including causal direction validity, causal strength, and behavior during intervention.

[0196] Ethical deviation is interpreted as an index indicating the degree of ethical deviation of an information element based on the risk of ethical violation. Ethical deviation is interpreted as a concept that includes assessments of legal compliance, compliance with internal regulations, human rights violation risk, discrimination risk, privacy violation risk, and social acceptability. Ethical deviation may employ a configuration that combines rule-based judgment, scoring models, natural language processing models, and human evaluation.

[0197] Freshness is interpreted as an indicator of the newness of information over time. Freshness may be calculated using a framework that includes not only the creation date and time or collection date and time, but also the frequency of updates, the occurrence of related events, and the degree of change in the external environment. Freshness may be modeled using a monotonically decreasing function, or it may be modeled using a non-monotonic time function that increases or decreases in response to events.

[0198] Trustworthiness is interpreted as an indicator of the degree of reliability of an information element based on the source, creator, verification status by third parties, and internal verification results by the system. Trustworthiness may employ a structure that includes indicators such as performance-based credibility, tamper detection results, signature and certificate verification results, and evaluations by external rating agencies. Trustworthiness may also employ a structure that expresses it as a numerical score, rank, classification label, or a combination thereof.

[0199] Local scores are interpreted as evaluation values ​​that integrate multiple attribute values ​​for an information element. Local scores are interpreted as a concept encompassing the results of linear combinations, nonlinear combinations, multi-objective optimization, fuzzy inference, and machine learning models. Local scores are not limited to simple scalar values; they may be represented as probability distributions, interval estimates, and scenario-specific score sets.

[0200] The validity score, transparency index, and accountability score are interpreted as macro-level metrics that assess the nature of the decision-making process based on local scores and decision-making structure data. These metrics may be expressed as a single value or as vector metrics with values ​​for multiple perspectives. These metrics may be calculated using simple arithmetic mean methods, as well as methods involving weighted mean, median, quantile, robust statistics, and aggregated results from trained models.

[0201] The Integrated Integrity Index is interpreted as a composite index that integrates the validity score, transparency index, and accountability score. The Integrated Integrity Index may be expressed as a single real number or as a vector value with multiple dimensions. The Integrated Integrity Index may be used in a format that includes a standardized score, rank, evaluation category, and threshold determination result.

[0202] A logical path is interpreted as a path connecting claim information, evidence information, counter-evidence information, and source information. Logical paths may be defined not only as simple paths on a directed graph, but also as paths involving node revisits, subgraph patterns, and sets of paths. A logical path is interpreted as a concept encompassing inference chains, evidence chains, and verification procedures used to describe a decision-making process.

[0203] A support path is interpreted as a logical path consisting of informational elements that support the asserted information. A support path is interpreted as a concept that includes not only paths connecting supporting information that directly supports the asserted information, but also paths containing meta-evidence that reinforces the reliability and validity of the supporting information. A rebuttal path is interpreted as a logical path consisting of informational elements that provide rebuttal to the asserted information. A rebuttal path may be expressed as a path containing supporting information that contradicts the asserted information, information that negates the assumptions, and risk information.

[0204] Metadata is interpreted as information including source information, creation date and time, author information, edit history, review history, access permission information, and external evaluation results. Metadata is interpreted as a concept that includes not only attributes associated with a single information element, but also attributes associated with multiple information elements or an entire logical path.

[0205] An indicator of the comprehensiveness of the explanation is interpreted as an indicator of the extent to which the collection of explanatory documents, comments, annotations, and document links covers each component of the decision-making process. The indicator of comprehensiveness of the explanation may employ a structure that calculates based on an evaluation that includes the completeness of the checklist, the presence or absence of required items, the extent to which recommended items are included, and the consistency between explanatory documents.

[0206] Reproducibility metrics are interpreted as indicators of whether a third party can re-execute the decision-making process based on recorded information and explanations and arrive at the same or an acceptable conclusion. Reproducibility metrics may employ a structure that calculates based on an assessment that includes the retention of data used, the recording of parameters and settings, the clarity of the execution procedure, and the degree of environmental dependence.

[0207] Metrics indicating structural complexity are interpreted as indicators calculated based on structural information, including the number of nodes, edges, branches, presence or absence of circular structures, and hierarchical depth in decision-making structure data. These metrics may employ configurations that utilize not only simple size indicators but also complexity indicators related to ease of understanding, ease of visualization, and ease of maintenance.

[0208] Indicators representing the stakeholders and routing structure of the approval process are interpreted as indicators that evaluate the composition of roles, positions, organizational units, and external stakeholders involved in the decision-making process, and the structure representing the approval flow among them. Indicators representing the stakeholders and routing structure of the approval process may employ a structure calculated based on an evaluation that includes the number of approval levels, the diversity of stakeholders, the degree of separation of duties, and the presence or absence of checks and balances.

[0209] The sensitivity coefficient is interpreted as an index representing the degree to which changes in the local score of each information element affect changes in the validity score, transparency index, and accountability score. The sensitivity coefficient may employ a configuration that calculates it based on analytical differentiation, numerical differentiation, simulation, sensitivity analysis algorithms, and importance estimation using machine learning models. The sensitivity coefficient may be defined not only as a single scalar value, but also as a higher-order sensitivity index that includes asymmetric sensitivity to changes in the up and down directions, local sensitivity near the threshold, and interactions between multiple indices.

[0210] Audited decision cases are interpreted as a concept that includes decision-making processes evaluated through one or more audit processes, such as internal audits, external audits, third-party verifications, and expert reviews. Audited decision cases are interpreted as a concept that includes not only historical cases, but also decision-making processes based on scenarios generated for verification, decision-making processes executed in simulation environments, and decision-making processes based on benchmark datasets.

[0211] The evaluation score is interpreted as an evaluation value assigned by experts to audited decision-making cases. The evaluation score is interpreted not only as a single overall score, but also as a concept encompassing evaluation values ​​across multiple axes, such as appropriateness, transparency, accountability, ethics, and risk level. The evaluation score may adopt a structure that expresses qualitative evaluations as numerical values, ranks, category labels, or combinations thereof.

[0212] The model's parameter set is interpreted as a concept that includes weight coefficients, bias terms, regularization coefficients, learning rates, thresholds, and other adjustable settings. The model's parameter set is interpreted as a broader concept that includes not only the weight coefficients in the integrated exponential calculation unit, but also the integration rules for attribute values ​​in the local score calculation unit, the decay function based on freshness, the formula for calculating sensitivity coefficients, and the threshold value.

[0213] The generation unit is interpreted as having a function to generate decision structure data from data related to the decision-making process. The generation unit may employ a configuration that uses means such as natural language processing, information extraction, syntactic analysis, semantic analysis, machine learning, rule-based transformation, and manual input assistance, either individually or in combination. The generation unit may employ a configuration that updates the decision structure data in real time as an online process, or a configuration that generates decision structure data in batches from past logs as a batch process.

[0214] A directed graph structure is a graph composed of nodes and edges, and is interpreted as a structure in which the edges are assigned a direction. A directed graph structure may employ a configuration that includes not only nodes corresponding to claim information, evidence information, counter-evidence information, and source information, but also auxiliary metadata nodes, process nodes, and external system integration nodes. A directed graph structure is interpreted as a concept that includes tree structures, DAG structures, structures including cycles, and multi-graph structures.

[0215] In this embodiment, a configuration that uses multiple types of attribute values, evaluation indicators, score calculation processes, and evaluation procedures in combination has been described as an example. However, a configuration that calculates, references, or executes only at least one of these may also be used. For example, attribute values ​​such as bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability, validity scores, transparency indicators, accountability scores, integrated integrity indexes, and other integrity-related indicators, types of explanation items, evaluation procedures for reproduction trials, and evaluation indicators related to approval routes may be used individually or in any combination. This is interpreted to include configurations that adopt only some of these or adopt all of them, depending on the purpose of use and the characteristics of the target business. [Explanation of symbols]

[0216] 100... Information processing system, 110... Information processing device, 120... Client terminal, 130... External system.

Claims

1. An information processing system for evaluating the integrity of a decision-making process, A generation unit that generates decision-making structure data related to the decision-making process based on data related to the decision-making process, An attribute calculation unit performs a process to calculate different attribute values ​​for each of a plurality of information elements, each of which includes an information element corresponding to each node in the aforementioned decision structure data, based on at least two of the following: statistical properties, semantic similarity, causal relationship model, risk of ethical violation, time elapsed, and reliability regarding the source, creator, or third-party verification status, and calculates the results of the attribute calculation process as a plurality of attribute values ​​for the information element. A local score calculation unit that integrates the multiple attribute values ​​for each of the aforementioned multiple information elements using a predetermined calculation and calculates a local score corresponding to the information element, An integrity index calculation unit calculates a validity score indicating the validity of the decision-making process, a transparency index indicating the transparency of the decision-making process, and an accountability score indicating the accountability of the decision-making process, based on the decision-making structure data and the local score calculated by the local score calculation unit. An integrated index calculation unit performs an integrated processing on the validity score of the decision-making process, the transparency index of the decision-making process, and the accountability score of the decision-making process based on a plurality of pre-set weighting coefficients, and calculates an integrated integrity index that indicates the integrity of the decision-making process. An output unit that outputs the integrated integrity index of the aforementioned decision-making process, Equipped with, The generation unit generates a directed graph structure as decision-making structure data, which includes a plurality of nodes corresponding to claim information, evidence information, counter-evidence information, and source information constituting the decision-making process, and a plurality of edges connecting the plurality of nodes and indicating support relationships, counter-evidence relationships, and source relationships between the claim information, evidence information, counter-evidence information, and source information. The integrity index calculation unit is: Based on the support and refutation relationships indicated by multiple edges of the directed graph structure, support and refutation paths leading to the node corresponding to the assertion information are identified; local scores corresponding to nodes included in the support paths are aggregated on a path-by-path basis as positive contributions; local scores corresponding to nodes included in the refutation paths are aggregated on a path-by-path basis as negative contributions; and the validity score is calculated by integrating the results of these aggregations. The transparency index is calculated by aggregating metadata associated with nodes or edges of the directed graph structure, relational information indicated by the multiple edges, information regarding reproducibility for replication by a third party, and information regarding the structural complexity of the directed graph structure, and integrating the aggregated results of the metadata, relational information, and reproducibility information as positive contributions and the aggregated result of the structural complexity information as a negative contribution. The accountability score is calculated by aggregating the explanation logs, reproduction trial logs, and approval process information related to the decision-making process, and integrating the aggregated results of the explanation logs, reproduction trial logs, and approval process information as a positive contribution to accountability. Information processing system.

2. The information processing system according to claim 1, The attribute calculation unit, for each of the plurality of information elements, (i) A process to calculate the degree of bias, which indicates the degree to which the information element deviates from the distribution, based on its statistical properties. (ii) A process to calculate a consistency deviation that indicates the degree of inconsistency between the content of the information element and other information elements, based on semantic similarity. (iii) A process to calculate the degree of causal consistency, which indicates the degree of consistency of the causal relationship between the information element in question and other information elements, based on a causal relationship model. (iv) A process to calculate the degree of ethical deviation, which indicates the magnitude of the ethical deviation of the information element, based on the risk of ethical violation. (v) A process for calculating freshness, which indicates the newness of information as time passes, and (vi) A process for calculating a confidence level indicating the degree of reliability of an information element, based on the reliability of the source, creator, and verification status by third parties. Perform at least two of the above, and calculate at least two of the above bias, consistency deviation, causal consistency, ethical deviation, freshness, and reliability as the multiple attribute values. Information processing system.

3. The information processing system according to claim 2, The attribute calculation unit calculates, for each of the plurality of information elements, the degree of bias of the information element, the consistency deviation of the information element, the causal consistency of the information element, the degree of ethical deviation of the information element, the freshness of the information element, and the reliability of the information element as the plurality of attribute values. The local score calculation unit treats each of the plurality of information elements as having a positive contribution from its reliability and causal consistency, and as having a negative contribution from its bias, consistency deviation, and ethical deviation. It then integrates the attribute values ​​of reliability, causal consistency, bias, consistency deviation, and ethical deviation according to predetermined rules, and calculates the local score of the information element based on a value obtained by weighting the integrated value according to its freshness. Information processing system.

4. The information processing system according to claim 1, The integrity index calculation unit is: The disclosure rate is calculated based on the disclosure status of metadata associated with the nodes or edges of the aforementioned directed graph structure. Based on the explanation of the meaning or reasoning related to the support, counter-support, and source relationships indicated by the aforementioned multiple edges, the relationship explanation rate is calculated. The recall rate is calculated based on the reproducibility of the results in replication by a third party. A density penalty index is calculated based on the structural complexity of the directed graph structure. The transparency index is calculated by integrating the disclosure rate, the relationship explanation rate, and the recall rate as positive contributions, and the density penalty index as a negative contribution. Information processing system.

5. The information processing system according to claim 1, The integrity index calculation unit is: Based on the comprehensiveness of the explanation regarding the aforementioned decision-making process, the degree of explanation sufficiency is calculated. Based on the reproducibility of the aforementioned decision-making process for replication by a third party, the success rate of the replication trial is calculated. Based on the composition of stakeholders and the path structure of the approval process in the aforementioned decision-making process, an approval path balance index is calculated. The accountability score is calculated by integrating the explanation sufficiency, the success rate of the reproduction trial, and the approval pathway balance index as positive contributions. Information processing system.

6. The information processing system according to claim 1, The integrity index calculation unit uses predetermined thresholds for the validity score of the decision-making process, the transparency index of the decision-making process, and the accountability score of the decision-making process, and extracts information elements or logical paths containing such information elements that contribute to the calculation of index values ​​below the thresholds as candidates for integrity improvement. The output unit outputs improvement suggestion information for the integrity improvement candidates extracted by the integrity index calculation unit. Information processing system.

7. The information processing system according to claim 1, The update unit includes an update unit that performs update processing of the integrated integrity index of the aforementioned decision-making process, The integrated index calculation unit calculates sensitivity coefficients for estimating the changes in the validity score of the decision-making process, the transparency index of the decision-making process, and the accountability score of the decision-making process when the local score of each of the multiple information elements changes, based on the rate of change of each index with respect to the local score. The aforementioned update unit is, If any of the multiple information elements included in the aforementioned decision-making structure data have their multiple attribute values ​​or local scores updated, the local scores of the updated information elements are recalculated, and the integrated integrity index of the decision-making process is updated differentially based on the sensitivity coefficient of the information element. The error resulting from the differential update is estimated, and if the error is less than or equal to a predetermined error limit, the result of the differential update is adopted. If the error exceeds a predetermined threshold, the integrated integrity index is recalculated for the entire decision-making structure data, and the integrated integrity index is updated. Information processing system.

8. The information processing system according to claim 1, The system includes a parameter learning unit that learns a set of model parameters, including the plurality of weight coefficients and the weight coefficients used in the predetermined calculation, by the local score calculation unit. The parameter learning unit, Using multiple audited decision-making cases as input data, a training dataset is obtained that includes the evaluation score assigned by experts to each audited decision-making case as training labels. For each of the aforementioned multiple audited decision cases, the parameter set of the model is updated to minimize an objective function that includes an integrated integrity index calculated from the audited decision case by the integrated index calculation unit, an error index based on the difference between the audited decision case and the teacher label corresponding to the audited decision case, and a regularization term based on the magnitude of the parameter set. The integrated index calculation unit calculates the integrated integrity index of the decision-making process using the set of model parameters updated by the parameter learning unit. Information processing system.

9. An information processing method performed by an information processing system that evaluates the integrity of a decision-making process, The generation unit of the information processing system generates decision-making structure data related to the decision-making process based on data related to the decision-making process, The attribute calculation unit of the information processing system performs a different attribute value calculation process for each of the multiple information elements, including information elements corresponding to each node in the decision-making structure data, based on at least two of the following: statistical properties, semantic similarity, causal relationship model, risk of ethical violation, time elapsed, and reliability regarding the source, creator, or third-party verification status, and calculates the results of the attribute value calculation process as multiple attribute values ​​for the information element. The local score calculation unit of the information processing system integrates the multiple attribute values ​​for each of the multiple information elements using a predetermined calculation and calculates a local score corresponding to that information element. The integrity index calculation unit of the information processing system calculates a validity score indicating the validity of the decision-making process, a transparency index indicating the transparency of the decision-making process, and an accountability score indicating the accountability of the decision-making process, based on the decision-making structure data and the local score calculated by the local score calculation unit. The integrated index calculation unit of the information processing system performs integrated processing on the validity score of the decision-making process, the transparency index of the decision-making process, and the accountability score of the decision-making process based on a plurality of pre-set weighting coefficients, and calculates an integrated integrity index that indicates the integrity of the decision-making process. The output unit of the information processing system outputs the integrated integrity index of the decision-making process, Includes, The generation unit generates a directed graph structure as decision-making structure data, which includes a plurality of nodes corresponding to claim information, evidence information, counter-evidence information, and source information constituting the decision-making process, and a plurality of edges connecting the plurality of nodes and indicating support relationships, counter-evidence relationships, and source relationships between the claim information, evidence information, counter-evidence information, and source information. The integrity index calculation unit is: Based on the support and refutation relationships indicated by multiple edges of the directed graph structure, support and refutation paths leading to the node corresponding to the assertion information are identified; local scores corresponding to nodes included in the support paths are aggregated on a path-by-path basis as positive contributions; local scores corresponding to nodes included in the refutation paths are aggregated on a path-by-path basis as negative contributions; and the validity score is calculated by integrating the results of these aggregations. The transparency index is calculated by aggregating metadata associated with nodes or edges of the directed graph structure, relational information indicated by the multiple edges, information regarding reproducibility for replication by a third party, and information regarding the structural complexity of the directed graph structure, and integrating the aggregated results of the metadata, relational information, and reproducibility information as positive contributions and the aggregated result of the structural complexity information as a negative contribution. The accountability score is calculated by aggregating the explanation logs, reproduction trial logs, and approval process information related to the decision-making process, and integrating the aggregated results of the explanation logs, reproduction trial logs, and approval process information as a positive contribution to accountability. Information processing methods.

Images