Learning system, learning method, and non-transitory computer-readable

Abstract

To provide a learning system, a learning method and a learning program for more correctly presenting new trust to be output on the basis of a new real world or for correctly adding trust to data to be output on the basis of the new real world according to the degree of trust obtained by evaluating the real world.SOLUTION: Acquiring information related to a real world and trust information evaluated for the real world, generating a learning model in which the information related to the real world is input and the trust information is output, re-learning the learning model based on new information related to the real world and the trust information output for the new real world, and weighting the trust information when re-learning the learning model.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to a learning system, a learning method, and a learning program. [Background technology]

[0002] In recent years, various machine learning models, including generative AI (Artificial Intelligence), have been developed.

[0003] For example, Patent Document 1 describes a learning device that, when a model trained using normal data is retrained using attention data so that the user specifically requests the model to output the correct answer, can accurately give the correct answer to the attention data while maintaining generalizability by increasing the weight of the attention data and retraining.

[0004] Patent Document 2 describes an anomaly detection and diagnosis method that detects abnormalities or signs of abnormalities in a plant or equipment from sensor data and operation data, links the abnormalities or signs of abnormalities to past countermeasures using information on the maintenance history for similar past abnormalities, suggests countermeasures based on the results of the linking, and adjusts the sensitivity of the anomaly detection based on the accuracy of the countermeasures. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-99702 [Patent Document 2] Patent No. 5808605 Summary of the Invention [Problem to be solved by the invention]

[0006] In Society 5.0, which may be realized in the future, it will be necessary to handle real-world data that reflects the physical real world in virtual space (cyberspace) through machine learning and other methods, and provide feedback to improve the real world, thereby achieving both the resolution of important social issues such as sustainability and economic development.

[0007] However, if real-world data such as satellite data or IoT data is incorrect, falsified, missing, or unreliable, and therefore does not accurately reflect the real world, the data generated by generative AI and the results of machine learning will be incomplete or unstable, and services provided using these sources could potentially cause significant damage to the real world.

[0008] For example, if there are errors, missing data, or large discrepancies in the location data obtained from a positioning sensor due to the effects of solar flares or spoofing, when updating the system through re-learning, data containing incorrect location data may be treated as correct data, which could cause services provided by cyber-physical systems (such as autonomous driving control systems) to malfunction.Furthermore, in the modern era of generative AI, automatic prompt generation AI, automatic image and document generation AI, and automatic check AI for contracts, etc. are connected via APIs (application programming interfaces), making it unclear whether human checks were involved.

[0009] The present invention aims to provide a learning system, a learning method, and a learning program that allows new trust output based on a new real world to be presented more accurately depending on the degree of trust evaluated in the real world, or that allows trust to be correctly added to data output based on the new real world. [Means for solving the problem]

[0010] The learning system of the present invention comprises one or more processors, which acquire information about the real world and information about trust evaluated for that real world, generate a learning model that inputs information about the real world and outputs the trust information, re-train the learning model based on information about the new real world and the trust information output for the new real world, and weight the trust information when re-training the learning model.

[0011] The learning method of the present invention is a learning method executed on one or more processors, and includes the steps of acquiring information about the real world and information about trust evaluated for the real world, generating a learning model that inputs information about the real world and outputs the trust information, and re-learning the learning model based on information about the new real world and the trust information output for the new real world, and is characterized in that the trust information is weighted during the re-learning step of the learning model.

[0012] The learning program of the present invention is a learning program that is implemented by one or more processors, and has the following functions: acquiring information about the real world and information about the trust evaluated for that real world; generating a learning model that inputs information about the real world and outputs the trust information; re-learning the learning model based on information about the new real world and the trust information output for the new real world; and weighting the trust information in the re-learning step of the learning model. [Effects of the Invention]

[0013] According to the present invention, new trust output based on the new real world can be presented more accurately depending on the trust that reflects the real world, or trust can be correctly added to data output based on the new real world. [Brief explanation of the drawings]

[0014] [Figure 1A] FIG. 1A is a diagram showing the configuration of a learning system according to this embodiment. [Figure 1B] FIG. 1B is a diagram showing an example of a system configuration including a plurality of management servers 10A, 10B, . . . that perform distributed learning. [Figure 1C] FIG. 1C is a diagram showing an example of a system configuration including a plurality of management servers 10A, 10B, . . . that perform distributed learning and an overall management server 10Z that performs federated learning. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer used as a management server and a user terminal. [Figure 3] FIG. 3 is a diagram showing the functional configuration of the management server according to the present embodiment. [Figure 4] FIG. 4 is a diagram showing the functional configuration of a user terminal according to this embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of processing when responding to the real world according to this embodiment. [Figure 6] FIG. 6 is a flowchart showing the flow of generating a learning model according to this embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a deep learning model. [Figure 8A] FIG. 8A is a diagram showing an example of a screen displaying trust information for real-world data ("RW data" in the diagram). [Figure 8B] FIG. 8B is a diagram showing an example of a screen that presents countermeasures and trust information for real-world data ("RW data" in the diagram). [Figure 9A] FIG. 9A is a diagram showing an example of evaluation of weighting according to this embodiment. [Figure 9B] FIG. 9B is a diagram showing an example of evaluation of weighting according to this embodiment. [Figure 10]FIG. 10 is a diagram illustrating a learning model that outputs trust based on input of information about the real world. [Figure 11] FIG. 11 is a diagram illustrating an example of a learning period of a learning model. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings. Note that in the following description, a learning model that inputs information about the real world and outputs only trust information will be described, but the learning model may output not only trust information but also judgment result data such as simulation results for the real world, or data generated from the learning model as a generative AI. Furthermore, the learning model is not limited to outputting trust information externally, and may, for example, internally output trust information to be taken into account when performing a simulation for the real world in cyberspace.

[0016] <Learning model> First, a learning model according to this embodiment will be described. The learning model according to this embodiment receives information about the real world as input and outputs (internal output and / or external output) information about trust.

[0017] 10 is a diagram illustrating a learning model that outputs trust in response to input of information about the real world. In the example shown in FIG. 10, the learning model learns by linking information about real-world state A=(Xa, Yb, Zc, ) with trust 1, trust 2, and trust 3; learns by linking information about real-world state B=(Xb, Yb, Zb, ) with trust 1 and trust 4; and learns by linking information about real-world state C=(Xc, Yc, Zc, ) with trust 1, trust 3, and trust 6. By linking and learning information about the real world with evaluated trust in this way, a learning model that outputs trust in response to input of information about the real world is generated.

[0018] When operating a learning model, it may be necessary to periodically reassess and update the learning model to reflect the latest real-world and more accurate trust information. In the example shown in Figure 10, information about a newly occurring real-world state D = (Xd, Yd, Zd,...) is input, and trust 1 and trust 3 are output and presented by the learning model in response. A user of an application dealing with real-world D reevaluates trust 1 and 3 presented by the learning model, and further evaluates it as trust 7 in the example shown in Figure 10. Here, trust 7 is not presented by the learning model and is evaluated based on the user's judgment.

[0019] <Example of trust for individual inputs> Let's take this situation as an example of trust for real-world positions. For example, the position of a moving object in the real world is read as D = (Xd, Yd, Zd, ) by positioning sensors X (e.g., GPS (Global Positioning System) receiver), Y (e.g., Quasi-Zenith Satellite System Michibiki receiver), Z (e.g., MADOCA (Multi-GNSS Advanced Demonstration tool for Orbit and Clock Analysis) receiver). Based on the previously learned state positioning sensors X (e.g., GPS receiver), Y (e.g., Quasi-Zenith Satellite System Michibiki receiver), and Z (e.g., MADOCA-compatible receiver) and the learning results of their corresponding trust information, the learning model presents trust 1 for the value of positioning sensor X (e.g., GPS receiver) and presents trust 3 for Y (e.g., Quasi-Zenith Satellite System Michibiki receiver). In contrast, a user of an application dealing with real world D (e.g., a mobile operator) may be presented with location trusts of 1 and 3 as described above by the learning model, but may reassess the location at trust 7 on their own judgment, taking into account factors such as the absence of influence from solar flares or jamming.

[0020] <Example of trust for the total input> As another example, trust may be not only for individual inputs but also for the overall trust of multiple inputs. For example, in the accounting of a real-world transaction, the status of the transaction D = (Xd, Yd, Zd, . . .) is determined from product movement data X (e.g., product tracking data obtained from a location tag sensor), transactor data Y (e.g., the transactor's credit information and company and job rank data obtained from a human resources information system or credit information system), and accounting data Z (e.g., data entered into accounting software). Based on the previously learned values ​​of X, Y, and Z and their corresponding trust information, the learning model presents a trust of 1 for the value of X (the tracking data of the transaction target product) and a trust of 3 for Y (e.g., the transactor's credit information). In contrast, a user of an application dealing with the real world D (e.g., a certified public accountant conducting an audit) may be presented with trusts of 1 and 3 for the values ​​of X and Y by the learning model, but may then, based on their own judgment and taking into account factors such as an on-site inspection, reassess the value to trust 7 as a comprehensive assessment of the combination of inputs.

[0021] <Example of trust for a different output than the trust> As another example, trust is not limited to the input of a learning model, but may also be trust in the output of various models for real-world situations. Taking the example of autonomous driving control for a vehicle, when situation D occurs in the real world around the vehicle, situation D = (Xd, Yd, Zd, ). The vehicle's various sensors X, Y, Z, . . . read situation D = (Xd, Yd, Zd, . . .). The autonomous driving control model (which may be a learning model) can output various countermeasures for situation D. Based on the previously learned states A to C and the learning results of the corresponding trust information, the learning model may present, for example, trust 1 for the operation of turning the steering wheel to the right (countermeasure 1) and trust 3 for the operation of stepping on the brakes (countermeasure 2). The presentation of trust may be presented not only via a display screen or voice, but also in cases where physical assistance is presented, varying in strength, when physically assisting the steering in response to the operation of turning the steering wheel to the right or when physically assisting the operation of stepping on the brakes. That is, it includes trust output for adding trust to physical assist functions, voice assist functions, etc. Therefore, in this example, presenting trust 1 for the operation of turning the steering wheel to the right (measure 1) corresponds to steering assist to turn the steering wheel to the right with strength 1 corresponding to trust 1. Also, presenting the operation of stepping on the brakes (measure 2) with trust 3 corresponds to assisting the brake pedal operation with strength 3 corresponding to trust 3. In this case, it can be assumed that the user of the application dealing with real world D (in this example, the driver of an autonomous vehicle) is presented with measures 1 and 2 and the corresponding assists of trust 1 and 3 as described above by the learning model, but at his or her own discretion, responds (evaluates) the operation of turning the steering wheel to the left (measure 3) with trust 7 corresponding to strength 7.

[0022] <Examples of trust that the information is not generated by a generating AI, or that real human judgment was involved in the output of the generating AI> The following are examples of trust methods for assessing the trustworthiness of human intervention in generating AI: 1. Does the system as a whole include a human decision-making process? 2. The likelihood that the checker of the generated AI's output is a human (e.g., the number of increments used to correct the prompt, the time it took to correct it, the interval between corrections, changes in location information linked to the account (human presence), etc.) The above-mentioned cases and the like can be considered as examples. Here, the learning model is updated by relearning, linking newly generated information about the real world with information about the trust evaluated for the real world. In the example shown in Fig. 10, the learning model is updated by linking and learning information about state D in the real world with the trust evaluated by the user for state D in the real world.

[0023] Regarding the update of the learning model, it is possible to configure it so that old learning data is not reflected by defining a learning period for the learning model. The learning period of the learning model will be explained using Fig. 11. Fig. 11 is a diagram showing an example of the learning period for the learning model.

[0024] In the example shown in Figure 11, the learning model is updated every month, and the learning model used is one that has learned data from the most recent year. More specifically, the learning model used in January is a learning model generated by linking and learning the real world events that occurred over the year from January of the previous year to the most recent December with the evaluation content evaluated for each real world. If the learning model is updated and operated every month, the learning model used in April will include the results of user responses based on the trust information presented by the learning model used from January to March.

[0025] Here, when the real world improves by considering trusts other than those presented by the learning model, as in the case of real world D in the example shown in Figure 10, it is difficult to identify which trusts were truly important among the considered trusts. For this reason, when updating the learning model, all of the considered trusts are often treated as correct data. This can lead to incorrect noise caused by treating incorrect trusts as correct data, or bias caused by treating unimportant trusts as correct data without prioritizing them, which can result in the incorrect noise and bias continuing to amplify with each update of the learning model.

[0026] The amplification of incorrect noise and bias will be explained using the example of creating a learning model by learning data from the most recent year, as shown in Figure 11, and updating the model every month. For example, suppose that real-world state D shown in Figure 10 occurs in the learning model used in January. In this case, the learning model used in February will have learned trust 1, trust 3, and trust 7 as the correct answer data for real-world state D.

[0027] Here, if Trust 1 and Trust 3 make no contribution at all to improving real-world state D or if their contribution is relatively small, the learning model used in February will be one that has learned incorrect or unimportant trusts, Trust 1 and Trust 3, linked to real-world state D, and may contain incorrect noise and bias. Furthermore, the learning model used in March will be a model that has learned using the incorrect noise and bias contained in the models used in January and February as correct data, and the learning model used in April will be a model that has learned using the incorrect noise and bias contained in the models used from January to March as correct data.

[0028] In this way, if the learning model contains incorrect noise or bias, treating the incorrect noise or bias as correct data will amplify or maintain the incorrect noise or bias each time the learning model is updated, increasing the risk that it will be reflected in the model for the next month. Therefore, in this embodiment, weighting is applied to the correct data when updating the learning model.

[0029] <System configuration> 1A is a diagram showing the configuration of a system 1 according to this embodiment. The system 1 according to this embodiment includes a management server 10 and a user terminal 20. The management server 10 and the user terminal 20 are connected via a network 30.

[0030] The management server 10 is a server that manages information about the real world and history information about the trust evaluated for the real world. The information about the real world is information about the real world (also referred to as "real-world data"), and the trust information is information indicating what kind of trust has been evaluated for the real world. For example, when the situation in the real world is sensed by an IoT (Internet of Things) or a positioning sensor, information about the sensed situation in the real world is information about the real world, and information about the reliability of dealing with the real world or the reliability of the sensed situation in the real world is trust information. Note that reliability may be a concept that includes at least one of usefulness, confidentiality, and authenticity. Furthermore, the trust information may be information that includes at least one of reliability, usefulness, confidentiality, and authenticity.

[0031] Furthermore, the management server 10 learns by linking information about the real world with information about the trust evaluated for that real world, and generates a learning model that inputs information about the real world and outputs trust information. Then, when the generated learning model is re-learned based on information about a new real world and the trust information output for the new real world, weighting is performed on the trust information. The management server 10 is realized, for example, by a computer. The management server 10 may be configured by a single computer, or may be realized by distributed processing using multiple computers.

[0032] As a more specific example of the latter, if the management server 10 is implemented by distributed computing implemented by multiple computers across multiple regions, the learning model may be a learning model distributed across each region. Note that distributed learning is not limited to geographical or regional learning, and may also be individual-based learning or optimization, company-based learning or optimization, or industry-based learning or optimization.

[0033] FIG. 1B illustrates an example system configuration including multiple management servers 10A, 10B, and so on that perform distributed learning. In the example of FIG. 1B, each management server 10A, 10B collects real-world data and trust information from multiple user terminals 20 (IoT terminals 20 in the case of individuals) associated by region, individual, or company, and generates geographically, personally, or corporately distributed learning models ("distributed learning models" in the figure). As each distributed learning model (re)learns based on the real-world data and trust information from the associated terminals 20, it is expected that optimization will progress geographically, personally, or corporately, but other regional, individual, or corporate factors will not be learned. Therefore, by learning data from nearby regions, individuals with similar personas, or competitors, a more appropriate learning model can be developed.

[0034] To achieve this, as shown by L1 in FIG. 1B, for example, management server 10A may transmit the real-world data and trust information from the associated terminals 20 to management server 10B as raw data for re-learning, or may transmit pseudo-data or derived data generated from a learning model (e.g., including a knowledge graph) that has learned the real-world data and trust information for re-learning, or may transmit a learning model (e.g., a weighting coefficient dataset) that has learned the real-world data and trust information for re-learning. This propagation toward federated learning is expected to lead to a shift from local optimization (geographical, personal, or corporate) to broader, glocal, or global optimization. In this embodiment, "relearning" also refers to the propagation of real-world data and trust information values ​​and learning results from one learning model to another, as described above. During this relearning, the trust information is weighted.

[0035] As another example, FIG. 1C illustrates a system configuration example including multiple management servers 10A, 10B, and so on that perform distributed learning and an overall management server 10Z that performs federated learning. In the example of FIG. 1C, each of the management servers 10A and 10B collects real-world data and trust information from multiple terminals 20 associated by region, individual, or company, etc., and generates a distributed learning model, as in FIG. 1B. However, the example differs from FIG. 1B in that it includes an overall management server 10C that aggregates the distributed learning models and performs federated learning. In this embodiment, "relearning" also includes the overall management server 10Z collecting distributed learning results from the management servers 10A, 10B, and so on to construct a federated learning model. During this relearning L2, the trust information may be weighted. In addition, "relearning" also includes each management server 10A, 10B, etc. obtaining the overall optimization results and federated learning results from the overall management server 10Z and correcting and feeding back the learning model, and during this relearning L3, weighting of trust information may be performed.

[0036] In this way, in this embodiment, when re-learning, 1) When the learning model that has been subjected to distributed learning or local optimization is subjected to federated learning or global optimization, 2) When the distributed learning or locally optimized results are federated learning or global optimization to the learning model, and / or 3) This is a broad concept that includes cases where a learning model that has undergone federated learning or global optimization is fed back to a learning model that has undergone distributed learning or local optimization.

[0037] Therefore, in this embodiment, trust weighting may be performed when a locally optimized learning model is globally optimized, as re-learning; trust weighting may be performed when a locally optimized learning model is federated, as re-learning; and trust weighting may be performed when a globally optimized or federated learned learning model is fed back to a local learning model, as re-learning.

[0038] Returning to FIG. 1A , the user terminal 20 is an information processing device that inputs information about the real world. The user terminal 20 is connected to the management server 10 via a network 30. The user terminal 20 may be implemented, for example, by a computer, a smartphone, a tablet information terminal, or another information processing device. The user terminal 20 may input, for example, various data read by an IoT sensor or a positioning sensor, or transaction information input by a user, as information about the real world, and output trust information or information that takes trust information into account to the user (for example, as described above as an example of trust for the totality of inputs, trust information or transaction information that takes trust information into account, showing the results of matching or integrating and evaluating product transactions based on transaction information and product movements based on location information and product IDs). Here, the significance and effects of this embodiment will be explained again using an example of trust for the totality of inputs.

[0039] In the case of accounting, the initial trust trained in the learning model is tentative, and the aim and effect is to weight and incorporate the overall judgment and know-how of experts and other human experts during the re-training process to output true trust. For example, in the case of real-world accounting transactions, the results of matching product movement data X with accounting data Z are initially trained as tentative trust to generate a learning model. However, certified public accountants who conduct accounting audits can make higher-level situational judgments and detect accounting fraud and errors. For example, certified public accountants who conduct actual accounting audits compare information about a transaction's owner, such as their age, family structure, and annual income, to determine whether they fit the persona of a person who may commit accounting fraud. If they do, they may conduct on-site inspections to uncover actual fraud. Because such expert know-how is tacit knowledge or difficult to verbalize, it has been difficult to express it in a fixed algorithm. However, with AI, it is possible to train a learning model on know-how that has not been algorithmized or verbalized, without having to code the algorithm. Therefore, as described above, by incorporating the results (in this example, trust judgment results indicating accounting fraud) of a certified public accountant's judgment into the learning model without verbalizing or algorithmizing the criteria or know-how used by the certified public accountant, the retrained learning model can output true trust. In this example, the initially generated learning model could only output trust based on the matching results of the transaction status D = (Xd, Yd, Zd, . . .) using product movement data X (e.g., product tracking data obtained by a location tag sensor) and accounting data Z (e.g., input data into accounting software). However, a learning model retrained with trust that reflects the certified public accountant's know-how is highly expected to be able to output trust as a high-dimensional judgment result that takes into account not only product movement data X and accounting data Z, but also transaction data Y (e.g., information from a human resources information system, the transaction's credit information, and data on the company and job rank).

[0040] The network 30 is an information communication network that handles communication between the management server 10 and the user terminal 20. The type of network 30 is not particularly limited as long as it is capable of transmitting and receiving data, and may be, for example, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), etc. The communication line used for data communication may be wired or wireless. Furthermore, the network may be configured to connect each device via multiple networks or communication lines.

[0041] <Computer hardware configuration> FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer used as the management server 10 and the user terminal 20. The computer 100 includes a processor 101, a read-only memory (ROM) 102, and a random access memory (RAM) 103. The processor 101 is, for example, a central processing unit (CPU), uses the RAM 103 as a working area, and executes a program read from the ROM 102. The computer 100 also includes a communication interface 104 for connecting to a network and a display mechanism 105 for displaying output on a display. The computer 100 also includes an input device 106 through which an operator of the computer 100 performs input operations. Note that the configuration of the computer 100 shown in FIG. 2 is merely an example, and the computer used in this embodiment is not limited to the example configuration shown in FIG. 2. Note that various processes performed in this embodiment are executed by one or more processors.

[0042] <Management Server Functional Configuration> Next, the functional configuration of the management server 10 will be described. FIG. 3 is a diagram showing the functional configuration of the management server 10 according to the present embodiment. As shown in FIG. 3, the management server 10 includes a real-world information acquisition unit 11 that acquires information about the real world, a trust information acquisition unit 12 that acquires information about trust evaluated for the real world, and a history information storage unit 13 that stores the history of the acquired information about the real world and trust information. The management server 10 also includes a learning unit 14 that learns by linking information about the real world with information about trust evaluated for the real world, and generates a learning model that inputs information about the real world and outputs trust information. The management server 10 also includes a trust information prediction unit 15 that predicts trust information corresponding to the acquired information about the real world, a trust information output unit 16 that outputs the predicted trust information, and a weight determination unit 17 that determines a weight to assign to the trust information when re-learning the learning model.

[0043] 3 is realized by the computer 100 shown in FIG. 2, the functions of the real-world information acquisition unit 11, the trust information acquisition unit 12, and the trust information output unit 16 are realized by, for example, the communication interface 104. The history information storage unit 13 is realized by, for example, the ROM 102. The functions of the learning unit 14, the trust information prediction unit 15, and the weight determination unit 17 are realized by, for example, the processor 101 executing a program.

[0044] <User terminal functional configuration> Fig. 4 is a diagram showing the functional configuration of user terminal 20 according to this embodiment. As shown in Fig. 4, user terminal 20 includes a real-world information acquisition unit 21 that acquires information about the real world that has occurred, a trust information acquisition unit 22 that acquires information about trust evaluated for the real world, a transmission unit 23 that transmits the acquired information about the real world and trust information to management server 10, and a display unit 24 that displays the trust information acquired from the learning model.

[0045] 4 is realized by the computer 100 shown in FIG. 2, the real-world information acquisition unit 21, the trust information acquisition unit 22, and the transmission unit 23 are realized by, for example, the communication interface 104. The display unit 24 is realized by, for example, the display mechanism 105.

[0046] <Processing when responding to the real world> Next, the flow of processing when adapted to the real world will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of processing when adapted to the real world according to this embodiment.

[0047] 5, first, the real world information acquisition unit 21 of the user terminal 20 acquires real world information related to the real world (step S201). The real world information acquisition unit 21 acquires real world information related to the real world acquired by an input device 106 such as an IoT sensor, a positioning sensor, or a touch panel. The information related to the real world acquired by the real world information acquisition unit 21 is transmitted to the management server 10 by the transmission unit 23 of the user terminal 20 (step S202), and is acquired by the real world information acquisition unit 11 of the management server 10 (step S203). The information related to the real world acquired by the real world information acquisition unit 11 is stored in the history information storage unit 13 of the management server 10 (step S204).

[0048] Next, the trust information prediction unit 15 of the management server 10 predicts trust information based on the learning model (step S205). Then, the trust information output unit of the management server 10 outputs the trust information predicted by the trust information prediction unit 15 (step S206). The generation of the learning model used in step S205 and the processing during learning will be described in detail later.

[0049] Next, the trust information acquisition unit 22 of the user terminal 20 acquires the trust information output by the learning model (step S207), and the trust information acquired by the trust information acquisition unit 22 is presented on the display unit 24 of the user terminal 20 (step S208). The display format of the trust information will be described in detail later. The user deals with the real world based on the trust information displayed on the display unit 24 of the user terminal 20.

[0050] <Creating a learning model> Next, generation of a learning model will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of generating a learning model according to this embodiment. The learning unit 14 learns by linking information about the real world stored in the history information storage unit 13 with information about trust evaluated for that real world, and generates a learning model that predicts and presents trust information from information about the real world.

[0051] 6, first, real world information acquisition unit 21 of user terminal 20 acquires information about the generated real world (sometimes referred to as "real world information" in this embodiment) (step S301). The information about the real world acquired by real world information acquisition unit 21 is transmitted to management server 10 by transmission unit 23 of user terminal 20 (step S302), and is acquired by real world information acquisition unit 11 of management server 10 (step S303). The information about the real world acquired by real world information acquisition unit 11 is stored in history information storage unit 13 of management server 10 (step S304).

[0052] Next, the trust information acquisition unit 22 of the user terminal 20 acquires information on the trust evaluated by the user for the real world (step S305). The trust information acquisition unit 22 acquires the trust information inputted, for example, by the input device 106. As a specific example, the trust information acquisition unit 22 may acquire the trust information as a result of matching the transaction data with the tracking data of the product.

[0053] As another example, the trust information acquiring unit 22 may acquire trust information based on the type of the input device 106. For example, the trust information acquiring unit 22 may acquire low trust information when the real-world information is manually input by a user, medium trust information when the real-world information is automatically input by an IoT sensor, a positioning sensor, or the like, or high trust information when the real-world information is automatically input by an authenticated IoT sensor, a positioning sensor, or the like.

[0054] As another example, the trust information acquisition unit 22 may acquire high-trust trust information when real-world information is secured by error- and tamper-proof technologies such as signal authentication, electronic authentication, or blockchain, and may acquire low-trust trust information when real-world information is not secured.

[0055] In addition, in this embodiment, trust information may be acquired by a known method such as the Trust Insertion Technology of Project Alvarium, as exemplified below. That is, the trust information acquisition unit 22 may check (A) whether data verification is performed using a distributed ledger, (B) whether the storage destination is secure, (C) whether the gateway is authenticated, (D) whether there is provenance metadata indicating where the data originated, and (E) whether there is owner information based on a device signature, as well as (1) whether noise is removed from the data distribution using the 3 sigma rule, (2) whether the optimal polling frequency is exceeded using the Nyquist criterion and Parseval's theorem, (3) whether the optimal number of devices calculated from the distance λ between sensors, the optimal polling frequency, etc. is exceeded, and (4) whether the authenticity fingerprint is correct. ) match, and (5) the results of evaluating the reliability of real-world data (real-world information) using practical Byzantine Fault Tolerance (pBFT), etc., may be obtained as trust information (see U.S. Patent Application Publication US2021306819A1, U.S. Patent Publications US11256492B2, US11507698B2, US11824883B2, etc.).

[0056] 6 again, the trust information acquired by the trust information acquisition unit 22 is then transmitted to the management server 10 by the transmission unit 23 of the user terminal 20 (step S306), and is acquired by the trust information acquisition unit 12 of the management server 10 (step S307). The trust information acquired by the trust information acquisition unit 12 is stored in the history information storage unit 13 of the management server 10 (step S308).

[0057] Next, the learning unit 14 of the management server 10 links the information about the real world stored in the history information storage unit 13 with the trust information evaluated for that real world and learns (step S309). The learning unit 14 then generates or updates a learning model that inputs information about the real world and outputs trust information (step S310). Details of updating the learning model will be described later, but as an example, based on the know-how of certified public accountants who perform accounting audits, the results of determining accounting fraud and errors are weighted as trust close to the truth and re-learned.

[0058] <Learning section processing> Next, an example of the processing performed by the learning unit 14 in steps S309 and S310 in FIG. 6 will be described.

[0059] The learning unit 14 learns by linking information about the real world with information about the trust evaluated for that real world, and generates a learning model that inputs information about the real world and outputs trust information. The function of the learning unit 14 is realized, for example, by the processor 101 of the computer 100 executing a machine learning program.

[0060] A machine learning program is a program that learns by machine learning the relationship between information about the real world as input and trust information as output. When a machine learning program is given information about the real world and trust information evaluated for that real world as training data, it adjusts the variables of each layer that make up a deep learning model, for example, based on this training data. Then, when information about the real world is given as input, learning proceeds so that trust information for the real world is output.

[0061] Fig. 7 is a diagram illustrating an example of a deep learning model, in which a convolutional neural network (CNN) is illustrated as an example of the deep learning model.

[0062] A convolutional neural network consists of an input layer, an output layer, and many hidden layers in between. A convolutional layer, a typical example of a hidden layer, extracts features of information about the real world, followed by a pooling layer that extracts the average or maximum of the extracted features. A convolutional neural network has a multi-layer structure in which unit structures made up of convolutional layers and pooling layers are connected in multiple stages, and these processes are repeated to identify different features in each layer and progress in learning.

[0063] The learning model calculates the probability that each piece of trust information held as a candidate is correct, and outputs those with a probability equal to or greater than a predetermined threshold, or outputs a predetermined number of pieces with the highest probabilities. In the example shown in Figure 7, when information about the real world is input, one or more pieces of trust information are output from the trust information held as candidates.

[0064] As another example, Table 1 below shows an example of a learning model generated from greenhouse gas emission data (emission data) and trust data ("Confidence Data" in the table) registered in a distributed ledger ("Decentralized Single Source of Truth" in the table) using Project Alvarium's Trust Insertion Technology. Alvarium's server records a low trust value ("Confidence Score" in the table) in the distributed ledger if the emission data is manually entered (Manual Data), and a high trust value in the distributed ledger if the data is automatically entered by an IoT sensor (Autonomous Data). This emission data and trust information (Confidence Score) are associated and trained into a generative AI learning model that inputs emission data and outputs trust. This offers strong hope for the construction of a learning model that determines and outputs trust information based on the metadata of the emission data, etc. Furthermore, by relearning the true trust, it is strongly expected that the weight coefficient w linking the emitted data input with the false trust will approach 0, and the weight coefficient w linking the emitted data input with the trust close to the true will approach 1. [Table 1]

[0065] <Display format> Next, the display format in step S208 of Fig. 5 will be described with reference to Fig. 8A and Fig. 8B. Fig. 8A is a diagram showing an example of a screen that displays trust information for real-world data ("RW data" in the figure).

[0066] As shown in FIG. 8A, the display unit 24 of the user terminal 20 displays a real-world information type 401 and trust content 402. The display unit 24 also displays evaluation status 403 and improvement 404, and accepts input from the user. The display unit 24 also displays an add button 406 and a register button 407. In FIG. 8A, the display unit 24 of the user terminal 20 displays real world A in real-world information type 401. The display unit 24 also displays trust 1, trust 2, and trust 3 in trust content 402 for real world A. The display unit 24 also displays check boxes 405 in evaluation status 403 and improvement 404, and accepts input from the user as to whether they have evaluated each trust and whether the real world has improved as a result of the presented trust evaluation.

[0067] More specifically, when displaying the trust information, the display unit 24 of the user terminal 20 displays the trust information for the real-world data. As shown by the dashed-dotted line in FIG. 8A , the display unit 24 may display the trust information for a single piece of real-world data (positioning information in this example), or may display the trust information for a combination of multiple pieces of real-world data (a combination of positioning information and / or IoT information and transaction information in this example) as shown by the dotted or solid line in FIG. 8A . The display unit 24 may also display a screen that accepts user input, along with the trust information, on whether the user evaluated the accuracy of the presented trust and whether the presented trust information has improved the real world. As a specific example, when low trust information is presented for a combination of positioning information, IoT information, and transaction information as shown by the solid line in FIG. 8A , a user such as an accountant can reevaluate (confirm) the inconsistency between the movement of goods based on the positioning information and IoT information and the transaction information. If the accounting information improves as a result, the user can check the evaluation checkbox and the improvement checkbox to register the improvement.

[0068] As another example, FIG. 8B is a diagram showing an example of a screen presenting countermeasures and trust information for real-world data ("RW data" in the figure). In this example, it is assumed that a learning model has been generated in which real-world information is input and countermeasures for the real world and trust information are output as information other than trust information. As shown in FIG. 8B, the display unit 24 of the user terminal 20 presents countermeasures A to C for the real world along with the input real-world data such as positioning information, IoT information, and transaction information, and also presents trust information for each countermeasure.

[0069] If the user implements and evaluates the presented countermeasures A to C, the user can check the checkbox 405 for implementation evaluation 403, and if there is an actual improvement in the real world, the user can check the checkbox for improvement 404 and register the results. As a specific example, as shown in FIG. 8B, for a combination of positioning information, IoT information, and transaction information, if countermeasure A, a field inspection, is indicated by trust 1 information (low), countermeasure B, a telephone survey of the person in charge, is indicated by trust 2 information (medium), and countermeasure C, a cross-examination, is indicated by trust 3 information (high), a user such as a tax official can check the checkbox 405 depending on whether or not there is an evaluation of the implementation of these countermeasures A to C, and if there is an actual improvement among the implemented countermeasures, the user can check the checkbox for improvement 404 and register the results. In the example shown in FIG. 8B, the user evaluates the trust of all of countermeasures A (trust 1), countermeasure B (trust 2), and countermeasure C (trust 3). Furthermore, the real world does not improve based on the assessment of Treatment A Trust 1 and Treatment B Trust 2 (for example, fraud and errors do not improve through on-site or telephone surveys), but the assessment of Treatment C Trust 3 indicates that real world A has improved (for example, fraud and errors have improved through counter-surveys).

[0070] If the trust evaluations presented in Figures 8A and 8B do not improve the real world and the user evaluates other trust at their own discretion, the user can add the evaluated trust information (along with a solution in the example of Figure 8B) from Add button 406. The input is completed when the user presses Register button 407. Note that the method of presenting trust information and the method of inputting trust by the user are not limited to display input means such as a touch panel, and other input / output means (physical presentation means using an actuator and user input of steering, braking, etc. in response to the presentation) can also be used, as explained using Figure 10.

[0071] <Updating the learning model> In operating a learning model, it may be necessary to periodically update the learning model by performing re-learning to reflect the latest real-world information or higher-level trust information. When updating the learning model, for example, the management server 10 acquires information about the new real world and information about the trust evaluated for that real world, and updates the learning model by having the learning model re-learn. Note that, as described above, if a learning model is generated that inputs real-world information and outputs information other than trust information (information about how to deal with the real world, etc.) and trust information, it goes without saying that when updating the learning model, the management server 10 also acquires information about the new real world and information about how to deal with the real world, etc., and trust information, and updates the learning model by having the learning model re-learn.

[0072] When faced with a new real world, the user inputs information about the real world into the real-world information acquisition unit 21 of the user terminal 20, and deals with the real world based on the trust information displayed on the display unit 24 of the user terminal 20. The user then inputs the evaluated trust information into the trust information acquisition unit 22 of the user terminal 20.

[0073] If weighting is not performed during relearning, the management server 10 treats all trust information evaluated by users as correct data when updating the learning model and performs relearning. However, for example, if the real world improves due to trust evaluations other than the trust information presented by the learning model, the trust information presented by the learning model may be incorrect noise or bias. Also, as shown in Figures 8A and 8B, if the multiple trust information presented by the learning model includes a mixture of information that has improved the real world and information that has not improved, the trust information presented by the learning model may be incorrect noise or bias.

[0074] Therefore, in this embodiment, the management server 10 weights the correct answer data when updating the learning model, thereby suppressing the amplification and bias of incorrect answer noise in the learning model. Weighting is performed on the correct answer label linked to trust information. When re-learning is performed on the correct answer data, if no weighting is performed, the correct answer label is set to 1, and if weighting is performed, the value of the correct answer label is lowered (for example, if the answer is incorrect, the correct answer label is set to 0) and re-learning is performed.

[0075] The learning model according to this embodiment calculates the probability that the trust information held as a candidate is correct, for example. As the learning model is updated, the variables at each layer constituting the deep learning model fluctuate, causing the probability of the correct answer to fluctuate. By weighting the correct answer data during re-learning, the probability that trust information with a small weight is correct decreases, and the probability of it being presented decreases. This makes it possible to suppress the amplification and bias of noise from incorrect answers in the learning model.

[0076] An example of weighting evaluation according to this embodiment will be described with reference to Fig. 9A and Fig. 9B. Fig. 9A and Fig. 9B are diagrams showing an example of weighting evaluation according to this embodiment. Fig. 9A and Fig. 9B show an example in which trust 1 to 3 are presented as information on trust in the real world.

[0077] <Example 1 of weighting evaluation during re-learning> In evaluation example 1 of weighting during relearning, the weight determination unit 17 of the management server 10 weights the presented trust information based on the fact that trust information was presented in the past real world.

[0078] For example, the weight determination unit 17 assigns a smaller weight to the presented trust information the higher the rate at which trust information has been presented in the past real world. The learning model analyzes the contents of the real world to predict and present the corresponding trust information, but trust information that has been presented frequently in the past and has a high presentation rate (i.e., trust that is presented in any real-world situation) may be presented even in the case of an incorrect answer or bias. Therefore, when trust information that has been presented frequently in the past real world is presented, it is likely to have been presented regardless of the state of the real world, and a smaller weight is assigned to it.

[0079] On the other hand, when information with low trust that has been presented in the past in the real world is presented, it is likely to be correct information presented in an unusual real-world situation, so it is designed to be weighted higher than information with high trust that has a high presentation rate. Note that the format of weighting is not limited to this example, and weighting may be assigned regardless of the ratio, such as assigning a small weight if trust information has been presented in the past in the real world a predetermined number of times or more.

[0080] An example of weighting will be described with reference to Fig. 9A. Fig. 9A is a diagram showing an example in which the higher the rate at which a certain trust has been presented in the past real world, the smaller the weight assigned to that trust. In the example shown in Fig. 9A, the trust presented by the learning model in accordance with the real-world situation is weighted relative to the correct label of that trust based on the rate at which it has been presented in the past real world, regardless of whether it is the correct trust for the real world or not.

[0081] FIG. 9A shows trust information 501, model presentation 502, presentation rate 503, and correct answer label 504. Note that the trust information items may include information other than trust information, such as countermeasures. Trust information 501 indicates the trust evaluated as a response to the real world that has occurred (along with countermeasures, etc., in some cases). Model presentation 502 indicates whether the trust information is presented from a learning model or not (i.e., user-input trust). Presentation rate 503 indicates the presentation rate for the past real world when trust information 501 is presented from a learning model. Correct answer label 504 indicates the weight assigned to the correct answer label linked to trust information 501 when the learning model is trained using trust information 501 as correct answer data.

[0082] 9A, the weight determination unit 17 of the management server 10 weights the presented trust information 501 to 0.8 if the presentation rate 503 of the trust information 501 in the past to the real world is less than 1%, to 0.5 if it is 1% or more and less than 10%, and to 0.1 if it is 10% or more. In addition, the trust information 501 not presented by the model is considered to be the correct trust input by the user, and therefore the weighting of the correct label 504 is 1.

[0083] For example, trust 1 is not presented as trust information from the learning model, so the weighting of the correct label 504 is 1. Trust 2 is presented from the model, and the presentation rate 503 for the past real world is 20%, so the correct label 504 is weighted to 0.1. Trust 3 is presented from the model, and the presentation rate 503 for the past real world is 1%, so the correct label 504 is weighted to 0.5. Note that the weighting method shown in FIG. 9A is an example of weighting, and is not limited to this format.

[0084] <Example 2 of weighting evaluation during re-learning> In evaluation example 2 of weighting during relearning, the weight determination unit 17 of the management server 10 weights the presented trust information based on the results of an evaluation based on the trust information presented for the past real world. There are two cases in the results of an evaluation based on the trust information presented for the past real world. One is when the presented trust information improves the real world, and the other is when the presented trust information does not improve the KPI (Key Performance Indicator) of the real world, but other trust evaluated at the user's discretion improves the KPI of the real world. The weight determination unit 17 of the management server 10 weights the presented trust information according to, for example, the proportion of these two cases.

[0085] For example, the weight determination unit 17 assigns a smaller weight to the trust information presented by the learning model the higher the percentage of cases in which a KPI in the real world is improved by a trust different from the trust information presented by the learning model in the past. If the real world improves due to a user evaluation of trust different from the presented trust information, it cannot be concluded that the trust information presented by the learning model was incorrect. However, since the presented trust information did not directly improve the real world, the presented trust information is unlikely to be correct. Therefore, the higher the percentage of cases in which a user evaluation of trust that is not presented in this way improves the real world, the smaller the weight is assigned to the trust information presented by the learning model.

[0086] An example of weighting will be described using Figure 9B. Figure 9B is a diagram showing an example in which the higher the percentage of cases in which the real world is improved by trust information different from the trust information presented for the past real world, the lower the weight assigned to the presented trust information. In Figure 9B, Case A refers to a case in which the real world is improved by trust information presented for the past real world, and Case B refers to a case in which the real world is not improved by the trust information presented for the past real world, but is improved by other trust evaluated at the user's discretion. For example, "the number of Case A for Trust 1 is 100" means that in the past real world, the learning model presented Trust 1 and Trust 1 improved the KPI of that real world 100 times.

[0087] FIG. 9B shows trust information 505, model presentation 506, number of A cases 507, number of B cases 508, A case rate 509, and correct label 510. Trust information 505 indicates trust as a response to the real world that has occurred. Model presentation 506 indicates whether trust is presented from a learning model. Number of A cases 507 indicates the number of A cases in the past real world when trust information 505 is presented from a learning model. Number of B cases 508 indicates the number of B cases in the past real world when trust information 505 is presented from a learning model but improvement is made using a method other than the presented trust information. Case rate 509 indicates the rate of A cases in the past real world when trust information 505 is presented from a learning model. Correct label 510 indicates a weight assigned to the correct label associated with trust information 505 when the learning model is trained using trust information 501 as correct data.

[0088] 9B, the weight determination unit 17 of the management server 10 assigns weights based on the proportion of case A and case B in the entire real world in which trust information 505 was presented in the past. The weight determination unit 17 assigns a weight of 0.8 when the case A rate 509 is less than 10%, 0.5 when it is 10% or more but less than 50%, and 0.1 when it is 50% or more. Furthermore, trust information 505 not presented by the model is considered to be correct trust evaluated by the user's judgment, and therefore the correct answer label 510 is set to 1.

[0089] For example, trust 1 is not presented as trust information from the learning model, so no weighting is performed and the correct label 510 is 1. Trust 2 is presented from the learning model, and the A case rate 509 is 20%, so the correct label 510 is weighted to 0.5. Trust 3 is presented from the learning model, and the A case rate 509 is 75%, so the correct label 510 is weighted to 0.1. Note that the weighting method shown in FIG. 9B is an example of weighting, and is not limited to this format.

[0090] Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above evaluation format. Various modifications or improvements to the above evaluation format are also included within the technical scope of the present invention. For example, although the above description uses an example in which a user responds to the real world, the present invention is not limited to this format. It is also possible to use a format in which a user inputs real-world content about a real world that occurs at home, etc., and then provides corresponding instructions.

[0091] (Addendum) (((1))) one or more processors; the one or more processors: Acquire information about the real world and information about the trust assessed for the real world; generating a learning model that inputs information about the real world and outputs information about the trust; Re-training the learning model based on information about the new real world and information about the trust output for the new real world; When retraining the learning model, weighting is given to the trust information. A system characterized by: (((2))) the one or more processors: Weighting the trust information based on the fact that the trust information was presented to the real world in the past The system according to (((1))) is characterized in that (((3))) the one or more processors: The higher the rate at which the trust information has been presented in the past in the real world, the smaller the weight is assigned to the trust information. The system according to (((2))) is characterized in that (((4))) the one or more processors: Weighting the trust information based on the results of evaluation based on the trust information presented to the real world in the past The system according to (((1))) is characterized in that (((5))) the one or more processors: Weighting the presented information on the trust in accordance with a situation in which the real world has improved due to the trust evaluated based on the information on the trust presented for the real world in the past, and a situation in which the real world has improved due to an evaluation of a trust different from the presented information on the trust The system according to (((4))) is characterized in that (((6))) the one or more processors: The higher the percentage of cases where the real world has been improved by a trust evaluation that differs from the trust information presented for the real world in the past, the smaller the weight is given to the presented trust information. The system according to (((5))) is characterized in that (((7))) The information about the real world is information about financial transactions, and the information about the trust is information about the trust. A system according to any one of (((1))) to (((6))) characterized in that (((8))) A program to be implemented on one or more processors, A function for obtaining information about the real world and information about the trust assessed for the real world; A function of generating a learning model that inputs information about the real world and outputs information about the trust; a function of re-learning the learning model based on information about the new real world and information about the trust output for the new real world; A function of weighting the trust information when re-learning the learning model; A program with.

[0092] According to the inventions (((1))) and (((8))), the rate at which information on incorrect trusts is presented can be reduced compared to when all evaluated trusts are equally re-learned as correct data. According to the invention (((2))), the rate at which trust information with low importance is presented can be reduced. According to the invention (((3))), it is possible to lower the presentation rate of trust information that has a high presentation rate and low importance. According to the invention of (((4))), the presentation rate of trust information with low importance can be reduced based on the results of past trust. According to the invention of (((5))), the rate at which trust information of low importance is presented can be reduced based on whether the real world has improved as a result of trust information presented in the past. According to the invention of (((6))), the rate at which trust information is presented can be increased as the rate at which the real world has improved due to trust information presented in the past increases. According to the invention of (((7))), it is possible to reduce the rate at which incorrect or biased trust information is presented in relation to accounting fraud or errors.

[0093] [Other forms of evaluation] So far, we have explained the evaluation forms of the present invention, but in addition to the evaluation forms described above, the present invention may be evaluated in various different evaluation forms within the scope of the technical idea described in the claims.

[0094] For example, the learning system has been described as being implemented as a distributed computing system in separate housings that realize the functions, but this is not limited to this and it may be implemented in the same housing. In addition, each function of the control unit, memory unit, etc. can be functionally or physically distributed or integrated according to the functional load or in any unit.

[0095] Furthermore, although an example has been described in which the management server 10 of the learning system 1 performs processing in response to a request from a client terminal such as a user terminal 20 and returns the processing results to the client terminal, the learning system 1 may also be configured as an integrated unit including the management server 10, the user terminal 20, etc., to perform processing in a stand-alone form.

[0096] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods.

[0097] In addition, the processing procedures, control procedures, specific names, registered data for each process, information including parameters such as search conditions, screen examples, and database configurations shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0098] The present invention is not limited to the above-described embodiments, but includes various modifications. Furthermore, for example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

[0099] Furthermore, with regard to the learning system 1 and the like, the components shown in the figures are functional concepts, and do not necessarily have to be physically configured as shown in the figures.

[0100] For example, all or any part of the processing functions of each device in the learning system 1, particularly those performed by the control unit of the management server 10 (such as the real-world information acquisition unit 11 to the weight determination unit 17), may be implemented by a processor 101 such as a central processing unit (CPU) and a program interpreted and executed by the processor 101, or may be implemented as a hardware processor using wired logic. The program is recorded on a non-transitory computer-readable recording medium containing programmed instructions for causing a computer to execute the method of the present invention, as described below, and is mechanically read by the learning system 1 or the management server 10 as needed (or may be read by the learning system 1 or the management server 10 from a SaaS server, etc., as needed). That is, a computer program is recorded in a storage unit such as the ROM 102, RAM 103, or HDD (hard disk drive) for cooperating with an operating system (OS) to issue instructions to the CPU and perform various processes. The computer program is executed by being loaded into the RAM and cooperates with the CPU to form the control unit. Furthermore, the program code of the software that realizes the evaluation function may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or the storage medium. The program code that realizes the functions described in this embodiment may be implemented in a wide range of program or script languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0101] The management server 10 of the learning system 1 may propose data provision to users of the user terminals 20, or may discover, analyze, and evaluate data assets and propose data linkage and data provision to potential data providers. As a specific example, the management server 10 of the learning system 1 may estimate the possibility of data acquisition and the value of the obtainable data as trust based on real-world information about the resources of potential data providers (e.g., catalog data related to real-world information such as customer information, business partner information, logistics information, accounting information such as ledgers and ledger data, business model information, human resource information, and know-how information), and propose data provision to the user terminals 20. The learning system 1 may also discover various tangible and intangible assets by inferring original data from trained models and ontology information, and propose the provision of trust as the data value generated from those assets.

[0102] In addition, this computer program may be stored in an application program server connected to the user terminal 20, the learning system 1, the management server 20, etc. via any network, and all or part of it may be downloaded as needed.

[0103] The program according to the present invention may be stored on a computer-readable recording medium or configured as a program product. Here, the term "recording medium" includes any "portable physical medium" such as a memory card, USB memory, SD card, flexible disk, magneto-optical disk, ROM, EPROM, EEPROM, CD-ROM, MO, DVD, and Blu-ray (registered trademark) Disc.

[0104] Furthermore, a "program" refers to a data processing method written in any language or description method, regardless of the format, such as source code or binary code. Note that a "program" is not necessarily limited to a single entity, but also includes a distributed structure consisting of multiple modules or libraries, or a program that achieves its function by cooperating with a separate program, such as an OS (Operating System). Note that the specific configurations and reading procedures for reading the recording medium in each device shown in the evaluation form, as well as the installation procedures after reading, can be well-known configurations and procedures. The present invention may also be configured as a program product in which the program is recorded on a non-transitory computer-readable recording medium.

[0105] Various databases (learning models, etc.) stored in a memory unit such as ROM 102 are stored in a memory device such as RAM or ROM, a fixed disk device such as a hard disk, a flexible disk, or an optical disk, which stores various programs, tables, databases, files, etc. used for various processes and website provision. The memory unit for the learning models, etc. may be a volatile semiconductor memory such as a random access memory (RAM), or a non-volatile storage such as a hard disk drive (HDD) or flash memory. The control unit such as the learning unit 14 is, for example, a processor such as a central processing unit (CPU) or a digital signal processor (DSP). However, the control unit such as the learning unit 14 may include a purpose-specific electronic circuit such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor executes a program stored in a memory such as RAM (which may be a learning model). A collection of multiple processors is sometimes referred to as a "multiprocessor" or simply a "processor."

[0106] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the evaluation function. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the above-described evaluation function, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, solid-state drives (SSDs), optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0107] Furthermore, the learning system 1, user terminal 20, management server 10, etc. may be configured as a known information processing device such as a personal computer or workstation, or may be configured by connecting any peripheral device to the information processing device. Furthermore, the learning system 1, user terminal 20, management server 10, etc. may be realized by installing software (including programs, data, etc.) that causes the information processing device to implement the method of the present invention.

[0108] Furthermore, the specific form of distribution and integration of the equipment is not limited to that shown in the figure, and all or part of it can be configured by functionally or physically distributing and integrating in any unit according to various additions, etc., or according to the functional load. In other words, the above-mentioned evaluation forms may be evaluated in any combination, or the evaluation form may be selectively evaluated. [Explanation of symbols]

[0109] 1... learning system, 10... management server, 20... user terminal, 30... network, 100... computer

Claims

1. one or more processors; the one or more processors: Obtaining information about the real world and information about the trust assessed for the real world; generating a learning model that inputs information about the real world and outputs information about the trust; A learning system characterized by:

2. When the input may include information generated by artificial intelligence (AI) rather than information about the real world, 2. The learning system according to claim 1, wherein the trust information is information about the reliability of the information about the real world.

3. The "information about the real world" includes information generated by artificial intelligence (AI), but also information that has been verified by a real person.

2. The learning system according to claim 1, wherein the trust information is information about the reliability of the information about the real world being verified by a real person.

4. the one or more processors: obtaining information about the real world and / or the trust assessed with respect to the real world from an external system; The learning system according to claim 1 .

5. The one or more processors further Re-training the learning model based on information about the new real world and information about the trust output for the new real world; When retraining the learning model, weighting is given to the trust information.

2. The learning system according to claim 1.

6. the one or more processors: The trust information is weighted based on the evaluation result based on the trust information presented to the real world in the past.

6. The learning system according to claim 5.

7. the one or more processors: Weighting is performed on the presented information of the trust in accordance with a situation in which the real world has improved due to the trust evaluated based on the information of the trust presented for the real world in the past, and a situation in which the real world has improved due to an evaluation of a trust different from the presented information of the trust.

7. The learning system according to claim 6.

8. the one or more processors: The lower the percentage of cases where the real world has improved due to a trust evaluation that differs from the trust information presented for the real world in the past, the lower the weight given to the presented trust information.

8. The learning system according to claim 7.

9. the one or more processors: Weighting the trust information based on the fact that the trust information was presented to the real world in the past 6. The learning system according to claim 5.

10. the one or more processors: The lower the rate at which the trust information has been presented in the past real world, the greater the weight given to the trust information.

10. The learning system according to claim 9.

11. the information about the real world is information about greenhouse gas emissions; The trust information is information on the reliability of the greenhouse gas emission data.

11. A learning system according to any one of claims 1 to 10.

12. The trust information is output internally within the learning model or externally from the learning model.

11. A learning system according to any one of claims 1 to 10.

13. The trust information is information that conceptually includes at least one of usability, authenticity, and reliability.

11. A learning system according to any one of claims 1 to 10.

14. If you are re-learning, 1) When the learning model that has been subjected to distributed learning or local optimization is subjected to federated learning or global optimization, 2) When the distributed learning or locally optimized results are federated learning or global optimization for the learning model, and / or 3) Including cases where federated learning or globally optimized learning models are fed back to distributed learning or locally optimized learning models.

11. A learning system according to any one of claims 1 to 10.

15. The learning model is a generative AI (Artificial Intelligence), The information about the real world is input, and information other than the information about the trust is output.

11. A learning system according to any one of claims 1 to 10.

16. 1. A training method executed on one or more processors, comprising: obtaining information about the real world and information about the trust assessed for the real world; generating a learning model that inputs information about the real world and outputs information about the trust; A learning method comprising:

17. A program to be implemented on one or more processors, A function for obtaining information about the real world and information about the trust assessed for the real world; A function of generating a learning model that inputs information about the real world and outputs information about the trust; A learning program with:

Images