Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program for supporting the management of AI assets in a multi-cloud environment. [Solution] An information processing system that manages multiple AI models deployed on at least two or more cloud platforms, comprising: a standard information storage unit that stores standard information including at least one of the security policies and operational rules of the AI ​​models that apply to a user organization; a model information acquisition unit that acquires configuration information indicating settings for each AI model from the cloud platform; an audit processing unit that evaluates the conformance of the AI ​​models with the standard information based on the configuration information of each AI model; and a risk assessment unit that calculates a risk score for each AI model based on the conformance assessment results.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program for managing AI assets operated within an organization. [Background technology]

[0002] In recent years, attempts to introduce AI models have been underway in various industries with the aim of improving business efficiency, etc. For example, Patent Document 1 discloses a system for supporting sales activities that integrates generative AI with an in-house system via an API to generate responses to inquiries. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2025-045244 Summary of the Invention [Problem to be solved by the invention]

[0004] AI models, such as the generative AI utilized in Patent Document 1, are developed by multiple cloud providers, and organizations such as companies are increasingly utilizing multiple AI models distributed across multiple cloud platforms. Therefore, there is a need to integrate and manage multiple AI models used in a multi-cloud environment.

[0005] One of the objectives of exemplary embodiments of the present disclosure is to provide an information processing system, an information processing method, and a program for supporting management of AI assets in a multi-cloud environment. [Means for solving the problem]

[0006] An information processing system according to one aspect of the present disclosure includes: An information processing system that manages multiple AI models deployed on at least two or more cloud platforms, a standard information storage unit that stores standard information including at least one of a security policy and an operation rule of the AI ​​model that is applied to a user organization; a model information acquisition unit that acquires configuration information indicating settings for each of the AI ​​models from the cloud platform; an audit processing unit that evaluates the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; and a risk assessment unit that calculates a risk score for each AI model based on the results of the compatibility assessment.

[0007] By having the above-mentioned features, the information processing system can centrally manage multiple AI assets used in a multi-cloud environment within a user's organization.

[0008] Other problems and solutions disclosed in the present application will become apparent from the embodiments and drawings of the present disclosure. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an information processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of the management server shown in FIG. [Figure 3] FIG. 3 is a block diagram illustrating an example of the software configuration of the information processing system shown in FIG. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of a user interface that presents an AI model. [Figure 5] FIG. 5 is a schematic diagram showing an example of a user interface that presents the usage status of an AI model. [Figure 6] FIG. 6 is a flowchart showing an example of information processing executed in the information processing system shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] An information processing system according to an embodiment of the present disclosure will be described below with reference to the drawings. In the accompanying drawings, identical or similar elements are designated by identical or similar reference symbols and names, and duplicate descriptions of identical or similar elements may be omitted in the description of the embodiment. Note that the contents shown in the drawings are merely examples for explaining the present embodiment and are merely schematic examples for ease of explanation of the present embodiment. The contents of the drawings may be modified or changed within the scope of no technical problem.

[0011] <System Overview> The information processing system according to this embodiment is a system for managing multiple AI models deployed on at least two or more cloud platforms. Here, "cloud platform" refers to an operational infrastructure for cloud computing services that provides computing resources, storage resources, or application resources via a network. Examples of cloud platforms include public clouds such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), as well as private clouds exclusively used by specific organizations. Furthermore, "AI model" refers to a software component built based on machine learning and / or artificial intelligence technologies that performs predetermined computational processes, such as prediction, analysis, inference, or generation, on input data. Examples of AI models include models that perform predetermined computational processes, such as classification and regression, using machine learning algorithms, image recognition models using deep learning, natural language processing models, generative AI models such as large-scale language models (LLMs), and image generation models.

[0012] AI models such as those described above are being used in a variety of ways. For example, they are used to perform tasks such as data analysis for business improvement, planning business strategies such as sales strategies, and document classification or automatic generation, while also being incorporated into corporate products such as chatbots and image recognition functions. Furthermore, AI models are increasingly being used to generate new content and support intellectual tasks using large-scale language models and image generation models. Business organizations, such as corporations, are adopting a variety of AI models depending on their intended use. As AI models are increasingly being adopted, organizations are increasingly operating them in so-called multi-cloud environments, where AI models are distributed across multiple cloud platforms rather than centralized in a single environment. In such multi-cloud environments, the risks and operational data associated with each AI model are also distributed across each cloud, making it difficult to grasp the overall picture. Therefore, a system that can comprehensively manage multiple AI models in a multi-cloud environment is needed.

[0013] In this embodiment, the information processing system acquires configuration information for each AI model (information indicating AI model settings such as disclosure settings, access rights, region, encryption status, and authentication settings) from each cloud platform and then audits the configuration information based on standard information, such as security policies and / or operational rules, applied to the user organization. Furthermore, the audit evaluates the conformance of each AI model with the standard information, and then calculates a risk score for each AI model based on the conformance evaluation results. With this configuration, the information processing system of this embodiment can centrally perform unified audits and quantitative risk assessments against standard information for multiple AI models distributed across a multi-cloud environment, thereby improving the reliability and management efficiency of AI assets throughout the organization. Details of this system are described below using examples shown in the drawings.

[0014] <System configuration> As shown in FIG. 1, the information processing system of this embodiment includes a management server 1 and one or more user terminals 2. The management server 1 and the user terminals 2 are connected to each other so that they can communicate with each other via a network NW. The management server 1 and the user terminals 2 can also be connected to each cloud platform via the network NW. One or more AI models are deployed on each cloud platform. Multiple AI models may be deployed on one cloud platform, or substantially the same AI model (for example, copies of one AI model) may be deployed on multiple cloud platforms.

[0015] Each user operating the user terminal 2 can use the AI ​​model deployed on the cloud platform via the network NW. In this embodiment, the network NW is primarily assumed to be the Internet, but the network NW is not limited to the Internet and may be constructed using, for example, a public telephone network, a mobile phone network, a wireless communication network, Ethernet (registered trademark), or the like. Note that the illustrated system configuration is an example and is not limiting.

[0016] <Management Server 1> The management server 1 is an information processing device that performs various information processing related to the management of AI models operated by a business organization. The management server 1 may be deployed at a user organization that uses the system, or at a business that provides the system. The "user organization" that uses the system may be, but is not limited to, a corporate enterprise, research institution, non-profit organization (such as an educational institution or medical corporation), unincorporated organization, public organization, public interest corporation, or business association that uses AI models in specific business activities. The management server 1 may be configured on-premise using a general-purpose computer such as a workstation or personal computer, or may be logically implemented using cloud computing.

[0017] 2 is a block diagram illustrating an example of the hardware configuration of the management server 1. Note that the illustrated configuration is an example, and the management server 1 may have other configurations. The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmission / reception unit 13, an input / output unit 14, etc., which are electrically connected to one another via a bus 16.

[0018] The processor 10 is a computing device that controls the overall operation of the management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for application execution and authentication processing. For example, the processor 10 is a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit). Each function (means) of the management server 1 is realized by the processor 10 executing a program stored in the storage 12 and deployed in the memory 11.

[0019] The memory 11 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory), and an auxiliary memory configured with a non-volatile storage device such as a flash memory, an HDD (Hard Disc Drive), etc. The memory 11 is used as a work area for the processor 10, and also stores a BIOS (Basic Input / Output System) that is executed when the management server 1 starts up, various setting information, etc.

[0020] The storage 12 stores various programs such as application programs, and in particular stores programs for executing the various functions of the present system. A database storing data used for each process may also be constructed in the storage 12. For example, a storage unit 120 (described later) is realized as part of the storage area of ​​the memory 11 and / or the storage 12.

[0021] The transmitting / receiving unit 13 is a communication interface that enables the management server 1 to communicate with various information processing terminals such as the user terminal 2 via a communication network. The transmitting / receiving unit 13 may further include a short-range communication interface such as Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) and / or a USB (Universal Serial Bus) terminal.

[0022] The input / output unit 14 is an information input device such as a keyboard, a mouse, etc., and an output device such as a display, etc. The input / output unit 14 may include a touch panel or the like that has both functions of inputting and outputting information, and may also include a printer, a speaker, etc. as output devices.

[0023] A bus 16 is commonly connected to the above elements and transmits, for example, address signals, data signals and various control signals.

[0024] <User device 2> The user terminal 2 is an information processing terminal owned by a member (member refers to any person belonging to a business entity, regardless of job title or position) of a user organization that uses this system. The user terminal 2 may be, for example, a mobile terminal such as a smartphone or tablet terminal, or a general-purpose computer such as a workstation or personal computer. The user terminal 2 also includes a processor 20, memory 21, storage 22, a transmitter / receiver 23, an input / output unit 24, etc., which are electrically connected to each other via a bus 26. Each element in the hardware configuration of the user terminal 2 can be configured in the same way as the management server 1 shown in FIG. 2, and detailed description of each element of the user terminal 2 will be omitted.

[0025] <Management Server 1 Functions (Software Configuration)> FIG. 3 is a block diagram illustrating an example of functions (software configuration) implemented in the information processing system. The management server 1 may include, for example, a model information acquisition unit 101, an audit processing unit 102, a risk assessment unit 103, and a display control unit 104 as functions realized by the processor 10 executing a program. These functional units are illustrated as functions executed by the processor 10 of the management server 1, but may also be configured to be executed by a processor of another information processing device, such as a user terminal 2, instead of the management server 1. The storage unit 120 of the management server 1 may include various databases, such as a model information storage unit 121, a reference information storage unit 122, and a corrective action information storage unit 123. Each storage unit is realized as part of the storage area of ​​the memory 11 and / or the storage 12 in the management server 1.

[0026] The model information storage unit 121 stores information about AI models used by a user organization. For example, the model information storage unit 121 may store basic information about each AI model, such as the name of the AI ​​model, information identifying the cloud platform on which the AI ​​model is deployed, and information used to connect to the AI ​​model (e.g., API endpoint, communication method, etc.), linked to identification information for uniquely identifying the AI ​​model. Furthermore, the information about the AI ​​models stored in the model information storage unit 121 may include configuration information and operation information collected by the model information acquisition unit 101 (described later), linked to the identification information of the AI ​​model.

[0027] The configuration information of an AI model is information indicating settings related to the AI ​​model, and may include, for example, information indicating one or more of the following: public settings, access rights, region, encryption state, and authentication settings of the AI ​​model. The public settings of an AI model are settings that determine whether the AI ​​model is public or private within a user organization and / or outside the user organization. The access rights of an AI model are settings that determine the range of operations that a member or group of members (e.g., a department) in a user organization is allowed to perform on the AI ​​model. The region is a setting that determines the physical region where the AI ​​model is deployed or operated, the data center used in the operation of the AI ​​model, etc. The encryption state setting is a setting that determines whether or not the AI ​​model configuration (parameters, model data such as weight files, input / output data, execution environment, etc.) and / or storage related to the AI ​​model are encrypted, and the encryption method. The authentication setting is a setting that determines whether or not authentication methods such as API keys and OAuth are used, the login control method, etc. In addition to the above, the configuration information of an AI model may also include setting items for each cloud platform, and the setting items included in the configuration information are not necessarily limited.

[0028] The operational information of the AI ​​model is information indicating the usage status of the AI ​​model, and may include, for example, the number of requests to the AI ​​model (the number of processing requests sent to the cloud platform via an API or the like), information indicating usage frequency such as the number of users using the AI ​​model and the number of usage sessions, information indicating usage costs such as monthly usage fees, and information indicating logs of security alerts, anomaly detection, other errors, etc. In addition to the above, the model information storage unit 121 may also store histories such as evaluation results by the audit processing unit 102 (described below) and risk scores calculated by the risk assessment unit 103, linked to the identification information of the AI ​​model.

[0029] The standard information storage unit 122 stores standard information that applies to a user organization. The standard information is information that indicates standards that must be observed when utilizing information assets such as AI models in the user organization, and includes information that indicates standards that must be observed for various setting items related to the AI ​​model. Specifically, the standard information storage unit 122 stores standard information that includes at least one of a security policy and operation rules for the AI ​​model that is applied to the user organization. The standard information is linked to identification information that uniquely identifies the user organization and registered in the standard information storage unit 122. In one user organization, multiple pieces of standard information may be registered for each group within the user organization.

[0030] A "security policy" is a basic policy on information security formulated to protect information assets owned and / or used by a user organization (such as AI models and information related to AI models (knowledge bases, input / output data to AI models)). A security policy defines principles to be followed by the entire organization regarding, for example, information handling and usage environment. A security policy serves as a high-level policy for a user organization and includes highly abstract standards that apply to the entire system. Meanwhile, "operational rules" are norms that translate the security policy into specific operations and set out detailed agreements to be observed in the actual operation of AI models. Operational rules are norms specified in detail by an organization regarding the configuration and / or usage procedures of AI models, such as the method of granting access permissions and the conditions for using AI models. Operational rules function as specific criteria for effectively applying a security policy.

[0031] The corrective action information storage unit 123 stores information on corrective processing for correcting or adjusting the state of the settings (hereinafter referred to as corrective action information) corresponding to various setting items related to the AI ​​model. The corrective action information may include, for example, information on actions to correct the settings of the AI ​​model so that they conform to predetermined standards such as the reference information described above, or may include information on actions that can be referenced to maintain or improve the settings or usage environment of the AI ​​model to a predefined recommended state. Corrective action information is specified for each setting item and is stored in the corrective action information storage unit 123.

[0032] Specifically, the corrective action information storage unit 123 may store, as the corrective action information, a corrective action procedure corresponding to each setting item of the AI ​​model. The corrective action procedure is information indicating an operation flow and a setting change method for correcting a specific setting item to an ideal state, such as a state conforming to reference information. The corrective action procedure may be registered in the form of text data, or may be registered in a format that can be directly displayed or transitioned to in a user interface, such as an HTML document, configuration information of a window or dialog displayed on a UI, or information on a web page that presents the corrective action procedure. The corrective action information may include information indicating a link destination for presenting the corrective action procedure on the display unit of the user terminal 2.

[0033] Furthermore, the corrective action information storage unit 123 may store, as the corrective action information, link information indicating a reference to a setting screen for corrective actions corresponding to each setting item of the AI ​​model. The setting screen is a user interface that can be operated by a user (a member of the user organization) to execute corrective actions. The setting screen may include, for example, a setting screen of a management console provided by a cloud platform, or a management screen built on an internal system of the user organization, and is defined according to the setting item. The link information is information indicating a URL, an identifier, or an internal reference key for identifying such a setting screen, and is used to enable a user to directly access the setting screen.

[0034] In addition to the database described above, the memory unit 120 may also store various setting information used for AI model management by this system, such as prompt templates used for compatibility evaluation described below, risk score calculation rules, prompt templates used for risk score calculation, compatibility evaluation results, risk evaluation results, operational information, and other information related to the AI ​​model, etc., displayed on the user terminal 2.

[0035] The model information acquisition unit 101 executes a process of acquiring configuration information indicating settings related to each AI model from the cloud platform. As described above, the configuration information acquired by the model information acquisition unit 101 includes information indicating one or more settings of public settings, access rights, region, encryption state, and authentication settings, and may also include information indicating various other settings. In addition to the configuration information, the model information acquisition unit 101 may execute a process of acquiring, from the cloud platform, operation information indicating the usage status of each AI model.

[0036] The method for acquiring the configuration information and operation information of an AI model is not necessarily limited, and may be acquired through an API provided by the cloud platform, or may be collected from log data, a metadata base, or the like of the AI ​​model. For example, when using API integration, the model information acquisition unit 101 may call an API provided by the cloud platform to acquire configuration information registered for an AI model deployed on the cloud platform. Similar to the configuration information, the operation information can also be acquired using API integration technology, and in addition to the configuration information and operation information, other information included in the metadata of the AI ​​model, environment setting information of the cloud platform, and the like may be acquired. The process of acquiring the configuration information, etc. may be executed in response to an instruction operation from a user, or may be executed periodically.

[0037] The model information acquisition unit 101 may execute a process of linking the configuration information etc. acquired from the cloud platform with identification information of the AI ​​model and recording the information in the model information storage unit 121. When recording the configuration information, the model information acquisition unit 101 may execute a process of converting the configuration information acquired from the cloud platform into a format that can be used for calculation processing in the management server 1 (for example, a process of unifying the data format, a process of normalizing the setting items in the configuration information, etc.).

[0038] The audit processing unit 102 evaluates the conformance of each AI model with the reference information based on the configuration information of the AI ​​model. Specifically, the audit processing unit 102 compares the configuration information of the AI ​​model with the reference information, which includes at least one of the security policy and operation rules of the AI ​​model applied to the user organization, and executes a process of detecting, as non-conforming items, AI model settings that do not conform to the reference information from among the settings indicated in the configuration information. The audit processing unit 102 compares the configuration information with the reference information to detect improper disclosure settings such as deviation from the disclosure range, region violations, improper encryption settings such as missing encryption, and improper API settings. The audit processing unit 102 may also evaluate compliance with legal restrictions in the region corresponding to the set region. The above-mentioned evaluation targets are merely examples, and the setting items targeted in the conformance evaluation are not necessarily limited.

[0039] The conformance evaluation by the audit processing unit 102 may be performed by theoretical calculation processing based on predefined calculation rules for evaluation, or may be performed using a large-scale language model (LLM). In the latter case, the audit processing unit 102 may provide the LLM with a prompt including configuration information of the AI ​​model and instruction information for conformance evaluation, and execute processing to cause the LLM to detect settings of the AI ​​model that do not conform to the standard information as non-conforming items.

[0040] The instruction information set in the prompt refers to information that specifies the standards, conditions, evaluation method, and output format of the evaluation results for conformance assessment. The instruction information may include standard information such as security policies and operational rules. Evaluation standards for the instruction information may be set by attaching the standard information to the prompt. In this case, the instruction information may be composed of a standard phrase, such as, "List any items in the attached AI model configuration information that do not conform to the security policy or operational rules as non-conformance items." Alternatively, the instruction information may be constructed based on the standard information by specifying a summary of the key points of the standard information, the configuration items to be checked and the standard information standards corresponding to those configuration items, and the conditions for detecting non-conformance items. A prompt template that applies the standard information may be prepared in advance, or the audit processing unit 102 may generate a prompt by incorporating the AI ​​model configuration information into the prompt template, for example, and provide the prompt to the LLM. Alternatively, the audit processing unit 102 may provide the standard information to the LLM and cause the LLM to output a prompt template including instruction information for conformance assessment.

[0041] The audit processing unit 102 performs the above-described conformance evaluation for each AI model and generates a conformance evaluation result for each AI model, indicating the presence or absence of nonconformance items and the detected nonconformance items. The audit processing unit 102 may then execute a process of linking the generated conformance evaluation result to the identification information of the AI ​​model and recording it in the model information storage unit 121. Furthermore, if a nonconformance item is detected in the conformance evaluation, the audit processing unit 102 may output an alert indicating the detection of the nonconformance item to the user terminal 2, or may execute a process of notifying the user of the AI ​​model in which the nonconformance item was detected and the detected nonconformance item via a predetermined means such as email. The conformance evaluation may be triggered by the acquisition of configuration information by the model information acquisition unit 101, or may be executed in response to a user instruction operation regardless of the timing of the acquisition of configuration information, or may be executed periodically according to a predetermined schedule. The timing of the conformance evaluation is not particularly limited. The conformance evaluation may be executed for each AI model used in the user organization at the same time, or at different times.

[0042] The risk assessment unit 103 executes a process of calculating a risk score for each AI model according to the conformance assessment result by the audit processing unit 102. The risk score is information that expresses the degree of risk of the AI ​​model used in the user organization as a numerical value or a graded index (for example, index labels such as "high, medium, low" or "Level AE"). The risk score is calculated based on the presence or absence of nonconformance items indicated by the conformance assessment result and the detected nonconformance items.

[0043] Specifically, the risk assessment unit 103 may calculate a risk score for each AI model by adding points to the score according to non-conformance items detected in the conformance assessment based on predefined risk score calculation rules. The calculation rules are information defining standards for deriving risk scores. The calculation rules may define the setting items to be calculated, may define points to be added for each setting item of the AI ​​model, or may define weighting parameters according to the importance of the setting items. For example, the risk assessment unit 103 may calculate a risk score for each AI model by adding scores corresponding to non-conformance items identified in each AI model based on calculation rules that define standards such as +30 points for an incomplete disclosure setting, +20 points for a region violation, and +50 points for treating no encryption as a major risk.

[0044] The calculation of the risk score may be performed by a theoretical calculation process based on calculation rules, or may be performed using a large-scale language model (LLM). In the latter case, the risk assessment unit 103 may execute a process of providing the LLM with a prompt including the risk score calculation rule and the detected non-conformance items, and causing the LLM to calculate a risk score to be added according to the non-conformance items. In this process, a scoring prompt template with a preset calculation rule may be used, and the risk assessment unit 103 may generate a prompt by attaching information indicating the non-conformance items detected by each AI model to the prompt template.

[0045] The risk assessment unit 103 may execute a process for selecting an AI model to be subjected to corrective action, or may execute a process for determining the priority of AI models, indicating the priority of the corrective action. In these processes, the target model or priority of the corrective action may be determined solely based on the risk score calculated by the scoring described above, or operational information indicating the usage status of the AI ​​model may be referenced along with the risk score. That is, the risk assessment unit 103 may select an AI model to be subjected to corrective action from among multiple AI models based on the risk score and operational information. For example, the risk assessment unit 103 may calculate an evaluation value (e.g., a value obtained by multiplying the risk score by a coefficient corresponding to the usage frequency) that integrates the risk score and operational information based on a selection rule that specifies that a larger coefficient is assigned to an AI model with a higher usage frequency or usage cost, and identify AI models with this evaluation value equal to or greater than a predetermined threshold as targets for corrective action. The risk assessment unit 103 may also determine the priority of AI models, indicating the priority of the corrective action, based on the risk score and operational information. In this case, too, the AI ​​models may be ranked based on the evaluation value that integrates the risk score and operational information.

[0046] The risk assessment unit 103 may execute a process of generating risk assessment result information including at least the calculated risk score for each AI model, linking it to the identification information of the AI ​​model, and recording it in the model information storage unit 121. In addition to the risk score, the risk assessment result information may include information indicating whether or not the model has been selected as a target model for corrective action, information indicating the priority of the corrective action, and the like.

[0047] The display control unit 104 executes processing to control the display of a user interface (UI) that presents information related to AI models. In this system, the form of the UI to be displayed on the user terminal 2 is not necessarily limited. The display control unit 104 may execute processing to output a dashboard that presents multiple AI models used in the user organization together with one or more pieces of information selected from the group consisting of configuration information, compatibility evaluation results, risk scores, and operational information indicating the usage status of the AI ​​models.

[0048] The dashboard output by the display control unit 104 may include, for example, a dashboard (AI model management dashboard) presenting a list of AI models used in the user organization, as shown in FIG. 4. In the example of FIG. 4, non-conformance items (conformance evaluation results) and risk scores are presented in the display column for each AI model. The dashboard may also include a dashboard presenting operational information of the AI ​​model, as shown in FIG. 5. In the example of FIG. 5, the operational information of the AI ​​model is presented by analyzing the usage status of each model according to predetermined aggregation criteria, such as the number of sessions, usage cost, number of users, and most recent usage history. FIGS. 4 and 5 are merely examples, and the configuration of the UI presenting information about the AI ​​model is not necessarily limited. In the present system, multiple dashboards may be output according to the attributes of the information to be presented, and the display control unit 104 may switch between multiple dashboards on the UI of the present system.

[0049] In the UI output shown in FIGS. 4 and 5 , the display control unit 104 may change the display format of each AI model according to the information to be visualized, so as to visualize one or more types of information among the compatibility evaluation results, the risk score, and the operational information. For example, the display control unit 104 may execute a process of determining the display format of information about each AI model in the user interface according to the risk score of each AI model calculated by the risk assessment unit 103. Specifically, the display control unit 104 may execute one or more of the following according to the risk score: color-coding the display corresponding to each AI model; ranking the models to determine the display order of each model in the list display; highlighting the models; and adding a warning mark. Note that highlighting may include, for example, highlighting the AI ​​model by changing the size and shape of the display field, or the size, color, and font of the characters indicating the AI ​​model.

[0050] In the example dashboard shown in Figure 4, AI models are sorted in descending order of risk score, and the color of each AI model's display column changes depending on whether or not there are any non-conformances. Furthermore, AI model A, which was selected as a model to be prioritized for correction, is highlighted in a darker color than the other models and is marked with a flag (warning mark) such as "urgent" indicating that it is a priority target for corrective action. The display format shown in Figure 4 is merely an example, and the visualization method is not limited to the example shown in Figure 4. One or more types of information from the conformance assessment results, risk scores, and operational information may also be visually presented using charts (e.g., bar graphs, radar charts, histograms, heat maps, etc.). By controlling the display format on the dashboard as described above, the risks of multiple AI models deployed in a multi-cloud environment can be visualized and understood in a unified manner.

[0051] When the conformance evaluation results for nonconformance items and the like are presented on the UI, corrective action information corresponding to the nonconformance items may be provided. When providing the corrective action information, the display control unit 104 reads the corrective action information corresponding to the nonconformance items from the corrective action information storage unit 123 and sets an operation area on the UI for providing the corrective action information. Specifically, based on the conformance evaluation results, the display control unit 104 may set a link for displaying corrective action procedures corresponding to the nonconformance items in a display area in the user interface that notifies the user of nonconformance items, which are settings that do not conform to the standard information. Furthermore, based on the conformance evaluation results, the display control unit 104 may set a link to a setting screen for corrective action corresponding to the nonconformance items in a display area in the user interface that notifies the user of nonconformance items, which are settings that do not conform to the standard information.

[0052] For example, in the display column of each AI model as shown in Figure 4, non-conformance items may be presented and a "corrective action" button may be provided. Then, in response to a user's operation on the button, a window showing corrective action procedures may be opened or a transition to a setting screen for executing the corrective action may be made. By setting the corrective action procedures and / or links to the setting screen as described above, it is possible to not only present risks but also to efficiently guide users to take corrective action. Note that when notifying the results of the conformance evaluation by an alert or the like, a link to the corrective action procedures and / or settings screen may be set on the alert notification screen, not limited to the dashboard as shown in Figure 4.

[0053] If the user organization includes multiple groups such as departments, the display control unit 104 may present on the dashboard a summary of risks, analysis results, and usage status for each group based on the evaluation results (non-conformance items and risk scores) and operational information of the AI ​​model used in each group.

[0054] In the above, some or all of the functions of the model information acquisition unit 101, the audit processing unit 102, the risk assessment unit 103, and the display control unit 104 shown as functional units of the management server 1 may be configured to be realized by the processor 20 of the user terminal 2.

[0055] The UI operation reception unit 201 has a function of receiving input operations from the user via various user interfaces (for example, UIs such as those shown in FIGS. 4 and 5 output by the display control unit 104) displayed on the user terminal 2. The UI operation reception unit 201 acquires operation signals via various input means provided on the user terminal 2, such as a touch panel, a mouse, a keyboard, or a pointing device, and executes a process of transmitting the operation signals to the management server 1 via the transmission / reception unit 23.

[0056] <Example of information processing method> Next, an example of an information processing method executed by the information processing system of this embodiment will be described with reference to the flowchart illustrated in Fig. 6. Note that the flowchart illustrated in Fig. 6 is merely an example, and the information processing method according to this embodiment is not limited to this, and some of the processes may be omitted, the order may be changed, or other processes may be added.

[0057] First, the model information acquisition unit 101 of the management server 1 acquires configuration information indicating the settings for each AI model and operational information indicating the usage status of each AI model from the cloud platform (step SQ101). The configuration information may include disclosure settings, access rights, region, encryption status, authentication settings, etc., while the operational information may include the number of requests, number of usage sessions, usage cost, number of users, etc. The model information acquisition unit 101 records the information acquired from the cloud platform in the model information storage unit 121, linking it to the identification information of the AI ​​model.

[0058] Next, the audit processing unit 102 evaluates the conformance of each AI model with the reference information based on the configuration information acquired in step SQ101 (step SQ102). The reference information includes at least one of the security policy and operation rules of the AI ​​model stored in the reference information storage unit 122. The audit processing unit 102 compares the configuration information with the reference information and detects, as non-conforming items, any settings indicated in the configuration information that do not conform to the reference information. The audit processing unit 102 may detect non-conforming items by logical operation processing based on operation rules, or may provide a prompt including configuration information and instruction information to the LLM, causing the LLM to extract non-conforming items.

[0059] Next, the risk assessment unit 103 executes a process of calculating a risk score for each AI model based on the compatibility assessment result in step SQ102 (step SQ103). Specifically, the risk assessment unit 103 calculates the risk score for each AI model by adding points to the score according to the non-conformance items detected in step SQ102 based on predefined calculation rules. The risk score may be calculated using an LLM. In this case, the risk assessment unit 103 provides a prompt including the calculation rule and the non-conformance items to the LLM, causing the LLM to calculate a risk score that is added according to the non-conformance items. Furthermore, the risk assessment unit 103 may select an AI model to be subject to corrective action or determine the priority of the corrective action based on the calculated risk score and operation information. The risk assessment unit 103 associates the risk assessment result for each AI model with the identification information of the AI ​​model and records it in the model information storage unit 121.

[0060] The management server 1 summarizes the conformance evaluation results of step SQ102 and the risk evaluation results of step SQ103 on a predetermined UI screen, such as the dashboard format shown in FIG. 4, and outputs the results to the user terminal 2 (step SQ104). At this time, the display control unit 104 controls the display mode of the AI ​​models in the user interface based on the calculated risk scores and conformance evaluation results. Specifically, the display of each AI model is presented according to the risk score by color coding, ranking, highlighting, or adding a warning mark. Furthermore, if a nonconformity item is detected, a link to a corrective action procedure or setting screen corresponding to the nonconformity item is set on the UI, allowing the user to perform corrective action by operating the link.

[0061] In the information processing system of this embodiment, by performing the above-described information processing and calculating the risk score of each AI model, it is possible to centrally audit AI models that are distributed across multiple cloud platforms in a multi-cloud environment and comprehensively grasp individual configuration deficiencies and security risks for each model.

[0062] In addition, by comparing configuration information with standard information and identifying non-conforming items, it is possible to clarify the location of risks and make it easier to implement specific improvement measures. Furthermore, by utilizing large-scale language models to perform audit processing and / or risk score calculation, these processes can be made more efficient with a certain degree of accuracy. In addition, by combining risk scores and operational information to select or prioritize models for corrective action, it is possible to appropriately identify AI models that need to be addressed, taking into account their operational importance, and then take efficient measures.

[0063] Furthermore, in the information processing system of this embodiment, dynamic display control according to the risk score is performed in the user interface, allowing high-risk models to be intuitively grasped. In addition, by presenting the risk assessment results together with links to corrective action procedures and / or setting screens corresponding to nonconformance items, the flow from detection to correction can be seamlessly linked, enabling prompt corrective action.

[0064] The above-described embodiments are merely examples for facilitating understanding of the present disclosure, and are not intended to limit the present disclosure. The present disclosure can be modified or improved without departing from the spirit thereof, and it goes without saying that the present disclosure includes equivalents thereof.

[0065] For example, the audit processing unit 102 performs a conformance assessment by comparing the configuration information with the reference information. In addition, the audit processing unit 102 may compare the change history of the configuration information over time and extract changed setting items based on the difference between before and after the change. The audit processing unit 102 may then assess the conformance of the changed setting items with the reference information. By auditing change points in this way, risk factors that occur during operation can be efficiently detected.

[0066] Although the risk score calculated by the risk assessment unit 103 is exemplified as a numerical score, it may be configured to be converted into a qualitative assessment label and presented. For example, by managing labels such as "Do not use AI model," "Caution required," and "Safe" in association with risk score intervals, it is possible to present information that is intuitively easy for users to understand. Furthermore, the dashboard presented by the display control unit 104 is not limited to the UI of the user terminal 2, and may be sent to an external system as a notification message or report. For example, by generating a periodic report containing risk scores and non-conformance items and sending it to an email or chat system, users can grasp the risk situation without opening the management screen.

[0067] The corrective action information stored in the corrective action information storage unit 123 is not limited to procedure information and link information, but may also include automation scripts, parameter sets for API calls, and the like. This may enable semi-autonomous operation in which the system automatically executes corrective actions for detected nonconformities without user intervention. Furthermore, the standard information registered in the standard information storage unit 122 is not limited to security policies and operational rules, but may also include external standards based on industry standards and laws and regulations. In this case, consistency with both internal organizational standards and external standards may be evaluated simultaneously, thereby ensuring compliance with not only internal organizational standards but also legal compliance and industry standards.

[0068] Furthermore, the series of processes performed by the information processing system described herein may be implemented using software, hardware, or a combination of software and hardware. A computer program for implementing each function of the management server 1 according to this embodiment may be created and installed on a PC or the like. A computer-readable recording medium on which such a computer program is stored may also be provided. Examples of the recording medium include a magnetic disk, an optical disk, a magneto-optical disk, and a flash memory. Furthermore, the computer program may be distributed, for example, via a network, without using a recording medium.

[0069] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0070] The information processing system, information processing method, and program of the present disclosure may have the following configuration. [Item 1] An information processing system that manages multiple AI models deployed on at least two or more cloud platforms, a standard information storage unit that stores standard information including at least one of a security policy and an operation rule of the AI ​​model that is applied to a user organization; a model information acquisition unit that acquires configuration information indicating settings for each of the AI ​​models from the cloud platform; an audit processing unit that evaluates the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; An information processing system comprising: a risk assessment unit that calculates a risk score for each AI model based on the compatibility assessment result. [Item 2] Item 1. The information processing system according to item 1, wherein the audit processing unit compares the configuration information of the AI ​​model with the standard information and detects settings of the AI ​​model that do not conform to the standard information as non-compliant items. [Item 3] The information processing system described in item 1, wherein the audit processing unit provides a prompt to the LLM that includes the configuration information of the AI ​​model and instruction information for evaluating the conformance, and executes a process to cause the LLM to detect settings of the AI ​​model that do not conform to the standard information as non-conforming items. [Item 4] The information processing system described in any one of items 1 to 3, wherein the configuration information includes information indicating one or more settings of the AI ​​model's public settings, access rights, region, encryption status, and authentication settings. [Item 5] 4. The information processing system according to item 2 or 3, wherein the risk assessment unit calculates a risk score for each AI model by adding points according to the detected non-conformance items based on predefined risk score calculation rules. [Item 6] The information processing system described in item 2 or 3, wherein the risk assessment unit provides a prompt to the LLM including the calculation rule for the risk score and the detected non-compliant items, and executes a process to have the LLM calculate the risk score to which points are added according to the non-compliant items. [Item 7] The model information acquisition unit acquires operational information indicating a usage status of each of the AI ​​models from the cloud platform, Item 2. The information processing system according to item 1, wherein the risk assessment unit selects an AI model to be subjected to corrective action from among the plurality of AI models based on the risk score and the operational information. [Item 8] The model information acquisition unit acquires operational information indicating a usage status of each of the AI ​​models from the cloud platform, Item 1. The information processing system according to item 1, wherein the risk assessment unit determines a priority of the AI ​​model indicating the priority of corrective action based on the risk score and the operational information. [Item 9] a display control unit that controls the display of a user interface that presents information about the AI ​​model; Item 1. The information processing system according to item 1, wherein the display control unit determines a display format of information relating to each of the AI ​​models in the user interface according to the risk score. [Item 10] Item 10. The information processing system of item 9, wherein the display control unit performs one or more of color coding, ranking, highlighting, and adding a warning mark to the display corresponding to each of the AI ​​models according to the risk score. [Item 11] a corrective action information storage unit that stores corrective action procedures corresponding to each setting item of the AI ​​model; a display control unit that controls the display of a user interface that presents information about the AI ​​model, The information processing system described in item 1, wherein the display control unit sets a link in a display area in the user interface that notifies non-compliant items, which are settings that do not comply with the standard information, based on the results of the conformity evaluation, to display the corrective action procedure corresponding to the non-compliant item. [Item 12] a corrective action information storage unit that stores link information indicating a reference destination to a setting screen for corrective action corresponding to each setting item of the AI ​​model; a display control unit that controls the display of a user interface that presents information about the AI ​​model, The information processing system described in item 1, wherein the display control unit sets a link to the setting screen for the corrective action corresponding to the non-conforming item in a display area in the user interface that notifies of non-conforming items, which are settings that do not conform to the standard information, based on the conformity evaluation results. [Item 13] a display control unit that controls the display of a user interface that presents information about the AI ​​model; Item 1. The information processing system according to item 1, wherein the display control unit outputs a dashboard that presents the plurality of AI models used in the user organization together with one or more of the configuration information, the compatibility evaluation results, the risk score, and operational information indicating the usage status of the AI ​​models. [Item 14] An information processing method for managing multiple AI models deployed on at least two or more cloud platforms, Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; obtaining configuration information from the cloud platform indicating settings for each of the AI ​​models; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; and calculating a risk score for each of the AI ​​models according to the results of the suitability evaluation. [Item 15] A program for causing a computer to execute an information processing method related to the management of multiple AI models deployed on at least two or more cloud platforms, Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; obtaining configuration information from the cloud platform indicating settings for each of the AI ​​models; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; and calculating a risk score for each AI model according to the results of the suitability evaluation. [Explanation of symbols]

[0071] 1 Management Server 122 Standard information storage unit 101 Model information acquisition unit 102 Audit Processing Unit 103 Risk Assessment Department

Claims

1. An information processing system that manages a plurality of AI models deployed on at least two or more cloud platforms, a standard information storage unit that stores standard information including at least one of a security policy and an operation rule of the AI ​​model that is applied to a user organization; a model information acquisition unit that acquires configuration information indicating settings related to each of the AI ​​models from the cloud platform; an audit processing unit that evaluates the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; a risk assessment unit that calculates a risk score for each of the AI ​​models according to the evaluation result of the suitability, The model information acquisition unit acquires operational information indicating the usage status of each of the AI ​​models from the cloud platform, The risk assessment unit selects an AI model to be subjected to corrective action from among the plurality of AI models based on the risk score and the operational information.

2. An information processing system that manages multiple AI models deployed on at least two or more cloud platforms, a standard information storage unit that stores standard information including at least one of a security policy and an operation rule of the AI ​​model that is applied to a user organization; a model information acquisition unit that acquires configuration information indicating settings related to each of the AI ​​models from the cloud platform; an audit processing unit that evaluates the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; a risk assessment unit that calculates a risk score for each of the AI ​​models according to the evaluation result of the suitability, The model information acquisition unit acquires operational information indicating the usage status of each of the AI ​​models from the cloud platform, The risk assessment unit determines the priority of the AI ​​model indicating the priority of corrective action based on the risk score and the operational information.

3. The information processing system according to claim 1 or 2, wherein the inspection processing unit compares the configuration information of the AI ​​model with the standard information and detects settings of the AI ​​model that do not conform to the standard information as non-conforming items.

4. 3. The information processing system according to claim 1, wherein the audit processing unit provides a prompt to an LLM including the configuration information of the AI ​​model and instruction information for evaluating the conformance, and executes a process to cause the LLM to detect settings of the AI ​​model that do not conform to the standard information as non-conforming items.

5. The information processing system according to claim 1 or 2, wherein the configuration information includes information indicating one or more settings of a public setting, an access right, a region, an encryption state, and an authentication setting of the AI ​​model.

6. 4. The information processing system according to claim 3, wherein the risk assessment unit calculates a risk score for each AI model by adding points according to the detected non-conformance items based on predetermined risk score calculation rules.

7. The information processing system described in Claim 4, wherein the risk assessment unit calculates a risk score for each AI model by adding points according to the detected non-conforming items based on pre-defined risk score calculation rules.

8. The information processing system of claim 3, wherein the risk assessment unit provides a prompt to an LLM including a calculation rule for the risk score and the detected non-compliant items, and executes a process to have the LLM calculate the risk score to which points are added according to the non-compliant items.

9. The information processing system described in Claim 4, wherein the risk assessment unit provides the LLM with a prompt including the calculation rule for the risk score and the detected non-compliant items, and performs a process to have the LLM calculate the risk score to which points are added according to the non-compliant items.

10. A display control unit that controls the display of a user interface that presents information about the AI ​​model, The information processing system according to claim 1 or 2, wherein the display control unit determines a display mode of information regarding each of the AI ​​models in the user interface according to the risk score.

11. 11. The information processing system according to claim 10, wherein the display control unit performs one or more of color coding, ranking, highlighting, and adding a warning mark to a display corresponding to each of the AI ​​models according to the risk score.

12. a corrective action information storage unit that stores corrective action procedures corresponding to each setting item of the AI ​​model; A display control unit that controls the display of a user interface that presents information about the AI ​​model, The information processing system of claim 1 or 2, wherein the display control unit sets a link in a display area in the user interface that notifies non-compliant items, which are settings that do not conform to the standard information, based on the results of the conformity evaluation, to display the corrective action procedure corresponding to the non-compliant item.

13. a corrective action information storage unit that stores link information indicating a reference destination to a setting screen for corrective action corresponding to each setting item of the AI ​​model; A display control unit that controls the display of a user interface that presents information about the AI ​​model, The information processing system of claim 1 or 2, wherein the display control unit sets a link to the setting screen for the corrective action corresponding to the non-conforming item in a display area in the user interface that notifies of non-conforming items, which are settings that do not conform to the standard information, based on the conformity evaluation results.

14. A display control unit that controls the display of a user interface that presents information about the AI ​​model, 3. The information processing system according to claim 1, wherein the display control unit outputs a dashboard that presents the plurality of AI models used in the user organization together with one or more pieces of information selected from the configuration information, the suitability evaluation results, the risk scores, and operational information indicating the usage status of the AI ​​models.

15. An information processing method for managing a plurality of AI models deployed on at least two or more cloud platforms, comprising: Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; Acquiring configuration information indicating settings for each of the AI ​​models and operational information indicating a usage status of each of the AI ​​models from the cloud platform; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; Calculating a risk score for each of the AI ​​models according to the suitability evaluation results; and selecting an AI model to be subjected to corrective action from among the plurality of AI models based on the risk score and the operational information.

16. An information processing method for managing multiple AI models deployed on at least two or more cloud platforms, comprising: Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; Acquiring configuration information indicating settings for each of the AI ​​models and operational information indicating a usage status of each of the AI ​​models from the cloud platform; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; Calculating a risk score for each of the AI ​​models according to the suitability evaluation results; and determining a priority of the AI ​​model indicating a priority of corrective action based on the risk score and the operational information.

17. A program for causing a computer to execute an information processing method related to management of multiple AI models deployed on at least two or more cloud platforms, Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; Acquiring configuration information indicating settings for each of the AI ​​models and operational information indicating a usage status of each of the AI ​​models from the cloud platform; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; Calculating a risk score for each of the AI ​​models according to the suitability evaluation results; and selecting an AI model to be subjected to corrective action from among the plurality of AI models based on the risk score and the operational information.

18. A program for causing a computer to execute an information processing method relating to management of multiple AI models deployed on at least two or more cloud platforms, Storing standard information including at least one of a security policy and an operational rule of the AI ​​model applied to a user organization; Acquiring configuration information indicating settings for each of the AI ​​models and operational information indicating a usage status of each of the AI ​​models from the cloud platform; Evaluating the conformity of the AI ​​model with the reference information based on the configuration information of each of the AI ​​models; Calculating a risk score for each of the AI ​​models according to the suitability evaluation results; and determining a priority of the AI ​​model indicating a priority of corrective action based on the risk score and the operational information.

Images