Information processing system, information processing method, and program

Abstract

Only AI content deemed useful can be published on the website, effectively promoting the sharing of knowledge through AI content. [Solution] The information processing system 1 comprises an information processing device 100 that provides a generation AI service to user A, and a web server 20 accessible to user C. The information processing device 100 receives a request from user A, obtains a response to the request using the generation AI provided by the generation AI service, stores the request and response in association, determines the usefulness of the response, generates web page content from the response based on the determination result, and uploads the content to the web server 20.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Among the websites provided through the Internet, there are useful websites created by experts, and there are also websites of unknown origin that are useless, and they are a mixture of good and bad. In recent years, there is a technology that uses generative AI to assist users in creating websites (see, for example, Patent Document 1).

[0003] Patent Document 1 discloses a technology in which text is input into generative AI to obtain source code, and after the user checks a preview screen of a website based on the source code, it is published on the Internet through a web server.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the technology described in Patent Document 1 above is to judge the quality of the source code generated by the user who participated in the creation of the source code using generative AI by checking the preview screen of the website, and does not judge the usefulness of AI products.

[0006] The present invention has been made in consideration of such circumstances, and an object thereof is to provide an information processing system, an information processing method, and a program that can publish only AI content judged to be useful on a website and can suitably promote the sharing of knowledge by AI content. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides an information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who is not receiving the generation AI service, wherein the first device comprises: a reception unit that receives requests from the first user; a response acquisition unit that obtains a response to the request using the generation AI provided by the generation AI service; a storage unit that stores the request and the response in association; a determination unit that determines the usefulness of the response; a content generation unit that generates web page content from the response based on the determination result of the determination unit; and an upload unit that uploads the content to the second device.

[0008] Furthermore, in order to solve the above problems, the present invention provides an information processing method in an information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who is not receiving the generation AI service, wherein the first device includes: a reception step of receiving a request from the first user; a response acquisition step of obtaining a response to the request using the generation AI provided by the generation AI service; a storage step of storing the request and the response in association; a judgment step of determining the usefulness of the response; a content generation step of generating web page content from the response based on the judgment result of the judgment step; and an upload step of uploading the content to the second device.

[0009] Furthermore, in order to solve the above problems, the present invention provides a computer-readable program for execution on an information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who does not receive the generation AI service, characterized in that the first device is made to execute: a reception step of receiving a request from the first user; a response acquisition step of obtaining a response to the request using the generation AI provided by the generation AI service; a storage step of storing the request and the response in association; a judgment step of determining the usefulness of the response; a content generation step of generating web page content from the response based on the judgment result of the judgment step; and an upload step of uploading the content to the second device. [Effects of the Invention]

[0010] According to the present invention, only AI content deemed useful can be published on a website, effectively promoting the sharing of knowledge through AI content. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of an information processing system relating to one embodiment of the present invention. [Figure 2] This is a block diagram showing the configuration of an information processing server related to one embodiment of the present invention. [Figure 3] This table schematically shows an example of the configuration of a database in which the storage unit stores requests and responses in correspondence in an information processing device according to one embodiment of the present invention. [Figure 4] This table schematically shows an example of a database configuration in an information processing device according to one embodiment of the present invention, in which the storage unit stores requests and responses in correspondence, as well as user actions, etc. [Figure 5]This figure shows an example of the hardware configuration of an information processing server according to one embodiment of the present invention. [Figure 6] This flowchart illustrates the AI ​​content generation process procedure in an information processing device according to one embodiment of the present invention. [Figure 7] This flowchart illustrates the web page generation process procedure in an information processing device according to one embodiment of the present invention. [Figure 8] This flowchart illustrates the procedure for publishing a web page in an information processing device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0013] In the following descriptions, unless otherwise specified, each component of each device / system refers to a functional block, not a hardware-level configuration. Each component of each device / system is realized through any combination of hardware and software, centered around the CPU, memory, programs loaded into memory, storage media such as a hard disk that stores those programs, and a network connection interface of any computer. Furthermore, there are various variations in the implementation methods and devices.

[0014] <Configuration of Information Processing System 1> Figure 1 is a block diagram showing the configuration of an information processing system 1 according to one embodiment of the present invention. As shown in Figure 1, the information processing system 1 has a function that enables a user A to provide AI content generated using a generation AI, along with an input request, to another user B within a system 10 that provides generation AI services via cloud services such as SaaS (referred to as a "generation AI service provision system").

[0015] The generation AI service providing system 10 includes an information processing device 100, a user terminal 200, a generation AI 300, and a user terminal 400, which are connected to each other via the Internet N1. In the generation AI service providing system 10, users such as individuals and companies receive the issuance of identification information (such as user IDs) and passwords through a contract with a service provider that provides this generation AI service, and can use the generation AI service. Although each component device of the generation AI service providing system 10 is directly connected to the Internet N1, only contract users can access and use the service, so it is a system with a closed environment.

[0016] The above-mentioned generation AI service is a service that can provide appropriate AI content for the content of a request (prompt) by allowing a user to select and use multiple types of generation AI within a closed environment. For example, in the generation AI service providing system 10, a user A who is an individual user or an employee user of a corporate contract uses the user terminal 200 to create articles, reports, and business reports by using the generation AI service provided by the information processing device 100 using the generation AI 300. And other users B other than user A can also view the AI content such as the created articles, reports, and business reports using the user terminal 400.

[0017] The information processing device 100 provides the generation AI service to the users of the generation AI service providing system 10, and is a computer or server that uses the generation AI 300 to generate AI content based on a prompt, which is the text input by user A using the user terminal 200.

[0018] The user terminal 200 is a computer such as a smartphone, a tablet terminal, a notebook PC, or a desktop PC for inputting a prompt for user A to generate AI content using the generation AI 300 and outputting and displaying the AI content.

[0019] The generative AI 300 is a generative AI that utilizes an LLM to generate AI content based on the prompts input by User A. In this embodiment, examples of the LLM used in the generative AI 300 include GPT-3.5, GPT-4, GPT-4 Turbo, GPT-4o related to "GPT (registered trademark)" of OpenAI, and Claude 3, Claude 3 Haike, Claude 3 Sonnet, Claude 3 Opus related to "CLAUDE (registered trademark)" of ANTHROPIC, etc.

[0020] Note that in this embodiment, the generative AI 300 may be equipped with multiple types of generative AI, and the generative AI to be used may be switched. For example, the user may be allowed to select which generative AI to use in consideration of the content of the request input by the user.

[0021] The LLM is trained from a large amount of text data and is used as a generative AI that generates text for a specific prompt. In this embodiment, various LLMs are also used in the generative AI 300, but the application of the present invention is not limited to the LLM for the generative AI. For example, in addition to the LLM as the generative AI, GANs (Generative Adversarial Networks) which is an image generation AI, a music generation AI, a video generation AI, or a language model corresponding to a multi-modal that can understand and generate multiple different types of data such as text, images, and voices may be used. The language model corresponding to the multi-modal is, for example, called MMLLM (Multi Modal Large Language Model) or a multi-modal model. The language model corresponding to the multi-modal may generate an answer based on text data or may generate an answer including image data or voice data.

[0022] In this embodiment, the generating AI 300 can generate AI content by referring to a large amount of data (typically text data) called a training dataset, which is pre-built by collecting data from various sources such as web pages, books, articles, and papers. Here, the training dataset provides the generating AI with knowledge about specific tasks and serves as the foundation for the generating AI to produce high-quality responses and efficient task processing. For example, when an employee creates a proposal document for their company's products to a client, information such as the company's homepage and product catalogs is prepared as the training dataset, and the generating AI can be expected to produce a high-quality proposal document by generating appropriate responses and content based on this information.

[0023] Furthermore, the information processing device 100 may incorporate the generating AI 300 into its internal structure, so that the information processing device 100 itself functions as the generating AI. Furthermore, the generating AI 300 may have an API (Application Programming Interface) for enabling a predetermined LLM to function in the information processing device 100. In this case, the LLM may be configured to function as a generating AI by cooperating with an external device.

[0024] User terminal 400 is a computer such as a smartphone, tablet, notebook PC, or desktop PC used by user B, who is different from user A, to display and view AI content generated by user A using generation AI 300. It is also possible for User B to input a prompt (request) into User Terminal 400 and generate AI content using Generation AI 300, and for User A to view that AI content using User Terminal 200, but since the configuration and processing procedure are the same in that case as well, the explanation is omitted in this embodiment.

[0025] Internet N1 refers to wired or wireless communication networks, also known as public networks or public telecommunications networks, that are built outside of a company, and include communication over 4G or 5G lines.

[0026] The web server 20 is connected to the internet N1 and is a public server that publishes the AI ​​content generated by the AI ​​generation service provision system 10 onto the internet N1, allowing general users C outside the closed environment of the AI ​​generation service provision system 10 to view web pages based on the AI ​​content using user terminal T1.

[0027] User terminal T1 is a smartphone, tablet, notebook PC, or desktop PC, etc., used by a regular user C to view web pages based on AI content provided by the web server 20 on the open environment of the internet N1.

[0028] <Configuration of the information processing device 100> Figure 2 is a block diagram showing the configuration of an information processing device 100 according to one embodiment of the present invention. The information processing device 100 functions as an information processing server that provides generated AI services to users in the generated AI service provision system 10. As shown in Figure 2, the information processing device 100 includes a reception unit 101, a response acquisition unit 102, a storage unit 103, a judgment unit 104, a content generation unit 105, an upload unit 106, an action acquisition unit 107, a collection unit 108, an evaluation criterion revision unit 109, a confidential information determination unit 110, a specification unit 111, a masking unit 112, and a display unit 113.

[0029] The reception unit 101 receives a request, which is a prompt for the generating AI, entered by the first user, User A, via the user terminal 200.

[0030] The response acquisition unit 102 acquires the response, which is AI content generated by the generation AI 300 in response to the request received by the reception unit 101.

[0031] The memory unit 103 stores the request entered by the first user and the response generated by the generation AI 300 in association with that request. Figure 3 is a schematic table showing an example of the configuration of a database in which the storage unit 103 stores requests and responses in correspondence in an information processing device 100 according to one embodiment of the present invention. For example, in this embodiment, as shown in Figure 3, the content of the request 1032, which is a prompt, and the content of the response 1033, which is AI content generated by inputting the request to the request generation AI 300, are associated with a user 1031, who can be identified by a user ID, and stored in the database.

[0032] As shown in Figure 1, the generated AI service provided by the information processing device 100 in the generated AI service provision system 10 can be enjoyed by user A using user terminal 200, and similarly by another user B using user terminal 400. First, the response obtained when user A inputs a request is stored in the storage unit 103, and even after user A inputs another request, the contents of that response can be displayed on the user terminal 200 by the display unit 113 as part of the past generation history. Furthermore, the display unit 113 of the information processing device 100 has the function of displaying requests and responses from user A (first user) stored in the storage unit 103 on the user terminal 400 of user B (third user), who is different from user A. User B can then view these responses displayed on the user terminal 400, and if they like them, export them to use in creating their own requests.

[0033] The judgment unit 104 determines whether the content of the response generated by the generation AI 300 is useful or not. In this embodiment, if the content of the AI ​​content generated by the generation AI 300 is deemed to provide information useful to the user, the judgment unit 104 determines that the content is "highly useful". Specifically, the judgment of usefulness by the judgment unit 104 includes two types of judgments: one in which a human is involved and one in which a human is not involved.

[0034] Human-involved decisions are those that involve user reactions to content, and include, for example, the following: 1. User Feedback The system uses an evaluation mechanism that allows users to rate the content of the generated AI-generated content. For example, this could include star ratings or the ability to click a "like" button (also known as a "like" button or "agree" button). It also includes comment and review functions to allow users to provide specific feedback. The system automatically collects user evaluation metrics for the generated AI content (e.g., the number of clicks on the "like" button, average time spent on the content, etc.) and determines its usefulness based on these metrics. For example, the "Goods" metric counts the number of clicks on the "Good" button to indicate how many positive ratings ("Goods") the generated AI content has received. The average time spent measures how much time other users spend viewing the generated AI content, and the average time is used to determine its usefulness. It should be noted that the user feedback mentioned above does not only include feedback from other users who were not involved in the generation of the AI ​​content, but also feedback from users who were involved in the generation of the AI ​​content themselves.

[0035] 2. Frequency of use We determine that a particular generated result is useful if it is used by many users (regardless of whether they are other users). For example, we use engagement metrics to analyze data such as how much time users spend on the generated AI content and how much that AI content is shared.

[0036] 3. Similarity Matching We check whether highly-rated AI content is reused when other users enter similar questions or prompts. For example, we use topic modeling to evaluate how relevant the generated content is to a particular topic.

[0037] 4. Engagement The system tracks how often the generated AI content is reused, either by itself or by other users, and determines its usefulness if the reuse frequency is high. It also automatically measures how many times the generated AI content is shared with other users or bookmarked, and determines its usefulness if these figures are high compared to thresholds. Furthermore, the AI-generated content is evaluated based on the number of followers and retweets it receives on internal social media platforms, and if the results are high compared to a threshold, it is deemed useful.

[0038] 5. Expert Review Experts in various fields review the content of the AI-generated content and evaluate its usefulness.

[0039] 6. Crowdsourcing In addition to internal evaluations, we will use a crowdsourcing platform to gather feedback from many users, thereby assessing the general usefulness of the AI ​​content.

[0040] While the judgments described above can be made using any one of the judgment methods, combining multiple judgment methods makes it possible to more accurately determine whether the results (AI content) generated by the generating AI are useful.

[0041] Next, examples of decisions that do not involve human intervention include the following: 1. Machine Learning Models (1) Quality evaluation model We will build a quality evaluation model that uses natural language processing (NLP) techniques to evaluate the quality of generated AI content (generated results). For example, the following evaluation points will be used. • Grammatical accuracy: Is the generated result grammatically correct? • Contextual appropriateness: Does the generated result fit the context? • Semantic consistency: Is the generated result semantically consistent? (2) Recommendation algorithm The algorithm uses past user behavior data to predict how useful newly generated AI content will be to other users.

[0042] 2. Contextual evaluation (1) Similarity assessment An algorithm is used to evaluate how similar the generated AI content is to other highly-rated results, and it is deemed useful if there are many similar highly-rated results. (2) Topic Modeling The generated AI content is evaluated for its relevance to a specific topic, and if the topic is useful to many users, the generated result is deemed useful.

[0043] 3. Automated evaluation metrics (1) BLEU (BiLingual Evaluation Understudy) score The system automatically evaluates the quality of generated AI content using BLEU scores, which are used to assess the quality of machine translation and text generation. (2) ROUGE (Recall-Oriented Understudy for Gisting Evaluation) score The quality of the generated AI content is evaluated using the ROUGE score, which is used to assess the quality of summarization in natural language processing.

[0044] While the judgments described above can be made using any one of the judgment methods, combining multiple judgment methods makes it possible to determine the usefulness of AI content automatically generated by software without human evaluation, that is, without human subjectivity. Furthermore, by using a complex evaluation criterion that combines the aforementioned judgments based on frequency of use mediated by human judgment with judgments based on behavioral data that do not involve human judgment, it becomes possible to leverage the strengths of each judgment method and make more accurate judgments about usefulness. Furthermore, the judgment unit 104 stores information regarding the "evaluation criteria" that serve as the basis for judging usefulness as described above, for example, in an internal database of the judgment unit 104, and compares it with information regarding the user's actions in response to the response to determine usefulness.

[0045] As described above, as a decision made without user involvement, the decision unit 104 within the information processing device 100, when the response is text, analyzes the inherent quality of the response using, for example, natural language processing (NLP) to determine the usefulness of the response. Here, inherent quality refers to the quality of the content of the response generated by the generative AI, which is not determined by human judgment, but rather is automatically evaluated by the aforementioned machine learning model and automated evaluation metrics without human involvement, from the perspective of whether the expression is appropriate like a human-written text, whether it is natural, whether it is easy to understand, or to what extent the response (text) reflects the bias of the training data and algorithm.

[0046] For example, the judgment unit 104 uses a character recognition algorithm to recognize text data in the response, extracts specific words from the recognized text data, and processes the data using an NLP algorithm that tags and reorders each word, thereby determining, for example, whether the grammar used in the response is correct and determining its usefulness. Furthermore, if the response is text, the determination unit 104 can also evaluate the inherent quality of the response and determine its usefulness using a large-scale language model (LLM), which is a model that learns linguistic patterns and structures from the text. Such an LLM can be provided by the generating AI 300 of the generating AI service provision system 10 or other generating AIs not shown.

[0047] The content generation unit 105 generates web page content (e.g., source code, etc.) from the AI ​​content (response) based on the "usefulness" judgment result from the judgment unit 104. The content generation unit 105 may perform the following processes, for example. (1) Analysis process: Analyze the text data and understand its structure. (2) Conversion process: Based on the analysis results, the text data is converted into source code. Specifically, certain patterns and phrases in the text data are converted into the syntax and instructions of the corresponding programming language. (3) Generation process: Output the converted source code as a file. To perform this series of processes, the content generation unit 105 has the function of deeply understanding the syntax and semantics of the target programming language and applying them to text data. Furthermore, when the content generation unit 105 performs the above processing, a user (for example, an administrator) within the generation AI service provision system 10 may interactively assist with some of the processing.

[0048] Furthermore, the content generation unit 105 may include in the generated content link information for a general user C (second user) who is outside the closed environment of the generation AI service provision system 10, which operates the user terminal T1 on the internet N1, to access the generation AI service provided by the information processing device 100 (first device). This makes it possible to guide users who are not receiving or are unaware of the existence of the generation AI service provided by the information processing device 100 to receive the said generation AI service.

[0049] The upload unit 106 uploads the content generated by the content generation unit 105 to a second device on the Internet N1, which is a web server 20, or to cloud storage where the web server 20's public files are stored.

[0050] As described above, an information processing system 1, which is one embodiment of the present invention, comprises an information processing device 100 (first device) that provides generated AI services to user A (first user), and a web server 20 (second device) that functions as a web server accessible to user C (second user). The information processing device 100 (first device) includes a reception unit 101 that receives requests from the first user, a response acquisition unit 102 that acquires responses to requests using the generation AI 300 provided by the generation AI service, a storage unit 103 that stores requests and responses in association, a judgment unit 104 that determines the usefulness of the response, a content generation unit 105 that generates web page content from the response based on the judgment result of the judgment unit 104, and an upload unit 106 that uploads the content to the web server 20 (second device). With this configuration, the content (request) that user A queries to the generating AI from user terminal 200 and the AI's generated response (AI content) are converted into a web page, and user C can view the uploaded page from user terminal T1. This allows the content of queries to the generating AI and the answers to be shared as knowledge among multiple users, promoting knowledge sharing among many users.

[0051] As described above, in the information processing device 100 (first device), which is one embodiment of the present invention, the content generation unit 105 includes link information in the content for user C (second user) to access the generation AI service provided by the information processing device 100 (first device). This makes it possible to inform users (User C) outside the generation AI service provision system 10 who are not currently using the generation AI service provided by the information processing device 100 about the generation AI service provided by the information processing device 100, and also allows them to easily access sites where they can receive the generation AI service (for example, product introduction sites or application sites) from the link information.

[0052] The action acquisition unit 107 acquires information regarding the actions of user B (third user) on user terminal 400 in response to the response acquired by the response acquisition unit 102. Here, "action" refers to the user's reaction to the response. "Information regarding the action" can be acquired through user operations on the user terminal (e.g., clicks) or viewing of the response, as shown below.

[0053] For example, the following can be used as an example of User B's "actions" and "information about those actions." (1) Copy: The act of copying the response to the clipboard. The number of copies made (copy count) becomes information related to the action. (2) Export: The act of saving the response as an external file (e.g., a text file or CSV file). The number of times it has been exported (export count) is information about the action. (3) Pressing the Good button: This refers to pressing the "Like!" button or the "I agree" button, which indicates agreement with or support for the content of the response. The number of times the button is pressed (number of goods) is used as information about the action. (4) Pressing the Bad Button: This is the act of pressing the "Disagree" button, indicating that the user does not agree with or support the content of the response. The number of times this button is pressed (number of Bads) is used as information about the action. (5) Posting a comment: The act of posting a comment on the content. For example, the number of comments (number of comments) is information about the action. (6) Impression: The content of the response is displayed on the user's device. The number of times the response is displayed on the devices of multiple users (number of impressions) is considered information about the action. Note that displays on the user's own device that were involved in creating the response that are subject to impression counting may not be counted. (7) Rating: The act of assigning a rating to content, for example, on a 5-point scale. The average rating for the content in question becomes information about the action. (8) Browsing: The act of a user viewing a page or other content. The number of times a user views a page (page views) is considered information about the action. (9) Viewing time (time spent): The time a user spends viewing a page or other content, i.e., the time they spend on that page or other content. The average viewing time (average time spent) is used to provide information about the user's actions.

[0054] Information regarding the actions of users A and B in the generation AI service provision system 10 described above is stored, for example, in a database managed by the storage unit 103, as shown in Figure 3. Specifically, as shown in Figure 3, a table is stored that associates the content of the prompt request 1032 with the content of the response 1033, which is AI content generated by inputting the request to the generation AI 300, for each user 1031, who can be identified by a user ID. Furthermore, information regarding the user's actions (user actions) 1034 for each response is stored for each type of action. For example, as shown in Figure 3, the actions by user B exemplified in (1) through (9) above are stored in the database as the number of copies, number of exports, number of likes, number of dislikes, number of comments, number of impressions, average rating, number of page views, and average time spent, as shown in user action 1034 in Figure 3.

[0055] As described above, in an information processing system 1 which is one embodiment of the present invention, the information processing device 100 (first device) further includes a display unit 113 that displays requests and responses from user A (first user) stored in the storage unit 103 on the user terminal 400 of user A or another user B (third user), and an action acquisition unit 107 that acquires information about the actions of user A or user B in the generation AI service provision system 10 in response to the response. At this time, the storage unit 103 stores information about the actions of user A or user B in association with the requests and responses. The judgment unit 104 compares the information about the actions of user A or user B in response to the response stored in the storage unit 103 with evaluation criteria stored in a database stored inside the judgment unit 104 to determine the usefulness of the response.

[0056] The collection unit 108 collects information regarding the actions based on the act of viewing a web page containing content uploaded from the upload unit 106 to the web server 20 (second device) and published on the internet N1 by the web server 20, as feedback information when a general user C on the internet N1 views this web page.

[0057] The feedback information collected by the collection unit 108 is stored in a database managed by the storage unit 103. Figure 4 is a schematic table showing an example of the configuration of a database in the information processing device 100 in one embodiment of the present invention, in which the storage unit 103 stores requests and responses in correspondence, as well as user actions, etc. For example, in this embodiment, as shown in Figure 4, the content of the request 1032, which is a prompt, and the content of the response 1033, which is AI content generated by inputting the request to the generating AI 300, are associated with a user 1031, who can be identified by a user ID. Information regarding the actions of user A or user B for each response (user action) 1034 and information regarding the actions of a general user C (user feedback) 1035 are stored for each type of action.

[0058] As shown in Figure 4, the data collected by the collection unit 108, including user feedback (user feedback 1035) such as the number of impressions, the number of page views, and the average time spent on the site (average viewing time), is stored in the database.

[0059] As mentioned above, the evaluation criteria revision unit 109 revises the information regarding the "evaluation criteria" that serve as the basis for judging usefulness, which is stored in the internal database or the like by the judgment unit 104, into new evaluation criteria that reflect the collected user feedback 1035. In this embodiment, various methods can be used for revising the information regarding "evaluation criteria" by the evaluation criteria revision unit 109, but for example, the following mechanism can be considered. (1) If user actions within the closed environment of the AI ​​generation service provision system 10 show a high number of page views and likes, indicating that the content is good, but user feedback in the open environment after uploading to the internet N1 shows few user actions and the content is not well-regarded, then the information regarding the "evaluation criteria" may be revised to set higher standards for content similar to that content in terms of field, response content / expression, and sentence structure to be considered good content within the company. (2) If user actions in a closed environment are evaluated as having relatively few page views or likes, but user feedback in an open environment shows a very high number of user actions and the content is evaluated as good, then the information regarding "evaluation criteria" may be revised to lower the standards for content similar to that content in terms of field, content / expression of answers, and sentence structure to be evaluated as good content within the company.

[0060] As described above, in an information processing system 1 which is one embodiment of the present invention, the information processing device 100 (first device) further comprises a collection unit 108 that collects feedback (user feedback) 1035 from user C (second user) on web pages published on the web server 20 (second device), and an evaluation criteria revision unit 109 that revises the evaluation criteria stored in the database of the judgment unit 104 to new evaluation criteria that reflect the external feedback collected by the collection unit 108. In other words, if user ratings are good in an open environment, the rating criteria (thresholds, etc.) for similar or related content in a closed environment should be lowered. Conversely, if user ratings are poor in an open environment, the rating criteria (thresholds, etc.) for similar or related content in a closed environment should be raised. Here, "similar or related content" refers to other content that has similar attributes. For example, in Figure 3, user ID 10038 has received content (responses) about "utilizing generative AI." In such cases, other content related to "generative AI" can be considered "similar or related content" to this content. By revising the evaluation criteria in this way, incorporating user feedback from an open environment, we can create a system that receives high praise from many general users outside the company, thereby increasing general user interest in the system.

[0061] The confidential information determination unit 110 determines whether the response acquired by the response acquisition unit 102 and stored in the storage unit 103 contains confidential information. In this embodiment, "confidential information" refers to general personal information, information whose free use by others is restricted, or information that can only be known by a specific organization or individual (for example, information that can only be known by users within the generation AI service provision system 10), and whose disclosure may harm business interests or cause legal problems.

[0062] In this embodiment, the confidential information shall specifically include the following information: (1) Personal information: Information that can identify a specific individual, such as name, address, telephone number, email address, and credit card number. (2) Business secrets: Confidential information relating to business activities, such as commercial strategies, sales data, customer lists, and unreleased product information. (3) Intellectual property: Legally protected information such as patents, trademarks, and copyrights. (4) Security information: Security information such as passwords, API keys, and encryption keys.

[0063] When a generating AI produces AI content containing confidential information as described above, there is a risk of information leakage and infringement of others' rights. Therefore, it is necessary to properly monitor and manage the output of the generating AI and filter or modify it as needed. Therefore, in this embodiment, for example, if the content generation unit 105 determines that the response acquired by the response acquisition unit 102 and stored by the storage unit 103 contains confidential information, it will not generate web page content from that response. The confidential information determination unit 110 can automatically check whether the content contains confidential information by, for example, using data loss prevention (DLP) tools or information detection tools to detect specific patterns or formats of information such as credit card numbers or information that falls under the Personal Information Protection Act. In addition, the confidential information determination unit 110 can also determine the presence or absence of confidential information by using AI-based contextual analysis that understands the context of the content and evaluates whether or not it contains confidential information, using natural language processing (NLP) or machine learning.

[0064] As described above, in an information processing system 1, which is one embodiment of the present invention, the information processing device 100 (first device) further includes a confidential information determination unit 110 that determines whether the response contains confidential information. If the confidential information determination unit 110 determines that the response contains confidential information, the content generation unit 105 refrains from generating web page content from the response. By refraining from publishing risky content on the internet, it becomes possible to protect the privacy of individuals and organizations, avoid legal issues that may arise from publication, reduce the risk of data breaches, and ultimately strengthen system security.

[0065] The identification unit 111 identifies the confidential information contained in the response acquired by the response acquisition unit 102. The identification of confidential information by the identification unit 111 has the same function as the confidential information determination by the confidential information determination unit 110 described above, determining whether or not confidential information is contained in the content, but the identification unit 111 also identifies the location of the confidential information within the content.

[0066] The masking unit 112 masks the information (for example, text such as personal information) in the confidential information portion of the content identified by the identification unit 111. There are various methods of masking, but for example, the following methods can be used. (1) Substitution: Replace confidential information with other information, for example, a personal name with a name like "Taro Yamada". (2) Deletion: Completely delete the confidential information. (3) Randomization: Replace confidential information with random information. This is effective for numerical data, for example. (4) Hashing: By converting secret information into a hash value, it becomes virtually impossible to recover the original information. (5) Tokenization: By replacing sensitive information with a specific token, it becomes possible to process data without knowing the original information. (6) Masking of audio and images: Covering the confidential audio and image portions of content identified by speech recognition or image recognition with blur or mosaic effects. (7) Data masking: Replacing data in tables or columns of a database, etc., with dummy data that is different from the actual data. Furthermore, the methods described above can be used in combination as needed.

[0067] As described above, in an information processing system 1 which is one embodiment of the present invention, the information processing device 100 (first device) further comprises an identification unit 111 that identifies confidential information contained in the response, and a masking unit 112 that masks the confidential information identified by the identification unit 111. The content generation unit 105 then generates content in which the confidential information contained in the response has been masked as shown in (1) to (7) above. By keeping risky parts of the content private, it becomes possible to protect the privacy of individuals and organizations, avoid legal issues that may arise from disclosure, reduce the risk of data breaches, and improve system reliability and security.

[0068] Furthermore, the content generated by the content generation unit 105 may be subject to SEO measures applied to the internal structure of the site, such as HTML and control files that make up the website, so that the web pages can be quickly crawled and accurately indexed by search engines and their thematic relevance can be correctly evaluated. For example, examples of on-page SEO measures include on-page measures related to crawlability, such as setting up breadcrumbs, and on-page measures related to indexing, such as optimizing page titles by including keywords.

[0069] As described above, the content generated by the content generation unit 105 of the information processing device 100 (first device) according to this embodiment can be made to have SEO measures applied. By implementing these on-page SEO measures, when the content is published on the internet N1 via the web server 20, it can enjoy benefits such as improved search engine rankings, increased user engagement, higher conversion rates, improved cost efficiency, and enhanced credibility.

[0070] <Example Hardware Configuration> Figure 5 shows an example of the hardware configuration of an information processing device 100 according to one embodiment of the present invention. The information processing device 100 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0071] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0072] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0073] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0074] The storage device 1040 is an auxiliary storage device that can be implemented as a removable media such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or memory card, or as ROM (Read Only Memory), and has a recording medium. The recording medium of the storage device 1040 stores program modules that realize each function of the information processing device 100. The processor 1020 reads these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to that program module.

[0075] The input / output interface 1050 is an interface for connecting the information processing device 100 with various input / output devices.

[0076] The network interface 1060 is an interface for connecting the information processing device 100 to network B1. The method by which the network interface 1060 connects to network B1 may be wireless or wired. The information processing device 100 may also communicate with a user terminal 200 or the like via the network interface 1060.

[0077] The hardware configuration of the web server 20 is the same as that of the information processing device 100 described above, so its explanation is omitted here. Furthermore, user terminals 200, 400, and T1 are smartphones and personal computers, and their hardware configurations are the same as those of the information processing device 100 shown in Figure 5, so a detailed explanation is omitted here.

[0078] <Example of operation> In the embodiments described below, the information processing procedure in the information processing device 100 with the above-described configuration will be specifically explained using the flowchart shown in Figure 6. Since the specific individual processing and operations in each part of the information processing device 100 are as described above, this section will mainly explain the information processing flow.

[0079] <Generating responses using AI> Figure 6 is a flowchart illustrating the AI ​​content generation process procedure in an information processing device 100 according to one embodiment of the present invention.

[0080] In the information processing system 1, a user (User A) of the generation AI service provision system 10 generates AI content using the generation AI service provided by the information processing device 100. First, User A receives the generation AI service using their user terminal 200 and inputs a request, which is a prompt for the generation AI, from the user terminal 200. The request entered by User A on the user terminal 200 is received by the reception unit 101 of the information processing device 100 (step S101).

[0081] Requests received by the reception unit 101 are processed by the generation AI 300 used in the generation AI service provided by the information processing device 100 to generate content, and a response, which is an AI product (AI content), is generated. The response is then acquired by the response acquisition unit 102 of the information processing device 100 (step S102).

[0082] In the information processing device 100, the response acquired by the response acquisition unit 102 is associated with the request entered by user A, and the storage unit 103 stores them (step S103). For example, in this embodiment, the request and response are associated and stored (stored) in a database provided by or managed by the storage unit 103 in a table format as shown in Figure 3 (step S103). Furthermore, in the generation AI service provision system 10, the database information stored in the storage unit 103 of the information processing device 100 can be viewed by user A and other users (for example, user B) of the generation AI service provision system 10. That is, the storage unit 103 stores requests entered by user A from the user terminal 200 and the responses generated by the generation AI based on those requests. In addition, the user terminals of other users who enjoy the generation AI service in the generation AI service provision system 10 (for example, user B's user terminal 400) can also view the requests and responses related to user A stored in the storage unit 103.

[0083] <User actions and web page generation> Figure 7 is a flowchart illustrating the web page generation process procedure in an information processing device 100 according to one embodiment of the present invention. When user B accesses the AI ​​service using user terminal 400 at any time, such as when user B uses the AI ​​service provision system 10, the request entered by user A and the response generated based on that request are displayed on user terminal 400, and user B can view them. In addition to viewing the response, user B can take actions (internal actions) on the response, such as copying, exporting, pressing the good button, pressing the bad button, writing comments, and rating, as described above. Information regarding user B's actions is then acquired by the action acquisition unit 107 of the information processing device 100 (step S201).

[0084] Next, the determination unit 104 of the information processing device 100 determines the usefulness of the response using the determination method described above, that is, it determines whether the response is useful or not (step S202). The determination of usefulness by the determination unit 104 is as described above.

[0085] As a result, if the determination unit 104 determines that the response is useful, the content generation unit 105 generates web page content from that response (step S203). Furthermore, when generating content for a web page, the content generation unit 105 may embed link information (e.g., URLs) into the web page, i.e., insert link information, such as a site introducing the generation AI service provided by the generation AI service provision system 10 or a site for applying for the generation AI service. This allows non-users of the generation AI service to directly access specific websites, such as introduction pages or application pages for the generation AI service. This not only provides useful information but also facilitates directing users to specific websites, thereby increasing the visibility of those websites.

[0086] On the other hand, if the determination unit 104 determines in step S202 that the response is not useful, the content generation unit 105 will not generate any content from the response, and the process will terminate. The action acquisition process in the action acquisition unit 107 may be executed in real time at all times, but it may also be set to acquire actions at regular intervals, or to acquire them at any arbitrary time, or the timing of acquisition may be set within the system.

[0087] Furthermore, during the content generation process by the content generation unit 105 in step S203, as described above, if the secret information determination unit 110 determines that the content contains secret information, the content may not be generated. Furthermore, when generating content by the content generation unit 105 in step S106, as described above, the content can also be generated after the masking unit 112 has performed a masking process on the portion of the content whose confidential information has been identified by the identification unit 111.

[0088] <Web page publication> Figure 8 is a flowchart illustrating the web page publishing process procedure in an information processing device 100 according to one embodiment of the present invention. The upload unit 106 of the information processing device 100 uploads the web page content generated by the content generation unit 105 to the web server 20 (step S301). At this time, as described above, data (web page content) is uploaded from the closed environment generation AI service provision system 10 to the web server 20 on the public network (Internet N1) (including devices managed and designated by the web server 20). As a result, the web server 20, which is a public server, publishes the web page on the Internet N1, and user C can view the web page generated based on the content from user terminal T1.

[0089] When user C browses a webpage from user terminal T1, if they find the content or structure to be good, they may take actions such as viewing the page multiple times or spending a long time browsing (staying) on ​​it. In this case, the collection unit 108 of the information processing device 100 collects such actions by user C as user feedback (step S302).

[0090] Then, the evaluation criteria revision unit 109 revises the evaluation criteria stored in the judgment unit 104 to new evaluation criteria information that reflects the user feedback collected by the information information unit 108 (step S303). This makes it possible to create evaluation criteria that further reflect feedback from general users in addition to the original evaluation criteria, thereby improving the reliability of the processing of the generation AI service provision system 10 and the processing of the entire information processing system 1.

[0091] It should be noted that the flowcharts shown in Figures 6 to 8 in this embodiment are merely one embodiment. Without changing the spirit of the present invention, there may be processes other than those described in Figures 6 to 8, some of the processes described in Figures 6 to 8 may be omitted, or the order of the processes may be changed.

[0092] While embodiments of the present invention have been described above, the embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of this invention is indicated not by the above description but by the claims, and all modifications within the meaning and scope of the claims are intended to be included. [Explanation of Symbols]

[0093] 1. Information Processing System 10. Generation AI Service Provision System 20. Web server (second device) 100 Information Processing Device (First Device) 101 Reception Department 102 Response Acquisition Unit 103 Storage section 104 Judgment Department 105 Content Generation Department 106 Upload Section 107 Action Acquisition Section 108 Collection Department 109 Evaluation Criteria Revision Department 110 Confidential Information Determination Department 111 Specific section 112 Masking section 113 Display section 200 user terminals 300 Generation AI 400 user terminals 1010 Bus 1020 Processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces N1 Internet T1 User Terminal

Claims

1. An information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who does not receive the generation AI service, The first apparatus is A reception unit that receives requests from the aforementioned first user, A response acquisition unit that acquires a response to the request using the generation AI provided by the generation AI service, A storage unit that stores the aforementioned request and the aforementioned response in association, A determination unit for determining the usefulness of the response, A content generation unit that generates web page content from the response based on the determination result of the determination unit, The device includes an upload unit that uploads the aforementioned content to the second device. An information processing system characterized by the following:

2. The information processing system according to claim 1, The content generation unit includes in the content link information for the second user to access the generated AI service provided by the first device. An information processing system characterized by the following:

3. The information processing system according to claim 1, The determination unit analyzes the inherent quality of the response using natural language processing (NLP) to determine the usefulness of the response. An information processing system characterized by the following:

4. The information processing system according to claim 3, The determination unit evaluates the inherent quality of the response using a large-scale language model (LLM) to determine its usefulness. An information processing system characterized by the following:

5. The information processing system according to claim 1, The first apparatus is A display unit that displays the requests and responses from the first user stored in the storage unit on the user terminal of the first user or on the user terminal of a third user who is a different user from the first user and is receiving the generation AI service, The system further includes an action acquisition unit that acquires information regarding the actions of the first user or the third user in response to the aforementioned response. The storage unit stores information relating to the actions of the first user or the third user in association with the request and the response. The determination unit compares the information regarding the actions of the first user or the third user in response to the response stored in the storage unit with predetermined evaluation criteria to determine the usefulness of the response. An information processing system characterized by the following:

6. The information processing system according to claim 5, The first apparatus is A collection unit for collecting feedback from the second user on a web page published in the second device, The system further includes an evaluation criteria revision unit that revises the predetermined evaluation criteria into new evaluation criteria that reflect the collected feedback. An information processing system characterized by the following:

7. The information processing system according to claim 1, The first apparatus is The system further comprises a secret information determination unit that determines whether the response contains secret information, If the content generation unit determines that the response contains the confidential information, it shall not generate the content for the web page from the response. An information processing system characterized by the following:

8. The information processing system according to claim 1, The first apparatus is A unit for identifying confidential information contained in the response, The system further comprises a masking unit that performs masking of the confidential information identified by the specified unit, The content generation unit generates the content in which the secret information included in the response has been masked. An information processing system characterized by the following:

9. The information processing system according to claim 1, The content generated by the content generation unit of the first device is subject to SEO measures. An information processing system characterized by the following:

10. An information processing method in an information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who does not receive the generation AI service; The first apparatus is A receiving step for receiving a request from the first user, A response acquisition step which involves obtaining a response to the request using the generation AI provided by the generation AI service, A storage step of storing the request and the response in association, A judgment step for determining the usefulness of the response, A content generation step that generates content for a web page from the response based on the determination result of the aforementioned determination step, The upload step includes uploading the content to the second device. An information processing method characterized by the following:

11. A computer-readable program for execution in an information processing system comprising: a first device that provides a generation AI service to a first user; and a second device that functions as a web server accessible to a second user who does not receive the generation AI service, The first device, A receiving step for receiving a request from the first user, A response acquisition step which involves obtaining a response to the request using the generation AI provided by the generation AI service, A storage step of storing the request and the response in association, A judgment step for determining the usefulness of the response, A content generation step that generates content for a web page from the response based on the determination result of the aforementioned determination step, An upload step of uploading the content to the second device, A computer-readable program for executing a command.

Images