Program, method, information processing device, and system

Abstract

There is an issue in that it is not possible to create suitable FAQs that refer to images. [Solution] A program to be executed by a computer having a processor and a memory unit, wherein the processor executes the following steps: a document acquisition step of acquiring a document; an element acquisition step of acquiring one or more document elements contained in the document; an FAQ acquisition step of acquiring one or more FAQ data consisting of questions related to the document and their answers; a first instruction input step of inputting a first prompt to a large-scale language model based on the one or more document elements and the one or more FAQ data, the first prompt including an instruction to identify at least a portion of one or more FAQ data related to at least a portion of the one or more document elements; and an association step of associating and storing at least a portion of the one or more document elements with at least a portion of the identified one or more FAQ data based on the output content from the large-scale language model in response to the first prompt.

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] 2. Description of the Related Art Information processing systems are known for processing information related to frequently asked questions (FAQs), which are composed of pairs of questions and answers. Patent Document 1 discloses a program, a method, an information processing device, and a system for creating a suitable FAQ. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7611621 Summary of the Invention [Problem to be solved by the invention]

[0004] There is an issue in that it is not possible to create suitable FAQs that refer to images. Therefore, the present disclosure has been made to solve the above problem, and its purpose is to provide a technology for creating a suitable FAQ that references images. [Means for solving the problem]

[0005] A program to be executed by a computer having a processor and a storage unit, wherein the processor executes the following steps: a document acquisition step of acquiring a document; an element acquisition step of acquiring one or more document elements contained in the document; an FAQ acquisition step of acquiring one or more FAQ data consisting of questions related to the document and their answers; a first instruction input step of inputting a first prompt to a large-scale language model based on the one or more document elements and the one or more FAQ data, the first prompt including an instruction to identify at least a portion of one or more FAQ data related to at least a portion of the one or more document elements; and an association step of associating and storing at least a portion of the one or more document elements with at least a portion of the identified one or more FAQ data based on output content from the large-scale language model in response to the first prompt. [Effects of the Invention]

[0006] According to the present disclosure, it is possible to create a suitable FAQ that references images. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a block diagram showing the functional configuration of the system 1. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the server 10. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a user terminal 20. [Figure 4] FIG. 10 is a diagram showing the data structure of a user table 1012. [Figure 5] FIG. 10 is a diagram showing the data structure of a document table 1013. [Figure 6] FIG. 10 is a diagram showing the data structure of an element table 1014. [Figure 7] FIG. 10 is a diagram showing the data structure of an FAQ table 1015. [Figure 8] FIG. 10 is a diagram showing the data structure of an association table 1021. [Figure 9] 10 is a flowchart showing the operation of an association process. [Figure 10] 10 is a flowchart showing the operation of an FAQ creation process. [Figure 11] 10 is a flowchart showing the operation of a learning process. [Figure 12] 10 is a flowchart showing the operation of an FAQ answering process. [Figure 13] 10 is a screen example showing the operation of the FAQ creation process. [Figure 14] 10 is a screen example showing the operation of the FAQ answer processing. [Figure 15] FIG. 2 is a block diagram showing the basic hardware configuration of a computer 90. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings describing the embodiments, common components are designated by the same reference numerals, and repeated description will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are necessarily essential components of the present disclosure. Furthermore, each drawing is a schematic diagram and is not necessarily a precise illustration.

[0009] <System 1 Configuration> The system 1 in the present disclosure is an information processing system that provides an information processing service that manages reference information such as manuals and FAQs. The system 1 includes an information processing device including a server 10, a user terminal 20, and a generating AI 50, all of which are connected via a network N. FIG. 1 is a block diagram showing the functional configuration of the system 1. As shown in FIG. FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG. FIG. 3 is a block diagram showing the functional configuration of the user terminal 20. As shown in FIG.

[0010] Each information processing device is configured by a computer equipped with an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. For each of the server 10, the user terminal 20, and the generation AI 50, descriptions that overlap with the basic hardware configuration and basic functional configuration of the computer will be omitted.

[0011] <Server 10 configuration> The server 10 is an information processing device that provides an information processing service for managing reference information such as manuals and FAQs. The server 10 includes a storage unit 101 and a control unit 104 .

[0012] <Configuration of the storage unit 101 of the server 10> The storage unit 101 of the server 10 includes an application program 1011 , a user table 1012 , a document table 1013 , an element table 1014 , an FAQ table 1015 , and an association table 1021 .

[0013] The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit. Application programs 1011 include applications such as a web browser application.

[0014] User table 1012 is a table that stores and manages information about member users (hereinafter, "users") who use the service. When a user registers to use the service, the user's information is stored in a new record in user table 1012. This allows the user to use the service according to the present disclosure. The user table 1012 is a table having columns of user ID, user name, and user data, with the user ID as the primary key. FIG. 4 is a diagram showing the data structure of the user table 1012.

[0015] The user ID is an item that stores user identification information for identifying a user. The user identification information is an item that is set with a unique value for each user. The user name is an item for storing the name of the user. The user name may be set to any character string such as a nickname instead of a name. The user data includes information specific to the individual user and attribute information relating to the user's characteristics and background. The user's unique information includes information unique to the user, such as the user's date of birth (age), sex, and the like. The user's attribute information includes information such as the user's educational history (highest level of education, major, graduation year), occupation, work history, interests, place of residence, and language.

[0016] The document table 1013 is a table for storing and managing information about documents (document information). The document table 1013 is a table having a document ID as a primary key and columns of document ID and document data. FIG. 5 is a diagram showing the data structure of the document table 1013.

[0017] The document ID is an item that stores document identification information for identifying a document. The document identification information is an item that has a unique value set for each piece of document information. Document data is an item that stores information about a document. The document data includes document files in formats such as PDF documents, Word documents, Markdown text, XML files, HTML files, text files, and image files (including OCR documents). In the present disclosure, document data refers to an item that stores document information related to product or service manuals. The document data may be stored directly, or may be configured to store information (path, URL, etc.) that identifies document data stored in an external directory.

[0018] The element table 1014 is a table for storing and managing information about elements (document elements). The element table 1014 is a table having the element ID as a primary key, and columns of element ID, document ID, and element data. FIG. 6 is a diagram showing the data structure of the element table 1014. As shown in FIG.

[0019] The element ID is an item that stores element identification information for identifying an element. The element identification information is an item that has a unique value set for each document element. The document ID is an item for storing document identification information for identifying a document. The element data is an item that stores information about the elements that make up the document. The element data includes one or more of the following information. The element data may be in text format or any structured data format such as html, xml, json, etc. The element data may be stored directly, or may be configured to store information (path, URL, etc.) that identifies element data stored in an external directory. Text elements: Text content such as paragraphs, sentences, headings, bullet points, and table cells Image elements: visual elements such as charts, photographs, illustrations, graphs, etc. Location information: The location of the element within the document (page number, coordinate information [x1,y1,x2,y2], etc.) Structural information: Identifiers that indicate the type of element (heading, body, figure, caption, etc.) Layout information: Information about the element's formatting (font, size, color, etc.) References: References to other elements (e.g., a text element referring to a figure element, such as "See Figure 1") The element data may be configured to store information referencing image elements included in the document data as tags, in addition to text information such as paragraphs, sentences, headings, bullet points, and table cells of the document data. For example, the tag may include a document ID that identifies the document related to the image element, and coordinate information that indicates the range of the image element in the identified document. The element data may also be configured so that the image data (binary data) itself is directly embedded, or may include any reference information that references the image data (such as a path to an image file or a URL), or may include information that references the element ID of another document element.

[0020] The FAQ table 1015 is a table for storing and managing information related to FAQs (FAQ information). The FAQ table 1015 is a table having the FAQ ID as a primary key, and columns of FAQ ID, document ID, question data, and answer data. FIG. 7 is a diagram showing the data structure of the FAQ table 1015.

[0021] The FAQ ID is an item that stores FAQ identification information for identifying FAQs. The FAQ identification information is an item in which a unique value is set for each piece of FAQ information. The document ID is an item for storing document identification information for identifying a document. The question data is an item that stores information about a question in the FAQ. The answer data is an item that stores information about answers to FAQs.

[0022] The association table 1021 is a table for storing and managing information relating to associations (association information). The association table 1021 is a table having columns for element ID and FAQ ID. FIG. 8 is a diagram showing the data structure of the association table 1021. As shown in FIG.

[0023] The element ID is an item for storing element identification information for identifying an element. The FAQ ID is an item for storing FAQ identification information for identifying an FAQ.

[0024] <Configuration of the control unit 104 of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041. The control unit 104 executes an application program 1011 stored in the storage unit 101, thereby realizing each functional unit.

[0025] The user registration control unit 1041 performs processing to store information about users who wish to use the service according to the present disclosure in the user table 1012. The information stored in the user table 1012 is generated when a user opens a web page operated by a service provider from any information processing terminal, enters information into a predetermined input form, and transmits the information to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, completing the user registration. This allows the user stored in the user table 1012 to use the service. Before the user registration control unit 1041 registers the user information in the user table 1012, the service provider may conduct a predetermined examination to restrict whether or not the user is permitted to use the service. The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or may be automatically set by the user registration control unit 1041.

[0026] <Configuration of User Terminal 20> The user terminal 20 is an information processing device operated by a user who uses a service. The user terminal 20 may be, for example, a mobile terminal such as a smartphone or tablet, a stationary personal computer (PC) or a laptop PC, or a wearable terminal such as a head mounted display (HMD) or a wristwatch terminal. The user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 , and an output device 208 .

[0027] <Configuration of the storage unit 201 of the user terminal 20> The storage unit 201 of the user terminal 20 includes a user ID 2011 and an application program 2012 .

[0028] The user ID 2011 is the user's account ID. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the user with the service according to the present disclosure. The user ID 2011 includes information such as a session ID temporarily assigned by the server 10 to identify the user using the user terminal 20.

[0029] The application program 2012 may be stored in advance in the storage unit 201, or may be configured to be downloaded from a web server or the like operated by a service provider via a communication IF. The application programs 2012 include applications such as a web browser application. The application program 2012 includes an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored on the user terminal 20.

[0030] <Configuration of the control unit 204 of the user terminal 20> The control unit 204 of the user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 executes an application program 2012 stored in the storage unit 201, thereby realizing each functional unit.

[0031] <Configuration of the input device 206 of the user terminal 20> The input device 206 of the user terminal 20 includes a camera 2061 , a microphone 2062 , a position information sensor 2063 , a motion sensor 2064 , and a touch device 2065 .

[0032] <Configuration of the output device 208 of the user terminal 20> The output device 208 of the user terminal 20 includes a display 2081 and a speaker 2082 .

[0033] <Configuration of generated AI50> Generative AI50 refers to large-scale artificial intelligence models used in the field of natural language processing (NLP). These models are trained on large amounts of text data (web pages, books, articles, etc.) to understand patterns in human language and effectively perform natural language generation (NLG) tasks. Generative AI 50 is used in many NLP tasks, such as generating answers to specific questions, automatically generating sentences, summarizing text, translation, and sentiment analysis. It can also be used for a variety of applications, including education, entertainment, customer service, and product development. Examples of generative AI 50 include the following. In this disclosure, a large-scale language model that primarily outputs text information as output information will be described as a type of generative AI 50. OpenAI ChatGPT Google Gemini ·Anthropic Claude Stable Diffusion Midjourney The generated AI 50 may be configured to be implemented as a part of the functions of the server 10.

[0034] <System 1 Operation> Each process of the system 1 will be explained below. FIG. 9 is a flowchart showing the operation of the association process. FIG. 10 is a flowchart showing the operation of the FAQ creation process. FIG. 11 is a flowchart showing the operation of the learning process. FIG. 12 is a flowchart showing the operation of the FAQ answering process. FIG. 13 is an example of a screen showing the operation of the FAQ creation process. FIG. 14 is an example of a screen showing the operation of the FAQ answer processing.

[0035] <Association process> The association process is a process of associating a document element related to a document such as a manual with one or more FAQ data items consisting of questions related to the document and their answers.

[0036] <Overview of association process> The association process is a series of steps that acquire a document such as a manual, extract document elements from the acquired document, present the extracted elements to the user while accepting edits from the user, store the document elements, associate the document elements with related FAQ data, and store the document elements and the FAQ data in association with each other.

[0037] <Details of association process> The association process will be described in detail below.

[0038] In step S101, the control unit 104 of the server 10 executes a document acquisition step for acquiring a document. Specifically, the control unit of the server 10 refers to the document table 1013 to acquire document information including document data. The control unit of the server 10 may acquire only document information for which the association process has not been performed. Alternatively, the control unit 104 of the server 10 may periodically perform the association process on the document information in the document table 1013. The control unit 104 of the server 10 may identify the document ID by searching the element table 1014 based on the element ID included in the association information stored in the association table 1021, and may start processing by targeting only the document information not included in the identified document ID as the target of the association process. In this disclosure, the association process for document data included in one piece of document information is disclosed as an example. When multiple pieces of document information are acquired, the following steps S102 and subsequent steps are executed for each piece of document information.

[0039] In step S102, the control unit 104 of the server 10 executes an element acquisition step of acquiring one or more document elements contained in the document. The element acquisition step includes a fourth instruction input step of inputting a fourth prompt to the large-scale language model, the fourth prompt including instructions for extracting information identifying the document, document elements (text elements, image elements, etc.) contained in the document, and image areas of images related to the document elements, and a step of acquiring the document elements by extracting them based on the output content from the large-scale language model in response to the fourth prompt. Specifically, the control unit 104 of the server 10 acquires the fourth instruction stored in the storage unit 101. The control unit 104 of the server 10 creates a fourth prompt by adding the document data acquired in step S101 to the location of ${document data} in the fourth instruction. Note that the document data may be included in the fourth instruction by substituting the document data for the location of ${document data} in the fourth instruction.

[0040] An example of the fourth instruction is shown below. <4th instruction> Identify and extract the document elements contained in the document below and the image regions associated with each document element. Document elements are extracted by dividing the document into sections, subsections, chapters, and paragraphs. Document: ${document data} Output format: #Element ID: [sequential number] #Element data: [Body text of document element] Illustration [number]: <bb> [Left X coordinate], [Top Y coordinate], [Right X coordinate], [Bottom Y coordinate]< / bb> Extract all document elements and associated image regions in a document and organize them in the specified format. - Enter the image area in the BB tag in the format "left X coordinate, top Y coordinate, right X coordinate, bottom Y coordinate." Each document element should be assigned a unique ID number and linked to any associated illustrations.

[0041] The control unit 104 of the server 10 sends a request including the created fourth prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to the server 10.

[0042] An example of the fourth response output by the generation AI 50 is shown below. The fourth response divides the document data into document elements such as sections, subsections, chapters, and paragraphs contained in the document data, extracts the document elements, and includes coordinate information that identifies the image area of the image related to the document elements. <bb> 953,391,1327,760< / bb> " is information that indicates the range of the image within the entire document data, with the pixel value at the top left of the original document data (assuming PDF format or image data) as the origin (0,0) and the pixel value at the bottom right as the end point (XMAX,YMAX). Specifically, " <bb> 953,391,1327,760< / bb> " is information indicating the range of a rectangle whose upper left coordinate is (953,391) and whose lower right coordinate is (1327,760). In this way, document elements are stored in association with not only the text information contained in the document elements but also information indicating the area range of the image. As a result, for document data such as a manual, the image area in the manual and the explanatory document (text document) are associated and stored for each document element. In the present disclosure, as an example, an image area tag " <bb> ···< / bb> " has been disclosed as an example of identifying an image area, but the present invention is not limited to this. In the element table 1014, explanatory text and image elements may be stored as elements of different element types (text element, image element), and the element data may include a character string related to identification information that identifies the document element, such as an element ID, in the element table 1014. This disclosure contemplates the use of any data format that associates explanatory text with image elements. Furthermore, the fourth response obtained from the generation AI 50 is allowed to acquire data in any structured format such as JSON format, XML format, etc. This allows the control unit 104 of the server 10 to store each item included in the acquired fourth response in the element data item of a new record in the element table 1014.

[0043] <Fourth Response> #Element ID: 001 #Element data: Cleaning and disinfecting the inside of the suction tube While immersing the tube in a diluted solution of neutral detergent, flush the inside of the tube with the solution to clean it, then disinfect it with disinfectant ethanol. After disinfection, dry thoroughly. Illustration 1: <bb> 953,391,1327,760< / bb> Illustration 2: <bb> 390,840,860,1244< / bb> #Element ID: 002 #Element data: Cleaning and disinfecting the exterior of the suction tube Clean with a soft sponge using a solution of neutral detergent. Illustration 3: <bb> 949,391,1327,760< / bb>

[0044] The control unit 104 of the server 10 stores each of the multiple document elements included in the fourth response in the element ID and element data fields of the new record in the element table 1014. Note that the document ID field stores the document ID of the document information acquired in step S101 (document ID of the document to be processed in the association process).

[0045] In step S103, the control unit 104 of the server 10 executes an element editing step of accepting an editing operation on a document element stored in the element table 1014 from the user. The element acquisition step includes the steps of presenting the extracted one or more document elements to the user in an editable manner, accepting edits to the presented one or more document elements, and acquiring the edited one or more document elements. It is also possible to accept edits from the user for the document elements acquired from the generation AI 50. Specifically, the configuration may be such that edits for the document elements acquired from the generation AI 50 are accepted from the user before or after the document elements are stored in the element table 1014. Specifically, the control unit 104 of the server 10 transmits the document elements acquired from the generation AI 50 to the user terminal 20. The control unit 204 of the user terminal 20 displays the element data of the received document elements on the display 2081 of the user terminal 20 and presents it in an editable format. This allows the user to check the document elements acquired from the generation AI 50. The user edits the element data by operating the input device 206 of the user terminal 20, etc. For example, the user can modify the explanatory text, the coordinates of the image area, etc. included in the element data. The user transmits the edited element data, together with the element ID of the document element, to the server 10 by operating the input device 206 of the user terminal 20, etc. The element editing step may be executed as part of the element acquisition step or as an independent step. Specifically, the target of the editing operation in the element editing step is not limited to the document elements output from the generation AI 50. Any document element stored in the element table 1014 may be editable.

[0046] In step S104, the control unit 104 of the server 10 executes an element storage step. Specifically, the control unit 104 of the server 10 searches the element ID item in the element table 1014 based on the received element ID, and overwrites the element data item of the identified record with the element data received from the user terminal 20. This makes it possible to extract document elements and images related to the document elements from documents such as manuals, etc., and improves the efficiency of editing document elements.

[0047] Note that, although the present disclosure has disclosed an example in which document elements are obtained using the generation AI 50, this is not limiting. For example, a user may manually extract document elements and store them in the element table 1014. Alternatively, document elements may be obtained using any region detection program, object detection program, or any other trained model without using the generation AI 50. The association process of the present disclosure can use any method for extracting one or more document elements from document data. The document elements stored in the element table 1014 may be document elements obtained from the generation AI 50 as they are, or may be document elements edited by the user. Additionally, the results of performing any processing on document elements may also be stored as document elements. All of these are included in the category of document elements.

[0048] In step S105, the control unit 104 of the server 10 executes an FAQ acquisition step of acquiring one or more FAQ data items each consisting of a question related to the document and its answer. Specifically, the control unit 104 of the server 10 refers to the FAQ table 1015 to acquire one or more FAQ data items each consisting of a question related to the document and its answer. The process of creating FAQ data will be explained in the FAQ creation process described later.

[0049] In step S105, the control unit 104 of the server 10 executes a first instruction input step of inputting a first prompt including an instruction to identify at least a portion of one or more FAQ data related to at least a portion of the one or more document elements to the large-scale language model based on one or more document elements and one or more FAQ data. Specifically, the control unit 104 of the server 10 acquires the first instruction stored in the storage unit 101. The control unit 104 of the server 10 creates a first prompt by adding one or more document elements acquired in step S104 and one or more pieces of FAQ information (question data, answer data) acquired to the ${document element} and ${FAQ data} portions of the first instruction, respectively. Note that the document elements and FAQ data may be included in the first instruction by substituting the document elements and FAQ information (question data, answer data) for the ${document element} and ${FAQ data} portions of the first instruction.

[0050] An example of the first instruction is shown below. This instruction is to output document elements and FAQ data in association with each other. <1st instruction> Based on the document elements and FAQ data below, identify at least some of the FAQ data that is related to at least some of the document elements. Document element: ${document element} FAQ data: ${FAQ data} Output format: |Element ID|FAQ ID|

[0051] The control unit 104 of the server 10 sends a request including the first prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to the server 10.

[0052] The following is an example of the first response output by the generation AI 50. The first response is based on the document elements and the FAQ data and includes information that identifies at least a portion of the FAQ data related to at least a portion of the document elements. <First response> |E001|FA001| |E001|FA004| |E012|FA012| |E014|FA022|

[0053] In step S106, the control unit 104 of the server 10 executes an association step of associating and storing at least a part of one or more document elements with at least a part of the identified one or more FAQ data based on the output content from the large language model for the first prompt. Specifically, the control unit 104 of the server 10 extracts pairs of one or more element IDs and FAQ IDs included in the first response, and stores each of them in the element ID and FAQ ID items of a new record in the element table 1014. Thereby, the document element and the FAQ data are associated. Also, since the document element includes the coordinate information of the image area, the coordinate information of the image area and the FAQ data are associated. Thereby, for the question received from the user, an answer can be output by referring to the image area of the document element.

[0054] <FAQ Creation Process> The FAQ creation process is a process of creating one or more FAQ data consisting of questions and their answers from documents such as manuals.

[0055] <Overview of FAQ Creation Process> The FAQ creation process is a series of processes that acquire a document such as a manual, extract FAQ data from the acquired document, present the extracted FAQ data to the user, and accept editing from the user.

[0056] <Details of FAQ Creation Process> The details of the FAQ creation process will be described below.

[0057] In step S301, the control unit 104 of the server 10 executes a document acquisition step of acquiring a document. Since step S301 is the same as step S101, the description is omitted.

[0058] In step S302, the control unit 104 of the server 10 executes a FAQ extraction step of extracting one or more FAQ data consisting of questions related to the document and their answers. The control unit 104 of server 10 extracts one or more FAQ data from the document data acquired in step S301.

[0059] <FAQ Extraction Step (First Embodiment)> As an example, the process of extracting FAQ data from document data using the generation AI 50 is shown below. The control unit 104 of server 10 acquires the fifth instruction stored in the storage unit 101. The control unit 104 of server 10 creates the fifth prompt by adding the document data acquired in step S301 to the location of ${document data} in the fifth instruction. Note that the document data may be included in the fifth instruction by replacing the location of ${document data} in the fifth instruction with the document data. In the present disclosure, predetermined document data that is the target of the FAQ creation process is included. Note that the FAQ creation process can be executed for any document data.

[0060] An example of the fifth instruction is shown below. <Fifth Instruction> Based on the following document, please create a FAQ consisting of the questions (F) assumed from the user for the document and the answers (Q) to those questions. Document: ${document data} Output format: #FAQID: [Sequential number] #Question data: [Question assumed from the user] #Answer data: [Answer to the question]

[0061] The control unit 104 of server 10 sends a request including the created fifth prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to server 10.

[0062] An example of the fourth response output by the generation AI 50 is shown below. The fourth response includes one or more FAQ data.

[0063] <Fifth Response> #FAQID:001 #Question data: Please teach me how to handle the emergency cock for supplying water in an emergency. #Answer data: The handling method of the emergency cock is as follows. 1. Remove the bottle cap seal 2. Insert the emergency cock 3. Turn the lever to the right to discharge water 4. When the remaining water runs out, tilt the bottle or remove the cap to pour

[0064] <FAQ extraction step (second embodiment)> As an example, the process of creating FAQ data from document elements and FAQ data using the generative AI 50 is shown below. Specifically, by using the FAQ data extracted in the FAQ extraction step (first embodiment) as a basis and executing the process according to the FAQ extraction step (second embodiment), more suitable FAQ data can be created. Note that the FAQ data used in the FAQ extraction step (second embodiment) does not necessarily have to be the FAQ data extracted in the FAQ extraction step (first embodiment), and it may also be FAQ data created manually by the user. Any data can be used as long as it is the FAQ data stored in the FAQ table 1015. The control unit 104 of the server 10 executes a second instruction input step of inputting a second prompt including an instruction to output an answer B of the FAQ data B that refers to the document element B for the question B to the large language model based on the question B of the FAQ data B and the document element B associated with the question B. Specifically, the control unit 104 of the server 10 acquires the second instruction stored in the storage unit 101. The control unit 104 of the server 10 creates a second prompt by adding the FAQ information (question data, answer data) and document elements (element data) extracted in step S302 to the ${FAQ data} and ${document elements} portions of the second instruction, respectively. Note that the FAQ data and document elements may be included in the second instruction by substituting the FAQ information (question data, answer data) and document elements (element data) for the ${FAQ data} and ${document elements} portions of the second instruction. The FAQ data included in the ${FAQ data} section does not have to be all of the FAQ information (question data, answer data) stored in the FAQ table 1015, but may be one or more pieces of FAQ information (question data, answer data) stored in the FAQ table 1015. Furthermore, the document elements included in the ${document element} section do not have to be all of the document elements stored in the element table 1014, but may be one or more pieces of document elements (element data) stored in the element table 1014. Furthermore, the document element to be included in the ${document element} portion is preferably one or more documents (element data) associated with one or more pieces of FAQ information (question data, answer data) via the association table 1021. Specifically, the control unit 104 of the server 10 searches the FAQ ID item in the association table 1021 based on the FAQ ID of the one or more pieces of FAQ information, and acquires one or more predetermined element IDs. The control unit 104 of the server 10 searches the element ID item in the element table 1014 based on the predetermined element ID, and acquires one or more predetermined element data. The control unit 104 of the server 10 creates the second prompt by adding the acquired predetermined element data to the ${document element} portion of the second instruction. This makes it possible to create FAQ information that includes more suitable answer data that references the image areas included in the original document based on the predetermined FAQ information. In the FAQ extraction step (second embodiment), the FAQ information to be drafted and the document elements that include information that references the image areas extracted from the original document are used as input information, and more suitable answer data that references the image areas included in the original document can be included.

[0065] The control unit 104 of the server 10 sends a request including the created second prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to the server 10.

[0066] Below is an example of the second response output by the generation AI 50. The second response includes one or more FAQ data. <Second Response> #FAQID:001 #Question: Please tell me how to use the emergency cock to supply water in an emergency. #Answer data: Here's how to use the emergency cock: 1. Remove the cap seal from the bottle <bb> 392,397,785,729< / bb> 2. Insert the emergency cock <bb> 390,840,860,1244< / bb> 3. Turn the lever to the right to release water. <bb> 949,391,1327,760< / bb> 4. When there is no more water left, tilt the bottle or remove the cap and pour. <bb> 949,916,1741,1241< / bb>

[0067] The control unit 104 of the server 10 executes an answer storage step of storing answer B of FAQ data B that references document element B for question B in association with question B based on the output content from the large-scale language model in response to the second prompt. Specifically, the control unit 104 of the server 10 stores each of the multiple pieces of FAQ information (FAQ ID, question data, answer data) included in the second response in the FAQ ID, question data, and answer data fields of the FAQ table 1015. Note that the document ID field stores the document ID of the document information of the document to be processed in the FAQ creation process, acquired in step S301. Note that if there is a record with the same FAQ ID in the FAQ table 1015, that record may be overwritten with the FAQ information created in the FAQ extraction step (second embodiment), or multiple different versions of FAQ information may be stored for the same FAQ ID. By providing the FAQ table 1015 with a column for storing version information (not shown), multiple versions of FAQ information may be stored in a manner that allows them to be selectively used.

[0068] In step S303, the control unit 104 of the server 10 executes an FAQ editing step of accepting an editing operation on the FAQ information stored in the FAQ table 1015 from the user. Note that edits to the FAQ information acquired from the generation AI 50 may be accepted from the user. Specifically, edits to the FAQ information acquired from the generation AI 50 may be accepted from the user before or after the FAQ information is stored in the FAQ table 1015. Specifically, the control unit 104 of the server 10 transmits the FAQ information acquired from the generation AI 50 to the user terminal 20. The control unit 204 of the user terminal 20 displays the question data and answer data of the received FAQ information on the display 2081 of the user terminal 20 in an editable format. This allows the user to check the FAQ information acquired from the generation AI 50. The user edits the question data and answer data by operating the input device 206 or the like of the user terminal 20. For example, the user can modify the explanatory text, coordinates of the image area, and the like included in the question data and answer data. The user transmits the edited question data and answer data to the server 10 together with the FAQ ID of the FAQ information by operating the input device 206 or the like of the user terminal 20. The FAQ editing step may be executed as part of the FAQ extraction step, or may be executed as an independent step. Specifically, the target of the editing operation in the FAQ editing step is not limited to the FAQ information output from the generation AI 50. Any FAQ information stored in the FAQ table 1015 may be editable.

[0069] FIG. 13 is an example of a screen showing the operation of the FAQ creation process. The control unit 204 of the user terminal 20 displays an FAQ editing screen for question page D1 on the display 2081 of the user terminal 20. The FAQ editing screen for question page D1 includes a document display area D101 and Q&A editing areas D111 and D112. The Q&A editing areas include question data and answer data. The document display area D101 includes text document elements D121, D122, and D123 and image document elements D131, D132, and D133. The Q&A editing areas D111 and D112 display the FAQ information to be edited in an editable manner. The user can freely edit the question data and answer data output to the Q&A editing areas D111 and D112 by operating the input device 206 of the user terminal 20, etc. It is also possible to set the Q&A editing area D111 to be the FAQ information before editing, and D112 to be the FAQ information after editing. This allows the user to create more suitable question data and answer data while checking the question data and answer data before editing. The user selects the save button D151 by operating the input device 206 or the like of the user terminal 20. The control unit 204 of the user terminal 20 transmits the question data and answer data entered in the Q&A editing areas D111 and D112 to the server 10. The control unit 104 of the server 10 stores the received question data and answer data in the question data and answer data items of the FAQ table 1015. The control unit 104 of the server 10 may overwrite the corresponding record in the FAQ table 1015, or may register it as a new record. If the user determines that the question data and answer data entered in the Q&A editing areas D111 and D112 are sufficient, the user selects the check box for the completion flag D153. As a result, a work completion flag (True) is stored in association with the FAQ information related to the question data and answer data entered in the Q&A editing areas D111 and D112. Specifically, it is stored that the FAQ creation process for a specific page of the document displayed in the document display area D101 has been completed. Specifically, the work completion flag is stored in association with the specific page of the document. The user selects the image switching button D152 by operating the input device 206 or the like of the user terminal 20. The control unit 204 of the user terminal 20 sends a request to the server 10 for another image for which the work is not completed. The control unit 104 of the server 10 identifies pages that are not associated with a work completion flag (True) from among the documents subject to the FAQ creation process, and sends them to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the received pages in the document display area D101 of the FAQ editing screen for the question page D1. The user can edit the question data and answer data of the page by editing the Q&A editing areas D111 and D112. By repeating this process, the user can create FAQ information for a document such as a manual.

[0070] In step S304, the control unit 104 of the server 10 executes an FAQ storage step. Specifically, the control unit 104 of the server 10 searches the FAQ ID item in the FAQ table 1015 based on the received FAQ ID, and overwrites the question data and answer data items of the identified record with the question data and answer data received from the user terminal 20. This makes it possible to extract FAQ information and images related to that FAQ information from documents such as manuals, thereby improving the efficiency of editing FAQ information.

[0071] Although the present disclosure has been described as an example in which FAQ information is acquired using the generation AI 50, the present disclosure is not limited to this. For example, a user may manually extract FAQ information and store it in the FAQ table 1015. The FAQ information stored in the FAQ table 1015 may be the FAQ information acquired from the generation AI 50 as is, or may be FAQ information edited by the user. Additionally, the results of performing any processing on the FAQ information may be stored as FAQ information. All of these are included in the category of FAQ information.

[0072] <Learning process> The learning process is a process of training a learning model based on FAQ data and document elements.

[0073] <Learning process overview> The learning process is a series of processes that acquires questions from FAQ data, acquires document elements, and trains a learning model using the questions and document elements as input data so that answers to the FAQ data can be output.

[0074] <Learning process details> The learning process will be described in detail below.

[0075] In step S501, the control unit 104 of the server 10 executes a question acquisition step of acquiring question data stored in the FAQ table 1015. Specifically, the control unit 104 of the server 10 acquires one or more pieces of question data by referring to the FAQ table 1015. Note that the control unit 104 of the server 10 does not need to acquire all of the question data stored in the FAQ table 1015, and may be configured to acquire only one or more pieces of question data stored in the FAQ table 1015.

[0076] In step S502, the control unit 104 of the server 10 executes a question acquisition step of acquiring answer data stored in the FAQ table 1015. Specifically, the control unit 104 of the server 10 acquires one or more pieces of answer data by referring to the FAQ table 1015. Note that the control unit 104 of the server 10 does not need to acquire all of the answer data stored in the FAQ table 1015, and may be configured to acquire only one or more pieces of answer data stored in the FAQ table 1015. Specifically, the control unit 104 of the server 10 acquires, from the FAQ table 1015, answer data having the same FAQ ID as the question data acquired in step S501. The control unit 104 of the server 10 searches the FAQ ID item in the FAQ table 1015 based on the FAQ ID of the predetermined question data acquired in step S501, and acquires one or more predetermined answer data.

[0077] In step S503, the control unit 104 of the server 10 executes a learning step in which, using question A of FAQ data A and document element A associated with question A as input, the learning model is trained so that answer A of FAQ data A is output.

[0078] <Steps for creating training data> The control unit 104 of server 10 executes a process of preparing, as a pair, the answer data (hereinafter, Answer A) corresponding to the question data (hereinafter, Question A) acquired from the FAQ table 1015. The control unit 104 of server 10 adds, as context information of Question A, document element A (element data including text elements and image elements) associated with Question A via the association table 1021, and creates these as one learning sample. Specifically, the control unit 104 of server 10 searches the FAQ ID item of the association table 1021 based on the FAQIDA of Question A, and acquires the item of one or more element IDs. The control unit 104 of server 10 searches the element ID item of the element table 1014 based on the one or more acquired element IDs, and acquires one or more document elements (document element A). The control unit 104 of server 10 creates input A to the learning model by adding (combining) document element A to Question A. The control unit 104 of server 10 creates learning data such that input A is input data and Answer A is output data. The learning unit 1051 of server 10 creates data sets such as training data, test data, and verification data for training the deep neural network of the learning model based on the learning data. The control unit 104 of server 10 learns the learning parameters of the deep neural network included in the learning model by deep learning so that input A is input data and Answer A is output data. Note that the learning model does not necessarily have to be an untrained deep learning model, and it may be a generative AI 50. In this case, the generative AI 50 is re-trained (fine-tuned) with the learning data. The control unit 104 of server 10 is configured to be able to refer to image information during answer generation, such as incorporating, as metadata in which the image area information (coordinate information) included in document element A is also textified, into the learning data for the learning model.

[0079] <FAQ Answer Processing> The FAQ answer processing is a process of presenting a suitable answer to a question received from a user. The answer content for the user is generated using a large language model based on FAQ data and document elements.

[0080] <Overview of FAQ Answer Processing> The FAQ answer processing is a series of processes that receive a question from a user, obtain a question for the received answer, and present the obtained answer.

[0081] <Details of FAQ Answer Processing> The details of the FAQ answer processing will be described below.

[0082] In step S701, the control unit 104 of the server 10 executes a question reception step of receiving a question from the user regarding a predetermined product or service. Specifically, the first user operates the input device 206 of the user terminal 20, executes a browser application or the like, and opens the question page D1 by inputting the URL of a web page (question page) for executing question processing. The control unit 204 of the user terminal 20 transmits a request including the user ID 2011 for opening the question page to the server 10.

[0083] When the server 10 receives the request, it generates a question page and transmits it to the user terminal 20. The control unit 204 of the user terminal 20 displays and presents the question page on the display 2081 of the user terminal 20. FIG. 14 shows an example of a question screen in the FAQ answering process. A question page D3 is displayed on the display 2081 of the user terminal 20. The question page D3 includes a question input field D301, an answer output field D302, and a send button D303. The question page D3 may present information indicating the product, service, etc. that is the subject of the question (information such as the product model number and service name), as well as options that can accept a selection. For example, the user can select the product name, service name, etc. that is the subject of the question from the options. The question page D3 may also be configured to be provided for each product or service (provided by a fixed URL, etc.). In this case, the question page D3 becomes a screen that can exclusively accept questions about a specific product or service. In this case, information (character strings) indicating the product, service, etc. is embedded in the URL, etc. The control unit 104 of the server 10 identifies a document ID related to the product, service, etc. selected by the user as the subject of a question from the document table 1013, based on information indicating the product, service, etc. (information such as product model number, service name, etc.) and character strings indicating the product, service, etc. included in the URL, etc. Specifically, the control unit 104 of the server 10 identifies a document ID including document information related to a document such as a manual related to the product, service, etc. A user can freely input questions about how to use a product, service, etc., or about any points of uncertainty, in the question input field D301 of the question page D3, and the control unit 104 of the server 10 can provide an FAQ service (chatbot service) that can automatically answer questions received from users by referring to document information, document elements, FAQ information, etc. The user inputs a predetermined question (which may include not only text but also information such as images and audio) in the question input field D301 of the question page D3 and selects the send button D303. The control unit 204 of the user terminal 20 transmits the predetermined question (question A) input in the question input field D301 to the server 10. The control unit 104 of the server 10 receives and accepts the question. In the present disclosure, as an example, a case will be described in which question A such as "I can't insert the emergency cock into the bottle" is received from the user.

[0084] In step S702, the control unit 104 of the server 10 executes an answer acquisition step of acquiring an answer to the question received in step S701.

[0085] <Answer Acquisition Step (First Embodiment)> The control unit 104 of the server 10 inputs the request including the question A acquired in step S701 to the learning model trained in the learning process. The learning model may be a learning model stored in the storage unit 101 of the server 10, or may be the retrained generated AI 50. If the learning model is the generated AI 50, the request is sent to an endpoint provided by the generated AI 50. The learning model outputs a response to the request to the control unit 104 of the server 10. The control unit 104 of the server 10 receives and acquires a response including answer A to question A. In the present disclosure, an example in which a learning model trained in advance by a learning process is used in the answer acquisition step (first embodiment) has been described as an example, but this is not limiting. For example, a learning process may be executed in the FAQ answering process, and the answer acquisition step (first embodiment) may be executed using the acquired learning model. Furthermore, in addition to the learning process, all or part of the association process, FAQ creation process, and learning process may be executed in the answer acquisition step (first embodiment). In other words, the control unit 104 of the server 10 may acquire document information such as a manual (which may be received from the user rather than from the document table 1013), and based on the document, execute all or part of the association process, FAQ creation process, and learning process, and then execute the answer acquisition step (first embodiment) based on the execution results.

[0086] <Answer Acquisition Step (Second Embodiment)> In step S702, the control unit 104 of the server 10 executes a third instruction input step of inputting a third prompt including an instruction to output answer A to question A received in the question receiving step into the large-scale language model based on the answer contained in the specified FAQ data and specified document elements. Specifically, the control unit 104 of the server 10 acquires the third instruction stored in the storage unit 101. The control unit 104 of the server 10 creates a third prompt by adding the question A acquired in step S701, the FAQ information (question data, answer data) acquired by referring to the FAQ table 1015, and the element information (element data) acquired by referring to the element table 1014 to the ${question}, ${FAQ data}, and ${document element} parts of the third instruction, respectively. Note that the question A, the FAQ information (question data, answer data), and the element information (element data) may be included in the third instruction by replacing the question A, the FAQ information (question data, answer data), and the element information (element data) in the ${question}, ${FAQ data}, and ${document element} parts of the third instruction. Furthermore, the third prompt may be created by including in the third instruction information that associates the element information stored in the association table 1021 with the FAQ information. This makes it possible to obtain a more suitable answer A by taking into consideration the mutual association between the document element and the FAQ information.

[0087] The control unit 104 of the server 10 sends a request including the created third prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to the server 10. Unlike the answer acquisition step (third embodiment), by including all of the information in the element table 1014 and the FAQ table 1015 in the third prompt, it is possible to acquire a suitable answer A from a broader perspective to the question A received from the user.

[0088] An example of the third response output by the generation AI 50 is shown below. The third response includes answer A to question A. Answer A is not the answer data stored in the FAQ table 1015, but is generated and provided by the generation AI 50 while referring to the document elements in the element table 1014, to be more suitable for answer A received from the user. Specifically, the third response (second embodiment) differs from the third response (third embodiment) in that it includes answer contents such as "1. Find a level surface and place the bottle so that the bottom is level" and "2. Make sure the bottle cap is facing upwards." In the answer acquisition step (second embodiment), the generation AI 50 can generate an answer A to the question A by taking into consideration document elements and FAQ information that are less relevant to the question A. The generation AI 50 can generate an answer A that is more suitable for the user's problem. <Third Response (Second Embodiment)> #Question data: I can't insert the emergency cock into the bottle #Response data: The bottle cap has a seal in it, and the emergency cock cannot be inserted unless the seal is removed. Follow the steps below to remove the seal from the bottle. 1. Find a level surface and place the bottle so that the bottom is level. <bb> 120,124,222,248< / bb> 2. Make sure the bottle cap is facing up. <bb> 120,248,222,448< / bb> 3. Remove the seal from the bottle cap. <bb> 392,397,785,729< / bb> 4. Insert the emergency cock into the top of the bottle cap. <bb> 390,840,860,1244< / bb> You can also release water by inserting the emergency cock into the bottle cap and turning the lever to the right. <bb> 949,391,1327,760< / bb>

[0089] <Answer Acquisition Step (Third Embodiment)> In step S702, the control unit 104 of the server 10 executes an FAQ specifying step of specifying predetermined FAQ data based on the question A received in the question receiving step. The control unit 104 of the server 10 executes an element specifying step of specifying one or more predetermined document elements stored in association with the predetermined FAQ data in the associating step.

[0090] In the answer acquisition step (second embodiment), the FAQ data included in the place of ${FAQ data} does not need to be all of the FAQ information (question data, answer data) stored in the FAQ table 1015, but may be one or more pieces of FAQ information (question data, answer data) that are part of the FAQ table 1015. In particular, in the FAQ answer processing step, it is preferable to include FAQ information (question data, answer data) that is similar to and related to question A. For example, the control unit 104 of the server 10 may calculate the similarity between question A and each piece of question data included in the FAQ table 1015, and extract FAQ information corresponding to question data whose similarity is equal to or greater than a predetermined threshold. The following methods can be used to calculate the similarity. (1) Method using embedded representation of sentences: The control unit 104 of the server 10 inputs question A and each question data into a sentence embedding model and calculates the cosine similarity between the resulting embedding vectors. BERT, Sentence-BERT, or other pre-trained language models can be used as the embedding model. FAQ information corresponding to question data whose cosine similarity exceeds a predetermined threshold (e.g., 0.7 or higher) is extracted. (2) Keyword matching method: The control unit 104 of the server 10 extracts important keywords from question A and each piece of question data, and calculates the similarity based on the number and ratio of common keywords. For example, the control unit 104 calculates the similarity using the Jaccard coefficient or TF-IDF, and extracts question data whose similarity is equal to or greater than a threshold. (3) Method using morphological analysis and word weighting: The control unit 104 of the server 10 performs morphological analysis on question A and each question data, assigns high weights to important parts of speech such as nouns and verbs, and calculates the similarity. This method enables similarity calculations that more accurately reflect the structure and meaning of sentences. The control unit 104 of the server 10 identifies FAQ information having question data similar to question A from the FAQ table 1015 by using any of the above methods or a combination of these methods, and includes the identified FAQ information in the ${FAQ data} section of the third prompt. It is also desirable to set an appropriate number of FAQ information items to be extracted, such as extracting the top N (for example, the top 5) FAQ information items. By considering the character limit for the prompt and processing efficiency, and including a necessary and sufficient number of FAQ information items in descending order of similarity, the large-scale language model can generate an appropriate answer A for question A.

[0091] Furthermore, the document elements included in the ${document element} location do not need to be all the document elements stored in element table 1014, and may be one or more document elements (element data) that are part of the document elements stored in element table 1014. The document elements included in the ${document element} location are preferably one or more documents (element data) associated with one or more pieces of FAQ information (question data, answer data) via association table 1021. Specifically, control unit 104 of server 10 searches the FAQ ID item in association table 1021 based on the FAQ ID of one or more pieces of FAQ information, and acquires one or more predetermined element IDs. Control unit 104 of server 10 searches the element ID item in element table 1014 based on the predetermined element ID, and acquires one or more predetermined element data. Control unit 104 of server 10 creates a third prompt by adding the acquired predetermined element data to the ${document element} location of the second instruction. This makes it possible to create an answer A that includes more suitable answer data that references the image areas included in the original document based on the specified FAQ information. In the answer acquisition step (second embodiment), the input information is the FAQ information that serves as a starting point and a document element that includes information that references the image areas extracted from the original document, and more suitable answer data that references the image areas included in the original document can be included.

[0092] The control unit 104 of the server 10 sends a request including the created third prompt to the URL endpoint provided by the generation AI 50. The generation AI 50 outputs a response to the request to the server 10.

[0093] An example of the third response output by the generation AI 50 is shown below. The third response includes answer A to question A. Answer A is not the answer data stored in the FAQ table 1015, but is generated and provided by the generation AI 50 while referring to the document elements in the element table 1014, and is more suitable for answer A received from the user. Specifically, the third response (third embodiment) allows the generation AI 50 to generate an answer A to question A by considering only FAQ information and document elements that are more relevant to question A. This makes it possible to prevent the generation AI 50 from generating an answer A that is irrelevant to question A. <Third Response (Third Embodiment)> #Question data: I can't insert the emergency cock into the bottle #Response data: The bottle cap has a seal in it, and the emergency cock cannot be inserted unless the seal is removed. Follow the steps below to remove the seal from the bottle. 1. Stand the bottle upright with the cap facing upwards. 2. Remove the seal from the bottle cap. <bb> 392,397,785,729< / bb> 2. Insert the emergency cock into the top of the bottle cap. <bb> 390,840,860,1244< / bb> You can also release water by inserting the emergency cock into the bottle cap and turning the lever to the right. <bb> 949,391,1327,760< / bb>

[0094] In the answer acquisition step (second embodiment) and the answer acquisition step (third embodiment), an example in which association information is acquired from the association table 1021 has been disclosed as an example, but the present invention is not limited to this. For example, an association process and an FAQ creation process may be executed in the FAQ answering process, and an answer may be acquired using the association information, document elements, FAQ information, etc. obtained thereby. In other words, the control unit 104 of the server 10 may acquire document information such as a manual (which may be received from the user rather than from the document table 1013), and based on the document, execute all or part of the association process and FAQ creation process, and based on the execution results, execute the answer acquisition step (second embodiment) and the answer acquisition step (third embodiment).

[0095] In step S703, the control unit 104 of the server 10 executes an element presentation step of presenting information based on the predetermined document element identified in the element identification step. The element presentation step includes a step of presenting an image included in the original document based on information that identifies an image area included in the predetermined document element identified in the element identification step. Specifically, the control unit 104 of the server 10 transmits one or more document elements included in the answer A acquired in step S702 to the user terminal 20. The user terminal 20 displays the acquired one or more document elements on the display 2081 of the user terminal 20, and presents them to the user. Specifically, the answer output field D302 displays not only the text data included in the answer data, but also the image area tag " <bb> ···< / bb> The image etc. specified by " can also be displayed and presented on the display 2081 of the user terminal 20.

[0096] In step S703, the control unit 104 of the server 10 executes an answer presentation step of presenting to the user information based on the output content from the large-scale language model in response to the third prompt. Specifically, in step S702, in the answer acquisition step (second embodiment) and the answer acquisition step (third embodiment), information based on the answer A acquired from the generation AI 50 is displayed and presented in the answer output field D302 of the question page D3.

[0097] FIG. 14 is an example of a screen showing the operation of the FAQ answer processing. The control unit 204 of the user terminal 20 displays the FAQ answer screen of question page D3 on the display 2081 of the user terminal 20. The FAQ answer screen of question page D3 includes the question received from the user (the question entered in the question input field D301) and answers D311 to D315. The answers D311 to D315 are example answers presented based on one or more answers acquired in answer acquisition step S702. D311 to D315 may be presented based on one answer (response), or each may be presented based on multiple answers (responses). Image elements D312 and D314 may be stored in an image data format such as jpg, png, or svg in a column (not shown) of FAQ table 1015, or document table 1013 may be referenced based on the document ID to obtain an area extracted based on an image tag for the document data, and the area may be output to D312 and D314. Any method can be selected as the output method for the image elements.

[0098] <Basic computer hardware configuration> 15 is a block diagram showing the basic hardware configuration of a computer 90. The computer 90 includes at least a processor 901, a main memory device 902, an auxiliary memory device 903, and a communication IF 991 (interface), which are electrically connected to one another by a communication bus 921.

[0099] The processor 901 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0100] The main memory device 902 is used to temporarily store programs, data to be processed by the programs, etc. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0101] The auxiliary storage device 903 is a storage device for saving data and programs, such as a flash memory, a hard disk drive (HDD), a magneto-optical disk, a CD-ROM, a DVD-ROM, or a semiconductor memory.

[0102] The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using wired or wireless communication standards. The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a direct connection using a USB (Universal Serial Bus) cable, etc.

[0103] It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration across multiple computers 90 and interconnecting them via a network. In this way, the computer 90 is a concept that includes not only a computer 90 housed in a single housing or case, but also a virtualized computer system.

[0104] <Basic functional configuration of computer 90> A description will now be given of the functional configuration of a computer realized by the basic hardware configuration (FIG. 15) of the computer 90. The computer comprises at least the functional units of a control unit, a storage unit, and a communication unit.

[0105] The functional units of the computer 90 can also be realized by distributing all or part of the functional units among multiple computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

[0106] The control unit is realized by the processor 901 reading out various programs stored in the auxiliary storage device 903, expanding them in the main storage device 902, and executing processing in accordance with the programs. The control unit can realize functional units that perform various types of information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0107] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes programs stored in memory. In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions. If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0108] The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Furthermore, the processor 901 can allocate a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 in accordance with the programs. Furthermore, the control unit can cause the processor 901 to execute processes for adding, updating, and deleting data stored in the storage unit in accordance with the various programs.

[0109] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated. Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0110] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0111] Furthermore, the information, data, etc. disclosed in this disclosure are intended to exemplify one concept of the information, data, etc., and are not intended to limit the concept. For example, information, data, etc. also include information, data, etc. that has been converted or summarized. For example, in the case where user data is data related to a user, such as the user's name, date of birth, occupation, etc., the user data may also include text describing an overview of the user that has been converted (summarized) using predetermined rules or other artificial intelligence systems such as generation AI.

[0112] The communication unit is realized by the communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input the information to the control unit. The control unit can cause the processor 901 to execute information processing on the received information in accordance with various programs. In addition, the communication unit can transmit information output from the control unit to other computers 90.

[0113] <About the prompt> A prompt is a query (including text, character strings, images, videos, audio, etc.) that is input to artificial intelligence systems, such as generative AI and large-scale language models. By inputting a prompt to an artificial intelligence system, a user can instruct the system on how to process information. A user can input a prompt to the AI system so that the AI system will output a desired result. Note that the prompt does not have to be a character string, but can be an image, video, audio, or other prompt. For example, a user's gesture, voice instruction, or other prompt can also be a prompt.

[0114] <About creating prompts> A prompt can be input by a user's input operation on a user terminal. Also, a document file, an image, a video, an audio, etc. can be uploaded and input as a prompt. A user can create a prompt by combining an instruction input by inputting on a user terminal with other instructions, data, etc. In this case, the prompt may include one or more instruction sentences, data, etc. Also, the prompt may include only one or more instruction sentences or one or more data. A prompt can be created by including part of an instruction, data, etc. with another instruction, data, etc. A prompt can be created by inserting a portion of an instruction, data, etc. into another instruction, data, etc. Prompts can be created by combining instructions, data, etc. with other instructions, data, etc. A prompt can be created by combining instructions, data, etc. with other instructions, data, etc. A prompt can be created by adding instructions, data, etc. to other instructions, data, etc. In this disclosure, expressions such as "include," "insert," "combine," "combine," and "append" used when creating prompts from directives, data, and the like are used as terms that refer to the same information processing process. In other words, these can be treated as the same information processing process in that the same prompt is created based on one or more directives, data, and the like. For example, the term "include" encompasses information processing such as "insert," "combine," "combine," and "append." Similarly, "insert," "combine," "combine," and "append" also encompass information processing such as "include," "insert," "combine," "combine," and "append," respectively.

[0115] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0116] (Appendix 1) A program to be executed by a computer having a processor and a storage unit, wherein the processor executes the following steps: a document acquisition step of acquiring a document; an element acquisition step (S102) of acquiring one or more document elements contained in the document; an FAQ acquisition step (S105) of acquiring one or more FAQ data consisting of questions related to the document and their answers; a first instruction input step (S105) of inputting a first prompt to a large-scale language model based on the one or more document elements and the one or more FAQ data, the first prompt including an instruction to identify at least a portion of one or more FAQ data related to at least a portion of the one or more document elements; and an associating step (S106) of storing at least a portion of the one or more document elements in association with at least a portion of the identified one or more FAQ data based on output content from the large-scale language model in response to the first prompt. This allows each FAQ about a specific product to be stored in association with the document elements that make up a document such as a manual for the specific product. Questions about the FAQ can be answered by referring to the document elements of the manual.

[0117] (Appendix 2) 10. The program of claim 1, wherein the document element includes information identifying an image region of an image included in the original document. This makes it possible to create FAQ data that can provide answers to questions about FAQs, including images contained in documents such as product manuals.

[0118] (Appendix 3) 10. The program of claim 1, wherein the document element includes string information that references an image area of an image included in the original document. This makes it possible to create FAQ data that can provide answers to questions about FAQs, including images contained in documents such as product manuals.

[0119] (Appendix 4) A program described in Appendix 1, which executes a learning step (S503) in which the program inputs question A of FAQ data A and document element A associated with question A and trains a learning model so that answer A of FAQ data A is output. This makes it possible to create a learning model that can output answers to FAQ questions that include images contained in documents such as product manuals.

[0120] (Appendix 5) 10. The program according to claim 1, wherein the processor executes a second instruction input step (S302) of inputting a second prompt to a large-scale language model, the second prompt including an instruction to output an answer B of the FAQ data B that references the document element B for the question B, based on the question B of the FAQ data B and the document element B associated with the question B, and an answer storage step (S304) of storing the answer B of the FAQ data B that references the document element B for the question B in association with the question B, based on the output content from the large-scale language model in response to the second prompt. This allows for storing, for each FAQ about a specific product, answers to questions about the FAQ, including answers that refer to document elements such as manuals, in association with each other.

[0121] (Appendix 6) A program as described in Appendix 1, in which a processor executes a question receiving step of receiving question A from a user, an FAQ identification step (S702) of identifying specified FAQ data based on question A received in the question receiving step, an element identification step (S702) of identifying one or more specified document elements stored in association with the specified FAQ data in an association step, and an element presentation step (S703) of presenting information based on the specified document elements identified in the element identification step. This allows the user to obtain an appropriate FAQ answer to question A, and also to check document content (document elements) related to the FAQ in a document such as a manual. If the document element contains information that references images, drawings, etc. in the manual, the user can obtain not only a text answer from the FAQ, but also an answer that references images, drawings, etc. in the manual.

[0122] (Appendix 7) The program described in Appendix 6, in which a processor executes a third instruction input step (S702) of inputting a third prompt to a large-scale language model, the third prompt including an instruction to output answer A to question A received in the question receiving step based on an answer included in predetermined FAQ data and predetermined document elements, and an answer presentation step (S703) of presenting information to a user based on the output content from the large-scale language model in response to the third prompt. This allows the user to obtain a specific answer A to the question A that he or she has entered by referring to an appropriate FAQ and a document such as a manual. Answer A is not limited to general answers from FAQs, manuals, etc., but is an individual, specific answer output by a large-scale language model, and the user can obtain an answer that is convenient for them.

[0123] (Appendix 8) The program described in Appendix 6, wherein the specified document element includes information that identifies an image area of an image included in the original document, and the element presentation step (S703) includes a step of presenting an image included in the original document based on information that identifies the image area included in the specified document element identified in the element identification step. This allows the user to obtain a suitable answer to the question A that he or she has entered, including an image that is described in a document such as a manual. The user can get a better answer to the question.

[0124] (Appendix 9) The program described in Appendix 1, wherein the element acquisition step (S102) includes a fourth instruction input step of inputting a fourth prompt to the large-scale language model, the fourth prompt including instructions for extracting information identifying the document element contained in the document and the image area of the image related to the document element, and a step of extracting and acquiring the document element contained in the document based on the output content from the large-scale language model in response to the fourth prompt. This makes it possible to extract document elements and images related to the document elements from documents such as manuals, etc., and improves the efficiency of editing document elements.

[0125] (Appendix 10) A program as described in Appendix 1, wherein the element acquisition step (S102) of the processor includes the steps of presenting the extracted one or more document elements to the user in an editable manner, accepting edits to the presented one or more document elements, and acquiring the edited one or more document elements. This allows the user to create more suitable document elements by editing the document elements extracted from the large-scale language model and the images related to the document elements.

[0126] (Appendix 11) A method executed by an information processing device having a processor and a memory unit, wherein the processor executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 9. This allows each FAQ about a specific product to be stored in association with the document elements that make up a document such as a manual for the specific product. Questions about the FAQ can be answered by referring to the document elements of the manual.

[0127] (Appendix 12) An information processing device comprising a processor and a storage unit, wherein the processor executes all of the steps executed in the invention according to any one of Supplementary Note 1 to Supplementary Note 9. This allows each FAQ about a specific product to be stored in association with the document elements that make up a document such as a manual for the specific product. Questions about the FAQ can be answered by referring to the document elements of the manual.

[0128] (Appendix 13) A system comprising means for performing all steps performed in any of the inventions according to any one of appendixes 1 to 9. This allows each FAQ about a specific product to be stored in association with the document elements that make up a document such as a manual for the specific product. Questions about the FAQ can be answered by referring to the document elements of the manual. [Explanation of symbols]

[0129] 1 System, 10 Server, 101 Memory Unit, 104 Control Unit, 106 Input Device, 108 Output Device, 20 User Terminal, 201 Memory Unit, 204 Control Unit, 206 Input Device, 208 Output Device, 50 Generation AI, 501 Memory Unit, 504 Control Unit, 506 Input Device, 508 Output Device

Claims

1. A program to be executed by a computer having a processor and a storage unit, the processor: a document acquisition step of acquiring a document; an element acquisition step of acquiring one or more document elements included in the document; an FAQ acquisition step of acquiring one or more FAQ data items consisting of questions related to the document and their answers; a first instruction input step of inputting a first prompt including an instruction to identify at least a portion of the one or more FAQ data related to at least a portion of the one or more document elements based on the one or more document elements and the one or more FAQ data to a large-scale language model; an associating step of associating at least a portion of the one or more document elements with at least a portion of the identified one or more FAQ data based on output content from the large-scale language model in response to the first prompt, and storing the associated data; To execute program.

2. The document element includes information identifying an image region of an image included in the original document. The program according to claim 1.

3. The document element includes string information that references an image area of an image included in the original document. The program according to claim 1.

4. The program, a learning step of training a learning model using a question A of FAQ data A and a document element A associated with the question A as input so as to output an answer A to the FAQ data A; To execute The program according to claim 1.

5. the processor: a second instruction input step of inputting a second prompt including an instruction to output an answer B of the FAQ data B to the question B, the answer B referring to the document element B, based on the question B of the FAQ data B and the document element B associated with the question B, to the large-scale language model; an answer storage step of storing an answer B of FAQ data B that references document element B to question B in association with question B based on output content from the large-scale language model in response to the second prompt; To execute The program according to claim 1.

6. the processor: a question receiving step of receiving a question A from a user; an FAQ specifying step of specifying predetermined FAQ data based on the question A received in the question receiving step; an element specifying step of specifying one or more predetermined document elements stored in association with the predetermined FAQ data in the associating step; an element presentation step of presenting information based on the predetermined document element identified in the element identification step; To execute The program according to claim 1.

7. the processor: a third instruction input step of inputting a third prompt including an instruction to output an answer A to the question A received in the question receiving step to a large-scale language model based on the answer included in the predetermined FAQ data and the predetermined document element; an answer presentation step of presenting to the user information based on output content from the large-scale language model in response to the third prompt; To execute The program according to claim 6.

8. the predetermined document element includes information identifying an image area of an image included in the original document; The element presenting step includes: presenting an image included in the original document based on information specifying the image area included in the predetermined document element specified in the element specifying step; The program according to claim 6.

9. The element acquisition step includes: a fourth instruction input step of inputting a fourth prompt including instructions for extracting information identifying a document, a document element included in the document, and an image region of an image associated with the document element to the large-scale language model; extracting and obtaining document elements included in the document based on the output content from the large-scale language model in response to the fourth prompt; Including, The program according to claim 1.

10. the processor: The element acquisition step includes: presenting the extracted one or more document elements to a user in an editable manner; accepting edits to the one or more submitted document elements; obtaining the one or more edited document elements; Including, The program according to claim 1.

11. A method executed by an information processing device including a processor and a storage unit, wherein the processor executes all of the steps executed in the invention according to any one of claims 1 to 9.

12. 10. An information processing device comprising a processor and a storage unit, wherein the processor executes all of the steps executed in any one of the inventions according to claims 1 to 9.

13. A system comprising means for executing all steps performed in any one of the inventions according to claims 1 to 9.

Images