Program, method, and information processing device

Abstract

In a system that generates answers to questions, there is a need for a technology that makes it easier for users to determine whether the generated answers are valid. [Solution] The program causes a processor to perform the following steps: accepting a question input from a user; extracting multiple pieces of content by referencing data in a graph-type data structure based on the content of the question accepted in the accepting step; generating an answer using the content of the question and the multiple pieces of content extracted in the extracting step; and presenting the answer generated in the generating step together with the multiple pieces of content that served as the basis for generating the answer; in the presenting step, the multiple pieces of content are presented in order of priority based on the reference relationships in the data in the graph-type data structure for the content that served as the basis.

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, and an information processing device. [Background technology]

[0002] When searching for content, the content is evaluated based on the search term and presented as a search result. Content is also recommended to the user.

[0003] Patent Document 1 describes a "recommendation information generating device that generates recommendation information."

[0004] Patent Document 1 points out as an issue that "there is a problem that recommendation information cannot be determined unless the access history increases. In other words, when selecting users with similar tastes, it is necessary for them to select the same content at least once, so if they do not select the same content, they will not be recognized as similar users. Therefore, it is necessary to accumulate a certain amount of access history. Therefore, it has been difficult to generate recommendation information based on similar users when each user has little access history."

[0005] Patent Document 1 describes the technical means for solving the problem of "being able to accurately select similar users even when there is little historical information" as follows: "acquiring a user characteristic vector for each user," "calculating the similarity between users based on the acquired user characteristic vector for each user," "selecting similar users who are similar to one user based on the calculated similarity," and "generating recommendation information for one user based on the characteristic vectors of the selected similar users." [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-252177 Summary of the Invention [Problem to be solved by the invention]

[0007] Some systems that generate answers to questions using large-scale language models present the answers to users while also showing the information sources that served as the basis for generating the answers. In these systems, information sources are extracted according to the content of the question. However, when generating answers to questions using large-scale language models, inappropriate answers may be generated due to bias in the training data used to train the large-scale language models, for example.

[0008] Therefore, in a system that generates answers to questions, there is a need for a technology that makes it easier for users to determine whether the generated answers are valid. [Means for solving the problem]

[0009] According to one embodiment of the present disclosure, a program for operating a computer including one or more computer processors is provided. A storage unit is configured to store data in a graph-type data structure that defines reference relationships between multiple types of content. The program causes the one or more computer processors to execute the following steps: accepting a question input from a user; extracting multiple pieces of content by referencing the data in the graph-type data structure based on the content of the question accepted in the accepting step; generating an answer using the content of the question and the multiple pieces of content extracted in the extracting step; and presenting the answer generated in the generating step together with the multiple pieces of content that served as a basis for generating the answer. In the presenting step, the multiple pieces of content are presented in order of priority based on the reference relationships in the data in the graph-type data structure for the content that served as the basis. [Effects of the Invention]

[0010] According to the present disclosure, in a system that generates answers to questions, it is possible to make it easier for a user to determine whether the generated answers are valid. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing the configuration of the system 1. [Figure 2] FIG. 2 is a diagram showing the configuration of the server 20. As shown in FIG. [Figure 3] FIG. 3 is a diagram showing the configuration of the terminal 10. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing the data structure of the user database 211. As shown in FIG. [Figure 5] FIG. 5 is a diagram showing the data structure of the legal content database 212. As shown in FIG. [Figure 6] FIG. 6 is a diagram showing the data structure of the content usage history database 213. As shown in FIG. [Figure 7] FIG. 7 is a diagram showing the data structure of the graph type data structure 214. As shown in FIG. [Figure 8] FIG. 8 is a diagram showing the flow of processing for creating a graph database using information on multiple legal content items. [Figure 9] FIG. 9 is a diagram showing a processing flow using a law-related graph database. [Figure 10] FIG. 10 is an example of an operation screen that displays the answer to a user's question generated by referring to a legally related graph-type data structure 214. [Figure 11] FIG. 11 is a diagram showing the flow of processing for identifying laws and regulations corresponding to a user's question and presenting the identified laws and regulations to the user. [Figure 12] FIG. 12 is an example of an operation screen that presents to the user laws and regulations that have been identified as corresponding to the user's question. [Figure 13]FIG. 13 is a diagram showing the flow of processing for generating a database that specifies information on legal issues for information on a plurality of legal content items. [Figure 14] FIG. 14 is a diagram showing the flow of a process for searching for content on a legal issue that corresponds to a user's question. [Figure 15] FIG. 15 shows an example of an operation screen that responds to a user's question with the results of searching various databases based on the corresponding legal points. [Figure 16] FIG. 16 is a diagram showing the flow of processing for identifying laws and regulations corresponding to portions of content that are identified as the basis for answers to questions, and outputting the latest laws and regulations. [Figure 17] FIG. 17 shows an example of a screen on an operation screen for viewing content, which displays the content of the content and a table of contents as an outline of the content. [Figure 18] FIG. 18 shows an example of a screen on an operation screen for viewing content, which displays the content details and other content related to the displayed content. [Figure 19] FIG. 19 shows an example of a screen on an operation screen for viewing content that displays information on laws and legal amendments as other content related to the displayed content. [Figure 20] FIG. 20 is a diagram showing the flow of processing for displaying a part of another content related to the displayed content while presenting an operation screen for viewing the content. [Figure 21] FIG. 21 shows the process flow for generating an answer by determining whether to use a first mode for generating an answer quickly or a second mode for generating an answer slowly and in detail, depending on the content of the question. [Figure 22] Figure 22 is an example of a screen that analyzes the content of the question entered and displays the results of a decision made before accepting the operation to send the question, as to whether to answer in the first mode (quick answer mode) or the second mode (detailed answer mode). [Figure 23]FIG. 23 shows an example of a screen that displays the determination result of the mode for generating an answer on the operation screen during the period from when a question is sent until an answer is generated using an external information processing system. [Figure 24] FIG. 24 shows an example of a screen that displays the determination result of the mode for generating an answer on the screen that displays the answer generated for the question. [Figure 25] FIG. 25 is a diagram showing the flow of a process for accepting a follow-up question that is a more in-depth investigation of a question input by a user. [Figure 26] FIG. 26 shows an example of a screen that accepts follow-up questions that are further clarified regarding the question entered by the user. [Figure 27] FIG. 27 is a diagram showing the flow of a process for presenting an answer to a question while displaying a list of content related to the answer, and changing the display format of the list in response to a user operation. [Figure 28] FIG. 28 shows an example of a screen displaying a list in a first manner that makes it easier to grasp the overall picture of the content when presenting an answer to a question and displaying a list of content related to the answer. [Figure 29] Figure 29 is an example of a screen displaying a list in a second manner in which, when presenting an answer to a question and displaying a list of content related to the answer, the amount of information in each piece of content is increased to assist in selecting content for further viewing. [Figure 30] FIG. 30 shows a graph-type data structure for each type of content, such as laws, precedents, books, and guidelines. [Figure 31] FIG. 31 is a diagram showing the data structure of the abbreviated name dictionary database 215 that defines rules for abbreviating laws and regulations. [Figure 32] Figure 32 shows the process flow for identifying the places in the content where laws and regulations are mentioned, depending on the various ways in which laws and regulations are written, and defining a reference relationship with information about the laws and regulations having a graph-type data structure to generate a graph-type data structure. [Figure 33]Figure 33 shows the processing flow for searching various contents in response to a question, referencing a graph-type data structure for the search results, extracting related contents, and generating an answer to the question. [Figure 34] FIG. 34 is a diagram showing the flow of processing for generating a graph database in which reference relationships are defined between each clause of the company regulations and a graph data structure that associates laws, precedents, books, and guidelines. [Figure 35] FIG. 35 shows the flow of processing in which each clause of the company regulations is associated with laws and precedents, and information on amendments to laws and precedents is acquired to extract clauses of the company regulations that will be affected. [Figure 36] FIG. 36 shows the flow of processing for generating an answer to a question based on laws, precedents, books, guidelines, and also company regulations. [Figure 37] FIG. 37 is a diagram showing the flow of processing for detecting questions from in-house posts and conversations and generating answers. [Figure 38] Figure 38 shows the process flow for analyzing the reference relationships in a graph-type data structure, determining the priority for presenting multiple types of content that served as the basis for generating an answer to a question, and presenting that content. [Figure 39] FIG. 39 is a diagram showing the flow of processing for evaluating the difficulty level of a document based on the citation relationships in a graph database. [Figure 40] Figure 40 shows the process flow for controlling the difficulty level of documents presented to a user in accordance with the user's browsing habits (difficulty level trends) by utilizing the evaluation results of document difficulty based on a graph database. [Figure 41] FIG. 41 is a diagram showing the flow of processing in which a question is asked, an answer is generated, and the content that served as the basis for generating the answer is presented on an author-by-author basis. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0013] <Outline of embodiment> In the first embodiment, we will explain a method for generating a data structure that defines reference relationships between legal content based on the wording (names of laws, etc.) contained in laws, precedents, legal books, and legal guidelines, and a method for generating answers to user questions using the graph-type data structure generated thereby.

[0014] <1.1 Overall system configuration> FIG. 1 is a diagram showing the configuration of the system 1.

[0015] 1 includes a server 20, a user terminal 10, a case law search service server 91, a law search service server 92, a book browsing service server 93, an information media service server 94, an artificial intelligence (large-scale language model) service server 95 (hereinafter also referred to as the "large-scale language model service server 95"), a business operator server 96, an SNS server 97, and a user terminal 10A. These devices are connected for communication via a network 80.

[0016] In the illustrated example, terminals such as terminal 10 and terminal 10A are shown as terminals used by users of the services provided by server 20, and each user operates a terminal.

[0017] In this embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, a collection of devices can be considered as one "information processing device," and system 1 can be formed as a collection of multiple devices. The way in which multiple functions required to realize system 1 according to this embodiment are allocated to one or multiple pieces of hardware can be determined appropriately in consideration of the processing capacity of each piece of hardware and / or the specifications required for system 1.

[0018] The terminal 10 is a device operated by a user. In this embodiment, the terminal 10 is operated by a user who conducts legal research, judgments, etc. The terminal 10 and the terminal 10A have the same functional configuration. The terminal 10 is realized, for example, as follows. · Handheld devices such as smartphones and tablets Desktop PCs (Personal Computers), laptop PCs Wearable devices worn by users (wristwatches, glasses, etc.) The terminal 10 includes a communication IF (Interface) 12 , an input device 13 , an output device 14 , a memory 15 , a storage 16 , and a processor 19 .

[0019] The communication IF 12 is an interface for inputting and outputting signals so that the terminal 10 can communicate with an external device.

[0020] The input device 13 is a device for receiving input operations from a user (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).

[0021] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0022] The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0023] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0024] The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0025] The server 20 is a device for accepting legal questions and providing users with a service that answers the questions while presenting multiple pieces of content related to the relevant laws. In this embodiment, the server 20 accepts legal questions input from users of the terminals 10 in chat format using free text or voice input, and outputs answers to the accepted questions. When generating answers to the questions, the server 20 may have an artificial intelligence (large-scale language model) service server 95 generate the answers, and provides the service by returning the generated results to the users.

[0026] In this embodiment, the server 20 provides the following services to users. Users who handle legal matters, such as the legal department of a business company, and provide legal advice. Users who do not necessarily perform legal work exclusively, such as business divisions of a business company Users who provide legal advice to clients as a professional, such as law firms The server 20 may accept requests for legal consultation from clients to legal experts by matching users who provide legal consultation as experts, such as law firms, with users who request legal consultation from experts, such as business companies or individuals. For example, the content of the consultation may be posted on a bulletin board that can be viewed by third parties, and the expert's response may also be made public. Alternatively, the content of the consultation may be kept private and not disclosed to third parties, allowing the client to consult with the expert. The server 20 may store such consultation content from the client and the expert's response.

[0027] The server 20 includes a communication IF 22 , an input / output IF 23 , a memory 25 , a storage 26 , and a processor 29 .

[0028] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0029] The input / output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user.

[0030] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0031] The storage 26 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0032] The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0033] The precedent search service server 91 has a database of precedents and allows users to search for judgments made by courts and the like.

[0034] The server 92 for the legal search service has a database of legal documents and allows users to search for legal documents.

[0035] The book browsing service server 93 is a service that allows users to browse electronic content such as magazines, books, etc. For example, by paying a fixed fee periodically, users can browse electronic books in fields such as law.

[0036] The information media service server 94 provides information services, such as collecting and making available blog posts, Q&A sites, news articles, IR information, etc. The information media service server 94 stores information on interpretations based on laws and regulations, as well as guidelines prepared by government agencies and other organizations to publicize operational rules.

[0037] The large-scale language model service server 95 is a server that executes language processing tasks using a language model constructed by a learning process including artificial intelligence (AI). An LLM (Large Language Model) is a server that has previously learned a large amount of large-scale data (text data, etc.), such as web content on the Internet, or a large amount of data stored in a specified database, and can execute various language processing tasks by providing the task.

[0038] The large-scale language model service server 95 accepts prompt inputs such as text, images, and voice, and generates and responds to the prompts. Examples of LLMs include GPT-3, GPT-4, and GPT-4o developed by OpenAI, and Gemini developed by Google.

[0039] The business's server 96 stores data generated in the course of business activities. This data has access rights, and access to the data is restricted to users belonging to the business and external users not belonging to the business.

[0040] The SNS server 97 provides services that encourage interaction between users, such as a service that allows users to view each other's posts. For example, there are services that allow users to view posts by searching the Internet even if they do not have a user account on the SNS, and services that allow users to view posts by other users if they have a user account.

[0041] <1.2 Functional configuration of server 20> 2 is a diagram showing the configuration of the server 20. As shown in FIG. 2, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0042] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0043] The storage unit 202 stores various databases such as a user database 211, a legal content database 212, a content usage history database 213, a graph-type data structure 214, an abbreviation notation dictionary database 215, a prompt database 218, and a legal term database 219.

[0044] The user database 211 is a database that manages information about each user, as will be described in detail later.

[0045] The legal content database 212 is a database that stores information on legal content, as will be described in detail later.

[0046] The content usage history database 213 is a database of the history of users' usage of legal content, as will be described in detail later.

[0047] The graph-type data structure 214 illustrates a graph-type data structure in which reference relationships are defined for parts of each content.

[0048] The abbreviation notation dictionary database 215 is a database that manages rules for the abbreviation notation of laws and regulations, as will be described in detail later.

[0049] The prompt database 218 is a database that manages templates of prompts to be sent to the server 95 of the large-scale language model service.

[0050] The legal term database 219 is a database of legal terms, including, for example, the names of issues set forth in laws and regulations (such as "claims," ​​"debts," "damages," and "risk allocation"), terms set forth in legal precedents, and the like.

[0051] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. By operating in accordance with the program, the control unit 203 performs functions shown as a reception control module 2041, a transmission control module 2042, a user management module 2043, a question processing module 2044, an LLM usage module 2045, a data structure definition module 2046, and a content presentation module 2047.

[0052] The reception control module 2041 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0053] The transmission control module 2042 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0054] The user management module 2043 is a module for managing information about each user who uses the system 1. Specifically, the user management module 2043 accepts registration of information about each user and updates the user database 211.

[0055] The question processing module 2044 is a program module that generates an answer to a question input by a user while searching the content managed in the legal content database 212 based on the graph-type data structure 214.

[0056] The LLM usage module 2045 is a program module that generates a prompt and sends the generated prompt to the large-scale language model service server 95, thereby generating a response to the prompt.

[0057] The data structure definition module 2046 is a program module that defines a graph-type data structure 214 based on the content managed in the legal content database 212 .

[0058] The content presentation module 2047 is a program module that provides the user with an operation screen for viewing content managed by each device, such as the graph-type data structure 214, the legal content database 212, the case law search service server 91, the law search service server 92, the book viewing service server 93, and the information media service server 94, and performs processing to allow the user to view the content in accordance with their operation.

[0059] <1.3 Configuration of Terminal 10> FIG. 3 is a diagram showing the configuration of the terminal 10. As shown in FIG.

[0060] As shown in FIG. 3, the terminal 10 includes multiple antennas (antenna 111, antenna 112), communication units (first communication unit 120, second communication unit 121) corresponding to the respective antennas, an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a memory unit 180, and a control unit 190. The terminal 10 also has functions and configurations (e.g., a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not specifically shown in FIG. 3. As shown in FIG. 3, the blocks included in the terminal 10 are electrically connected by a bus or the like.

[0061] The antenna 111 emits a signal emitted by the terminal 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first communication unit 120.

[0062] The antenna 112 emits a signal emitted by the terminal 10 as a radio wave. The antenna 112 also receives a radio wave from space and provides the received signal to the second communication unit 121.

[0063] The first communication unit 120 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 111 so that the terminal 10 can communicate with other wireless devices. The second communication unit 121 performs modulation / demodulation processing and the like for transmitting and receiving signals via the antenna 112 so that the terminal 10 can communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are communication modules including a tuner, a received signal strength indicator (RSSI) calculation circuit, a cyclic redundancy check (CRC) calculation circuit, a high-frequency circuit, and the like. The first communication unit 120 and the second communication unit 121 perform modulation / demodulation, frequency conversion, and the like for wireless signals transmitted and received by the terminal 10, and provide the received signals to the control unit 190.

[0064] Input device 130 has a mechanism for accepting input operations by a user. Specifically, input device 130 is configured as a touch screen and includes touch-sensitive device 131. Touch-sensitive device 131 accepts input operations by a user of terminal 10. Touch-sensitive device 131 detects the user's touch position on the touch panel, for example, by using a capacitive touch panel. Touch-sensitive device 131 outputs a signal indicating the user's touch position detected by the touch panel to control unit 190 as an input operation.

[0065] The display 132 displays data such as images, videos, and text under the control of the control unit 190. The display 132 is realized by, for example, an LCD, an organic EL display, or the like.

[0066] The audio processing unit 140 modulates and demodulates audio signals. The audio processing unit 140 modulates a signal provided from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is realized, for example, by a processor for audio processing. The microphone 141 accepts audio input and provides an audio signal corresponding to the audio input to the audio processing unit 140. The speaker 142 converts the audio signal provided from the audio processing unit 140 into audio and outputs the audio to the outside of the terminal 10.

[0067] The location information sensor 150 is a sensor that detects the location of the terminal 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. The satellite positioning system receives signals from at least three or four satellites, and detects the current location of the terminal 10 equipped with the GPS module based on the received signals.

[0068] The camera 160 is a device that receives light with a light receiving element and outputs the received light as a captured image. The camera 160 is, for example, a depth camera that can detect the distance from the camera 160 to a subject being photographed.

[0069] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10 .

[0070] The storage unit 180 is configured with, for example, a flash memory, and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.

[0071] The control unit 190 controls the operation of the terminal 10 by reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 is, for example, an application processor. By operating in accordance with the program, the control unit 190 fulfills the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a storage control unit 195.

[0072] Operation acceptance unit 191 performs processing to accept a user's input operation to an input device such as touch-sensitive device 131. Operation acceptance unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation, based on information about the coordinates where the user has touched touch-sensitive device 131 with a finger or the like.

[0073] The transmitting / receiving unit 192 performs processing for the terminal 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol.

[0074] The data processing unit 193 performs calculations on the data that the terminal 10 has received as input in accordance with a program, and outputs the calculation results to a memory or the like.

[0075] The notification control unit 194 performs processing for displaying a display image on the display 132, processing for outputting sound from the speaker 142, and processing for generating vibrations.

[0076] The storage control unit 195 controls the storage of data in the storage unit 180 .

[0077] A description will be given of various types of information stored in storage unit 180. In one aspect, storage unit 180 stores various types of information such as user information 181.

[0078] The user information 181 is information about a user who uses the services of the server 20 .

[0079] <2 Data Structure> 4 is a diagram showing the data structure of the user database 211. The user database 211 includes the following items: "User ID," "Name," "Email address," "Business ID," "Department," "Job title," "Date of employment," "Date of retirement," and "Qualifications held."

[0080] The item "user ID" is information that identifies each user.

[0081] The item "Name" is information indicating the name of the user.

[0082] The item "email address" is information about the email address used to contact the user.

[0083] Specifically, the item "email address" includes information about an email address as information for identifying a user, in order to accept the user's login to the service provided by the server 20.

[0084] The item "business ID" is information that identifies the organization to which the user belongs.

[0085] Specifically, the item "business ID" is information for identifying the organization to which the user belongs, and the following organizations are possible. Business companies ·Law firm The item "Department" is information about the department to which the user belongs.

[0086] Specifically, the item "Department" may include the following information regarding the department to which the user belongs: Departments such as the legal department that are expected to carry out legal work - Business divisions, sales divisions, etc. that are not expected to have dedicated legal affairs staff The item "position" is information about the position of the user.

[0087] Specifically, the item "job title" may include the following information as the user's job title: Position with decision-making authority (management, etc.) A position that does not have decision-making authority but demonstrates expertise and a role of assisting the administrative department Not holding a position The item "Date of joining" is information about the date on which the user joined the organization.

[0088] Specifically, the item "Date of employment" may include the following information: Start date if already a member The date when a person is scheduled to join an organization but has not yet joined (it is possible that the person has decided to join the organization but has not yet joined, and therefore has not been assigned permission to view organizational information) · The employee is assigned to the company but the start date has not been finalized (the start date may still be being adjusted) The item "Resignation Date" is information about the date on which the user left the organization and left employment.

[0089] Specifically, the item "retirement date" may include the following information: Date of leaving the company (when you leave the company, you may lose the authority to view organizational information) ·Currently employed The item "Qualifications held" is information indicating qualifications held by the user.

[0090] Specifically, the item "Qualifications held" is information for identifying the qualifications held by the user, and may include the following qualifications. · National qualifications such as lawyer, patent attorney, and certified public accountant that demonstrate expertise A qualification recognized by a business, general incorporated association, or other organization as a qualification that indicates expertise FIG. 5 is a diagram showing the data structure of the legal content database 212. As shown in FIG.

[0091] The item "content ID" is information that identifies each piece of legal content.

[0092] The item "type" is information on the type of each legal content.

[0093] The item "Type" may include the following types of legal content: ·Laws ·Case Law Legal books ·Websites containing legal information The item "information source" is information about the information source from which the content is provided.

[0094] The item "information source" may include the following as information sources: ·Law search service server 92 Case law search service server 91 · A book browsing service server 93 that provides legal books for browsing · Information media service server 94, which is a website that contains legal information The item "title" is information about the name of the content.

[0095] The item "Title" may include the following as the name of the content: In the case of laws and regulations, the name of the law, the article number, the heading of the article, etc. In the case of precedents, case number, name of the case, etc. - In the case of legal books, the title of the book, etc. In the case of legal guidelines, the name of the guidelines, etc. The item "content tag" is information about the name of the tag given to the content.

[0096] The item "content tag" may include the following as information about tags assigned to content. Tags that represent the content of the content extracted by the large-scale language model service server 95 by summarizing data that indicates the content of the content using keywords, a certain number of characters, etc. A list of tags to be assigned to the entire content is held in advance in the server 20, and tags identified by the server 95 of the large-scale language model service from the tags included in the list based on the content of the content. The item "data" is information about the data file of the content.

[0097] The item "tagged location" is information on the location of a tag set at an arbitrary location of data constituting the content, using the name of a law, precedent, legal book, legal guideline, etc. as a tag.

[0098] The item "Tagged location" may include the following as tag setting locations: In the case of laws and regulations, tags may be set in association with specific clause numbers, etc. In the case of legal precedents, the judgment may be divided into multiple blocks based on the paragraphs, headings, arguments of both parties, and the court's decision on each point of contention, and tags may be set at any point in the text within the block. In the case of a legal book, you can divide the book into multiple blocks based on chapters, headings, paragraphs, etc., and set tags at any point in the text within the block. In the case of legal guidelines, the guideline text may be divided into multiple blocks in the same way as above, and tags may be set at any point in the text within the block. For example, if a book on law contains names such as the name of a law, the name of a clause, or the number of a precedent, the server 20 may set a tag at the portion where the name is written.

[0099] The item "Name tag" is information about the name of the tag that has been set.

[0100] The item "Name Tag" may include the following tags attached to the name: Name of the law, article number, etc. Case number and other names of precedents Legal book titles Name of legal guidelines The item "Name tag score" is information on the evaluation result for a tag indicating a name.

[0101] More specifically, the item "Name Tag Score" is a score that is calculated by associating tags with content in various services provided by servers such as a legal search service server 92, a case law search service server 91, a book browsing service server 93, and an information media service server 94, and evaluating the tags based on the history of content viewing based on the tags.

[0102] The server 20 may use the results of evaluating tags for names in this manner to determine the priority of tags corresponding to search terms when a user searches (for example, the higher the evaluation of a tag, the more priority it will have as a tag corresponding to the user's search), and search for legal content based on tags with higher priority.

[0103] The item "Name tag associated content ID" is information that identifies the content that defines the reference relationship based on the location where the tag is set when the same tag is set in multiple contents for the tag set in the item "Name tag".

[0104] In the example shown, the item "Name tag associated content ID" indicates the result of defining a reference relationship between multiple pieces of legal content that each contain the name of a specific law.

[0105] The item "location where issue tag is attached" is information on the location where a tag is attached to an arbitrary location of data that constitutes the content, using a legal issue as a tag.

[0106] The item "Issue Tag" is information on the name of the tag of the set issue.

[0107] The item "Issue Tag Score" is information on the evaluation result of the tag indicating the issue.

[0108] More specifically, the item "Issue Tag Score" is an evaluation of issue tags based on the classification of legal books by issue on the book viewing service server 93, etc., accepting an operation to search for books to be viewed based on an issue, and the user's history of viewing legal books based on the search results.

[0109] The item "Issue Tag Associated Content ID" is information that identifies the content that defines the reference relationship based on the location where the tag is set when the same tag is set in multiple contents for the issue tag set in the item "Issue Tag."

[0110] FIG. 6 is a diagram showing the data structure of the content usage history database 213. As shown in FIG.

[0111] The "Search ID" field is information that identifies each search performed to access legal content.

[0112] The item "Search User ID" is information that identifies the user who searched for legal content.

[0113] The item "Search User ID" may be associated with the item "User ID" of the user database 211.

[0114] The item "search term" is information about the search term used to search for the content.

[0115] More specifically, the item "search term" is a word specified by the user, or a sentence in which the user inputs a question in natural language.

[0116] The item "search issue tag" is information on the name of the issue tag set for the search term entered by the user for a search.

[0117] More specifically, the item "Search Issue Tag" is an issue tag extracted by matching the search term entered by the user with a list of issue tags, or information on legal issues set by a large-scale language model service server 95, etc.

[0118] The item "search date and time" is information about the timing of the search.

[0119] The item "information source" is information in a database that is referenced when a user performs an operation to search for content.

[0120] The item "information source" may include the following as databases referenced by the search: - legal search service server 92 Case law search service server 91 Book browsing service server 93 ·Information media service servers 94 The item "viewed content ID" is information for identifying the content viewed by the user.

[0121] The item "Viewed Content ID" may be associated with the item "Content ID" in the legal content database 212.

[0122] The item "content viewing location" is information about the location where the user viewed the content.

[0123] More specifically, in the case of a book about law, the item "content viewed portion" is information about the portion (page, or page viewed for a certain period of time or more) viewed by the user in the book.

[0124] FIG. 7 is a diagram showing the data structure of the graph type data structure 214. As shown in FIG.

[0125] 7 shows that information on a block of text 214A, which is associated with document A (e.g., a legal book) and includes a passage from document A about a specific provision of a law as its name, is stored in legal content database 212. Similarly, information on a block of text 214B, which is associated with document B (e.g., a guideline) and includes a passage from document B about a specific provision of a law as its name, is stored in legal content database 212.

[0126] In the illustrated example, a reference relationship (edge) is defined between portions (sentences 214A and 214B) of document A and document B that contain the same clause of a law. In this way, data structure definition module 2046 defines, as nodes, portions of multiple contents in legal content database 212 that contain names of laws, precedents, legal books, guidelines, etc., and generates graph-type data structure 214 by defining nodes and edges based on these names. In the illustrated example, a reference relationship is defined between two documents, document A and document B, but the number of documents is not limited to two.

[0127] <3. Operation (First Embodiment)> FIG. 8 is a diagram showing the flow of processing for creating a graph database using information on multiple legal content items.

[0128] In step S821, the data structure definition module 2046 of the server 20 acquires information on legal content and information that is a candidate for tagging by referring to the legal content database 212, the legal terminology database 219, and the like.

[0129] In this way, the server 20 stores information on a plurality of pieces of content related to law in the legal content database 212 of the storage unit 202.

[0130] In step S823, the data structure definition module 2046 of the server 20 identifies a portion of the legal content that includes information identifying laws and regulations, information identifying case law, information identifying legal books, and information identifying legal guidelines.

[0131] In this way, the data structure definition module 2046 defines the following for each of the plurality of legal contents: Information identifying the law; Information identifying the case; Information identifying the law book; Information identifying legal guidelines; A part containing at least one of the above is identified.

[0132] More specifically, the data structure definition module 2046: Information identifying the law, including at least the name or clause of the law; Information to identify the case includes the case number, Information identifying the legal book includes the title of the legal book, Information identifying the legal guideline includes the name of the guideline, Alternatively, a part including at least one of the above may be specified.

[0133] More specifically, the data structure definition module 2046: Information identifying the law, including at least the name or clause of the law; Information to identify the case includes the case number, Information identifying the legal book includes the title of the legal book, Information identifying the legal guideline includes the name of the guideline, Alternatively, a certain amount of text containing at least one of the above may be specified as a part.

[0134] In step S825, the data structure definition module 2046 of the server 20 associates the identified portion of the content with the content and stores the associated portion in the legal content database 212.

[0135] In this way, the data structure definition module 2046 stores the portion identified in step S823 in the storage unit 202 in association with the identified content.

[0136] Here, the data structure definition module 2046 may evaluate a portion based on the portion associated with the content, depending on the usage status of search results in a service that accepts content search operations (e.g., the case law search service server 91, the law search service server 92, the book browsing service server 93, the information media service server 94, etc.). For example, a tag associated with the content may be evaluated based on the search results of the tag in the book browsing service (e.g., the more frequently searched a tag is, the more books are browsed based on the search results for the tag, and the higher the evaluation value may be). The data structure definition module 2046 may associate the evaluation result of the portion with the content and store it in the legal content database 212 (e.g., as an item "name tag score"). For example, when identifying a tag corresponding to a user's question in a graph-structured database, the server 20 may select the tag based on its evaluation value (e.g., prioritize a tag with a high evaluation value as a tag corresponding to the user's question), and search the graph-structured database using the selected tag. This improves the response of searches based on graph-structured tags compared to searching each content included in the legal content database 212 using keywords, while further improving the accuracy of search results (presentation of legal content appropriate to the user).

[0137] By processing the above steps S823 and S825, it is possible to identify the locations where names of laws, precedents, books, guidelines, etc. are written, and to assign tags to the content based on these names.

[0138] In step S827, the data structure definition module 2046 of the server 20 extracts the identified portions of the plurality of contents that are of the same type. The data structure definition module 2046 defines a reference relationship between the extracted portions that are of the same type for each content.

[0139] More specifically, when defining a reference relationship, the data structure definition module 2046 may extract parts of multiple pieces of content that are the same type, and may define the reference relationship between pieces of content by extracting parts that have the same name for the identified parts.

[0140] For example, if the same case law number is written in a first e-book and a second e-book, the location where the case law number is written in the first e-book and the location where the case law number is written in the second e-book are extracted as being of the same type. The data structure definition module 2046 defines a reference relationship between the extracted locations where the case law number is written in the first e-book and the second e-book.

[0141] In this way, the data structure definition module 2046 extracts, from a plurality of pieces of content, pieces of content that have the same identified portion, and defines the reference relationships between the pieces of content for each of the extracted pieces of content, thereby constructing a graph-structured database. For example, the data structure definition module 2046 may store the reference relationships defined for the plurality of pieces of content in the legal content database 212.

[0142] Here, the data structure definition module 2046 may define a reference relationship between portions of a certain amount of text in a plurality of pieces of content.

[0143] As a result, when multiple pieces of content contain the same names of laws, precedents, books, and guidelines, a graph-type data structure that defines the reference relationships between the pieces of content can be generated. For example, if a graph-type data structure is generated based on the name of a law, searching for the name of the law can display search results for multiple information sources, such as the original text of the law, precedents corresponding to the law, books that mention the law, and guidelines that mention the law. Because the search is performed using a graph-type data structure, search responsiveness can be further improved compared to searching these information sources sequentially using the above keywords.

[0144] In step S829, the data structure definition module 2046 of the server 20 outputs information on the reference relationships defined between the multiple pieces of content to the terminal 10. After performing the process of defining the reference relationships between the pieces of content, the data structure definition module 2046 may not output the information to the terminal 10 or the like, or may present the information on the reference relationships defined between the pieces of content in response to a request from the terminal 10 or the like.

[0145] In step S811, the terminal 10 displays information indicating the reference relationships of the contents.

[0146] FIG. 9 is a diagram showing a processing flow using a law-related graph database.

[0147] In step S921, the question processing module 2044 of the server 20 outputs to the terminal 10 an operation screen that accepts an input of a question about law and displays the answer.

[0148] As described above, server 20 holds data structures for information on law-related content (legal content database 212, graph-type data structure 214). Server 20 stores information on a plurality of law-related content items in storage unit 202. Server 20 stores, in storage unit 202, the results of defining reference relationships between pieces of content that are similar in part to each other and that include at least one of information identifying laws and regulations, information identifying legal precedents, information identifying legal books, and information identifying legal guidelines.

[0149] In step S911, the terminal 10 accepts an input of a question about law from the user.

[0150] In step S923, the question processing module 2044 of the server 20 assigns a tag representing the content of the question to the question input by the user of the terminal 10 via the large-scale language model service server 95. For example, the question processing module 2044 generates a prompt that includes, along with the content of the question, at least one of an instruction to summarize the content of the question input by the user, an instruction to refer to legal terms in the legal terminology database 219 or the like to assign a tag to the content of the question, etc. The question processing module 2044 sends the prompt via the LLM usage module 2045 to the large-scale language model service server 95, and assigns a tag to the question by receiving an output result corresponding to the prompt from the large-scale language model service server 95.

[0151] In step S925, the question processing module 2044 of the server 20 uses the tag assigned to the question to search for tagged content by referring to the graph data structure 214. This reduces the amount of processing required to search for tagged content compared to searching a table-type content database, and further improves the response speed of search results.

[0152] In step S927, the question processing module 2044 of the server 20 instructs the server 95 of the large-scale language model service to generate an answer to the question based on the search result, thereby obtaining the generated answer. Specifically, the question processing module 2044 generates a prompt including the search result (legal content searched based on the tag assigned to the question) and the question entered by the user in steps S911 and S923 by referring to the graph data structure 214, and transmits the generated prompt to the server 95 of the large-scale language model service, thereby receiving the answer generated by the server 95 of the large-scale language model service.

[0153] In response to a legal question input from a user, the question processing module 2044 may use the search results of the content searched in step S925 to output the search results to the user terminal 10 without generating an answer to the question using the large-scale language model service server 95. This allows the legal content searched for by the graph data structure 214 in response to the user's question to be returned to the user, reducing the amount of processing required for the search and enabling faster return of search results compared to sequentially searching each data item in a table-type content database.

[0154] In this way, the server 20 has a data structure 214 of information on legal content that is used to identify a portion (tag) in the database that corresponds to the search request in response to a search request for information on multiple legal content, and to respond to the search request based on the reference relationship between the contents for the identified portion.

[0155] As described above, the data related to the data structure is used in the process of receiving a question input from a user and responding to a search request to search for a portion of a plurality of contents corresponding to a question based on the reference relationships between the contents in response to the search request, in order to generate an answer to the question by the large-scale language model service server 95.The answer to the question is generated by the large-scale language model based on the search results of the plurality of contents and the question from the user.

[0156] In step S929, the server 20 outputs the answer to the question to the terminal 10 based on the tag corresponding to the question.

[0157] In step S913, the terminal 10 displays the answer to the question.

[0158] <4. Screen Example (First Embodiment)> FIG. 10 is an example of an operation screen that displays the answer to a user's question generated by referring to a legally related graph-type data structure 214.

[0159] The operation screen 1000 is an operation screen that displays the results of searching laws, precedents, legal books, guidelines, etc. in response to a question while referring to the graph-type data structure 214.

[0160] The operation screen 1000 corresponds to each process such as step S921 in FIG. 9, step S1121 in FIG. 11, and step S1321 in FIG.

[0161] The question specification section 1002 is an operation member that accepts specification of the question content from the user.

[0162] More specifically, the question specification unit 1002 accepts a question input from the user in natural language.

[0163] The sending operation unit 1004 is an operation member that accepts an operation to generate an answer to a question input by the user.

[0164] In the illustrated example, the transmission operation unit 1004 transmits the question input to the question specification unit 1002 to the server 20 in response to a user operation.

[0165] The account display area 1006 is an area for displaying information about the account of a user who uses a service provided by the server 20.

[0166] In the example shown, the account display area 1006 also displays the user's affiliation, but it may also display the user's account fee plan so that the differences in available functions depending on the fee plan (for example, limitations on the number of times a question can be entered, limitations on the range of laws, precedents, legal books, guidelines, etc. that can be referenced) can be easily recognized.

[0167] The answer display area 1008 is an area where an answer generated by the server 20 in response to a question entered by a user is displayed.

[0168] The question content display area 1010 is an area for displaying the content of the question input by the user in the question specification section 1002 .

[0169] In the illustrated example, the question content display area 1010 displays the question for which the server 20 has generated an answer, making it easy to recognize what question the answer is to. For example, if an answer is generated each time a question is entered, the question to which the answer was given can be easily confirmed even when scrolling back to past questions.

[0170] The question tag display area 1012 is an area that displays tags that have been assigned to the input question.

[0171] The question tag display area 1012 corresponds to the processing of step S923 in FIG.

[0172] The law display area 1014 is an area that displays the laws that are searched for in response to the input question by referring to the graph-type data structure 214 .

[0173] In the illustrated example, the law display area 1014 displays the name of the law and the clause number, and also has a "View details" button. In response to a user's operation on the "View details" button, the terminal 10 may display the details of the law within the operation screen 1000 (for example, by expanding the area of ​​the law display area 1014) so ​​that the details of the law (for example, the original text of the clause) can be viewed, or may open a separate window so that the details of the law can be checked.

[0174] The case display area 1016 is an area that displays the case that has been searched for in response to the input question by referring to the graph-type data structure 214 .

[0175] In the illustrated example, the case law display area 1016 displays the case law number and has a "View Details" button. In response to a user's operation on the "View Details" button, the terminal 10 may display the details of the searched case law on the operation screen 1000 or the like so that the user can view the details of the searched case law (such as an outline of the case in the case law, as will be described later).

[0176] The legal book display area 1018 is an area that displays legal books searched for in response to an input question by referring to the graph-type data structure 214 .

[0177] In the illustrated example, the legal book display area 1018 presents the details of the searched legal book to the user (for example, by accepting a user operation for "View details" shown in the laws and regulations display area 1014, etc.). The legal book display area 1018 displays an overview of the book, such as the name, publisher, and author, while the book details display area 1020, etc., described below, displays an excerpt from the text of the book.

[0178] The book details display area 1020 is an area that displays a portion of the searched legal book extracted according to the searched tag.

[0179] In the illustrated example, the book details display area 1020 displays a portion of the text displayed in the evidence display area 1022 (described later) and the text before and after it. For example, in a legal book, the text may be divided into multiple blocks by chapter, paragraph, etc., and a block containing a text indicated in the "tagged portion" item in the legal content database 212 may be extracted.

[0180] The grounds display area 1022 is an area that displays the part of the legal book where the description corresponding to the tag is found.

[0181] In the example shown, the basis display area 1022 highlights (e.g., highlights or bolds) the text shown in the "tagged location" item in the legal content database 212, making it easy to identify the location in the text that corresponds to the tag.

[0182] The guideline display area 1024 is an area that displays the guideline searched for in response to the input question by referring to the graph-type data structure 214 .

[0183] In the illustrated example, the guideline display area 1024 displays the name of the guideline and has a "View Details" button. In response to a user's operation on the "View Details" button, the terminal 10 may display details of the case law on the operation screen 1000 or the like so that the details of the searched guideline can be viewed.

[0184] <5. Operation (Second Embodiment)> Next, a second embodiment will be described.

[0185] When receiving a legal consultation, it can sometimes be difficult to identify the relevant law from the content of the consultation. The legal provisions themselves are often defined in abstract terms, and even if search terms are defined based on the content of the consultation, the results may not correspond to the wording of the legal provisions. Therefore, the usual procedure involves searching for cases that correspond to the content of the consultation, and then identifying the relevant law and provisions from those cases. For those responding to the consultation, if the field is unfamiliar, the above-mentioned research procedure can be time-consuming.

[0186] Therefore, in the second embodiment, a technology for accepting input of natural language and presenting related laws and clauses will be described.

[0187] Specifically, in the second embodiment, Accepting questions in natural language from users; The server 95 of the large-scale language model service identifies what law the question is about (for example, a specific law such as the Companies Act); Tags are assigned in advance to legal books and web articles related to law (the tagging of the legal books may be performed by the large-scale language model service server 95), and the content of legal books, etc. is searched for those whose tags match the content of the question (searching by keyword, vectorizing the question or the content of legal books, etc., and searching for vectors of content related to law based on the vectors, etc.); Retrieving statutes and provisions cited in the text of the searched content; Do the following.

[0188] FIG. 11 is a diagram showing the flow of processing for identifying laws and regulations corresponding to a user's question and presenting the identified laws and regulations to the user.

[0189] In step S1121, the question processing module 2044 of the server 20 outputs to the terminal 10 an operation screen that accepts an input of a question about law and displays the answer.

[0190] Server 20 stores at least one of legal books, legal guidelines, and legal information media articles in storage unit 202 as information on a plurality of legal contents. Storage unit 202 stores information on a plurality of legal books, and stores tags indicating the content of each legal book in association with the plurality of legal books.

[0191] For example, server 20 may set tags indicating the contents of legal books by referring to classifications set for each book on book browsing service server 93 (classifications entered by a human, such as the service operator or the book publisher), or server 95 of the large-scale language model service may assign tags indicating the contents of legal books (for example, by sending a prompt to server 95 of the large-scale language model service to summarize the contents of the legal books).

[0192] In step S1111, the terminal 10 accepts an input of a question about law from the user.

[0193] In step S1123, the question processing module 2044 of the server 20 assigns a tag corresponding to the question to the question input by the user by the server 95 of the large-scale language model service.

[0194] The question processing module 2044 assigns tags to questions by providing a prompt to the large-scale language model service server 95 including an instruction to assign a legal tag to the question, thereby obtaining information on the results of tagging output by the large-scale language model service server 95, and assigning tags to the question using the obtained information on the results of tagging.

[0195] In addition, the question processing module 2044 may assign tags to questions by inputting the question entered by the user in step S1111 into a trained model that has been trained to output tags for questions based on the results of tags being assigned to questions.

[0196] In addition, the question processing module 2044 may also store a list of tags to be assigned to questions based on keywords contained in the question text, and assign tags to the question by comparing the question entered in step S1111 with the list.

[0197] In step S1125, the question processing module 2044 of the server 20 uses the tag assigned to the question to refer to the database and search for content to which the tag has been assigned.

[0198] In this way, the question processing module 2044 searches for information on multiple pieces of law-related content based on the input question. Here, the question processing module 2044 searches for information on at least one of legal books, legal guidelines, and legal information media articles. The question processing module 2044 may search for legal books based on tags assigned to the question and tags associated with the legal books.

[0199] Furthermore, in a service (book viewing service server 93) that allows browsing of multiple legal books, scores of tags associated with legal books are determined (legal content database 212, content usage history database 213). The question processing module 2044 may search for legal books based on the scores of tags associated with the legal books. For example, when searching for tags associated with each legal book in the database that correspond to the tag assigned to the question, tags with higher scores may be given priority in the search results.

[0200] As described above, server 20 stores information on a plurality of pieces of law-related content in storage unit 202. Storage unit 202 defines reference relationships between pieces of content that are similar in each piece of content, each piece containing at least one of information identifying a law, information identifying a precedent, information identifying a legal book, and information identifying a legal guideline, and stores the results as law-related data structures (legal content database 212, graph-type data structure 214). Question processing module 2044 searches for information on a plurality of pieces of law-related content in response to a question by referencing the law-related data structure.

[0201] In step S1127, question processing module 2044 of server 20 identifies information about laws and regulations cited in the content of the search results. For example, question processing module 2044 may search for content based on the tag assigned to the question and information associated with legal books held in legal content database 212 (such as the item "Title" and the item "Content Tag" in legal content database 212), and identify information about laws and regulations included in the searched content (for example, the item "Tagged Location" and the item "Name Tag" in legal content database 212).

[0202] In step S1129, the question processing module 2044 of the server 20 presents the information on the identified law to the user of the terminal 10.

[0203] In step S1113, the terminal 10 displays information about the laws and regulations corresponding to the question.

[0204] <6. Screen Example (Second Embodiment)> FIG. 12 is an example of an operation screen that presents to the user laws and regulations that have been identified as corresponding to the user's question.

[0205] The law detail display area 1026 is an area that displays a part of the law that has been extracted according to the searched tag for the searched law.

[0206] In the illustrated example, the law detail display area 1026 displays the clause of the law corresponding to the searched tag. For example, the server 20 may display only the clause corresponding to the question, rather than all clauses included in the article.

[0207] The case detail display area 1028 is an area that displays summaries of multiple cases extracted according to the searched tags.

[0208] In the illustrated example, the case detail display area 1028 displays the results of summaries generated by the large-scale language model service server 95 for multiple cases retrieved by referencing the graph-type data structure 214 according to the tag corresponding to the question (for example, the results of generating a summary of the case by prompting the server 95 to generate an outline of the case and an outline of the conclusion in a specified number of characters).

[0209] <7. Operation (Third embodiment)> Next, a third embodiment will be described.

[0210] Typically, when conducting legal research, searching for legal content by keyword will result in a list of content that corresponds to that keyword. However, especially in areas that are unfamiliar to the researcher, it can be difficult to determine whether the listed content (e.g., legal books) is important, and the researcher will have to read each piece of content one by one, which is a significant burden.

[0211] Therefore, in the third embodiment, a question written in natural language is input and points of content related to the input question are presented to the user. Contents such as legal books are organized and presented to the user for each point of content.

[0212] Specifically, in the third embodiment, Constructing a graph-type data structure for the issues (the issues of each legal content may be identified, for example, by a large-scale language model service server 95. Legal content such as laws, precedents, legal books, and guidelines are associated with the issues and constructed as a graph-type data structure); Accepting questions from users in natural language; Identifying the issue corresponding to the question based on a database constructed to hold information on legal content based on the issue (for example, identifying the issue from keywords contained in the question, vectorizing the question and searching for the issue based on the vector of the legal content), Based on the issues corresponding to the question and a graph-type data structure of the issues, the system responds with laws, precedents, legal books, guidelines, etc. that correspond to the issues. Do the following.

[0213] FIG. 13 is a diagram showing the flow of processing for generating a database that specifies information on legal issues for information on a plurality of legal content items.

[0214] In step S1321, the data structure definition module 2046 of the server 20 acquires information on content related to law by referring to the legal content database 212 or the like.

[0215] In this way, the server 20 stores information on a plurality of pieces of law-related content in the storage unit 202. By storing information on a plurality of pieces of law-related content in the storage unit 202, the server 20 stores at least any of information on laws and regulations, information on legal precedents, information on legal books, and information on legal guidelines.

[0216] In step S1323, the data structure definition module 2046 of the server 20 generates a prompt including an instruction to identify a legal issue corresponding to the content of the legal content. The data structure definition module 2046 may also generate a prompt including an instruction to extract a portion of the content that is the basis for identifying a legal issue corresponding to the content.

[0217] In step S1325, the data structure definition module 2046 of the server 20 provides the generated prompt to the server 95 of the large-scale language model service, thereby obtaining a result identifying legal issues regarding the content of the content from the server 95 of the large-scale language model service.

[0218] In this way, the data structure definition module 2046 provides the content for which the prompt is to be generated and the prompt generated for that content to the large-scale language model service server 95, and receives the output result from the large-scale language model service server 95, thereby obtaining the results identifying the legal issues for each piece of content.

[0219] In step S1327, the data structure definition module 2046 of the server 20 updates the legal content database 212 by associating the content with the identified legal issue.

[0220] Here, the data structure definition module 2046 may associate the extracted portion of the content with the identified legal issue and store it in the legal content database 212 .

[0221] In addition, based on the legal issues associated with the content, the issues may be evaluated according to the usage of search results in a service that accepts operations to search for content (for example, a server 93 for a book browsing service), and the evaluation results of the issues may be associated with the content and stored in the legal content database 212 (item ``Issue Tag Score'').

[0222] In step S1329, the data structure definition module 2046 of the server 20 identifies portions of the content related to the same point for the plurality of pieces of content, defines reference relationships for the identified portions, and updates the database. The data structure definition module 2046 outputs information on the reference relationships defined for the points of content for the plurality of pieces of content to the terminal 10.

[0223] In this way, the data structure definition module 2046 may identify portions of multiple pieces of content related to the same issue, and define reference relationships between the identified portions of content to define a graph-type data structure 214.

[0224] It is not necessary to update the database that defines the reference relationship and then output the update to the terminal 10; instead, in response to a request from the terminal 10, it is possible to respond with the status of the reference relationship defined in the database.

[0225] In step S1311, the terminal 10 displays information indicating the reference relationships of the contents regarding the point of content.

[0226] FIG. 14 is a diagram showing the flow of a process for searching for content on a legal issue that corresponds to a user's question.

[0227] In step S1421, the question processing module 2044 of the server 20 outputs to the terminal 10 an operation screen that accepts an input of a question about law and displays the answer.

[0228] The server 20 stores information on multiple legal content items in a legal content database 212 and a graph-type data structure 214 in the memory unit 202, and stores information on legal issues associated with each of the multiple content items.

[0229] The graph-type data structure 214 is used to accept legal questions input from users, identify the issues corresponding to the questions by analyzing the input questions, and search for legal content associated with the identified issues by referring to the information on the legal issues stored in the graph-type data structure 214 based on the identified issues.

[0230] The graph-type data structure 214 is configured to store, for each of a plurality of pieces of content, information on legal issues associated with the content and portions of the content corresponding to the legal issues in association with each other. In the graph-type data structure 214, a reference relationship is defined between portions of a plurality of pieces of content relating to the same issue. The graph-type data structure 214 is used in a process of searching, based on the reference relationship, for a plurality of pieces of content for which a reference relationship is defined for a specified issue.

[0231] In step S1411, the terminal 10 accepts an input of a question about law from the user.

[0232] In step S1423, the question processing module 2044 of the server 20 analyzes the input question to identify the legal issues corresponding to the question. More specifically, the question processing module 2044 generates a prompt for the input question (for example, the prompt may include an instruction to refer to a list of issues) including an instruction to identify the legal issues for the question, and provides the generated prompt to the server 95 of the large-scale language model service, thereby receiving the output result from the server 95 of the large-scale language model service and identifying the legal issues for the question.

[0233] In step S1425, the question processing module 2044 of the server 20 uses the issue identified for the question to search for content tagged with the issue by referring to the legal content database 212 and the graph-type data structure 214.

[0234] In this way, the question processing module 2044 searches for information on legal content associated with the issue corresponding to the identified question by referring to information on the legal issue stored in the memory unit 202.

[0235] In step S1427, the question processing module 2044 of the server 20 outputs the search results for content using the issue tag to the terminal 10, including a portion of these contents that corresponds to the issue (the ``Issue tagged portion'' item in the legal content database 212).

[0236] Here, the question processing module 2044 may provide the server 95 of the large-scale language model service with a prompt including the search results for content using the issue tag and the user's question accepted in steps S1411 and S1423, thereby causing the server 95 of the large-scale language model service to generate an answer to the user's question while referring to the search results for content using the issue tag, and output the generated answer to the terminal 10.

[0237] In step S1413, the terminal 10 displays the search results of legal content together with the points of discussion corresponding to the question.

[0238] <8 Screen Example (Third Embodiment)> FIG. 15 shows an example of an operation screen that responds to a user's question with the results of searching various databases based on the corresponding legal points.

[0239] The issue display area 1030 is an area that displays information about the issue corresponding to the question.

[0240] The point display area 1030 corresponds to the processing of step S1427 in FIG.

[0241] The issue summary display area 1032 is an area that displays an overview of the search results that have been searched for as issues corresponding to the question.

[0242] In the illustrated example, the issue summary display area 1032 displays information on the issues searched for in response to the question by referencing the graph-type data structure 214, and the results of an answer generated by the large-scale language model service server 95 based on the search results and the question.

[0243] The individual issue display area 1034 is an area that displays the details of one of the multiple issues.

[0244] The reference literature display area 1036 is an area for displaying literature (legal books in the illustrated example) that is the basis for the argument.

[0245] In the illustrated example, the references display area 1036 displays multiple legal books searched in response to the issue tags.

[0246] The grounds display area 1038 is an area that displays the part of the legal book where the description corresponding to the point tag is found.

[0247] The grounds display area 1040 is an area that displays the part of the legal book where the description corresponding to the point tag is found.

[0248] <Fourth embodiment> In the fourth embodiment, we will explain a technology that identifies the laws and regulations corresponding to the search location in the content and displays information on the latest laws and legal amendments, making it easy to refer to information on the laws and regulations that were assumed at the time the content was published and subsequent legal amendments.

[0249] As time passes since the time of publication, content such as books may become outdated because it no longer reflects legal changes or the facts that are the basis for legal changes. When referring to such content, there is a risk that people will not realize that there have been subsequent legal changes, and will not realize that the content is outdated.

[0250] However, there is a problem in that checking whether there have been any legal amendments, etc., requires a lot of time and effort, such as checking the history of whether there have been any legal amendments in the first place.

[0251] Therefore, in the fourth embodiment, we will explain a technology that makes it easy to view content that has become outdated over time, taking into account subsequent legal changes, etc. This will further reduce the effort required for research.

[0252] <9. Operation (Fourth embodiment)> FIG. 16 is a diagram showing the flow of processing for identifying laws and regulations corresponding to portions of content that are identified as the basis for answers to questions, and outputting the latest laws and regulations.

[0253] An outline of each process is as follows:

[0254] Terminal 10: The user inputs a question, and the terminal 10 sends the question to the server 20 Server 20: The server 20 generates a provisional answer to the question and searches for related content based on the provisional answer. Server 20: Identifies relevant laws and regulations from a portion of the identified content and acquires the latest information on legal amendments. Server 20: Organizes information to be presented to users, including legal information prior to the publication of the content, and presents it to users. Terminal 10: The terminal 10 displays the received information to the user, highlighting important parts. By displaying related information in an integrated manner on the same screen, the user can easily refer to the content and the latest legal amendments. These processing flows allow users to simultaneously obtain the latest legal amendment information even when referencing older content, such as content that has been published some time ago, thereby reducing the effort required for research.

[0255] Next, each step will be described.

[0256] In step S1621, the question processing module 2044 of the server 20 presents the user with an operation screen for accepting input of a question.

[0257] Here, the server 20 manages multiple contents to be viewed in the memory unit 202 (such as a legal content database 212, a graph-type data structure 214, data from the server 91 for the case law search service, data from the server 92 for the law search service, data from the server 93 for the book viewing service, and data from the server 94 for the information media service).

[0258] The content managed in storage unit 202 may include content that indicates the publication date of the content. More specifically, the content managed in legal content database 212 of storage unit 202 may include at least one of books, guidelines, and media articles that indicate the publication date of the content. Here, content includes books, articles, and other content that mainly consists of text and images, as well as videos.

[0259] Information on a plurality of pieces of law-related content is held as a plurality of pieces of content to be viewed in the storage unit 202. More specifically, the storage unit 202 may store information that defines a reference relationship between pieces of content of the same type for a portion of the content, such as information that identifies laws and regulations, information that identifies precedents, information that identifies legal books, and information that identifies legal guidelines, for each of the plurality of pieces of law-related content.

[0260] In step S1611, the terminal 10 accepts a question input from the user. The terminal 10 transmits the entered question to the server 20. For example, the server 20 may provide the terminal 10 with an operation screen with a text box and accept the question input in natural language or by keyword. Furthermore, the server 20 may provide an audio guide on the operation screen or accept the question input by voice input from the user.

[0261] In step S1622, the question processing module 2044 of the server 20 obtains a provisional answer to the question entered by the user by sending a prompt to an external information processing system to generate a provisional answer in the form of an answer sentence based on the content of the received question.

[0262] More specifically, the question processing module 2044 generates a prompt including the content of the received question and an instruction sentence that outputs the content of the question in the form of an answer, and transmits the generated prompt to an external information processing system to receive output corresponding to the prompt from the external information processing system. As described below, the question processing module 2044 searches multiple content items based on the content of the received output in the form of an answer. Some content items, such as books and guidelines, are written in the form of answer sentences. For example, in content items such as books, using a question (e.g., "When ~~, can ~~ be done?" or "What should be noted about ~~?") in the form of an answer sentence (e.g., "When ~~, ~~ can ~~ be done?" or "What should be noted about ~~ is ~~") can make it easier to identify a portion of the book content that serves as the basis for the answer corresponding to the content of the question, compared to searching by the content of the question. This makes it easier to identify the portion of the content that serves as the basis for the answer when generating an answer to a question, which can further improve the quality of answers to questions.

[0263] In addition, the question processing module 2044 may analyze the content of the question entered by the user using morphological analysis or the like, as in step S1622, to extract words, and then use the extracted words or the like to search for information sources, or may identify a portion of the content based on the content of the question using a vector search, and generate an answer to the question based on this information.

[0264] In step S1623, the question processing module 2044 of the server 20 searches the database of each content in the graph data structure 214 based on the sentence of the provisional answer generated in step S1622. The question processing module 2044 identifies a portion (snippet) of the content corresponding to the question, and obtains the answer to the question by transmitting a prompt including an instruction sentence for generating an answer based on the identified content and the user's question to an external information processing system.

[0265] In this way, based on the content of the question accepted as input in steps S1611 and S1622, question processing module 2044 identifies a portion of the content of the content corresponding to the question by searching multiple contents in storage unit 202. Question processing module 2044 may also identify a portion of the content of the content corresponding to the question based on the reference relationship between the contents.

[0266] In step S1624, the question processing module 2044 of the server 20 outputs the generated answer to the terminal 10.

[0267] In step S1612, the terminal 10 displays the answer together with information about a portion of the content that was the basis for generating the answer. The terminal 10 accepts an operation from the user to specify content.

[0268] In step S1625, the content presentation module 2047 of the server 20 extracts relevant laws and regulations by searching (e.g., vector search) based on the content of a portion (snippet) of the specified content that is identified as the basis for generating an answer.

[0269] In this way, the content presentation module 2047 may extract information on laws and regulations related to the portion of the content by performing a vector search of a laws and regulations database (such as data from the server 92 of the law search service) based on the portion of the content identified in step S1623. The question processing module 2044 may present to the user information on laws and regulations related to the extracted portion of the content in step S1626, which will be described later.

[0270] In step S1626, the content presentation module 2047 of the server 20 refers to a law database (such as the server 92 for a law search service) and acquires the latest amendment information for the relevant law. The content presentation module 2047 may acquire information on legal amendments since the publication date of the content designated by the user to be viewed, as well as the latest legal provisions. The content presentation module 2047 may also acquire information on laws and regulations prior to the publication date of the content before amendments. The content presentation module 2047 may acquire information on legal amendments depending on whether the date was before or after the publication date of the content, as compared with the publication date of the content.

[0271] In step S1627, the content presentation module 2047 of the server 20 presents the user with a screen for viewing the content designated by the user. The content presentation module 2047 outputs legal amendment information, legal provisions, and content publication date information to the terminal 10.

[0272] In this way, when information on laws and regulations is included in a specified portion of the specified content, the content presentation module 2047 presents information on legal amendments to the relevant laws and regulations to the user. For example, when information on laws and regulations is included in a portion of the content displayed on the screen for viewing the content, the content presentation module 2047 may present information on legal amendments to the user of the terminal 10 regardless of when the content was published.

[0273] The content presentation module 2047 may present to the user information on legal amendments, including the latest legal amendments, that have been enacted since the time of publication of the identified content. This allows the user to view the content while referring to information on laws and regulations that have been amended since the time of publication of the content, making it easier to make legal decisions, etc., taking legal amendments into consideration.

[0274] The content presentation module 2047 may display information about the legal amendments together with information about the provisions of the relevant laws and regulations on the operation screen that displays the identified portion.

[0275] The content presentation module 2047 may display information on the publication date of the identified content, information on legal amendments, and information on the provisions of the relevant laws and regulations on the operation screen. This allows the content to be viewed while comparing it with the publication date of the content and referring to information on the revised legal amendments and provisions, making it easier to make legal decisions, etc., taking legal amendments into consideration.

[0276] The content presentation module 2047 may display, on the operation screen, information on legal amendments made after the time of publication of the identified content, as well as information on laws and regulations before the time of publication. This allows a user to view the content while checking information on the laws and regulations that the content is based on, and also to view the content while taking into account information on legal amendments made after the content was published, making it even easier to make legal decisions, etc., taking legal amendments into account.

[0277] The content presentation module 2047 may display the content on the operation screen and highlight a portion of the content that corresponds to the question. This allows the user to easily check the part of the content that is the basis for the answer to the question, making it easier to confirm the validity of the answer to the question and facilitating legal judgments, etc.

[0278] In step S1613, the terminal 10 displays the information received from the server 20 (content viewing screen, legal amendments, legal provisions, etc.) to the user.

[0279] In step S1614, the terminal 10 presents a screen for viewing the content to the user. The terminal 10 highlights a portion of the content (the portion that is the basis for the answer to the user's question) and displays information about legal amendments to laws and regulations related to that portion.

[0280] More specifically, the terminal 10 highlights a portion of the content corresponding to the question, and when the user selects the content, displays information about legal amendments to the laws and regulations related to the selected portion. The terminal 10 displays the publication date of the content, legal amendment information, and the provisions of the laws and regulations on the same screen.

[0281] <10 Screen Example (Fourth Embodiment)> FIG. 17 shows an example of a screen on an operation screen for viewing content, which displays the content of the content and a table of contents as an outline of the content.

[0282] It also includes an explanation of an example screen according to the fifth embodiment.

[0283] The viewing target operation screen 1700 is an operation screen for accepting an operation to view content.

[0284] The viewing target operation screen 1700 corresponds to each process of steps S1627 and S1613 in Fig. 16, steps S2021 and S2012 in Fig. 20, which will be described later, etc. For example, as shown in Fig. 10, 12, 15, etc., a screen displaying an answer to a question displays content (such as legal books) that served as the basis for generating the answer, and in response to an operation to designate these contents, the viewing target operation screen 1700 for viewing the designated content is displayed. For example, in response to an operation by the user to designate each document (basis display areas 1038, 1040) displayed in the reference document display area 1036 in Fig. 15, the viewing target operation screen 1700 is displayed with the document as the viewing target content.

[0285] The account display area 1702 is an area where the user's account is displayed. The viewing target display area 1704 is an area for displaying the content to be viewed.

[0286] In the illustrated example, the viewing subject display area 1704 displays a portion of a legal book as content.

[0287] The snippet display area 1706 is an area that displays a part of the content that is highlighted.

[0288] In the illustrated example, as shown in Figures 10, 12, 15, etc., snippet display area 1706 displays the content (such as a legal book) that served as the basis for generating an answer, and accepts a user operation to designate content to display viewing target operation screen 1700. In such a case, a section including a portion that served as the basis for generating an answer is displayed in response to the designation operation, and the portion is highlighted by highlighting, etc. This allows the user to easily check the portion that served as the basis and also refer to the surrounding descriptions when checking the book, etc. that served as the basis while confirming the answer to a question such as that shown in Figure 10, etc.

[0289] The sub-window display area 1708 is an area for displaying various information related to the content to be viewed.

[0290] The bookshelf addition operation section 1710 is an operation member that accepts an operation to add a bookmark to the displayed content (book).

[0291] The content cover image display area 1712 is an area for displaying the image of the cover of the content (book) to be viewed.

[0292] The content bibliographical information display area 1714 is an area for displaying the bibliographical information of the content to be viewed.

[0293] More specifically, the content bibliographic information display area 1714 displays information such as the name of the content (the title in the case of a book), the edition of the content, the time the content was published, the entity that published the content, and the author of the content. In the example shown, an operation to copy this bibliographic information is accepted. This allows a user to easily cite the content information that serves as the basis when reporting an opinion based on the description in a book.

[0294] The table of contents designation section 1716 is an operation member that accepts designation of the table of contents as various information of the content to be viewed.

[0295] In the illustrated example, when displaying various information related to the content to be viewed, the table of contents designation unit 1716 displays tabs for switching the object, such as a "Table of Contents" that shows the table of contents of the content, "Other Related Content" that displays other content (laws, precedents, guidelines, etc. in the illustrated example) different from the content to be viewed (a book in the illustrated example), a "Bookmark" that displays a list of parts of the content selected by the user, and a "Binder" that collectively manages various contents for the content or parts of the content. In the example of Fig. 17, the "Table of Contents" tab is designated and is displayed so as to be distinguishable from the other tabs.

[0296] The related information designation section 1718 is an operation member that accepts designation of other related content as various information of the content to be viewed.

[0297] In the illustrated example, the related information designation section 1718 shows a state in which the related information designation section 1718 is designated by the user in the example of FIG.

[0298] The bookmark designation section 1720 is an operation member that accepts designation to display a list of locations selected by the user in the content as various information about the content to be viewed.

[0299] The binder designation section 1722 is an operation member that accepts designation for registering content or a portion of content in a "binder" that collectively manages various content.

[0300] In the illustrated example, the binder designation unit 1722 accepts an operation to register the content itself or a portion of the content (a portion of the description in the case of a book) in association with a "binder" that the user can arbitrarily name. This allows the user to easily check the content or a portion of the content registered in the "binder" when organizing their views based on the issue, for example, if the user sets a "binder" according to the issue, making it easy to review the content again.

[0301] The details display area 1724 is an area for displaying details of various information related to the content to be viewed.

[0302] In the illustrated example, the details display area 1724 in Fig. 17 displays the table of contents, and the position in the table of contents corresponding to the currently viewed section (book page) displayed in the viewing target display area 1704 is highlighted. This allows the user to easily grasp the relationship between the overall picture of the content and the section they are currently viewing, and makes it easy for the user to refer to other sections of the content. Also, in Fig. 18, information on legal amendments, laws, guidelines, and precedents is displayed as other content related to the book.

[0303] The legal reference relationship display area 1726 is an area that displays the legal reference relationship that defines the legal reference relationship.

[0304] In the example shown, the legal reference relationship display area 1726 is set with a link to a legal database in which the reference relationship is defined, and depending on the user's operation, the area may transition to the legal search service server 92 to display the legal acts provided by the service, or the legal acts may be displayed by overlay or the like on the screen shown in Figure 17, etc.

[0305] The legal reference relationship display area 1728 is an area that displays the legal reference relationship that defines the legal reference relationship.

[0306] The case reference relationship display area 1730 is an area that displays the case in which the reference relationship is defined.

[0307] In the example shown, the case reference relationship display area 1730 is set with a link to the case database in which the reference relationship is defined, and depending on the user's operation, it may transition to the case search service server 91 to display the case laws provided by that service, or it may display the case laws by overlay or the like on the screen shown in Figure 17, etc.

[0308] FIG. 18 shows an example of a screen on an operation screen for viewing content, which displays the content details and other content related to the displayed content.

[0309] The related content update operation section 1732 is an operation member that accepts an operation to update the display contents of other related content for the content to be viewed, which is displayed in the details display area 1724.

[0310] In the illustrated example, the related content update operation unit 1732, in response to a user operation, updates the display of information such as legal amendments, statutes, guidelines, and precedents related to a portion of the content displayed in the browsing target display area 1704. This corresponds to the processing of step S2023 in FIG. 20 .

[0311] The legal amendment information display area 1734 is an area for displaying information on legal amendments.

[0312] The legal amendment information display area 1734 corresponds to the processing of step S1627 etc. in Fig. 16. In the example shown in the figure, an operation to specify information about legal amendments is accepted, and in response to the specifying operation, details of the information about legal amendments are displayed as shown in Fig. 19.

[0313] The legal information display area 1736 is an area where legal information is displayed.

[0314] The legal information display area 1736 corresponds to the processing of step S1627 etc. in Fig. 16. In the example shown in the figure, a link is set to the information on legal information, and in response to an operation to specify the link, information on the legal database etc. is displayed as described above.

[0315] The guideline information display area 1738 is an area where guideline information is displayed.

[0316] The guideline information display area 1738 corresponds to the processing of step S2022 in FIG.

[0317] The case information display area 1740 is an area for displaying information about the case.

[0318] The case information display area 1740 corresponds to the processing of step S2022 in FIG.

[0319] FIG. 19 shows an example of a screen on an operation screen for viewing content that displays information on laws and legal amendments as other content related to the displayed content.

[0320] The legal amendment details display area 1742 is an area for displaying detailed information about legal amendments.

[0321] In the illustrated example, the legal amendment details display area 1742 displays the legal amendment details display area 1742 instead of the display of the sub-window display area 1708 shown in Figures 17, 18, etc. Alternatively, the information on the legal amendment may be displayed in a separate window, or the information on the legal amendment may be displayed in the sub-window display area 1708 in response to a user operation.

[0322] The law confirmation operation section 1744 is an operation member that accepts an operation to confirm the law the details of which are displayed.

[0323] In the illustrated example, the law confirmation operation unit 1744 displays information about laws and regulations from the law database, etc., in response to a user's operation, as described above. This makes it easy to check each provision of the law.

[0324] The window close operation section 1746 is an operation member that accepts an operation to close the legal amendment details display area 1742 .

[0325] In the illustrated example, the window close operation unit 1746 erases the display of the legal amendment details display area 1742 in response to a user operation, and displays the sub-window display area 1708 shown in FIGS. 17 and 18, etc.

[0326] The legal amendment information display area 1748 is an area that displays detailed information about legal amendments.

[0327] The legal amendment information display area 1748 corresponds to the processing of step S1627 etc. in Fig. 16. In the example shown, information on the latest legal amendment and information before the amendment are displayed. In addition, it is also possible to display information on the publication date of the content to be viewed while also displaying information on legal amendments before and after the publication date.

[0328] <Fifth embodiment> In the fifth embodiment, a technology will be described in which, while viewing content such as a book, other content (e.g., laws, precedents, guidelines, etc.) related to the open section (part of a page in the book) is also displayed.

[0329] When researching content, questions are posed to a large-scale language model, and the answers generated by the model can provide an overview, but the answers may be too abstract and lead to misunderstandings or may be incorrect, so it is necessary to confirm whether the answers are correct. For example, when trying to organize a legal opinion, it is necessary to thoroughly understand the basis for that judgment and then summarize the opinion while referring to evidence.

[0330] There are also book browsing services available, which allow users to check out books, but it is not easy to find the book they need to find what they want to know, or the relevant section of that book that corresponds to what they want to know.

[0331] Therefore, we will explain a technology that can further reduce the effort required to investigate original sources of evidence by allowing users to simultaneously check various pieces of information related to the content of specialized books and other publications while viewing the content.

[0332] <11 Operation (Fifth embodiment)> FIG. 20 is a diagram showing the flow of processing for displaying a part of another content related to the displayed content while presenting an operation screen for viewing the content.

[0333] An outline of each process is as follows:

[0334] Browsing content and providing related information: While browsing content such as legal books, users can simultaneously refer to laws, precedents, and guidelines related to the page they are viewing. This makes it easier to gather detailed information and gain a more accurate understanding. User interface configuration: The screen is roughly divided into the first area (main content display) and the second area (overview and related information display), and the content displayed in the second area can be flexibly switched by user operation. Identifying related content: The server 20 identifies other related content based on the content being viewed or selected. In this case, information on the association between content items is utilized. Related content information updates: When a user changes the content they are viewing, related information updates dynamically (or in response to user actions). This makes it easier to find the information most relevant to what they are currently viewing. Linkage with question-answering function: In response to a user's question, the server 20 performs a search and links with an external information processing system (a large-scale language model service server 95), and the system provides an answer and its rationale, allowing users to easily check the details of the original content. Research support: This system allows users to easily access primary information on other related content, such as relevant laws and precedents, when reading content such as specialized books, thereby facilitating information gathering and understanding.

[0335] These processes allow users to efficiently acquire legal knowledge, for example in legal research, and reduce the time and effort required for research and study. Furthermore, providing related information in a centralized manner can prevent oversights and misunderstandings, and can support more reliable judgments and analysis. Next, each step will be described.

[0336] In step S2011, the terminal 10 presents the user with a screen for viewing any of the content such as laws and regulations, precedents, legal books, guidelines, etc. The terminal 10 presents the user with a screen for viewing the content (laws and regulations, precedents, legal books, guidelines, etc.) specified by the user. The terminal 10 accepts an operation from the user to specify the content the user wishes to view.

[0337] Here, the storage unit 202 of the server 20 stores portions of at least a plurality of pieces of content, such as laws, precedents, legal books, and legal guidelines, in association with each other (legal content database 212, graph-type data structure 214). The content presentation module 2047 presents to the user a screen for viewing any of the various pieces of content, such as laws, precedents, legal books, and legal guidelines, stored in the storage unit 202.

[0338] The content presentation module 2047 identifies a portion of other content that is associated with the portion of content to be viewed on the viewing screen.

[0339] The server 20 may also generate an answer to a user's question. The question processing module 2044 receives a user's question, searches various contents in the storage unit based on the question, creates a prompt including the search results from the search step and the user's question, the prompt including an instruction sentence that answers the question by referring to the search results, and transmits the created prompt to an external information processing system (the server 95 for the large-scale language model service) to obtain an answer generated for the question from the external information processing system (the server 95 for the large-scale language model service). The content presentation module 2047 may receive from the user an operation to specify a portion of the content to be displayed as the basis for generating the answer, and present a screen to the user for viewing with the specified portion of the content displayed.

[0340] In step S2021, the content presentation module 2047 of the server 20 transmits to the terminal 10 the details of the content designated by the user.

[0341] In step S2012, the terminal 10 displays the acquired content in a first area of ​​the operation screen. The terminal 10 displays a second area on the operation screen that is different from the first area, and initially displays an overview of the content (such as a table of contents). The terminal 10 may display the first area and the second area side by side on the operation screen. For example, the second area may be provided with tabs for changing the content to be displayed, and the displayed content may be switched according to the tab designated by the user.

[0342] In step S2013, the terminal 10 accepts an operation to display "other related content" in the second area. For example, the terminal 10 may display "other related content" in the second area in response to an operation of a tab for changing the content to be displayed.

[0343] In this way, the content presentation module 2047 of the server 20 provides a function of switching the content displayed in the second area between a "content summary" (such as a table of contents of the content) and "other related content" in response to a user operation. The terminal 10 accepts a user operation to select display of "other related content" in the second area. In this way, the content presentation module 2047 displays a portion of the identified other content in response to a user operation.

[0344] As described above, the content presentation module 2047 displays, on the screen for viewing content, a first area that displays the content to be viewed, and a second area that is different from the first area, and switches the object displayed in the second area between an outline of the content to be viewed and a portion of other identified content, depending on the user's operation.

[0345] More specifically, the content presentation module 2047 displays the contents of a legal book as the content to be viewed in a first area of ​​the screen on which the content is viewed, and in a second area, depending on the user's operation, switches between displaying an overview of the legal book including the table of contents, or displaying at least a portion of a statute, precedent, or legal guideline as other content associated with the portion of the legal book to be viewed.

[0346] In step S2022, the content presentation module 2047 of the server 20 outputs a portion of other related content, such as a law, precedent, or guideline, associated with a portion of the content (e.g., a book) currently being viewed. More specifically, the content presentation module 2047 identifies other content, such as a law, precedent, or guideline, associated with the portion of the content currently being viewed. The content presentation module 2047 acquires the portion of the identified other content from various databases, etc.

[0347] In step S2014, the terminal 10 displays a portion of other related content in the second area.

[0348] In step S2015, the terminal 10 accepts an operation to change the viewing range of the content (such as scrolling or page movement) and an operation to select a part of the content displayed in the first area (such as text selection).

[0349] In step S2023, the content presentation module 2047 of the server 20, in response to a change in the viewing range of the content (in the case of a book, a change in the page being viewed) or an operation to select a portion of the content (in the case of a book, the user selects a portion of the text), re-identifies other content related to the portion, and outputs it to the terminal 10. For example, the content presentation module 2047 may be provided with an operation member (e.g., an "update" button) that accepts an operation to re-identify other related content, and in response to an operation on the operation member, re-identify other content related to the range being viewed at the time the operation was performed or the range in which a portion of the content was selected, and output it to the terminal 10.

[0350] In this way, the content presentation module 2047 may re-identify other related content based on the range of the newly viewed content and update the displayed content. The content presentation module 2047 may also identify a portion of other content (laws, precedents, guidelines) related to the portion of content selected by the user and update the displayed content.

[0351] In step S2016, the terminal 10 displays the updated related content in the second area.

[0352] <12 Screen Example (Fifth Embodiment)> The screen example in the fifth embodiment is similar to that shown in FIG. 18 and the like described in the fourth embodiment.

[0353] Sixth Embodiment In the sixth embodiment, we will explain a technology that uses an external information processing system such as a large-scale language model service server 95 to generate answers to questions, providing a function that allows switching between search algorithms such as simple, fast answers and slow but in-depth answers to key points according to the answer style and search depth desired by the user, thereby satisfying a wide range of search needs.

[0354] We will also explain the technology for determining the mode for generating answers, more specifically, the technology for analyzing the query entered by the user and the literature searched based on the query, and determining the answer style and search depth desired by the user based on the query, the nature of the search field, the variability of the literature in the search results, etc. We will also explain the technology for automatically switching search algorithms such as simple, fast answers and slow but in-depth answers to key points depending on the results of this determination, thereby satisfying a wide range of search needs.

[0355] When generating answers to questions using an external information processing system such as a large-scale language model, it is difficult for users to determine which of the various models to use. Furthermore, if a model that always generates detailed answers is used, there is a risk that the number of times such a model can be used will be reduced unnecessarily if there are restrictions on the number of times such a model can be used (e.g., the number of times a question can be asked within a certain period of time). Furthermore, if a user attempts to use a model that always generates detailed answers, it will take longer to receive an answer to a question. The more diverse the questions a user asks, the longer it will take to receive an answer, which could disrupt the smooth running of the user's work.

[0356] Therefore, in the sixth embodiment, in order to generate answers that correspond to the user's questions while also shortening and optimizing the time required for research work and helping the user to carry out their work smoothly, we will explain a technology that determines whether to narrow the research scope to generate an answer or to expand the research scope and generate an answer, and if narrowing the research scope and generating an answer quickly can result in an appropriate answer, then the answer is generated in a mode that generates an answer quickly.

[0357] <13 Operation (Sixth embodiment)> FIG. 21 shows the process flow for generating an answer by determining whether to use a first mode for generating an answer quickly or a second mode for generating an answer slowly and in detail, depending on the content of the question.

[0358] An outline of each process is as follows:

[0359] Mode determination: The server 20 analyzes the content and difficulty of the question, the variance of search results, the number of topics in the field, etc., and determines whether the quick mode or the detailed answer mode is appropriate. User behavior feedback: The behavior of the user after confirming the answer, such as whether they ask additional questions or quickly leave, is recorded and reflected in the next mode determination. By providing such mode determination for each user, it is possible to provide the optimal answering style for each user. Waiting time optimization: By displaying the current processing mode while generating an answer, users can better predict waiting times. Answer relevance: Along with the final answer, the mode in which the answer was generated is displayed, allowing users to easily understand the depth and detail of the answer. According to the sixth embodiment, users can obtain answers that are in depth and suited to their needs at the appropriate time, which is expected to improve work efficiency and reduce the time required for research.

[0360] Next, each step will be described.

[0361] In step S2121, the question processing module 2044 of the server 20 presents the user with an operation screen for accepting input of a question.

[0362] In step S2111, the terminal 10 accepts a question input from the user and transmits the question to the server 20.

[0363] In step S2122, the question processing module 2044 of the server 20 analyzes the content of the question received from the terminal 10 and determines whether it is in high-speed mode or detailed mode.

[0364] In this way, the question processing module 2044 analyzes the content of the user's question to determine whether to generate an answer in a first mode that generates an answer quickly, or in a second mode that generates an answer more slowly and in more detail than the first mode.

[0365] (Method of determining the mode in which to operate an external information processing system to generate an answer)

[0366] (1) Variation in sources of information that address questions The question processing module 2044 analyzes the content of the user's question, and searches for information sources including books by referencing the graph data structure 214, the legal content database 212, etc. for the question. The question processing module 2044 determines whether to generate an answer in the first mode or the second mode depending on the variation in the fields of the books. More specifically, the question processing module 2044 may determine that the greater the variation in the fields of the books, the more detailed the answer should be generated in the second mode, and that the smaller the variation in the fields, the faster the answer should be generated in the first mode.

[0367] For example, the question processing module 2044 may determine whether the search results for a group of books are of the same field (for example, books that have been pre-classified according to fields such as law) by determining whether the proportion of books in the same field is above a certain level, or whether the fields of the books are diverse, depending on the degree of variation in the fields, etc.

[0368] The question processing module 2044 may determine whether to generate an answer in the first mode or the second mode depending on the number of issues in the field, taking into account variations in the field. If the number of issues in the question is equal to or greater than a certain number, the answer may be generated in the second mode, and if the number of issues is less than a certain number, the answer may be generated in the first mode.

[0369] (2) The server 95 of the large-scale language model service determines whether to generate a detailed answer. The question processing module 2044 analyzes the content of the user's question and generates an evaluation prompt including the content of the question and an instruction sentence for evaluating whether the question requires further consideration. The question processing module 2044 transmits the generated evaluation prompt to an external information processing system (the large-scale language model service server 95). The question processing module 2044 acquires the evaluation result from the external information processing system. The question processing module 2044 may determine whether to generate an answer in the first mode or the second mode depending on the acquired evaluation result.

[0370] Here, the question processing module 2044 may generate an evaluation prompt that includes the content of the question and an instruction to evaluate whether the question requires further consideration in terms of whether the question has a large number of issues.

[0371] (3) Judgment based on the behavioral history of users who have been presented with answers generated for questions The server 20 may store, in the storage unit 202, behavioral history information that is a history of a user's behavior regarding a question after an answer to the question has been presented to the user. For example, the server 20 associates and stores the content of the question, the answer generated by the large-scale language model service server 95 for the question, and information on the user's behavior after the answer has been presented to the user, such as the time spent viewing the answer and the number of times the user has asked additional questions.

[0372] The question processing module 2044 may analyze the content of the user's question and determine whether to generate an answer in the first mode or the second mode based on the user's behavior history shown in the behavior history information regarding past questions related to the content of the question.

[0373] More specifically, the question processing module 2044 may make a determination based on at least one of the extent to which the user asked follow-up questions in past questions, which is a history of the user's behavior regarding past questions indicated in the behavior history information, or the viewing time during which the user confirmed the answers. For example, if answers to similar past questions have been presented to the user and the number of follow-up questions is small (the number of follow-up questions, the number of characters in the follow-up questions, etc., is below a certain level), the question processing module 2044 may determine that the answers presented to the questions are sufficient and answer in the first mode. Furthermore, if the viewing time during which the answers to similar past questions were confirmed is below a certain level, the question processing module 2044 may determine that the answers presented to the questions are sufficient and answer in the first mode.

[0374] The server 20 may be configured to store, in the storage unit 202, a trained model that is trained to output the content of the user's behavior in response to the content of the question, using the content of the question and the history of the behavior as learning data based on the behavior history information. The question processing module 2044 may make a determination based on the content of the question received in steps S2111 and S2122 and the output result of the content of the user's behavior output based on the trained model.

[0375] (Timing of presenting the result of determining the mode that will generate an answer to the question) The question processing module 2044 displays on the screen an input field for accepting input of a question and an operating member for accepting transmission of the question entered in the input field, and when accepting input of a question entered in the input field in response to a user's operation on the operating member, determines the mode in which to generate an answer to the question entered in the input field, and may present information indicating whether the answer will be generated in the first mode or the second mode as a result of the determination to the user without accepting a user's operation on the operating member (i.e., regardless of whether an operation to generate an answer to the entered question has been performed). This makes it easier for the user to understand whether detailed consideration is required before sending a question, and the length of the waiting time until an answer is obtained, etc.

[0376] In step S2112, the terminal 10 displays to the user that processing is in progress and whether the mode is fast or detailed until an answer is generated. Until an answer is generated by the question processing module 2044 of the server 20, the terminal 10 displays to the user that processing is in progress and the mode being used to generate the answer.

[0377] In this way, the question processing module 2044 may present to the user information indicating whether the answer is being generated in the first mode or the second mode during the period in which the answer is being generated before it is presented to the user. This makes it easier for the user to estimate the waiting time until the answer is generated, eliminating the difficulty of operation due to the waiting time being unknown, and may have the effect of making the waiting time seem shorter.

[0378] In step S2123, based on the determination result of step S2122, the question processing module 2044 of the server 20 reduces the number of information sources referenced based on the question to a certain level or less in the high-speed mode (first mode), while referencing more information sources in the detailed answer mode (second mode).

[0379] The question processing module 2044 prepares to generate an answer in the selected mode based on the determination result of step S2122. In the fast mode, the number of information sources to be referenced is reduced. In the detailed mode, more information sources are referenced. For example, in the fast mode, the number of books to be referenced may be limited to a certain number (such as 10 books), and a certain number of the top books (such as the top 3) may be presented as the basis for generating the answer. In the detailed answer mode, the number of books to be referenced may be increased by a certain multiple compared to the fast mode to generate an answer.

[0380] In step S2124, the question processing module 2044 of the server 20 generates a prompt including the question and the information source referenced in response to the question, and transmits it to the external information processing system (the large-scale language model service server 95).

[0381] Here, when generating an answer in the first mode, the question processing module 2044 may refer to a smaller number of information sources than in the second mode. The question processing module 2044 generates a prompt that includes the content of the user's question and the referenced information sources, and that includes an instruction statement for generating an answer to the question based on the information sources, and transmits the generated prompt to the external information processing system, thereby obtaining an answer from the external information processing system.

[0382] In step S2125, the question processing module 2044 of the server 20 acquires the answer from the external information processing system (the large-scale language model service server 95). The question processing module 2044 transmits the acquired answer to the terminal.

[0383] In step S2126, the question processing module 2044 of the server 20 may record the behavior history of the user who views the answer to the question and store it in the storage unit 202. In this way, the question processing module 2044 may record the behavior history of the user who views the answer to the question and use it for future mode determination.

[0384] The question processing module 2044 may present the answer together with information indicating whether the answer was generated in the first mode or the second mode, thereby allowing the user to easily recognize in what mode the answer was generated.

[0385] In step S2113, the terminal 10 displays the generated answer. The terminal 10 also displays whether the answer was generated in the high-speed mode or the detailed browsing mode.

[0386] <14 Screen Example (Sixth Embodiment)> Figure 22 is an example of a screen that analyzes the content of the question entered and displays the results of a decision made before accepting the operation to send the question, as to whether to answer in the first mode (quick answer mode) or the second mode (detailed answer mode).

[0387] The mode determination result display area 2202 is an area that displays the determination result of whether to use the high-speed mode or the detailed answer mode as the mode for generating an answer at the stage before the question is sent (when the question has been entered in the question specification section 1002 but before any operation has been performed on the send operation section 1004).

[0388] The mode determination result display area 2202 corresponds to each process shown in Fig. 21. In the example shown, it is displayed that it has been determined that an answer will be generated in the detailed answer mode.

[0389] The high-speed mode designation section 2204 is an operation member that accepts designation to generate an answer in high-speed mode.

[0390] In the illustrated example, the high-speed mode designation section 2204 is displayed so as to be positioned inside the mode determination result display area 2202, but it may also be positioned near the sending operation section 1004, etc., and the user may be able to manually switch between generating an answer in high-speed mode or detailed answer mode.

[0391] FIG. 23 shows an example of a screen that displays the determination result of the mode for generating an answer on the operation screen during the period from when a question is sent until an answer is generated using an external information processing system.

[0392] The processing progress display area 2206 is an area that displays an overall picture of each process when generating an answer to a question, and the status of the processing situation.

[0393] In the illustrated example, the processing progress display area 2206 displays steps that have been processed, steps that are in progress, and steps that have not yet been processed, in a distinguishable manner.

[0394] The mode display area 2208 is an area that displays the mode in which the answer is generated.

[0395] The mode display area 2208 corresponds to the processing of step S2112 in FIG.

[0396] FIG. 24 shows an example of a screen that displays the determination result of the mode for generating an answer on the screen that displays the answer generated for the question.

[0397] The mode display area 2210 is an area that displays the mode in which the answer was generated when an answer has been generated.

[0398] The mode display area 2210 corresponds to the processing of steps S2125, S2113, etc. in FIG.

[0399] Seventh Embodiment In the seventh embodiment, a technology will be described that supports brushing up questions so as to make it easier to obtain suitable answers in a system that generates answers to questions.

[0400] In systems that generate answers to questions, such as interactive systems, users may not know how to ask a question. Even if they enter a highly abstract, general question, the answer may be broad and shallow, and may not be very useful because it is not specific.

[0401] Therefore, we will explain a technology that allows users to quickly recognize the question they want to ask and easily obtain answers to their questions. More specifically, we will explain a technology that makes it easier for users to input questions by presenting question candidates in response to the user's question.

[0402] <15. Operation (Seventh embodiment)> FIG. 25 is a diagram showing the flow of a process for accepting a follow-up question that is a more in-depth investigation of a question input by a user.

[0403] An outline of each process is as follows:

[0404] Question input and analysis: Analyze the question entered by the user and evaluate its specificity Encouragement of question specificity: If a question needs to be more specific, relevant information is presented to prompt the user to enter a specific question. Answer generation: Based on the specified question, information sources are searched and an answer is generated by the large-scale language model service server 95. Providing an answer: Present the answer to the user These processing flows enable users to clarify what they want to know and obtain appropriate answers. Even if a question is vague, the processing of the server 20 can help make the question more specific, improving the user experience.

[0405] Next, each step will be described.

[0406] In step S2521, the question processing module 2044 of the server 20 presents the user with an operation screen for accepting input of a question.

[0407] In the memory unit 202, the server 20 stores, as an information source, a graph database (graph-type data structure 214) which is a graph-type data structure that defines reference relationships between parts of the content of multiple pieces of content, and stores attribute information set for at least one of the parts of the content for which reference relationships are defined or the defined reference relationships in the graph database.

[0408] The memory unit 202 stores, as a graph database, reference relationships that associate portions of the content of at least several of the various types of content, such as laws, precedents, legal books, and legal guidelines, and stores attribute information that includes at least information on the points of contention.

[0409] In step S2511, the terminal 10 accepts an input of a question (first input) from the user, and transmits the input question to the server device.

[0410] In step S2522, the question processing module 2044 of the server 20 receives the content of the user's question. The question processing module 2044 determines whether the received question is specific.

[0411] (Determine whether the question is specific and whether additional questions or deeper questions are needed) The question processing module 2044 determines whether the content of the question related to the first input requires a more specific question to generate an answer. If it is determined that a more specific question is required, the question processing module 2044 performs the process of step S2544 below (prompting the user to specify the question). If it is determined that a more specific question is not required, the question processing module 2044 may generate a prompt including the content of the question related to the first input and search results obtained by searching information sources based on the content of the question, the prompt including an instruction statement for generating an answer by referring to the search results of searching information sources for the question, and transmit the generated prompt to an external information processing system (the server 95 of the large-scale language model service) so that the external information processing system (the server 95 of the large-scale language model service) can obtain an answer created in response to the prompt and output the obtained answer to the user.

[0412] (The large-scale language model service server 95 determines whether the question needs further digging.) The question processing module 2044 may generate a prompt including the content of the question related to the first input as shown in step S2511, and including an instruction sentence for determining whether a more specific question is required to generate an answer to the question, and may make a determination by sending the generated prompt to an external information processing system (server 95 for the large-scale language model service) and obtaining the result of the determination made by the external information processing system (server 95 for the large-scale language model service) in response to the prompt.

[0413] (Determined based on the scope of reference to the graph database) The question processing module 2044 may refer to a graph database (graph-type data structure 214) as an information source based on the content of the question related to the first input as shown in step S2511, thereby obtaining information on the range of the referenced graph database, and making a judgment depending on the extent of the obtained referenced range.

[0414] The question processing module 2044 may make a determination based on at least one of the number of contents included in the referenced range and the amount of data including the number of characters in the contents, as information about the range referenced in the graph database (graph-type data structure 214). For example, the question processing module 2044 may determine that the greater the number of contents included in the referenced range (for example, the greater the number of extracted books), the more necessary it is to dig deeper and specify the question. Furthermore, the question processing module 2044 may determine that the greater the number of characters in the referenced content, the more necessary it is to dig deeper and specify the question.

[0415] If it is determined that a more specific question is required, the question processing module 2044 may present candidate questions to the user in the following step S2524.

[0416] (Question closed) The server 20 may store information on upper limits to the extent that the user is prompted to input specific questions in the storage unit 202. For example, the server 20 may store information such as an upper limit on the number of questions, a range of references to the graph data structure 214, etc. as the upper limit information.

[0417] When the question processing module 2044 repeatedly receives a question input from the user through the processing of step S2511 in this embodiment, step S2513 described below, etc. (e.g., the user asks a question after a display prompting the user to ask a more specific question), and the extent to which information sources have been consulted reaches an upper limit of information, the question processing module 2044 may terminate the processing of prompting the user to input a more specific question. Here, the question processing module 2044 may terminate the processing of prompting the user to input a more specific question by presenting the user with a response that the information sources were consulted in response to the question but no information corresponding to the question was found. This allows the user to confirm that the search was conducted but no relevant information was found.

[0418] In step S2523, if the question is insufficient (abstract), the question processing module 2044 of the server 20 creates specific question candidates to be presented to the user by referring to information sources within a certain range and generating a list of related keywords and points of discussion. For example, the list is generated by giving priority to keywords and points of discussion that are included in the results of referring to the information sources within a certain range and have a high score related to the question.

[0419] In step S2524, the question processing module 2044 of the server 20 transmits to the terminal 10 related information together with a message prompting the user to specify the question.

[0420] In this way, the question processing module 2044 refers to information sources to a certain extent in response to the content of the user's question related to the first input, and presents the information in the referenced range to the user by responding to the user with the information in the referenced range, while prompting the user to enter a question that is more specific than the question related to the first input.

[0421] When responding to a user's question related to the first input with information from the referenced range, the question processing module 2044 may present the user with the information from the referenced range without using a process to identify an information source corresponding to the question using an external information processing system that generates an answer in response to a prompt input.

[0422] The question processing module 2044 references a graph database (graph-type data structure 214) as an information source based on the content of the question related to the first input. The question processing module 2044 presents to the user information extracted based on multiple attributes (such as points of discussion) included in the referenced range. At this time, the question processing module 2044 may present a prompt to prompt the user to input a specific question using the extracted information as an example.

[0423] The question processing module 2044 may present the user with a list of issues and keywords included in the scope of the graph database based on the content of the question related to the first input, and prompt the user to enter a more specific question.

[0424] (Determine whether the question contains multiple themes) The question processing module 2044 may determine whether a question input by a user includes multiple themes.

[0425] If it is determined that a question includes multiple themes, the question processing module 2044 may classify the question according to the theme and present the classified results to the user.

[0426] More specifically, the question processing module 2044 generates a prompt including the received input question, the prompt including an instruction sentence for determining whether the question includes multiple themes. The question processing module 2044 transmits the generated prompt to an external information processing system (the server 95 of the large-scale language model service). The question processing module 2044 acquires information on the result of determination made in accordance with the instruction sentence from the external information processing system (the server 95 of the large-scale language model service). The question processing module 2044 may make a determination based on the information on the result of determination acquired from the server 95 of the large-scale language model service.

[0427] Furthermore, the question processing module 2044 may refer to the graph database (graph-type data structure 214) based on the accepted input question, thereby acquiring information on the range referenced in response to the question, and may make a determination based on the acquired information on the range referenced. For example, when the graph-type data structure 214 is referenced in response to a question, it may be determined whether multiple themes (e.g., multiple points of discussion) are included based on information on attributes set in the nodes and edges set in the referenced range.

[0428] In step S2512, the terminal 10 displays specific question candidates and related information to the user, and displays a message prompting the user to input further questions.

[0429] In step S2513, the terminal 10 accepts an input of a specific question (second input) from the user. The terminal 10 transmits the question to the server 20.

[0430] In step S2525, the question processing module 2044 of the server 20 searches information sources based on the question and obtains search results. In this way, the question processing module 2044 receives a specific question entered by the user, searches information sources based on the specific question, and obtains search results.

[0431] In step S2526, the question processing module 2044 of the server 20 generates a prompt including the question and the search results, and sends it to the external information processing system (the server 95 of the large-scale language model service). The question processing module 2044 obtains an answer from the external information processing system (the server 95 of the large-scale language model service).

[0432] In this way, the question processing module 2044 generates a prompt that includes the content of the user's question related to the second input and the search results of the searching step, and that includes an instruction statement for generating an answer to the question while referring to the search results. The question processing module 2044 transmits the generated prompt to an external information processing system (the large-scale language model service server 95) to obtain an answer created by the external information processing system in response to the prompt.

[0433] In step S2527, the question processing module 2044 of the server 20 transmits the answer acquired from the server 95 of the large-scale language model service to the terminal 10.

[0434] In step S2514, the terminal 10 displays the answer to the user.

[0435] <16 Screen Example (Seventh Embodiment)> FIG. 26 shows an example of a screen that accepts follow-up questions that are further clarified regarding the question entered by the user.

[0436] The question content display area 2602 is an area for displaying the question (first question) input by the user.

[0437] In the illustrated example, the question content display area 2602 displays the content of the question received in the process of step S2511 in FIG.

[0438] The question candidate display area 2604 is an area for displaying question candidates when it is determined that further investigation of the question is necessary.

[0439] The question candidate display area 2604 corresponds to the processing of steps S2524, S2512, etc. in FIG.

[0440] The question content display area 2606 is an area for displaying a question (second input) further input by the user.

[0441] In the illustrated example, the question content display area 2606 displays the content of the question received in the process of step S2513 in FIG.

[0442] <Eighth embodiment> In the eighth embodiment, for example, a technology will be described in which, in a system that generates answers to questions, a list of content that forms the basis of the answer is displayed in multiple stages, and switching between a mode that makes it easy to grasp the overall picture of many pieces of content and browse through quoted passages in each piece of content, and a mode that makes it easy to compare and consider which content to view in detail.

[0443] If the content is a book, the first stage displays the book title and a certain number of characters for each piece of content. The second stage displays a larger number of characters from the quoted passage. The window is wider than in the first stage, allowing users to see more of the quoted passage, making it easier to select a book.

[0444] Because answers from large-scale language models are not necessarily correct, it is difficult to rely solely on the answers from large-scale language models when conducting research, and it is necessary to check materials with more evidence. On the other hand, content such as books is often created by authors with their own purposes, and may contain a variety of descriptions.

[0445] Therefore, it is not easy for users to identify the materials they should check to find out what they want to know.

[0446] Here, we will explain a technology that further reduces the workload required for research by making it easier for users to identify materials that contain evidence while getting an overall understanding of what they want to know.

[0447] <17 Operation (Eighth embodiment)> FIG. 27 is a diagram showing the flow of a process for presenting an answer to a question while displaying a list of content related to the answer, and changing the display format of the list in response to a user operation.

[0448] An outline of each process is as follows:

[0449] Two-stage display: The user first receives the answer and a concise list of content (first mode). If the user wants to see more details, the user can increase the number of citations in each content, resulting in a more informative display (second mode). Switching display modes: When switching from the first mode to the second mode, an animation is performed in which the display area expands vertically and horizontally, visually informing the user that the amount of quoted text being displayed is increasing. On the other hand, when switching from the second mode back to the first mode, an animation is performed in which the display area shrinks gradually. Content Listing by Issue: The content list is presented to the user organized by legal issue. As the user switches between issues, the displayed issue name and associated content list are updated. Providing information on research scope: Users can check the number of sources referenced, the number of characters, and the time required calculated from their average reading speed. This makes it easy to understand the extent to which the time and amount of information required for research has been reduced compared to when browsing each book or other content individually. Utilizing a graph database: The server 20 refers to a graph database to identify appropriate sources of information for a query. The graph database includes laws, precedents, legal books, guidelines, etc. Answer generation by the large-scale language model service server 95: Based on the identified information source, an answer is generated using the large-scale language model service server 95. The answer includes evidence of the information source, making it possible to provide the user with highly reliable information. Improved user operability: Users can easily switch display modes and select topics, allowing them to easily access the information they need. This process flow allows users to efficiently check the documents that form the basis of their responses and easily access detailed information as needed, reducing the workload involved in the survey and enabling higher quality information collection.

[0450] Next, each step will be described.

[0451] In step S2721, the question processing module 2044 of the server 20 presents the user with an operation screen for accepting input of a question.

[0452] A graph database (graph-type data structure 214) that is a graph-type data structure that defines reference relationships between parts of the content for a plurality of pieces of content is stored in storage unit 202. In storage unit 202, the graph database (graph-type data structure 214) that serves as an information source stores at least information about the content of legal books.

[0453] In step S2711, the terminal 10 accepts a question input from the user. The terminal 10 transmits the input question to the server 20.

[0454] In step S2722, the question processing module 2044 of the server 20 refers to the graph database (graph type data structure 214) to identify content that will be an information source to be referred to in response to the question.

[0455] In step S2723, the question processing module 2044 of the server 20 creates a prompt to generate an answer based on the identified information source (including a book) and sends it to the external information processing system (server 95 of the large-scale language model service), thereby obtaining an answer from the external information processing system (server 95 of the large-scale language model service).

[0456] Based on the identified information source, the question processing module 2044 generates answer information including the content of the answer to the question and information on the information source that is the basis for the answer.

[0457] More specifically, the question processing module 2044 creates a prompt including the content of the question and the identified information source, and includes an instruction to generate an answer by referring to the information source for the question, and generates answer information by transmitting the created prompt to an external information processing system (the server 95 of the large-scale language model service) and obtaining the answer generated by the external information processing system (the server 95 of the large-scale language model service).In this way, the server 95 of the large-scale language model service generates an answer to the question along with the basis for the answer.

[0458] In step S2724, the question processing module 2044 of the server 20 generates the content of the answer and a list of the contents (information sources) that are the basis of the answer. The question processing module 2044 transmits the answer and the list of contents to the terminal 10.

[0459] Here, when generating the content of the answer and a list of content (information sources) that served as the basis for the answer, the question processing module 2044 may calculate the number of information sources referenced and the number of characters, and calculate the required time from the average reading speed, and may transmit this calculated information (such as the number of information sources referenced) to the terminal 10 along with the answer and the list of content.

[0460] More specifically, the question processing module 2044 presents to the user a list of multiple content sources that served as the basis for the content of the answer in either a first manner or a second manner in which a greater amount of relevant parts of the content of each content that served as the basis for generating the answer are presented than in the first manner.

[0461] (Display mode and display mode switching) The question processing module 2044 displays the content of the answer and a list of the multiple pieces of content that served as the basis for the answer on the screen. The question processing module 2044 receives an operation from the user to switch between displaying the list of content in the first format or the second format. The question processing module 2044 switches between displaying the list in the first format and the second format in response to the switching operation.

[0462] More specifically, in the second mode, the question processing module 2044 may display the list in a mode in which the display area and display amount of the content of each content in the list are increased compared to the first mode. This makes it easy to grasp the overall feel of the content included in the list in the first mode, while in the second mode, it becomes easy to consider and select from among candidate content for which the user wishes to check the details.

[0463] The question processing module 2044 may display on the screen both the content of the answer and a list of the multiple pieces of content that served as the basis for the answer, and may display on the screen, in association with the list, an operation member that accepts an operation to switch whether the list of content is displayed in the first mode or the second mode. When switching from the first mode to the second mode in response to a user's operation on the operation member, the question processing module 2044 may transition to the second mode by rendering the display area of ​​each piece of content in the first mode so as to be gradually expanded at least vertically or horizontally. This allows the user to visually recognize that the amount of content that can be confirmed in each piece of content is greater in the second mode than in the first mode in response to the user's operation, making the user's operation even easier.

[0464] When switching from the second mode to the first mode in response to a user's operation on an operating member displayed in association with the list, the question processing module 2044 may transition to the first mode by rendering the display area of ​​each content in the second mode so as to be gradually reduced in at least one of the vertical and horizontal directions. This makes it easier for the user to recognize that when switching from the second mode to the first mode, the amount of content that can be confirmed for each content is reduced, making it easier for the user to grasp the overall picture of each content included in the list, and makes the user's operation even easier.

[0465] More specifically, the question processing module 2044 may display, as a list of multiple contents, a list of legal books related to each legal issue included in the answer to the question. The question processing module 2044 displays an operation member that accepts an operation to switch the issue in the list of legal books. In response to a user's operation on the operation member for switching the issue, the question processing module 2044 may update the name of the issue as the title of the list of legal books and display a list of legal books corresponding to the issue to be displayed. This allows a user to easily check the list of legal books for each issue, and when selecting a book for which the user wants to view details, the user can easily select it from the displayed list by narrowing down the issue.

[0466] (Check the amount of data in the source of information) When presenting the user with a list of the contents of legal books that are the source of information, the question processing module 2044 may also present the user with information on the amount of information in the source of information.

[0467] More specifically, the question processing module 2044 may present to the user information on the amount of information in the information sources by presenting to the user at least one of the number of contents in the referenced information sources and the amount of data included in the contents in the referenced information sources. The question processing module 2044 may present to the user information on the amount of information in the information sources by calculating the time required for a human to read and understand the referenced information sources and presenting the calculated time required to the user.

[0468] In step S2712, the terminal 10 presents the answer received from the server and a list of contents to the user (first mode). The terminal 10 presents the number of information sources referred to, the number of characters, and the required time as information on the range of research to the user.

[0469] In step S2713, the terminal 10 switches the display mode (switching from the first mode to the second mode, switching the issue, switching from the second mode to the first mode) to change the content of the list, the amount of information, and the display area of ​​the list, in response to a user operation on the content list.

[0470] The terminal 10 switches to the second mode by receiving an operation from the user to display in detail the list of content displayed in the first mode. At this time, the terminal 10 displays an animation that gradually expands the display area and amount of information for each piece of content in the content list in response to the user's operation, as a switch to the second mode. In the second mode, the quoted portions of each piece of content are increased and detailed information is displayed. When the user performs an operation to switch the content display mode again, an animation that gradually reduces the display area is displayed, and the terminal transitions from the second mode to the first mode.

[0471] The terminal 10 accepts an operation to switch the topic from the user. In response to the switching of the topic, the terminal 10 updates the displayed list of content related to the topic name.

[0472] In step S2714, the terminal 10 accepts from the user a designation of the content that the user wishes to view in detail.

[0473] In step S2725, the content presentation module 2047 of the server 20 acquires detailed information about the specified content and responds to the terminal 10. For example, if a legal book is specified, the contents of the legal book are output to the terminal 10.

[0474] In step S2715, the terminal 10 presents the details of the acquired content to the user.

[0475] <18 Screen Example (Eighth Embodiment)> FIG. 28 shows an example of a screen displaying a list in a first manner that makes it easier to grasp the overall picture of the content when presenting an answer to a question and displaying a list of content related to the answer.

[0476] Figure 29 is an example of a screen displaying a list in a second manner in which, when presenting an answer to a question and displaying a list of content related to the answer, the amount of information in each piece of content is increased to assist in selecting content for further viewing.

[0477] The operation screen 2800 is an operation screen that displays the answer generated in response to the question.

[0478] The operation screen 2800 corresponds to the processing of step S2724 in FIG.

[0479] The answer display area 2802 is an area where the answer generated for the question is displayed.

[0480] The input question display area 2804 is an area for displaying the content of the question input by the user.

[0481] The answer summary display area 2806 is an area that displays a summary of the answer included in the generated answer.

[0482] The issue display area 2808 is an area that displays a list of issues included in the generated answer.

[0483] The research scope display area 2810 is an area that displays the scope of information sources that were consulted to generate the answer.

[0484] The first issue display area 2812 is an area that displays the answer to the first issue among the multiple issues included in the generated answer.

[0485] The document list designation section 2814 is an operation member that accepts designation to display a list of documents that served as the basis for generating the answer to the first point of contention.

[0486] In the illustrated example, the document list designation unit 2814 switches the list of documents displayed in the document list display area 2902 to one corresponding to the first point of discussion in response to user operation (switches the group of documents to be displayed in the list group display area 2904).

[0487] The document list display area 2902 is an area that displays a list of content (books, guidelines, etc.) that was referenced when generating the answer.

[0488] The list group display area 2904 is an area for displaying a group of content to be displayed.

[0489] In the illustrated example, the list group display area 2904 displays groups corresponding to the issues shown in the issue display area 2808. The groups to be displayed are switched in response to user operations on the list group display area 2904.

[0490] The list enlargement / reduction operation unit 2906 is an operation member that accepts operations to expand or reduce the display area for each content in order to increase the amount of information quoted from each content as a way of displaying the list of content displayed in the literature list display area 2902.

[0491] In the illustrated example, the list enlargement / reduction operation unit 2906 transitions from the state shown in FIG. 28 to the state shown in FIG. 29 in response to a user operation. Specifically, in response to a user operation on the list enlargement / reduction operation unit 2906, the horizontal width of the document list display area 2902 is expanded, while the amount of information on the citations of each piece of content in the content display area 2910 is increased. In the illustrated example, in response to a user operation on the list enlargement / reduction operation unit 2906, the horizontal width of the answer display area 2802 is reduced, and the line breaks in the answer text are changed accordingly. However, it is also possible to leave the horizontal width of the answer display area 2802 unchanged, and to move the display position of the answer display area 2802 horizontally as the display width of the document list display area 2902 is expanded. In the example of FIG. 28, the display mode of the list enlargement / reduction operation unit 2906 indicates that the document list display area 2902 is to be expanded. In the example of FIG. 29, the display mode of the list enlargement / reduction operation section 2906 indicates that the document list display area 2902 is to be reduced.

[0492] The window close operation section 2908 is an operation member that accepts an operation to close the document list display area 2902 .

[0493] The content display area 2910 is an area for displaying an outline of each piece of reference content (such as a book).

[0494] In the illustrated example, the content display area 2910 displays bibliographic information such as the title of the content, the content classification (e.g., book, guideline, etc.), and the publication date of the content, as well as citations of portions of the content that served as the basis for generating the answer to the question. Compared to the example shown in FIG. 29, the amount of citation information in the content display area 2910 is reduced in the example shown in FIG. 28. As a result, compared to the example shown in FIG. 29, the example shown in FIG. 28 displays the reference content in a manner that makes it easier to grasp the overall picture of the reference content. For example, when a user refers to the answers displayed in the answer display area 2802 and considers which original source, such as a book, to refer to for each answer, the user can easily narrow down the issues by viewing the answers displayed in the answer display area 2802 and checking the list of references in the literature list display area 2902 for each issue. On the other hand, in the example shown in FIG. 29, the amount of information on the citations in each content is increased, making it easier to determine which content to review in detail. Depending on the user's operation, each content is transitioned to a content viewing screen such as that shown in FIG. 17, etc.

[0495] <Ninth embodiment> In the ninth embodiment, the following will be described.

[0496] (1) Flow for creating a graph-type data structure that corresponds to the abbreviated notation of laws and regulations. Depending on the various notations of laws and regulations, the system identifies the places in the content where laws and regulations are mentioned, defines the reference relationship with the information of the laws and regulations that has a graph-type data structure, and generates a graph-type data structure.

[0497] (2) A flow that performs a vector search based on a question, obtains information from the citation graph using the retrieved documents, and provides the information obtained from the question, the retrieved documents, and the graph database to the LLM to generate an answer. Various content is searched based on the question, and related content is extracted from the search results by referencing a graph-type data structure, which serves as the basis for generating an answer to the question.

[0498] In order to make appropriate decisions in legal practice, it is necessary to comprehensively consider a variety of information sources, including relevant laws and regulations, similar precedents, and explanations in specialized books and guidelines.

[0499] Therefore, we will explain the technology that constructs a citation graph that comprehensively links laws, precedents, books, and guidelines, and serves as a source of information to be referenced in legal practice.

[0500] Specifically, the following will be explained. -How to build a citation graph that includes various sources such as books and guidelines in addition to laws and precedents - A method for handling question-answering tasks that require providing reasons for answers, such as in legal practice A method for providing appropriate evidence for questions by combining a large-scale language model with a constructed citation graph. A method for generating answers that are more favorable to legal professionals. <19 Data Structure of the Ninth Embodiment> FIG. 30 shows a graph-type data structure for each type of content, such as laws, precedents, books, and guidelines.

[0501] The example shown in the figure shows an example of a graph-type data structure in which information identifying a book (information identifying the content, such as the title of the book, the year of publication of the book, the publisher of the book, the author of the book, the edition of the book, etc.) and each part (chunk) into which the main text of the book is divided are each represented as vertices, and edges are defined in a parent-child relationship between the vertices of information identifying the book and the vertices representing the chunks.

[0502] The example shown in the figure shows an example of a graph-type data structure for laws and regulations, with information identifying the law (such as the name of the law, when the law was revised, when it came into effect, etc.) and articles, paragraphs, and subdivisions as vertices, and edges defined by parent-child relationships.

[0503] Although not shown in the figure, guidelines are similarly structured as follows: information identifying the guideline (guideline name, issuer of the guideline, publication date of the guideline, etc.) and each part into which the guideline text is divided are each treated as vertices, and a graph-type data structure is created in which edges are defined in a parent-child relationship between the vertex representing the information identifying the guideline and the vertex representing the chunk.

[0504] Although not shown in the figure, similarly for legal precedents, the data structure is a graph in which the vertices are information identifying the case (date of trial, type of trial, court information, trial number, etc.) and at least one of the summary of the case or a chunk of the main text of the judgment, and edges are defined by parent-child relationships in the same way as above.

[0505] 31 is a diagram showing the data structure of the abbreviation notation dictionary database 215, which defines rules for abbreviating laws and regulations. The abbreviation notation dictionary database 215 includes an item "abbreviation notation rule ID," an item "formal name of law," an item "law abbreviation," an item "abbreviation notation rule," an item "law ID," and an item "content ID."

[0506] The item "abbreviation notation rule ID" is an ID for uniquely identifying an abbreviation notation rule.

[0507] The item "Formal name of law" is information about the formal name of the law.

[0508] The item "Law abbreviation" is information on the abbreviated name of the law.

[0509] The item "abbreviation notation rules" is detailed information on the application rules and patterns of abbreviation notation.

[0510] The item "Law ID" is an ID for uniquely identifying a law.

[0511] The item "content ID" is an ID for uniquely identifying content to which the abbreviation notation rule applies.

[0512] <20. Operation (Processing Flow of Ninth Embodiment)> Figure 32 shows the process flow for identifying the places in the content where laws and regulations are mentioned, depending on the various ways in which laws and regulations are written, and defining a reference relationship with information about the laws and regulations having a graph-type data structure to generate a graph-type data structure.

[0513] In step S3221, the data structure definition module 2046 of the server 20 acquires information on various content to be viewed, such as book information from the server 93 of the book viewing service, guideline information from the server 94 of the information media service, and case law information from the server 91 of the case law search service.

[0514] The data structure definition module 2046 integrates the book title and its chunks (each part into which the content is divided) into a graph structure as a parent-child relationship. The data structure definition module 2046 detects periods, commas, and line breaks in the main text of the book, and extracts each part that reaches a certain number of characters as a chunk. For example, in the main text of the book, a section where a sentence exceeds a certain number of characters and a period, comma, or line break is detected may be extracted as a single chunk.

[0515] The data structure definition module 2046 integrates the guideline title and its chunks (each part into which the content is divided) into a graph structure as a parent-child relationship.

[0516] The data structure definition module 2046 integrates information identifying the case (such as the case number and the date of the trial) and information on the content of the judgment (for example, the case summary and chunks of the main text of the judgment) as a parent-child graph structure.

[0517] In the memory unit 202, the server 20 manages information on content to be viewed, which is at least either a book or a guideline (such as the legal content database 212, data from the server 93 for the book viewing service, data from the server 94 for the information media service, etc.), and information on laws and regulations including the provisions of the laws and regulations (such as data from the server 92 for the law search service).

[0518] The server 20 manages content information and statutory information as data with a graph-type data structure in the storage unit 202. More specifically, the server 20 stores statutory information in the storage unit 202 in a graph-type data structure in which the statutory name, article, paragraph, and subparagraph are each defined as vertices, and reference relationships are defined in which the vertices of the article, paragraph, and subparagraph are in a parent-child relationship with the statutory name vertex.

[0519] Here, in the memory unit 202, the server 20 classifies information on laws and regulations based on at least one of the revision date or the implementation date, and stores each information in a graph-type data structure according to at least one of the revision date or the implementation date.

[0520] In the memory unit 202, the server 20 manages content information as data in a graph-type data structure in which information indicating the title of a book or guideline is used as a vertex, and each part into which the content of the book or guideline is divided is used as a vertex, and reference relationships are defined between the vertices of each part as parent-child relationships with the vertex indicating the title of the book or guideline.

[0521] The server 20 manages information about legal precedents in the storage unit 202, and manages it as data with a graph-type data structure in which the information that identifies the legal precedent is the vertex, and at least one of the summary of the legal precedent or each part into which the judgment of the legal precedent is divided is the vertex, and a reference relationship is defined between the vertex of the information that identifies the legal precedent and the vertex of the summary of the legal precedent or each part as a parent-child relationship. Here, the information that identifies the legal precedent includes the following elements. Court name: The name of the court where the trial was held. The abbreviation of the court may also be used. For example, "Supreme Court (Saichi)" or "Tokyo High Court (Tokyo High Court)." Type of trial: judgment, order, decision Date of trial: The date of the trial. For example, "September 9, 1993" or "December 10, 2020" Source: The case collection or magazine in which it is published (abbreviated name and page number). For example, "Hanrei Jiho (Hanji)" or "Hanrei Times (Hanta)" As described above, the information for identifying a legal precedent is composed of a regular expression that includes the character strings of the above elements. The data structure definition module 2046 extracts the character strings of the above abbreviations using regular expressions and performs processing to convert abbreviations such as court name, type of trial, and source back to their official names. This makes it possible to uniquely identify a legal precedent and define reference relationships to legal precedents from books and guidelines.

[0522] In step S3222, the data structure definition module 2046 of the server 20 identifies the part of the content that mentions laws and regulations, even if the name of the law is written in a different way from the official name, based on the rules for mentioning laws and regulations in the content. The data structure definition module 2046 extracts the part of the content that mentions laws and regulations by text analysis.

[0523] (1) Extraction of laws and regulations by referring to the abbreviation dictionary database 215 that manages abbreviations The server 20 refers to dictionary information (abbreviation dictionary database 215) that has a correspondence between abbreviations and full names of laws and regulations, and identifies the locations in the content where the laws and regulations are abbreviated. The data structure definition module 2046 refers to the abbreviation dictionary database 215 to identify the full names of laws and regulations in the portions of the content that mention laws and regulations. In this way, the data structure definition module 2046 may identify the full names of laws and regulations using dictionary information according to the type of abbreviation style associated with the content.

[0524] The abbreviated name notation dictionary database 215 manages the following examples of abbreviated names: Unique abbreviations used in books, etc. For example, the phrase "Article 124, Paragraph 3 of the Rules" may refer to "Article 124, Paragraph 1, Paragraph 3 of the Enforcement Regulations of the Companies Act" (in books, etc., the names of laws and regulations are sometimes written using their own abbreviations). For example, the phrase "Article 27-2, Paragraph 1, Item 3 of the Financial Instruments and Exchange Act" may refer to "Article 27-2, Paragraph 1, Item 3 of the Financial Instruments and Exchange Act." For example, "Commercial Registration Act 54IV" may refer to "Article 54, Paragraph 4 of the Commercial Registration Act" (the numbering system differs between articles and paragraphs, for example, articles are written in Arabic numerals and paragraphs in Roman numerals). Here, the official name of the law may be obtained by obtaining information on the official name provided by the server 92 of the law search service (for example, the official name of the law listed in the file name that can be obtained from the e-GOV law search).

[0525] (2) Extraction of laws and regulations when the names of laws and regulations are abbreviated The data structure definition module 2046 may extract parts of the content that do not contain the name of a law but contain a description of an article, paragraph, or subsection, and may identify the official name of the law for the extracted part by referring to the title of the content or other parts of the content.

[0526] Some books and guidelines do not include the full name of a statute in the text, but simply use the word "law" or use a phrase such as "the same law" to suggest that the name of the preceding statute is inherited. For these abbreviations (e.g., "law" or "the same law"), the data structure definition module 2046 may infer the name of the statute in the abbreviated form by extracting the name of the statute from the title, table of contents, and surrounding text of the book or guideline where the abbreviation is made. For example, even if the text simply includes the abbreviation "law," the name of the statute in question may be inferred by extracting the name of the statute in the book title, the name of the statute in the table of contents, and the names of the statutes written around the abbreviated description. In this way, when the name of a statute is abbreviated, the data structure definition module 2046 infers the exact statute by referring to the context of the description and other parts of the content, and defines the statute's clause number and reference relationship.

[0527] (3) Defining reference relationships to multiple articles, clauses, and items The data structure definition module 2046 may extract a portion of the content that contains a specific phrase used together with an article, paragraph, or subparagraph, and may define a reference relationship for that portion to multiple articles, paragraphs, or subparagraphs depending on the specific phrase in the extracted portion.

[0528] The data structure definition module 2046 detects whether phrases such as "and" and "·" are included in the text of books and guidelines along with the abbreviations of laws and regulations. When these specific phrases are used, the data structure definition module 2046 analyzes that multiple articles, clauses, and items are being mentioned, and defines the reference relationships to the multiple articles, clauses, and items in the graph.

[0529] For example, if a book contains the words "Article 364 and Article 367 of the Civil Code," the data structure definition module 2046 defines a reference relationship between "Article 364 of the Civil Code" and "Article 367 of the Civil Code" as legal information for the part (chunk) in the book where the wording is contained, assuming that this means a reference to these two.

[0530] Similarly, when the phrase "·" is used in a book or guideline (for example, when it is written as "Article 210, Paragraph 1, Article 214 of the Corporate Reorganization Act"), the data structure definition module 2046 defines a reference relationship to multiple articles, paragraphs, and items, just as when the phrase "and" is used in a book or guideline along with information about laws and regulations (for example, it defines a reference relationship between "Article 210, Paragraph 1 of the Corporate Reorganization Act" and "Article 214 of the Corporate Reorganization Act").

[0531] Furthermore, if a book or guideline uses expressions specifying the scope of a clause (for example, "from", "to") along with legal information such as the abbreviation of the law, the data structure definition module 2046 extracts the expressions specifying the scope of the clause and defines a reference relationship between the body of the book or guideline and each clause in that range (for example, if the body of the book states "Civil Rehabilitation Act Articles 258 to 260", a reference relationship between the part of the book containing that statement and "Civil Rehabilitation Act Article 258", "Civil Rehabilitation Act Article 259", and "Civil Rehabilitation Act Article 260" is defined (for example, if the body of the book states "Civil Act Articles 138 to 141", a reference relationship between the part of the book containing that statement and "Civil Code Article 138", "Civil Code Article 139", "Civil Code Article 140", and "Civil Code Article 141" is defined).

[0532] As described above, the abbreviation notation dictionary database 215 of the storage unit 202 is configured to store dictionary information indicating the correspondence between the abbreviated notation forms of laws and regulations and the official names of laws and regulations. Here, there are multiple types of abbreviated notation forms of laws and regulations. The server 20 is configured to store dictionary information for each type of abbreviated notation form of laws and regulations in the abbreviated notation dictionary database 215 of the storage unit 202. Each piece of content is stored in association with information that specifies the type of abbreviated notation form of laws and regulations.

[0533] In step S3223, the data structure definition module 2046 of the server 20 analyzes the statute name, article, paragraph, and subsection identified in the content, and identifies the corresponding vertices in the graph data structure of the statute.

[0534] The data structure definition module 2046 refers to the publication date of the content (for example, the publication date of a book, the publication date of a guideline) and identifies the vertex of the latest version of the law (revision history) for the relevant period in the graph-type data structure for the law.

[0535] In step S3224, the data structure definition module 2046 of the server 20 defines a reference relationship between the identified law information and the mention portion in the content, and adds a new edge to the graph data structure. In this way, the data structure definition module 2046 defines a reference relationship between the portion in the content that mentions the identified law and the information on the mentioned law. More specifically, the data structure definition module 2046 defines a reference relationship between the portion in the content that mentions the identified law and the vertex corresponding to the law name, article, paragraph, or subparagraph of the mentioned law.

[0536] The data structure definition module 2046 may refer to information on the publication date of the content, and define a reference relationship between the part that mentions a specified law in the content and the vertex corresponding to the law name, article, paragraph, or subparagraph of the mentioned law, targeting the latest information on laws and regulations at the time of publication of the content.

[0537] In this way, the data structure definition module 2046 defines a reference relationship between a part identified as mentioning a law and the information about the mentioned law, even if the official name of the law is not stated in the content.

[0538] The data structure definition module 2046 may define a reference relationship between the vertex of each portion of the content that mentions a legal precedent and at least one of the vertices of each portion of the legal precedent summary or judgment of the legal precedent.

[0539] In step S3225, the data structure definition module 2046 of the server 20 calculates a parameter indicating the degree of similarity based on the reference relationships defined between the contents, and assigns it to the parent vertex of each content. The final graph data structure may be saved in a database for use in processes such as a question-answering system that responds to content searches with content.

[0540] The data structure definition module 2046 calculates a parameter indicating the degree of similarity between each piece of content based on information about the reference relationship defined for the content. For example, by vectorizing other content (including portions of other content) for which a reference relationship is defined for each piece of content, a vector for the reference relationship between each piece of content can be identified. For example, text for portions of other content for which a reference relationship is defined for the content can be aggregated and vectorized, or a vector based on the reference relationship between the content can be obtained by performing a calculation based on the vectors of portions of other content for which a reference relationship is defined for the content. By comparing these vectors, the similarity of the reference relationship between each piece of content (the degree to which similar other content is referenced) can be calculated. The data structure definition module 2046 associates the calculated parameter indicating the similarity with the content and stores it in the storage unit 202.

[0541] Here, the server 20 may be a system that generates an answer to a question. The question processing module 2044 may search for content based on the content of a question entered by a user, further extract content similar to the searched content based on a parameter indicating the similarity of the searched content, and generate an answer to the question using the extracted content.

[0542] For example, the question processing module 2044 may extract content by referring to a graph-type data structure based on the content of a question entered by a user, generate a prompt including the content of the question and the extracted content, and including an instruction to generate an answer to the question by referring to the extracted content, and provide the generated prompt to an information processing system that performs language processing (server 95 of the large-scale language model service), thereby obtaining an answer generated in response to the prompt from the information processing system that performs language processing, and presenting the obtained answer to the user.

[0543] Figure 33 shows the processing flow for searching various contents in response to a question, referencing a graph-type data structure for the search results, extracting related contents, and generating an answer to the question.

[0544] In step S3311, the terminal 10 accepts a question input from the user. The terminal 10 transmits the content of the accepted question to the server 20. In step S3321, the question processing module 2044 of the server 20 vectorizes the content of the received user question.

[0545] Here, the data structures of various data held by the server 20 and used in the processing will be described.

[0546] The server 20 is configured to store, in the graph-type data structure 214 of the storage unit 202, data of a graph-type data structure that defines reference relationships between parts that indicate the content of each content for multiple contents to be viewed.

[0547] More specifically, the graph-type data structure 214 of the memory unit 202 holds multiple types of content as multiple contents, and is configured to store data of a graph-type data structure that defines reference relationships even between different types of content, and is configured to store, as multiple types of content, at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law.

[0548] The graph-type data structure 214 of the storage unit 202 holds information on each part (chunk) of the book content or guideline content divided into multiple parts for at least either book information or guideline information held as multiple types of content. The graph-type data structure 214 of the storage unit 202 holds data that defines the reference relationship between parts of other content and at least one part of the book or guideline content as data of the graph-type data structure.

[0549] More specifically, the storage unit 202 stores data in a graph-type data structure in which the title of at least one of the books or guidelines is at the vertex, and the content of at least one of the books or guidelines is at the vertex, and data defining a reference relationship between the title vertex and the content vertex in a parent-child relationship.

[0550] More specifically, the memory unit 202 stores data in a graph-type data structure in which the name of the law is the vertex, and each of the articles, paragraphs, and subparagraphs that make up the law is the vertex, with reference relationships defined between the vertex and the articles, paragraphs, and subparagraphs in the order of parent-child relationships with the vertex of the name of the law.

[0551] More specifically, the memory unit 202 stores data in a graph-type data structure in which the information identifying the precedent is at the vertex, and the content of the precedent is the summary of the precedent, and at least one of the parts into which the content of the judgment has been divided is at the vertex, and data that defines a reference relationship between the vertex of the information identifying the precedent and the summary of the precedent and at least one of the parts of the content of the judgment in a parent-child relationship.

[0552] In addition, the data structure definition module 2046 defines reference relationships between parent vertices by aggregating the reference relationships of child vertices to parent vertices based on the reference relationships defined from parts of the content to other content, and stores the reference relationships as a graph-type data structure 214.

[0553] In step S3322, the question processing module 2044 of the server 20 vectorizes the content of the question and searches for a part corresponding to the content of the question by comparing the vector of the content of the question with the vector of each part of the content. The question processing module 2044 compares the vectorized question with the vector of each part (chunk) of content such as books and guidelines in the database to search for chunks related to the question (searches for chunks with similar vectors).

[0554] In this way, the question processing module 2044 searches for a plurality of pieces of content based on the content of the question. More specifically, the question processing module 2044 searches for a plurality of types of content. Based on the content of the question, the question processing module 2044 searches for at least one of book information and guideline information from among the plurality of types of content.

[0555] In step S3323, the question processing module 2044 of the server 20 extracts related content (first content) based on the ranking of the chunks in the search results.

[0556] More specifically, the question processing module 2044 searches for a portion of a book or guideline that corresponds to the content of the question, and determines the ranking of the content that corresponds to the content of the question based on the search ranking of each portion of the content. For example, the search ranking of the content is determined by aggregating the search rankings of each chunk of content and weighting those with higher rankings, calculating the average search ranking of the chunks for each content, extracting a certain number of the top chunks from the search rankings, and tallying the number of search results for each content. For example, the more of each content chunk that ranks highly based on vector comparison, the higher the search ranking of the content may be.

[0557] In step S3324, the question processing module 2044 of the server 20 selects, for one or more first contents that are the search results, second contents that have a reference relationship defined with the first contents based on the data of the graph data structure. The question processing module 2044 refers to the graph data structure 214 for the first contents and extracts other related contents (second contents).

[0558] As described above, the data structure definition module 2046 extracts, as a group, a plurality of pieces of content having similar reference relationships defined from portions of the content to other pieces of content. The question processing module 2044 may select, as second content, other pieces of content that are extracted as a group because they have similar reference relationships with respect to a first piece of content. In this way, the question processing module 2044 may extract second content related to a first piece of content based on the similarity of the reference relationships in the graph data structure for the first piece of content (by comparing vectorized reference relationships defined for the content between pieces of content, content having similar reference relationships is extracted).

[0559] In step S3325, the question processing module 2044 of the server 20 generates a prompt including a question sentence, a first content, and a second content, and formats the prompt as input data to the LLM so that an answer to the question is generated using the prompt.

[0560] In this manner, the question processing module 2044 generates a prompt that includes the content of the question, the retrieved first content, and the selected second content, and that includes an instruction to generate an answer to the content of the question by referencing the first content and the second content.

[0561] In step S3326, the question processing module 2044 of the server 20 provides the generated prompt to the LLM (large scale language model service server 95) to generate an answer. In this way, the question processing module 2044 provides the generated prompt to an information processing system that performs language processing, thereby obtaining the answer output by the information processing system that performs language processing.

[0562] In step S3327, the question processing module 2044 of the server 20 acquires the answer output from the LLM (the large scale language model service server 95) and presents it to the user.

[0563] In step S3312, the terminal 10 presents the acquired answer to the user.

[0564] <Modification> The above-described embodiments may be combined with each other.

[0565] Furthermore, the above-described embodiments may be modified as follows.

[0566] (1) Sources of information that serve as the basis for answers to questions In the above embodiment, an example was given in which a graph-type data structure is defined for laws, precedents, legal books, and legal guidelines, and a search is performed using the graph-type data structure in response to a user's question, and an answer including the search results is output.

[0567] In addition, a graph-type data structure may be defined in the same manner as above based on information that is not necessarily made public, such as information held on the business's server 96, and the same type of information as that held on the business's server 96 may be searched for in response to a question based on the graph-type data structure, and an answer to the question may be output. For example, if a business has accumulated results of decisions made on specific cases based on legal issues (for example, based on issues of the Act against Unjustifiable Premiums and Misleading Representations), a summary of the cases within the business may be presented in the same way as presenting court cases such as those shown in Figure 12 above.

[0568] For example, information held by a business may include internal regulations, manuals, incident reports, documents such as Q&As and minutes stored in document tools, and inquiries between users within the business (e.g., responses from users to inquiries to accounting, legal, etc.) in an internal communication tool (e.g., a tool in which users are assigned to groups such as channels and can send and receive messages between users). A graph-type data structure may be defined for this data using information on the internal organizational chart, internal terminology, general terminology, etc. As described in the above embodiment, answers to questions about internal regulations, etc. may be provided by having the large-scale language model service server 95 generate answers to user questions by referencing the graph-type data structure. For example, these data files may be stored in a specific folder, and the large-scale language model service server 95 may generate answers by referencing the information in the folder.

[0569] In addition to the above, a graph-type data structure similar to the above may be defined based on information that may be made public to the general public, such as on a social networking service server 97, and may be referenced when answering a user's question. For example, the content posted by a specific person's account may be cited as the basis for answering a question.

[0570] (2) How to generate a prompt As described in the above embodiments, in a system for generating answers to questions, it is not easy to create prompts to be given to a large-scale language model.

[0571] Therefore, the prompt may be created in the manner described in each of the above embodiments.

[0572] <21 Operation (Processing Flow of Tenth Embodiment)> In the tenth embodiment, we will explain a technology that supports the appropriate revision of company rules. This technology makes it easy to find clauses in company rules that need to be changed due to legal amendments or changes in precedents. In addition, in order to clearly indicate the relationship between company rules and laws and regulations, it clarifies whether company rules are mandatory as required by law or are unique rules for the organization.

[0573] FIG. 34 is a diagram showing the flow of processing for generating a graph database in which reference relationships are defined between each clause of the company regulations and a graph data structure that associates laws, precedents, books, and guidelines.

[0574] The following processing flow links each clause of the company's internal regulations with external information such as books, guidelines, and other content, creating a system that can easily extract and display the affected parts of the company's internal regulations when legal amendments or changes in precedents occur.

[0575] For example, when a user inputs new company regulations data from a terminal, the server analyzes the content and matches it with previously stored information on laws, precedents, and related guidelines. The server automatically defines links between nodes, which are then reflected in the graph database. This makes it possible to extract relevant updates to regulations using the same processing flow when regulations are changed in the future, potentially improving the efficiency of revising regulations in line with changes in laws and precedents.

[0576] In step S3411, the terminal 10 accepts input of information on the company rules of the organization (such as the text of each clause and associated metadata) from the user. The terminal 10 transmits the input company rules data to the server 20 via the network.

[0577] The server 20 associates the company regulations data with laws, legal precedents, books, and guidelines as follows: As described above, the storage unit 202 associates laws and legal precedents, and books and guidelines, and further associates the company regulations with laws, legal precedents, and legal precedents as described below.

[0578] As described above, the storage unit 202 is configured to store each provision of the law as data in a graph-type data structure.

[0579] Furthermore, as described above, the memory unit 202 is configured to store at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on legal precedents, and to store data in a graph-type data structure that defines reference relationships between the contents of these pieces of content.

[0580] In step S3421, the data structure definition module 2046 of the server 20 accepts input of information on company rules that define the company rules of an organization. The data structure definition module 2046 associates the received company rule data with information about the organization and stores it in the storage unit 202. The data structure definition module 2046 analyzes and extracts each clause from the company rule data and recognizes it as a node. For example, as described in FIG. 30, a reference relationship may be defined in which the title of the company rule is the parent node and each clause of the company rule is the child node, indicating a parent-child relationship.

[0581] In step S3422, the data structure definition module 2046 of the server 20 acquires information relevant to the company rules from external information sources such as laws, precedents, books, guidelines, etc. The data structure definition module 2046 analyzes the contents, keywords, semantic similarities, etc. between the acquired external information and each clause of the company rules, and identifies the reference relationships that associate them with each other.

[0582] More specifically, the data structure definition module 2046 acquires information that mentions the company rules from an information source that mentions the company rules. For example, a book that explains the establishment of company rules describes the theme of the company rules, example sentences for the company rules, the laws and regulations that are the background to the company rules, and the provisions of the laws and regulations. The information source mentions the laws and regulations that are the basis for the company rules. By referring to the information from the information source for the information on the company rules of the organization that was accepted as input in step S3421, the data structure definition module 2046 identifies the laws and regulations related to the content of each clause of the company rules of the organization.

[0583] As described above, the details of the types of data stored in the storage unit 202 include books, guidelines, laws and regulations, and legal precedents. In this way, the storage unit 202 has a graph-type database in which reference relationships are defined between the contents of books, guidelines, laws and regulations, and legal precedents. A reference relationship is defined between information on the organization's internal regulations and information on books and guidelines. Also, a reference relationship is defined between information on books and guidelines and information on laws and regulations and legal precedents.

[0584] The data structure definition module 2046 acquires, as an information source, information on at least one of legal books and legal guidelines that mention the company regulations. The data structure definition module 2046 defines a reference relationship between each clause of the company regulations of the organization and at least one of the books or guidelines by referring to at least one of legal books and legal guidelines as an information source, as follows:

[0585] In step S3423, the data structure definition module 2046 of the server 20 defines reference relationships (links) as edges between the clauses of each in-house regulation and the nodes of each piece of content, such as laws, precedents, books, and guidelines, based on the association results, to construct a graph data structure. The data structure definition module 2046 stores the generated graph data structure as a graph database, making it available for subsequent search processing and answer generation processing.

[0586] In this way, the data structure definition module 2046 associates the laws and regulations identified in step S3422 with the contents of each clause of the organization's internal regulations and stores them in the storage unit 202.

[0587] The data structure definition module 2046 stores information about the organization's internal regulations and each clause of the laws and regulations that define a reference relationship with the book or guideline identified in step S3422, and stores the information about the organization's internal regulations and each clause of the laws and regulations as a graph-type data structure.

[0588] In step S3424, the data structure definition module 2046 of the server 20, if necessary, associates information on the organization's internal regulations with the graph-type data structure and transmits the generated results (an overview of the graph structure and association information) to the terminal 10, allowing the user to confirm and provide feedback.

[0589] FIG. 35 shows the flow of processing in which each clause of the company regulations is associated with laws and precedents, and information on amendments to laws and precedents is acquired to extract clauses of the company regulations that will be affected.

[0590] The following processing flow makes it possible to extract relevant clauses of company regulations in conjunction with legal amendments and changes in precedents, thereby realizing a system that allows users to quickly respond to revisions to company regulations.

[0591] In step S3511, the terminal 10 displays a screen for checking information on legal amendments and precedent changes on the dashboard or notification screen for the user. In this way, the terminal 10 prompts the user to check the impact of changes in laws, regulations, and precedents. The terminal 10 accepts input from the user of a request for impact confirmation (e.g., start of check, specification of target company regulations, etc.) and transmits the request to the server 20.

[0592] The server 20 performs the following process to prompt the user to update the organization's internal rules in response to legal amendments and changes in precedents. Information about the organization's internal rules is associated with laws, precedents, books, and guidelines. The server 20 extracts internal rules that need to be changed due to legal amendments and changes in precedents. The server 20 then prompts the user to confirm and change those internal rules.

[0593] At least one of information on books including legal books and information on guidelines including legal guidelines, information on laws and regulations, information on legal precedents, and information on the organization's internal regulations is stored in the storage unit 202. The storage unit 202 is configured to store graph-type data structure data that defines a reference relationship between at least information on each clause of the organization's internal regulations and at least one of information on each clause of laws and regulations or information on legal precedents.

[0594] In step S3521, the data structure definition module 2046 of the server 20 periodically or on-demand acquires the latest change information from an external legal amendment information providing system or precedent change information providing system (e.g., the server 91 for the precedent search service, the server 92 for the law search service) or data source. For example, articles about legal amendments and precedent change may be acquired as information provided by the server 94 for the information media service. The data structure definition module 2046 compares the acquired information about legal amendments and precedent change information with the information about laws and precedents in the graph database and identifies the items to be changed (such as the name of the law or the clause of the law).

[0595] In this way, the data structure definition module 2046 acquires at least one of information on legal amendments and information on changes in precedents. Based on the acquired information on legal amendments or changes in precedents, the data structure definition module 2046 identifies at least one of related provisions of laws and regulations or information on related precedents.

[0596] In step S3522, the data structure definition module 2046 of the server 20 extracts each clause of the company rules associated with the identified laws and precedents from the graph data structure and analyzes the extent of their impact. Based on the extraction results, the data structure definition module 2046 generates and organizes a list of affected company rule clauses and necessary amendment proposals (recommended operation information).

[0597] In this way, the data structure definition module 2046 extracts the clauses of the organization's internal regulations that have a defined reference relationship to the clauses of the relevant laws and regulations or the information of the relevant precedents, which were identified in step S3521.

[0598] Here, the server 20 may present a proposal for amending the company rules in response to changes in laws and precedents.

[0599] The data structure definition module 2046 generates proposed amendments to the clauses of the extracted internal regulations of the organization based on at least one of the information on legal amendments and information on legal precedent changes acquired in step S3521, and presents the generated proposed amendments to the clauses of the internal regulations to the user. For example, the server 20 acquires articles explaining legal amendments and legal precedent changes from the information media service server 94. These articles may include not only the details of the legal amendments and legal precedent changes, but also proposed amendments to the clauses of the internal regulations. The data structure definition module 2046 may acquire these explanatory articles on legal amendments and legal precedent changes, as well as information on the affected internal regulations, and generate proposed amendments to the clauses of the internal regulations. For example, the data structure definition module 2046 may generate a prompt that includes information about clauses of the organization's internal regulations that will be affected by a legal amendment or a change in precedent, and an explanatory article about the legal amendment or the change in precedent, taking these explanatory articles about the legal amendment or the change in precedent into consideration, and may send the generated prompt to the server 95 of the large-scale language model service, thereby obtaining the generated amendment result from the server 95 of the large-scale language model service.

[0600] In step S3523, the data structure definition module 2046 of the server 20 returns the generated information on the affected clauses of the internal regulations that will be affected by the legal amendment or change in precedent and the proposed amendments to the terminal 10 so that the user can check them.

[0601] In this way, the data structure definition module 2046 presents the information on the internal rules of the organization extracted in step S3522 to the user as internal rules that are affected by the amendment of the law or the change in precedent.

[0602] The server 20 may display information on legal amendments and precedents together with the company regulations that will be affected.

[0603] The data structure definition module 2046 may display on the screen both the information on the clauses of the organization's internal regulations extracted in step S3522 and at least one of the information on legal amendments or information on changes in precedent obtained in the same step.

[0604] Furthermore, when the user approves the proposed amendment, the server 20 may update the database of the company rules. At this time, the database may be stored for each version of the company rules.

[0605] In this way, the data structure definition module 2046 may receive an operation from a user approving a proposed amendment to a clause of the company regulations, and in response to receiving an operation approving the proposed amendment, update information on the company regulations of the organization based on the proposed amendment.

[0606] In step S3512, the user of the terminal 10 checks the returned information on the screen, and performs operations such as approving or correcting the proposed revision, providing feedback, etc., to complete the final update process.

[0607] <Modification of the Tenth Embodiment> In the above embodiment, an example was described in which a reference relationship was defined between each provision of the internal regulations and information on laws and precedents. This allows the system to determine whether each provision of the internal regulations is required by law or precedent (a reference relationship is defined between the internal regulations and information on laws and precedents) or whether the internal regulations are established independently by the organization and not by law or precedent (a provision of the internal regulations that does not define a reference relationship with information on laws and precedents). The server 20 may present the internal regulations to the user in a manner that allows the user to visually confirm whether a reference relationship with information on laws and precedents is defined for each provision of the internal regulations. More specifically, the server 20 may change the display mode for each provision of the internal regulations depending on whether a reference relationship with information on laws and precedents is defined in the graph data structure. This allows the system to revise the regulations after confirming whether the revision is required because of a legal requirement or because the internal regulations are unique to the organization.

[0608] <22 Operation (Processing Flow of Eleventh Embodiment)> In the eleventh embodiment, an example of a system for generating answers to questions will be described in which, as explained in the tenth embodiment, a graph-type database defines reference relationships between the contents of laws, precedents, books, and guidelines, and the reference relationships are defined with each clause of an organization's internal regulations as a node. In the eleventh embodiment, a technology will be described for generating answers to questions while also generating answers in accordance with the internal regulations, including information about the internal regulations, for the content that served as the basis for generating the answers.

[0609] FIG. 36 shows the flow of processing for generating an answer to a question based on laws, precedents, books, guidelines, and also company regulations.

[0610] The following processing flow shows an example of a consistent process that starts with user input, then proceeds to natural language analysis on the server side, vector search, extraction of related information using a graph database, and generation of an answer using LLM. This ultimately enables the realization of a system that displays an answer on the terminal along with supporting information, including company regulations.

[0611] In step S3611, the terminal 10 accepts input of various questions from the user, such as business questions (for example, inquiries about the company's risks and standards). The terminal 10 transmits the input question text to the server 20.

[0612] The server 20 has a graph-type data structure in which company regulations are associated with laws, regulations, legal precedents, books, and guidelines. When the server 20 receives a question from a user, it generates an answer based on the books, guidelines, laws, legal precedents, and company regulations. This process is explained below.

[0613] At least one of information on books including legal books and information on guidelines including legal guidelines, information on laws and regulations, information on precedents, and information on the organization's internal regulations is stored in the storage unit 202. The storage unit 202 is configured to store graph-type data structure data in which a reference relationship is defined between information on each clause of the organization's internal regulations and at least one of information on each clause of laws and regulations, information on precedents, information on books, and information on guidelines.

[0614] In step S3621, the question processing module 2044 of the server 20 performs natural language analysis on the received question sentence, extracts keywords, and performs vectorization processing. In this way, the question processing module 2044 converts the context of the question into a numerical expression in preparation for the subsequent search.

[0615] In this way, the question processing module 2044 accepts input of a question from the user.

[0616] In step S3622, the question processing module 2044 of the server 20 uses vector search to compare the vector information of pre-registered books and guidelines, and searches for and extracts highly relevant documents.

[0617] In step S3623, the question processing module 2044 of the server 20 identifies information on laws and precedents linked to the book and guideline by utilizing the reference relationships between nodes in the graph database.

[0618] In this way, the question processing module 2044 identifies at least one of information on laws and regulations, case law, book information, or guideline information related to the content of the question by referring to data in a graph-type data structure based on the content of the question.

[0619] More specifically, the question processing module 2044 searches for books and guidelines based on the content of the question, identifies laws and precedents that define a reference relationship with the searched books and guidelines, and extracts in-house regulations associated with the identified laws and precedents.

[0620] In this way, the question processing module 2044 searches for at least one of book information and guideline information based on the content of the question. The question processing module 2044 identifies at least one of the searched book information and guideline information and at least one of information on laws and regulations or information on legal precedents for which a reference relationship is defined in the data of the graph data structure.

[0621] In this way, the question processing module 2044 performs a vector search of books and guidelines based on the content of the question, and further identifies laws, precedents, and company regulations using a graph-type data structure. The question processing module 2044 may vectorize the content of the question and search for at least one of book information and guideline information by comparing the vector of the content of the question with the vector of at least one of book information and guideline information.

[0622] In step S3624, the question processing module 2044 of the server 20 extracts the relevant company regulations information (including each clause and information on the scope of impact) based on the link information on the graph in conjunction with the identified laws and precedent information.

[0623] In this way, the question processing module 2044 extracts information on the internal rules of the organization that defines a reference relationship to the information identified in step S3623. The question processing module 2044 extracts at least one of the information on laws and regulations or the information on legal precedents identified in step S3623 and information on the internal rules of the organization that defines a reference relationship in the data with a graph data structure.

[0624] In step S3625, the question processing module 2044 of the server 20 integrates the question, the extracted books, guidelines, laws, precedents, and company regulations, and generates a prompt to be given to the LLM (large scale language model service server 95).

[0625] The question processing module 2044 generates a prompt including the content of the question, at least one of the information on laws and regulations, legal precedents, books, and guidelines identified in step S3623, and the information on the internal regulations of the organization extracted in step S3624. The prompt includes an instruction to generate an answer to the content of the question by referring to at least one of the information on laws and regulations, legal precedents, books, and guidelines, and the extracted information on the internal regulations of the organization.

[0626] In step S3626, the question processing module 2044 of the server 20 inputs the generated prompt into the LLM (server 95 of the large-scale language model service) and performs language processing to generate an answer that is in line with the inquiry content and the basis for that answer (including information on company regulations).

[0627] In this way, the question processing module 2044 causes an information processing system that performs language processing to generate an answer using at least the information on the internal regulations of the organization extracted in step S3624. The question processing module 2044 provides the generated prompt to the information processing system that performs language processing, thereby causing the information processing system that performs language processing to generate an answer in accordance with the prompt.

[0628] In step S3627, after receiving the response from the LLM (large-scale language model service server 95), the question processing module 2044 of the server 20 formats the content, verifies it, and performs processing to compile the entire response sentence into a final data set.

[0629] The question processing module 2044 presents to the user the answer generated in step S3626, which includes information on the internal rules of the organization that are the basis for the answer.

[0630] In step S3612, the terminal 10 displays the final answer (answer text and basis information) transmitted from the server device on the user's screen so that the user can check it.

[0631] <Modification of the eleventh embodiment> The server 20 may collect information on cases in which changes to company rules resulted in disadvantageous changes and cases in which changes to company rules did not result in disadvantageous changes by referring to information on legal precedents regarding changes to company rules.The server 20 may accept input of a question regarding changes to company rules, and generate a prompt including information on the content of the change to company rules related to the question and legal precedents showing cases in which the change resulted in disadvantageous changes and cases in which the change did not result in disadvantageous changes, and including an instruction for evaluating whether the content of the change to company rules related to the question constitutes a disadvantageous change.The server 20 may provide the generated prompt to the server 95 of the large-scale language model service, thereby receiving an answer in which the server 95 of the large-scale language model service evaluates the likelihood that the change to company rules related to the content of the question constitutes a disadvantageous change, and present the answer to the user.

[0632] In addition, when generating an answer to a question, the server 20 may generate the answer in accordance with the company's internal regulations, distinguishing whether the internal regulations are legally required (whether a reference relationship is defined between the provisions of the internal regulations and information on laws and regulations or information on precedents) or whether the internal regulations are unique and not legally required (whether a reference relationship is not defined between the provisions of the internal regulations and information on laws and regulations or information on precedents), and present the user with information on the internal regulations that are the basis for the answer.

[0633] <23 Operation (Processing Flow of the Twelfth Embodiment)> In the twelfth embodiment, an example of generating questions in a system that collects internal posts and meeting audio, or receives posts about whether they violate laws and regulations or constitute harassment, and generates answers to the questions will be described. In this way, a technology for evaluating whether internal posts and meeting audio constitute violations of laws and regulations or harassment will be described.

[0634] FIG. 37 is a diagram showing the flow of processing for detecting questions from in-house posts and conversations and generating answers.

[0635] The following processing flow can realize a system that automatically detects questions and concerns from transcription data of internal posts and conversations and generates appropriate answers. For example, it can detect messages that may violate laws and regulations in real time from internal posts and voice conversations, generate questions and answers, and provide feedback on potential violations of laws and regulations as they occur, thereby establishing a system to prevent violations of laws and regulations.

[0636] In step S3711, the terminal 10 accepts a post on a chat tool or a conference system, or an audio conversation from the user (a natural statement including a question or doubt may be made). In the case of a conversation, speech recognition processing is performed in real time or after recording, and the content of the statement is converted into text data.

[0637] In step S3712, the terminal 10 transmits the text-converted post / conversation data to the server 20 via the network.

[0638] The question processing module 2044 of the server 20 detects the content of comments from posts and voice conversations within the organization, generates questions, obtains answers to the generated questions, and presents them to the user. This will be explained below.

[0639] The memory unit 202 is configured to store at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law, and to store data in a graph-type data structure that defines a reference relationship between information on each provision of laws and regulations, information on case law, information on books, or information on guidelines.

[0640] The memory unit 202 is configured to store information on the organization's internal regulations, and to store graph-type data structure data that defines a reference relationship between information on each clause of the organization's internal regulations and at least one of information on each clause of laws and regulations, information on legal precedents, information on books, and information on guidelines.

[0641] In step S3721, the question processing module 2044 of the server 20 performs preprocessing on the received text data, such as format standardization and noise removal, to prepare the data for easy analysis. Using natural language analysis and keyword extraction techniques, the module detects from the preprocessed data any comments that may indicate the intent of the question or concern about legal violations.

[0642] In this way, the question processing module 2044 acquires at least one of information on communication between users and information transmitted to the outside in business activities.

[0643] (1) Automatically detects user comments and notifies them of possible legal violations The question processing module 2044 may acquire at least one of information on communication between users or information transmitted to the outside in business activities, and may acquire this information without the user having to perform an operation to specify this information.

[0644] (2) Source of information questioning whether a violation of laws and regulations exists More specifically, the following may be sources of information to inquire into whether a violation of laws and regulations exists: Communication information between users: There may be messaging tools, emails, document creation tools, etc. that each user can post and view within the organization. Information disseminated externally in business activities: This may include materials used in public relations and sales, catalogs, banners, landing page images, and text. The question processing module 2044 may acquire information on communication between users by acquiring data accumulated in at least one of a messaging tool that allows users to view messages posted by other users, email, transcription data of voice conversations between users, and a document tool that allows users to set shared access rights.

[0645] The question processing module 2044 acquires information to be disseminated externally in business activities by acquiring data created by at least either the public relations department or the sales department, and may acquire at least one of the following data: information in a sales catalog, a web page banner, and an image or text on a landing page.

[0646] (3) Identification of subjects for legal violation judgment The question processing module 2044 analyzes text posts between users and voice conversation logs to extract targets for judgment from the vast amount of text data. Based on a collection of case studies, the question processing module 2044 identifies parts that are similar to cases of legal violations and compliance violations. For example, databases that can be referenced include cases of violations by other companies published by government organizations, cases of administrative sanctions (consumer protection), books (books that explain specific laws and regulations such as the Subcontract Act), etc.

[0647] The question processing module 2044 acquires information on communication between users, such as text posts between users and logs of voice conversations. The question processing module 2044 may refer to data on a collection of legal violation case studies and detect, from the information acquired in step S3721, any portions similar to cases recorded in the collection of legal violation case studies.

[0648] More specifically, the question processing module 2044 may perform the above detection based on the degree of agreement with a collection of case studies of statements such as harassment.

[0649] The question processing module 2044 may refer to data on a collection of legal violation case studies and detect, from the information acquired in step S3721, any parts that are similar to cases recorded in the collection of case studies about statements that violate legal violations.

[0650] More specifically, the question processing module 2044 may perform the above detection based on the degree of similarity between the content of the business request and a collection of cases that constitute a violation of laws and regulations (for example, a violation of the Subcontract Act) based on data such as documents and messages sent and received with users outside the organization, such as application forms and the contents of emails with other companies.

[0651] The question processing module 2044 may refer to data on a collection of legal violation cases and detect any parts of the information acquired in step S3721 that are similar to cases recorded in the collection of cases regarding the content of work requests that violate laws and regulations.

[0652] The question processing module 2044 may read wording from images of catalogs, banners, and landing pages, and detect whether there is any legal violation or whether it conforms to the standards of these pages. The question processing module 2044 may refer to at least either data on a collection of examples of illegal expressions or information on design standards, and detect at least either parts of the information acquired in step S3721 that are similar to examples included in the collection of examples of illegal expressions or parts that do not conform to the design standards.

[0653] The question processing module 2044 may extract a portion of a conversation between users that indicates an intention to ask a question. For example, the question processing module 2044 may extract a portion of a conversation that indicates an intention to confirm whether or not a legal violation has occurred, such as, "Is this okay (such as harassment or possible violation of law)?" The question processing module 2044 may acquire information about communication between users by acquiring at least one of text posts between users and logs of voice conversations. The question processing module 2044 may detect a portion of the information acquired in step S3721 that includes a statement indicating an intention to evaluate the possibility of a legal violation.

[0654] The above processing allows the amount of data to be processed by an information processing system that performs language processing (for example, a server 95 for a large-scale language model service) to be an extracted portion rather than the entire text of the post or conversational speech transcription data, thereby reducing the processing load on the information processing system that performs language processing. Furthermore, it is expected that the time it takes to receive a response from the information processing system that performs language processing will be shortened, and it may be possible to respond in real time while the post or conversational speech is being made.

[0655] In step S3722, the question processing module 2044 of the server 20 extracts, based on the detected question and the detected user's doubt, evidence information and related documents such as books, laws, precedents, guidelines, and company regulations from the graph database using vector search and reference relationships in the graph data structure.

[0656] In this way, the question processing module 2044 searches for each piece of data in the graph data structure based on the information acquired in step S3721.

[0657] For example, the question processing module 2044 may first search for books and guidelines, extract laws and precedents that have a reference relationship with the books and guidelines, and generate an answer to the question. The question processing module 2044 searches for at least one of book information and guideline information based on the content of the question. The question processing module 2044 identifies at least one of the searched book information and guideline information and at least one of law information and precedent information that have a reference relationship defined in the data of the graph data structure.

[0658] When a reference relationship is also defined for information on company regulations in the graph-type data structure, the question processing module 2044 may generate an answer taking the company regulations into consideration. The question processing module 2044 searches for at least one of book information or guideline information based on the content of the question. The question processing module 2044 identifies at least one of the searched book information or guideline information and at least one of the legal information or precedent information for which a reference relationship is defined in the data of the graph-type data structure. The question processing module 2044 extracts information on the company regulations of the organization that define a reference relationship for at least one of the searched book information or guideline information and at least one of the identified legal information or precedent information.

[0659] In step S3723, the question processing module 2044 of the server 20 combines the search results (related documents, citation graph information, etc.) with the original question content and automatically generates a prompt to input to the LLM (language processing system; large-scale language model service server 95).

[0660] The question processing module 2044 generates a prompt that includes at least a part of the information acquired in step S3721 and that includes an instruction for evaluating the possibility of a violation of the law.

[0661] The question processing module 2044 may generate a prompt that includes at least one of information about the searched book or information about the guideline, and at least one of information about laws and regulations or information about case law identified based on reference relationships from data in a graph-type data structure, and that includes an instruction to generate an answer by referring to this information, and to evaluate the possibility of a violation of laws and regulations.

[0662] The question processing module 2044 may generate a prompt that includes at least one of information on the searched book or information on the guideline, at least one of information on laws and regulations or information on case precedents identified based on reference relationships from data in a graph-type data structure, and information on the extracted internal regulations of the organization, and the prompt includes the above instruction sentence.

[0663] The question processing module 2044 may generate a prompt by including the similar portion detected in step S2731 in the prompt.

[0664] In step S3724, the question processing module 2044 of the server 20 inputs the generated prompt into the LLM (large scale language model service server 95) and generates an answer to the question and an answer including an assessment of the likelihood of legal or compliance violations.

[0665] In this way, the question processing module 2044 provides the prompt generated in step S3723 to the information processing system that performs language processing, and accepts the answer that the information processing system that performs language processing outputs in response to the prompt.

[0666] In step S3725, the question processing module 2044 of the server 20 adds information on the books, laws, precedents, guidelines, and company regulations that form the basis of the answer to the generated answer, formats it, and compiles it into a format that can be presented to the user. The question processing module 2044 again transmits the completed answer and the basis information to the terminal 10, and notifies and provides feedback to the user.

[0667] In this way, the question processing module 2044 presents to the user an answer including the result of evaluating the possibility of a violation of the law, which is output by the information processing system that performs language processing.

[0668] Here, the question processing module 2044 presents to the user an answer including information on books, guidelines, laws, and precedents that served as the basis for generating the answer. The question processing module 2044 may be configured to present to the user an answer that includes at least one of information on books or guidelines, and at least one of information on laws and precedents that served as the basis for generating the answer by the information processing system that performs language processing.

[0669] The question processing module 2044 may present to the user an answer that includes information on books, guidelines, laws, and precedents that served as the basis for generating an answer, as well as company rules. Alternatively, the question processing module 2044 may arbitrarily select and present to the user information on books, guidelines, laws, precedents, and company rules as the basis for generating an answer. For example, the question processing module 2044 may present information on company rules rather than information on books, guidelines, laws, and precedents as the basis for generating an answer. Furthermore, depending on the user asking a question, a combination of information on books, guidelines, laws, precedents, and company rules may be determined and presented as the basis for generating an answer. For example, the user asking a question may place importance on company rules as the basis for generating an answer. For example, the question processing module 2044 may accept an operation from the user to specify at least one of books, guidelines, laws, precedents, and company rules as the basis for generating an answer, and may generate an answer according to the user's specification, or may present to the user content that serves as the basis for generating an answer according to the user's specification.

[0670] In this way, the question processing module 2044 may include in the answer to the user at least one of book information or guideline information, at least one of legal information or precedent information, and information on the organization's internal regulations, which were the basis for generating the answer using the information processing system that performs language processing.

[0671] In step S3713, the terminal 10 displays the notified answer and the basis information to the user, thereby enabling the user to check the answer and the content on which the answer is based, and to ask additional questions or provide feedback as necessary.

[0672] <Modification of the twelfth embodiment> As described above, a technology has been described in which a system for generating answers to questions evaluates user posts and user conversation voices for legal violations and harassment. When the server 20 presents a user with an answer indicating that the user's post, etc., may violate the law through these processes, the server 20 may accept an operation from the user to anonymously consult with a person in a specific department within the organization or an external expert. In response to this operation, a message including the user's post, etc., that was the subject of the question, and the answer generated by the server 20 may be sent to the person in the specific department or the external expert.

[0673] <24 Operation (Processing Flow of the Thirteenth Embodiment)> In the thirteenth embodiment, we will explain a technology for ranking content such as books based on the reference relationships in a graph-type data structure for multiple types of content. Based on the results of these rankings, it is possible to set the priority of documents when extracting documents that serve as evidence for a question, and to control the display order when presenting content to a user.

[0674] Figure 38 shows the process flow for analyzing the reference relationships in a graph-type data structure, determining the priority for presenting multiple types of content that served as the basis for generating an answer to a question, and presenting that content.

[0675] In the following processing flow, the terminal accepts input from the user, and the server extracts content according to the question, analyzes the reference relationships between content in a graph-type data structure, determines the priority of the content, and then generates an answer using LLM and integrates the results. This allows the user to easily determine which evidence content they should focus on referring to along with the answer.

[0676] In addition, the following processing flow makes it even easier for users to select from a large amount of content by displaying content with many citations, systematic basic books, or types of literature that are considered important in a particular field (such as those that emphasize guidelines) at the top of the list.

[0677] In step S3811, the terminal 10 accepts a question input from the user and transmits the content of the question to the server 20.

[0678] The question processing module 2044 of the server 20 determines the ranking of content by taking into account the reference relationships in the graph-type data structure. In a system that generates answers to questions using LLM, when presenting supporting literature, the content is displayed in the ranking based on the reference relationships in the graph-type data structure. This is explained below.

[0679] The storage unit 202 is configured to store data with a graph-type data structure that defines reference relationships between multiple types of content.

[0680] The memory unit 202 is configured to store multiple types of content, including at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law, and to store data in a graph-type data structure that defines a reference relationship between information on each provision of laws and regulations, information on case law, information on books, or information on guidelines.

[0681] The memory unit 202 is configured to store information on the organization's internal regulations, and to store data of a graph-type data structure that defines a reference relationship between information on each clause of the organization's internal regulations and at least one of information on each clause of laws and regulations, information on legal precedents, information on books, and information on guidelines.

[0682] In step S3821, the question processing module 2044 of the server 20 accepts a question input from the user. Based on the received question, the question processing module 2044 extracts initial related candidate content from the graph data structure using a method such as vector search. Based on the content of the accepted question, the question processing module 2044 extracts multiple pieces of content by referring to the data in the graph data structure.

[0683] The question processing module 2044 performs a search (vector search) based on the content of the question, extracts content that defines a reference relationship with the content of the search results from the graph database, and generates an answer using these search result contents and extracted content as the basis for generating an answer using LLM (large scale language model service server 95).The question processing module 2044 searches multiple types of content based on the content of the received question, and identifies content that defines a reference relationship in a graph data structure for the content that becomes the search result.

[0684] Furthermore, the question processing module 2044 may perform the following processing by further referencing a graph-type database associated with internal regulations, and assigning a higher score to clauses of internal regulations that are more frequently cited, thereby increasing the priority of display. The question processing module 2044 searches for at least one of book information or guideline information based on the content of the question. The question processing module 2044 identifies at least one of legal information or precedent information for which a reference relationship is defined in the graph-type data structure data with at least one of the searched book information or guideline information. The question processing module 2044 may extract information on the internal regulations of the organization that define a reference relationship with at least one of the searched book information or guideline information, and at least one of the identified legal information or precedent information.

[0685] In step S3822, the question processing module 2044 of the server 20 analyzes the reference relationships (citation and cited information, designation of basic texts, field-specific weighting, etc.) between the extracted candidate contents.

[0686] In step S3823, the question processing module 2044 of the server 20 applies evaluation rules based on the number of citations and document type (book, guideline, law, precedent, company rule, etc.) to each candidate content, and calculates and determines the presentation priority.

[0687] In step S3824, the question processing module 2044 of the server 20 constructs a prompt (question content and extracted and ranked candidate content information) for generating an answer in accordance with the determined priority, and requests the LLM to generate an answer.

[0688] In this way, the question processing module 2044 generates an answer using the content of the question and the multiple pieces of content extracted in the extracting step.

[0689] Here, the question processing module 2044 generates a prompt including the information resulting from the search performed in step S3821 and the content of the question, and provides the generated prompt to an information processing system (the large-scale language model service server 95) that performs language processing, thereby causing the information processing system that performs language processing to generate an answer corresponding to the prompt. Here, the question processing module 2044 generates a prompt including the content that is the search result of the search performed in step S3821, the content identified in the graph-type data structure, and the content of the question, and including an instruction statement for generating an answer to the question by referencing these contents. The question processing module 2044 also generates a prompt that includes at least one of information on the searched book or information on the guideline, at least one of information on laws and regulations or information on legal precedents identified based on reference relationships from the data in the graph-type data structure, and information on the extracted internal regulations of the organization, and includes the content of the question.

[0690] In step S3825, the question processing module 2044 of the server 20 integrates the answer generated from the LLM with the multiple underlying contents (listed in order of priority) and returns the answer to the terminal 10 as the generated result.

[0691] The question processing module 2044 presents the answer generated in step S3824 to the user of the terminal 10 along with the plurality of pieces of content that served as the basis for generating the answer. Here, the question processing module 2044 presents the plurality of pieces of content in order of priority based on the reference relationships in the data of the graph data structure for the pieces of content that served as the basis.

[0692] Furthermore, the question processing module 2044 presents the content that is the search result of the search performed in step S3821 and the content identified in the graph data structure as the content that served as the basis for the answer, together with the answer.

[0693] (1) Priority of content presentation based on graph-type data structure: Cited The question processing module 2044 may prioritize a book or guideline by giving it a higher score the more times it is cited (the more edges) in the graph-type data structure.

[0694] The question processing module 2044 may display multiple pieces of content that served as the basis for generating an answer, giving them a higher priority the more reference relationships with other content defined in the graph-type data structure (the more citations there are).

[0695] (2) Priority of presenting content based on graph-type data structures: designated as systematic basic texts The question processing module 2044 may assign a higher score to a book that is a more systematic basic book.

[0696] The question processing module 2044 may display a plurality of pieces of content that serve as the basis, with higher priority given to books among the content that serve as the basis, such that the more systematic the book, the higher the priority.

[0697] For example, the question processing module 2044 may set a book designated as a basic book or a book determined to be a basic book by summarizing the contents of a book review as a systematic basic book. The question processing module 2044 may set a high priority for a book designated as a systematic basic book by a user or a book determined to be a systematic basic book based on the contents of a book review.

[0698] (3) Regarding internal regulations, determine the priority of content presentation based on a graph-type data structure The question processing module 2044 may display multiple pieces of content that serve as the basis by assigning a higher priority to the information on the internal regulations of the organization that serves as the basis, the more reference relationships with other content are defined in the graph-type data structure.

[0699] (4) Identifying important content types for each field The question processing module 2044 may refer to the graph-type data structure to determine the type of important content for each field (e.g., which of books or guidelines is more important?). For example, the question processing module 2044 may analyze the citation relationships of content obtained by searching literature, tally the number of citations for each type of content (book, guideline), and determine which type is more important. In this way, the question processing module 2044 may determine the importance of each type of content by analyzing the reference relationships defined between multiple types of content that served as evidence and other content in the graph-type data structure.

[0700] The question processing module 2044 may preferentially present to the user content of the type determined to be important in the above manner. In this way, the question processing module 2044 may control the display order of each content based on the results of analyzing the citation relationships for each type, such as books and guidelines, for each field (if a guideline is cited many times, the guideline is important).

[0701] In this way, the question processing module 2044 may preferentially present the content that is the basis for a question of higher importance based on the results of determining the level of importance among the multiple types of content that are the basis for a question.

[0702] For example, the question processing module 2044 may prioritize displaying the type of content with the highest importance for each field, and when switching between types (books, guidelines, etc.) using tabs, the prioritized tab may be displayed as selected in the initial display.

[0703] For example, when displaying contents sorted by type, the question processing module 2044 may display more important types of contents at the top.

[0704] In step S3812, the terminal 10 displays the prioritized list of answers and supporting content received from the server 20 to the user.

[0705] <Modification of the thirteenth embodiment> In the above embodiment, an example has been described in which documents are evaluated based on the citation relationships of each document in a graph-type data structure, documents are extracted in response to a question, and the priorities of the documents to be referenced in a prompt to the large-scale language model service server 95 are determined according to the evaluation results of the documents. Alternatively, when extracting documents in response to a question, the documents may be extracted without using the evaluation results of the documents based on the citation relationships of each document in the graph-type data structure, and the documents may be provided as documents to be referenced in a prompt to the large-scale language model service server 95 according to the scores of each document in the extraction results, and when presenting to the user a list of documents that served as the basis for generating an answer, the list of documents that served as the basis may be presented based on the results of evaluating the documents based on their citation relationships.

[0706] <25. Operation (Processing Flow of the Fourteenth Embodiment)> In the fourteenth embodiment, a technique for controlling the display order of documents presented to a user (reader) will be described according to the reader's experience, knowledge, ability, etc. The difficulty level of a document is evaluated by analyzing the reference relationships of each document in a graph-type data structure, and documents with many citations are considered to be of high difficulty.

[0707] FIG. 39 is a diagram showing the flow of processing for evaluating the difficulty level of a document based on the citation relationships in a graph database.

[0708] The following processing flow shows a method for objectively evaluating the difficulty level of documents by utilizing the reference relationships (citation relationships) between nodes in a graph database, thereby building a foundation for presenting appropriate documents in response to user requests.

[0709] Furthermore, in the processing flow below, the difficulty level of a document can be evaluated by combining analysis of vocabulary, writing style, and technical terminology in addition to citation relationships.

[0710] In step S 3911 , the terminal 10 receives a request from the user to select a document to be evaluated or to evaluate the difficulty level of each document, and transmits the request information to the server 20 .

[0711] The data structure definition module 2046 evaluates the difficulty of documents using a graph database, as explained below.

[0712] The storage unit 202 is configured to store data with a graph-type data structure that defines reference relationships between multiple types of content.

[0713] The memory unit 202 is configured to store multiple types of content, including at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law, and to store data in a graph-type data structure that defines a reference relationship between information on each provision of laws and regulations, information on case law, information on books, or information on guidelines.

[0714] The storage unit 202 stores data with a graph-type data structure in which the title of at least one of the books or guidelines is at the apex, and the content of at least one of the books or guidelines is at the apex, and data defining a reference relationship between the title apex and the content apex in a parent-child relationship.

[0715] In step S3921, after receiving a user request, the data structure definition module 2046 of the server 20 accesses the graph database and retrieves the node (title and content) of the document to be evaluated and information on the citation and reference relationships of other content linked to it.

[0716] In step S3922, the data structure definition module 2046 of the server 20 analyzes the citation relationships set for each child node (divided part of the document content) corresponding to the target document, and extracts the number of other contents referenced by each child node.

[0717] In this way, data structure definition module 2046 identifies the number of times that content references other content by referring to the graph-type data structure in storage unit 202. Here, data structure definition module 2046 may identify the number of times that content references other content by counting the number of reference relationships defined (for example, when reference relationships are defined in multiple places in the same content, the module may count these individually defined reference relationships), or may count the number of other content pieces with reference relationships defined (for example, when reference relationships are defined in multiple places in the same content, the module may count the number of other content pieces as referencing one content piece, regardless of the count results of these individually defined reference relationships). ·Method of counting the number of times other content is referenced The data structure definition module 2046 may also tally the number of other contents with which each child node defines a reference relationship in a graph database that associates laws, precedents, books, and guidelines, where the content is defined as a parent node and the content is divided into child nodes to define a parent-child relationship.

[0718] The data structure definition module 2046 may determine the number of references by counting the number of other contents for which the child vertices corresponding to each portion of the book or guideline content define a reference relationship.

[0719] The data structure definition module 2046 may aggregate the number of other contents for which child nodes define reference relationships into the parent node. The data structure definition module 2046 may aggregate and store the number of other contents for which child vertices corresponding to each part of the content of a book or guideline define reference relationships into the parent vertex.

[0720] In step S3923, the data structure definition module 2046 of the server 20 aggregates the number of citations of each extracted child node into the parent node and calculates a difficulty index based on the citation relationships as a whole (e.g., the more citations, the higher the difficulty). If necessary, the data structure definition module 2046 may integrate information on the results of text analysis, such as vocabulary difficulty, writing style, and terminology usage, with information on the results of evaluation based on citation relationships, to calculate an overall difficulty score for each document.

[0721] As described above, the data structure definition module 2046 may evaluate each piece of content based on the number of other pieces of content that define a reference relationship that is aggregated at the parent vertex for the content.

[0722] Here, the data structure definition module 2046 may evaluate the difficulty of a document as being higher for the reader the more other content is cited, as a method of evaluating the difficulty of the document. For example, the document may be argued with abundant citations to support the argument, and therefore may be longer than a certain amount.

[0723] The data structure definition module 2046 may evaluate the content such that the greater the number of references made by the content to other content, the higher the degree of difficulty of the content.

[0724] Furthermore, the data structure definition module 2046 may evaluate the difficulty level higher the more other content that the content that the content references references (the more difficult the content that is cited).The data structure definition module 2046 may evaluate the difficulty level of the content as higher the more other content that the content references.

[0725] In step S3924, the data structure definition module 2046 of the server 20 classifies the documents into multiple levels, from low to high, based on the calculated difficulty scores. This makes it possible to set a display priority that takes into account the user's future browsing history and cooperation with the user's demographic information.

[0726] In step S3925, the data structure definition module 2046 of the server 20 returns information on the final evaluation results and document classification results to the terminal 10.

[0727] In step S3912, the terminal 10 presents to the user on the screen evaluation information such as the difficulty level, classification, and display priority order of each document based on the evaluation results received from the server.

[0728] Figure 40 shows the process flow for controlling the difficulty level of documents presented to a user in accordance with the user's browsing habits (difficulty level trends) by utilizing the evaluation results of document difficulty based on a graph database.

[0729] The following processing flow can improve research efficiency, learning efficiency, etc. by presenting literature of an appropriate level of difficulty for each user based on the user's past browsing history and profile information. In addition, by comparing the evaluation results of the literature's difficulty level with the user's browsing habits, the display order can be dynamically changed, making it possible to present content in line with the user's improving ability.

[0730] In the following explanation, as described above, an example will be described in which the server 20 accepts a question input from a user, extracts the content that forms the basis of the answer by referring to a graph-type data structure, and then uses an information processing system (server 95 of the large-scale language model service) that performs language processing to output an answer in response to a prompt that generates an answer by referring to the literature that forms the basis, and presents the answer and the content that formed the basis for generating the answer to the user, in which the content that formed the basis for generating the answer is displayed in a row according to the difficulty level tendency for each user.

[0731] The user management module 2043 accepts operations from users to register their own profile information (e.g., qualifications, years of experience) and past literature browsing history (such as the difficulty level trends of the literature they browsed). Based on the received user information, the user management module 2043 analyzes and aggregates each user's literature browsing trends and difficulty level preferences to generate a user profile (difficulty level trend profile).

[0732] In step S4021, the question processing module 2044 of the server 20 acquires the difficulty evaluation results (evaluation results based on citation relationships and other text analysis results) of documents that have already been evaluated from the graph database.

[0733] In step S4022, the question processing module 2044 of the server 20 compares the acquired document evaluation results with the user's difficulty tendency profile and calculates the priority (display order) of documents that match the user's ability. For example, the question processing module 2044 sets documents with low difficulty to the top for a user who tends to prefer low difficulty.

[0734] In step S4023, the question processing module 2044 of the server 20 compiles the list of documents after adjusting the order according to the difficulty level of the documents and the evaluation information of the documents (e.g., number of citations, usage of technical terms, etc.) and formats it as data for generating answers.

[0735] In this way, the question processing module 2044 classifies documents according to their level of difficulty and presents them to the user. The question processing module 2044 classifies each piece of content by level of difficulty based on the results of evaluating the level of difficulty of each piece of content. The question processing module 2044 presents each piece of content classified by level of difficulty to the user.

[0736] In a system for generating an answer to a question as described above, the question processing module 2044 presents the answer along with the literature that served as the basis for generating the answer. The question processing module 2044 accepts a question input from a user. The question processing module 2044 searches for content based on the content of the accepted question. The question processing module 2044 generates a prompt that includes the searched content and the content of the question, and that includes an instruction to generate an answer by referring to the searched content. The question processing module 2044 provides a prompt to an information processing system that performs language processing, thereby obtaining an answer corresponding to the prompt from the information processing system that performs language processing. The question processing module 2044 then presents the obtained answer, including the content that served as the basis for generating the answer, to the user. At this time, as described above, the content that served as the basis is presented in order according to the difficulty level of the content.

[0737] The question processing module 2044 may present documents according to the level of difficulty based on the trend (history) of the difficulty of documents viewed by the user. The question processing module 2044 may present to the user content that served as the basis for generating an answer, and may present to the user content according to the level of difficulty based on the trend of the evaluation results of the difficulty of the content that was viewed in the user's content viewing history.

[0738] In a system that generates answers to questions, the question processing module 2044 may generate answers tailored to the user's ability (difficulty level trend) in accordance with the difficulty level of the documents viewed by the user. The question processing module 2044 accepts a question input from a user. The question processing module 2044 searches for content based on the content of the accepted question. The question processing module 2044 generates a prompt including the searched content and the content of the question, the prompt including an instruction statement to generate an answer by referring to the searched content. The question processing module 2044 provides a prompt to an information processing system that performs language processing, thereby acquiring an answer corresponding to the prompt from the information processing system that performs language processing. The question processing module 2044 presents the acquired answer to the user, including the content that served as the basis for generating the answer. Here, in acquiring the answer, the question processing module 2044 may generate a prompt including an instruction statement to generate an answer tailored to the difficulty level in accordance with the trend in the difficulty level evaluation results of the content viewed by the user in the user's content viewing history. The question processing module 2044 may obtain, from a system that performs language processing, an answer generated according to the level of difficulty of the user as an answer in response to the prompt.

[0739] The question processing module 2044 may present literature of a difficulty level according to whether the user has a qualification and how long the user has held the qualification. The question processing module 2044 receives at least one of user information, such as whether the user has a specific qualification and how long the user has held the specific qualification. The question processing module 2044 sets the difficulty level of content to be presented to the user according to the user information. The question processing module 2044 sets the priority of content to be presented to the user based on the evaluation result of the difficulty of each content and the difficulty level set for the user. The question processing module 2044 may present content to the user according to the set priority.

[0740] In step S4011, the terminal 10 receives the literature list and evaluation information that are tailored to the user's browsing habits and that are returned from the server 20, and presents them on the screen as optimal literature information.

[0741] <26 Operation (Processing flow of the 15th embodiment)> In the fifteenth embodiment, a technology for searching for experts in a certain field will be described. For example, a technology will be described that utilizes a graph-type data structure to accept a query to find "an expert or lawyer who is knowledgeable in a specific field (e.g., cyber attacks)" and present experts based on the accumulated literature.

[0742] FIG. 41 is a diagram showing the flow of processing in which a question is asked, an answer is generated, and the content that served as the basis for generating the answer is presented on an author-by-author basis.

[0743] The processing flow below is different from conventional expert matching services, which match people seeking advice with experts. Instead, the service automatically extracts information on relevant laws, precedents, books, etc. from the content of the question and presents it by author, allowing users to grasp at a glance the source and reliability of each piece of information (assessed by the number of citations), making it even easier to find experts based on these perspectives.

[0744] In step S4111, the user accesses the question input screen and inputs the consultation content, question, or problem in the text form of the terminal 10. The input question data is sent to the server 20 via the communication function.

[0745] To make it easier to find an expert, the question processing module 2044 of the server 20 refers to the graph database to extract content such as books that correspond to the question, and displays the extracted results by author.

[0746] The storage unit 202 is configured to store data in a graph-type data structure that defines reference relationships between multiple types of content. Author information is associated with each content.

[0747] The memory unit 202 is configured to store multiple types of content, including at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law, and to store data in a graph-type data structure that defines a reference relationship between information on each provision of laws and regulations, information on case law, information on books, or information on guidelines.

[0748] In step S4121, the question processing module 2044 of the server 20 analyzes the received question and converts it into a vector representation for internal processing (preparation for vector search). The question processing module 2044 executes a vector search algorithm to search for content related to the question. For the search result content, the graph database is referenced to extract content with defined reference relationships. The question processing module 2044 also obtains information such as the citation relationships of each content (the number of other content citations, the number of times it has been cited, etc.).

[0749] In this way, the question processing module 2044 receives a question input from the user. The question processing module 2044 performs a search based on the content of the received question, and searches for content corresponding to the content of the question by referring to data in a graph data structure.

[0750] The question processing module 2044 performs a search based on the content of the question, and extracts the content of the search results and the content for which a reference relationship is defined in the data of the graph type data structure.

[0751] In step S4122, the question processing module 2044 of the server 20 groups each of the retrieved and extracted contents based on the accompanying author information, and determines the presentation order for each author based on the number of citations and the magnitude of the reference relationships.

[0752] In step S4123, the question processing module 2044 of the server 20 integrates the question content and the supporting content organized by author to generate a prompt for the LLM (Language Learning Module). The generated prompt is sent to the LLM, and a result including the answer to the question and the content information that is the supporting content for generating the answer is received.

[0753] In this way, the question processing module 2044 refers to a graph database that associates laws, precedents, books, and guidelines, and presents an answer to the question while also presenting the content on which the answer is based, by author. Specifically, the question processing module 2044 generates a prompt that includes information on the searched content and the question. The question processing module 2044 provides the generated prompt to an information processing system that performs language processing, causing the information processing system that performs language processing to generate an answer in accordance with the prompt.

[0754] In generating a prompt, the question processing module 2044 generates a prompt that includes the content of the search results, the extracted content, and the content of the question, and that includes an instruction to generate an answer by referring to these contents.

[0755] In step S4124, the question processing module 2044 of the server 20 further organizes and formats the answer and evidence content information received from the information processing system that performs language processing (the large-scale language model service server 95) by author, and converts it into a data format for presentation to the user. The question processing module 2044 returns the formatted answer and the categorized and prioritized evidence content information to the terminal 10.

[0756] In this way, the question processing module 2044 classifies the search result contents by author and presents them to the user.

[0757] The question processing module 2044 presents answers to questions and the content that served as the basis for generating the answers, and classifies and presents the content that served as the basis for generating the answers by author. · How content is presented by author The question processing module 2044 may determine the priority of content to be presented for each author according to the magnitude of the citation relationship. For example, the more citations there are in a content, the higher the priority may be. The question processing module 2044 may also prioritize content that has a greater number of reference relationships defined with other content in a graph-type data structure. For example, it is possible to easily find the author of a work that cites many other documents in its discussion. -Display the number of extracted documents by author The question processing module 2044 may display the number of extracted contents for each author of the presented content. For example, if five works are extracted for a certain author, the number of extracted contents for that author will be displayed as "5." This makes it easy to find authors who have produced many works in the field corresponding to the question. Workflow after selecting experts When the question processing module 2044 receives an operation from the user to select an expert, the question processing module 2044 may be configured to enable messages to be sent and received between the selected expert and the user.

[0758] The question processing module 2044 receives an operation to specify an author from the user. For example, an answer to a question may be generated and displayed while the underlying content is displayed, and the author may be specified while information about the author of the content is displayed. The question processing module 2044 transmits and receives messages between the author specified by the specification operation and the user.

[0759] When the question processing module 2044 receives a question from a user that seeks an expert, it presents content by author, but if the question does not include a request for an expert, it may display the answer to the question while presenting the content that is the basis for the answer without classifying it by author.

[0760] When the question processing module 2044 receives an input of a question that seeks an expert, it may present content by author, whereas when the question processing module 2044 receives an input of a question that does not involve the seeking of an expert, it may present content without classifying it by author.

[0761] In step S4112, the terminal 10 displays the received results together with the answer portion on the user interface as evidence content classified by author (with indexes such as the number of citations and priority).

[0762] <Modification of the fifteenth embodiment> In the above embodiment, an example has been described in which, when a question is received and content such as books is searched for based on the question, the content is grouped by author, and a prompt including an instruction to refer to the content grouped by author is generated in a prompt to be sent to the server 95 of the large-scale language model service.

[0763] Alternatively, in response to a question, literature may be searched for using a vector search or the like based on the question, and at the stage of generating a prompt, answers corresponding to the prompt may be generated by the large-scale language model service server 95 without grouping by author, and when presenting the literature that served as the basis for generating the answer, the answers may be grouped and displayed by the author of the literature.

[0764] <Other> The above-described embodiments may be combined with each other.

[0765] The network is composed of the Internet, a LAN, various mobile communication systems constructed by wireless base stations, etc. For example, the network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi (registered trademark)) that can connect to the Internet via a predetermined access point. In the case of a wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), and Bluetooth (registered trademark). In the case of a wired connection, the network also includes a direct connection using a USB (Universal Serial Bus) cable, etc.

[0766] In addition, a computer can be virtually realized by distributing all or part of each hardware configuration across multiple computers and connecting them via a network. In this way, the concept of a computer includes not only a computer housed in a single housing or case, but also a virtualized computer system.

[0767] A database refers to a relational database, which manages data sets called masters and tables in a tabular format structurally defined by rows and columns, by relating them to each other. In a database, a table is called a table, a master, a column in a table is called a column, and a row in a table is called a record. In a relational database, relationships between tables and masters can be set and associated.

[0768] Typically, each table and each master has a column set as a primary key to uniquely identify a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, or update records in specific tables and masters stored in the storage unit according to various programs.

[0769] Furthermore, by storing data, various programs, and various databases in the storage unit, it can be considered that the information processing device and information processing system according to the present disclosure have been manufactured.

[0770] Note that the databases and masters in this disclosure may include any data structure in which information is structurally defined (such as a list, dictionary, associative array, or object). The data structure also includes data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

[0771] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that implements the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for providing such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tape, non-volatile memory cards, and ROMs.

[0772] Furthermore, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, and Java (registered trademark).

[0773] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

[0774] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes programs stored in memory.

[0775] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0776] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0777] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0778] (Addendum) The matters described in the above embodiments will be supplemented below.

[0779] (Appendix 1) A program for operating a computer having one or more computer processors, configured to store in a memory unit data of a graph-type data structure that defines reference relationships between multiple types of content, the program causing the one or more computer processors to execute the following steps: accepting a question input from a user; extracting multiple pieces of content by referring to the data of the graph-type data structure based on the content of the question accepted in the accepting step; generating an answer using the content of the question and the multiple pieces of content extracted in the extracting step; and presenting the answer generated in the generating step together with the multiple pieces of content that served as the basis for generating the answer, wherein in the presenting step, the multiple pieces of content are presented in order of priority based on the reference relationships in the data of the graph-type data structure for the content that served as the basis.

[0780] (Appendix 2) The program described in Appendix 1 is configured to store, in a memory unit, as multiple types of content, at least one of information on books including legal books or information on guidelines including legal guidelines, information on laws and regulations, and information on case law, and to store data of a graph-type data structure in which reference relationships are defined between information on each clause of the laws and regulations, information on case law, information on books, or information on guidelines, and in a generating step, to search for each piece of data in the graph-type data structure based on the content of the question received in the receiving step, to generate a prompt including the information resulting from the search and the content of the question, and to provide the generated prompt to an information processing system that performs language processing, thereby causing the information processing system that performs language processing to generate an answer corresponding to the prompt.

[0781] (Appendix 3) A program as described in Appendix 2, wherein in the generating step, multiple types of content are searched for based on the content of the question received in the receiving step, an...

Claims

1. A program for operating a computer having one or more computer processors, The storage unit is configured to store data of a graph-type data structure that defines reference relationships between multiple types of content, The program causes the one or more computer processors to: receiving a question input from a user; extracting a plurality of contents by referring to the data in the graph data structure based on the content of the question received in the receiving step; generating an answer using the content of the question and the plurality of contents extracted in the extracting step; a step of presenting the answer generated in the generating step together with a plurality of contents that served as a basis for generating the answer; In the presenting step, the program presents the plurality of pieces of content that serve as the basis, with a higher priority being given to the content that has a greater number of reference relationships with other content defined in the graph-type data structure data.

2. In the storage unit, the plurality of types of content include: At least one of information from books, including legal books, and information from guidelines, including legal guidelines; Information on legislation and and storing data of the graph-type data structure in which a reference relationship is defined between at least one of information on each provision of the law, information on the precedent, information on the book, and information on the guideline, In the generating step, searching for each data in the graph-type data structure based on the content of the query received in the receiving step; generating a prompt including information resulting from the search and the content of the question; 2. The program according to claim 1, wherein the generated prompt is provided to an information processing system that performs language processing, thereby causing the information processing system that performs language processing to generate a response corresponding to the prompt.

3. In the generating step, searching the plurality of types of content based on the content of the question received in the receiving step, and identifying content for which a reference relationship is defined in the graph-type data structure for the content resulting from the search, generating a prompt including the search result content, the content identified in the graph data structure, and the question, the prompt including an instruction to generate an answer by referring to these contents; 3. The program according to claim 2, wherein in the presenting step, the content that is the search result and the content identified in the graph-type data structure are presented as the content that served as the basis together with the answer.

4. 3. The program according to claim 2, wherein in the presenting step, among the content items that serve as the basis, books are given a higher priority as the books become more systematic basic works, and the plurality of content items that serve as the basis are displayed.

5. The program described in claim 4, wherein in the presenting step, a book designated as a systematic basic book by a user or determined to be a systematic basic book based on the content of a review of the book is given a higher priority as a book that is a systematic basic book.

6. the storage unit is configured to store information on internal company regulations of an organization, and to store data of the graph-type data structure in which a reference relationship is defined between information on each clause of the internal company regulations of the organization and at least one of information on each clause of the laws and regulations, information on the precedents, information on the books, and information on the guidelines, In the extracting step, searching for at least one of information about the book and information about the guideline based on the content of the question; Identifying at least one of the retrieved book information or guideline information and the law information or the case law information that has a reference relationship defined in the graph-type data structure data; and extracting information on the internal regulations of the organization that define a reference relationship with at least one of the retrieved book information or guideline information, and at least one of the identified law information or precedent information; In the generating step, generating the prompt, the prompt including at least one of the retrieved book information or guideline information, at least one of the law information or precedent information identified based on a reference relationship from the graph data structure data, and extracted information on internal regulations of the organization, the prompt including the content of the question; The program of claim 2, wherein in the presentation step, the information on the organization's internal regulations that served as the basis is given a higher priority the more reference relationships with other content defined in the graph-type data structure, and the multiple pieces of content that served as the basis are displayed.

7. 1. A computer-implemented method having one or more computer processors, comprising: The storage unit is configured to store data of a graph-type data structure that defines reference relationships between multiple types of content, The method further comprises the step of: receiving a question input from a user; extracting a plurality of contents by referring to the data in the graph data structure based on the content of the question received in the receiving step; generating an answer using the content of the question and the plurality of contents extracted in the extracting step; presenting the answer generated in the generating step together with the plurality of contents that served as the basis for generating the answer; In the presenting step, the method presents the plurality of pieces of content that serve as the basis, giving the content a higher priority the more reference relationships with other content that are defined for the content in the graph-type data structure.

8. An information processing device, The storage unit is configured to store data of a graph-type data structure that defines reference relationships between multiple types of content, a control unit of the information processing device, receiving a question input from a user; extracting a plurality of contents by referring to the data in the graph data structure based on the content of the question received in the receiving step; generating an answer using the content of the question and the plurality of contents extracted in the extracting step; presenting the answer generated in the generating step together with the plurality of contents that served as the basis for generating the answer; In the presenting step, an information processing device presents the plurality of pieces of content that serve as the basis by giving a higher priority to the content that serves as the basis, the greater the number of reference relationships with other content defined in the data of the graph-type data structure.

Images