Programs, methods, and information processing devices.

Abstract

There is a need for technology that makes it easier for organizations to carry out their activities in accordance with internal regulations. [Solution] The program causes the processor to perform the following steps: receive input of a question from the user; identify at least one of the following information related to the content of the question by referring to data in a graph-type data structure based on the content of the received question: information of laws and regulations, case law, book information, or guideline information; extract information of the organization's internal regulations for which a reference relationship is defined with respect to the information identified in the identification step; generate an answer using an information processing system that performs language processing, using at least the information of the organization's internal regulations extracted in the extraction step; and present to the user the answer generated in the generation step, which includes information of the organization's internal regulations that form the basis of the answer.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a program, a method, and an information processing apparatus. BACKGROUND ART

[0002] To operate organizations such as corporations, internal rules and regulations are established. When establishing internal rules and regulations, care must be taken to avoid conflict with regulatory laws and regulations. Internal rules and regulations are revised as appropriate in accordance with changes in laws and social conditions.

[0003] Patent Document 1 describes "a rule document management apparatus capable of easily changing the content of related rule documents from among a large number of rule documents when the content of one rule document is changed".

[0004] Patent Document 1 points out, as a problem, that "when a certain rule document is changed, the task of identifying which related rule documents are involved and what content should be corrected to is performed manually. These manual tasks not only require a large amount of work time, but also frequently cause errors in correction work".

[0005] Patent Document 1 describes the following technical means for solving the problem: "a regulation document management device for managing multiple regulation documents, which stores the links where the contents of the regulation documents are related to each other as static link information"; "when a modification operation is performed on a link in a first regulation document, the modification operation performed on the first regulation document is presented as a recommended operation to the link in the second regulation document corresponding to the link in the first regulation document"; "when editing mode is set, a modification operation is performed on the content of a link in a target regulation document, and as a result of the modification operation on the target regulation document, modification operations on the link in other related regulation documents are performed in a chain reaction based on static link information, the link in the other related regulation document is stored as impact scope information for the link in the target regulation document"; and "when a modification operation is performed on a link in a target regulation document," "the modification operation performed on the link in the target regulation document is presented as a recommended operation to the link in the related regulation document corresponding to the previously stored impact scope information." Patent Document 1 describes "static link information" as a specific example, stating that "static link information La, Lb, and Lc may be similar words between two regulation documents, patterns of notation for official positions, or related regulation names." Patent Document 1 states that, "when editing mode is set, a modification operation is performed on the content of a link in a target regulation document, and this modification operation on the target regulation document triggers a chain reaction of modification operations on link locations in other related regulation documents based on static link information. The system then stores the link locations in the other related regulation documents as impact scope information for the link locations in the target regulation document. The next time a modification operation is performed on a link in the target regulation document, the system presents the modification operation performed on the link locations in the target regulation document as a recommended operation for the link locations in the related regulation documents corresponding to the previous impact scope information stored in the impact scope information storage means. Therefore, when the content of one regulation document is modified, the content of related regulation documents from among a large number of regulation documents can be easily modified." [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2010-262391 [Overview of the project] [Problems that the invention aims to solve]

[0007] While internal regulations should be established to comply with regulatory laws, organizations may also develop their own unique rules. It is important for organizations to promote their business activities and other various activities in accordance with these internal regulations.

[0008] Therefore, there is a need for technologies that make it easier for organizations to carry out their activities in accordance with internal regulations. [Means for solving the problem]

[0009] According to one embodiment shown in this disclosure, a program is provided for operating a computer having one or more computer processors. The memory unit is configured to store at least one of the following: information on books, including books on law; information on guidelines, including legal guidelines; information on laws and regulations; information on precedents; and information on the organization's internal regulations. The memory unit is configured to store data in a graph-type data structure in which reference relationships are defined between each provision of the organization's internal regulations and at least one of the information on each provision of laws and regulations, precedents, books, or guidelines. The program causes one or more computer processors to perform the following steps: receiving a question input from a user; identifying at least one of the following information related to the question: laws and regulations, precedents, books, or guidelines, by referring to data in a graph-type data structure based on the content of the received question; extracting information of the organization's internal regulations for which reference relationships are defined with respect to the information identified in the identification step; having an information processing system perform language processing to generate an answer using at least the information of the organization's internal regulations extracted in the extraction step; and presenting the answer generated in the generation step, which includes the information of the organization's internal regulations that forms the basis of the answer, to the user. [Effects of the Invention]

[0010] This disclosure makes it even easier for organizations to carry out their activities in accordance with their internal regulations. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows the configuration of System 1. [Figure 2] Figure 2 shows the configuration of server 20. [Figure 3] Figure 3 shows the configuration of terminal 10. [Figure 4] Figure 4 shows the data structure of the user database 211. [Figure 5]Figure 5 shows the data structure of the legal content database 212. [Figure 6] Figure 6 shows the data structure of the content usage history database 213. [Figure 7] Figure 7 shows the data structure of graph-type data structure 214. [Figure 8] Figure 8 shows the process flow for creating a graph database using information from multiple legal content sources. [Figure 9] Figure 9 shows the processing flow using a graph database related to laws. [Figure 10] Figure 10 is an example of an operation screen that displays the results of generating answers to user questions while referring to a graph-type data structure 214 related to the law. [Figure 11] Figure 11 shows the process flow for identifying the relevant laws and regulations in response to the user's question and presenting those laws and regulations to the user. [Figure 12] Figure 12 shows an example of an operation screen that presents the user with the relevant laws and regulations identified in response to the user's question. [Figure 13] Figure 13 illustrates the process flow for generating a database that identifies legal issues from information on multiple legally related content. [Figure 14] Figure 14 shows the process flow for searching for content related to legal issues that address a user's question. [Figure 15] Figure 15 shows an example of an operation screen that responds to a user's question by searching various databases based on the corresponding legal issues. [Figure 16] Figure 16 shows the process flow for identifying the relevant laws and regulations in the content that are identified as the basis for the answers to the questions, and outputting the latest laws and regulations. [Figure 17] Figure 17 shows an example screen of a user interface for viewing content, which displays both the content itself and a table of contents summarizing it. [Figure 18] Figure 18 is a screen example of a screen that displays the content of a content and other content related to the displayed content on an operation screen for viewing the content. [Figure 19] Figure 19 is a screen example of a screen that displays information on laws and regulations and legal revisions as other content related to the displayed content on an operation screen for viewing content. [Figure 20] Figure 20 is a diagram showing a flow of processing for presenting an operation screen for viewing content and displaying a part of other content related to the displayed content. [Figure 21] Figure 21 is a diagram showing a flow of processing for generating an answer by determining, as a mode for generating an answer according to the content of a question, whether to use a first mode for generating an answer at high speed or a second mode for generating an answer in detail at low speed. [Figure 22] Figure 22 is a screen example of a screen that analyzes the content of an input question and displays the result of determining whether to answer in the first mode (high-speed mode) or the second mode (detailed answer mode) at a stage before receiving an operation to transmit the question. [Figure 23] Figure 23 is a screen example of a screen that displays the determination result of the answer generation mode on the operation screen during the period from when a question is transmitted until an answer is generated using also an external information processing system. [Figure 24] Figure 24 is a screen example in which the determination result of the answer generation mode is displayed on the screen that displays the answer generated for the question. [Figure 25] Figure 25 is a diagram showing a flow of processing for accepting a follow-up re-question that delves deeper into a question input by a user. [Figure 26] Figure 26 is a screen example of a screen that accepts a follow-up re-question that delves deeper into a question input by a user. [Figure 27] Figure 27 is a diagram showing a flow of processing for presenting an answer to a question, displaying a list of content related to the answer, and changing the display mode of the list according to a user's operation. [Figure 28]Figure 28 is an example of a screen that displays a list of content related to a question while presenting the answer, in a first manner that facilitates understanding the overall picture of the content. [Figure 29] Figure 29 is an example screen of a second mode in which a list of content related to a question is displayed, with the amount of information in a portion of each content increased to assist in selecting content to view in more detail. [Figure 30] Figure 30 shows a graph-type data structure for each type of content: laws, precedents, books, and guidelines. [Figure 31] Figure 31 shows the data structure of the abbreviation dictionary database 215, which defines the rules for abbreviating laws and regulations. [Figure 32] Figure 32 shows the process flow for identifying sections in the content that refer to laws and regulations, depending on the various forms of legal notation, defining reference relationships with the information of the laws and regulations which have a graph-type data structure, and generating a graph-type data structure. [Figure 33] Figure 33 shows the process flow for generating answers to questions by searching for various types of content in response to a question, extracting relevant content by referring to a graph-type data structure for the search results, and using this as the basis for generating an answer to the question. [Figure 34] Figure 34 shows the process flow for generating a graph-type database in which reference relationships are defined between each clause of the company regulations and a graph-type data structure that associates them with laws, precedents, books, and guidelines. [Figure 35] Figure 35 illustrates the process of how each clause of the company's internal regulations is linked to laws and precedents, and how information on legal revisions and changes in precedents is obtained to extract the affected clauses of the internal regulations. [Figure 36] Figure 36 shows the process flow for generating answers to questions based on laws, precedents, books, guidelines, and internal company regulations. [Figure 37]Figure 37 shows the process flow for detecting questions from internal company posts and conversations and generating answers. [Figure 38] Figure 38 shows the process flow for analyzing the reference relationships of a graph-type data structure, determining the priority order for presenting multiple types of content that formed the basis for generating answers to questions, and then presenting that content. [Figure 39] Figure 39 shows the process flow for evaluating the difficulty level of literature based on citation relationships in a graph-type database. [Figure 40] Figure 40 shows the process flow for controlling the difficulty level of literature presented to a user according to the user's browsing trends (difficulty trends), utilizing the evaluation results of literature difficulty levels based on a graph-type database. [Figure 41] Figure 41 shows the processing flow for asking questions, generating answers, and presenting the content that formed the basis for generating those answers, on an author-by-author basis. [Modes for carrying out the invention]

[0012] Embodiments of the present disclosure will be described below with reference to the drawings. In the following description, identical parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions of them will not be repeated.

[0013] <Outline of the Embodiment> In the first embodiment, we will describe a method for generating a data structure that defines reference relationships between legal content based on wording (names of laws, etc.) contained in laws, precedents, legal books, and legal guidelines, and a method for generating answers to user questions using the graph-type data structure generated thereby.

[0014] <1.1 System Configuration Diagram> Figure 1 shows the configuration of System 1.

[0015] System 1, shown in Figure 1, includes a server 20, a user terminal 10, a case law search service server 91, a legal search service server 92, a book browsing service server 93, an information media service server 94, an artificial intelligence (large-scale language model) service server 95 (hereinafter sometimes referred to as "large-scale language model service server 95"), a business operator server 96, an SNS server 97, and a user terminal 10A. These devices communicate with each other via a network 80.

[0016] In the illustrated example, terminals used by users of the service provided by server 20 are shown as terminal 10, terminal 10A, etc., and each user operates their own terminal.

[0017] In this embodiment, each device (terminal device, server, etc.) can also be considered as an information processing device. That is, the collection of each device can be considered as a single "information processing device," and System 1 may be formed as a collection of multiple devices. The way in which the multiple functions required to realize System 1 according to this embodiment are distributed to one or more hardware can be appropriately determined in view of the processing capacity of each hardware and / or the specifications required for System 1.

[0018] Terminal 10 is a device operated by the user. In this embodiment, the user who conducts legal research, makes judgments, etc., operates Terminal 10. Terminal 10 and Terminal 10A have similar functional configurations. Terminal 10 is implemented, for example, as follows. • Handheld mobile devices such as smartphones and tablets • Stationary PC (Personal Computer), Laptop PC • Wearable devices worn by the user (watch-type, glasses-type, etc.) Terminal 10 includes a communication interface (IF) 12, an input device 13, an output device 14, memory 15, storage 16, and a processor 19.

[0019] The communication interface 12 is an interface for inputting and outputting signals so that terminal 10 can communicate with an external device.

[0020] The input device 13 is a device for receiving input operations from the user (for example, a touch panel, touchpad, pointing device such as a mouse, keyboard, etc.).

[0021] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0022] Memory 15 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0023] Storage 16 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0024] The processor 19 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0025] Server 20 is a device for providing a service to users that accepts legal questions and answers them by presenting multiple pieces of relevant legal content. In this embodiment, Server 20 accepts legal questions from the user of Terminal 10 in a chat format via free text or voice input, and outputs answers to the received questions. In generating answers to such questions, Server 20 may also have Server 95, an artificial intelligence (large-scale language model) service, generate the answers, and provides the service by responding to the user with the generated results.

[0026] In this embodiment, the server 20 provides services to the following types of users. • Users who perform legal duties and provide answers to legal consultations, similar to the legal departments of business companies. • Users who do not necessarily have dedicated legal staff, such as business divisions within a company. • Users who provide legal consultations to clients as a service, such as law firms. Server 20 may accept requests for legal consultations from clients to experts by matching users who provide legal consultation services as professionals, such as law firms, with users who request legal consultations from experts such as lawyers, such as businesses or individuals. For example, the content of the consultation may be posted on a bulletin board that is accessible to third parties, and the expert's response may also be made public, or the content of the consultation may be kept confidential and not disclosed to third parties, allowing the consultation to take place privately between the client and the expert. Server 20 may also store such client consultation content and expert response content.

[0027] The server 20 includes a communication interface 22, an input / output interface 23, memory 25, storage 26, and a processor 29.

[0028] Communication IF22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0029] Input / Output IF23 functions as an interface between an input device for receiving user input operations and an output device for presenting information to the user.

[0030] Memory 25 is for temporarily storing programs and data processed by programs, etc., and is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0031] Storage 26 is for storing data, and can be, for example, flash memory or an HDD (Hard Disk Drive).

[0032] The processor 29 is hardware for executing the instruction set described in the program, and consists of an arithmetic unit, registers, peripheral circuits, etc.

[0033] Server 91 of the case law search service has a database of case law, allowing users to search for judgments made by courts and other authorities.

[0034] Server 92 of the legal search service has a database of laws and regulations and allows users to search for those laws and regulations.

[0035] The book browsing service server 93 provides a service that makes electronic content such as magazines and books available for viewing. For example, users can access ebooks in fields such as law by paying a fixed fee on a regular basis.

[0036] Server 94 of the Information Media Services provides information services, such as collecting and making available blog posts, Q&A sites, news articles, and IR information. Server 94 of the Information Media Services also stores information on interpretations based on laws and regulations, as well as guidelines from government agencies and other organizations prepared to disseminate operational rules.

[0037] Server 95 of the Large-Scale Language Model Service is a server that executes language processing tasks using language models built through learning processes including artificial intelligence (AI). An LLM (Large Language Model) is a model that has been pre-trained on large amounts of data (such as text data), for example, a large amount of web content on the internet, or a large amount of data stored in a designated database, and can perform various language processing tasks by being given a task.

[0038] The server 95 of the large-scale language model service accepts prompt input in the form of text, images, audio, etc., and generates and responds with answers to those prompts. Examples of LLMs include GPT-3, GPT-4, and GPT-4o developed by OpenAI, and Gemini developed by Google.

[0039] Server 96, operated by the service provider, stores data generated as a result of the service provider's business activities. Access to this data is restricted, with viewing permissions set for users belonging to the service provider and external users not belonging to the service provider, depending on the type of data.

[0040] SNS Server 97 provides services that facilitate interaction between users, such as a service that allows users to mutually view each other's posts. For example, there are services that can be viewed via internet search even without a user account on the SNS, and services that allow users with a user account to view each other's posts.

[0041] <1.2 Functional Configuration of Server 20> Figure 2 shows the configuration of server 20. As shown in Figure 2, server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0042] The communications unit 201 performs processing to enable the server 20 to communicate with external devices.

[0043] The memory unit 202 stores various databases, including a user database 211, a legal content database 212, a content usage history database 213, a graph-type data structure 214, an abbreviation dictionary database 215, a prompt database 218, and a legal terminology database 219.

[0044] User database 211 is a database that manages information for each user. Further details will be provided later.

[0045] Legal Content Database 212 is a database that holds information on legal content. Further details will be provided later.

[0046] Content Usage History Database 213 is a database of user usage history for legal content. Further details will be provided later.

[0047] The graph-type data structure 214 illustrates a graph-type data structure in which reference relationships are defined for a portion of each content.

[0048] The Abbreviation Dictionary Database 215 is a database that manages the rules for using abbreviations when referring to laws and regulations. Further details will be provided later.

[0049] The prompt database 218 is a database that manages prompt templates to be sent to the server 95 of the large-scale language model service.

[0050] Legal Terminology Database 219 is a database of legal terms. Legal terms include, for example, the names of various legal issues as defined in laws and regulations ("claims," ​​"debts," "damages," "risk allocation," etc.), and terms used in case law.

[0051] The control unit 203 is realized when the processor 29 reads a program stored in the memory unit 202 and executes instructions contained in the program. By operating according to the program, the control unit 203 performs the functions shown as the reception control module 2041, transmission control module 2042, user management module 2043, question processing module 2044, LLM utilization module 2045, data structure definition module 2046, and content presentation module 2047.

[0052] The receive control module 2041 controls the process by which the server 20 receives signals from external devices according to a communication protocol.

[0053] The transmission control module 2042 controls the process by which the server 20 transmits signals to external devices according to a communication protocol.

[0054] The user management module 2043 is a module for managing information for each user using System 1. Specifically, the user management module 2043 accepts registration of each user's information and updates the user database 211.

[0055] The question processing module 2044 is a program module that, in response to a question entered by a user, searches for content managed in the legal content database 212 based on a graph-type data structure 214 and generates an answer to the question.

[0056] The LLM utilization module 2045 is a program module that generates prompts and sends the generated prompts to the server 95 of the large-scale language model service, thereby generating responses to the prompts.

[0057] The data structure definition module 2046 is a program module that defines a graph-type data structure 214 based on the content managed in the legal content database 212.

[0058] The content presentation module 2047 is a program module that provides users with an operation screen for viewing content managed by various devices such as the graph-type data structure 214, the legal content database 212, the case law search service server 91, the legal search service server 92, the book browsing service server 93, and the information media service server 94, and processes the content to be viewed according to the user's operations.

[0059] <1.3 Configuration of Terminal 10> Figure 3 shows the configuration of terminal 10.

[0060] As shown in Figure 3, terminal 10 includes multiple antennas (antenna 111, antenna 112), communication units corresponding to each antenna (first communication unit 120, second communication unit 121), an input device 130 (including a touch-sensitive device 131), a display 132, an audio processing unit 140, a microphone 141, a speaker 142, a position information sensor 150, a camera 160, a motion sensor 170, a storage unit 180, and a control unit 190. Terminal 10 also has functions and configurations not specifically shown in Figure 3 (for example, a battery for maintaining power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.). As shown in Figure 3, each block included in terminal 10 is electrically connected by a bus or the like.

[0061] Antenna 111 radiates signals emitted by terminal 10 as radio waves. Antenna 111 also receives radio waves from space and provides the received signals to first communication unit 120.

[0062] Antenna 112 radiates signals emitted by terminal 10 as radio waves. Antenna 112 also receives radio waves from space and provides the received signals to the second communication unit 121.

[0063] The first communication unit 120 performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 111 in order to communicate with other wireless devices. The second communication unit 121 also performs modulation and demodulation processing, etc., for the terminal 10 to transmit and receive signals via the antenna 112 in order to communicate with other wireless devices. The first communication unit 120 and the second communication unit 121 are a communication module that includes a tuner, an RSSI (Received Signal Strength Indicator) calculation circuit, a CRC (Cyclic Redundancy Check) calculation circuit, a high-frequency circuit, etc. The first communication unit 120 and the second communication unit 121 perform modulation and demodulation, frequency conversion, etc., of the wireless signals transmitted and received by the terminal 10, and provide the received signal to the control unit 190.

[0064] The input device 130 has a mechanism for receiving user input operations. Specifically, the input device 130 is configured as a touchscreen and includes a touch-sensitive device 131. The touch-sensitive device 131 receives user input operations of the terminal 10. The touch-sensitive device 131 detects the user's contact position with the touch panel, for example, by using a capacitive touch panel. The touch-sensitive device 131 outputs a signal indicating the user's contact position detected by the touch panel to the control unit 190 as an input operation.

[0065] The display 132 displays data such as images, videos, and text in accordance with the control of the control unit 190. The display 132 is implemented by, for example, an LCD or an organic EL display.

[0066] The audio processing unit 140 modulates and demodulates the audio signal. The audio processing unit 140 modulates the signal received from the microphone 141 and provides the modulated signal to the control unit 190. The audio processing unit 140 also provides the audio signal to the speaker 142. The audio processing unit 140 is implemented, for example, by an audio processing processor. The microphone 141 receives an audio input and provides the audio signal corresponding to that audio input to the audio processing unit 140. The speaker 142 converts the audio signal received from the audio processing unit 140 into sound and outputs the sound to the outside of the terminal 10.

[0067] The location information sensor 150 is a sensor that detects the location of the terminal 10, and is, for example, a GPS (Global Positioning System) module. A GPS module is a receiving device used in a satellite positioning system. In a satellite positioning system, signals are received from at least three or four satellites, and the current location of the terminal 10, which is equipped with a GPS module, is detected based on the received signals.

[0068] Camera 160 is a device that receives light using a photodetector and outputs it as an image. Camera 160 is, for example, a depth camera that can detect the distance from camera 160 to the object being photographed.

[0069] The motion sensor 170 includes an acceleration sensor, an angular velocity sensor, etc., and detects the movement of the terminal 10.

[0070] The storage unit 180 is composed of, for example, flash memory and stores data and programs used by the terminal 10. The various types of information stored in the storage unit 180 will be described later.

[0071] The control unit 190 controls the operation of the terminal 10 by reading the program stored in the memory unit 180 and executing the instructions contained in the program. The control unit 190 is, for example, an application processor. By operating according to the program, the control unit 190 performs the functions of an operation reception unit 191, a transmission / reception unit 192, a data processing unit 193, a notification control unit 194, and a memory control unit 195.

[0072] The operation reception unit 191 processes input operations from the user to an input device such as a touch-sensitive device 131. Based on the coordinate information of the touch-sensitive device 131 where the user's finger or the like has made contact, the operation reception unit 191 determines the type of operation, such as whether the user's operation is a flick operation, a tap operation, or a drag (swipe) operation.

[0073] The transmitting / receiving unit 192 performs processing to enable the terminal 10 to send and receive data with an external device such as a server 20 in accordance with a communication protocol.

[0074] The data processing unit 193 performs calculations on the data received as input by the terminal 10 according to the program and outputs the calculation results to memory or other locations.

[0075] The notification control unit 194 performs the following processes: displaying the display image on the display 132, outputting sound to the speaker 142, and generating vibrations.

[0076] The memory control unit 195 controls the storage of data to the memory unit 180.

[0077] The various types of information stored by the memory unit 180 will now be explained. In a given situation, the memory unit 180 stores various types of information, such as user information 181.

[0078] User information 181 is information about a user who uses the services of server 20.

[0079] <2 Data Structure> Figure 4 shows the data structure of the user database 211. The user database 211 includes the following fields: "User ID", "Name", "Email Address", "Business ID", "Department", "Position", "Date of Employment", "Date of Resignation", and "Qualifications Held".

[0080] The "User ID" field is information that identifies each user.

[0081] The "Name" field contains information indicating the user's name.

[0082] The "Email Address" field contains the user's email address information for contact purposes.

[0083] Specifically, the item "email address" contains email address information, which serves as user identification information for accepting user logins to services provided by server 20.

[0084] The "Business ID" field is information that identifies the organization to which the user belongs.

[0085] Specifically, the "Business ID" field is information that identifies the organization to which the user belongs, and the following types of organizations are possible: • Business company ·Law firm The "Department" field contains information about the department to which the user belongs.

[0086] Specifically, the item "Department" may include the following information as the department to which the user belongs: • Departments such as the legal department that are expected to perform legal duties. • Departments such as business divisions and sales divisions that do not plan to have someone dedicated to legal affairs. The "Job Title" field contains information about the user's job title.

[0087] Specifically, the "Job Title" field may include the following information regarding the user's job title: • Positions with decision-making authority (e.g., management) • A position that does not have decision-making authority but demonstrates expertise and a role in supporting the management department. • Not holding a managerial position. The "Date of Joining" field contains information about the date the user became a member of the organization.

[0088] Specifically, the item "Date of Joining the Company" may include the following information: • Start date if you are already a member • A date when they are scheduled to join the organization but are not yet affiliated (They may have been hired but not yet have access to the organization's information because their start date has not yet arrived). • While affiliated with the company, the start date is not yet confirmed (the start date may still be being finalized). The "Resignation Date" field contains information about the date on which the user left the organization and resigned.

[0089] Specifically, the item "Retirement Date" may include the following information: • Date of resignation (Upon resignation, access to organizational information may be lost) • Currently employed The "Qualifications Held" item contains information indicating the qualifications held by the user.

[0090] Specifically, the item "Qualifications Held" may include the following qualifications, which serve as information to identify the qualifications held by the user. • National qualifications that demonstrate expertise, such as lawyer, patent attorney, and certified public accountant. • Qualifications recognized by businesses, general incorporated associations, and other organizations as proof of expertise. Figure 5 shows the data structure of the legal content database 212.

[0091] The "Content ID" field is information that identifies each piece of legally relevant content.

[0092] The "Type" field contains information about the type of legal content.

[0093] The "Type" item may include the following as types of legal content: ·Laws Case law • Books on law • Websites containing legal information The item "Source" refers to the information about the source from which the content is provided.

[0094] The item "Sources" may include the following as sources: • Server 92 for the legal search service • Server 91 of the case law search service • Server 93 of the book browsing service that provides access to legal books. • Information media service server 94, which contains legal information. The "Title" field contains information about the name of the content.

[0095] The "Title" field may include the following as the name of the content: In the case of laws and regulations, the name of the law, the article number, the headings attached to the articles, etc. • In the case of precedents, the case number, the name of the case, etc. • In the case of books on law, the title of the book, etc. • In the case of legal guidelines, the name of the guideline, etc. The item "Content Tags" contains information about the names of the tags assigned to the content.

[0096] The item "Content Tags" may include the following as information about tags assigned to the content: • The large-scale language modeling service server 95 extracts tags representing the content by summarizing data indicating the content using keywords, a certain number of characters, etc. • Server 20 maintains a list of tags to be assigned to the entire content in advance, and based on the content, the server 95 of the large-scale language model service identifies tags from the tags included in the list. The item "Data" contains information about the content's data file.

[0097] The item "Tagged Location" refers to information about the location where tags are set, using the names of laws, precedents, legal books, legal guidelines, etc., as tags, and where these tags are set at any point in the data that makes up the content.

[0098] The item "Tagging Location" may include the following as locations where tags are set: In the case of laws and regulations, tags may be set in association with specific clause numbers, etc. In the case of case law, the judgment can be divided into multiple blocks based on the paragraphs, headings, arguments of both parties, and the sections where the court's judgments on each issue are described, and tags can be set at any point in the text within those blocks. • In the case of legal books, the book may be divided into multiple blocks based on chapters, headings, paragraphs, etc., and tags may be set at any point in the text within those blocks. • In the case of legal guidelines, the guideline text may be divided into multiple blocks in the same manner as above, and tags may be set at any point in the text within those blocks. For example, if a book on law contains names such as the names of laws and regulations, the names of articles, or the numbers of precedents, server 20 may set tags on the parts where such names are written.

[0099] The "Name Tag" field contains information about the name of the tag that has been set.

[0100] The item "Name Tag" may include the following as tags attached to the name: • Name of the law, article number, etc. • Name of the case number, etc. of the precedent • Titles of legal books • Name of the legal guidelines The "Name Tag Score" item contains information about the evaluation results for the tags that indicate the name.

[0101] More specifically, the "Name Tag Score" item is an evaluation of tags based on the history of content viewing in various services provided by the legal search service server 92, the case law search service server 91, the book browsing service server 93, the information media service server 94, etc., where tags and content are associated.

[0102] Server 20 may use the results of the evaluation of name tags in this manner to determine the priority of tags corresponding to search terms when a user performs a search (for example, the higher the evaluation of a tag, the higher the priority given to it as a tag corresponding to the user's search), and search for legal content based on the tags with the highest priority.

[0103] The "Name Tag Associated Content ID" field is information that identifies the content based on the location where the tag is set, when the same tag is set for multiple pieces of content.

[0104] In the illustrated example, the item "Name Tag Associated Content ID" shows the result of defining reference relationships between content containing the name of a specific law, where each piece of content related to multiple laws contains the name of that specific law.

[0105] The item "Location of Issue Tagging" refers to information about the location where legal issues are tagged and set as tags in arbitrary parts of the data that make up the content.

[0106] The item "Issue Tag" contains information about the names of the tags assigned to the issues.

[0107] The item "Issue Tag Score" contains information about the evaluation results for tags that indicate issues.

[0108] More specifically, the "Issue Tag Score" item is determined by classifying legal books by issue on server 93 of the book browsing service, etc., accepting operations to search for books to be viewed based on issues, and evaluating the issue tags based on the user's browsing history of legal books in response to the search results.

[0109] The item "Issue Tag Associated Content ID" is information that identifies the content that defines the reference relationship based on where the issue tag is set, when the same issue tag is set in multiple pieces of content, as set in the item "Issue Tag".

[0110] Figure 6 shows the data structure of the content usage history database 213.

[0111] The "Search ID" field is information that identifies each search performed to access legal content.

[0112] The "Search User ID" field identifies the user who searched for legal content.

[0113] The item "Search User ID" may be associated with the item "User ID" in user database 211.

[0114] The "Search Terms" field contains information about the search terms used to find the content.

[0115] More specifically, the "search term" field is a word specified by the user, or a sentence entered by the user in natural language as a question.

[0116] The item "Search Issue Tag" contains information about the names of the issue tags assigned to the search terms entered by the user.

[0117] More specifically, the item "Search Issue Tag" is either a tag for an issue extracted by matching the search term entered by the user with a list of issue tags, or information on a legal issue set by the server 95 of the large-scale language model service, etc.

[0118] The "Search Date and Time" field contains information about the timing of the search.

[0119] The "Source" item is database information that is referenced when a user performs a search for content.

[0120] The item "Information Source" may include the following as databases referenced by the search: • Server 92 of the legal search service • Server 91 of the case law search service • Server 93 for the book browsing service • Information Media Services Server 94 The "Viewed Content ID" field is information that identifies the content viewed by the user.

[0121] The item "Viewed Content ID" may be associated with the item "Content ID" in the legal content database 212.

[0122] The item "Content Viewed Location" contains information about the specific locations where the user viewed content.

[0123] More specifically, the item "Content Viewed" refers to information about the sections (pages; it may also be defined as pages viewed for a certain period of time or longer) within a book related to law.

[0124] Figure 7 shows the data structure of graph-type data structure 214.

[0125] In the example shown in Figure 7, it is indicated that information for a set of text 214A, which includes the relevant section of a specific law contained in document A, is stored in the legal content database 212, associated with document A (e.g., a legal book). Similarly, it is indicated that information for a set of text 214B, which includes the relevant section of a specific law contained in document B, is stored in the legal content database 212, associated with document B (e.g., guidelines), as the name of the set of text 214B.

[0126] In this example, a reference relationship (edge) is defined between parts of document A and document B that contain the same legal provision (document 214A, document 214B). In this way, the data structure definition module 2046 generates a graph-type data structure 214 by defining nodes for the parts of the legal content database 212 that contain the names of laws, precedents, legal books, guidelines, etc., and defining nodes and edges based on these names. The example shown illustrates the definition of a reference relationship between two documents, A and B, but it is not limited to just two.

[0127] <3. Operation (First Embodiment)> Figure 8 shows the process flow for creating a graph database using information from multiple legal content sources.

[0128] In step S821, the data structure definition module 2046 of the server 20 obtains information on legal content and information that can be used as tagging candidates by referring to the legal content database 212, the legal terminology database 219, etc.

[0129] In this way, the server 20 stores information on multiple legal content in the legal content database 212 of the storage unit 202.

[0130] In step S823, the data structure definition module 2046 of server 20 identifies a portion of the legal content that includes information identifying laws and regulations, information identifying precedents, information identifying legal books, and information identifying legal guidelines.

[0131] Thus, in each of the multiple legal content items, the data structure definition module 2046 is defined as follows: Information that identifies laws and regulations, Information that identifies a case, Information identifying legal books, Information identifying legal guidelines, Identify a portion that includes at least one of the following.

[0132] More specifically, the data structure definition module 2046 is: As information identifying the law, at least one of the law's name or provision, Information that identifies a case includes the case number, Information that identifies a legal book includes the title of the legal book, Information that identifies legal guidelines, such as the name of the guideline, It may also be necessary to identify a portion that includes at least one of the following.

[0133] More specifically, data structure definition module 2046 is: As information identifying the law, at least one of the law's name or provision, Information that identifies a case includes the case number, Information that identifies a legal book includes the title of the legal book, Information that identifies legal guidelines, such as the name of the guideline, It is also possible to identify a certain amount of text as a part that contains at least one of the following.

[0134] In step S825, the data structure definition module 2046 of the server 20 associates the identified portion of the content with the content itself and stores it in the legal content database 212.

[0135] In this way, the data structure definition module 2046 stores the portion identified in step S823 in the storage unit 202 in association with the content related to that identification.

[0136] Here, the data structure definition module 2046 may evaluate a portion of the content based on the portion associated with the content, according to the usage of the search results in a service that accepts operations to search for content (for example, the server 91 for the case law search service, the server 92 for the legal search service, the server 93 for the book browsing service, the server 94 for the information media service, etc.). For example, regarding tags associated with content, the tags may be evaluated according to the search history of those tags in the book browsing service mentioned above (e.g., the evaluation value is increased for frequently searched tags and for tags that lead to books being viewed through searches). The data structure definition module 2046 may store the evaluation results of the evaluated portion in the legal content database 212 in association with the content (for example, the item "Name Tag Score"). For example, when the server 20 identifies a tag corresponding to a user's question in a graph-structured database, it may select a tag according to its evaluation value (e.g., prioritizing tags with high evaluation values ​​as tags corresponding to the user's question), and then search the graph-structured database using the selected tag. This improves the response time of tag-based searches using a graph structure compared to searching each piece of content in the legal content database 212 using keywords, while further enhancing the accuracy of search results (presenting legal content suitable for the user).

[0137] By processing steps S823 and S825 described above, it is possible to identify the locations where the names of laws, precedents, books, guidelines, etc., are mentioned, and to assign tags based on these names to the content.

[0138] In step S827, the data structure definition module 2046 of the server 20 extracts identical portions of multiple content items. The data structure definition module 2046 defines reference relationships for the portions of each content item that have been extracted as containing identical items.

[0139] More specifically, the data structure definition module 2046 may define reference relationships by extracting content that has a part of the same type, and by extracting content with the same name relating to the identified part.

[0140] For example, if the same case law number is described in both the first and second ebooks, the section in the first ebook where the case law number is described and the section in the second ebook where the case law number is described are extracted as being of the same type. The data structure definition module 2046 defines reference relationships between the extracted sections in the first and second ebooks where the above-mentioned case law number is described.

[0141] In this way, the data structure definition module 2046 constructs a graph-structured database by extracting content with similar identified parts from multiple content items and defining reference relationships between the content items for each extracted content item. For example, the data structure definition module 2046 may store the reference relationships defined for multiple content items in the legal content database 212.

[0142] Here, the data structure definition module 2046 may define reference relationships between portions of text, which are fixed amounts of text, from multiple contents.

[0143] This allows for the generation of a graph-type data structure that defines reference relationships between multiple pieces of content when those pieces of content contain the same names of laws, precedents, books, and guidelines. For example, if a graph-type data structure is generated based on the name of a law, searching for that law's name can display search results from multiple sources, such as the original text of the law, corresponding precedents, books that mention the law, and guidelines that mention the law. Because the search is performed using a graph-type data structure, the responsiveness of the search can be further improved compared to sequentially searching these sources using the keywords mentioned above.

[0144] In step S829, the data structure definition module 2046 of the server 20 outputs information about the reference relationships defined for multiple contents to the terminal 10. Alternatively, the data structure definition module 2046 may choose not to output the information to the terminal 10 after defining the reference relationships between the contents, or it may present the information about the reference relationships defined between the contents upon request from the terminal 10.

[0145] In step S811, terminal 10 displays information indicating the content reference relationships.

[0146] Figure 9 shows the processing flow using a graph database related to laws.

[0147] In step S921, the question processing module 2044 of the server 20 receives input of a legal question and outputs an operation screen to the terminal 10 that displays the answer.

[0148] As described above, Server 20 holds a data structure for information on legal content (legal content database 212, graph-type data structure 214). Server 20 stores information on multiple legal content items in its storage unit 202. In the storage unit 202, Server 20 stores the results of defining reference relationships between content items for each of the multiple content items, where at least one of the following is included: information that identifies laws and regulations, information that identifies precedents, information that identifies legal books, and information that identifies legal guidelines.

[0149] In step S911, terminal 10 receives input from the user regarding legal questions.

[0150] In step S923, the question processing module 2044 of server 20 assigns a tag representing the content of the question to the question entered by the user of terminal 10, with the help of the server 95 of the large-scale language model service. For example, the question processing module 2044 generates a prompt that includes, along with the content of the question, at least one of the following: an instruction to summarize the content of the question entered by the user, an instruction to refer to legal terms such as the legal terminology database 219 regarding the content of the question and assign a tag to it. The question processing module 2044 sends the prompt to the server 95 of the large-scale language model service via the LLM utilization module 2045, and assigns a tag to the question by receiving an output result corresponding to the prompt from the server 95 of the large-scale language model service.

[0151] In step S925, the question processing module 2044 of the server 20 uses the tags assigned to the question to refer to the graph-type data structure 214 and search for tagged content. This reduces the processing required to search for tagged content compared to searching a database of table-type content, and further improves the response speed of search results.

[0152] In step S927, the question processing module 2044 of server 20 obtains the generated answer by instructing the server 95 of the large-scale language model service to generate an answer to the question based on the search results. Specifically, the question processing module 2044 generates a prompt that includes the search results (legal content searched based on tags assigned to the question) retrieved by referring to the graph-type data structure 214 and the question entered by the user in steps S911 and S923, and sends the generated prompt to the server 95 of the large-scale language model service, thereby receiving the answer generated by the server 95 of the large-scale language model service.

[0153] Furthermore, the question processing module 2044 may, in response to a legal question input from the user, output the search results of the content retrieved in step S925 to the user's terminal 10 without further generating an answer to the question by the server 95 of the large-scale language model service. This allows the system to respond to the user's question with legal content retrieved using the graph-type data structure 214, reducing the processing required for the search and enabling a faster response of search results compared to sequentially searching each data in a table-type content database.

[0154] Thus, the server 20 has a data structure 214 for information on legal content, which is used in the process of responding to a search request to retrieve information on multiple legal content items. This structure identifies a portion (tag) in the database that corresponds to the search request and responds to the search request based on the reference relationships between the content items for the identified portion.

[0155] Here, as described above, the data related to the data structure is used in a process where, in response to a search request that retrieves a portion of multiple contents that corresponds to the question, the server 95 of the large-scale language model service receives a question input from the user and generates an answer to the question. The server 95 retrieves a portion of multiple contents that corresponds to the question and responds with search results for multiple contents based on the reference relationships between the contents for that portion. The large-scale language model generates an answer to the question based on the search results for multiple contents and the question from the user.

[0156] In step S929, the server 20 outputs an answer to the question to the terminal 10 based on the tag corresponding to the question.

[0157] In step S913, terminal 10 displays the answer to the question.

[0158] <4. Screen Example (First Embodiment)> Figure 10 is an example of an operation screen that displays the results of generating answers to user questions while referring to a graph-type data structure 214 related to the law.

[0159] The user interface 1000 displays the results of a search for laws, precedents, legal books, guidelines, etc., in response to a question, while referring to the graph-type data structure 214.

[0160] The operation screen 1000 corresponds to each process, such as step S921 in Figure 9, step S1121 in Figure 11, and step S1321 in Figure 13.

[0161] The question designation unit 1002 is an operating component that receives the user's designation of the question content.

[0162] More specifically, the question specification unit 1002 accepts questions from the user in natural language.

[0163] The transmission operation unit 1004 is an operation component that receives an operation to generate an answer to a question entered by the user.

[0164] In the illustrated example, the transmission operation unit 1004 transmits the question entered in the question specification unit 1002 to the server 20 in response to the user's operation.

[0165] The account display area 1006 is an area that displays account information for users who use the services provided by server 20.

[0166] In the illustrated example, the account display area 1006 also displays the user's affiliation, but it may also display the user's account pricing plan to make it easier to recognize the differences in available features depending on the pricing plan (for example, limitations on the number of questions that can be entered, and limitations on the scope of laws, precedents, legal books, guidelines, etc. that can be referenced).

[0167] The answer display area 1008 is an area that displays the answer generated by the server 20 in response to the question entered by the user.

[0168] The question content display area 1010 is an area that displays the content of the question entered by the user in the question specification unit 1002.

[0169] In the illustrated example, the question content display area 1010 displays the question for which the server 20 has generated an answer, making it easy to recognize which question the answer is for. For example, if an answer is generated each time a question is entered, even when scrolling back through past questions, it is easy to confirm which question the answer is for.

[0170] The question tag display area 1012 is an area that displays the tags assigned to the entered question.

[0171] The question tag display area 1012 corresponds to the processing shown in step S923 of Figure 9.

[0172] The legal information display area 1014 is an area that displays the legal information retrieved by referring to the graph-type data structure 214 in response to the entered question.

[0173] In the illustrated example, the legal information display area 1014 displays the name of the law and the article number, and also includes a "View Details" button. In response to the user's operation of the "View Details" button, terminal 10 may display the details of the law (for example, the original text of the article) within the operation screen 1000 (for example, by expanding the area of ​​the legal information display area 1014), or it may open a separate window to allow the user to view the details of the law.

[0174] The case law display area 1016 is an area that displays case law retrieved by referring to the graph-type data structure 214 in response to the entered question.

[0175] In the illustrated example, the case law display area 1016 displays the case law number and includes a "View Details" button. Terminal 10 may also display the details of the searched case law (such as an overview of the case, as described later) on the operation screen 1000 or elsewhere, in response to the user's operation of the "View Details" button.

[0176] The legal books display area 1018 is an area that displays legal books found by referring to the graph-type data structure 214 in response to the entered question.

[0177] In the illustrated example, the legal book display area 1018 presents the user with details of the searched legal book (for example, after receiving a user action such as "View Details" shown in the legal information display area 1014, etc.). The legal book display area 1018 displays an overview of the book, such as the title, publisher, and author, while the book details display area 1020, etc., described later, displays an excerpt from the book's text.

[0178] The book details display area 1020 is an area that displays a portion of a legal book extracted according to the searched tags.

[0179] In the illustrated example, the book details display area 1020 displays a portion of the information displayed in the evidence display area 1022, which will be described later, as well as the surrounding text. For example, in a legal book, the text may be divided into multiple blocks by chapters, paragraphs, etc., and the blocks containing the text indicated in the "Tagged Locations" item of the legal content database 212 may be extracted.

[0180] The evidence display area 1022 is an area that displays the location in legal textbooks where the corresponding information is found.

[0181] In the illustrated example, the evidence display area 1022 highlights the text shown in the "Tagged Location" item of the legal content database 212 (for example, by highlighting it with a marker or making it bold), making it easy to identify the location of the text corresponding to the tag.

[0182] The guideline display area 1024 is an area that displays the guidelines found by referring to the graph-type data structure 214 in response to the entered question.

[0183] In the illustrated example, the guideline display area 1024 displays the name of the guideline and includes a "View Details" button. Terminal 10 may also display the details of the case within the operation screen 1000 or elsewhere to allow the user to view the details of the searched guideline in response to the user's operation of the "View Details" button.

[0184] <5 Operation (Second Embodiment)> Next, a second embodiment will be described.

[0185] When a legal consultation is received, it can sometimes be difficult to identify the relevant law from the consultation content alone. This is because legal provisions are often defined using abstract language, and even if search terms are defined based on the consultation content, the results may not correspond to the wording of the legal provision. Therefore, the usual procedure involves searching for cases that correspond to the consultation content and then identifying the relevant law and provision from those cases. For the person providing the consultation, especially in an unfamiliar field, this investigation procedure can be time-consuming.

[0186] Therefore, in the second embodiment, we will describe a technology that accepts natural language input and presents relevant laws and regulations.

[0187] Specifically, in the second embodiment, Accepting questions from users in natural language, The server 95 of the large-scale language model service will be used to identify which law the question pertains to (for example, a specific law such as the Companies Act). Legal books and web articles related to law are pre-tagged (the tagging of such legal books may be performed by the server 95 of the large-scale language model service), and the content of legal books, etc., is searched for those whose content matches the tags (searching by keyword, vectorizing the question text and the content of legal books, etc., and then searching for vectors of law-related content based on the vectors), To obtain laws and regulations cited in the text of searched content, To do so.

[0188] Figure 11 shows the process flow for identifying the relevant laws and regulations in response to the user's question and presenting those laws and regulations to the user.

[0189] In step S1121, the question processing module 2044 of the server 20 receives input of a legal question and outputs an operation screen to the terminal 10 that displays the answer.

[0190] Server 20 stores information on multiple legal content in its storage unit 202, including at least one of the following: legal books, legal guidelines, and articles from legal information media. Storage unit 202 stores information on multiple legal books, and each of these legal books is associated with a tag indicating its content.

[0191] For example, server 20 may set tags indicating the content of legal books by referring to the classification set for each book in server 93 of the book browsing service (classifications entered by a human, such as the service operator or the book publisher), or server 95 of the large-scale language modeling service may assign tags indicating the content of legal books (for example, by sending a prompt to server 95 of the large-scale language modeling service to summarize the content of legal books).

[0192] In step S1111, terminal 10 receives input from the user regarding legal questions.

[0193] In step S1123, the question processing module 2044 of server 20 assigns a tag corresponding to the question entered by the user, using the server 95 of the large-scale language model service.

[0194] The question processing module 2044 assigns tags to questions by providing a prompt to the server 95 of the large-scale language model service that includes instructions to assign legal tags to the questions. The module then obtains information on the tag assignment results output by the server 95 of the large-scale language model service, and assigns tags to the questions based on the obtained information on the tag assignment results.

[0195] Alternatively, the question processing module 2044 may assign tags to questions by inputting the questions entered by the user in step S1111 into a trained model that has been trained to output tags to questions based on the results of tagging questions.

[0196] In addition, the question processing module 2044 also maintains a list of tags to be assigned to questions based on keywords contained in the question text, and may assign tags to the question by comparing the question entered in step S1111 with this list.

[0197] In step S1125, the question processing module 2044 of the server 20 uses the tags assigned to the question to refer to the database and search for the tagged content.

[0198] In this way, the question processing module 2044 searches for information from multiple legal content sources based on the entered question. Here, the question processing module 2044 searches for information from at least one of the following: legal books, legal guidelines, or articles from legal information media. The question processing module 2044 may also search for legal books based on tags assigned to the question and tags associated with legal books.

[0199] Furthermore, in a service that allows users to view multiple legal books (the server 93 for the book viewing service), the scores of tags associated with legal books are determined (legal content database 212, content usage history database 213). The question processing module 2044 may search for legal books based on the scores of the tags associated with them. For example, when searching for tags in the database that correspond to the tags assigned to the question, tags with higher scores may be given priority in the search results.

[0200] As described above, the server 20 stores information on multiple legal content items in the storage unit 202. In the storage unit 202, for each of the multiple content items, the results of defining reference relationships between content items that are of the same type, including at least one of the following: information that identifies laws and regulations, information that identifies precedents, information that identifies legal books, and information that identifies legal guidelines, are stored as a legal data structure (legal content database 212, graph-type data structure 214). The question processing module 2044 retrieves information on multiple legal content items related to a question by referring to the legal data structure.

[0201] In step S1127, the question processing module 2044 of the server 20 identifies information about laws and regulations cited in the search results content. For example, the question processing module 2044 may search for content based on the tags assigned to the question and information associated with legal books held in the legal content database 212 (such as the "Title" and "Content Tag" items in the legal content database 212), and identify information about laws and regulations included in the searched content (for example, the "Tagged Location" and "Name Tag" items in the legal content database 212).

[0202] In step S1129, the question processing module 2044 of the server 20 presents the identified legal information to the user of the terminal 10.

[0203] In step S1113, terminal 10 displays information on laws and regulations corresponding to the question.

[0204] <6. Screen Example (Second Embodiment)> Figure 12 shows an example of an operation screen that presents the user with the relevant laws and regulations identified in response to the user's question.

[0205] The legal details display area 1026 is an area that displays a portion of the legal information extracted according to the searched tags for the searched legal information.

[0206] In the illustrated example, the legal details display area 1026 displays the legal provisions corresponding to the searched tag. For example, the server 20 may display only the provisions corresponding to the question, rather than all of the provisions contained in the legal text.

[0207] Case details display area 1028 is an area that displays summaries of multiple case precedents extracted according to the searched tags.

[0208] In the illustrated example, the case details display area 1028 displays the results of generating summaries of multiple case precedents, which were retrieved by referring to the graph-type data structure 214 according to the tags corresponding to the question, using the server 95 of the large-scale language model service (for example, the results of generating summaries of case precedents by prompting the generation of an overview of the case and an overview of the conclusion within a predetermined number of characters).

[0209] <7 Operation (Third Embodiment)> Next, a third embodiment will be described.

[0210] Typically, when conducting legal research, searching for legal content using keywords will result in a list of relevant articles. However, especially in areas unfamiliar to the researcher, it can be difficult to determine the importance of the listed articles (e.g., legal books), requiring the researcher to meticulously read through each individual article, which is a significant burden.

[0211] Therefore, in the third embodiment, the system accepts a question in natural language and presents the user with issues related to the entered question. For each issue, the system presents the user with organized content such as legal books.

[0212] Specifically, in the third embodiment, To construct a graph-type data structure for the issues (for example, the issues of each piece of legal content may be identified by the server 95 of the large-scale language model service. For each issue, legal content such as laws, precedents, legal books, and guidelines are associated and constructed as a graph-type data structure), The system accepts questions from users in natural language. Based on a database constructed to hold information on legal content based on the issue at hand, the system identifies the issue corresponding to the question (for example, identifying the issue from keywords contained in the question, or vectorizing the question and searching for the issue based on the vector of legal content). Based on the issues corresponding to the questions and a graph-type data structure for those issues, the response will be to provide relevant laws, precedents, legal books, guidelines, etc. To do so.

[0213] Figure 13 illustrates the process flow for generating a database that identifies legal issues from information on multiple legally related content.

[0214] In step S1321, the data structure definition module 2046 of the server 20 obtains information on legal content by referring to the legal content database 212, etc.

[0215] Thus, the server 20 stores information on multiple legal content in its storage unit 202. Specifically, the server 20 stores information on multiple legal content in its storage unit 202, including at least one of the following: information on laws and regulations, information on precedents, information on legal books, and information on legal guidelines.

[0216] In step S1323, the data structure definition module 2046 of the server 20 generates a prompt that includes instructions to identify legal issues corresponding to the content of the legally relevant content. The data structure definition module 2046 may also generate a prompt that includes instructions to extract the portion of the content that formed the basis for identifying the legal issues corresponding to the content.

[0217] In step S1325, the data structure definition module 2046 of server 20 provides the generated prompt to the server 95 of the large-scale language model service, thereby obtaining the results of identifying legal issues regarding the content from the server 95 of the large-scale language model service.

[0218] In this way, the data structure definition module 2046 provides the content for prompt generation and the prompt generated for that content to the server 95 of the large-scale language model service, and by receiving the output results from the server 95 of the large-scale language model service, it obtains the results of identifying the legal issues for each piece of content.

[0219] In step S1327, the data structure definition module 2046 of the server 20 updates the legal content database 212 by associating the content with the identified legal issues.

[0220] Here, the data structure definition module 2046 may associate the extracted portion of the content with the identified legal issues and store them in the legal content database 212.

[0221] Furthermore, based on the legal issues associated with the content, the system may evaluate the legal issues in accordance with the usage of search results in a service that accepts searches for content (for example, a book browsing service server 93), and store the evaluation results of the legal issues in association with the content in a legal content database 212 (item "issue tag score").

[0222] In step S1329, the data structure definition module 2046 of the server 20 identifies a portion of the content relating to the same issue for multiple content items, defines a reference relationship for the identified portion, and updates the database. The data structure definition module 2046 outputs the information of the reference relationships defined for the issues of the multiple content items to the terminal 10.

[0223] Thus, the data structure definition module 2046 may define a graph-type data structure 214 by identifying parts of multiple content related to the same issue and defining reference relationships between the identified parts of the multiple content.

[0224] Furthermore, it is not necessary to update the database defining the reference relationships and then output the update to terminal 10; instead, the system may respond to a request from terminal 10 by indicating the status of the defined reference relationships in the database.

[0225] In step S1311, terminal 10 displays information indicating the reference relationships of content related to the issue.

[0226] Figure 14 shows the process flow for searching for content related to legal issues that address a user's question.

[0227] In step S1421, the question processing module 2044 of the server 20 receives input of a legal question and outputs an operation screen to the terminal 10 that displays the answer.

[0228] Server 20 stores information on multiple legal content items in its legal content database 212 and graph-type data structure 214 in its storage unit 202, and stores information on legal issues associated with each of the multiple content items.

[0229] The graph-type data structure 214 is used to receive legal questions from users, analyze the input questions to identify the legal issues corresponding to the questions, and, based on the identified legal issues, search for legal content related to those issues by referring to the information on legal issues stored in the graph-type data structure 214.

[0230] In the graph-type data structure 214, each of the multiple content items is configured to store information about the legal issue associated with the content and a portion of the content corresponding to that legal issue. In the graph-type data structure 214, reference relationships are defined between multiple content portions relating to the same issue. The graph-type data structure 214 is used in processes that search for multiple content items for which reference relationships are defined regarding a specified issue, based on those reference relationships.

[0231] In step S1411, terminal 10 receives input from the user regarding legal questions.

[0232] In step S1423, the question processing module 2044 of the server 20 identifies the legal issues corresponding to the input question by analyzing the input question. More specifically, the question processing module 2044 generates a prompt for the input question that includes instructions to identify the legal issues related to the question (for example, the prompt may include instructions to refer to a list of issues), and provides the generated prompt to the server 95 of the large-scale language model service. The server 95 then receives the output result of the large-scale language model service and identifies the legal issues related to the question.

[0233] In step S1425, the question processing module 2044 of the server 20 uses the issue identified for the question to refer to the legal content database 212 and the graph-type data structure 214 to search for content tagged with that issue.

[0234] As described above, the question processing module 2044 searches for information on content related to laws associated with the relevant issue by referring to the legal issue information stored in the storage unit 202, based on the issue corresponding to the identified question.

[0235] In step S1427, the question processing module 2044 of the server 20 outputs the content search results using the issue tags to the terminal 10, including the portions corresponding to the issues for these contents (the item "issue tagging location" in the legal content database 212).

[0236] Here, the question processing module 2044 may provide a prompt including the content search results using the issue tags and the user's question accepted in steps S1411 and S1423 to the server 95 of the large language model service, so as to cause the server 95 of the large language model service to generate an answer to the user's question with reference to the content search results using the issue tags, and output the generated answer to the terminal 10.

[0237] In step S1413, the terminal 10 displays the search results of the legal content together with the issues corresponding to the question.

[0238] <8 Screen Example (Third Embodiment)> Figure 15 is a screen example of an operation screen that responds to a user's question with the results of searching various databases based on the corresponding legal issues.

[0239] The issue display area 1030 is an area for displaying information on the issues corresponding to the question.

[0240] The issue display area 1030 corresponds to the processing such as step S1427 in Figure 14.

[0241] The issue summary display area 1032 is an area for displaying the summary of the search results retrieved as the issues corresponding to the question.

[0242] In the illustrated example, the issue summary display area 1032 displays information about the issue retrieved by referring to the graph-type data structure 214 in response to the question, and the results of the answer generated by the large-scale language model service server 95 based on the search results and the question.

[0243] The individual issue display area 1034 is an area that displays the details of one of several issues.

[0244] Reference display area 1036 is the area that displays the source material (a legal book in the illustrated example) that supports the issue at hand.

[0245] In the illustrated example, the reference display area 1036 shows multiple legal books that were found by searching for a tag corresponding to the issue.

[0246] The evidence display area 1038 is the area that displays the section in legal textbooks that contains descriptions corresponding to the issue tags.

[0247] The evidence display area 1040 is an area that displays the section in legal textbooks that contains descriptions corresponding to the issue tags.

[0248] <Fourth Embodiment> In the fourth embodiment, we describe a technology that identifies the laws and regulations corresponding to the searched portion in the content and displays the latest laws and regulations and information on legal amendments, thereby making it easy to refer to the laws and regulations that were assumed at the time the content was published, as well as information on subsequent legal amendments.

[0249] As time passes since publication, content such as books may become outdated as it does not reflect legal revisions or the facts underlying those revisions. When referring to such content, there is a risk that one may not be aware of subsequent legal revisions and therefore not realize that the content is no longer relevant to the present time.

[0250] However, to confirm whether there are any legal revisions or the like, there is a problem that the effort is considerable, such as investigating the history of whether any legal revisions have occurred in the first place.

[0251] Accordingly, in the fourth embodiment, a technology will be described that facilitates viewing content in consideration of subsequent legal revisions and the like, even if time has passed since the content was issued and the content is becoming or has become outdated. This can further reduce the effort required for research.

[0252] <9 Operation (Fourth Embodiment)> Figure 16 is a diagram showing the flow of processing for identifying laws and regulations corresponding to portions in content that are specified as the basis for answers to questions, and outputting the latest laws and regulations.

[0253] The outline of each process is as follows.

[0254] Terminal 10: A user inputs a question, and the terminal 10 transmits the question to the server 20 Server 20: The server 20 generates a tentative answer to the question, and searches for related content based on the tentative answer Server 20: The server 20 identifies relevant laws and regulations from a portion of the specified content, and acquires the latest legal revision information Server 20: The server 20 organizes information to be presented to the user, including laws and regulations information from before the content was issued, and presents the information to the user Terminal 10: The terminal 10 displays the received information to the user, and highlights important portions. By integrating and displaying related information on the same screen, the user can easily refer to the content and the latest legal revision information With these processing flows, even when referring to old content for which time has passed since the issuance point, the user can simultaneously obtain the latest legal revision information, and the effort required for research can be reduced.

[0255] Next, each step will be described.

[0256] In step S1621, the question processing module 2044 of the server 20 presents the user with an operation screen that accepts the input of a question.

[0257] Here, the server 20 manages multiple pieces of content to be viewed in the storage unit 202 (such as the legal content database 212, the graph-type data structure 214, the data from the case law search service server 91, the data from the legal search service server 92, the data from the book browsing service server 93, and the data from the information media service server 94).

[0258] The content managed in the memory unit 202 may include information indicating the publication date of the content. More specifically, the content managed in the legal content database 212, etc., of the memory unit 202 may include at least one of the following: books, guidelines, or media articles, which indicate the publication date of the content. Here, content includes books, articles, and other materials primarily consisting of text and images, as well as videos.

[0259] The memory unit 202 stores information on multiple legal content items as multiple content items that are the subject of viewing. More specifically, the memory unit 202 may store information defining reference relationships between content items of the same type for each of the multiple legal content items, such as information that identifies laws and regulations, information that identifies precedents, information that identifies legal books, and information that identifies legal guidelines.

[0260] In step S1611, terminal 10 receives a question from the user. Terminal 10 sends the entered question to server 20. For example, server 20 may provide terminal 10 with an operation screen with text boxes and accept the question in natural language, or it may accept it as a keyword. Alternatively, it may provide voice guidance on the operation screen, or it may accept the question from the user via voice input.

[0261] In step S1622, the question processing module 2044 of the server 20 obtains a provisional answer to the question entered by the user by sending a prompt to an external information processing system to generate a provisional answer in the form of an answer statement based on the content of the received question.

[0262] More specifically, the question processing module 2044 generates a prompt that includes the content of the received question and an instruction that outputs the content of the question in answer format, and sends the generated prompt to an external information processing system, thereby receiving output corresponding to the prompt from the external information processing system. As described later, the question processing module 2044 searches multiple contents based on the content of the received answer format output. In the case of books, guidelines, and other contents, some are written in the form of answer sentences. For example, in the case of books and other contents, compared to searching by the content of the question, using the question (e.g., "In the case of ~~, is it possible to ~~?", "What should be noted about ~~?") in the form of an answer sentence ("In the case of ~~, it is possible to ~~", "What should be noted about ~~ is ~~") can make it easier to identify a part of the book content that serves as the basis for the answer corresponding to the content of the question. This makes it easier to identify a part of the content that serves as the basis for the answer when generating an answer to a question, and the quality of the answer to the question can be further improved.

[0263] The question processing module 2044 may also analyze the content of the question entered by the user using morphological analysis or the like, as in step S1622, extract words, search for information sources using the extracted words, or identify a part of the content using vector search based on the content of the question, and then generate an answer to the question based on this information.

[0264] In step S1623, the question processing module 2044 of the server 20 searches the database of each content in the graph-type data structure 214 based on the provisional answer text generated in step S1622. The question processing module 2044 identifies a portion (snippet) of the content corresponding to the question and obtains an answer to the question by sending a prompt containing instructions to an external information processing system to generate an answer based on the identified content and the user's question.

[0265] In this way, the question processing module 2044 identifies a portion of the content corresponding to the question by searching multiple contents in the storage unit 202 based on the content of the question received in steps S1611 and S1622. The question processing module 2044 may also identify a portion of the content corresponding to the question based on the reference relationships between the contents.

[0266] In step S1624, the question processing module 2044 of the server 20 outputs the generated answer to the terminal 10.

[0267] In step S1612, terminal 10 displays the answer along with information about a portion of the content that formed the basis for generating the answer. Terminal 10 accepts an operation from the user to specify the content.

[0268] In step S1625, the content presentation module 2047 of the server 20 extracts relevant laws and regulations by search (e.g., vector search) based on the content of a snippet of the specified content that is identified as the basis for generating the answer.

[0269] Thus, the content presentation module 2047 may extract information on laws and regulations related to a portion of the content by performing a vector search of the legal database (such as the data on the legal search service server 92) based on the portion of the content identified in step S1623. The question processing module 2044 may then present the user with information on laws and regulations related to the extracted portion of the content in step S1626, which will be described later.

[0270] In step S1626, the content presentation module 2047 of server 20 refers to a legal database (such as the legal search service server 92) and obtains the latest amendment information for the relevant laws. The content presentation module 2047 may also obtain legal amendment information and the latest legal provisions from after the publication date of the content specified by the user for viewing. The content presentation module 2047 may also obtain legal information prior to the publication date of the content. The content presentation module 2047 may also obtain legal amendment information depending on whether it is before or after the publication date of the content.

[0271] In step S1627, the content presentation module 2047 of the server 20 presents the user with a screen for viewing the content specified by the user. The content presentation module 2047 outputs information on legal amendments, legal provisions, and the publication date of the content to the terminal 10.

[0272] Thus, the content presentation module 2047 presents the user with information on legal amendments to applicable laws when a specified portion of the content contains legal information. For example, the content presentation module 2047 may present the user of terminal 10 with information on legal amendments to applicable laws when a portion of the content displayed on the content viewing screen contains legal information, regardless of when the content was published.

[0273] Content presentation module 2047 may also present users with information on legal amendments, including the most recent legal amendments, that have been in effect since the publication date of the specified content. This allows users to view the content while referring to information on legal amendments made since the publication date of the content, making it easier to make legal judgments and other decisions in consideration of legal amendments.

[0274] The content presentation module 2047 may also display information about legal amendments and information about the relevant articles of law on the user interface that displays the identified portion.

[0275] The content presentation module 2047 may display information on the publication date of the specified content, information on legal amendments, and information on the relevant legal provisions on the user interface. This allows users to view the content while referring to the information on amended legal provisions and provisions compared to the publication date of the content, making it even easier to make legal judgments and other decisions while taking legal amendments into consideration.

[0276] The content presentation module 2047 may also display information on legal amendments made after the publication date of the specified content, along with information on laws and regulations prior to the publication date, on the user interface. This allows users to view the content while confirming the information on the laws and regulations on which the content is based, and also to view the content while considering information on legal amendments made after the publication date, making it even easier to make legal judgments and other decisions in consideration of legal changes.

[0277] The content presentation module 2047 may display the content on the user interface and highlight a portion of the content that corresponds to the question. This allows the user to easily identify the portion of the content that forms the basis of the answer to the question, making it easier to verify the validity of the answer and facilitating legal judgments.

[0278] In step S1613, terminal 10 displays information received from server 20 (such as a screen for viewing content, legal amendments, and legal provisions) to the user.

[0279] In step S1614, terminal 10 presents the user with a screen for viewing content. Terminal 10 highlights a portion of the content (the portion that forms the basis for the answer to the user's question) and displays information about legal amendments related to that portion.

[0280] More specifically, terminal 10 highlights a portion of the content corresponding to the question, and when it accepts the user's selection of content, it displays information on legal amendments related to the selected portion. On the same screen, terminal 10 displays the content's publication date, legal amendment information, and the legal text.

[0281] <10. Screen Example (Fourth Embodiment)> Figure 17 shows an example screen of a user interface for viewing content, which displays both the content itself and a table of contents summarizing it.

[0282] This also includes a description of a screen example relating to the fifth embodiment.

[0283] The target operation screen 1700 is an operation screen that accepts operations for viewing content.

[0284] The browsing target operation screen 1700 corresponds to each process such as steps S1627 and S1613 in Figure 16, and steps S2021 and S2012 in Figure 20, which will be described later. For example, as shown in Figures 10, 12, and 15, the screen that displays the answer to a question displays the content (such as legal books) that formed the basis for generating the answer, and in response to an operation to specify this content, the browsing target operation screen 1700 that allows viewing of the specified content is displayed. For example, in response to an operation in which the user specifies each document (basis display area 1038, 1040) displayed in the reference document display area 1036 of Figure 15, the browsing target operation screen 1700 is displayed with that document as the content to be viewed.

[0285] The account display area 1702 is the area that displays the user's account. The viewing area 1704 is the area that displays the content to be viewed.

[0286] In the illustrated example, the viewing area 1704 displays a portion of a legal book as content.

[0287] Snippet display area 1706 is the area that displays a portion of the content that is highlighted.

[0288] In the illustrated example, the snippet display area 1706, as shown in Figures 10, 12, and 15, displays the content (such as legal books) that formed the basis for generating the answer, and accepts user input to specify content, displaying the target operation screen 1700. In such cases, the section containing the part that formed the basis for generating the answer is displayed in response to the user's specified operation, and that part is highlighted by a highlighter or similar method. This makes it easy for the user to check the answer to a question, as shown in Figure 10, and to easily refer to the source book or other document while also checking the section that formed the basis.

[0289] The sub-window display area 1708 is an area that displays various information related to the content being viewed.

[0290] The bookshelf addition operation unit 1710 is an operation component that accepts the operation to add a bookmark to the displayed content (book).

[0291] The content cover image display area 1712 is the area that displays the cover image of the content (book) being viewed.

[0292] The content bibliographic information display area 1714 is the area that displays the bibliographic information of the content being viewed.

[0293] More specifically, the content bibliographic information display area 1714 displays information such as the name of the content (title in the case of a book), the edition of the content, the date of publication, the entity that published the content, and the author of the content. In the illustrated example, the system accepts the operation of copying this bibliographic information. This makes it easy for users to cite the content information that serves as the basis when reporting their opinions or other opinions based on the content of a book.

[0294] The table of contents specification unit 1716 is an operating component that accepts the specification of a table of contents as various information of the content to be viewed.

[0295] In the illustrated example, the table of contents designation section 1716 displays tabs to switch between different types of information related to the content being viewed. These tabs include "Table of Contents" to show the table of contents of the content, "Other Related Content" to display other content different from the content being viewed (a book in the illustrated example) (laws, precedents, guidelines, etc. in the illustrated example), "Bookmarks" to display a list of locations selected by the user within the content, and "Binder" to manage various types of content together for the content or a part of the content. In the example in Figure 17, the "Table of Contents" tab is designated and displayed to distinguish it from the other tabs.

[0296] The related information designation unit 1718 is an operating component that accepts the designation of other related content as various types of information for the content being viewed.

[0297] In the illustrated example, the related information designation unit 1718 is shown as being designated by the user in the example of Figure 18.

[0298] The bookmark designation unit 1720 is an operating component that receives a designation to display a list of locations selected by the user within the content being viewed, as various information about the content being viewed.

[0299] The binder designation section 1722 is an operating component that accepts designations to register content or a part of content in a "binder" that manages various types of content together.

[0300] In the illustrated example, the binder designation unit 1722 accepts the operation of registering the content itself, or a part of the content (in the case of a book), in association with a "binder" whose name can be arbitrarily set by the user. This allows, for example, if the user sets up a "binder" corresponding to a particular issue, the user can easily check the content or a part of the content registered in the "binder" when organizing their views based on that issue, making it easier to review the content again.

[0301] The detail display area 1724 is an area that displays detailed information related to the content being viewed.

[0302] In the illustrated example, the detail display area 1724 displays the table of contents in Figure 17, highlighting the position in the table of contents that corresponds to the currently viewed section (page number in the book) displayed in the viewing target display area 1704. This makes it easy to understand the relationship between the overall content and the section the user is viewing, and makes it easier for the user to refer to other parts of the content. In Figure 18, information on legal revisions, laws, guidelines, and case law is displayed as other content related to the book.

[0303] The legal reference relationship display area 1726 is an area that displays the legal laws in which the reference relationship is defined.

[0304] In the illustrated example, the legal reference relationship display area 1726 is set to a link to a legal database in which the reference relationship is defined. Depending on the user's operation, the system may transition to the service provided by the legal search service server 92 to display the relevant laws, or the laws may be displayed on a screen such as Figure 17 as an overlay.

[0305] The legal reference relationship display area 1728 is an area that displays the legal laws in which the reference relationship is defined.

[0306] The case reference relationship display area 1730 is an area that displays court cases in which a reference relationship is defined.

[0307] In the illustrated example, the case law reference relationship display area 1730 is set to a case law database in which the reference relationship is defined. Depending on the user's operation, the system may transition to the case law search service server 91 to display the case law provided by that service, or the case law may be displayed on a screen such as Figure 17 as an overlay.

[0308] Figure 18 shows an example of a screen in a content viewing operation screen, which displays the content itself and other related content.

[0309] The related content update operation unit 1732 is an operation component that receives an operation to update the display content of other related content for the content being viewed, which is displayed in the detailed display area 1724.

[0310] In the illustrated example, the related content update operation unit 1732 updates the display of information such as legal revisions, laws, guidelines, and precedents related to a portion of the content displayed in the viewable display area 1704, in response to user operations. This corresponds to the process shown in step S2023 of Figure 20.

[0311] The legal amendment information display area 1734 is an area for displaying information about legal amendments.

[0312] The legal amendment information display area 1734 corresponds to the processing shown in step S1627 of Figure 16. In the illustrated example, it accepts an operation to specify information about legal amendments, and in response to this operation, it displays the details of the legal amendment information as shown in Figure 19.

[0313] The legal information display area 1736 is an area for displaying legal information.

[0314] The legal information display area 1736 corresponds to the process shown in step S1627 of Figure 16. In the illustrated example, links are set to the legal information, and information such as the legal database is displayed as described above in response to the operation of specifying the link.

[0315] Guideline information display area 1738 is the area for displaying guideline information.

[0316] The guideline information display area 1738 corresponds to the process shown in step S2022 of Figure 20.

[0317] The case law information display area 1740 is an area for displaying case law information.

[0318] The case law information display area 1740 corresponds to the processing shown in step S2022 of Figure 20.

[0319] Figure 19 shows an example screen of a user interface for viewing content, where information on laws and legal amendments is displayed as other content related to the content being viewed.

[0320] The legal amendment details display area 1742 is an area that displays detailed information about legal amendments.

[0321] In the illustrated example, the legal amendment details display area 1742 is displayed in place of the sub-window display area 1708 shown in Figures 17, 18, etc. Alternatively, the legal amendment information may be displayed in a separate window, or the legal amendment information may be displayed in the sub-window display area 1708 in response to user operations.

[0322] The legal compliance confirmation operation unit 1744 is an operating component that accepts operations to confirm the legal compliance that is being displayed in detail.

[0323] In the illustrated example, the legal information verification unit 1744 displays legal information, such as that found in the legal database, in response to user operations. This makes it easy to verify each article of the law.

[0324] The window close operation unit 1746 is an operating member that receives an operation to close the legal amendment details display area 1742.

[0325] In the illustrated example, the window close operation unit 1746, in response to user operation, erases the display of the legal amendment details display area 1742 and displays the sub-window display area 1708 shown in Figures 17 and 18, etc.

[0326] The legal amendment information display area 1748 is an area that displays details of legal amendment information.

[0327] The legal amendment information display area 1748 corresponds to the processing shown in step S1627 of Figure 16. In the illustrated example, the latest legal amendment information and the information before the amendment are displayed. Alternatively, information on the publication date of the content being viewed may be displayed, along with information on legal amendments before and after that publication date.

[0328] <Fifth Embodiment> The fifth embodiment describes a technology that displays other content (such as laws, precedents, guidelines, etc.) related to the currently open section (a portion of a page in a book) while viewing content such as books.

[0329] When conducting research on content, asking questions to a large-scale language model and examining the responses it generates can provide an overview. However, the responses may be too abstract, leading to misunderstandings or even errors. Therefore, it is necessary to verify the accuracy of the answers. For example, when organizing legal opinions, it is necessary to thoroughly understand the basis for those opinions and then summarize them while referring to evidence.

[0330] While there is a book browsing service that allows you to view books, it's not easy to find the specific book you need for what you want to know, or to locate the relevant section within that book.

[0331] Therefore, this document describes a technology that allows users to view various pieces of information related to the section they are viewing simultaneously, thereby further reducing the effort required to research original sources that serve as evidence.

[0332] <11 Operation (Fifth Embodiment)> Figure 20 shows the process flow for displaying a portion of other content related to the content being displayed, while simultaneously presenting an operation screen for viewing content.

[0333] The outline of each process is as follows:

[0334] Content Viewing and Presentation of Related Information: Users can view legal books and other content while simultaneously referencing relevant laws, precedents, and guidelines for the page they are viewing. This facilitates the collection of detailed information and accurate understanding. User Interface Configuration: The screen is broadly divided into a first area (main content display) and a second area (overview and related information display). The content displayed in the second area can be flexibly switched by the user's operation. Identifying related content: Server 20 identifies other related content based on the content being viewed or selected. Information about the relationships between content is used in this process. Related Content Information Update: When a user changes the content they are viewing, related information is dynamically updated accordingly (or in response to user actions). This makes it easier to obtain the most relevant information for the content currently being viewed. Integration with the question answering function: In response to user questions, the server 20 performs searches and integrates with external information processing systems (the server 95 of the large-scale language model service) to present answers and their supporting evidence, making it easy to check the details of the original content. Research Support: This system allows users to easily access primary sources of other relevant content, such as laws and precedents, when reading specialized books and other materials, thereby facilitating information gathering and understanding.

[0335] These processes allow users, for example in legal research, to efficiently acquire legal knowledge and reduce the effort required for research and learning. Furthermore, the centralized provision of relevant information helps prevent overlooking or misunderstanding information, supporting more reliable judgment and analysis. Next, we will explain each step.

[0336] In step S2011, terminal 10 presents the user with a screen for viewing one of the following contents: laws, precedents, legal books, guidelines, etc. Terminal 10 presents the user with a screen for viewing the content specified by the user (one of the following: laws, precedents, legal books, guidelines, etc.). Terminal 10 accepts an operation from the user to specify the content they wish to view.

[0337] Here, the storage unit 202 of the server 20 stores at least multiple types of content, including laws, precedents, legal books, and legal guidelines, by associating parts of the content with each other (legal content database 212, graph-type data structure 214). The content presentation module 2047 presents the user with a screen that allows them to view any of the content stored in the storage unit 202, including laws, precedents, legal books, and legal guidelines.

[0338] The content presentation module 2047 identifies parts of other content that are associated with the part of the content being viewed on the screen being viewed.

[0339] Furthermore, the server 20 may generate answers to user questions. The question processing module 2044 receives a user question, searches various contents in the memory based on the question, creates a prompt that includes the search results in the search step and the user's question, and includes instructions to answer the question by referring to the search results, sends the created prompt to an external information processing system (the server 95 of the large-scale language model service) to obtain the answer generated for the question from the external information processing system (the server 95 of the large-scale language model service), and displays the content of the answer and a portion of the content that formed the basis for generating the answer to the user as the obtained answer. The content presentation module 2047 may accept an operation from the user to specify a portion of the content that is displayed as the basis for generating the answer, and then present a screen to the user displaying the portion of the content that was specified.

[0340] In step S2021, the content presentation module 2047 of the server 20 sends the content specified by the user to the terminal 10.

[0341] In step S2012, terminal 10 displays the acquired content in the first area of ​​the operation screen. Terminal 10 displays a second area on the operation screen that is different from the first area, and initially displays an overview of the content (such as a table of contents). Terminal 10 may also display the first area and the second area side by side on the operation screen. For example, tabs for changing the displayed content may be placed in the second area, and the displayed content may be switched according to the tab specified by the user.

[0342] In step S2013, terminal 10 accepts an operation to display "other related content" in the second area. For example, terminal 10 may display "other related content" in the second area in response to an operation on a tab to change the content to be displayed.

[0343] Thus, the content presentation module 2047 of server 20 provides a function to switch the content displayed in the second area between a "content overview" (such as a table of contents) and "other related content" in response to user interaction. Terminal 10 accepts the user's operation to select the display of "other related content" in the second area. In this way, the content presentation module 2047 displays a portion of the identified other content in response to user interaction.

[0344] As described above, the content presentation module 2047 displays a first area on the content viewing screen that displays the content to be viewed, and a second area different from the first area. Depending on the user's operation, the module switches whether the content displayed in the second area is an overview of the content to be viewed or a part of other specified content.

[0345] More specifically, the content presentation module 2047 displays the contents of a legal book as the content to be viewed in the first area of ​​the screen where the content is viewed, and in the second area, depending on the user's operation, switches between displaying an overview including the table of contents of the legal book, or displaying at least one part of laws, precedents, or legal guidelines as other content associated with the part of the legal book that is to be viewed.

[0346] In step S2022, the content presentation module 2047 of server 20 outputs portions of other related content, such as laws, precedents, and guidelines, associated with the portion of the content currently being viewed (e.g., a book). More specifically, the content presentation module 2047 identifies other content, such as laws, precedents, and guidelines, associated with the portion of the content currently being viewed. The content presentation module 2047 retrieves portions of the identified other content from various databases, etc.

[0347] In step S2014, terminal 10 displays a portion of other related content in the second area.

[0348] In step S2015, terminal 10 accepts operations to change the content viewing range (such as scrolling or page turning) and operations to select a portion of the content displayed in the first area (such as text selection).

[0349] In step S2023, the content presentation module 2047 of the server 20 identifies other content related to the selected area in response to a change in the content viewing range (in the case of a book, changing the page being viewed) or an operation to select a portion of the content (in the case of a book, the user selects a portion of the text), and outputs it to the terminal 10. For example, the content presentation module 2047 may be equipped with an operation component that accepts an operation to identify other related content (for example, an "update" button), and in response to an operation on the operation component, it may identify other content related to the range being viewed at the time the operation was performed, or the range in which a portion of the content has been selected, and output it to the terminal 10.

[0350] Thus, the content presentation module 2047 may re-identify other related content based on the scope of newly viewed content and update its display. Alternatively, the content presentation module 2047 may identify portions of other content (laws, precedents, guidelines) related to portions of content selected by the user and update their display.

[0351] In step S2016, terminal 10 displays the updated related content in the second area.

[0352] <12. Screen Example (Fifth Embodiment)> The example screen in the fifth embodiment is the same as that shown in Figure 18, etc., described in the fourth embodiment above.

[0353] <Sixth Embodiment> In the sixth embodiment, when generating answers to questions using an external information processing system such as the server 95 of the large-scale language model service, a function is provided that allows switching between simple, high-speed answers and slower but more in-depth, point-based answers, according to the user's desired answer style and search depth, thereby meeting a wide range of search needs.

[0354] Furthermore, we will describe the technology for determining the mode of response generation, more specifically, the technology for analyzing the user-entered question query and the literature searched based on that query, and determining the user's desired response style and search depth based on the nature of the question query, the search target field, and the variability of the search results. Here, we will describe the technology for automatically switching between simple, fast responses and slower but more in-depth, point-based responses depending on the determination result, thereby meeting a wide range of search needs.

[0355] For users, when generating answers to questions using external information processing systems such as large-scale language models, it is not easy to determine which of the various models to use. Furthermore, if a user always relies on a model that generates detailed answers, and there are restrictions on the number of times such models can be used (for example, a limit on the number of questions that can be asked within a certain period), there is a risk of unnecessarily reducing the number of times the model can be used. Also, if a user always tries to use a model that generates detailed answers, the time it takes to receive answers will increase. The more questions a user asks, the longer it will take to receive answers, which may hinder the smooth progress of the user's work.

[0356] Therefore, in the sixth embodiment, in order to generate answers that correspond to user questions while also shortening and optimizing the time required for research work and supporting the smooth progress of the user's work, the present invention describes a technology that determines whether to narrow the scope of the research to generate an answer or broaden the scope of the research to generate an answer, and if it is possible to generate an answer quickly by narrowing the scope of the research, the present invention describes a technology that responds in a mode that generates an answer quickly.

[0357] <13 Operation (Sixth Embodiment)> Figure 21 shows the process flow for generating an answer, which determines whether to use a first mode that generates answers quickly or a second mode that generates answers slowly and in detail, depending on the content of the question.

[0358] The outline of each process is as follows:

[0359] Mode Determination: Server 20 analyzes the content and difficulty of the question, the variability of the search results, the number of points of discussion in each field, etc., to determine whether fast mode or detailed answer mode is appropriate. User behavior feedback: Record the user's actions after confirming their answers, such as whether they ask additional questions or leave the system quickly, and use this information to determine the next mode. By providing such a mode determination for each user, it is possible to offer the most suitable response style for each user. Optimizing waiting times: By displaying the current processing mode while generating answers, users can more easily predict waiting times. Validity of the answer: Along with the final answer, the mode in which the answer was generated is displayed. This allows users to easily understand the depth and detail of the answer. According to the sixth embodiment, users can obtain answers of a depth that suits their needs at an appropriate time, which is expected to improve work efficiency and reduce the time required for research.

[0360] Next, we will explain each step.

[0361] In step S2121, the question processing module 2044 of the server 20 presents the user with an operation screen that accepts the input of a question.

[0362] In step S2111, terminal 10 receives a question input from the user. Terminal 10 sends the question to server 20.

[0363] In step S2122, the question processing module 2044 of the server 20 analyzes the content of the question received from the terminal 10 and determines whether it is in high-speed mode or detailed mode.

[0364] In this way, the question processing module 2044 analyzes the content of the user's question to determine whether to generate the answer in a first mode that generates an answer quickly, or in a second mode that generates an answer more slowly and in more detail than the first mode.

[0365] (Method for determining the mode to operate the external information processing system for generating the answer)

[0366] (1) Variation in the sources of information that answer the question The question processing module 2044 analyzes the content of the user's question, referring to a graph-type data structure 214, a legal content database 212, etc., and searching for information sources, including books. The question processing module 2044 determines whether to generate an answer in the first mode or the second mode, depending on the variability in the fields of the books. More specifically, the question processing module 2044 may determine to generate a detailed answer in the second mode if the variability in the fields of the books is large, and to generate an answer quickly in the first mode if the variability in fields is small.

[0367] For example, the question processing module 2044 may determine whether the genres of the books are scattered, based on whether the proportion of books in the same field (for example, books that have been pre-classified according to the field of law, etc.) is above a certain level, and the degree of dispersion across genres.

[0368] The question processing module 2044 may determine, based on the variability of the fields, whether to generate the answer in the first mode or the second mode, depending on the number of issues in the field. If the number of issues in the question is above a certain level, the answer may be generated in the second mode, and if the number of issues is below a certain level, the answer may be generated in the first mode.

[0369] (2) The server 95 of the large-scale language model service determines whether or not to generate a detailed response. The question processing module 2044 analyzes the content of the user's question and generates an evaluation prompt that includes the content of the question and instructions to evaluate whether further consideration of the question is necessary. The question processing module 2044 sends the generated evaluation prompt to an external information processing system (the server 95 of the large-scale language model service). The question processing module 2044 obtains the evaluation results from the external information processing system. Depending on the obtained evaluation results, the question processing module 2044 may determine whether to generate an answer in the first mode or the second mode.

[0370] Here, the question processing module 2044 may generate an evaluation prompt that includes the content of the question and instructions to evaluate whether the question requires further consideration in terms of whether or not there are many points to discuss.

[0371] (3) Determination based on the user's behavior history when presented with the generated answers to the questions. Server 20 may store in its memory unit 202 behavioral history information, which is a history of the user's actions regarding a question after the answer has been presented to the user. For example, Server 20 stores information relating the content of a question, the answer generated by the server 95 of the large-scale language model service in response to that question, and the user's actions after the answer has been presented to the user, such as the time spent viewing the answer and the number of additional questions asked.

[0372] The question processing module 2044 analyzes the content of the user's question and may determine whether to generate an answer in the first mode or the second mode based on the user's behavior history shown in the behavior history information regarding past questions related to the content of the question.

[0373] More specifically, the question processing module 2044 may make a determination based on the user's behavior history information regarding past questions, specifically the extent of additional questions the user asked in past questions, or the viewing time the user spent reviewing the answers, which is a history of the user's behavior in past questions. For example, if an answer was provided to the user for a similar past question and there were few additional questions (the number of additional questions, the number of characters in the additional questions, etc., were below a certain level), the module may determine that the provided answer to the question was sufficient and decide to respond in the first mode. Alternatively, if the viewing time the user spent reviewing the answers to similar past questions was below a certain level, the module may determine that the provided answer to the question was sufficient and decide to respond in the first mode.

[0374] Server 20 may be configured in its memory unit 202 to store a trained model that uses the content of the question and the history of the user's actions as training data based on the behavior history information, and is trained to output the content of the user's actions in response to the content of the question. Question processing module 2044 may make a decision based on the content of the question received in steps S2111 and S2122 and the output result of the content of the user's actions output based on the trained model.

[0375] (The timing for displaying the determination result of the mode for generating answers to questions) The question processing module 2044 displays an input field for receiving questions and an operating component for receiving the questions entered in the input field on the screen. When the module accepts the input of questions entered in the input field in response to user operation on the operating component, it determines the mode for generating an answer to the question entered in the input field. As a result of this determination, it may also present information to the user indicating whether the answer will be generated in the first mode or the second mode, even without accepting user operation on the operating component (i.e., regardless of whether an operation to generate an answer to the entered question has been performed). This makes it even easier for the user to understand whether detailed consideration is necessary before sending a question, and how long the waiting time until an answer is received will be.

[0376] In step S2112, terminal 10 displays that it is processing until the answer is generated, and whether it is in fast mode or detailed mode. Terminal 10 displays to the user that it is processing until the answer is generated by the question processing module 2044 of server 20, and the mode being used to generate the answer.

[0377] Thus, the question processing module 2044 may provide the user with information indicating whether it is generating the answer in the first mode or the second mode during the period in which it generates the answer before presenting it to the user. This makes it easier for the user to anticipate the waiting time until the answer is generated, eliminating the difficulty of operation caused by not knowing the waiting time, and potentially making the waiting time feel shorter.

[0378] In step S2123, the question processing module 2044 of the server 20, based on the determination result in step S2122, reduces the amount of information sources referenced based on the question to a certain level or less in the case of high-speed mode (first mode), while referencing more information sources in the case of detailed answer mode (second mode).

[0379] The question processing module 2044 prepares to generate an answer in the selected mode based on the determination result in step S2122. In fast mode, the amount of information sources referenced is reduced. In detailed mode, more information sources are referenced. For example, in fast mode, the number of books referenced may be limited to a certain number (e.g., 10 books), and the top a certain number (e.g., the top 3) may be presented as the basis for generating the answer. In detailed answer mode, the number of books referenced may be increased to a certain multiple compared to fast mode to generate the answer.

[0380] In step S2124, the question processing module 2044 of the server 20 generates a prompt containing the question and the information sources referenced in response to the question, and sends it to an external information processing system (the server 95 of the large-scale language model service).

[0381] Here, the question processing module 2044 may refer to a smaller amount of information sources when generating an answer in the first mode than in the second mode. The question processing module 2044 generates a prompt that includes the content of the user's question and the information sources it referenced, and includes instructions to generate an answer to the question based on the information sources, and obtains an answer from an external information processing system by sending the generated prompt to the external information processing system.

[0382] In step S2125, the question processing module 2044 of server 20 obtains an answer from an external information processing system (server 95 of the large-scale language model service). The question processing module 2044 sends the obtained answer to the terminal.

[0383] In step S2126, the question processing module 2044 of the server 20 may record the user's behavior history when viewing answers to questions and store it in the storage unit 202. In this way, the question processing module 2044 may record the user's behavior history when viewing answers to questions and utilize it for future mode determination.

[0384] The question processing module 2044 may also present the answer along with information indicating whether the answer was generated in the first or second mode. This allows the user to easily recognize which mode was used to generate the answer.

[0385] In step S2113, terminal 10 displays the generated response. Terminal 10 also displays whether the response was generated in high-speed mode or detailed viewing mode.

[0386] <14. Screen Example (Sixth Embodiment)> Figure 22 is an example screen that displays the result of determining whether to answer in the first mode (high-speed mode) or the second mode (detailed answer mode) before accepting the operation to submit the question, based on the content of the input question.

[0387] The mode determination result display area 2202 is an area that displays the determination result of whether to use high-speed mode or detailed answer mode as the mode for generating the answer, at the stage before the question is sent (when the question has been entered into the question specification unit 1002 but before the operation is performed on the transmission operation unit 1004).

[0388] The mode determination result display area 2202 corresponds to each process shown in Figure 21. In the illustrated example, it is displayed that it has been determined to generate an answer in detailed response mode.

[0389] The high-speed mode specification unit 2204 is an operating member that receives a specification to generate an answer in high-speed mode.

[0390] In the illustrated example, the high-speed mode selection unit 2204 is shown to be located inside the mode determination result display area 2202, but it may also be located near the transmission operation unit 1004, etc., and the user may be able to manually switch between generating an answer in high-speed mode and generating an answer in detailed answer mode.

[0391] Figure 23 is an example of a screen that displays the determination result of the mode for generating the answer on the user interface during the period from when a question is submitted until an answer is generated using an external information processing system.

[0392] The processing progress display area 2206 is an area that displays an overview of each process involved in generating an answer to a question, as well as the status of the processing.

[0393] In the illustrated example, the processing progress display area 2206 displays steps that have been completed, steps that are in progress, and steps that have not yet been processed, distinguishing between them.

[0394] The mode display area 2208 is an area that displays the mode used to generate the answer.

[0395] The mode display area 2208 corresponds to the process shown in step S2112 of Figure 21.

[0396] Figure 24 is an example of a screen that displays the result of determining the mode in which the answer was generated, on top of the screen that displays the answer generated for the question.

[0397] The mode display area 2210 is an area that displays the mode used to generate the answer, if an answer has been generated.

[0398] The mode display area 2210 corresponds to the processes in steps S2125, S2113, etc., in Figure 21.

[0399] <Seventh Embodiment> In the seventh embodiment, a technology is described that assists in refining questions in a system that generates answers to questions, in order to make it easier to obtain suitable answers.

[0400] In interactive systems that generate answers to questions, users may not know how to formulate their questions effectively. Even if they input a vague, abstract question, the resulting answer may be broad and superficial, lacking specificity and therefore unhelpful.

[0401] Therefore, this section describes technologies that enable users to quickly recognize the questions they want to ask, thereby making it easier for them to obtain answers to their questions. More specifically, it describes technologies that make it easier for users to input questions, such as by presenting suggested questions in response to their initial questions.

[0402] <15 Operation (Seventh Embodiment)> Figure 25 shows the processing flow for receiving follow-up questions that delve deeper into the questions entered by the user.

[0403] The outline of each process is as follows:

[0404] Question Input and Analysis: Analyzes user-entered questions and evaluates their specificity. Facilitating the Specificity of Questions: When the question needs to be more specific, provide relevant information to encourage the input of a more detailed question. Answer generation: Based on the specified question, information sources are searched and the answer is generated by the server 95 of the large-scale language model service. Providing answers: Present the obtained answers to the user. These processing flows allow users to clarify what they want to know and receive appropriate answers. Even vague questions can be helped to become more specific through the processing of server 20, improving the user experience.

[0405] Next, we will explain each step.

[0406] In step S2521, the question processing module 2044 of the server 20 presents the user with an operation screen that accepts the input of a question.

[0407] Server 20 maintains a graph database (graph-type data structure 214) in its storage unit 202 as an information source. This graph database is a graph-type data structure in which reference relationships are defined between parts of the content of multiple contents. The graph database also maintains information about attributes set on at least one of the parts of the content for which a reference relationship is defined, or on the reference relationship itself.

[0408] In the memory unit 202, as a graph database, at least multiple types of content, including laws, precedents, legal books, and legal guidelines, are stored, with reference relationships defined by linking parts of the content together, and information on attributes, including at least information on the issues at stake, is also stored.

[0409] In step S2511, terminal 10 receives a question (first input) from the user. Terminal 10 sends the entered question to the server device.

[0410] In step S2522, the question processing module 2044 of the server 20 receives the user's question. The question processing module 2044 determines whether the received question is specific.

[0411] (Assess whether the question is specific and whether additional questions or further clarification are needed.) The question processing module 2044 determines whether further specific questions are needed to generate an answer to the question related to the first input. If it is determined that further specific questions are needed, the question processing module 2044 performs the following step S2544 (prompting the specificization of the question). If it is determined that further specific questions are not needed, the module may generate a prompt that includes the content of the question related to the first input and the search results obtained by searching for information sources based on the content of the question, and includes an instruction to generate an answer by referring to the search results obtained by searching for information sources for the question. The generated prompt may also be sent to an external information processing system (large-scale language model service server 95) to obtain the answer created by the external information processing system (large-scale language model service server 95) in response to the prompt, and to output the obtained answer to the user.

[0412] (The server 95 of the large-scale language model service determines whether further questioning is necessary.) The question processing module 2044 may also perform a determination by generating a prompt that includes the content of a question relating to a first input, as shown in step S2511, and includes an instruction to determine whether a further specific question is required to generate an answer to the said question, and by sending the generated prompt to an external information processing system (large-scale language model service server 95), and obtaining the result of the determination created by the external information processing system (large-scale language model service server 95) in response to the prompt.

[0413] (Determined based on the scope of the graph database reference) The question processing module 2044 may obtain information within the range referenced from the graph database (graph-type data structure 214) by referring to the graph database as an information source based on the content of the question relating to the first input as shown in step S2511, and may make a determination according to the extent of the referenced range obtained.

[0414] The question processing module 2044 may make a determination based on at least one of the following as information about the range referenced in the graph database (graph-type data structure 214): the number of content items included in the referenced range, or the amount of data including the number of characters in the content. For example, the module may determine that the more content items included in the referenced range (for example, the more books extracted), the more the module needs to delve deeper into the question and make it more specific. Alternatively, the more characters in the referenced content, the more the module may determine that the more the module needs to delve deeper into the question and make it more specific.

[0415] If the question processing module 2044 determines that a more specific question is required, it may present the user with candidate questions in the following step S2524.

[0416] (End of questioning) The server 20 may store in its memory unit 202 information that is sufficient to prompt the user to input specific questions. For example, it may store information such as the upper limit on the number of questions asked and the range within which the graph-type data structure 214 is referenced as the above-mentioned upper limit information.

[0417] The question processing module 2044 may terminate the process of prompting the user to input a more specific question when it has repeatedly received question input from the user through processes such as step S2511 in this embodiment and step S2513 described later (for example, when the user asks a question after being prompted to ask a more specific question), and the extent to which the information sources have been consulted has reached the upper limit of information. Here, the question processing module 2044 may terminate the process of prompting the user to input a more specific question and present the user with a response indicating that it consulted the information sources for the question but could not find information corresponding to the question. This allows the user to confirm that they have investigated but could not find the relevant information.

[0418] In step S2523, if the question processing module 2044 of server 20 is insufficient (abstract), it creates candidate specific questions to present to the user by referencing information sources within a certain scope and generating a list of relevant keywords and points of discussion. For example, it generates a list prioritizing keywords and points of discussion that have a high score related to the question among the results of referencing the above information sources within a certain scope.

[0419] In step S2524, the question processing module 2044 of the server 20 sends relevant information to the terminal 10 along with a message prompting the terminal to elaborate on the question.

[0420] In this way, the question processing module 2044 refers to information sources within a certain range in response to the content of the user's question in the first input, and presents the information within that range to the user by responding with the information within that range, while also prompting the user to enter a more specific question than the one in the first input.

[0421] The question processing module 2044, in responding to the user's question related to the first input with referenced information within its scope, may present the referenced information to the user without using a process that causes an external information processing system, which generates an answer in response to prompt input, to identify the information source corresponding to the question.

[0422] The question processing module 2044 refers to the graph database (graph-type data structure 214) as an information source based on the content of the question related to the first input. The question processing module 2044 presents the user with information extracted based on multiple attributes (such as issues) included in the referenced range. At this time, the question processing module 2044 may also prompt the user to input a more specific question using the extracted information as an example.

[0423] The question processing module 2044 may present the user with a list containing the issues and keywords included in the range obtained by referring to the graph database based on the content of the question from the first input, and prompt the user to input a more specific question.

[0424] (Determine if the question includes multiple themes) The question processing module 2044 may also determine whether a question received from the user contains multiple themes.

[0425] The question processing module 2044 may, if it determines that a question contains multiple themes, classify the questions according to the themes and present the classification results to the user.

[0426] More specifically, the question processing module 2044 generates a prompt that includes the question it has received as input, and includes an instruction that determines whether the question contains multiple themes. The question processing module 2044 sends the generated prompt to an external information processing system (the server 95 of the large-scale language model service). The question processing module 2044 obtains information from the external information processing system (the server 95 of the large-scale language model service) regarding the determination made according to the instruction. The question processing module 2044 may also make a determination based on the determination result information obtained from the server 95 of the large-scale language model service.

[0427] Furthermore, the question processing module 2044 may also obtain information about the range referenced in response to the question by referring to the graph database (graph-type data structure 214) based on the question it has received as input, and then make a determination based on the information about the range it has obtained. For example, when the graph-type data structure 214 is referred to in response to a question, the module may determine whether multiple themes (e.g., multiple issues) are included based on the information about the nodes and edges set in the range it has referenced.

[0428] In step S2512, terminal 10 displays candidate questions and related information to the user, and displays a message prompting the user to enter further questions.

[0429] In step S2513, terminal 10 receives a specific question (second input) from the user. Terminal 10 sends the question to server 20.

[0430] In step S2525, the question processing module 2044 of the server 20 searches for information sources based on the question and retrieves the search results. In this way, the question processing module 2044 receives a specific question entered by the user, searches for information sources based on the specific question, and retrieves the search results.

[0431] In step S2526, the question processing module 2044 of server 20 generates a prompt containing the question and search results and sends it to an external information processing system (server 95 of the large-scale language model service). The question processing module 2044 retrieves the answer from the external information processing system (server 95 of the large-scale language model service).

[0432] Thus, the question processing module 2044 generates a prompt that includes the content of the user's question related to the second input and the search results of the search step, and that includes instructions to generate an answer to the question while referring to the search results. The question processing module 2044 sends the generated prompt to an external information processing system (the server 95 of the large-scale language model service), thereby obtaining the answer created by the external information processing system in response to the prompt.

[0433] In step S2527, the question processing module 2044 of server 20 sends the answer obtained from the server 95 of the large-scale language model service to terminal 10.

[0434] In step S2514, terminal 10 displays the answer to the user.

[0435] <16. Screen Example (Seventh Embodiment)> Figure 26 shows an example screen for receiving follow-up questions that delve deeper into the questions entered by the user.

[0436] The question content display area 2602 is the area that displays the question entered by the user (the first question).

[0437] In the illustrated example, the question content display area 2602 displays the content of the question received in step S2511 of Figure 25.

[0438] The question candidate display area 2604 is an area that displays candidate questions when it is determined that further investigation of the question is necessary.

[0439] The question candidate display area 2604 corresponds to the processes in steps S2524, S2512, etc., in Figure 25.

[0440] The question content display area 2606 is an area that displays the question (second input) that the user has entered.

[0441] In the illustrated example, the question content display area 2606 displays the content of the question received in step S2513 of Figure 25.

[0442] <Eighth Embodiment> In the eighth embodiment, for example, a system that generates answers to questions will display a list of content that forms the basis of the answer in multiple stages, and will describe a technique for switching between a mode that makes it easier to grasp the overall picture of many contents and to browse through the quoted sections of each content, and a mode that makes it easier to compare and consider which content to view in detail.

[0443] If the content is a book, in the first stage, the book title and a certain number of characters are displayed for each piece of content. In the second stage, a larger number of characters from the quoted passages of the book are displayed. The window is enlarged compared to the first stage to allow users to view more of the quoted text and make it easier to select a book.

[0444] Since responses from large-scale language models are not always accurate, it is difficult to rely solely on their responses when conducting research, and it is necessary to consult more evidence-based materials. On the other hand, content such as books is often created by authors according to their own purposes, and may contain a variety of information.

[0445] Therefore, it is not easy for users to determine which materials they should consult to find the information they need.

[0446] Therefore, this document describes a technology that further reduces the workload required for research by providing users with an overall understanding of what they want to know and making it easier for them to identify evidence-based materials.

[0447] <17 Operation (Eighth Embodiment)> Figure 27 shows the process flow for displaying a list of content related to a question while presenting the answer, and changing the display of the list according to the user's actions.

[0448] The outline of each process is as follows:

[0449] Two-stage display: The user first receives the answer and a list of concise content (first mode). If the user wants to see more details, the number of quoted sections for each content increases through user interaction, resulting in a more information-rich display (second mode). Display Mode Switching: When switching from the first mode to the second mode, an animation is performed in which the display area expands sequentially both vertically and horizontally, visually notifying the user that the amount of quoted text displayed is increasing. Conversely, when returning from the second mode to the first mode, an animation is performed in which the display area shrinks sequentially. Content List by Issue: The content list is organized and presented to the user by legal issue. When the user switches issues, the name of the displayed issue and the list of related content are updated. Research Scope Information Provision: Users can see the number of information sources consulted, the number of words, and the estimated time taken based on their average reading speed. This makes it easy to understand the extent to which the time and amount of information required for research have been reduced compared to individually viewing each book or other content. Utilizing a graph database: Server 20 refers to the graph database to identify appropriate information sources for a question. The graph database includes laws, precedents, legal books, guidelines, etc. Answer generation by server 95 of the large-scale language model service: Based on the identified information sources, answers are generated using server 95 of the large-scale language model service. The answers include evidence of the information sources, providing reliable information to the user. Improved user experience: Switching display modes and selecting discussion points is easier, allowing users to easily access the information they need. This processing flow allows users to efficiently review the documents that formed the basis of their answers and easily access detailed information as needed. As a result, the workload associated with the survey is reduced, and higher quality information can be collected.

[0450] Next, we will explain each step.

[0451] In step S2721, the question processing module 2044 of the server 20 presents the user with an operation screen that accepts the input of a question.

[0452] The memory unit 202 holds a graph database (graph-type data structure 214), which is a graph-type data structure that defines reference relationships between parts of the content of multiple contents. The memory unit 202 holds at least the information of the legal book content in the graph database (graph-type data structure 214) as an information source.

[0453] In step S2711, terminal 10 receives a question from the user. Terminal 10 sends the entered question to server 20.

[0454] In step S2722, the question processing module 2044 of the server 20 refers to the graph database (graph-type data structure 214) to identify the content that will serve as the information source to be referenced for the question.

[0455] In step S2723, the question processing module 2044 of the server 20 creates a prompt to generate an answer based on the identified information source (including books) and sends it to an external information processing system (the server 95 of the large-scale language model service), thereby obtaining the answer from the external information processing system (the server 95 of the large-scale language model service).

[0456] The question processing module 2044 generates answer information that includes the content of the answer to the question and information about the source on which the answer is based, based on the identified information sources.

[0457] More specifically, the question processing module 2044 creates a prompt that includes the content of the question and identified information sources, and that includes instructions to generate an answer by referring to the information sources regarding the question. By sending the created prompt to an external information processing system (the server 95 of the large-scale language model service), the module generates answer information by obtaining the answer generated by the external information processing system (the server 95 of the large-scale language model service). In this way, the module causes the server 95 of the large-scale language model service to generate an answer to the question, along with the basis for the answer.

[0458] In step S2724, the question processing module 2044 of the server 20 generates the content of the answer and a list of the content (information sources) on which the answer is based. The question processing module 2044 sends the answer and the list of content to the terminal 10.

[0459] Here, the question processing module 2044 may calculate the number and character count of the referenced sources when generating the answer and a list of the content (sources) on which the answer is based, and calculate the time taken from the average reading speed. It may also send this calculated information (such as the number of referenced sources) along with the answer and the list of content to terminal 10.

[0460] More specifically, the question processing module 2044 presents the user with a list of multiple source contents that formed the basis of the answer, either in a first manner or in a second manner that presents a larger portion of the relevant sections of each source content that formed the basis of the answer than in the first manner.

[0461] (Display mode and switching between display modes) The question processing module 2044 displays both the answer and a list of the supporting content on the screen. The question processing module 2044 accepts user input to switch between displaying the content list in either the first or second format. The question processing module 2044 switches between the first and second formats to display the list according to the user's input.

[0462] More specifically, in the second embodiment, the question processing module 2044 may display the list in a manner that increases the display area and amount of content for each item in the list compared to the first embodiment. This makes it easy to grasp the overall picture of the content included in the list in the first embodiment, while in the second embodiment, it makes it easy to consider and select from among the candidates for content to which the user wishes to check the details.

[0463] The question processing module 2044 may display the answer content and a list of supporting content on the screen, and may also display an operation member on the screen that accepts an operation to switch whether to display the content list in the first or second form, in association with the list. When switching from the first form to the second form in response to the user's operation on the operation member, the question processing module 2044 may transition to the second form by drawing the display area of ​​each content in the first form to expand sequentially in at least one of the vertical or horizontal directions. This makes it possible to visually recognize to the user that the amount of content that can be viewed in each content is greater in the second form than in the first form, in response to the user's operation, making the user operation even easier.

[0464] The question processing module 2044 may transition to the first mode by sequentially shrinking the display area of ​​each content in the second mode in at least one of the vertical or horizontal directions when switching from the second mode to the first mode in response to user operation on the control element displayed in association with the list. This makes it easier for the user to recognize that the amount of content that can be viewed is reduced when switching from the second mode to the first mode, making it easier to grasp the overall picture of each content included in the list, and thus makes user operation even easier.

[0465] More specifically, the question processing module 2044 may display a list of legal books related to each legal issue included in the answer to the question, as a list of multiple contents. The question processing module 2044 displays an operation element that accepts the operation of switching between legal issues in the list of legal books. In response to the user's operation on the operation element for switching legal issues, the question processing module 2044 may update the name of the legal issue as the title of the legal book list and display a list of legal books corresponding to the legal issue to be displayed. This makes it easy to check a list of legal books for each legal issue, and makes it easy to select a book to check in detail from the list displayed by narrowing down the legal issues.

[0466] (Checking the amount of data in the referenced information sources) When the question processing module 2044 presents the user with a list of the legal books that served as the basis or source of information, it may also present the user with information about the amount of information contained in each source.

[0467] More specifically, the question processing module 2044 may present the user with information about the amount of information in the information source, by presenting the user with at least one of the following: the number of contents in the referenced information source or the amount of data contained in the contents of the referenced information source. The question processing module 2044 may also present the user with information about the amount of information in the information source, by calculating the time it would take a human to read the referenced information source and presenting the calculated time to the user.

[0468] In step S2712, terminal 10 presents the user with the response received from the server and a list of content (first mode). Terminal 10 also presents the user with the number of information sources referenced, the number of characters, and the time taken, as information within the scope of the research.

[0469] In step S2713, terminal 10 switches the display mode to change the content, amount of information, and display area of ​​the list in response to user operations on the list of content (switching from the first mode to the second mode, switching the topic, switching from the second mode to the first mode).

[0470] Terminal 10 switches to the second mode when it receives an operation from the user to display a detailed view of the list of content displayed in the first mode. At this time, in response to the user's operation, terminal 10 displays an animation that sequentially expands the display area and amount of information for each content in the list of content as a switch to the second mode. In the second mode, the quoted portion of each content is increased and detailed information is displayed. If the user performs an operation to switch the content display mode again, an animation that sequentially shrinks the display area is displayed and the system transitions from the second mode to the first mode.

[0471] Terminal 10 accepts user requests to switch topics. In response to the topic switch, Terminal 10 updates the list of displayed topic names and related content.

[0472] In step S2714, terminal 10 receives a specification from the user regarding the content they wish to view in detail.

[0473] In step S2725, the content presentation module 2047 of the server 20 retrieves detailed information of the specified content and responds to the terminal 10. For example, if a legal book is specified, the contents of that legal book are output to the terminal 10.

[0474] In step S2715, terminal 10 presents the user with details of the acquired content.

[0475] <18 Screen example (8th embodiment)> Figure 28 is an example of a screen that displays a list of content related to a question while presenting the answer, in a first manner that facilitates understanding the overall picture of the content.

[0476] Figure 29 is an example screen of a second mode in which a list of content related to a question is displayed, with the amount of information in a portion of each content increased to assist in selecting content to view in more detail.

[0477] Operation screen 2800 is an operation screen that displays the generated answers to questions.

[0478] The operation screen 2800 corresponds to the process shown in step S2724 and other steps in Figure 27.

[0479] The answer display area 2802 is the area that displays the answers generated in response to the question.

[0480] The input question display area 2804 is an area that displays the content of the question entered by the user.

[0481] The answer summary display area 2806 is an area that displays a summary of the answers included in the generated responses.

[0482] The issue display area 2808 is an area that displays a list of issues included in the generated response.

[0483] The research scope display area 2810 is the area that displays the range of information sources referenced to generate the answer.

[0484] The first issue display area 2812 is the area that displays the answer to the first of the multiple issues included in the generated answer.

[0485] The bibliography selection unit 2814 is an operating member that accepts a specification to display a list of documents that served as the basis for generating the answer to the first point of contention.

[0486] In the illustrated example, the bibliography selection unit 2814 switches the list of bibliographies displayed in the bibliography display area 2902 to one that corresponds to the first issue, in response to user operation (switching the group of bibliographies to be displayed in the list group display area 2904).

[0487] The reference list display area 2902 is an area that displays a list of content (books, guidelines, etc.) that was referenced when generating the answer.

[0488] The list group display area 2904 is the area that displays the group of content to be displayed.

[0489] In the illustrated example, the list group display area 2904 is divided into groups corresponding to the issues shown in the issue display area 2808. The displayed groups are switched according to the user's actions on the list group display area 2904.

[0490] The list expansion / contraction operation unit 2906 is an operation member that accepts operations to expand or contract the display area for each content item in order to increase the amount of information cited from each content item, in order to display a list of content items that are displayed in the bibliography list display area 2902.

[0491] In the illustrated example, the list expansion / contraction unit 2906 transitions from the state shown in Figure 28 to the state shown in Figure 29 in response to user operation. Specifically, in response to user operation on the list expansion / contraction unit 2906, the horizontal width of the bibliography display area 2902 is expanded while increasing the amount of information displayed for citations in each content item in the content display area 2910. In the illustrated example, in response to user operation on the list expansion / contraction unit 2906, the width of the answer display area 2802 is reduced, and the line breaks of the answer text are changed accordingly. However, the width of the answer display area 2802 itself may not be changed, and the display position of the answer display area 2802 may be moved horizontally in accordance with the expansion of the display width of the bibliography display area 2902. In the example in Figure 28, the display mode of the list expansion / contraction unit 2906 is shown as expanding the bibliography display area 2902. In the example shown in Figure 29, the display mode of the list expansion / contraction operation unit 2906 is shown as reducing the size of the bibliography list display area 2902.

[0492] The window close operation unit 2908 is an operating member that receives an operation to close the bibliography list display area 2902.

[0493] Content display area 2910 is an area that displays an overview of each reference content (such as books).

[0494] In the illustrated example, the content display area 2910 displays the content title, content classification (book, guideline, etc.), bibliographic information such as the publication date of the content, and a citation of a portion of the content that served as the basis for generating the answer to the question. Compared to the example shown in Figure 29, the example in Figure 28 reduces the amount of citation information in the content display area 2910. As a result, in the reference list display area 2902, compared to the example in Figure 29, the example in Figure 28 displays the content in a way that makes it easier to grasp the overall picture of the references. For example, when a user refers to the answers displayed in the answer display area 2802 and considers which answers to refer to the original source such as a book, it becomes easier to narrow down the points by viewing the answers displayed in the answer display area 2802 while also checking the list of references in the reference list display area 2902 for each point. On the other hand, in the example in Figure 29, the amount of information on the citations for each piece of content is increased, making it easier to decide which pieces of content to examine in detail. Depending on the user's specified operation, each piece of content transitions to a content viewing screen as shown in Figure 17, etc.

[0495] <Ninth Embodiment> The ninth embodiment will be described as follows.

[0496] (1) Flowchart for creating a graph-type data structure that corresponds to abbreviated notations of laws and regulations. Depending on the various forms of notation of laws and regulations, the section in the content that refers to the law is identified, and the reference relationship with the information of the law that has a graph-type data structure is defined to generate a graph-type data structure.

[0497] (2) A flow in which a vector search is performed based on the question, information is obtained from the citation graph using the retrieved documents, and the information obtained from the question, retrieved documents, and graph database is provided to the LLM to generate an answer. Various content is searched according to the question, and relevant content is extracted by referring to a graph-type data structure for the content of the search results, which is used as the basis for generating the answer to the question.

[0498] In order to make appropriate judgments in legal practice, it is necessary to comprehensively consider a variety of information sources, including relevant laws and regulations, similar precedents, and explanations in specialized books and guidelines.

[0499] Therefore, this paper explains a technique for creating citation graphs that comprehensively link laws, precedents, books, and guidelines, thereby serving as a source of information to be referenced in legal practice.

[0500] Specifically, the following will be explained. • How to construct citation graphs that include diverse information sources such as laws and precedents, as well as books and guidelines. • How to handle question-answering tasks that require presenting the basis for the answer, simulating legal practice. • A method for presenting appropriate evidence in response to questions by combining large-scale language models with constructed citation graphs. A method for generating more favorable answers for legal professionals. <19 Data structure of the ninth embodiment> Figure 30 shows a graph-type data structure for each type of content: laws, precedents, books, and guidelines.

[0501] The example shown illustrates a graph-type data structure where each vertex represents a part (chunk) of the book's text, and edges are defined in a parent-child relationship between the vertices representing the book-identifying information and the vertices representing the chunks.

[0502] The illustrated example shows a graph-type data structure for laws and regulations, where information identifying the law (such as the name of the law, the date of amendment, and the date of enforcement) and the articles, paragraphs, and subparagraphs are each used as vertices, and edges are defined by parent-child relationships.

[0503] Although not shown in the diagram, the guidelines are similarly structured as a graph-type data structure, with each vertex representing information that identifies the guideline (guideline name, issuing body, publication date, etc.) and each part of the guideline text divided into sections, and edges defined in a parent-child relationship between the vertex representing the information that identifies the guideline and the vertex representing the chunk.

[0504] Although not shown in the diagram, the data structure for precedents is similar, with each vertex representing either information identifying the precedent (date of trial, type of trial, court information, trial number, etc.) or at least one of the summaries of the precedents or chunks of the judgment text, and edges defined in a parent-child relationship as described above.

[0505] Figure 31 shows the data structure of an abbreviation dictionary database 215 that defines rules for abbreviating laws and regulations. The abbreviation dictionary database 215 includes the following items: "Abbreviation Rule ID", "Official Name of Law or Regulation", "Abbreviation of Law or Regulation", "Abbreviation Rule", "Law or Regulation ID", and "Content ID".

[0506] The item "Abbreviation Rule ID" is an ID used to uniquely identify an abbreviation rule.

[0507] The item "Official Name of Law" contains information about the official name of the law.

[0508] The item "Abbreviated Name of Law" contains information on the abbreviated name of the law.

[0509] The "Abbreviation Rules" section provides detailed information on the rules and patterns for applying abbreviations.

[0510] The item "Legal ID" is an ID used to uniquely identify a legal or statutory law.

[0511] The "Content ID" field is an ID used to uniquely identify content to which abbreviation rules apply.

[0512] <20 Operation (Processing flow of the 9th embodiment)> Figure 32 shows the process flow for identifying sections in the content that refer to laws and regulations, depending on the various forms of legal notation, defining reference relationships with the information of the laws and regulations which have a graph-type data structure, and generating a graph-type data structure.

[0513] In step S3221, the data structure definition module 2046 of server 20 acquires information on various content to be viewed, such as book information from server 93 of the book viewing service, guideline information from server 94 of the information media service, and case law information from server 91 of the case law search service.

[0514] The data structure definition module 2046 integrates the book title and its chunks (each divided part of the content) into a graph structure with parent-child relationships. The data structure definition module 2046 extracts chunks from the book's text, detecting periods, commas, and line breaks, and each section reaching a certain number of characters. For example, in the book's text, it may extract a single chunk up to the point where a period, comma, or line break is detected after the sentence exceeds a certain number of characters.

[0515] Data structure definition module 2046 integrates the guideline title and its chunks (each part of the content divided into sections) into a graph structure with parent-child relationships.

[0516] The data structure definition module 2046 integrates information identifying a case (such as the case number and the date of the judgment) and information about the content of the judgment (for example, a summary of the case and chunks of the judgment text) into a parent-child relationship graph structure.

[0517] Server 20 manages, in its storage unit 202, information on content that is subject to viewing and is at least one of books or guidelines (such as the legal content database 212, data from the book viewing service server 93, and data from the information media service server 94), and information on laws and regulations that include provisions of laws and regulations (such as the data from the law search service server 92).

[0518] Server 20 manages content information and legal information as graph-type data structures in its storage unit 202. More specifically, Server 20 stores legal information in its storage unit 202 using a graph-type data structure in which the legal name, article, paragraph, and item are each vertices, and reference relationships are defined such that the vertices of the legal name are in a parent-child relationship with the vertices of the article, paragraph, and item in order.

[0519] Here, the server 20 stores information on laws and regulations in the storage unit 202, distinguishing them based on at least one of the amendment date or the enforcement date, and storing them in a graph-type data structure according to at least one of the respective amendment date or enforcement date.

[0520] Server 20 manages content information in its storage unit 202 as data in a graph-type data structure, where the title of the book or guideline is at one end, and each part of the book or guideline's content is at another end, with each part's vertex representing a parent-child relationship defined between the title of the book or guideline and the vertex representing the part.

[0521] Server 20 manages case law information in its memory unit 202, and manages it as data in a graph-type data structure in which the information identifying the case is at the top, and at least one of the case summary or each part obtained by dividing the judgment text of the case is at the top, with reference relationships defined as parent-child relationships between the vertices of the case summary or each part of the case and the vertices of the information identifying the case. Here, the information identifying the case includes the following elements. ·Court name: The name of the court that rendered the judicial decision. Abbreviated notations of the court name may also be used. For example, "Supreme Court (Saikō)", "Tokyo High Court (Tokyo Kōsai)" ·Type of judicial decision: Judgment (Han), Order (Mei), Ruling (Ketsu) ·Date of judicial decision: The date of the judicial decision. For example, "Hei 5.9.9", "Reiwa 2-nen 12-gatsu 10-nichi" ·Source: The published case law collection or magazine (abbreviated notations allowed), and the page number. For example, "Hanrei Jiho (Hanji)", "Hanrei Times (Hantai)" As described above, the information for identifying a judicial precedent is constituted by a regular expression including the character strings of the above elements. The data structure definition module 2046 extracts the character strings using the above abbreviations with a regular expression, and performs processing to restore abbreviations such as the court name, type of judicial decision, and source to their official names. This makes it possible to uniquely identify a judicial precedent, and to define the reference relationship from portions of books and guidelines to the judicial precedent.

[0522] In step S3222, the data structure definition module 2046 of the server 20 identifies portions that refer to laws and regulations in the content, based on the rules for referring to laws and regulations in the content, even if the notation is different from the official name of the law or regulation. The data structure definition module 2046 extracts the portions referring to laws and regulations within the content through text analysis.

[0523] (1) Extraction of laws and regulations by referring to the abbreviation notation dictionary database 215 that manages abbreviation notations The server 20 refers to dictionary information (abbreviation notation dictionary database 215) that holds the correspondence between the abbreviation notations of laws and regulations and their official names, and identifies locations where the abbreviation notation of a law or regulation appears in the content. The data structure definition module 2046 refers to the abbreviation notation dictionary database 215 to identify the official name of the law or regulation in the portion referring to the law or regulation in the content. In this way, the data structure definition module 2046 may identify the official name of a law or regulation using dictionary information corresponding to the type of abbreviation notation mode associated with the content.

[0524] The abbreviation dictionary database 215 manages abbreviated expressions such as the following examples. • Unique abbreviations used in books, etc. For example, the phrase "Article 124, Item 3 of the Company Regulations" may refer to "Article 124, Paragraph 1, Item 3 of the Companies Act Enforcement Regulations" (in books and other publications, the names of laws and regulations may be abbreviated in unique ways). For example, the phrase "Article 27-2, Paragraph 1, Item 3 of the Financial Instruments and Exchange Act" may refer to "Article 27-2, Paragraph 1, Item 3 of the Financial Instruments and Exchange Act." For example, the notation "Commercial Registration 54IV" may refer to "Article 54, Paragraph 4 of the Commercial Registration Act" (the notation of numbers differs between the article and the paragraph; for example, the article is written in Arabic numerals and the paragraph in Roman numerals). Here, the official name of the law may be obtained by acquiring information on the official name provided by the legal search service server 92 (for example, the official name of the law listed in the file name obtained from e-GOV legal search).

[0525] (2) Extraction of laws and regulations when the name of the law or regulation is abbreviated. The data structure definition module 2046 may extract portions of the content that do not contain the name of a law but do contain descriptions of articles, paragraphs, or subparagraphs, and then identify the official name of the law by referring to the title of the content or other parts of the content for the extracted portions.

[0526] In some books and guidelines, the full name of a law may not be stated in the text, and instead, it may simply be referred to as "the Law" or as "the Law," suggesting that it is following the name of a preceding law. In response to these omissions (e.g., "the Law," "the Law"), Data Structure Definition Module 2046 may infer the name of the law in the abbreviated notation by referring to the title of the book or guideline, the table of contents, and the text surrounding the abbreviated section to extract the name of the law. For example, even if the text is simply abbreviated as "the Law," the name of the law may be inferred by extracting the name of the law as shown in the book's title, the table of contents, and the name of the law written around the abbreviated section. In this way, when the name of a law is abbreviated, Data Structure Definition Module 2046 infers the correct law by referring to the context of the description and other parts of the content, and defines the article number and reference relationships of the law.

[0527] (3) Definition of reference relationships to multiple articles, paragraphs, and clauses The data structure definition module 2046 may extract portions of the content that contain specific phrases used with articles, paragraphs, or subparagraphs, and define reference relationships between those portions and multiple articles, paragraphs, or subparagraphs, depending on the specific phrases in those portions.

[0528] Data structure definition module 2046 detects when phrases such as "and" and "·" are included in the text of books and guidelines, along with abbreviations of laws and regulations. When these specific phrases are used, data structure definition module 2046 analyzes that multiple articles, paragraphs, or items are being referred to and defines the reference relationships between these multiple articles, paragraphs, or items in the graph.

[0529] For example, if the book contains the phrase "Articles 364 and 367 of the Civil Code," the data structure definition module 2046 will define the reference relationships for each of these articles, treating the portion (chunk) of the book containing that phrase as legal information, signifying a reference to "Article 364 of the Civil Code" and "Article 367 of the Civil Code."

[0530] Similarly, when the phrase "·" is used in a book or guideline (e.g., when it is written as "Article 210, Paragraph 1 · Article 214 of the Corporate Reorganization Act"), the data structure definition module 2046 defines a reference relationship to multiple articles, paragraphs, and items, just as when the phrase "and" is used in a book or guideline along with information about the law (e.g., it defines a reference relationship between "Article 210, Paragraph 1 of the Corporate Reorganization Act" and "Article 214 of the Corporate Reorganization Act")).

[0531] Furthermore, if a book or guideline uses expressions that specify the scope of an article (for example, "from," "~") along with legal information such as abbreviations of laws and regulations, the data structure definition module 2046 extracts the expressions that specify the scope of the article and defines the reference relationships between the main text of the book or guideline and each article within that scope (e.g., if the main text of the book states "Articles 258 to 260 of the Civil Rehabilitation Act," the module defines the reference relationships between the part of the book in which that statement is made and "Article 258 of the Civil Rehabilitation Act," "Article 259 of the Civil Rehabilitation Act," and "Article 260 of the Civil Rehabilitation Act"). (e.g., if the main text of the book states "Articles 138 to 141 of the Civil Code," the module defines the reference relationships between the part of the book in which that statement is made and "Article 138 of the Civil Code," "Article 139 of the Civil Code," "Article 140 of the Civil Code," and "Article 141 of the Civil Code").

[0532] As described above, the abbreviation dictionary database 215 of the memory unit 202 is configured to store dictionary information showing the correspondence between the abbreviated form of a law and its formal name. There are multiple types of abbreviated forms of laws. The server 20 is configured to store dictionary information for each type of abbreviated form of a law in the abbreviated dictionary database 215 of the memory unit 202. For each piece of content, it is configured to store information associated with identifying the type of abbreviated form of the law.

[0533] In step S3223, the data structure definition module 2046 of the server 20 parses the law names, articles, paragraphs, and subparagraphs identified within the content and identifies the corresponding vertices in the graph-type data structure of the law.

[0534] The data structure definition module 2046 refers to the publication date of the content (e.g., the publication date of a book, the publication date of a guideline) and identifies the vertex of the latest legal version (revision history) for that period in the graph-type data structure for the law.

[0535] In step S3224, the data structure definition module 2046 of server 20 defines the reference relationship between the identified legal information and the references in the content, adding a new edge to the graph-type data structure. In this way, the data structure definition module 2046 defines a reference relationship between the parts of the content that refer to the identified legal information and the information of the referred legal information. More specifically, the data structure definition module 2046 defines a reference relationship between the parts of the content that refer to the identified legal information and the vertices corresponding to the legal name, article, paragraph, or subparagraph of the referred legal information.

[0536] The data structure definition module 2046 may refer to information about the publication date of the content and define reference relationships between the portion of the content that refers to a specific law and the corresponding vertex of the law, article, paragraph, or subparagraph of the law, with respect to the most recent legal information available at the time of publication of the content.

[0537] Thus, the data structure definition module 2046 defines a reference relationship between the portion of the content that is identified as mentioning a law, even if the full name of the law is not explicitly stated, and the information of the law being mentioned.

[0538] The data structure definition module 2046 may define reference relationships between each vertex of the content that refers to a case and at least one of the vertices of each part of the case summary or judgment text.

[0539] In step S3225, the data structure definition module 2046 of the server 20 calculates a parameter indicating similarity based on the reference relationships defined between the contents and assigns it to the parent vertex of each content. The final graph-type data structure may be stored in a database in preparation for use in processes that respond to content searches, such as a question answering system.

[0540] The data structure definition module 2046 calculates a parameter indicating the similarity of each content based on the reference relationship information defined for each content. For example, by vectorizing other content (including parts of other content) that has a reference relationship defined for each content, a vector for the reference relationship of each content can be identified. For example, the text of parts of other content that have a reference relationship defined for the content may be aggregated and vectorized, or a vector based on the reference relationship of the content may be obtained by performing calculations on the vectors of parts of other content that have a reference relationship defined for the content. By comparing these vectors, the similarity of the reference relationships of each content (the degree to which they refer to similar other content) can be calculated. The data structure definition module 2046 associates the calculated similarity parameter with the content and stores it in the storage unit 202.

[0541] Here, server 20 may be a system that generates answers to questions. The question processing module 2044 may search for content based on the content of the question entered by the user, further extract content similar to the searched content based on a parameter indicating the similarity of the searched content, and generate an answer to the question using the extracted content.

[0542] For example, the question processing module 2044 may perform the following functions: extract content by referring to a graph-type data structure based on the content of the question entered by the user; generate a prompt that includes the content of the question and the extracted content, and includes instructions that refer to the extracted content in response to the question to generate an answer; and provide the generated prompt to an information processing system (large-scale language model service server 95) that performs language processing, thereby obtaining the answer generated in response to the prompt from the information processing system that performs language processing, and presenting the obtained answer to the user.

[0543] Figure 33 shows the process flow for generating answers to questions by searching for various types of content in response to a question, extracting relevant content by referring to a graph-type data structure for the search results, and using this as the basis for generating an answer to the question.

[0544] In step S3311, terminal 10 receives a question from the user. Terminal 10 sends the content of the received question to server 20. In step S3321, the question processing module 2044 of the server 20 vectorizes the content of the user's question that it has received.

[0545] Here, we will explain the data structure of the various data held by server 20 that are used in the processing.

[0546] The server 20 is configured to store data in a graph-type data structure 214 of the storage unit 202, in which reference relationships are defined between the parts that represent the content of each piece of content for multiple pieces of content that are to be viewed.

[0547] More specifically, the graph-type data structure 214 of the memory unit 202 is configured to store data in a graph-type data structure that holds multiple types of content as multiple contents, and defines reference relationships between different types of content. The multiple types of content are configured to store at least one of either book information including books on law or guideline information including legal guidelines, as well as information on laws and regulations and information on precedents.

[0548] In the graph-type data structure 214 of the memory unit 202, information is stored for each part (chunk) obtained by dividing the content of the book or guideline into multiple parts, for at least one of the book information or guideline information that is held as multiple types of content. In the graph-type data structure 214 of the memory unit 202, data is stored as graph-type data that defines the reference relationships between parts of the content of at least one of the book or guideline and parts of other content.

[0549] More specifically, the memory unit 202 holds data in a graph-type data structure, where the title of at least one of the books or guidelines is a vertex, and the content portion of at least one of the books or guidelines is a vertex, with a parent-child relationship defined between the title vertex and the content portion vertex.

[0550] More specifically, the memory unit 202 stores data in a graph-type data structure, with the name of the law as one vertex, and each of the articles, paragraphs, and clauses that make up the law as another vertex, and a parent-child relationship defined between the name of the law and each vertex in the order of articles, paragraphs, and clauses.

[0551] More specifically, the memory unit 202 holds data in a graph-type data structure, with information identifying the precedent as the vertex, and the content of the precedent as the vertex, with at least one of the summaries of the precedent or the content of the judgment divided into multiple parts as the vertex, defining a parent-child relationship between the vertex identifying the precedent and at least one of the vertices of the summaries of the precedent or the content of the judgment.

[0552] Furthermore, the data structure definition module 2046 defines reference relationships between parent vertices by aggregating the reference relationships of child vertices to parent vertices based on the reference relationships defined from one part of the content to other parts of the content, and stores them as a graph-type data structure 214.

[0553] In step S3322, the question processing module 2044 of the server 20 vectorizes the content of the question and searches for the part corresponding to the content of the question by comparing the vector of the question content with the vectors of each part of the content. The question processing module 2044 compares the vectorized question with the vectors of each part (chunk) of the content such as books and guidelines in the database to search for chunks related to the question (searches for chunks with similar vectors).

[0554] Thus, the question processing module 2044 searches multiple types of content based on the content of the question. More specifically, the question processing module 2044 searches multiple types of content. Based on the content of the question, the question processing module 2044 searches at least one of the multiple types of content, namely information from books or information from guidelines.

[0555] In step S3323, the question processing module 2044 of the server 20 extracts relevant content (first content) based on the ranking of the search result chunks.

[0556] More specifically, the question processing module 2044 searches for the relevant section of a book or guideline based on the content of the question, and determines the ranking of the content based on the search ranking of each section of the content. For example, the search ranking of the content can be determined by aggregating the search rankings of each chunk of content and weighting those with higher rankings, calculating the average of the search rankings of the chunks for each piece of content, or extracting a certain number of top-ranking chunks from the search rankings and aggregating the number of chunk search results included in each piece of content. For example, content with a large number of high-ranking chunks based on vector comparisons may be given a higher search ranking.

[0557] In step S3324, the query processing module 2044 of the server 20 selects a second content that has a defined reference relationship with the first content, based on the data in a graph-type data structure, for one or more first content items that are the search results. The query processing module 2044 then references the graph-type data structure 214 for the first content and extracts other related content (second content).

[0558] As described above, the data structure definition module 2046 extracts multiple content items as a group if the reference relationships defined from one content item to another are similar. The question processing module 2044 may select the other content items extracted as a group due to similar reference relationships with respect to the first content item as the second content item. In this way, the question processing module 2044 may extract the second content item related to the first content item by finding that the reference relationships of the graph-type data structure are similar for the first content item (by comparing the vectorized reference relationships defined for the content items with each other, thereby extracting content items with similar reference relationships).

[0559] In step S3325, the question processing module 2044 of the server 20 generates a prompt that includes a question statement, a first content, and a second content, and formats it as input data to the LLM so that the prompt generates an answer to the question.

[0560] Thus, the question processing module 2044 generates a prompt that includes the content of the question, the retrieved first content, and the selected second content, and includes instructions that refer to the first content and the second content to generate an answer to the content of the question.

[0561] In step S3326, the question processing module 2044 of server 20 provides the generated prompt to the LLM (Large-Scale Language Model Service server 95) to generate an answer. In this way, the question processing module 2044 provides the generated prompt to the information processing system that performs language processing, thereby obtaining the answer output by the information processing system that performs language processing.

[0562] In step S3327, the question processing module 2044 of server 20 retrieves the answer output from LLM (Large-Scale Language Model Service server 95) and presents it to the user.

[0563] In step S3312, terminal 10 presents the acquired response to the user.

[0564] <Variation> The embodiments described above may also be combined in various ways.

[0565] Furthermore, each embodiment described above may be modified as follows.

[0566] (1) Sources of information on which answers to questions are based In the above description of the embodiment, a graph-type data structure is defined for laws, precedents, legal books, and legal guidelines, and an example is described in which a user's question is answered by searching using this graph-type data structure and outputting a response that includes the search results.

[0567] In addition, a graph-type data structure may be defined in the same manner as above, based on information that is not necessarily made public to an unspecified number of people, such as information held on the operator's server 96. Then, in response to a question, the same type of information as that held on the operator's server 96 may be searched based on the graph-type data structure, and the answer to the question may be output. For example, if the operator has accumulated results of judgments made within the company regarding specific cases based on legal issues (for example, issues related to the Premiums and Representations Act, etc.), a summary of the internal cases of the operator may be presented, similar to presenting court cases such as those shown in Figure 12 above.

[0568] For example, the information held by a business operator may include internal regulations, manuals, incident reports, documents such as Q&A and meeting minutes stored in document tools, and internal user inquiries in internal communication tools (for example, a system where users are assigned to groups such as channels, enabling the sending and receiving of messages between users) (for example, user responses to inquiries to accounting, legal, etc.). A graph-type data structure may be defined for this data using information from the company's organizational chart, internal terminology, general terminology, etc. As described in the above embodiment, answers to user questions can be provided by having the server 95 of the large-scale language model service generate answers by referring to the graph-type data structure. For example, these data files may be stored in a specific folder, and the server 95 of the large-scale language model service may generate answers by referring to the information in that folder.

[0569] In addition to the above, a graph-type data structure similar to the above can be defined based on information that may be made public to an unspecified number of people, such as on SNS server 97, and this structure can be referenced when answering user questions. For example, the content of posts made by a specific person's account may be cited as the basis for answering a question.

[0570] (2) How to generate a prompt As described in each of the embodiments above, it has not been easy to create prompts to provide to a large-scale language model in a system that generates answers to questions.

[0571] Therefore, prompts may be created using the methods described in each of the above embodiments.

[0572] <21 Operation (Processing flow of the 10th embodiment)> The tenth embodiment describes a technology that supports the appropriate revision of internal company regulations. It makes it easy to find the clauses in the internal regulations that need to be changed due to legal revisions or changes in case law. In addition, it clarifies whether the internal regulations are legally required or unique to the organization, in order to clarify the relationship between internal regulations and laws.

[0573] Figure 34 shows the process flow for generating a graph-type database in which reference relationships are defined between each clause of the company regulations and a graph-type data structure that associates them with laws, precedents, books, and guidelines.

[0574] The following processing flow links each clause of internal regulations with external information such as books and guidelines, enabling a system that can easily extract and present the affected sections of internal regulations when legal revisions or changes in case law occur.

[0575] For example, when a user inputs new company regulations data from a terminal, the server analyzes the content and matches it with pre-stored information on laws, precedents, and related guidelines. The server automatically defines links between nodes, which are then reflected in a graph-type database. This allows for the extraction of relevant updates to the regulations using a similar processing flow when regulations are changed later, potentially improving the efficiency of revising regulations in accordance with legal revisions and changes in precedents.

[0576] In step S3411, terminal 10 receives input from the user regarding the organization's internal regulations (text of each clause, accompanying metadata, etc.). Terminal 10 then transmits the entered internal regulations data to server 20 via the network.

[0577] Server 20 associates internal regulations data with laws, precedents, books, and guidelines as follows: As described above, the storage unit 202 associates laws with precedents and books with guidelines, and further associates internal regulations with laws and precedents as explained below.

[0578] As described above, the memory unit 202 is configured to store each article of the law as data in a graph-type data structure.

[0579] Furthermore, as described above, the memory unit 202 is configured to store at least one of the following: information on books including books on law or information on guidelines including legal guidelines, information on laws and regulations, and information on precedents, and to store data in a graph-type data structure that defines reference relationships between the contents of these contents.

[0580] In step S3421, the data structure definition module 2046 of the server 20 receives input of information about the internal regulations that define the organization's internal rules. The data structure definition module 2046 associates the received internal regulations data with the organization's information and stores it in the storage unit 202. The data structure definition module 2046 analyzes and extracts each clause from the internal regulations data and recognizes them as nodes. For example, as explained in Figure 30, the title of the internal regulations may be defined as the parent node, and each clause of the internal regulations may be defined as a child node to show the parent-child relationship.

[0581] In step S3422, the data structure definition module 2046 of server 20 retrieves information that corresponds to the internal regulations from external sources such as laws, precedents, books, and guidelines. The data structure definition module 2046 analyzes the content, keywords, and semantic similarities between the retrieved external information and each clause of the internal regulations, and identifies the reference relationships that link them together.

[0582] More specifically, the data structure definition module 2046 retrieves information that refers to internal regulations from sources that refer to internal regulations. For example, a book explaining how to develop internal regulations might describe the themes of the internal regulations, example sentences, the laws and regulations that form the background of the internal regulations, and the articles of those laws and regulations. The sources refer to the laws and regulations that form the basis of the internal regulations. The data structure definition module 2046, with respect to the information on the organization's internal regulations received as input in step S3421, refers to the information from the sources to identify the laws and regulations relevant to the content of each clause of the organization's internal regulations.

[0583] As described above, the types of data held in the memory unit 202 include books, guidelines, laws, and case precedents. Thus, the memory unit 202 has a graph-type database in which reference relationships are defined between the contents of books, guidelines, laws, and case precedents. Reference relationships are defined between the information of the organization's internal regulations and the information of books and guidelines. Reference relationships are also defined between the information of books and guidelines and the information of laws and case precedents.

[0584] Data Structure Definition Module 2046 retrieves information from at least one legal book or legal guideline that refers to the internal regulations as a source of information. Data Structure Definition Module 2046 defines reference relationships between each clause of the organization's internal regulations and at least one of the legal books or legal guidelines as a source of information, as follows:

[0585] In step S3423, the data structure definition module 2046 of server 20 defines reference relationships (links) as edges between the clauses of each internal regulation and the respective content nodes of laws, precedents, books, and guidelines, based on the associated results, and constructs a graph-type data structure. The data structure definition module 2046 stores the generated graph-type data structure as a graph-type database, making it available for subsequent search processing and answer generation processing.

[0586] In this way, the data structure definition module 2046 associates the content of each clause of the laws and regulations and the organization's internal rules identified in step S3422 and stores them in the storage unit 202.

[0587] The data structure definition module 2046 maintains the information of the organization's internal regulations and the provisions of laws and regulations as a graph-type data structure by identifying the information of the provisions of laws and regulations that define reference relationships with each provision of the organization's internal regulations and each provision of laws and regulations, for each provision of the laws and regulations that defines reference relationships with each provision of the laws and regulations that defines reference relationships with each provision of the laws and regulations that defines reference relationships with each provision of the laws and regulations that defined

[0588] In step S3424, the data structure definition module 2046 of the server 20 sends the results (summary of the graph structure and association information) generated by associating the organization's internal regulations information with a graph-type data structure, as needed, to the terminal 10, enabling user confirmation and feedback.

[0589] Figure 35 illustrates the process of how each clause of the company's internal regulations is linked to laws and precedents, and how information on legal revisions and changes in precedents is obtained to extract the affected clauses of the internal regulations.

[0590] The following processing flow allows for the extraction of relevant internal regulations in conjunction with legal revisions and changes in case law, enabling a system that allows users to quickly adapt to revisions in internal regulations.

[0591] In step S3511, terminal 10 displays a screen on the dashboard or notification screen for the user to check information on legal revisions and changes in precedents. In this way, terminal 10 prompts the user to check the impact of changes in laws and precedents. Terminal 10 receives input from the user for a request to check the impact (e.g., start the check, specify the target company regulations, etc.) and sends the request to server 20.

[0592] Server 20 processes the process to prompt the organization to update its internal regulations in response to legal revisions and changes in case law, as follows: The organization's internal regulations are linked to laws, case law, books, and guidelines. The server extracts the internal regulations that need to be changed due to legal revisions and changes in case law. The server prompts the user to review and change those internal regulations.

[0593] The memory unit 202 stores at least one of the following: information on books, including books on law; information on guidelines, including legal guidelines; information on laws and regulations; information on precedents; and information on the organization's internal regulations. The memory unit 202 is configured to store data in a graph-type data structure in which reference relationships are defined between at least the information on each clause of the organization's internal regulations and at least one of the information on each clause of laws and regulations or the information on precedents.

[0594] In step S3521, the data structure definition module 2046 of server 20 periodically or on demand obtains the latest change information from external legal amendment information provision systems and case law change information provision systems (e.g., server 91 for the case law search service, server 92 for the legal law search service) and data sources. For example, it may obtain articles about legal amendments and articles about case law changes as information provided by server 94 of the information media service. The data structure definition module 2046 matches the obtained legal amendment information and case law change information with the legal law and case law information in the graph database to identify the items subject to change (e.g., legal name, legal clause).

[0595] Thus, the data structure definition module 2046 obtains at least one of either information on legal amendments or information on changes in case law. Based on the obtained information on legal amendments or changes in case law, the data structure definition module 2046 identifies at least one of either the relevant legal provisions or relevant case law information.

[0596] In step S3522, the data structure definition module 2046 of server 20 extracts each clause of the internal regulations associated with the identified laws and precedents from the graph-type data structure and analyzes the scope of impact. Based on the extraction results, the data structure definition module 2046 generates and organizes a list of affected internal regulations clauses and necessary proposed amendments (recommended operation information).

[0597] Thus, the data structure definition module 2046 extracts the relevant legal provisions or relevant case law information and the provisions of the organization's internal regulations that define the reference relationships identified in step S3521.

[0598] Server 20 may, in response to legal revisions or changes in case law, present a draft amendment to the internal regulations.

[0599] The data structure definition module 2046 generates proposed amendments to the internal regulations clauses of the extracted organization, based on at least one of the information on legal amendments or case law changes obtained in step S3521, and presents the generated proposed amendments to the internal regulations clauses to the user. For example, server 20 obtains articles explaining legal amendments and articles explaining case law changes from the information media service server 94. These articles may include not only the content of legal amendments and case law changes, but also proposed amendments to the internal regulations clauses. The data structure definition module 2046 may also obtain these explanatory articles on legal amendments and case law changes, along with information on the affected internal regulations, to generate proposed amendments to the internal regulations clauses. For example, the data structure definition module 2046 may generate a prompt that includes the clauses of the organization's internal regulations that are affected by legal amendments or changes in case law, and information on explanatory articles on legal amendments or changes in case law, and generate a prompt that generates a draft amendment to the clauses of the internal regulations, taking these explanatory articles on legal amendments or changes in case law into consideration, and send the generated prompt to the server 95 of the large-scale language model service, thereby obtaining the result of the draft amendment from the server 95 of the large-scale language model service.

[0600] In step S3523, the data structure definition module 2046 of the server 20 sends back the generated information on the affected areas and the proposed amendments for the clauses of the internal regulations that are affected by legal revisions or changes in case law to the terminal 10, making them available for the user to review.

[0601] Thus, the data structure definition module 2046 presents the information on the organization's internal regulations extracted in step S3522 to the user as internal regulations that are subject to legal revisions or changes in case law.

[0602] Server 20 may display both information on legal revisions and precedents, as well as the affected internal company regulations.

[0603] The data structure definition module 2046 may display on the screen at least one of the information on the clauses of the organization's internal regulations extracted in step S3522, along with the information on legal amendments or changes in case law obtained in the same step.

[0604] Furthermore, server 20 may update the database of internal regulations once the user approves the revised draft. In this case, it may also maintain separate databases for each version of the internal regulations.

[0605] Thus, the data structure definition module 2046 may accept an operation from the user to approve a proposed revision of the internal regulations clause, and in response to accepting the operation to approve the revision, update the information in the organization's internal regulations based on the revised regulations.

[0606] In step S3512, terminal 10 allows the user to view the returned information on the screen, perform actions such as approving, correcting, or providing feedback on the proposed amendments, and complete the final update process.

[0607] <Modified form of the 10th embodiment> In the above embodiment, an example of defining reference relationships between each clause of the internal regulations and information on laws and precedents was described. This allows for the determination of whether each clause of the internal regulations is required by law or precedent (a reference relationship is defined between the clause of the internal regulations and the information on laws and precedents), or whether it is an internal regulation established independently by the organization rather than by law or precedent (a clause of the internal regulations for which a reference relationship with information on laws and precedents is not defined). Server 20 may present the information on the internal regulations to the user in a manner that allows for visual confirmation of whether a reference relationship is defined between each clause of the internal regulations and information on laws and precedents. More specifically, Server 20 may display each clause of the internal regulations differently depending on whether a reference relationship is defined between each clause of the internal regulations and information on laws and precedents in a graph-type data structure. This allows for confirmation of whether revision is necessary because it is required by law or whether it is an internal regulation unique to the organization, before revising the regulations.

[0608] <22 Operation (Processing flow of the 11th embodiment)> The 11th embodiment describes an example of a system for generating answers to questions in a graph-type database where reference relationships are defined for each content item of an organization's internal regulations, as described in the 10th embodiment, and each clause of the internal regulations is defined as a node. The 11th embodiment describes a technology that generates answers to questions while also generating answers in accordance with the internal regulations, including information from the internal regulations as the basis for generating the answers.

[0609] Figure 36 shows the process flow for generating answers to questions based on laws, precedents, books, guidelines, and internal company regulations.

[0610] The following processing flow demonstrates an example where the entire process, from user input to server-side natural language processing, vector search, extraction of relevant information using a graph database, and finally response generation using LLM, is carried out consistently. This enables a system that ultimately presents the response to the terminal along with supporting information, including company regulations.

[0611] In step S3611, terminal 10 accepts input from the user for various questions, such as business-related questions (e.g., inquiries about the company's risks or standards). Terminal 10 sends the input question text to server 20.

[0612] Server 20 has a graph-type data structure in which internal regulations, laws, precedents, books, and guidelines are linked. When Server 20 receives a question from a user, it generates an answer based on books, guidelines, laws, precedents, and internal regulations. The following describes this process.

[0613] The memory unit 202 stores at least one of the following: information on books, including books on law; information on guidelines, including legal guidelines; information on laws and regulations; information on precedents; and information on the organization's internal regulations. The memory unit 202 is configured to store data in a graph-type data structure in which reference relationships are defined between the information on each clause of the organization's internal regulations and at least one of the information on each clause of laws and regulations, information on precedents, information on books, or information on guidelines.

[0614] In step S3621, the question processing module 2044 of the server 20 performs natural language analysis on the received question sentence, extracts keywords, and performs vectorization. In this way, the question processing module 2044 converts the context of the question into a numerical representation, preparing it for subsequent searches.

[0615] In this way, the question processing module 2044 receives a question from the user.

[0616] In step S3622, the question processing module 2044 of the server 20 uses vector search to search for and extract highly relevant documents by comparing them with vector information of books and guidelines that have been registered in advance.

[0617] In step S3623, the question processing module 2044 of server 20 identifies information on laws and precedents associated with books and guidelines by utilizing the reference relationships between nodes in the graph database.

[0618] Thus, the question processing module 2044 identifies at least one of the following information related to the question: legal information, case law information, book information, or guideline information, based on the content of the question it receives, by referring to data in a graph-type data structure.

[0619] More specifically, the question processing module 2044 searches for books and guidelines based on the content of the question, and identifies the laws and precedents that define the reference relationships between the search results (books and guidelines). The question processing module 2044 then extracts internal company regulations associated with the identified laws and precedents.

[0620] Thus, the question processing module 2044 searches for at least one of the book information or guideline information based on the content of the question. The question processing module 2044 identifies at least one of the searched book information or guideline information and at least one of the law information or case law information for which a reference relationship is defined in the data of the graph-type data structure.

[0621] Thus, the question processing module 2044 performs a vector search of books and guidelines based on the content of the question, and further identifies laws, precedents, and internal regulations using a graph-type data structure. The question processing module 2044 may also vectorize the content of the question and search for at least one of the book information or guideline information by comparing the vector of the question content with at least one of the vectors of the book information or guideline information.

[0622] In step S3624, the question processing module 2044 of server 20 extracts relevant internal regulations information (including each clause and scope of impact information) based on the link information on the graph, in conjunction with the identified laws and precedent information.

[0623] Thus, the question processing module 2044 extracts information about the organization's internal regulations that define reference relationships to the information identified in step S3623. The question processing module 2044 extracts information about the organization's internal regulations that define reference relationships to at least one of the legal information or case law information identified in step S3623, in the data of the graph-type data structure.

[0624] In step S3625, the question processing module 2044 of server 20 integrates the question text, extracted books, guidelines, laws, precedents, and internal regulations to generate a prompt to be given to the LLM (Large-Scale Language Modeling Service server 95).

[0625] The question processing module 2044 generates a prompt that includes the content of the question, at least one of the legal information, case law information, book information, or guideline information identified in step S3623, and the organization's internal regulations information extracted in step S3624. The prompt includes instructions to generate an answer to the question content by referring to at least one of the legal information, case law information, book information, or guideline information and the extracted organization's internal regulations information.

[0626] In step S3626, the question processing module 2044 of server 20 inputs the generated prompt to the LLM (Large-Scale Language Model Service server 95) and performs language processing to generate an answer that matches the content of the inquiry and the basis for that answer (including internal regulations information).

[0627] In this way, the question processing module 2044 uses at least the information on the organization's internal regulations extracted in step S3624 to cause the information processing system that performs language processing to generate an answer. The question processing module 2044 provides the generated prompt to the information processing system that performs language processing, thereby causing the information processing system that performs language processing to generate an answer corresponding to the prompt.

[0628] In step S3627, the question processing module 2044 of server 20 receives a response from LLM (Large-Scale Language Model Service server 95), formats the content, performs verification, and compiles the entire response into a final dataset.

[0629] The question processing module 2044 presents the user with the answer generated in step S3626, which includes information on the organization's internal regulations that form the basis of the answer.

[0630] In step S3612, terminal 10 displays the final response (response text and supporting information) sent from the server device on the user's screen, making it available for the user to review.

[0631] <Variation of the 11th embodiment> Server 20 may collect information on cases where changes to internal regulations were detrimental and cases where they were not by referring to case law information regarding changes to internal regulations. Server 20 may accept input questions about changes to internal regulations and generate prompts that include information on the content of the changes to internal regulations related to the question, cases in case law showing detrimental and non-detrimental changes, and instructions to evaluate whether the content of the changes to internal regulations related to the question constitutes a detrimental change. Server 20 may provide the generated prompts to Server 95 of the large-scale language model service, receive a response from Server 95 of the large-scale language model service evaluating the possibility that the changes to internal regulations related to the question constitute a detrimental change, and present that response to the user.

[0632] Furthermore, when generating an answer to a question, the server 20 may generate the answer in accordance with internal regulations, distinguishing whether the internal regulations are legally required (i.e., whether reference relationships are defined between the clauses of the internal regulations and information on laws and precedents) or whether they are unique and not legally required (i.e., whether reference relationships are defined between the clauses of the internal regulations and information on laws and precedents), and then presenting the user with information on the internal regulations that formed the basis of the answer.

[0633] <23 Operation (Processing flow of the 12th embodiment)> The twelfth embodiment describes an example of generating questions in a system that collects internal posts and meeting audio, or accepts posts regarding legal violations or harassment, and generates answers to those questions. In this way, the technology for evaluating whether internal posts and meeting audio constitute legal violations or harassment is described.

[0634] Figure 37 shows the process flow for detecting questions from internal company posts and conversations and generating answers.

[0635] The following processing flow can enable a system that automatically detects questions and concerns from transcribed data of internal company posts and conversations and generates appropriate answers. For example, it can detect potentially illegal messages in real time from internal company posts and voice conversations, generate questions and answers, and provide timely feedback on potential legal violations, thereby establishing a system to prevent legal violations.

[0636] In step S3711, terminal 10 receives messages from the user via chat tools, postings on a conferencing system, and voice conversations (sometimes including questions or doubts in a natural way). In the case of voice conversations, speech recognition processing is performed in real time or after recording to convert the spoken content into text data.

[0637] In step S3712, terminal 10 sends the text-based post / conversation data to server 20 via the network.

[0638] The question processing module 2044 on server 20 detects the content of posts and voice conversations within the organization, generates questions, obtains answers to the generated questions, and presents them to the user. This is explained below.

[0639] The memory unit 202 is configured to store at least one of the following: information on books including books on law or information on guidelines including legal guidelines, information on laws and regulations, and information on precedents. It also stores data in a graph-type data structure in which reference relationships are defined between the information on each article of the law, the information on precedents, the information on books, or the information on guidelines.

[0640] The memory unit 202 is configured to store information on the organization's internal regulations and to store data in a graph-type data structure in which reference relationships are defined between each clause of the organization's internal regulations and at least one of the following: information on the clauses of laws and regulations, information on precedents, information on books, or information on guidelines.

[0641] In step S3721, the question processing module 2044 of server 20 performs preprocessing on the received text data, such as formatting standardization and noise reduction, to prepare it for analysis. Using natural language processing and keyword extraction techniques, it detects statements from the preprocessed data that contain the intent of the question or concerns about legal violations.

[0642] Thus, the question processing module 2044 obtains at least one of the following: information about communication between users, or information transmitted externally in business activities.

[0643] (1) Automatically detect user statements and notify of potential legal violations. The question processing module 2044 may acquire at least one of the following: information about communication between users, or information transmitted externally in business activities, without requiring the user to perform an operation to specify this information.

[0644] (2) Sources of information to determine whether a violation of the law has occurred More specifically, the following could be sources of information for determining whether a violation of the law has occurred: • Information on communication between users: This could include messaging tools, email, and document creation tools that allow each user within the organization to post and view information. • Information disseminated externally in business activities: This may include materials used in public relations and sales, catalogs, banners, landing page images, and text. The question processing module 2044 may acquire information about communication between users by acquiring data accumulated in at least one of the following: a messaging tool that makes messages posted by users viewable by other users, email, transcript data of voice conversations between users, or a document tool that allows users to set shared access permissions.

[0645] The question processing module 2044 acquires information to be disseminated externally in business activities, and this includes acquiring data created by at least one of the public relations department or the sales department, which may include data such as information from sales catalogs, banners on web pages, and images or text on landing pages.

[0646] (3) Selection of subjects for determination of legal violations The Question Processing Module 2044 analyzes user-generated text posts and voice conversation logs to extract data for evaluation from a vast amount of text. Based on case studies, the Question Processing Module 2044 identifies sections similar to cases of legal violations and compliance violations. For example, databases it may refer to could include violation cases of other companies published by administrative organizations, administrative sanction cases (consumer protection), and books (such as books explaining specific laws like the Subcontracting Act).

[0647] The question processing module 2044 acquires information on communication between users, specifically obtaining at least one of the following: user-submitted text messages or logs of voice conversations. The question processing module 2044 may also refer to data from a collection of legal violation cases and detect sections of the information acquired in step S3721 that are similar to cases included in the collection of legal violation cases.

[0648] More specifically, the question processing module 2044 may perform the above detection based on the degree of agreement with a collection of examples of statements, such as harassment.

[0649] The question processing module 2044 may refer to data from a collection of legal violation cases and detect sections of the information obtained in step S3721 that are similar to cases included in the collection of legal violation cases.

[0650] More specifically, the question processing module 2044 may perform the above detection based on the degree to which the content of the business request matches a collection of examples of legal violations (for example, violations of the Subcontracting Act) based on data such as application forms and email content exchanged with users outside the organization.

[0651] The question processing module 2044 may refer to data from a collection of legal violation cases and detect sections of the information obtained in step S3721 that are similar to cases included in the collection of cases concerning requests for work that violate laws and regulations.

[0652] The question processing module 2044 may read text from images in catalogs, banners, and landing pages to detect whether there are any legal violations or whether these pages conform to the standards. The question processing module 2044 may also refer to at least one of the data of examples of legally violating expressions or information on design standards to detect at least one of the parts of the information obtained in step S3721 that are similar to examples included in the collection of legally violating expressions, or parts that do not conform to the design standards.

[0653] The question processing module 2044 may extract parts of conversations between users that indicate an intention to ask a question. For example, it may extract parts of a conversation that indicate an intention to confirm whether something is a violation of the law, such as "Is this okay (harassment, potential legal violation, etc.)?" The question processing module 2044 obtains information on communication between users, obtaining at least one of the following: text posts between users or logs of voice conversations. The question processing module 2044 may also detect parts of the information obtained in step S3721 that contain statements indicating an intention to evaluate the possibility of a legal violation.

[0654] Through the above process, the amount of data to be processed by the information processing system that performs language processing (for example, the server 95 of the large-scale language model service) can be reduced to an extracted portion rather than the entire text of the transcribed posts and conversation audio. This reduces the processing load on the information processing system that performs language processing. Furthermore, it is expected that the time it takes to receive a response from the information processing system that performs language processing will be shortened, and it may be possible to respond in real time while posts and conversation audio are being made.

[0655] In step S3722, the question processing module 2044 of the server 20 extracts supporting information and related documents such as books, laws, precedents, guidelines, and internal regulations from the graph database based on the detected question and the detected user's inquiry, using vector search and reference relationships in the graph data structure.

[0656] In this way, the question processing module 2044 searches each data point in the graph-type data structure based on the information obtained in step S3721.

[0657] For example, the question processing module 2044 may first search for books and guidelines, extract laws and precedents for which reference relationships are defined with those books and guidelines, and then generate an answer to the question. Based on the content of the question, the question processing module 2044 searches for information on books or at least one of the guidelines. The question processing module 2044 identifies at least one of the laws or precedents for which reference relationships are defined in the data of a graph-type data structure, along with at least one of the searched book information or guideline information.

[0658] If internal regulations also have reference relationships defined in the graph-type data structure, the question processing module 2044 may also consider the internal regulations when generating an answer. Based on the content of the question, the question processing module 2044 searches for at least one of the book information or guideline information. The question processing module 2044 identifies at least one of the statutory information or case law information for which a reference relationship is defined in the data of the graph-type data structure, along with at least one of the retrieved book information or guideline information. The question processing module 2044 extracts information from the organization's internal regulations that define reference relationships to at least one of the retrieved book information or guideline information, and at least one of the identified statutory information or case law information.

[0659] In step S3723, the question processing module 2044 of server 20 combines the search results (related documents, citation graph information, etc.) with the original question content and automatically generates a prompt for input to the LLM (Language Processing System; server 95 of the Large-Scale Language Model Service).

[0660] The question processing module 2044 generates a prompt that includes at least some of the information obtained in step S3721, and includes instructions to evaluate the possibility of a legal violation.

[0661] The question processing module 2044 may generate a prompt that includes at least one of the information from the searched book or guideline, and at least one of the information from laws or precedents identified based on reference relationships from the data in a graph-type data structure, and which includes instructions to generate an answer by referring to this information, and to assess the likelihood of a legal violation.

[0662] The question processing module 2044 may generate a prompt that includes the above-mentioned instruction, which includes at least one of the information from the searched book or guideline, at least one of the information from the laws or precedents identified based on reference relationships from the data in the graph-type data structure, and the extracted information from the organization's internal regulations.

[0663] The question processing module 2044 may generate a prompt by including the similar parts detected in step S2731 in the prompt.

[0664] In step S3724, the question processing module 2044 of server 20 submits the generated prompt to the LLM (Large-Scale Language Model Service server 95) to generate an answer that includes an response to the question and an assessment of the possibility of legal or compliance violations.

[0665] In this way, the question processing module 2044 receives the response output by the information processing system that performs language processing by providing the prompt generated in step S3723 to the information processing system that performs language processing.

[0666] In step S3725, the question processing module 2044 of the server 20 adds information on the books, laws, precedents, guidelines, and internal regulations that form the basis of the answer to the generated answer, formats it, and compiles it into a format that can be presented to the user. The question processing module 2044 then sends the completed answer and the supporting information back to the terminal 10 to notify and provide feedback to the user.

[0667] In this way, the question processing module 2044 presents the user with an answer that includes the results of an evaluation of the likelihood of a legal violation, which has been output by the information processing system that performs language processing.

[0668] Here, the question processing module 2044 presents the user with an answer that includes information on the books, guidelines, laws, and precedents that formed the basis for generating the answer. The question processing module 2044 may also present the user with an answer that includes at least one of the information from books or guidelines, and at least one of the information from laws or precedents, that formed the basis for generating the answer by the information processing system that performs language processing.

[0669] Question processing module 2044 may present the user with an answer that includes information from books, guidelines, laws, and precedents that form the basis for generating the answer, as well as internal company regulations. Alternatively, question processing module 2044 may arbitrarily select and present information from books, guidelines, laws, precedents, and internal company regulations to the user as the basis for generating the answer. For example, question processing module 2044 may not present information from books, guidelines, laws, and precedents as the basis for generating the answer, but instead present information from internal company regulations. Furthermore, depending on the user asking the question, the module may determine a combination of this information from books, guidelines, laws, precedents, and internal company regulations and present it as the basis for generating the answer. For example, a user asking a question may place importance on internal company regulations as the basis for generating the answer. For example, question processing module 2044 may accept an operation from the user to specify at least one of books, guidelines, laws, precedents, and internal company regulations as the basis for generating the answer, and either generate an answer according to the user's specification, or present the user with content that forms the basis for generating the answer according to the user's specification.

[0670] Thus, the question processing module 2044 may present the user with the answer including at least one of the following: information from books or guidelines, information from laws or precedents, and information from the organization's internal regulations, which formed the basis for the information processing system that performs language processing to generate the answer.

[0671] In step S3713, terminal 10 displays the notified answer and supporting information to the user. This allows the user to review the answer and the content supporting it, and to ask additional questions or provide feedback as needed.

[0672] <Modified form of the 12th embodiment> As described above, in a system that generates answers to questions, we have explained a technology for evaluating whether user posts and user conversation audio constitute legal violations or harassment. If, through these processes, the server 20 presents an answer to the user indicating that the user's posts, etc., may constitute a legal violation, the server 20 may accept an anonymous request from the user to consult with a person in a specific department within the organization or an external expert. In response to such a request, the server 20 may send a message containing the user's posts, etc. that were the subject of the question, and the result of the answer generated by the server 20, to the person in the specific department or the external expert.

[0673] <24 Operation (Processing flow of the 13th embodiment)> In the 13th embodiment, a technology is described for ranking multiple types of content, such as books, based on reference relationships in a graph-type data structure. Based on the results of these rankings, it is possible to set the priority of literature when extracting supporting literature in response to a question, and to control the display order when presenting content to the user.

[0674] Figure 38 shows the process flow for analyzing the reference relationships of a graph-type data structure, determining the priority order for presenting multiple types of content that formed the basis for generating answers to questions, and then presenting that content.

[0675] In the following processing flow, the terminal receives input from the user, and the server extracts content corresponding to the question, analyzes the reference relationships between content in a graph-type data structure, determines the priority of the content, and generates answers and integrates the results using LLM. This allows the user to easily determine which supporting content should be referenced in detail along with their answer.

[0676] Furthermore, the following processing flow makes it even easier for users to select and filter through a large amount of content, by displaying content with many citations, systematic textbooks, or types of literature that are considered important in specific fields (such as guidelines) at the top of the list.

[0677] In step S3811, terminal 10 receives a question input from the user and sends the content of the question to server 20.

[0678] The question processing module 2044 on server 20 determines the ranking of content by considering the reference relationships of a graph-type data structure. In a system that generates answers to questions using LLM, when presenting supporting literature, it displays it in order based on the reference relationships of the graph-type data structure. This is explained below.

[0679] The memory unit 202 is configured to store data in a graph-type data structure that defines reference relationships between multiple types of content.

[0680] The memory unit 202 is configured to store multiple types of content, including at least one of either book information (including books on law) or guideline information (including guidelines on law), information on laws and regulations, and information on precedents, and to store data in a graph-type data structure in which reference relationships are defined between at least one of the information on each article of the law, the information on precedents, the information on books, or the information on guidelines.

[0681] The memory unit 202 is configured to store information on the organization's internal regulations and to store data in a graph-type data structure in which reference relationships are defined for each clause of the organization's internal regulations with at least one of the following: information on the clauses of laws and regulations, information on precedents, information on books, or information on guidelines.

[0682] In step S3821, the question processing module 2044 of the server 20 receives a question input from the user. Based on the received question, the question processing module 2044 extracts initial relevant candidate content from the graph-type data structure using methods such as vector search. Based on the content of the received question, the question processing module 2044 extracts multiple content items by referring to the data in the graph-type data structure.

[0683] The Question Processing Module 2044 performs a search (vector search) based on the content of the question, extracts content from a graph-type database that defines reference relationships with the search results, and uses this search result content and extracted content as the basis for generating the answer, which is then used by the LLM (Large-Scale Language Model Service Server 95) to generate the answer. The Question Processing Module 2044 searches multiple types of content based on the content of the received question and identifies content in a graph-type data structure that defines reference relationships with the search results.

[0684] Furthermore, the Question Processing Module 2044 may also refer to a graph-type database that associates internal regulations and prioritize the display of internal regulations that are frequently cited, by assigning higher scores to those regulations. To achieve this, the following processing may be performed: The Question Processing Module 2044 searches for information from books or guidelines based on the content of the question. The Question Processing Module 2044 identifies information from laws or precedents for which a reference relationship is defined in the graph-type data structure, at least one of the searched book information or guideline information. The Question Processing Module 2044 may also extract information from the organization's internal regulations that define a reference relationship to at least one of the searched book information or guideline information, and at least one of the identified law information or precedents.

[0685] In step S3822, the question processing module 2044 of server 20 analyzes the reference relationships between the extracted candidate content (citation and cited information, specification of basic textbooks, subject-specific weighting, etc.).

[0686] In step S3823, the question processing module 2044 of server 20 applies evaluation rules based on the number of citations and the type of document (books, guidelines, laws, precedents, internal regulations, etc.) to each candidate content, calculates the priority for presentation, and determines it.

[0687] In step S3824, the question processing module 2044 of server 20 constructs a prompt for answer generation (the question content and the extracted and ranked candidate content information) according to the determined priority and requests the LLM to generate the answer.

[0688] In this way, the question processing module 2044 generates an answer using the content of the question and the multiple pieces of content extracted in the extraction step.

[0689] Here, the question processing module 2044 generates a prompt that includes the information obtained from the search in step S3821 and the content of the question, and provides the generated prompt to the information processing system (server 95 of the large-scale language model service) that performs language processing, causing the information processing system to generate an answer corresponding to the prompt. Here, the question processing module 2044 generates a prompt that includes the content of the search results obtained in step S3821, the content identified in the graph-type data structure, and the content of the question, and also includes instructions that refer to these contents to generate an answer to the question. The question processing module 2044 also generates a prompt that includes the content of the question, which includes at least one of the searched book information or guideline information, at least one of the law information or case law information identified based on reference relationships from the data in the graph-type data structure, and extracted information on the organization's internal regulations.

[0690] In step S3825, the question processing module 2044 of server 20 integrates the answer generated from LLM with multiple supporting content (lists arranged in order of priority) and returns the generated answer to terminal 10.

[0691] The question processing module 2044 presents the answer generated in step S3824 to the user of terminal 10, along with multiple pieces of content that formed the basis for generating that answer. Here, the question processing module 2044 presents multiple pieces of content in order of priority based on the reference relationships in the data of the graph-type data structure.

[0692] Furthermore, the question processing module 2044 presents the content resulting from the search in step S3821 and the content identified in the graph-type data structure as the supporting content, along with the answer.

[0693] (1) Prioritization of content presentation based on graph-type data structure: citations Question processing module 2044 may prioritize books or guidelines by assigning higher scores to those with more citations (more edges) in a graph-type data structure.

[0694] The question processing module 2044 may also prioritize the content that formed the basis for generating the answer, displaying multiple pieces of content that served as the basis, with higher priority given to content that has a greater number of reference relationships defined with other content in a graph-type data structure (i.e., the more it is cited).

[0695] (2) Prioritization of content presentation based on graph-type data structures: Systematic basic texts and designated items The question processing module 2044 may assign higher scores to books that are more systematic and comprehensive.

[0696] The question processing module 2044 may prioritize books among the source content, displaying multiple sources of evidence, with books that are systematic and fundamental texts receiving higher priority.

[0697] For example, the question processing module 2044 may designate books that have been specified as basic texts, or books that have been determined to be basic texts based on a summary of their reviews, as systematic basic texts. The question processing module 2044 may also give higher priority to books that have been designated as systematic basic texts by the user, or books that have been determined to be systematic basic texts based on the content of their reviews, as books that are systematic basic texts.

[0698] (3) Regarding internal regulations, the priority of content presentation will be determined based on a graph-type data structure. The question processing module 2044 may also display multiple pieces of content that serve as the basis for the question, prioritizing those with a higher number of reference relationships defined with other content in a graph-type data structure, based on the information from the organization's internal regulations that formed the basis of the question.

[0699] (4) Determination of important content types for each field. The question processing module 2044 may refer to a graph-type data structure to determine the type of important content in each field (for example, whether books or guidelines are more important). For example, the question processing module 2044 may analyze the citation relationships of content obtained by searching literature, aggregate the number of citations for each type of content (books, guidelines), and determine which type is more important. In this way, the question processing module 2044 may determine the importance of each type of content by analyzing the reference relationships defined between multiple types of content that served as the basis for the question in a graph-type data structure.

[0700] The question processing module 2044 may prioritize presenting the user with content of the type it has determined to be important as described above. In this way, the question processing module 2044 may control the display order of each piece of content based on the results of an analysis of citation relationships for each type of content, such as books and guidelines, for each field (guidelines are considered important if they are cited frequently).

[0701] Thus, the question processing module 2044 may prioritize presenting the most important type of evidence content based on its determination of importance among the various types of evidence content.

[0702] For example, the question processing module 2044 may prioritize displaying content of high importance within each field, and when switching between categories (books, guidelines, etc.) using tabs, the prioritized tab may be displayed as selected in the initial display.

[0703] For example, when the question processing module 2044 displays content sorted by type, it may display content of the most important type at the top.

[0704] In step S3812, terminal 10 displays to the user a prioritized list of the answers and supporting content received from server 20.

[0705] <Modified form of the 13th embodiment> In the above description of the embodiment, an example was described in which literature is evaluated based on the citation relationships of each document in a graph-type data structure, literature is extracted in response to a question, and the priority of the literature to be referenced in the prompt to the server 95 of the large-scale language model service is determined according to the evaluation results of the literature. Alternatively, when extracting literature in response to a question, literature may be extracted without using the evaluation results of the literature based on the citation relationships of each document in the graph-type data structure, and the literature to be referenced in the prompt to the server 95 of the large-scale language model service may be provided according to the score of each document in the extraction result, while presenting the user with a list of literature that formed the basis for generating the answer, the list of literature that formed the basis may be presented based on the evaluation results of the literature based on citation relationships.

[0706] <25 Operation (Processing flow of the 14th embodiment)> The 14th embodiment describes a technique for controlling the display order of documents presented to a user, based on the reader's experience, knowledge, and abilities. To evaluate the difficulty level of the documents, the reference relationships of each document in a graph-type data structure are analyzed, and those with many citations are considered to be of high difficulty.

[0707] Figure 39 shows the process flow for evaluating the difficulty level of literature based on citation relationships in a graph-type database.

[0708] The following processing flow demonstrates a method for objectively evaluating the difficulty level of literature by utilizing the reference relationships (citation relationships) between nodes in a graph-type database. This allows for the creation of a foundation for providing appropriate literature according to user requirements.

[0709] Furthermore, the following processing flow allows for the evaluation of literature difficulty by combining analysis of vocabulary, style, and technical terms in addition to citation relationships.

[0710] In step S3911, terminal 10 inputs a request from the user to select the literature to be evaluated, or to evaluate the difficulty level of each document, and sends this request information to server 20.

[0711] Data structure definition module 2046 performs literature difficulty evaluation using a graph database. This is explained below.

[0712] The memory unit 202 is configured to store data in a graph-type data structure that defines reference relationships between multiple types of content.

[0713] The memory unit 202 is configured to store multiple types of content, including at least one of either book information (including books on law) or guideline information (including guidelines on law), information on laws and regulations, and information on precedents, and to store data in a graph-type data structure in which reference relationships are defined between at least one of the information on each article of the law, the information on precedents, the information on books, or the information on guidelines.

[0714] The memory unit 202 stores data in a graph-type data structure, where the title of at least one of the books or guidelines is a vertex, and the content portion of at least one of the books or guidelines is a vertex, with a reference relationship defined between the title vertex and the content portion vertex in a parent-child relationship.

[0715] In step S3921, after receiving a user request, the data structure definition module 2046 of the server 20 accesses the graph database and retrieves information on the nodes (titles and content sections) of the literature to be evaluated, as well as the citation and reference relationships of other content associated with them.

[0716] In step S3922, the data structure definition module 2046 of the server 20 analyzes the citation relationships set for each child node (divided portion of the document content) corresponding to the target document and extracts the number of other contents referenced by each child node.

[0717] In this way, the data structure definition module 2046 identifies the number of contents that reference other contents by referring to the graph-type data structure of the storage unit 202. Here, the data structure definition module 2046 may identify the number of contents that reference other contents by aggregating the number of reference relationships that are defined (for example, if reference relationships are defined in multiple places on the same content, it aggregates the reference relationships that are defined individually), or it may aggregate the number of other contents that reference a reference relationship (for example, if reference relationships are defined in multiple places on the same content, it aggregates that one content is referenced regardless of the aggregation result of the reference relationships that are defined individually). • How to count the number of references to other content The data structure definition module 2046 may also aggregate the number of other contents for which each child node defines a reference relationship in a graph-type database that associates laws, precedents, books, and guidelines, where content is defined as a parent node and the content is divided into child nodes to define a parent-child relationship.

[0718] The data structure definition module 2046 may also determine the number of references by aggregating the number of other contents that define reference relationships for each child vertex corresponding to the content of the book or guideline.

[0719] The data structure definition module 2046 may aggregate the number of other contents for which child nodes define reference relationships in the parent node. The data structure definition module 2046 may aggregate and maintain in the parent vertex the number of other contents for which child vertices corresponding to each part of the content of a book or guideline define reference relationships.

[0720] In step S3923, the data structure definition module 2046 of the server 20 aggregates the number of citations of each extracted child node into the parent node and calculates a difficulty index based on the overall citation relationships (e.g., the more citations, the higher the difficulty). The data structure definition module 2046 may also, if necessary, integrate information from the text analysis results, such as vocabulary difficulty, writing style, and usage of technical terms, with information from the evaluation results based on citation relationships, to calculate an overall difficulty score for each document.

[0721] As described above, the data structure definition module 2046 may evaluate each content based on the number of other contents that define reference relationships, which are aggregated at the parent vertex.

[0722] Here, the data structure definition module 2046 suggests that a higher number of citations from other content may indicate a higher level of difficulty for the reader when evaluating the difficulty of a document. For example, a document might be one that uses numerous citations to support its arguments, and therefore be of a certain length or more.

[0723] The data structure definition module 2046 may evaluate content by determining that the more references a piece of content has to other content, the more difficult the content is considered to be.

[0724] Furthermore, the data structure definition module 2046 may rate the difficulty level higher the more content that references other content (i.e., the more difficult the references are). The data structure definition module 2046 may also rate the difficulty level of the content higher the more other content that references the content.

[0725] In step S3924, the data structure definition module 2046 of the server 20 classifies the literature into multiple levels from low to high based on the calculated difficulty score. This allows for setting display priorities that take into account future user browsing history and the user's demographic information.

[0726] In step S3925, the data structure definition module 2046 of the server 20 sends the final evaluation results and literature classification results back to the terminal 10.

[0727] In step S3912, terminal 10 displays evaluation information such as the difficulty level, classification, and priority display order of each document to the user on the screen, based on the evaluation results received from the server.

[0728] Figure 40 shows the process flow for controlling the difficulty level of literature presented to a user according to the user's browsing trends (difficulty trends), utilizing the evaluation results of literature difficulty levels based on a graph-type database.

[0729] The following processing flow can improve research and learning efficiency by presenting literature of appropriate difficulty level to each user, based on their past browsing history and profile information. Furthermore, by matching the evaluation results of literature difficulty with the user's browsing trends, the display order is dynamically changed, allowing content to be presented in line with the user's improving abilities.

[0730] In the following explanation, as described above, the server 20 receives a question input from the user, extracts the content that forms the basis of the answer by referring to a graph-type data structure, and performs language processing using an information processing system (server 95 of the large-scale language model service) to output an answer in response to a prompt that generates an answer by referring to the supporting literature. The server then presents the answer and the content that formed the basis for generating the answer to the user. In this explanation, we describe an example in which the content that formed the basis for generating the answer is displayed in order according to the difficulty level tendencies of each user.

[0731] User Management Module 2043 accepts user requests to register their profile information (e.g., qualifications, years of experience) and past literature browsing history (e.g., difficulty level trends of viewed literature). Based on the received user information, User Management Module 2043 analyzes and aggregates each user's literature browsing trends and difficulty level preferences to generate a user profile (difficulty level trend profile).

[0732] In step S4021, the question processing module 2044 of server 20 retrieves the difficulty level evaluation results (evaluation results based on citation relationships and other text analysis results) of already evaluated literature from the graph database.

[0733] In step S4022, the question processing module 2044 of the server 20 compares the acquired literature evaluation results with the user's difficulty level tendency profile and calculates the priority (display order) of literature that matches the user's ability level. For example, for users who tend to prefer lower difficulty levels, the question processing module 2044 sets lower difficulty literature at the top.

[0734] In step S4023, the question processing module 2044 of server 20 combines the list of references, which has been sorted according to the difficulty level of the references, with the evaluation information of the relevant references (evidence information such as the number of citations and the usage of technical terms), and formats it into data for generating answers.

[0735] In this way, the question processing module 2044 classifies the literature according to its difficulty level and presents it to the user. The question processing module 2044 classifies each piece of content according to its difficulty level based on the results of evaluating the difficulty level of each piece of content. The question processing module 2044 then presents each piece of content, classified according to its difficulty level, to the user.

[0736] The question processing module 2044, in a system that generates answers to questions as described above, presents the literature that served as the basis for generating the answer along with the answer itself. The question processing module 2044 receives a question from the user. The question processing module 2044 searches for content based on the content of the received question. The question processing module 2044 generates a prompt that includes the searched content and the content of the question, and includes an instruction to generate an answer by referring to the searched content. The question processing module 2044 obtains an answer corresponding to the prompt from the information processing system that performs language processing by providing a prompt to the information processing system that performs language processing. The question processing module 2044 then presents the obtained answer to the user, including the content that served as the basis for generating the answer. In this case, as described above, the content that served as the basis is presented in an order according to the difficulty level of the content.

[0737] The question processing module 2044 may present literature of appropriate difficulty based on the user's viewing history and trends in difficulty levels. The question processing module 2044 may also present the user with the content that formed the basis for generating the answer, and may present content of appropriate difficulty levels based on the user's viewing history and the resulting difficulty level evaluations of the content viewed.

[0738] The question processing module 2044, in a system that generates answers to questions, may also generate answers that are tailored to the user's skill level (tendency in difficulty) according to the difficulty level of the literature viewed by the user. The question processing module 2044 receives a question from the user. The question processing module 2044 searches for content based on the content of the received question. The question processing module 2044 generates a prompt that includes the searched content and the content of the question, and includes an instruction to generate an answer by referring to the searched content. The question processing module 2044 obtains an answer corresponding to the prompt from an information processing system that performs language processing by providing a prompt to the information processing system that performs language processing. The question processing module 2044 presents the obtained answer to the user, including the content that formed the basis for generating the answer. Here, in obtaining an answer, the question processing module 2044 may also generate a prompt that includes an instruction to generate an answer tailored to the difficulty level, according to the trend in the evaluation results of the difficulty level of the content viewed by the user in their content viewing history. The question processing module 2044 may also obtain a response from a language processing system that is generated according to the user's difficulty level as a response to a prompt.

[0739] The question processing module 2044 may present literature of varying difficulty levels based on whether the user is qualified and the length of time they have held that qualification. The question processing module 2044 receives information about at least one of the following user details: whether the user has a specific qualification, and the length of time the user has held that qualification. The question processing module 2044 sets the difficulty level of the content to present to the user based on the user information. The question processing module 2044 sets the priority of the content to present to the user based on the evaluation result of the difficulty level of each piece of content and the difficulty level set for the user. The question processing module 2044 may present the content to the user according to the set priority level.

[0740] In step S4011, terminal 10 receives a list of literature and evaluation information tailored to the user's browsing habits, which has been returned from server 20, and presents it on the screen as the most suitable literature information.

[0741] <26 Operation (Processing flow of the 15th embodiment)> The description of the 15th embodiment describes a technique for finding experts in a particular field. For example, it describes a technique that utilizes a graph-type data structure to accept a question such as "I want to find an expert or lawyer who is knowledgeable in a specific field (e.g., cyberattacks)," and then presents experts based on the accumulated literature.

[0742] Figure 41 shows the processing flow for asking questions, generating answers, and presenting the content that formed the basis for generating those answers, on an author-by-author basis.

[0743] The following processing flow, unlike conventional expert matching services that match clients with experts, automatically extracts relevant laws, precedents, books, etc., from the content of the question and presents them on an author-by-author basis. This allows users to grasp the source and reliability (evaluated by the number of citations) of each piece of information at a glance, making it even easier to find experts from these perspectives.

[0744] In step S4111, terminal 10 receives the user's question input screen and enters the consultation details, questions, and doubts into the text form. The entered question data is then sent to server 20 via the communication function.

[0745] Server 20's question processing module 2044 extracts relevant content, such as books, from a graph-type database to make it easier to find experts, and displays the extracted results by author. This is explained below.

[0746] The memory unit 202 is configured to store data in a graph-type data structure that defines reference relationships between multiple types of content. The content is associated with the author's information.

[0747] The memory unit 202 is configured to store multiple types of content, including at least one of either book information (including books on law) or guideline information (including guidelines on law), information on laws and regulations, and information on precedents, and to store data in a graph-type data structure in which reference relationships are defined between at least one of the information on each article of the law, the information on precedents, the information on books, or the information on guidelines.

[0748] In step S4121, the question processing module 2044 of the server 20 analyzes the received question content and converts it into a vector representation for internal processing (preparation for vector search). The question processing module 2044 executes a vector search algorithm to search for content related to the question content. For the content of the search results, it refers to a graph-type database and extracts content for which reference relationships are defined. The question processing module 2044 also obtains information such as the citation relationships of each piece of content (number of times other content is cited, number of times other content is cited, etc.).

[0749] In this way, the question processing module 2044 receives a question from the user. Based on the content of the received question, the question processing module 2044 performs a search, referring to data in a graph-type data structure to find content corresponding to the content of the question.

[0750] The question processing module 2044 performs a search based on the content of the question and extracts the content of the search results and the content in the graph-type data structure where reference relationships are defined.

[0751] In step S4122, the question processing module 2044 of the server 20 groups each searched and extracted content based on the accompanying author information, and determines the presentation order for each author based on the number of citations and the relative importance of references.

[0752] In step S4123, the question processing module 2044 of server 20 integrates the question content with the supporting content organized by author and generates a prompt for the LLM (Language Processing System). The generated prompt is sent to the LLM, and the result is received, which includes the answer to the question and the content information that underlies its generation.

[0753] In this way, the question processing module 2044 refers to a graph-type database that links laws, precedents, books, and guidelines, and while providing answers to questions, it also presents the content that formed the basis of those answers, on an author-by-author basis. Specifically, the question processing module 2044 generates a prompt that includes information on the searched content and the content of the question. The question processing module 2044 provides the generated prompt to an information processing system that performs language processing, causing the information processing system to generate an answer corresponding to the prompt.

[0754] The question processing module 2044 generates a prompt that includes the content of the search results, the extracted content, and the content of the question, and includes instructions to generate an answer by referring to this content.

[0755] In step S4124, the question processing module 2044 of server 20 takes the information on the answers and supporting content received from the information processing system (server 95 of the large-scale language model service) that performs language processing, organizes and formats it on an author-by-author basis, and converts it into a data format for presentation to the user. The question processing module 2044 sends the formatted answers and the information on the categorized and prioritized supporting content back to terminal 10.

[0756] In this way, the question processing module 2044 classifies the content of the search results by author and presents it to the user.

[0757] The question processing module 2044 presents the answer to the question and the content that formed the basis for generating the answer, classifying and presenting the content that formed the basis for generating the answer by author. • Method of presenting content on an author-by-author basis The question processing module 2044 may also determine the priority of content presented on an author-by-author basis, according to the degree of citation relationships. For example, content with more citations may be given a higher priority. The question processing module 2044 may also prioritize the presentation of content that has a large number of reference relationships defined with other content in a graph-type data structure. For example, it would be easier to find authors of works that discuss their work while citing numerous other sources. • Display the number of extracted documents per author. The question processing module 2044 may also display the number of extracted content pieces for each author. For example, if five works are extracted for a particular author, the number of extracted content pieces for that author will be displayed as "5". This makes it easy to find authors with many works in the field corresponding to the question. • Business workflow after extracting experts When the question processing module 2044 receives an operation from the user to select an expert, it may enable the user and the selected expert to send and receive messages.

[0758] The question processing module 2044 accepts an operation from the user to specify the author. For example, it may generate and display an answer to a question while displaying the supporting content, and then display the author's information for that content, allowing the user to specify the author. The question processing module 2044 then sends and receives messages between the specified author and the user based on the operation specified.

[0759] The question processing module 2044, when receiving a question from a user that asks for an expert, may present content on an author-by-author basis. However, if the question does not include a request to find an expert, it may display the answer to the question while presenting the content that formed the basis of the answer, without classifying it by author.

[0760] The question processing module 2044 may, when it receives a question that seeks an expert, present content on an author-by-author basis, while when it receives a question that does not include a request to find an expert, it may present content without classifying it by author.

[0761] In step S4112, terminal 10 displays the received results, along with the response portion, on the user interface as evidence content classified by author (with indicators such as citation count and priority).

[0762] <Modified form of the 15th embodiment> In the above description of the embodiment, an example was described in which, when a question is received and content such as books is searched based on the question, the content is grouped by author, and a prompt is generated in the prompt given to the server 95 of the large-scale language model service that includes an instruction to refer to the content grouped by author.

[0763] Alternatively, in response to a question, the system may search for literature using vector search or similar methods based on the question, generate a prompt, and then, without grouping by author, have the server 95 of the large-scale language model service generate an answer corresponding to the prompt. When presenting the literature that formed the basis for generating the answer, the literature may be grouped and displayed by author.

[0764] <Other> The concepts described in each of the above embodiments can be combined.

[0765] A network consists of various mobile communication systems, such as the internet, LANs, and wireless base stations. For example, a network includes 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks that can connect to the internet via designated access points (e.g., Wi-Fi®). When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via a wired connection, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0766] Furthermore, by distributing all or part of each hardware configuration across multiple computers and connecting them to each other via a network, a computer can be virtually realized. Thus, the concept of a computer includes not only computers housed in a single enclosure or case, but also virtualized computer systems.

[0767] A database, specifically a relational database, is used to manage and link together tabular data sets called masters, which are structurally defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters.

[0768] Typically, each table and master has a primary key column to uniquely identify records, but setting a primary key column is not mandatory. The control unit can instruct the processor 901 to add, delete, or update records in specific tables and masters stored in the memory unit, according to various programs.

[0769] Furthermore, by storing data, various programs, and various databases in the memory unit, the information processing device and information processing system related to this disclosure can be considered to have been manufactured.

[0770] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0771] Furthermore, each of the above-mentioned configurations, functions, processing units, processing means, etc., may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. The present invention can also be implemented by software program code that realizes the functions of the embodiment. In this case, a storage medium on which the program code is recorded is provided to a computer, and the processor of that computer reads the program code stored in the storage medium. In this case, the program code read from the storage medium itself realizes the functions of the embodiment described above, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media used to supply such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, and the like.

[0772] Furthermore, the program code that implements the functions described in this embodiment can be implemented in a wide range of programming or scripting languages, such as assembler, C / C++, Perl, Shell, PHP, and Java (registered trademark).

[0773] Furthermore, the program code for the software that implements the functions of the embodiment may be distributed via a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the computer's processor may read and execute the program code stored in the storage means or storage medium.

[0774] The functions realized by the components described herein may be implemented in a circuit or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the functions described herein. A processor is considered to be a circuit or processing circuitry, including transistors and other circuits. A processor may be a programmed processor that executes a program stored in memory.

[0775] In this specification, circuitry, unit, and means are hardware programmed to perform or execute the functions described herein. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to perform or execute the functions described herein.

[0776] If the hardware is a processor that is considered to be a type of circuitry, then the circuitry, means, or unit is a combination of hardware and software used to constitute the hardware and / or processor.

[0777] While several embodiments of this disclosure have been described above, these embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications are permitted without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents.

[0778] (Note) The details described in each of the above embodiments are noted below.

[0779] (Note 1) A program for operating a computer having one or more computer processors, wherein the memory unit stores at least one of the following: information on books including books on law or information on guidelines including legal guidelines, information on laws and regulations, information on precedents, and information on the organization's internal regulations, and is configured to store data in a graph-type data structure in which reference relationships are defined between each clause of the organization's internal regulations and at least one of the information on each clause of laws and regulations, information on precedents, information on books, or information on guidelines, and the program is configured to operate a computer having one or more computer processors, a step of receiving input of a question from a user, a step of identifying at least one of the information on laws and regulations, information on precedents, information on books, or information on guidelines related to the content of the question by referring to data in a graph-type data structure based on the content of the received question, a step of extracting information on the organization's internal regulations for which reference relationships are defined with respect to the information identified in the identification step, and a step of generating an answer by an information processing system that performs language processing using at least the information on the organization's internal regulations extracted in the extraction step. A program that performs the following steps: presenting the user with the answer generated in the generation step, which includes information on the organization's internal regulations that form the basis of the answer.

[0780] (Note 2) The program described in Appendix 1, wherein in the identification step, it searches for at least one of the book information or guideline information based on the content of the question, identifies at least one of the statutory information or case law information for which a reference relationship is defined in the data of a graph-type data structure, and in the extraction step, it extracts at least one of the statutory information or case law information identified in the identification step and information of the organization's internal regulations for which a reference relationship is defined in the data of a graph-type data structure.

[0781] (Note 3) The program described in Appendix 2, which, in the identification step, vectorizes the content of the question and searches for at least one of the book information or guideline information by comparing the vector of the question content with at least one of the vectors of the book information or guideline information.

[0782] (Note 4) A program as described in any of the appendices 1 to 3, which in the generation step generates a prompt that includes the content of a question, at least one of the information on laws and regulations, case law, books, or guidelines identified in the identification step, and information on the organization's internal regulations extracted in the extraction step, and provides the generated prompt to an information processing system that performs language processing, thereby causing the information processing system to generate an answer corresponding to the prompt.

[0783] (Note 5) A method performed by a computer having one or more computer processors, wherein the storage unit is configured to store at least one of the following in its memory: information on books including books on law or information on guidelines including legal guidelines, information on laws and regulations, information on precedents, and information on the organization's internal regulations, and to store data in a graph-type data structure in which reference relationships are defined for each provision of the organization's internal regulations with at least one of the information on each provision of laws and regulations, information on precedents, information on books, or information on guidelines, the method is a method in which one or more computer processors perform the steps of: receiving input of a question from a user; identifying at least one of the information on laws and regulations, information on precedents, information on books, or information on guidelines related to the content of the question by referring to data in a graph-type data structure based on the cont...

Claims

1. A program for operating a computer having one or more computer processors, In the memory unit, Information from books, including legal textbooks, or information from guidelines, including legal guidelines, at least one of the following: Information on laws and regulations, Information on case law, It is configured to store information on the organization's internal regulations, and to store data in a graph-type data structure in which reference relationships are defined for each clause of the organization's internal regulations with at least one of the following: information on each clause of the aforementioned laws and regulations, information on the aforementioned precedents, information on the aforementioned books, or information on the aforementioned guidelines. The program is configured on one or more computer processors. Steps to receive questions from the user, Based on the content of the question received, the step of identifying at least one of the information on laws and regulations, case law, books, or guidelines related to the content of the question by referring to the data in the graph-type data structure, The steps include: extracting information from the organization's internal regulations that define reference relationships to the information identified in the aforementioned identification step; The steps include generating a response using an information processing system that performs language processing, with at least the information on the internal regulations of the organization extracted in the extraction step, A program that performs the steps of: presenting to the user the answer generated in the above-mentioned step, which includes information on the internal regulations of the organization that form the basis of the answer.

2. In the aforementioned step of identification, Based on the content of the above questions, search for at least one of the information in the above book or the information in the above guidelines, Identify at least one of the information in the searched book or the information in the guideline, and at least one of the information in the graph-type data structure that has a reference relationship defined with the information in the law or the information in the case law. The program according to claim 1, wherein in the extraction step, it extracts information of the organization's internal regulations, which have a reference relationship defined in the data of the graph-type data structure, with at least one of the information of laws and regulations or the information of precedents identified in the identification step.

3. The program according to claim 2, wherein in the step of identifying, the content of the question is vectorized, and at least one of the information in the book or the information in the guideline is retrieved by comparing the vector of the content of the question with the vector of at least one of the information in the book or the information in the guideline.

4. In the step of generating, To generate a prompt that includes the content of the aforementioned question, at least one of the information on the aforementioned laws and regulations, the aforementioned case law, the aforementioned book, or the aforementioned guideline information identified in the aforementioned identification step, and the information on the organization's internal regulations extracted in the aforementioned extraction step, The program according to claim 1, wherein the generated prompt is provided to the information processing system that performs language processing, thereby causing the information processing system that performs language processing to generate a response corresponding to the prompt.

5. A method performed by a computer having one or more computer processors, In the memory unit, Information from books, including legal textbooks, or information from guidelines, including legal guidelines, at least one of the following: Information on laws and regulations, Information on case law, It is configured to store information on the organization's internal regulations, and to store data in a graph-type data structure in which reference relationships are defined for each clause of the organization's internal regulations with at least one of the following: information on each clause of the aforementioned laws and regulations, information on the aforementioned precedents, information on the aforementioned books, or information on the aforementioned guidelines. The above method involves one or more computer processors, Steps to receive questions from the user, Based on the content of the question received, the step of identifying at least one of the information on laws and regulations, case law, books, or guidelines related to the content of the question by referring to the data in the graph-type data structure, The steps include: extracting information from the organization's internal regulations that define reference relationships to the information identified in the aforementioned identification step; The steps include generating a response using an information processing system that performs language processing, with at least the information on the internal regulations of the organization extracted in the extraction step, A method comprising the steps of: presenting to the user the answer generated in the above-mentioned step, which includes information on the internal regulations of the organization that form the basis of the answer.

6. An information processing device, In the memory unit, Information from books, including legal textbooks, or information from guidelines, including legal guidelines, at least one of the following: Information on laws and regulations, Information on case law, It is configured to store information on the organization's internal regulations, and to store data in a graph-type data structure in which reference relationships are defined for each clause of the organization's internal regulations with at least one of the following: information on each clause of the aforementioned laws and regulations, information on the aforementioned precedents, information on the aforementioned books, or information on the aforementioned guidelines. The control unit of the information processing device, Steps to receive questions from the user, Based on the content of the question received, the step of identifying at least one of the information on laws and regulations, case law, books, or guidelines related to the content of the question by referring to the data in the graph-type data structure, The steps include: extracting information from the organization's internal regulations that define reference relationships to the information identified in the aforementioned identification step; The steps include generating a response using an information processing system that performs language processing, with at least the information on the internal regulations of the organization extracted in the extraction step, An information processing device that performs the steps of: presenting to the user the answer generated in the above-mentioned step, the answer which includes information on the internal regulations of the organization that form the basis of the answer.

Images