Information processing device and information processing method

Abstract

The present invention provides a technique for extracting a review viewpoint for reviewing the quality of a document. An information processing device according to one embodiment includes: an extraction unit that extracts the review viewpoint of a document using a first large-scale language model from teaching data including document data generated in the past; and storage unit that stores the extracted review viewpoint in a database. The information processing device includes: a review unit for reviewing a new document using a second large-scale language model on the basis of the review viewpoint stored in the database; and an output unit for outputting a review result by the review unit.

Description

Information processing device and information processing method

[0001] The present invention relates to a technique for an information processing device and an information processing method.

[0002] For example, Japanese Patent Application Laid-Open No. 2009-124222 discloses an invention that determines in advance the strengths and weaknesses of users and selects reviewer candidates suitable for a document to be reviewed from among these users.

[0003] Patent No. 6676792

[0004] The invention described in Patent Document 1 simply selects reviewer candidates.

[0005] In response to this, the present invention provides a technique for extracting review viewpoints for reviewing the quality of a document.

[0006] An information processing device according to one aspect of the present disclosure includes an extraction unit that extracts review perspectives for a document from training data including document data generated in the past using a first large-scale language model, and a storage unit that stores the extracted review perspectives in a database.

[0007] An information processing method according to another aspect of the present disclosure includes a step of extracting review perspectives for a document from training data including document data generated in the past using a first large-scale language model by a computer, and a step of storing the extracted review perspectives in a database.

[0008] According to the present invention, review perspectives for reviewing the quality of a document can be extracted.

[0009] 1 is a diagram illustrating a system configuration of an information processing system 1 according to an embodiment. A diagram illustrating a functional configuration of the information processing system 1. A diagram illustrating a hardware configuration of an information processing device 10. A sequence chart illustrating a method for extracting review perspectives in the information processing system 1. A diagram illustrating a prompt P11 for requesting extraction of review perspectives. A diagram illustrating a document. A diagram illustrating a teacher database 1001. A diagram illustrating a review perspective database 1002. A sequence chart illustrating a method for searching for review perspectives in the information processing system 1. A sequence chart illustrating a method for reviewing a document in the information processing system 1. A diagram illustrating a prompt P21 for requesting review of a document. A diagram illustrating a document review result.

[0010] 1. Configuration FIG. 1 illustrates an exemplary system configuration of an information processing system 1 according to an embodiment. In this example, the information processing system 1 (or simply the system) is a system for providing a document review service. The document review service is a service related to document review and has a function for automatically extracting perspectives (hereinafter referred to as "review perspectives") for reviewing the quality of documents, such as documents, based on past data. A document refers to information that is output in a format that can be perceived visually or aurally by humans, and is a general term that includes, for example, presentation slides, text documents, spreadsheets, emails, reports, materials, documents, and academic papers. A document also includes computer data created using existing software tools. A review perspective is knowledge that summarizes the quality requirements that a document must satisfy based on its content. Past data (an example of training data) refers to documents created in the past. The information processing system 1 reviews documents created by users based on the extracted review perspectives. Note that the term "user" here refers to a user who uses the review perspective extraction function in the information processing system 1 and a document creator who uses the review function.

[0011] Document creation and review account for a large portion of business operations. However, in traditional business settings, document quality requirements are often not verbalized, and review criteria have traditionally been implicitly understood (i.e., relying on personal evaluation). As a result, evaluation criteria vary depending on the person in charge, and review criteria change when the person in charge changes. This has led to problems such as a decline in document quality, an inability to ensure reproducibility, and ultimately a decrease in efficiency in terms of time and effort. To address these issues, the inventors focused on using AI specialized for document work. In particular, large language models (LLMs), which have shown remarkable technological innovation in the AI ​​field, have language processing capabilities using natural language, analytical capabilities, huge databases, and user affinity. Therefore, LLMs are highly compatible with document work. To perform document creation and review work efficiently and with high accuracy, the information processing system 1 has the following configuration.

[0012] The information processing system 1 includes an information processing device 10, an administrator terminal 20, a creator terminal 30, a first LLM 110, and a second LLM 120. In this example, the components of the system are connected via a network 9 as shown in Fig. 1. In this example, the network 9 is a computer network such as the Internet or a mobile network.

[0013] The information processing device 10 is an information processing device or server device in the information processing system 1. In this example, the information processing device 10 extracts and stores review perspectives and reviews documents created by users. The information processing device 10 realizes various functions by working with AI, such as an LLM. In this example, the first LLM 110 (an example of a first large-scale language model) and the second LLM 120 (an example of a second large-scale language model) are both AI in the information processing system 1. The first LLM 110 has a function of extracting review perspectives based on documents. The second LLM 120 has a review function of reviewing user documents.

[0014] In this example, the information processing device 10 is connected to an administrator terminal 20 and a creator terminal 30 via a network 9. The information processing device 10 cooperates with a first LLM 110 in response to a request from the administrator terminal 20 to extract review perspectives. The information processing device 10 stores the extracted review perspectives in a dedicated database. The information processing device 10 acquires a document from the creator terminal 30. The information processing device 10 cooperates with a second LLM 120 to review the document in accordance with the review perspectives.

[0015] The administrator terminal 20 is a terminal owned and used by an administrator of a company or the like. The administrator terminal 20 includes, for example, a smartphone, a tablet, or a personal computer. In this example, the administrator refers to an employee of the target company or organization (hereinafter referred to as the "target company"). In this system, the "administrator" is defined as a superior (e.g., a department manager, a division manager, etc.) who manages the use of the information processing system 1 at the target company. In addition, the employee who actually creates documents such as materials at the target company is defined as a "creator." The administrator terminal 20 cooperates with the information processing device 10 in accordance with the administrator's instructions and extracts review perspectives necessary for business operations.

[0016] The creator terminal 30 is a terminal owned and used by the creator of a document. Like the administrator terminal 20, the creator terminal 30 includes, for example, a smartphone, a tablet, or a personal computer. The creator terminal 30 is used, for example, as a terminal when the creator creates a document. By using the information processing system 1, the creator can read review perspectives stored in the database and use them as a reference when creating a document. Alternatively, the creator can request a review of a document they created from the information processing device 10 via the creator terminal 30.

[0017] 2 is a diagram illustrating an example of the functional configuration of the information processing system 1. In this embodiment, the information processing device 10 has functional blocks (components) including an extraction unit 11, an accumulation unit 12, a correction unit 13, a review unit 14, an output unit 15, a storage unit 191, and a control unit 192. In this example, the storage unit 191 stores various data and programs including a database, for example. In this example, the control unit 192 performs various controls.

[0018] The extraction unit 11 extracts document review perspectives using a first large-scale language model from training data including document data (examples of documents) generated in the past. In this example, the training data is, for example, past documents to which information (or labels) indicating the quality of each document has been assigned. The training data includes document data of documents that did not pass review based on predetermined criteria (hereinafter referred to as "bad documents"). Documents that pass review are referred to as "good documents" here. Note that the predetermined criteria refer to, for example, evaluation criteria for determining the quality of a document. By referring to the labels, the AI ​​can identify whether each document in the training data is a good document or a bad document.

[0019] The first LLM 110 (an example of a first large-scale language model) receives input of good documents and bad documents from the extraction unit 11. The first LLM 110 extracts review perspectives based on these documents and outputs them to the extraction unit 11. In other words, the first LLM 110 can extract perspectives for determining a document as a "good document" from examples of "good documents" and "bad documents."

[0020] In another example, the extraction unit 11 requests the first large-scale language model to consider predetermined review perspective candidates already in the database, which can be considered by the first large-scale language model to avoid duplication with review perspectives already in the database or to extract integrated review perspectives.

[0021] Here, the information processing device 10 cooperates with various LLMs using Retrieval-Augmented Generation (RAG). In this example, RAG is a prompt extension technology that combines search functions and other functions in response to user requests to improve the accuracy of LLM responses. For example, the information processing device 10 includes a first RAG 130 that inputs prompts to the first LLM 110 in response to requests received from the administrator terminal 20. The first RAG 130 acquires information necessary for extracting review perspectives from the user, such as prompts and various documents. The first RAG 130 searches for past document data or existing review perspectives recorded in the database of the information processing device 10. The first RAG 130 inputs a prompt that integrates this data into the first LLM 110 and extracts review perspectives.

[0022] The storage unit 12 stores the extracted review perspectives in a database. The storage unit 12 acquires the review perspectives from the first LLM 110. The storage unit 12 records the review perspectives, good documents, and bad documents in association with each other in the database.

[0023] The correction unit 13 corrects the review perspectives extracted by the extraction unit 11 in accordance with instructions from the user. The correction unit 13 presents the review perspectives to the manager terminal 20. The manager terminal 20 accepts corrections to the review perspectives from the manager (an example of a user). The manager terminal 20 requests the accumulation unit 12 to record the corrected review perspectives. The accumulation unit 12 records the corrected review perspectives in a database.

[0024] The review unit 14 reviews the new document using the second large-scale language model based on the review perspectives stored in the database. In this example, the new document is a document created by a creator (an example of a user). The review unit 14 acquires the new document and the review perspectives from the user via the creator terminal 30. The second LLM 120 (an example of a second large-scale language model) acquires the new document and the review perspectives from the review unit 14. In this example, the review unit 14 requests the second large-scale language model to evaluate the new document with respect to the review perspectives retrieved from the database.

[0025] Here, the second RAG 140 is used to request a review of a document from the second LLM 120. The second RAG 140 acquires a prompt requesting a review and the document to be reviewed from the user. The second RAG 140 searches for the extracted review perspective recorded in the database of the information processing device 10. The second RAG 140 inputs the prompt to the second LLM 120 and acquires the review results of the document.

[0026] The output unit 15 outputs the review results by the review unit 14. In this example, the review results include problem indications and suggested corrections for each review perspective. The output unit 15 outputs the review results to the creator terminal 30. In this example, the user corrects the new document based on the review results.

[0027] FIG. 3 is a diagram illustrating an example of the hardware configuration of the information processing device 10. The information processing device 10 is physically configured as a computer including a processor 101, a memory 102, a storage 103, a communication device 104, an input device (optional), a display device (optional), and a bus connecting these. Each of these devices operates using power supplied from a battery (not shown). In the following description, the term "device" can be interpreted as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may be configured to include one or more of the devices shown in FIG. 3, or may be configured without including some of the devices. Furthermore, the information processing device 10 may be configured by communicating and connecting multiple devices each having a different housing.

[0028] Each function of the information processing device 10 is realized by loading specified software (programs) onto hardware such as the processor 101, memory 102, etc., so that the processor 101 performs calculations, controls communication via the communication device 104, and controls at least one of reading and writing data in the memory 102 and storage 103.

[0029] The processor 101 controls the entire computer by running, for example, an operating system. The processor 101 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. Furthermore, for example, a baseband signal processing unit, a call processing unit, etc. may be realized by the processor 101.

[0030] The processor 101 reads programs (program codes), software modules, data, etc. from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described below. The functional blocks of the information processing device 10 may be implemented by a control program stored in the memory 102 and running on the processor 101. Various processes may be executed by one processor 101, or may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The programs may be transmitted to the information processing device 10 via a telecommunications line.

[0031] The memory 102 is a computer-readable recording medium and may be configured by at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), etc. The memory 102 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 102 can store executable programs (program codes), software modules, etc. for implementing the method according to this embodiment.

[0032] Storage 103 is a computer-readable recording medium, and may be composed of at least one of, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Storage 103 may also be called an auxiliary storage device.

[0033] The communication device 104 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0034] Each device, such as the processor 101 and the memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used between each device.

[0035] The information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 101 may be implemented using at least one of these pieces of hardware.

[0036] In this example, the programs stored in the storage 103 include a program (hereinafter referred to as a "server program") for causing a computer to function as a server in the information processing system 1. When the processor 101 is executing the server program, the processor 101, the memory 102, the storage 103, and the communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of an extraction unit 11, an accumulation unit 12, a correction unit 13, a review unit 14, and a control unit 192. At least one of the memory 102 and the storage 103 is an example of a storage unit 191. The communication device 104 is an example of an output unit 15.

[0037] Although detailed description will be omitted, the administrator terminal 20 and the creator terminal 30 are computers having a processor, memory, storage, communication device, input device, and output device, specifically, for example, a smartphone, tablet terminal, or personal computer. In this example, the programs stored in the storage of the administrator terminal 20 and the creator terminal 30 include a program (hereinafter referred to as a "client program") for causing the computer to function as a client in the information processing system 1. The configuration of the information processing system 1 has been described above. Next, the operation of the information processing system 1 will be described.

[0038] 4 is a sequence chart illustrating a method for extracting review perspectives in the information processing system 1. Here, a function will be described in which an administrator who manages a business extracts review perspectives necessary for the business by using the information processing system 1. Note that the processing in the information processing device 10 in this sequence is executed by the first RAG 130.

[0039] In step S101, the first RAG 130 receives a request to extract review perspectives from the administrator terminal 20. In this case, the administrator inputs good documents and bad documents to the first RAG 130 via a UI (User Interface) of the administrator terminal 20. For example, the first RAG 130 uses the uploading of a document as a trigger for extracting review perspectives. Here, a description will be given of documents.

[0040] FIG. 5 is a diagram illustrating an example of a document. Slide S1 and slide S2 are both examples of documents. In this example, slide S1 is an example of a bad document. Slide S2 is an example of a good document. These slides all show the same content, but are documents whose quality has been judged by an administrator based on factors such as design, font, balance, and readability. The administrator uploads slides S1 and S2 to the first RAG 130 via the administrator terminal 20 along with an extraction request.

[0041] Returning to Fig. 4, in step S102, the first RAG 130 automatically generates a prompt for requesting extraction of review viewpoints, triggered by an output (or request) from the manager terminal 20. The prompt will now be described.

[0042] FIG. 6 illustrates a prompt P11 for requesting the extraction of review perspectives. FIG. 6 shows an example of a prompt automatically generated by the first RAG 130. As indicated by the opening sentence of the prompt P11, for example, "Of the two attached slide groups, the first slide group did not pass review, while the second slide group passed review. Please summarize review perspectives so that future reviews can be performed using the same criteria as for these slide groups," the first RAG 130 requests the extraction of review perspectives. At this time, the first RAG 130 inputs specifications (the third paragraph) regarding the items and content of the review perspectives as the output format. The first RAG 130 attaches a group of good documents and a group of bad documents to the prompt P11. The first RAG 130 can search the database for the good documents and the bad documents and provide them as a group of training data to the first LLM 110.

[0043] FIG. 7 is a diagram illustrating a teacher database 1001. In this example, the teacher database 1001 includes multiple records related to teacher data. Each record corresponds to information for each teacher data. Each record includes ID information and data content. The ID information is unique identification information for each teacher data. Identification information (document ID) corresponding to good documents and bad documents is assigned, respectively. In this example, good documents and bad documents can be classified in advance by an administrator and registered in the teacher database 1001. The data content is the specific content of various data (good documents and bad documents). The data content includes, for example, information such as the document name, content, and creator (administrator or department).

[0044] The first RAG 130 acquires past documents from the teacher database 1001. In this example, the first RAG 130 searches the database according to the documents acquired from the administrator terminal 20. The first RAG 130 acquires document data generated in the past from the database as teacher data.

[0045] Returning to FIG. 4 , in step S103, the first RAG 130 outputs an extraction instruction to the first LLM 110. The extraction instruction includes automatically generated prompts, documents (good documents and bad documents), and training data acquired from a database. The first LLM 110 extracts review perspectives in response to the extraction instruction. In the case of a generative AI that uses natural language, such as an LLM, the review perspectives are generated on a text-based basis. Note that the first LLM 110 may have previously learned training data acquired from the first RAG 130 or another server. The first LLM 110, which has previously learned the training data, may extract review perspectives in response to the prompts and documents acquired from the first RAG 130.

[0046] In step S104, the first RAG 130 acquires data related to the review perspectives extracted by the first LLM 110. The data acquired by the first RAG 130 may be in any format, such as text, comma separated values ​​(CSV), or a table format.

[0047] In step S105, the extracted review viewpoints are appropriately recorded in a database. For example, the first RAG 130 accumulates the review viewpoints in the database. Here, the database for managing the extracted review viewpoints will be described.

[0048] FIG. 8 is a diagram illustrating an example of the review perspective database 1002. In this example, the review perspective database 1002 includes multiple records related to the extracted review perspectives. Each record corresponds to information for each review perspective. Each record includes a review perspective ID, a perspective item, perspective content, and tag information. The review perspective ID is unique identification information for each extracted review perspective. The perspective item is an item related to the classification of the review perspective. The perspective items include, for example, visual clarity, organization of information, clarity of title, or consistency. The perspective items are named according to the perspective. The perspective content is information related to the content of the review perspective indicated by the perspective item. This content includes text-based explanations, sentences, or explanations. The tag information is information related to tags, labels, or keywords that can be assigned to the review perspective. The tag information is typically a tag related to the department, position, manager, or creator who is expected to use a particular review perspective. For example, the tag information includes information such as the sales department, section manager or above, or general. The tag information is used when the information processing system 1 reviews documents (described below). The review viewpoint database 1002 may also store data of the documents used at the time of extraction. In this way, the first RAG 130 can accumulate review viewpoints in the database.

[0049] The information used as tag information is included as tag information in, for example, teacher data. When generating a prompt for causing the first LLM 110 to extract review perspectives, the first RAG 130 may generate a prompt that instructs the first LLM 110 to extract review perspectives (e.g., by classifying the review perspectives by tag information) taking the tag information into consideration. In this case, the prompt may include an instruction to present tag information corresponding to the extracted review perspectives.

[0050] Furthermore, information on general review perspectives (not shown) is recorded in the review perspective database 1002. The general review perspectives (hereinafter referred to as "general review perspectives") are review items that define general perspectives according to various categories, such as slide design, font, typographical errors, format, or legality. For example, when reviewing a document, the information processing system 1 can refer to the general review perspectives in addition to the review perspectives independently extracted by the first LLM 110.

[0051] Returning to Fig. 4, in step S106, the first RAG 130 outputs the extracted results of the review viewpoints to the manager terminal 20. The manager terminal 20 presents the review viewpoint data to the manager in accordance with various UIs.

[0052] In step S107, the first RAG 130 accepts a revision of the review perspective from the administrator. The first RAG 130 updates the review perspective recorded in the database based on the revision made by the administrator. Note that the first RAG 130 may accept a revision of the review perspective from the administrator using any method. In this example, the information processing system 1 may rearrange various steps and confirm with the administrator whether or not the review perspective has been revised before recording the revision in the database.

[0053] As described above, the administrator can extract review perspectives by using the information processing system 1. The information processing system 1 can extract review perspectives for reviewing the quality of a document from past documents. The administrator can also store the extracted review perspectives in a database. Next, a case where the creator of a document uses the information processing system 1 will be described.

[0054] 9 is a sequence chart illustrating a method for searching for review perspectives in the information processing system 1. Here, a case will be described in which a creator searches for review perspectives necessary for document creation using the information processing system 1 when creating a document. In step S201, the creator terminal 30 outputs a review perspective search request to the information processing device 10 in response to input by the creator. The search request includes a request to access a database, input of a target review perspective, or specification of tag information (or keywords), etc.

[0055] In step S202, the information processing device 10 searches the review viewpoint database 1002. At this time, if the creator has specified a search criteria, the information processing device 10 obtains search results according to the specified criteria. For example, the creator can narrow down the review viewpoints based on tag information, etc.

[0056] In step S203, the information processing device 10 outputs the search results of the review perspectives to the creator terminal 30. The creator terminal 30 displays the search results acquired from the information processing device 10 in various formats. For example, the creator terminal 30 may display the review perspectives in a list, or may display them on a text-based or language-based basis.

[0057] As described above, the creator can search for review perspectives stored in the information processing system 1. The creator can create a document based on the searched review perspectives. This ensures the quality and reproducibility of the document and improves work efficiency. Next, a case where the information processing system 1 reviews a document created by the creator will be described.

[0058] 2-3. Document Review Method FIG. 10 is a sequence chart illustrating a document review method in the information processing system 1. Here, a function for reviewing a creator's document in the information processing system 1 will be described. Note that the processing in the information processing device 10 in this sequence is executed by the second RAG 140. In step S301, the second RAG 140 accepts a document review request from the creator terminal 30. The review request includes uploading of a document created by the creator. Hereinafter, the document to be reviewed is referred to as the "target document." The review request may include tag information such as the creator's attribute information.

[0059] In step S302, the second RAG 140 reads the database in response to the creator's review request. The second RAG 140 references the review perspective database 1002 to acquire review perspectives that can be used to conduct a review. In this example, the second RAG 140 acquires review perspectives from tag information corresponding to the creator's information (such as the creator's department or position). Alternatively, the second RAG 140 may acquire review perspectives specified by the creator, or may acquire review perspectives based on the content of the loaded document.

[0060] In step S303, the second RAG 140 outputs a review instruction to the second LLM 120. Regarding the review instruction, the second RAG 140 automatically generates a prompt to be input to the second LLM 120. The prompt includes, for example, the target document acquired from the creator terminal 30 and a review perspective acquired from the database. Here, the prompt will be described.

[0061] FIG. 11 illustrates a prompt P21 for requesting a document review. FIG. 11 shows an example of a prompt automatically generated by the second RAG 140. The prompt P21 may contain, for example, the following content: "Please review the next slide based on the review criteria listed below. If there are any problems with the slide, please point out the problems and propose corrections for each review criteria." The second RAG 140 generates a prompt upon receiving a review request from the creator terminal 30. The prompt P21 includes, for example, review instructions, the document (e.g., data) to be reviewed, and review criteria (e.g., item names). The second RAG 140 inputs the review criteria searched (and used) by the creator in Section 2-2 into the prompt. Alternatively, the second RAG 140 may identify these review criteria based on the creator's specifications, tag information, history information, or the like, and input them into the prompt.

[0062] In addition, various methods are possible for the second RAG 140 to acquire review perspectives from the review perspective database 1002. In this example, when a review perspective is specified from the creator terminal 30 (i.e., when the creator wants to clearly specify a review perspective), the second RAG 140 acquires a review perspective corresponding to the specification from the database. Alternatively, the second RAG 140 may acquire a review perspective based on tag information such as business or department. On the other hand, when the creator does not specify a review perspective (i.e., when the creator cannot clearly specify a review perspective), the second RAG 140 may search for similar documents created in the past for the document to be reviewed and acquire review perspectives associated with those similar documents. Alternatively, the second RAG 140 may extract keywords and other information contained in the document to be reviewed and compare (i.e., cross-reference) this information with tag information to select review perspectives with a high matching rate (e.g., the top five review perspectives). The second LLM 120 may cooperate with the second RAG 140 to acquire review viewpoints from the review viewpoint database 1002 as necessary.

[0063] In this way, the second LLM 120 can review the creator's document in response to a prompt. The second LLM 120 outputs the review results.

[0064] Returning to Figure 10, in step S304, the second RAG 140 acquires the document review results from the second LLM 120. The data acquired by the second RAG 140 may be in any format, for example, text-based or tabular format. Here, an example of the review results output by the second LLM 120 will be described.

[0065] FIG. 12 is a diagram illustrating document review results. FIG. 12 is a schematic diagram showing the document review results output by the second LLM 120. Review R1 is an explanation summarizing the review results for the target document by review perspective. Review R1 includes reviews based on items such as visual clarity, organization of information, or clarity of title for each review perspective. The review content includes pointed out problems, suggested corrections, or good points. The review content also includes content based on the creator's prompt or review perspective.

[0066] Returning to FIG. 10 , in step S305, the second RAG 140 outputs the review results to the creator terminal 30. The creator terminal 30 presents the review results to the creator using various UIs. In this example, the creator terminal 30 presents the creator with information about the document to be reviewed and information about the review results obtained from the second LLM 120. The creator can obtain the review results via the creator terminal 30.

[0067] As a result, the information processing system 1 can review the creator's document. This ensures the quality and reproducibility of the document. It also eliminates the problem of variations in evaluation criteria between reviewers. By verbalizing quality requirements, the creator's time and effort can be reduced. The creator can also revise the document based on the obtained review results and request another review of the revised document. This is expected to further improve the quality of the document.

[0068] 3. Modifications The present invention is not limited to the above-described embodiment, and various modifications are possible. Some modifications will be described below. Two or more of the following features may be combined and applied.

[0069] (1) Information Processing System 1 The hardware configuration and network configuration of the information processing system 1 are not limited to those exemplified in the embodiment. The information processing system 1 may have any hardware configuration and network configuration as long as the required functions can be realized. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, at least a portion of the information processing device 10 may be implemented in the first LLM 110 or the second LLM 120. For example, the first LLM 110 or the second LLM 120 may have at least a portion of the functions of the first RAG 130 or the second RAG 140 related to the information processing device 10.

[0070] (2) Information Processing Device 10 Some of the functions of the information processing device 10 may be implemented in another server. This server may be, for example, a physical server or a virtual server (including a so-called cloud). Furthermore, the correspondence between functional elements and hardware is not limited to that illustrated in the embodiment. For example, in the embodiment, at least some of the functions described as being implemented in the information processing device 10 may be implemented in another device or system, or conversely, at least some of the functions described as being implemented in another device or system may be implemented in the information processing device 10. In this example, at least some of the functions of the information processing device 10 may be implemented in the administrator terminal 20 or the creator terminal 30. For example, the administrator terminal 20 or the creator terminal 30 may have at least some of the first RAG 130 or the second RAG 140, among the functions related to the information processing device 10.

[0071] (3) Administrator Terminal 20 and Creator Terminal 30 The administrator terminal 20 and creator terminal 30 are not limited to those exemplified in the embodiment. The administrator terminal 20 and creator terminal 30 may perform the above-described processes using any display screen, input device, or various UIs. The administrator terminal 20 and creator terminal 30 may be configured to be able to arbitrarily access various databases of the information processing device 10. The administrator terminal 20 may perform processes related to modifying, adding, or deleting review perspectives. The creator terminal 30 may upload or download documents. The creator terminal 30 may search for review perspectives in any way.

[0072] (4) First LLM 110 and Second LLM 120 The first LLM 110 and the second LLM 120 are not limited to those exemplified in the embodiment. At least some of the functions of the first LLM 110 and the second LLM 120 may be implemented in the information processing device 10. The first LLM 110 and the second LLM 120 may be configured using the same generation AI, or different AIs. An AI that integrates the first LLM 110 and the second LLM 120 may be introduced. Furthermore, a single LLM as a system may combine the functions of both the first LLM 110 and the second LLM 120.

[0073] (5) Review Perspective Extraction Method: The sequence chart shown in FIG. 4 merely illustrates an example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S102, the first RAG 130 may use any type of training data (i.e., documents) to extract review perspectives. The training data may be arbitrarily selected from documents classified by company, department, position, etc. The training data may be data from past documents narrowed down to an arbitrary period. Among the training data, the general review perspective may be information regarding document creation guidelines. In step S103, the first LLM 110 may extract review perspectives based on prompts acquired via an API (Application Programming Interface). The first LLM 110 may arbitrarily access various databases of the information processing device 10. The prompts generated by the first RAG 130 may be of any type. For example, an administrator may input the prompts instead of the first RAG 130.

[0074] (6) Search Method for Review Perspectives The sequence chart shown in FIG. 9 merely shows one example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S201, the information processing device 10 may acquire a search request from the administrator terminal 20 via the second RAG 140. In this case, in step S202, the second RAG 140 may execute a search for review perspectives in the database. In step S203, the creator terminal 30 may execute processes such as displaying, filtering, and sorting the search targets in addition to displaying the search results.

[0075] (7) Document Review Method The sequence chart shown in FIG. 10 merely illustrates an example of the operation, and the operation of the information processing system 1 is not limited to this. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S301, the creator terminal 30 may accept a review request (document submission) from the creator on the same display screen as used to search for review perspectives as described in Section 2-2. In step S302, the second RAG 140 may search for and acquire review perspectives for the target document based on any information. For example, search conditions may include attribute information such as the user's department, information on past review performance, review perspective search history, or information on the order of newly accumulated review perspectives. The prompt generated by the second RAG 140 may be of any type. For example, the creator may enter the prompt instead of the second RAG 140.

[0076] (8) Database (Data) The database (or the data itself) of the information processing system 1 shown in Figures 5 to 8, 11, and 12 is not limited to the example shown in the embodiment. In this example, any type of data may be registered in the database. The teacher database 1001 may have any type of information added to the document. For example, tag information, keywords, labels, etc. may be added to the document. The tag information of the review perspective database 1002 may be any type. Common tag information may be used in various databases. The review R1 may be any type, for example, text, images, videos, or audio.

[0077] (9) The various other programs executed by the processor 101 may be provided by downloading via a network such as the Internet, or may be provided in a state recorded on a computer-readable non-transitory recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0078] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of hardware and / or software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (e.g., wired, wireless, etc.) and these multiple devices. The functional block may also be realized by combining software with the single device or multiple devices.

[0079] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, consideration, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0080] For example, the information processing device 10 according to an embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0081] Each aspect or embodiment described in the present disclosure may be applied to at least one of systems using LTE (Long Term Evolution), LTE-Advanced (LTE-A), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (New Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-Wide Band), Bluetooth (registered trademark), or other suitable systems, and next-generation systems enhanced based on these. Furthermore, a combination of multiple systems (e.g., a combination of at least one of LTE and LTE-A with 5G) may also be applied.

[0082] The order of the procedures, sequences, sequence charts, etc. of each aspect or embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order and are not limited to the particular order presented.

[0083] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0084] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0085] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0086] Software, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise, should be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc. Additionally, software, instructions, information, etc. may be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then such wired and / or wireless technologies are included within the definition of a transmission medium.

[0087] The information, signals, etc. described in this disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Note that terms described in this disclosure and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meanings.

[0088] Furthermore, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information.

[0089] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0090] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0091] The "unit" in the configuration of each of the above devices may be replaced with "means," "circuit," "device," or the like.

[0092] When the terms "include," "including," and variations thereof are used in this disclosure, these terms are intended to be inclusive, similar to the term "comprising." Furthermore, when the term "or" is used in this disclosure, it is not intended to be an exclusive or.

[0093] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0094] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different."

[0095] 1...information processing system, 10...information processing device, 20...administrator terminal, 30...creator terminal, 110...first LLM, 120...second LLM, 9...network, 11...extraction unit, 12...storage unit, 13...correction unit, 14...review unit, 15...output unit, 191...storage unit, 192...control unit, 130...first RAG, 140...second RAG, 1001...teacher database, 1002...review viewpoint database, P...prompt, R...review, S...slide

Claims

1. An information processing device having an extraction unit that extracts review viewpoints for documents using a first large-scale language model from training data including document data generated in the past, and a storage unit that stores the extracted review viewpoints in a database.

2. The information processing device according to claim 1, wherein the training data includes document data of documents that have not passed review based on predetermined criteria.

3. The information processing apparatus according to claim 1, further comprising a correction unit that corrects the review viewpoints extracted by said extraction unit in accordance with a user's instruction.

4. The information processing device according to claim 1, wherein the extraction unit requests the first large-scale language model to consider predetermined candidate review viewpoints.

5. An information processing apparatus according to claim 1, further comprising: a review unit that reviews new documents using a second large-scale language model based on review viewpoints stored in said database; and an output unit that outputs review results by said review unit.

6. The information processing apparatus according to claim 5, wherein the review unit requests the second large-scale language model to evaluate the new document with respect to the review perspectives retrieved from the database.

7. An information processing method comprising the steps of: a computer extracting document review perspectives from training data including document data generated in the past using a first large-scale language model; and storing the extracted review perspectives in a database.

Images