Information processing device, information processing method, and program

Abstract

This invention provides an information processing device, an information processing method, and a program that can generate a rule set used for generating project estimates based on multiple past project data, even under conditions where the amount of input to a large-scale language model is limited. [Solution] The information processing device, information processing method, and program use a controller connected to an input / output unit that generates a rule set used for generating estimates for cases based on multiple past case data. The controller acquires multiple past case data associated with a predetermined entity via the input / output unit and divides the multiple past case data into multiple batches based on a predetermined data volume criterion. For each batch, the controller inputs the past case data belonging to that batch into a large-scale language model, acquires intermediate rule information corresponding to that batch from the large-scale language model, inputs multiple intermediate rule information into the large-scale language model, acquires a single rule set formed by integrating multiple intermediate rule information from the large-scale language model, and outputs the rule set via the input / output unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In industries that handle highly individualized projects, such as the event planning and production industry, the construction industry, and the advertising industry, business materials such as estimates are created for each project according to the requests from customers. In the work of creating business materials, it was necessary to consider a wide variety of factors, such as the content of the project, the handling in past similar projects, the relationship with the customer, and industry-specific practices.

[0003] In recent years, techniques for performing text summarization, classification, information extraction, text generation, etc. using large language models (Large Language Model; hereinafter referred to as "large language models") that are trained based on a large amount of text data have been spreading. By applying such large language models to support the creation of business materials, it has been considered to reduce the degree of dependence on the experience of the person in charge. [[ID=1十七]]

[0004] For example, Patent Document 1 discloses a technique for obtaining basic information of a customer, information related to transactions with the customer, and external information of the customer, analyzing the text data included therein to identify keywords and the context of the keywords, structuring the potential issues of the customer as structured data, selecting the company's own services according to predefined selection rules, and generating sales materials for the customer by incorporating the selection results and issues into a predefined prompt template and inputting them into a large language model.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] In the technology disclosed in Patent Document 1, selection rules that define which in-house services to select and under what circumstances, and prompt templates that define the framework for instructions to large-scale language models, are both used as predefined. These selection rules and prompt templates are intended to reflect the specific circumstances of the entity handling the case (e.g., a company, business unit, office, etc.; the same applies hereinafter). However, Patent Document 1 did not adequately consider how to develop these rules specific to such entities.

[0007] Generally, the approach to business documents, the names of items used, and the unit price levels differ from one entity to another. Therefore, if it is possible to comprehensively analyze past case data specific to a particular entity and automatically generate rule information specific to that entity, it will be possible to support the creation of business documents tailored to that entity.

[0008] In this context, a large-scale language model could be used to automatically extract rule information specific to a given entity from past case data. However, there is an upper limit to the amount of data that can be input into a large-scale language model at once. When the number of past case data associated with an entity is large, it may not be possible to input all of the past case data into the large-scale language model at once, making it difficult to extract rule information from past case data without omission.

[0009] This disclosure has been made in view of the above-mentioned issues. Its purpose is to provide an information processing device, an information processing method, and a program that can generate a rule set used for generating estimates for projects based on multiple past project data, even under conditions where the amount of input to a large-scale language model is limited. [Means for solving the problem]

[0010] The information processing device, information processing method, and program related to this disclosure use a controller connected to an input / output unit that generates a rule set used for generating estimates for cases based on multiple past case data. The controller acquires multiple past case data associated with a predetermined entity via the input / output unit and divides the multiple past case data into multiple batches based on a predetermined data volume criterion. For each batch, the controller inputs the past case data belonging to that batch into a large-scale language model, acquires intermediate rule information corresponding to that batch from the large-scale language model, inputs multiple intermediate rule information into the large-scale language model, acquires a single rule set formed by integrating multiple intermediate rule information from the large-scale language model, and outputs the rule set via the input / output unit. [Effects of the Invention]

[0011] According to this disclosure, it is possible to provide an information processing device, an information processing method, and a program that can generate a rule set used for generating estimates for projects based on multiple past project data, even under conditions where the amount of input to a large-scale language model is limited. [Brief explanation of the drawing]

[0012] [Figure 1] This figure shows the overall configuration of the information processing system related to this disclosure. [Figure 2] This is a block diagram showing the functional configuration of the information processing device related to this disclosure. [Figure 3] This is a flowchart showing the flow of the rule set generation process. [Figure 4] This diagram shows the state transitions of the rule set. [Figure 5] This flowchart shows the flow of the estimation process related to this disclosure. [Modes for carrying out the invention]

[0013] Hereinafter, an embodiment of this disclosure will be described with reference to the drawings. In each drawing, common components may be denoted by the same reference numeral and their descriptions may be omitted. Also, the dimensional ratios of the components in each drawing do not necessarily reflect the actual dimensions.

[0014] [1. Overall Configuration of the Information Processing System] Figure 1 shows the overall configuration of the information processing system 1 according to this embodiment. The information processing system 1 includes an information processing device 10, an administrator terminal 20, a user terminal 30, a large-scale language model processing device 40, an embedded model processing device 41, a vector search device 50, a relational database device 60, an object storage device 70, and a job queue device 80. These devices are connected to each other so as to be able to communicate with each other via a network N.

[0015] Information processing system 1 stores data related to past projects associated with each of several entities (hereinafter referred to as "past project data") and generates a set of rule information (hereinafter referred to as "rule set") based on the past project data. Information processing system 1 also has a function to generate information related to estimates for new projects (hereinafter referred to as "new projects") corresponding to the entity (hereinafter referred to as "estimate information") by referring to the generated rule set. In this embodiment, the past project data includes data such as the text of meeting minutes and the text of estimates related to past projects.

[0016] The information processing device 10 is a device in the information processing system 1 responsible for analyzing past case data and generating rule sets, and is implemented as a server device. The information processing device 10 may consist of a single server device, or it may consist of multiple server devices working together. Furthermore, the information processing device 10 may be implemented as a virtual server device operating on a cloud environment, or it may be implemented as a set of functions launched on a serverless computing platform.

[0017] The administrator terminal 20 is a terminal device used by an administrator who performs operations related to the registration of past case data and operations related to the generation of rule sets. The administrator terminal 20 may be, for example, a personal computer or a tablet terminal. The user terminal 30 is a terminal device used by a user such as a salesperson when generating estimate information for a new case.

[0018] The large language model processing device 40 is a device in which a large language model that generates text as a response to an input prompt operates. In the present embodiment, the large language model processing device 40 is provided in a manner that can be used through an API (Application Programming Interface) by a business operator different from the operator of the information processing system 1. However, the large language model processing device 40 may be a device operated by the operator of the information processing system 1 itself.

[0019] The embedding model processing device 41 is a device that performs processing to convert an input text into a numerical vector (hereinafter referred to as an "embedding vector") that reflects the semantic features of the text. The embedding model processing device 41 may be provided by the same business operator as the large language model processing device 40 or may be provided by a different business operator.

[0020] The vector search device 50 is a device that stores an embedding vector and meta information associated with the embedding vector and performs processing to search for an embedding vector with a high degree of similarity to a given search vector. In the present embodiment, the vector search device 50 is operated in a multi-tenant configuration having tenants corresponding to each of a plurality of entities. Therefore, the embedding vectors stored in the tenant of a certain entity are configured not to be referenced from the tenants of other entities.

[0021] The relational database device 60 is a device that manages information related to entities, past case data, new cases, rule sets, etc. as a relational database. The object storage device 70 is a device that stores files of documents such as minutes of meetings and estimates (hereinafter referred to as "original files") registered by an administrator in units of objects.

[0022] The job queue device 80 is a device that provides a queue for temporarily holding jobs to be executed asynchronously in response to requests from the information processing device 10. The information processing device 10 retrieves and executes the jobs put into the queue of the job queue device 80 by a predetermined worker process. The worker process may be implemented as a part of the information processing device 10 or as a separate server device from the information processing device 10.

[0023] [2.Hardware Configuration of Information Processing Device] The information processing device 10 includes a controller 100 and an input / output unit 90 connected to the controller 100. The controller 100 includes a processor and a memory that stores programs executed by the processor and various data. The input / output unit 90 includes a communication interface for the information processing device 10 to input and output information to and from other devices via the network N.

[0024] The controller 100 executes processing as each functional unit described later by the processor executing the program stored in the memory. When the information processing device 10 is realized by the cooperation of a plurality of server devices, part or all of each functional unit described later may be distributed and arranged in different server devices. Also, when the information processing device 10 operates on a serverless computing platform, part or all of each functional unit described later may be implemented as a function started on the platform.

[0025] [3.Functional Configuration of Information Processing Device] Figure 2 is a block diagram showing the functional configuration of the information processing device 10. The controller 100, in terms of its functional configuration, includes a first acquisition unit 110, a splitting unit 120, an extraction unit 130, an integration unit 140, a storage unit 150, a progress management unit 160, a text acquisition unit 170, a rule set acquisition unit 180, a search unit 190, a generation unit 200, and a streaming distribution unit 210.

[0026] Of these functional units, the first acquisition unit 110, the splitting unit 120, the extraction unit 130, the integration unit 140, the storage unit 150, and the progress management unit 160 are primarily involved in the generation of rule sets. The text acquisition unit 170, the rule set acquisition unit 180, the search unit 190, the generation unit 200, and the streaming distribution unit 210 are primarily involved in the generation of estimation information. Input and output of information between each functional unit and other devices is performed via the input / output unit 90.

[0027] The first acquisition unit 110 acquires multiple past case data associated with a predetermined entity from at least one of the relational database device 60 and the object storage device 70 via the input / output unit 90. The first acquisition unit 110 targets only the case data that has completed the import process described later, out of the multiple case data managed in the relational database device 60, as past case data for acquisition.

[0028] The splitting unit 120 divides the multiple past case data acquired by the first acquisition unit 110 into multiple batches based on a predetermined data volume criterion. Here, a batch refers to a unit of multiple past case data that is treated as input for a single extraction process.

[0029] The extraction unit 130 inputs past case data belonging to each of the multiple batches obtained by the division unit 120 to the large-scale language model processing unit 40 via the input / output unit 90, and obtains intermediate rule information corresponding to the batch from the large-scale language model processing unit 40. Intermediate rule information refers to a description of rule information on a batch basis at a stage before it is integrated into the final rule set.

[0030] The integration unit 140 inputs the multiple intermediate rule information acquired by the extraction unit 130 to the large-scale language model processing unit 40 via the input / output unit 90, and obtains a single rule set from the large-scale language model processing unit 40 in which the multiple intermediate rule information has been integrated. During this integration, the integration unit 140 performs processing to control the conditions for registering an item in the rule set based on the occurrence indicator for each item. The occurrence indicator will be described later.

[0031] The storage unit 150 outputs the rule set acquired by the integration unit 140 to the relational database device 60 via the input / output unit 90 for storage. The storage unit 150 stores the rule set in association with a pair of entity and rule set version.

[0032] The progress management unit 160 records progress information indicating the progress of the acquisition of intermediate rule information by the extraction unit 130 and the integration by the integration unit 140 in a relational database device 60, which is an example of a storage device, via the input / output unit 90. The progress information may include the total number of batches, the number of batches for which the acquisition of intermediate rule information has been completed, the intermediate rule information acquired for each batch, and a value indicating the status.

[0033] The text acquisition unit 170 acquires text information representing the content of a new project (hereinafter referred to as "project text") from inputs such as meeting minutes and customer interaction records related to the new project.

[0034] The rule set acquisition unit 180 acquires the latest rule set, which has been generated, from the relational database device 60 via the input / output unit 90 for the entity corresponding to the new case.

[0035] The search unit 190 obtains a search vector from the embedded model processing device 41 based on the case text or a part thereof acquired by the text acquisition unit 170, and uses the search vector to search for past case data similar to the new case from the tenant corresponding to the subject in the vector search device 50.

[0036] The generation unit 200 inputs a prompt, which includes a system prompt containing the rule set acquired by the rule set acquisition unit 180, and a user prompt containing the case text acquired by the text acquisition unit 170 and past case data searched by the search unit 190, to the large-scale language model processing unit 40 via the input / output unit 90, causing the large-scale language model processing unit 40 to generate estimation information for the new case.

[0037] The streaming distribution unit 210 sequentially distributes the content output sequentially from the large-scale language model processing device 40 during the generation of estimation information by the generation unit 200 to the user terminal 30 via the input / output unit 90. In this embodiment, the streaming distribution unit 210 may perform the distribution using the Server-Sent Events (hereinafter referred to as "SSE") method.

[0038] [4. Data Structure] The relational database device 60 stores at least the following tables: a subject management table that manages the subject's identification information and name, and a case data table that manages the case data's identification information, the subject's identification information, a value indicating whether the case data is past case data or a new case, a value indicating the status of the import process, the text of the meeting minutes, the text of the quotation, and the text of the summary, etc.

[0039] Furthermore, the relational database device 60 stores a rule set management table that manages rule set identification information, subject identification information, version, status values, rule set text, and progress information. In the rule set management table, constraints are placed on uniqueness to prevent duplication of pairs of subject identification information and version.

[0040] Furthermore, the relational database device 60 may store a similar case management table that records past case data retrieved by the search unit 190 for new cases. The similar case management table may record identification information for new cases, identification information for retrieved past case data, a score indicating similarity, and a ranking, etc.

[0041] For example, four types of values ​​may be used to indicate the status of the import process in the case data table: "Not processed," "In processing," "Completed," and "Failed." For example, five types of values ​​may be used to indicate the status in the rule set management table: "Waiting for generation," "Extracting," "Integrating," "Completed," and "Failed."

[0042] The object storage device 70 stores original files uploaded from the administrator terminal 20, etc. Each original file is associated with the subject's identification information and the case data's identification information. In the vector search device 50, the subject's identification information is used as the tenant identifier, and each tenant stores an embedded vector obtained by converting the text of the case data summary, associated with metadata containing the case data's identification information.

[0043] [5. Importing past project data] The process for importing past case data will now be explained. The administrator uploads the original files of past cases related to the subject to the information processing device 10 via the administrator terminal 20. The information processing device 10 stores the received original files in the object storage device 70 and registers the record related to the case data in the case data table of the relational database device 60, along with a value indicating the status as "unprocessed".

[0044] Next, the information processing device 10 submits the job to the queue of the job queue device 80. When the worker process retrieves the job, it changes the value indicating the status of the target case data to "processing," retrieves the original file from the object storage device 70, and extracts text from the original file. This extraction process is carried out using a method appropriate to the format of the original file (for example, PDF (Portable Document Format), text format, or electronic spreadsheet file format). The extracted text is stored in the case data table as meeting minutes text or quotation text.

[0045] Next, the worker process inputs the extracted text into the large-scale language model processing unit 40, which generates a summary text suitable for search purposes. The generated summary text is stored in the case data table.

[0046] Next, the worker processing inputs the summary text to the embedding model processing device 41 to obtain an embedding vector, and stores the embedding vector, along with the metadata of the case data, in the tenant of the vector search device 50 corresponding to the subject's identification information.

[0047] Once the above processes are complete, the worker process changes the value indicating the status of the case data to "Completed". If an error occurs at any stage, the worker process changes the value indicating the status of the case data to "Failed" and records information indicating the nature of the error in the relational database device 60. Case data in the "Failed" state may be subject to a job submission for re-import based on an operation from the administrator terminal 20.

[0048] [6. Rule set generation process] Next, we will explain the process of generating a rule set. Figure 3 is a flowchart showing the flow of the rule set generation process. This process is performed by the controller 100 while communicating with each device via the input / output unit 90.

[0049] The administrator selects the target entity on the screen displayed on the administrator terminal 20 and then performs an operation to instruct the start of rule set generation. Upon receiving this operation, the information processing device 10 registers a new record in the rule set management table of the relational database device 60, along with a value indicating the "pending generation" status (step S101). At this time, the storage unit 150 assigns a value as the new version that is the value of the highest version among the records already registered for that entity plus 1. If no records are registered for that entity, the value of the initial version is assigned.

[0050] Next, the information processing device 10 submits a job related to the generation of a rule set to the queue of the job queue device 80 (step S102). This job includes at least identification information of the target rule set. In this embodiment, the process of acquiring intermediate rule information by the extraction unit 130 and the process of acquiring a single rule set by the integration unit 140 are both executed as asynchronous jobs submitted via the queue of the job queue device 80.

[0051] The worker process retrieves a job from the queue of the job queue device 80. Subsequently, the first acquisition unit 110 identifies the subject's identification information from the identification information of the rule set included in the job, and acquires only the case data from the relational database device 60 via the input / output unit 90 as past case data, where the value indicating the status of the import process is "completed" among the multiple case data associated with that subject (step S103). This makes it possible to suppress the disruption to the generation of intermediate rule information caused by case data that is in an incomplete or failed import process.

[0052] Next, the splitting unit 120 divides the multiple past case data acquired by the first acquisition unit 110 into multiple batches based on a predetermined data volume criterion (step S104).

[0053] In this embodiment, the data volume criterion is defined as an upper limit on the total number of characters in past case data belonging to a batch. This upper limit may be a predetermined value based on the upper limit of the amount of data that the large-scale language model processing device 40 can accept as a single input, and the conversion ratio between the number of characters and the amount of data.

[0054] The splitting unit 120 adds past case data one by one to the batch being compiled. If the total number of characters in the past case data belonging to the batch exceeds the upper limit, the past case data is not added to the batch being compiled, and a new batch is compiled.

[0055] Furthermore, the splitting unit 120 generates a batch consisting only of past case data that would exceed the upper limit if included individually. This allows past case data with a large number of characters to be included in the acquisition of intermediate rule information without being excluded from processing.

[0056] The splitting unit 120 records the total number of batches generated, the identification information of past case data included in each batch, and the total number of characters in each batch as progress information in the relational database device 60. Hereinafter, the total number of batches generated by the splitting will be referred to as the total number of batches.

[0057] Following step S104, the information processing device 10 changes the value indicating the status of the target rule set to "Extracting" (step S105).

[0058] Next, the extraction unit 130 sequentially acquires intermediate rule information for each of the multiple batches (step S106). Specifically, the extraction unit 130 generates a prompt by incorporating the contents of past case data belonging to the batch into a predetermined template for extraction prompts, and inputs the prompt to the large-scale language model processing unit 40. The template for extraction prompts includes instructions that rule information associating conditions with items should be output as intermediate rule information.

[0059] Intermediate rule information may be output in the form of a table with columns for, for example, "Conditions," "Item," "Unit Price," "Unit," and "Remarks." The "Conditions" column contains text indicating the conditions under which the item will be included in the estimate, and the "Item" column contains the name of the item that may be included in the estimate.

[0060] When the extraction unit 130 receives intermediate rule information from the large-scale language model processing unit 40, it records the intermediate rule information for that batch and the number of batches for which the acquisition of intermediate rule information has been completed as progress information in the relational database device 60 via the progress management unit 160.

[0061] If the extraction unit 130 fails to obtain intermediate rule information, it may retry up to a predetermined number of times. The interval between retries may be set to a waiting time that increases with the number of retries. If the intermediate rule information cannot be obtained even after the predetermined number of retries, the extraction unit 130 treats the processing for that batch as a failure.

[0062] Furthermore, if a predetermined number of errors (for example, 5 times) occur consecutively, the extraction unit 130 may change the value indicating the status of the target rule set to "failed" and terminate the process. This configuration makes it possible to suppress an increase in processing load and communication volume when the large-scale language model processing unit 40 is in an unresponsive state, etc.

[0063] [6-1. Control of Interruption and Resumption] In the process of acquiring intermediate rule information for each of the multiple batches, the extraction unit 130 interrupts the process if a predetermined interruption condition is met when the acquisition of intermediate rule information for some of the batches has been completed. Then, based on the progress information recorded in the relational database device 60, the extraction unit 130 makes it possible to resume processing from the batch for which intermediate rule information has not yet been acquired.

[0064] In this embodiment, the interruption condition includes the arrival of an elapsed time that has a predetermined margin relative to the maximum execution time defined in the execution environment on which the information processing device 10 operates. For example, if the information processing device 10 is implemented as a function that operates on a serverless computing platform, and the maximum time allowed for the execution of a single function is defined as 15 minutes, the extraction unit 130 may interrupt processing 10 minutes after the function is started. This value of 10 minutes is just an example, and other values ​​may be used depending on the maximum execution time defined in the execution environment, the average time required for processing per batch, etc.

[0065] If the interruption condition is met, the extraction unit 130 waits for the acquisition of intermediate rule information for the batch being processed to be completed before interrupting the process. The intermediate rule information acquired up to the point of interruption is recorded as progress information in the relational database device 60 and can be referenced even after the interruption.

[0066] When the extraction unit 130 interrupts processing, it resubmits a new job with identification information that identifies the unprocessed batch to the queue of the job queue device 80. This identification information may be, for example, an index indicating the unprocessed batch. When the worker process retrieves the job, it reads progress information from the relational database device 60 and resumes processing from the unprocessed batch identified by the identification information.

[0067] This control over interruption and resumption allows processing that would otherwise fall outside the maximum execution time defined in the execution environment to be completed across multiple divided execution units. Furthermore, because progress information is recorded in the relational database device 60, even if interruptions and resumptions are repeated, it is possible to suppress duplicate execution of processing for the same batch. Once the extraction unit 130 has finished acquiring intermediate rule information for all batches, it moves on to the next process.

[0068] [6-2. Integration Processing] Once the acquisition of intermediate rule information for all batches is complete, the information processing device 10 changes the value indicating the status of the target rule set to "Integrating" (step S107). Subsequently, the integration unit 140 reads the intermediate rule information for all batches recorded as progress information and performs the process of integrating them (step S108). The integration unit 140 generates a prompt by incorporating the multiple intermediate rule information into a predetermined template for integration prompts, and inputs the prompt to the large-scale language model processing device 40.

[0069] The template for the integration prompt includes two instructions. The first instruction states that all items appearing in each intermediate rule information should be comprehensively identified, and that items with similar names should be treated as the same item.

[0070] The second instruction states that, for a given item, the conditions for registering that item in the rule set should be controlled based on an occurrence index that indicates the proportion of batches from which intermediate rule information containing that item has been extracted. In this way, the integration unit 140 controls conditions based on occurrence index by inputting prompts to the large-scale language model processing device 40 that include instructions regarding the handling of items according to the occurrence index.

[0071] Specifically, the second instruction includes the following: if an item's occurrence index is equal to or greater than the first threshold, it is permitted to register that item in the rule set as an unconditional rule without any conditions; and if an item's occurrence index is less than the first threshold, it is permitted to register that item in the rule set as a conditional rule with at least one condition.

[0072] In this embodiment, the first threshold may be a value corresponding to the total number of items in the batch. That is, registration as an unconditional rule without any conditions is permitted only for items that appear in all batches.

[0073] In addition, the second instruction may include the following: if the occurrence index for a given item is below the second threshold, then at least one condition extracted from the intermediate rule information that includes that item must be attached to the item and registered in the rule set as a conditional rule.

[0074] In this embodiment, the second threshold may be a value corresponding to 50% of the total number of items in the batch. That is, items that appear in less than half of the batch will not be treated as unconditional rules under any circumstances, and at least one condition will be attached to them. This prevents items that can only be accounted for under specific conditions from being overgeneralized as rules common to all cases.

[0075] The first and second thresholds are not limited to the values ​​described above, and different values ​​may be used depending on operational requirements. For example, the first threshold may be set to a value corresponding to 80% of the total number of items in the batch, and the second threshold may be set to a value corresponding to 30% of the total number of items in the batch. Alternatively, multiple thresholds may be combined to divide the range of possible occurrences of the indicator into multiple categories, and different treatment may be defined for each category.

[0076] Furthermore, the determination of item identity used in calculating the occurrence index may be performed not only by a complete match of names, but also by semantic identity determination by the large-scale language model processing device 40, or by determination using a pre-prepared dictionary for item normalization.

[0077] The integration unit 140 inputs the above prompt to the large-scale language model processing unit 40 and obtains a single integrated rule set as a response. In this embodiment, the rule set is obtained as text in Markdown format.

[0078] The rule set may include, for example, a section listing conditional rules that associate conditions with items, a section listing items that are prone to being overlooked in accounting, and a section listing items that are not to be included in the rule set. If the integration process fails, the information processing device 10 changes the value indicating the status of the target rule set to "failed". If the integration process is successful, the storage unit 150 outputs the acquired rule set to the target record in the rule set management table of the relational database device 60 via the input / output unit 90 and saves it, and changes the value indicating the status to "completed" (step S109).

[0079] [6-3. State Transitions] Figure 4 shows the state transitions of a rule set. The "Waiting for generation" state is the state after a request for rule set generation has been received, until worker processing begins. Once worker processing begins, the state transitions to "Extracting". In the "Extracting" state, the acquisition of intermediate rule information in batches proceeds. Even if the processing is interrupted due to the fulfillment of an interruption condition and then resumed, the state remains "Extracting".

[0080] Once the acquisition of intermediate rule information is complete for all batches, the status transitions to "Integrating". When the integration process is completed in the "Integrating" state, the status transitions to "Completed". On the other hand, if the integration process fails, or if consecutive errors occur in the "Extracting" state, the status transitions to "Failed". Rule sets in the "Failed" state may be submitted for resubmission based on administrator actions.

[0081] [6-4. Managing Ruleset Versions] The rule set is stored in the relational database device 60, associated with the subject's identification information and the version of the rule set. As described above, in step S101, the newly generated rule set is assigned a new version different from the version of the rule set already generated for that subject. Therefore, multiple versions of the rule set can be stored for the same subject.

[0082] Rule sets of previously generated versions are not overwritten by new rule sets, but are retained as historical references. In the estimate generation process described later, the rule set of the record with the highest version among the records where the status value for the target entity is "Completed" is used. This ensures that the latest rule set is used, while also allowing reference to past rule set versions as needed.

[0083] [7. Estimate Generation Process] Next, we will explain the process of generating estimate information for new projects using the generated rule set. Figure 5 is a flowchart showing the flow of the estimate generation process.

[0084] The user registers a new project via the user terminal 30 and inputs text such as meeting minutes related to the new project (step S201). When the information processing device 10 receives a request from the user terminal 30 to generate estimate information, it starts an asynchronous job related to the generation of estimate information.

[0085] The worker process, via the rule set acquisition unit 180, acquires the latest version of the rule set from the relational database device 60 for the entity handling the new case, whose status value is "completed" (step S202).

[0086] Next, the text acquisition unit 170 acquires the case text of a new case, and the search unit 190 inputs the case text or a part thereof into the embedded model processing device 41 to obtain a search vector (step S203).

[0087] Next, the search unit 190 uses the acquired search vector to search for a predetermined number of past case data with high similarity from tenants corresponding to the target entity of the vector search device 50 (step S204). The search unit 190 may record the identification information and similarity score of the past case data obtained as a result of the search in a similar case management table.

[0088] Next, the search unit 190 retrieves the text of the meeting minutes or the text of the quotation for past case data obtained as a result of the search from the relational database device 60 (step S205).

[0089] Next, the generation unit 200 constructs a prompt for generating estimate information (step S206). The prompt includes a system prompt and a user prompt. The system prompt includes a description indicating the role of the person responsible for creating the estimate, the body of the rule set acquired by the rule set acquisition unit 180, the specifications of the output format (for example, a schema in the format of JSON (JavaScript Object Notation)), and instructions that the rules belonging to a specific category within the rule set should be finally reviewed.

[0090] The user prompt includes the text of the meeting minutes for the new project, the response to additional confirmations from the customer, and the text of the summary of past project data and the quotation obtained by the search unit 190. The generation unit 200 inputs the constructed prompt to the large-scale language model processing unit 40. In this embodiment, the large-scale language model processing unit 40 may have a function to return the internal reasoning process as an intermediate output, and the generation unit 200 sequentially receives the reasoning process and a portion of the quotation information.

[0091] The streaming distribution unit 210 sequentially distributes the content received sequentially by the generation unit 200 to the user terminal 30 using the SSE method (step S207). On the screen of the user terminal 30, the inference process and parts of the estimate information in the process of generation are sequentially displayed even before the generation of the estimate information is completed.

[0092] Once the generation of the quotation information is complete, the generation unit 200 analyzes the received quotation information and converts it into a predetermined structured data format (step S208). In this embodiment, the quotation information is configured as structured data having a hierarchical structure that includes multiple categories and one or more items belonging to each category. For each item, at least one of the following may be set: item name, unit price, quantity, unit, amount, description, and a value indicating importance.

[0093] The generation unit 200 stores the structured estimate information in the relational database device 60 and sends a completion notification to the user terminal 30 (step S209). The user can check the generated estimate information through the screen of the user terminal 30 and make corrections as needed.

[0094] In the above estimation generation process, estimation information is generated by referencing both a rule set specific to the entity and past project data specific to that entity. The rule set provides a formal framework common to all entities, while the retrieved past project data provides unit prices and descriptions specific to individual projects. Therefore, estimation information can be obtained that reflects the unique characteristics of the entity while also addressing the characteristics of individual new projects.

[0095] [8. Variation] Although one embodiment of the present disclosure has been described above, the present disclosure is not limited to the above embodiment. For example, in the above embodiment, the device for generating estimate information and the device for generating rule sets were described as the same information processing device 10, but these may be configured as separate devices. In this case, the device that performs the process of generating rule sets and the device that performs the process of generating estimate information may be configured to exchange rule set information via a relational database device 60.

[0096] Furthermore, while the above embodiments cited meeting minutes and quotations in the event planning and production industry as primary examples of past project data, the scope of application of this disclosure is not limited to these. The same configuration can be applied to project data in a wide range of fields, such as quotations, proposals, and designs in the construction, advertising, or manufacturing industries.

[0097] Furthermore, although the above embodiment describes a configuration in which the large-scale language model processing device 40 is provided in a manner that is available through an API, the large-scale language model processing device 40 may be a device installed on the same network as the information processing device 10. In this case, it becomes possible to complete the processing without sending information about the subject to the outside.

[0098] Furthermore, although the above embodiment mainly described an example in which intermediate rule information is obtained by the large-scale language model processing device 40, the method of obtaining intermediate rule information is not limited to this. For example, a process that extracts keywords and numerical values ​​from past case data based on predetermined rules may be used, and this process may be used in combination with the process by the large-scale language model processing device 40.

[0099] Furthermore, in the above embodiment, the appearance index was assumed to be the value obtained by dividing the number of batches in which the item appeared by the total number of batches. However, the appearance index may be any other index as long as it is a value that quantitatively indicates the extent to which the item appeared in a batch.

[0100] Alternatively, the above-described processing may be performed by recording a program for realizing the processing of each functional unit of the information processing device 10 in the above embodiment on a computer-readable recording medium, and then having a computer system read and execute the program recorded on the recording medium. The computer system referred to here includes the OS (Operating System) and hardware such as peripheral devices.

[0101] [Effects of the Embodiment] As described in detail above, the information processing device, information processing method, and program related to this disclosure use a controller connected to an input / output unit that generates a rule set used for generating estimates for cases based on multiple past case data. The controller acquires multiple past case data associated with a predetermined entity via the input / output unit and divides the multiple past case data into multiple batches based on a predetermined data volume criterion. For each batch, the controller inputs the past case data belonging to that batch into a large-scale language model, acquires intermediate rule information corresponding to that batch from the large-scale language model, inputs multiple intermediate rule information into the large-scale language model, acquires a single rule set formed by integrating multiple intermediate rule information from the large-scale language model, and outputs the rule set via the input / output unit.

[0102] This makes it possible to generate a set of rules used to generate project estimates based on multiple past project data, even under conditions where the amount of input to a large-scale language model is limited.

[0103] Multiple historical case data sets are divided into multiple batches based on predetermined data volume criteria, and input into the large-scale language model is performed on a batch-by-batch basis. This limits the amount of data input into the large-scale language model at one time to a range corresponding to the data volume criteria. Therefore, even if there is an upper limit to the amount of data that can be input into the large-scale language model at one time, all of the multiple historical case data sets associated with a subject can be processed, and intermediate rule information can be obtained across the entire set of historical case data.

[0104] Furthermore, the intermediate rule information obtained separately for each batch is combined into a single rule set that reflects the overall content of past case data.

[0105] The data volume criterion may be defined as an upper limit on the total number of characters in past case data belonging to a batch. In the splitting process, the controller may generate a batch consisting only of past case data that would exceed the upper limit if included individually.

[0106] This allows for the splitting of multiple past case data into multiple batches using a simple process: accumulating the number of characters in each past case data and comparing the sum to an upper limit. Since it does not require actually inputting data into a large-scale language model to determine whether splitting is possible, the processing load required for splitting can be reduced. Furthermore, by managing the total number of characters in the past case data belonging to each batch based on the upper limit, it is possible to prevent the amount of data input into the large-scale language model at one time from becoming excessive.

[0107] On the other hand, among multiple past case data sets, there may be at least one case whose character count exceeds the upper limit. If such past case data is included in a single batch along with other past case data, the amount of data belonging to that batch will increase even further.

[0108] In this configuration, past case data exceeding the upper limit is treated as a batch consisting only of that past case data, excluding other past case data. This allows past case data with a large number of characters to be included in the acquisition of intermediate rule information without being excluded from processing, and the amount of data included in the batch can be kept to the bare minimum.

[0109] The controller may record progress information indicating the progress of acquiring intermediate rule information for multiple batches in a storage device via an input / output unit. If a predetermined interruption condition is met when the acquisition of intermediate rule information for some of the batches is complete, the controller may interrupt processing and, based on the progress information recorded in the storage device, allow processing to be resumed from the batches for which intermediate rule information has not yet been acquired.

[0110] This allows for the acquisition of intermediate rule information for all batches to be completed through multiple processing cycles, even in execution environments where the time or resources allocated to a single process are limited. Furthermore, because batches for which intermediate rule information has not yet been acquired are identified based on the progress information, it is possible to prevent duplicate execution of processing for batches for which acquisition has already been completed. In addition, it is possible to suppress unnecessary increases in processing load and the amount of communication with large language models.

[0111] The interruption conditions may include the arrival of an elapsed time that has a predetermined margin relative to the maximum execution time defined in the execution environment in which the information processing device operates.

[0112] As a result, processing is interrupted before the maximum execution time elapses and the execution environment forcibly terminates the process. The time corresponding to this margin can be used to complete the acquisition of intermediate rule information for the batch being processed, record the progress information in storage, and prepare to resume processing for the unprocessed batches. Therefore, it is possible to prevent the process from being terminated prematurely due to forced termination upon the expiration of the maximum execution time, and to save the progress of acquiring intermediate rule information before interrupting and resuming the process.

[0113] The controller may execute the processes for acquiring intermediate rule information and acquiring rule sets as asynchronous jobs submitted via a queue, and when interrupting the process in response to the fulfillment of an interruption condition, it may resubmit the jobs to the queue with identification information that identifies the unprocessed batches.

[0114] As a result, resuming an interrupted process is achieved by having a worker retrieve the job from the queue and process the unprocessed batch identified by the relevant identification information, just as with the initial processing. Therefore, without requiring a separate special mechanism for interrupting and resuming processing, the interruption and resumption of processing can be handled uniformly by utilizing the mechanism for executing asynchronous jobs.

[0115] The controller outputs a rule set associated with a subject and a version of the rule set, and the output rule set may be assigned a new version different from the version of the rule set already generated for that subject.

[0116] This ensures that even when a new rule set is generated for the same entity, the previously generated rule set is not overwritten by the new rule set, but is maintained in a state of mutual distinction based on the entity and version pair. Therefore, multiple versions of rule sets can be maintained for the same entity, allowing the use of the latest version of the rule set while referencing past versions as needed.

[0117] A rule set may contain rule information that associates conditions with items. When retrieving a rule set, the controller may control the conditions for registering an item in the rule set based on an occurrence metric that indicates the percentage of batches in which intermediate rule information containing that item is extracted.

[0118] The occurrence metric can be used to indicate how commonly an item appears across multiple batches, that is, how widely the item appears in the overall historical transaction data. This allows for different conditions to be used when registering items in the rule set for items that appear in many batches and items that appear in only some batches.

[0119] Therefore, it is possible to prevent items that may only be accountable in specific cases from being overgeneralized as items that apply to all cases, and to obtain a rule set that reflects the trends of items that have appeared in past case data.

[0120] The controller may, when acquiring a rule set, allow an item to be registered in the rule set as an unconditional rule without conditions if its occurrence indicator is equal to or greater than a first threshold, and register an item in the rule set as a conditional rule with at least one condition if its occurrence indicator is less than the first threshold.

[0121] This allows for the registration of unconditional rules only for items that appear widely across multiple batches and are considered applicable to all cases. On the other hand, for items with a limited scope of appearance, at least one condition is always associated with them, thus preventing items that should only be accounted for under specific conditions from being treated as rules applicable to all cases. Therefore, the presence or absence of conditions associated with an item can be determined based on a clear criterion: a comparison between the appearance indicator and the first threshold.

[0122] In acquiring the rule set, the controller may also, if the occurrence indicator for a certain item is below a second threshold, add at least one condition extracted from the intermediate rule information that includes that item to the rule set as a conditional rule.

[0123] As a result, items with a limited range of appearance will not be registered in the rule set without conditions attached, and the conditions attached to such items will be supported by intermediate rule information based on past transaction data in which the item actually appeared. Therefore, for items that can only be accounted for in specific cases, the conditions for their accountability can be defined as specific details that are in line with the circumstances that appeared in past transaction data, thereby preventing the item from being accounted for when it is not appropriate.

[0124] The controller may include instructions regarding how to handle items based on their occurrence indicators in the prompts it inputs to the large-scale language model when retrieving the rule set.

[0125] As a result, control of conditions based on occurrence indicators is achieved by the large-scale language model generating a single rule set in accordance with those instructions. Therefore, it is not necessary to perform separate post-processing to reflect the handling based on occurrence indicators in the rule set output from the large-scale language model, and the integration of multiple intermediate rule information and control of conditions according to occurrence indicators can be performed together as a series of processes.

[0126] Furthermore, since the standards for handling items are defined as part of the instructions included in the prompt, these standards can be established without changing the program itself that defines the processing procedures.

[0127] Although several embodiments have been described, it is possible to modify or transform the embodiments based on the above disclosure. All components of the above embodiments, and all features described in the claims, may be taken individually and combined, provided that they do not conflict with each other. [Explanation of Symbols]

[0128] 1. Information Processing System 10 Information Processing Devices 20 Administrator terminals 30 User terminals 40 Large-scale language model processing unit 41 Embedded Model Processing Device 50 Vector Search Devices 60 Relational Database Devices 70 Object Storage Devices 80 Job Queue Device 90 Input / output section 100 controllers 110 First acquisition part 120 Division 130 Extraction part 140 Integration Department 150 Preservation Department 160 Progress Management Department 170 Text acquisition unit 180 Rule Set Acquisition Section 190 Search section 200 Generation part 210 Streaming Distribution Department N Network

Claims

1. An information processing device that generates a set of rules used to generate estimates for a project based on data from multiple past projects, It comprises an input / output unit and a controller connected to the input / output unit, The aforementioned controller, Through the input / output unit, multiple past case data associated with a predetermined entity are acquired. Multiple past case data sets are divided into multiple batches based on a predetermined data volume criterion. For each of the aforementioned batches, an extraction prompt is generated by incorporating the content of the past case data belonging to that batch into an extraction prompt template that includes instructions to output rule information as intermediate rule information, which associates conditions with items, and by inputting the extraction prompt into the large-scale language model, the intermediate rule information, which is rule information that associates conditions with items, is obtained from the large-scale language model. Multiple pieces of the aforementioned intermediate rule information are incorporated into a template for an integration prompt that includes instructions to comprehensively identify all items appearing in each of the aforementioned intermediate rule information and to treat items with similar names as the same item. This template is then used to generate an integration prompt, and this integration prompt is input into the large-scale language model. As a result, a single rule set, which is an integrated version of the multiple pieces of the aforementioned intermediate rule information, is obtained from the large-scale language model. The rule set is output via the input / output unit. Information processing device.

2. The aforementioned data volume standard is defined as an upper limit on the total number of characters in past case data belonging to a batch. In the division, the controller generates a batch consisting only of past case data for any past case data that exceeds the upper limit when a single past case data is included. The information processing apparatus according to claim 1.

3. The aforementioned controller, Each time the intermediate rule information is acquired for the batch, progress information identifying the batch for which the acquisition of the intermediate rule information has been completed is recorded in the storage device via the input / output unit. If a predetermined interruption condition is met when the acquisition of the intermediate rule information for some of the batches is completed, the process is interrupted by terminating without executing the process to acquire the intermediate rule information for the batches for which the intermediate rule information has not yet been acquired at that stage. The process involves reading the progress information recorded in the storage device to identify batches for which the intermediate rule information has not yet been acquired, and then executing a process to acquire the intermediate rule information for the identified batches, thereby restarting processing from the batches for which the intermediate rule information has not yet been acquired. The information processing apparatus according to claim 1.

4. The information processing device according to claim 3, wherein the interruption condition includes the arrival of an elapsed time having a predetermined margin with respect to the maximum execution time defined in the execution environment in which the information processing device operates.

5. The information processing apparatus according to claim 3, wherein the controller executes the process of acquiring the intermediate rule information and the process of acquiring the rule set as asynchronous jobs submitted via a queue, and when the process is interrupted in response to the fulfillment of the interruption condition, it resubmits a job to the queue with identification information that identifies the unprocessed batch.

6. The controller outputs the rule set in association with the subject and the version of the rule set. The resulting rule set will have a new version that differs from the version of the rule set already generated for that entity. The information processing apparatus according to claim 1.

7. The aforementioned rule set includes rule information that associates conditions with items, The controller controls the conditions based on the occurrence index by including in the integration prompt an instruction that, for a given item, conditions should be defined for registering that item in the rule set based on an occurrence index that shows the ratio of the number of batches from which the intermediate rule information containing that item is extracted to the total number of batches. The information processing apparatus according to any one of claims 1 to 6.

8. The instructions included in the integration prompt are: If the occurrence indicator for a certain item is equal to or greater than the first threshold, the item may be registered in the rule set as an unconditional rule without any conditions. If the occurrence index for a given item is less than the first threshold, that item should be registered in the rule set as a conditional rule with at least one condition. Includes provisions that define, The information processing apparatus according to claim 7.

9. The instruction further includes a provision that, if the occurrence index for a certain item is less than or equal to a second threshold, the item should be registered in the rule set as a conditional rule, with at least one condition extracted from the intermediate rule information including the item. The information processing apparatus according to claim 8.

10. An information processing method executed by a controller connected to an input / output unit, which generates a rule set used for generating estimates for a project based on data from multiple past projects, The aforementioned controller, Through the input / output unit, multiple past case data associated with a predetermined entity are acquired. Multiple past case data sets are divided into multiple batches based on a predetermined data volume criterion. For each of the aforementioned batches, an extraction prompt is generated by incorporating the content of the past case data belonging to that batch into an extraction prompt template that includes instructions to output rule information as intermediate rule information, which associates conditions with items, and by inputting the extraction prompt into the large-scale language model, the intermediate rule information, which is rule information that associates conditions with items, is obtained from the large-scale language model. Multiple pieces of the aforementioned intermediate rule information are incorporated into a template for an integration prompt that includes instructions to comprehensively identify all items appearing in each of the aforementioned intermediate rule information and to treat items with similar names as the same item. This template is then used to generate an integration prompt, and this integration prompt is input into the large-scale language model. As a result, a single rule set, which is an integrated version of the multiple pieces of the aforementioned intermediate rule information, is obtained from the large-scale language model. The rule set is output via the input / output unit. Information processing methods.

11. A program executed by a controller connected to an input / output unit, which generates a set of rules used to generate estimates for projects based on data from multiple past projects, The aforementioned controller, The steps include: acquiring a plurality of past case data associated with a predetermined entity via the input / output unit; The steps include dividing the aforementioned past case data into multiple batches based on a predetermined data volume criterion, For each of the aforementioned batches, the following steps are taken: generating an extraction prompt by incorporating the content of the past case data belonging to that batch into an extraction prompt template that includes instructions to output rule information as intermediate rule information, which associates conditions with items; and obtaining the intermediate rule information, which is rule information associating conditions with items, from the large-scale language model by inputting the extraction prompt into the large-scale language model; The steps include: generating an integration prompt by incorporating multiple intermediate rule information into an integration prompt template that includes instructions to comprehensively identify items appearing in each of the intermediate rule information and to treat items with similar names as the same item, and obtaining a single rule set from the large-scale language model by inputting the integration prompt into the large-scale language model; The steps include outputting the rule set via the input / output unit, A program that includes this.

Images