Information processing device, labeling method, and labeling program

Abstract

This makes it possible to analyze the series of tasks involved in creating deliverables in accordance with the actual state of those tasks. [Solution] The information processing device includes a creation location identification unit that identifies a portion of a deliverable created by a series of operations, which is created by a partial operation, and a labeling unit that determines a label to be assigned to the partial operation using a language model trained on natural language based on the location identified by the creation location identification unit. This information processing device makes it possible to visualize the contents of a series of operations, identify bottleneck operations, and ultimately optimize the entire series of operations.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a labeling method, and a labeling program.

Background Art

[0002] Techniques for supporting the efficiency improvement of various operations are known. As an example of a technique for supporting operation efficiency improvement, for example, there is an operation efficiency improvement support system described in Patent Document 1 below. This operation efficiency improvement support system classifies operation data indicating the content, person in charge, and deliverables of an operation into a plurality of work processes to create a current operation flow. Then, this operation efficiency improvement support system compares the current operation flows for a plurality of different operations and creates a standard operation flow consisting of standard work processes that are ideal work processes.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As performed in the operation efficiency improvement support system described in Patent Document 1, as a measure for improving the efficiency of an operation, it is effective to subdivide and analyze a series of operations in the operation. However, it is not easy to subdivide and analyze the operations in consideration of the specific operation content. For example, in the above operation efficiency system, the work processes are classified based on the attribute information input by the user. Therefore, if the input attribute information is not appropriate, it is considered highly likely that a standard operation flow not in line with the actual operation state will be created, and the effect of efficiency improvement will be limited.

[0005] This disclosure has been made in view of the above-mentioned issues, and one exemplary purpose thereof is to provide a technology that enables the analysis of a series of tasks involved in creating a deliverable in accordance with the actual nature of those tasks. [Means for solving the problem]

[0006] An information processing device relating to an exemplary aspect of this disclosure includes: a means for identifying a production location in a deliverable created by a series of operations, which is a part of the series of operations; and a labeling means for determining a label to be assigned to the part of the operations using a language model trained on natural language based on the location identified by the production location identification means.

[0007] In an illustrative aspect of the labeling method relating to this disclosure, at least one processor performs a creation location identification process to identify a portion of an output created by a series of operations, which is a part of the series of operations; and a labeling process to determine a label to be assigned to the portion of the operations using a language model trained on natural language based on the location identified in the creation location identification process.

[0008] A labeling program relating to an illustrative aspect of this disclosure causes a computer to function as a means for identifying a location in a deliverable created by a series of operations, which is a part of the series of operations, and a labeling means for determining a label to be assigned to the part based on the location identified by the means for identifying a location, using a language model trained on natural language. [Effects of the Invention]

[0009] One illustrative aspect of this disclosure is that it provides a technology that enables the analysis of a series of tasks involved in creating a deliverable in accordance with the actual circumstances of those tasks. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing the configuration of the information processing device related to this disclosure. [Figure 2] This is a flowchart showing the flow of the labeling method related to this disclosure. [Figure 3] This is a block diagram showing the configuration of other information processing devices related to this disclosure. [Figure 4] Figure 3 is a diagram illustrating the overview of the processes performed by the information processing device shown. [Figure 5] This diagram shows an example of labeling. [Figure 6] This figure shows an example of a display screen that the display control unit will show. [Figure 7] Figure 3 is a flowchart showing the processing flow executed by the information processing device. [Figure 8] This is a block diagram showing the configuration of a computer that functions as an information processing device related to this disclosure. [Modes for carrying out the invention]

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0013] (Configuration of Information Processing Device 1) The configuration of the information processing device 1 according to this exemplary embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the information processing device 1. As shown in Figure 1, the information processing device 1 includes a creation location identification unit 101 and a labeling unit 102.

[0014] The creation location identification unit 101 identifies the portion of the deliverable created by a series of operations that was created by a partial operation, which is a part of the series of operations. The entity performing the series of operations may be entirely human, or some or all of the series of operations may be performed by a non-human entity such as artificial intelligence. Furthermore, the series of operations may be performed using one or more electronic devices (e.g., a computer), or some or all of the series of operations may be performed without the use of electronic devices.

[0015] The above deliverable is an electronic deliverable, or in other words, electronic data. For example, the above deliverable may be document data such as a report, a design document, or a plan created using a computer. The content of the above deliverable is not particularly limited. For example, the above deliverable may be software-related data such as a software design document or a computer program. Also, for example, the above deliverable may be a medical document such as a diagnosis report. Thus, the information processing apparatus 1 can also be applied to the healthcare field. Note that the above deliverable can also be non-electronic (e.g., a physical product, etc.).

[0016] Also, the creation location specifying unit 101 may specify a part of the deliverable as a "location created by partial work" (hereinafter referred to as the creation location), or may specify a converted product obtained by converting a part of the deliverable as the creation location. Also, when the deliverable is completed by conversion, the creation location specifying unit 101 may specify a part of the deliverable before conversion as the creation location. For example, assume that the deliverable is a computer program. In this case, the creation location specifying unit 101 may specify a part of the source code before compilation as the creation location, or may specify a part of the object code obtained by compiling the above source code as the creation location. Similarly, for example, when text data that can be converted into an image is the deliverable, the creation location specifying unit 101 may specify a part of the text data as the creation location, or may specify a part of the above image as the creation location. The creation location specifying unit 101 may directly specify the creation location in the deliverable or may specify it indirectly.

[0017] Based on the above creation location specified by the creation location specifying unit 101, the labeling unit 102 determines a label to be assigned to the partial work using a language model obtained by training a natural language. Hereinafter, the language model used by the labeling unit 102 is referred to as language model M. The labeling unit 102 can be said to be naming the partial work or classifying the partial work. Therefore, in the following description, the determination of the label, the labeling, and the labeling, etc. can also be rephrased as naming or classifying, etc.

[0018] Here, learning a natural language, more specifically, means learning the arrangement of its components (such as words) in a natural language sentence and the arrangement of sentences in a text. Examples of language models that have learned natural languages include, for example, BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT approach), ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), and the like.

[0019] Also, determining a label based on the creation location means directly or indirectly using the creation location in determining the label. How the creation location is used in determining the label is not particularly limited. For example, if the creation location is text described in a natural language, the labeling unit 102 may directly input the creation location as it is into the language model M. Also, when the creation location is data in a format other than text, the labeling unit 102 may convert the data into text format and input it into the language model M. Also, there are language models that are configured and learned to be able to input data in a format other than text, such as images. When the labeling unit 102 uses such a language model (a language model capable of inputting data in a format other than text) as the language model M, it can input the creation location in a format other than text directly into the language model M. Also, for example, the labeling unit 102 can also perform labeling using an image taken of the creation location or a description text of the object shown in the image generated from the image.

[0020] As described above, the information processing apparatus 1 according to this exemplary embodiment employs a configuration comprising: a production location identification unit 101 that identifies a portion of a deliverable created by a series of operations, which is created by a partial operation that is a part of the series of operations; and a labeling unit 102 that determines a label to be assigned to the partial operation using a language model M trained on natural language based on the location identified by the production location identification unit 101.

[0021] According to the above configuration, the parts of the deliverable created by a series of operations that were created by a sub-operation are identified, and labels to be assigned to the sub-operations are determined based on the identified parts. Since the parts of the deliverable created by a sub-operation reflect the actual state of that sub-operation, determining labels based on these parts makes it possible to determine labels that are appropriate to the actual state of the sub-operation. Furthermore, by using the labels assigned to the sub-operations, it becomes possible to analyze the series of operations in accordance with the actual state of those operations. For example, by dividing a series of operations into multiple sub-operations and assigning a label to each sub-operation, it becomes possible to discover the sub-operation that is truly a bottleneck in the series of operations.

[0022] Thus, the information processing device 1 has the effect of making it possible to analyze a series of tasks in accordance with the actual work performed. Furthermore, since the content of the series of tasks is visualized by the labels determined by the information processing device 1, it becomes possible to identify bottleneck tasks and, consequently, optimize the entire series of tasks.

[0023] (Labeling Program) The functions of the information processing device 1 described above can also be implemented by a program. In this exemplary embodiment, the labeling program causes the computer to function as a creation location identification means that identifies a part of a series of operations that is created by a partial operation in an output product created by the series of operations, and a labeling means that determines a label to be assigned to the partial operation using a language model M that has been trained on natural language based on the location identified by the creation location identification means. This labeling program has the effect of making it possible to analyze the series of operations in accordance with the actual state of those operations.

[0024] (Labeling process) The flow of the labeling method according to this exemplary embodiment will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the labeling method. Note that the entity executing each step in this labeling method may be a processor provided in the information processing device 1, a processor provided in another device, or the entity executing each step may be a processor provided in a different device.

[0025] In S1 (creation location identification process), at least one processor identifies the parts of the deliverable created by a series of operations that were created by a partial operation, which is a part of that series of operations.

[0026] In S2 (labeling), at least one processor determines the label to be assigned to the subtask using a language model M trained on natural language learning, based on the parts identified in S1.

[0027] As described above, the labeling method according to this exemplary embodiment employs a configuration in which at least one processor performs a creation location identification process to identify the parts of a deliverable created by a series of operations that are created by a partial operation, and a labeling process to determine a label to be assigned to the partial operation using a language model M trained on natural language based on the parts identified in the creation location identification process. This labeling method has the effect of making it possible to analyze the series of operations that create a deliverable in accordance with the actual state of those operations.

[0028] [Second exemplary embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0029] (Configuration of Information Processing Device 1A) The configuration of the information processing device 1A according to this exemplary embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A is a device equipped with functions to support the analysis of work. The information processing device 1A may be a local device used by individual users, or it may be a server that provides work analysis support services to multiple users.

[0030] As shown in the figure, the information processing device 1A includes a control unit 10A that controls all parts of the information processing device 1A, and a storage unit 11A that stores various data used by the information processing device 1A. The information processing device 1A also includes a communication unit 12A for the information processing device 1A to communicate with other devices, an input unit 13A that receives input to the information processing device 1A, and an output unit 14A for the information processing device 1A to output data. The control unit 10A includes a creation location identification unit 101A, a labeling unit 102A, a data acquisition unit 103A, a division unit 104A, a reference data identification unit 105A, an aggregation unit 106A, and a presentation control unit 107A.

[0031] The creation location identification unit 101A, similar to the creation location identification unit 101 of the exemplary embodiment 1, identifies the creation location in the deliverable created by the series of operations, which is a part of the series of operations.

[0032] Similar to the labeling unit 102 in Exemplary Embodiment 1, the labeling unit 102A determines the label to be assigned to the above-mentioned work section based on the work section identified by the work section identification unit 101A, using a language model M trained on natural language. Details of the labeling by the labeling unit 102A will be described later.

[0033] The data acquisition unit 103A acquires various data necessary to provide analysis support services. For example, the data acquisition unit 103A acquires deliverables created by a series of operations to be analyzed. Also, for example, the data acquisition unit 103A acquires the history of operations performed to create the deliverables during the period in which the series of operations were performed, and historical information showing the changes in the deliverables during the period in which the series of operations were performed. The data acquisition unit 103A may acquire the data showing the operation history and the data showing the changes in the deliverables as separate historical information. The operation history and the changes in the deliverables can be acquired from the computer or other device on which the series of operations were performed. Also, for example, if the deliverables were created using a file management system such as Git, the information showing the changes in the deliverables can also be acquired from the file management system.

[0034] The division unit 104A divides a series of operations performed to create a predetermined deliverable into multiple sub-operations. Details of the processing performed by the division unit 104A will be described later.

[0035] The reference data identification unit 105A identifies the reference data that the creator of the deliverable referenced when creating the deliverable. Providing the reference data identification unit 105A is not mandatory. However, by providing the reference data identification unit 105A, it becomes possible to have the labeling unit 102A consider the reference data when determining the label, thereby improving the accuracy of the assigned label.

[0036] The aggregation unit 106A aggregates the work time for each of the multiple subtasks, grouping them by the labeling unit 102A that have the same label. Providing the aggregation unit 106A is not mandatory. However, by providing the aggregation unit 106A, in addition to the effects of the information processing device 1, it becomes possible to easily analyze the work time for each subtask.

[0037] The presentation control unit 107A presents various information related to the provision of analysis support services. For example, the presentation control unit 107A may present labels determined by the labeling unit 102A, or the aggregation results of the aggregation unit 106A. The manner in which the presentation control unit 107A presents information is arbitrary. For example, the presentation control unit 107A may output the above-mentioned labels, etc., to the output unit 14A, or it may output the labels, etc., to an external display device of the information processing device 1A (for example, a terminal device used by the user) via the communication unit 12A. The manner of output is also arbitrary; for example, the presentation control unit 107A may output the labels, etc., as a sound output, or as printed output. It is not necessary to present the labels determined by the labeling unit 102A, or the aggregation results of the aggregation unit 106A, etc., to the user. For example, this information may be used for work analysis without being presented to the user.

[0038] (Summary of the process) The overview of the processing performed by the information processing device 1A will be explained based on Figure 4. Figure 4 is a diagram illustrating the overview of the processing performed by the information processing device 1A. In the example in Figure 4, a text document D is created as an output through a series of operations using the computer 41, display device 42, keyboard 43, and mouse 44. Figure 4 also shows the time-series change in the activity level of input operations using the keyboard 43 and mouse 44 during this series of operations as graph G. The method for calculating the activity level will be described later.

[0039] In the example shown in Figure 4, the division unit 104A divides the series of tasks performed to create document D based on the time-series changes in activity levels shown in graph G. Specifically, the division unit 104A divides the series of tasks into three parts: the tasks performed in period T1, the tasks performed in period T2, and the tasks performed in period T3.

[0040] Next, the creation location identification unit 101A identifies the locations in document D that were created by each partial work. For example, in the example in Figure 4, the creation location identification unit 101A identifies description location P1, which is a part of document D, as a creation location created by a partial work performed during period T1. Although not shown in the illustration, the creation location identification unit 101A also identifies each creation location created by each partial work performed during periods T2 and T3.

[0041] Next, the labeling unit 102A determines the labels to be assigned to the subtasks performed during period T1, based on the description location P1 identified by the creation location identification unit 101A, using a language model M trained on natural language learning. Specifically, the labeling unit 102A inputs the description location P1 into the language model M to infer the labels to be assigned to the subtasks performed during period T1, and determines the labels to be assigned to the subtasks performed during period T1 based on the inference results. Although not shown in the diagram, the labeling unit 102A similarly labels the subtasks performed during periods T2 and T3.

[0042] Thus, the information processing device 1A can divide the series of tasks involved in creating document D according to the actual work shown in graph G, and label each of the resulting subtasks. This makes it possible to analyze the series of tasks involved in creating document D according to the actual work, thereby streamlining each subtask and optimizing the entire series of tasks.

[0043] (Regarding the division of work) As described above, the division unit 104A divides a series of tasks performed to create a predetermined deliverable into multiple subtasks. The details of the processing performed by the division unit 104A are described below. Note that the division unit 104A only needs to be able to divide a series of tasks into coherent chunks, and the method of division is not limited to the following examples.

[0044] For example, the division unit 104A may divide a series of tasks into multiple subtasks based on the time-series changes in the activity level of input operations to the computer 41 used to create the deliverables during the period in which the series of tasks are performed, as shown in the example in Figure 4. This provides the effect that, in addition to the effects of the information processing device 1, it is possible to automatically define subtasks according to the actual work without being bound by existing work classifications.

[0045] Here, the activity level of input operations is an indicator of how actively input operations are being performed. For example, when input operations are performed using two input devices, a keyboard 43 and a mouse 44, as in the example in Figure 4, the splitting unit 104A can calculate the activity level from the history of input operations received by these input devices. Specifically, the splitting unit 104A may calculate, for example, the number of keystrokes on the keyboard 43 as the activity level of keyboard operations, and the amount of movement of the mouse 44 as the activity level of mouse operations, and then calculate the overall activity level by combining these activity levels. The splitting unit 104A may also calculate the amount of movement of the cursor displayed on the display device 42 (i.e., the display object operated by the mouse) as the amount of movement of the mouse 44 (this can also be rephrased as the distance moved). Furthermore, the splitting unit 104A may also reflect the number of operations such as clicks on the mouse 44 in the activity level.

[0046] The splitting unit 104A may calculate the activity level from the history of keyboard operations only, or from the history of mouse operations only. Furthermore, the splitting unit 104A may calculate the activity level from the history of input operations using any input device other than a keyboard and mouse. For example, the splitting unit 104A may calculate the activity level from the history of input operations using at least one of a touch panel, stylus pen, and touchpad. Also, if voice input is performed during the creation of a deliverable, the splitting unit 104A may calculate the activity level from the total duration of the input voice, the number of characters in the text generated by speech recognition of the voice, etc.

[0047] Furthermore, various methods can be applied to the classification based on the time-series changes in activity levels. For example, the division unit 104A may detect a period within the time frame in which a series of tasks are performed in which the activity level remains below a predetermined threshold for a predetermined time or longer. The division unit 104A may then divide the series of tasks into multiple subtasks by dividing them into each detected period. In the example in Figure 4, the series of tasks for creating document D are divided into three subtasks using this method.

[0048] Furthermore, for example, the division unit 104A may detect predetermined operations performed at each division of work from the history of input operations to the computer 41 used to create the deliverables during the period in which the series of operations are performed. The division unit 104A may then divide the series of operations into multiple sub-operations by dividing the series of operations at each time a predetermined operation is detected. Even when such a configuration is adopted, in addition to the effects performed by the information processing device 1, the effect of being able to automatically define sub-operations according to the actual work, without being bound by existing work classifications, can be obtained.

[0049] The specific operations to be described above can be predetermined. For example, if the deliverable is electronic data, the splitting unit 104A may detect operations such as saving the electronic data or displaying a preview of the electronic data as the specified operations. Also, for example, if the task of creating new electronic data from the original data is performed, the original data and the new electronic data may be compared and verified at each stage of the task. For this reason, the splitting unit 104A may detect an operation to display the original data together with the created electronic data as the specified operations.

[0050] Furthermore, the division unit 104A may divide a series of tasks into multiple subtasks based on the behavior of the creator of the deliverable. For example, the division unit 104A may perform the above division based on the results of analyzing images of the creator taken while working. In this case, the division unit 104A should divide the series of tasks at the time when a predetermined action performed at each division of the task is detected. Examples of predetermined actions include standing up from a seat, stretching, eating or drinking, and operating devices not used for work, such as a smartphone.

[0051] Furthermore, the division unit 104A may perform the above divisions based on the operating status of various devices used or worn by the worker. For example, suppose the worker is wearing a wearable device capable of measuring vital data, such as a smartwatch. In this case, the division unit 104A may identify the time when the worker changed from a tense state to a relaxed state based on the time-series changes in vital data, such as heart rate, and divide the series of tasks at that time.

[0052] (Labeling details) The details of labeling by the labeling unit 102A will be explained with reference to Figure 5. Figure 5 shows an example of labeling. In the example in Figure 5, the information processing device 1A (more specifically, the labeling unit 102A) inputs a prompt 51 to the language model M, and the language model M outputs a response 52 to the prompt 51. Note that the language model M may be one that is provided outside the information processing device 1A (for example, on a server), as in the example in Figure 5, or the language model M may be stored in the information processing device 1A.

[0053] Prompt 51 is a prompt that instructs the user to infer a label to be assigned to a subtask. Specifically, Prompt 51 includes "deliverable" and "candidate," and instructs the user to select a label from the "candidate" that describes the task of creating the "deliverable." Here, "deliverable" in Prompt 51 does not refer to the entire deliverable created by the series of tasks, but rather to the part of the deliverable identified by the creation location identification unit 101A, that is, the creation location created by the subtask.

[0054] The parts of the prompt 51 other than the content of the "deliverable" are standard and can be templated. By storing this template in the storage unit 11A, the labeling unit 102A can input the description of the creation location identified by the creation location identification unit 101A into the "deliverable" part of the template and generate the prompt 51.

[0055] Furthermore, prompt 51 includes a sentence instructing the system to create a new label if no suitable candidates are found. Using prompts that include such sentences makes it possible to label without being constrained by candidates, while preventing the assignment of labels unsuitable for analysis or excessive label diversity. In addition, the sentence instructing the system to create a new label in prompt 51 specifies that the label to be created should be no more than 10 characters long. Thus, the sentence instructing the system to create a new label may also include conditions for the label to be created by the language model M. This allows the language model M to create a label that satisfies the desired conditions.

[0056] In the prompt 51 in Figure 5, the sentence "Digitize the documents,..." is entered in the "Deliverables" section. The prompt 51 also lists typical tasks performed during the design of a computer program, such as creating a system configuration diagram and creating a function list, as "candidates." In the example in Figure 5, the language model M outputs a response 52 in response to the input prompt 51, indicating that "Creating a function list," one of the above "candidates," is the label that should be assigned to the subtask. Based on this response 52, the labeling unit 102A can determine that the label to be assigned to the subtask that creates the "deliverable" that begins with "Digitize the documents" is "Creating a function list."

[0057] It is not mandatory to include a sentence instructing the language model M to generate a new label in the prompt input to the language model M. In this case, the language model M will output the most appropriate label among the candidates. Alternatively, the language model M may output the degree of suitability of each candidate as a label to be assigned to the subtask. In this case, the labeling unit 102A may decide to assign the label with the highest degree of suitability to that subtask. Alternatively, for example, the labeling unit 102A may have the presentation control unit 107A present each candidate and its degree of suitability, allowing the user to select which candidate to adopt. In this case, the labeling unit 102A will decide to assign the candidate selected by the user to the subtask.

[0058] It is not mandatory to include label candidates in the prompts input to the language model M. If label candidates are not included in the prompts input to the language model M, the labeling unit 102A only needs to generate a prompt instructing it to infer what label should be assigned to the subtask. Alternatively, the labeling unit 102A may generate a prompt instructing it to generate a label to be assigned to the subtask.

[0059] As described above, the labeling unit 102A may input a prompt to the language model M that includes the creation location identified by the creation location identification unit 101A, cause the language model M to infer a label to be assigned to the subtask, and then determine the label to be assigned to the subtask based on the result of the inference. This provides the effect that, in addition to the effects of the information processing device 1, it is possible to directly consider the creation location identified by the creation location identification unit 101A and determine an appropriate label.

[0060] Furthermore, as described above, the labeling unit 102A may input a prompt to the language model M that includes label candidates in addition to the creation location identified by the creation location identification unit 101A, to infer which candidate is appropriate as a label for the subtask, and determine the label to be assigned to the subtask based on the result of the inference. This provides the effect of being able to label subtasks within a range of candidates, in addition to the effects performed by the information processing device 1.

[0061] Furthermore, as described above, the prompt input to the language model M may also be a prompt that instructs the model to generate a new label if there is no suitable label for a subtask among the candidate labels. This provides the effect of enabling labeling without being constrained by candidates, while suppressing the assignment of labels unsuitable for analysis or excessive diversification of labels, in addition to the effects of the information processing device 1.

[0062] Furthermore, the labeling unit 102A may include various pieces of information in the prompts input to the language model M that are useful for inferring the labels to be assigned to the subtasks. This increases the likelihood that an appropriate label will be determined.

[0063] For example, as described above, the information processing device 1A may include a reference data identification unit 105A. The reference data identification unit 105A identifies reference data, which is data that the creator of a deliverable referenced when creating the deliverable. For example, the reference data identification unit 105A can identify reference data from the history of input operations to the computer used to create the deliverable during the period in which a series of operations for creating the deliverable are performed. Specifically, the reference data identification unit 105A may detect an operation to open (in other words, display or play back) data different from the deliverable, or an operation to display data different from the deliverable on the screen that was displaying the deliverable. The reference data identification unit 105A may then identify the data opened, displayed, or played back by such operation as reference data.

[0064] When the reference data identification unit 105A identifies reference data, the labeling unit 102A generates a prompt that includes the reference data identified by the reference data identification unit 105A, in addition to the creation location identified by the creation location identification unit 101A. This prompt should instruct the system to estimate a label considering the content of the reference data. The prompt may also include a statement indicating that the reference data was referenced during the execution of the subtask that is subject to labeling.

[0065] The labeling unit 102A then inputs the generated prompt to the language model M, which infers a label to assign to the subtask considering the reference data, and determines the label to assign to the subtask based on the result of this inference. This provides the effect of increasing the likelihood of determining an appropriate label, in addition to the effects of the information processing device 1.

[0066] For example, suppose the deliverable is a medical document, and during the period in which one sub-task of the series of tasks to create that medical document is being performed, an operation is performed to open (in other words, display) a patient's electronic medical record. In this case, the reference data identification unit 105A detects the above operation from the input operation history and identifies the electronic medical record opened by that operation as reference data. Then, the labeling unit 102A generates a prompt that includes the electronic medical record identified by the reference data identification unit 105A in addition to the creation location identified by the creation location identification unit 101A, and inputs it into the language model M. As a result, the contents of the electronic medical record are taken into consideration when inferring the label to be assigned to the sub-task, making it possible to increase the likelihood of determining an appropriate label.

[0067] (Example of result display) The display control unit 107A may present the label determined by the labeling unit 102A to the user. Alternatively, the display control unit 107A may present the aggregation results of the aggregation unit 106A. The presentation of labels, etc., by the display control unit 107A will be explained with reference to Figure 6. Figure 6 is a diagram showing an example of a display screen displayed by the display control unit 107A. More specifically, the display screen shown in Figure 6 is an example of a display screen for presenting the label determined by the labeling unit 102A together with the aggregation results of the aggregation unit 106A.

[0068] In the example display screen in Figure 6, a graph showing the time-series change in the activity level of a series of tasks is displayed as a work report for a series of tasks that generate a deliverable. Such a graph can be generated using the time-series activity level calculated by the division unit 104A. Furthermore, this graph is divided into three periods, T1 to T3. These divisions represent the division results by the division unit 104A. In other words, the tasks in each of the periods T1 to T3 are all partial tasks.

[0069] Furthermore, in the example display screen in Figure 6, labels such as "Create Function List" are displayed for each period T1 to T3 (in other words, each sub-task), along with the duration of each period (in other words, the time during which the sub-task was performed). These labels are determined by the labeling unit 102A.

[0070] Furthermore, in the example display screen in Figure 6, the total time for a series of tasks is displayed as "Work Time," and the "Work Breakdown" shows the aggregated work time for each subtask in each period from T1 to T3, grouped by subtask with the same label. Specifically, the subtask performed in period T1 and the subtask performed in period T3 are both assigned the same label, "Create Function List." Therefore, in the example display screen in Figure 6, the aggregated work time for "Create Function List" is shown to be 35 minutes, based on the work time for each of these subtasks (20 minutes and 15 minutes). Note that for the subtask in period T2, which is labeled "Create Screen Transition Diagram," there are no other subtasks with the same label, so the work time for period T2, 20 minutes, is displayed as the aggregated result. Such aggregation is performed by the aggregation unit 106A as described above.

[0071] (Process flow) The processing flow performed by the information processing device 1A will be explained with reference to Figure 7. Figure 7 is a flowchart showing the processing flow performed by the information processing device 1A. The flowchart in Figure 7 includes each process of the labeling method according to this exemplary embodiment. Figure 7 shows the processing performed after a series of operations for creating a deliverable has been completed, and after the history information showing the history of operations performed to create the deliverable during the period in which the series of operations were performed, and the changes in the deliverable during the period in which the series of operations were performed has been recorded.

[0072] In S11, the data acquisition unit 103A acquires the history information recorded as described above and the deliverables created by the series of operations. The method of acquiring the history information and deliverables is arbitrary. For example, the data acquisition unit 103A may acquire the history information and deliverables from another device (for example, the computer 41 shown in Figure 4) via the communication unit 12A, or it may acquire the history information and deliverables input via the input unit 13A. Note that the history information and deliverables do not necessarily have to be acquired at the same time. Furthermore, the history information and deliverables may be acquired using different acquisition methods.

[0073] In S12, the division unit 104A uses the history of operations during the period in which the series of operations were performed, as shown in the history information acquired in S11, to divide the series of operations performed to create the deliverable acquired in S11 into multiple sub-operations. For example, the division unit 104A may calculate the activity level of input operations at each time point during the period in which the series of operations were performed from the history, and detect a period in which the calculated activity level remains below a predetermined threshold for a predetermined time or longer. The division unit 104A may then divide the series of operations into multiple sub-operations by dividing it into each detected period. Alternatively, for example, the division unit 104A may detect a predetermined operation performed at each division of the operations from the history, and divide the series of operations into multiple sub-operations by dividing it into each time point in which the predetermined operation is detected.

[0074] In S13 (creation location identification process), the creation location identification unit 101A identifies the parts of the deliverable created by the series of operations that were created by a partial operation, which is a part of the series of operations. Specifically, the creation location identification unit 101A identifies which parts of the deliverable obtained in S11 were created by one of the partial operations separated in S12, and performs this process for each of the multiple partial operations separated in S12. For example, the creation location identification unit 101A may identify the start time and end time of each partial operation based on the separation result in S12. Then, based on the changes in the deliverable during the period in which the series of operations were performed, as shown in the history information obtained in S11, the creation location identification unit 101A identifies the parts of the deliverable obtained in S11 that were created during the period from the start time to the end time of one partial operation, i.e., the creation locations mentioned above. For example, the creation location identification unit 101A may identify the difference between the deliverable at the end time of one partial operation and the deliverable at the start time of that partial operation as the creation location. This process is performed for each individual task.

[0075] In S14, the reference data identification unit 105A uses the history of operations during the period in which the series of operations were performed, as shown in the history information acquired in S11, to identify the reference data for each sub-task defined by the processing in S12. However, data is not necessarily referenced in every sub-task. Therefore, in S14, there may be sub-tasks in which the reference data identification unit 105A cannot identify the reference data.

[0076] In S15, the labeling unit 102A generates a prompt that instructs it to infer a label to be assigned to the work that created the creation location identified in S13, i.e., the sub-work. The labeling unit 102A may generate a prompt for each of the sub-works, or it may generate a single prompt for all of the sub-works together. In the latter case, the labeling unit 102A only needs to generate a prompt that instructs it to infer a label to be assigned to each sub-work that created each creation location, including the creation locations from each of the sub-works.

[0077] In S16 (labeling process), the labeling unit 102A determines the labels to be assigned to the subtasks based on the creation locations identified in the S13 process, using a language model M trained on natural language learning. Specifically, the labeling unit 102A inputs a prompt (generated in S15) containing the creation locations identified in the S13 process to the language model M, and determines the labels to be assigned to the subtasks based on the output from the language model M. The label determination is performed for each subtask defined in the S12 process.

[0078] In S17, the aggregation unit 106A aggregates the work time for each of the multiple subtasks defined by the processing in S12, for each subtask that has the same label determined in S16.

[0079] In S18, the display control unit 107A displays the labels for each subtask determined in S16, along with the aggregated results from S17. For example, the display control unit 107A may display a display screen as shown in Figure 6. This completes the process shown in Figure 7.

[0080] [Variation] The entities executing each process described in the exemplary embodiments above are arbitrary and not limited to the examples given. For example, a system having the same functions as the information processing devices 1 and 1A can be constructed using multiple devices that can communicate with each other. Furthermore, the entities executing each process shown in the flowchart of Figure 7 may be a single device (which can also be called a processor) or multiple devices (which can also be called processors).

[0081] [Examples of implementation using software] Some or all of the functions of the information processing devices 1 and 1A (hereinafter also referred to as "the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0082] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as Computer C) is shown in Figure 8. Figure 8 is a block diagram showing the hardware configuration of Computer C, which functions as each of the above devices.

[0083] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program (labeling program) P that causes computer C to operate as each of the above-mentioned devices. In computer C, processor C1 reads program P from memory C2 and executes it, thereby realizing each of the above-mentioned devices.

[0084] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0085] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0086] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0087] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0088] [Additional Notes] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0089] (Note A1) An information processing device comprising: a means for identifying a production location in a deliverable created by a series of operations, which is created by a partial operation that is a part of the series of operations; and a labeling means for determining a label to be assigned to the partial operation based on the location identified by the production location identification means, using a language model trained on natural language.

[0090] (Appendix A2) The information processing apparatus according to Appendix A1, comprising a division means for dividing the series of operations into a plurality of sub-operations based on the time-series change in the activity level of input operations to the computer used to create the deliverable during the period in which the series of operations are performed.

[0091] (Note A3) The information processing apparatus according to Appendix A1, comprising a division means for dividing a series of tasks into a plurality of subtasks by detecting a predetermined operation performed at each division of the work from the history of input operations to the computer used to create the deliverable during the period in which the series of tasks are performed, and dividing the series of tasks at each time the predetermined operation is detected.

[0092] (Note A4) The labeling means inputs a prompt including the location identified by the creation location identification means to the language model, causes it to infer a label to be assigned to the subtask, and determines the label to be assigned to the subtask based on the result of the inference, as described in any of the appendices A1 to A3.

[0093] (Note A5) The labeling means inputs a prompt to the language model that includes, in addition to the location identified by the creation location identification means, candidates for the label, causing the language model to infer which candidate is appropriate as the label for the subtask, and determines the label to be assigned to the subtask based on the result of the inference, as described in Appendix A4.

[0094] (Note A6) The information processing device described in Appendix A5 is an information processing device that instructs the device to generate a new label if there is no suitable label for the subtask among the candidates.

[0095] (Note A7) The information processing apparatus according to any of the appendices A4 to A6, comprising: reference data identification means for identifying reference data, which is data referenced when creating the deliverable, from the history of input operations to the computer used to create the deliverable during the period in which the series of operations described above are performed; and labeling means inputting a prompt to the language model that includes the reference data in addition to the location identified by the creation location identification means, causing the language model to infer a label to be assigned to the sub-work considering the reference data, and determining a label to be assigned to the sub-work based on the result of the inference.

[0096] (Note A8) An information processing device according to any one of the appendices A1 to A7, comprising a summarization means for summarizing the work time for each of the aforementioned plurality of sub-works for each sub-work that has the same label determined by the labeling means.

[0097] (Note B1) A labeling method comprising: a creation location identification process in which at least one processor performs a creation location identification process to identify a portion of an output created by a series of operations, which is a part of the series of operations; and a labeling process to determine a label to be assigned to the portion of the operations using a language model trained on natural language based on the location identified in the creation location identification process.

[0098] (Note B2) The labeling method according to Appendix B1, which includes a division process in which at least one processor divides the series of operations into a plurality of sub-operations based on the time-series change in the activity level of input operations to the computer used to create the deliverable during the period in which the series of operations are performed.

[0099] (Note B3) The labeling method according to Appendix B1, which includes a division process in which at least one processor detects a predetermined operation performed at each division of work from the history of input operations to the computer used to create the deliverable during the period in which the series of work is performed, and divides the series of work into a plurality of sub-works at each time the predetermined operation is detected.

[0100] (Note B4) The labeling method according to any one of the appendices B1 to B3, wherein in the labeling process, the at least one processor inputs a prompt to the language model that includes the location identified in the creation location identification process, causes the language model to infer a label to be assigned to the sub-work, and determines the label to be assigned to the sub-work based on the result of the inference.

[0101] (Note B5) The labeling method according to Appendix B4, wherein in the labeling process, at least one processor inputs a prompt to the language model that includes, in addition to the location identified in the creation location identification process, a candidate label, and causes the language model to infer which candidate is appropriate as the label for the sub-work, and determines the label to be assigned to the sub-work based on the result of the inference.

[0102] (Note B6) The labeling method described in Appendix B5, wherein the prompt instructs the system to generate a new label if there is no suitable label for the subtask among the candidates.

[0103] (Note B7) The labeling method according to any one of Appendix B4 to B6, wherein the at least one processor includes a reference data identification process that identifies reference data, which is data referenced when creating the deliverable, from the history of input operations to the computer used to create the deliverable during the period in which the series of operations are performed, and in the labeling process, the at least one processor inputs a prompt to the language model that includes the reference data in addition to the location identified in the creation location identification process, causes the language model to infer a label to be assigned to the sub-work considering the reference data, and determines a label to be assigned to the sub-work based on the result of the inference.

[0104] (Note B8) A labeling method according to any one of Appendix B1 to B7, wherein at least one processor includes an aggregation process that aggregates the work time for each of the plurality of subtasks for each subtask that has the same label determined by the labeling process.

[0105] (Note C1) A labeling program that causes a computer to function as a means for identifying a part of a series of operations that creates a deliverable, and a means for identifying a part of a series of operations that is created by a partial operation, and a means for determining a label to be assigned to the partial operation based on the part identified by the means for identifying a part that was created by a partial operation that was created by a partial operation that was created by a partial operation.

[0106] (Note C2) The labeling program described in Appendix C1, which causes the computer to function as a division means for dividing the series of operations into a plurality of sub-operations based on the time-series change in the activity level of input operations to the computer used to create the deliverable during the period in which the series of operations are performed.

[0107] (Note C3) The labeling program described in Appendix C1, wherein the computer functions as a division means for dividing a series of tasks into a plurality of subtasks by detecting a predetermined operation performed at each division of the work from the history of input operations to the computer used to create the deliverable during the period in which the series of tasks are performed, and dividing the series of tasks at each time the predetermined operation is detected.

[0108] (Note C4) The labeling means is a labeling program according to any one of the appendices C1 to C3, which inputs a prompt including the location identified by the creation location identification means to the language model, causes it to infer a label to be assigned to the sub-work, and determines the label to be assigned to the sub-work based on the result of the inference.

[0109] (Note C5) The labeling program described in Appendix C4, wherein the labeling means inputs a prompt to the language model that includes, in addition to the location identified by the creation location identification means, candidate labels, to cause the language model to infer which candidate is appropriate as the label for the subtask, and determines the label to be assigned to the subtask based on the result of the inference.

[0110] (Appendix C6) The labeling program described in Appendix C5 is a prompt that instructs the program to generate a new label if there is no suitable label for the subtask among the candidates.

[0111] (Note C7) A labeling program as described in any of Appendix C4 to C6, wherein the computer functions as a reference data identification means for identifying reference data, which is data referenced when creating the deliverable, from the history of input operations to the computer used to create the deliverable during the period in which the series of operations are performed, and the labeling means inputs a prompt to the language model that includes the reference data in addition to the location identified by the creation location identification means, causes the language model to infer a label to be assigned to the sub-work considering the reference data, and determines the label to be assigned to the sub-work based on the result of the inference.

[0112] (Note C8) A labeling program as described in any of Appendix C1 to C7, which causes the computer to function as an aggregation means for aggregating the work time for each of the plurality of subtasks for each subtask where the label determined by the labeling means is the same.

[0113] (Note D1) An information processing device comprising at least one processor, wherein the at least one processor performs a creation location identification process to identify a portion of a deliverable created by a series of operations, which is created by a partial operation that is a part of the series of operations; and a labeling process to determine a label to be assigned to the partial operation based on the portion identified in the creation location identification process, using a language model trained on natural language.

[0114] The information processing device may also include memory. Furthermore, the memory may store a program that causes at least one processor to execute each of the aforementioned processes.

[0115] (Note D2) The information processing apparatus according to Appendix D1, wherein the at least one processor performs a division process that divides the series of operations into a plurality of sub-operations based on the time-series change in the activity level of input operations to the computer used to create the deliverable during the period in which the series of operations are performed.

[0116] (Note D3) The information processing apparatus according to Appendix D1, wherein at least one processor detects a predetermined operation performed at each division of work from the history of input operations to the computer used to create the deliverable during the period in which the series of work is performed, and divides the series of work into a plurality of sub-works by dividing the series of work at each time the predetermined operation is detected.

[0117] (Note D4) The information processing apparatus according to any one of the appendices D1 to D3, wherein in the labeling process, the at least one processor inputs a prompt to the language model that includes the location identified in the creation location identification process, causes the language model to infer a label to be assigned to the sub-work, and determines the label to be assigned to the sub-work based on the result of the inference.

[0118] (Note D5) The information processing apparatus according to Appendix D4, wherein in the labeling process, at least one processor inputs a prompt to the language model that includes, in addition to the location identified in the creation location identification process, a candidate label, to cause the language model to infer which candidate is appropriate as the label for the subtask, and determines the label to be assigned to the subtask based on the result of the inference.

[0119] (Note D6) The information processing device described in Appendix D5, wherein the prompt is a prompt that instructs the device to generate a new label if there is no suitable label for the subtask among the candidates.

[0120] (Note D7) The information processing apparatus according to any one of the appendices D4 to D6, wherein at least one processor performs a reference data identification process to identify reference data, which is data referenced when creating the deliverable, from the history of input operations to the computer used to create the deliverable during the period in which the series of operations are performed, and in the labeling process, the at least one processor inputs a prompt to the language model that includes the reference data in addition to the location identified in the creation location identification process, causes the language model to infer a label to be assigned to the sub-work considering the reference data, and determines a label to be assigned to the sub-work based on the result of the inference.

[0121] (Note D8) The information processing apparatus according to any one of the appendices D1 to D7, wherein at least one processor performs an aggregation process that aggregates the work time for each of the plurality of subtasks for each subtask that has the same label determined by the labeling process.

[0122] (Note E) A non-temporary recording medium that records a labeling program that causes a computer to perform a creation location identification process to identify a portion of a deliverable created by a series of operations, which is a part of the series of operations; and a labeling process that determines a label to be assigned to the portion of the operations using a language model trained on natural language learning based on the location identified in the creation location identification process. [Explanation of Symbols]

[0123] 1. Information Processing Device 101 Identifying the location of creation 102 Labeling section 1A Information Processing Device 101A Identification of the creation location 102A Labeling section 104A Split part 105A Reference Data Identification Unit 106A Aggregation Department M language model

Claims

1. A means for identifying the location of a deliverable created by a series of operations, which identifies the portion created by a partial operation that is part of the series of operations, An information processing device comprising: labeling means, which determines a label to be assigned to the sub-work using a language model trained on natural language based on the location identified by the creation location identification means.

2. The information processing apparatus according to claim 1, further comprising a division means for dividing the series of operations into a plurality of sub-operations based on the time-series change in the activity level of input operations to the computer used to create the deliverable during the period in which the series of operations are performed.

3. The information processing apparatus according to claim 1, comprising a division means for dividing a series of tasks into a plurality of subtasks by detecting a predetermined operation performed at each division of the work from the history of input operations to the computer used to create the deliverable during the period in which the series of tasks are performed, and dividing the series of tasks at each time the predetermined operation is detected.

4. The information processing apparatus according to any one of claims 1 to 3, wherein the labeling means inputs a prompt including the location identified by the creation location identification means to the language model, causes it to infer a label to be assigned to the sub-work, and determines a label to be assigned to the sub-work based on the result of the inference.

5. The information processing apparatus according to claim 4, wherein the labeling means inputs a prompt to the language model that includes, in addition to the location identified by the creation location identification means, candidates for a label, to cause the language model to infer which candidate is appropriate as a label for the subtask, and determines a label to be assigned to the subtask based on the result of the inference.

6. The information processing apparatus according to claim 5, wherein the prompt is a prompt that instructs the apparatus to generate a new label if there is no suitable label for the subtask among the candidates.

7. The system includes a reference data identification means that identifies reference data, which is data referenced when creating the deliverable, from the history of input operations on the computer used to create the deliverable during the period in which the aforementioned series of operations are performed. The information processing apparatus according to claim 4, wherein the labeling means inputs a prompt to the language model that includes the reference data in addition to the location identified by the creation location identification means, causes the language model to infer a label to be assigned to the sub-work considering the reference data, and determines a label to be assigned to the sub-work based on the result of the inference.

8. The information processing apparatus according to any one of claims 1 to 3, further comprising an aggregation means for aggregating the work time for each of the plurality of sub-works for each sub-work that has the same label determined by the labeling means.

9. At least one processor, A process for identifying the location of a deliverable created by a series of operations, which identifies the part of the deliverable created by a partial operation that is a part of the series of operations, A labeling method that performs a labeling process to determine a label to be assigned to the sub-work using a language model trained on natural language based on the location identified in the aforementioned creation location identification process.

10. Computers, A means for identifying the location of a deliverable created by a series of operations, which identifies the portion created by a partial operation that is part of the series of operations, and A labeling program that functions as a labeling means, which determines a label to be assigned to the sub-work using a language model trained on natural language based on the location identified by the creation location identification means.

Images