Work summarization device, work summarization method, and program

Abstract

To solve the problem that it is difficult to grasp work contents performed to a device by a user.SOLUTION: A model managing unit 111 that stores a prompt model that is information for acquiring, from the generated AI, information explaining the content of the work performed on the device by the user and that serves as a source for constructing a prompt to be given to the generated AI, a work information acquiring unit 131 that continuously acquires work information related to the work of the user, a prompt constructing unit 132 that constructs a prompt to be given to the generated AI by using the work information and the prompt model, and the prompt constructing unit 132 that passes the constructed prompt to the generated AI, with the work summarizing device 1 including the answer acquiring unit 133 that acquires an answer from generated AI, the summary acquiring unit 134 that acquires summary information that is information using the answer acquired by the answer acquiring unit 133 and that explains the content of work performed by a user, and the summary outputting unit 141 that outputs the summary information, it is easy to grasp the content of work performed on the device by the user.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a task summarizing device that acquires and outputs a summary of the content of tasks performed by a user on a device. [Background technology]

[0002] Conventionally, there has been an operator fraud detection system that monitors the entire room where an operator works remotely, detects fraud, and notifies an administrator (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-68624 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, it is difficult to grasp the content of the work that the user has performed on the device. [Means for solving the problem]

[0005] The work summarizing device of the first invention is a work summarizing device that includes a template management unit that stores prompt templates, which are information for obtaining from the generation AI information explaining the content of work performed by a user on a device and are the basis for constructing prompts to be given to the generation AI; a work information acquisition unit that continuously acquires work information related to work performed by the user on a device; a prompt construction unit that constructs prompts to be given to the generation AI using the work information and prompt templates acquired by the work information acquisition unit; an answer acquisition unit that passes the prompt constructed by the prompt construction unit to the generation AI and acquires an answer from the generation AI; a summary acquisition unit that acquires summary information, which is information using the answer acquired by the answer acquisition unit and explains the content of the work performed by the user; and a summary output unit that outputs the summary information.

[0006] With this configuration, a summary of the work that the user has performed on the device can be obtained using the generation AI, making it easier to understand the work that the user has performed on the device.

[0007] Furthermore, in the work summarization device of the second invention, compared to the first invention, the work information acquisition unit acquires work information for each of two or more summary unit times, which is a portion of the total time during which a series of tasks is performed by a user, the prompt composition unit composes a prompt for each summary unit work information, which is the work information for each of the two or more summary unit times, the summary acquisition unit acquires summary information for each of the two or more summary unit work information, and the summary output unit outputs summary information for each of the two or more summary unit times.

[0008] With this configuration, by using the generation AI to obtain a summary of the work content for each summary unit time, it becomes easier to understand the work content that the user performed on the device.

[0009] Furthermore, in the task summarizing device of the third invention, in contrast to the second invention, the summarizing unit time is a predetermined fixed time.

[0010] With this configuration, a summary of the work content for each unit time is obtained using the generation AI, making it easier to understand the work content that the user has performed on the device.

[0011] In addition, the work summarization device of the fourth invention is a work summarization device in which, compared to the second invention, the work information acquisition unit is equipped with a judgment means for judging whether the work performed by the user on the device satisfies the switching condition, and a work information acquisition means for acquiring work information for the summary unit time up to before the switching condition is satisfied, and for acquiring work information for the summary unit time after the switching condition is satisfied.

[0012] With this configuration, a summary of the work content for each time period of a series of coherent tasks can be obtained using the generation AI, making it easier to understand the work content that the user performed on the device.

[0013] In addition, the work summarizing device of the fifth invention is a work summarizing device according to any one of the first to fourth inventions, wherein the work information acquisition unit acquires work information paired with a user identifier that identifies a user, and further includes a report destination acquisition unit that acquires a report destination identifier paired with the user identifier paired with the work information from a report destination management unit that stores report destination identifiers that identify the report destination to which the summary information is to be sent, paired with each of two or more user identifiers, and the summary output unit transmits the summary information to the report destination identified by the report destination identifier acquired by the report destination acquisition unit.

[0014] With this configuration, a summary of the work performed by a user on a device can be reported to the reporter corresponding to the user.

[0015] Furthermore, the work summarizing device of the sixth invention is a work summarizing device in which, compared to any one of the first to fifth inventions, the work information acquisition unit acquires work information, which is information displayed on a screen on which a user performs work.

[0016] With this configuration, a summary of the work performed by the user on the device can be obtained using information displayed on the screen on which the user performs the work, making it easier to understand the work content.

[0017] Furthermore, in contrast to the sixth aspect of the present invention, the task summarizing device of the seventh aspect of the present invention is a task summarizing device in which the task information includes title information, which is a character string indicating the title of a window on the screen.

[0018] With this configuration, a summary of the work performed by the user on the device is obtained using the title information of the window displayed on the screen on which the user is performing the work, making it easier to understand the work content.

[0019] Furthermore, in contrast to the sixth aspect of the present invention, the task summarizing device of the eighth aspect of the present invention is a task summarizing device in which the task information includes a window image, which is an image of a window on a screen.

[0020] With this configuration, a summary of the work performed by the user on the device can be obtained using a screenshot of the window displayed on the screen on which the user is performing the work, making it easier to understand the work content.

[0021] Furthermore, in a work summarizing device according to a ninth aspect of the present invention, in addition to any one of the first to eighth aspects of the present invention, the work information acquisition unit includes word acquisition means for acquiring words that identify a work at each of two or more times; co-occurrence count acquisition means for acquiring, at each of two or more times, the number of co-occurrences of two words in a set of two words that occur at the same time using the words acquired by the word acquisition means; graph construction means for acquiring graph data in which each of two co-occurring words in each of the two or more sets is a node and the two nodes are connected by an edge whose weight is the number of co-occurrences of the two words; and graph construction means for acquiring graph data in which the graph construction means acquires a graph of the number of co-occurrences of the two words in each of the two or more sets. The work summarization device comprises a division means for dividing rough data into two or more subgraphs, which are graph data, using the node connections by edges and the edge weights, and a work information acquisition means for acquiring, for each of the two or more subgraphs, work information having words for each of the two or more nodes that the subgraph has, a prompt composition unit for each of the two or more subgraphs, assigning the work information to a prompt template to compose a prompt, and an answer acquisition unit for each of the two or more subgraphs, passing the prompt composed by the prompt composition unit to a generation AI and acquiring an answer from the generation AI.

[0022] With this configuration, by acquiring a summary of the work content that the user performed on the device for each word group that indicates the characteristics of the work, it becomes easier to understand the work content that the user performed on the device.

[0023] Furthermore, the work summarization device of the tenth invention is a work summarization device in which, compared to any one of the first to ninth inventions, the work information acquisition unit is equipped with an original information acquisition means for acquiring original information, which is a captured image of the screen of the device, at each of two or more times, and a work information acquisition means for acquiring work information from the original information acquired by the original information acquisition means at each of two or more times.

[0024] With this configuration, by acquiring work information from a captured image of the screen of the device on which the user is performing work, it becomes easier to understand the work content that the user has performed on the device.

[0025] Furthermore, in the work summarizing device of the eleventh aspect of the present invention, in contrast to the tenth aspect of the present invention, the summary output section stores summary information in pairs with the photographed image.

[0026] This configuration enables appropriate management of summaries of the work that a user has performed on a device. [Effects of the Invention]

[0027] The task summarizing device according to the present invention makes it easy for a user to understand the tasks performed on the device. [Brief explanation of the drawings]

[0028] [Figure 1] Conceptual diagram of information system A in embodiment 1 [Figure 2] Block diagram of Information System A [Figure 3] Block diagram of the work summarization device 1 [Figure 4] FIG. 10 shows an example of the graph data. [Figure 5] Image of the division method process [Figure 6] A flowchart illustrating an example of the operation of the work summarizing device 1 [Figure 7] Flowchart illustrating an example of unit time task information acquisition processing [Figure 8] Flowchart illustrating an example of the answer acquisition process [Figure 9] 10 is a flowchart illustrating an example of the output summary information acquisition process. [Figure 10] A flowchart illustrating an example of the operation of the terminal device 2 [Figure 11] Figure showing an example of the first prompt template [Figure 12] Figure 10 shows an example of the second prompt template [Figure 13] Figure showing an example of the first prompt template [Figure 14] Figure 10 shows an example of the report destination management table [Figure 15] FIG. 10 is a diagram showing an example of the acquired word information. [Figure 16] An example of the report is shown in the figure below. [Figure 17] A diagram showing an example of the active window [Figure 18] FIG. 10 shows an example of linking the summary information with a photographed image, etc. [Figure 19] A diagram showing an example of linking the same title information etc. with summary information [Figure 20] Block diagram of the computer system DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, embodiments of the work summarizing device and the like will be described with reference to the drawings. Note that components with the same reference numerals in the embodiments perform similar operations, and therefore repeated description may be omitted.

[0030] (Embodiment 1) In this embodiment, we will describe a task summarization device that acquires task information based on tasks that a user performs on a device, uses the task information and a prompt template to create a prompt, provides the prompt to a generation AI, acquires an answer from the generation AI, and acquires and outputs a summary of the user's task based on the answer.The device on which the user performs the task is, for example, a terminal device described below, but it may also be another device.

[0031] In this embodiment, a task summarization device that acquires summary information for each summarization unit time will be described. The summarization unit time may be a fixed time such as 30 minutes, or may be automatically determined by detecting a change in task.

[0032] In this embodiment, a task summarizing device will be described that transmits a summary of a task performed by a user to a report destination associated with the user identifier. The report destination may be, for example, the user's superior or a manager.

[0033] In addition, in this embodiment, we will describe a task summarization device that acquires information indicating word co-occurrence relationships from task information containing two or more words, determines word groups from the word co-occurrence relationships, constructs prompts for each word group, and acquires a summary for each word group.

[0034] Furthermore, in this embodiment, a task summarizing device that manages summary information together with screenshot images will be described.

[0035] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0036] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag on information Z, etc., and it is sufficient if information Z can be accessed.

[0037] 1 is a conceptual diagram of an information system A according to this embodiment. The information system A comprises an operation summarizing device 1, one or more terminal devices 2, and a generation AI device 3.

[0038] The work summarization device 1 is a device that acquires and outputs summaries of work performed by a user on the device. The work summarization device 1 may be, for example, a cloud server or an ASP server, but the type is not important. If the work summarization device 1 is a terminal, the information system A does not require a terminal device 2, or it can be thought that the work summarization device 1 also serves as the terminal device 2. If the work summarization device 1 is a terminal, the work summarization device 1 may be, for example, a so-called personal computer, smartphone, or tablet terminal, but the type is not important.

[0039] The terminal device 2 is a terminal used by a user. The terminal device 2 is typically a device on which a user performs work. The terminal device 2 may also be a terminal used by an administrator. A user is a person who performs work on a device. The terminal device 2 may be, for example, a personal computer, smartphone, or tablet device, but the type does not matter. The device on which work is performed is typically a computer, but the type does not matter. It is preferable that the device on which work is performed has a screen. The user typically operates the device while looking at the screen of the device. User work may also be referred to as user operation or user input. User work may be, for example, data entry, access to a database, etc., but the type does not matter.

[0040] The generation AI device 3 is a device that has the function of a generation AI. Here, the generation AI device 3 typically has the function of a text generation AI. The generation AI may be, for example, ChatGPT or Google Bard, but this does not matter. Google is a registered trademark. The generation AI device 3 may be, for example, a cloud server or an ASP server, but the type does not matter. The generation AI device 3 will be referred to as a generation AI as appropriate. The work summary device 1 may also have the function of a generation AI. In such a case, the generation AI device 3 is not necessary.

[0041] The work summarizing device 1 and one or more terminal devices 2, and the work summarizing device 1 and generation AI device 3, can communicate with each other via a network such as the Internet.

[0042] FIG. 2 is a block diagram of the information system A in this embodiment. FIG. 3 is a block diagram of the work summarizing device 1.

[0043] The work summarizing device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a template management unit 111 and a report destination management unit 112. The processing unit 13 includes a work information acquisition unit 131, a prompt construction unit 132, an answer acquisition unit 133, a summary acquisition unit 134, and a report destination acquisition unit 135. The work information acquisition unit 131 includes raw information acquisition means 1311, determination means 1312, and work information acquisition means 1313. The work information acquisition means 1313 includes word acquisition means 13131, co-occurrence count acquisition means 13132, graph construction means 13133, and division means 13134. The output unit 14 includes a summary output unit 141.

[0044] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit 26.

[0045] Various types of information are stored in the storage unit 11 that constitutes the work summarizing device 1. The various types of information include, for example, a prompt template (to be described later), a report destination (to be described later), and work information (to be described later).

[0046] One or more prompt templates are stored in the template management unit 111. For example, the template management unit 111 stores a first prompt template and a second prompt template.

[0047] A prompt template is information for creating a prompt to be given to the generation AI. A prompt template is information that serves as the basis for constructing a prompt to be given to the generation AI. A prompt template may have, for example, one or more variables. The prompt here is intended, for example, to obtain information from the generation AI that explains the content of the operation performed by the user on the device. The prompt is intended, for example, to summarize one or more sentences.

[0048] The first prompt template is a prompt template for obtaining information explaining the content of the work performed by the user from the generation AI based on the user's work information.

[0049] The second prompt template is a prompt template for summarizing two or more sentences.

[0050] One or more pieces of report destination information are stored in the report destination management unit 112. The report destination information is information that specifies a report destination to which summary information is to be reported. The report destination information has a user identifier and a report destination identifier.

[0051] A user identifier is information that identifies a user, such as a user ID, an email address, or a phone number.

[0052] The report destination identifier is information for identifying the report destination, such as the email address, phone number, or ID of the report destination, or the user ID of the notification tool used for reporting (for example, an SNS tool such as Slack).

[0053] The reception unit 12 receives various instructions and information. The various instructions and information include, for example, source information, operation information, and start instructions. The reception unit 12 usually receives the various instructions and information from the terminal device 2. However, the reception unit 12 may also directly receive the various instructions and information input by the user.

[0054] Here, acceptance typically refers to the reception of information transmitted via a wired or wireless communication line, but may also be a concept that includes the reception of information input from an input device such as a keyboard, mouse, or touch panel, or the reception of information read from a recording medium such as an optical disk, magnetic disk, or semiconductor memory.

[0055] The reception unit 12 receives, for example, two or more pieces of raw information from the terminal device 2. It is preferable that time information is associated with such raw information. The time information is information that specifies the time. The time information is, for example, the date and time, the time, or the time elapsed since the start time.

[0056] The raw information is the information that is the source from which work information is obtained. The raw information is information resulting from work performed by a user on a device. The raw information is usually information that can be obtained from the device on which the user is working. Examples of raw information include captured images, window images, input information, and event information.

[0057] A captured image is an image of a device screen. For example, a captured image is a screenshot of a device screen. For example, a captured image is an image of a device screen captured with a camera.

[0058] A window image is an image of a window on a device screen. A window image is, for example, a screenshot of a window on a device screen. A window image is, for example, an image of a window cut out from an image of the device screen captured by a camera. The window here is the window that the user operates. Such a window is called the active window or the current window.

[0059] Input information is information that a user inputs to a device. The input information is, for example, a command that a user inputs to a device, or input information. The command is, for example, an SQL statement for accessing a database. The input information may also be input data.

[0060] Event information is information that identifies an event that has occurred on a device due to a user's input to the device, such as LightButtonOn or DoubleClick.

[0061] The start instruction is an instruction to start obtaining summary information.

[0062] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by the task information acquisition unit 131, the prompt configuration unit 132, the answer acquisition unit 133, the summary acquisition unit 134, or the report destination acquisition unit 135.

[0063] The work information acquisition unit 131 acquires work information. For example, the work information acquisition unit 131 acquires work information stored on a server (not shown). For example, the work information acquisition unit 131 acquires work information from the storage unit 11. For example, the work information acquisition unit 131 acquires work information from a device on which a user performs work. The original storage location of the work information acquired by the work information acquisition unit 131 does not matter. It is preferable that the work information acquisition unit 131 continuously acquires work information. Continuously acquiring work information means acquiring work information at two or more points in time. Continuously acquiring work information may also mean acquiring work information from each original information at two or more points in time.

[0064] The work information is information about work performed by a user on a device. The work information is preferably information displayed on a screen on which the user performs work. The work information includes, for example, title information. The work information includes, for example, a window image.

[0065] Title information is a string of characters that indicates the title of a window on the screen. Title information is usually a string of characters that indicates the title of the active window.

[0066] It is preferable that the task information acquisition unit 131 acquires one or more pieces of task information for each of two or more summary unit times.

[0067] The summarization unit time is a portion of the total time taken for a series of tasks to be performed by a user. The summarization unit time is the time taken for the prompt composition unit 132 to acquire task information to be used when composing a single prompt. The summarization unit time is, for example, a predetermined fixed time.

[0068] It is preferable that the work information acquisition unit 131 acquires work information paired with a user identifier.

[0069] It is preferable that the work information acquisition unit 131 acquires work information that is displayed on a screen on which the user performs work.

[0070] The work information acquisition unit 131 acquires the title information of the active window from the original information, which is, for example, a captured image or a window image, by character recognition processing.

[0071] The work information acquisition unit 131 acquires, for example, a captured image or a window image received by the reception unit 12.

[0072] The work information acquisition unit 131 acquires work information, for example, periodically. The work information acquisition unit 131 acquires work information, for example, when an acquisition condition is met. The acquisition condition is a condition for acquiring work information. The acquisition condition is, for example, when an input from a user is accepted or when a screen is changed.

[0073] The work information acquisition unit 131 acquires work information using, for example, raw information acquisition means 1311, determination means 1312, and work information acquisition means 1313.

[0074] The raw information acquisition means 1311 acquires raw information at each of two or more times. The raw information acquisition means 1311 acquires raw information, which is, for example, a captured image, at each of two or more times.

[0075] The determination means 1312 determines whether the work performed by the user satisfies a switching condition. The switching condition is, for example, that a new window has become the active window. The switching condition is, for example, that the reception unit 12 has received specific event information. The switching condition is, for example, that a predetermined fixed time (for example, 30 minutes or 1 hour) has elapsed after the start of or switching to the work.

[0076] The task information acquisition means 1313 acquires task information for the summary unit time up to the time before the switching condition is satisfied, and also acquires task information for the summary unit time after the switching condition is satisfied.

[0077] The work information acquisition means 1313 acquires work information from the raw information acquired by the raw information acquisition means 1311 at each of two or more times. The work information acquisition means 1313, for example, acquires title information of the active window from the captured image or active window image acquired by the raw information acquisition means 1311 at each of two or more times. The work information acquisition means 1313, for example, acquires two or more pieces of event information acquired by the raw information acquisition means 1311 at each of two or more times.

[0078] It is preferable that the task information acquisition means 1313 acquires task information having words of each of two or more nodes that the subgraph has, for each of two or more subgraphs described later.

[0079] The word acquisition means 13131 acquires words that identify tasks at two or more times. The word acquisition means 13131 acquires one or more words from title information, which is, for example, a character string. The word acquisition means 13131 acquires one or more words by, for example, performing character recognition on a captured image or a window image. The word acquisition means 13131 acquires two or more words from two or more pieces of event information acquired by the raw information acquisition means 1311, for example, by referring to a correspondence table. The correspondence table here is a table having two or more pieces of correspondence information that have event information and character strings that describe the event. It is preferable that the word acquisition means 13131 associates the one or more acquired words with time information and stores them. In such a case, the one or more words may be associated with the stored captured image.

[0080] The co-occurrence count acquiring means 13132 acquires the number of co-occurrences of two words in a set of two words that occur at the same time at two or more times using the words acquired by the word acquiring means 13131. The co-occurrence count acquiring means 13132 acquires a set of co-occurrence relationship data having a structure of (target word A, target word B, co-occurrence count), for example. Such co-occurrence relationship data is, for example, (Google, Chrome, 2), (ABC-KINTAI, Cloud, 1), (email, Outlook, 1). The co-occurrence relationship data is data indicating the co-occurrence relationship between two words. It is preferable that the co-occurrence count acquiring means 13132 accumulates the co-occurrence relationship data in association with time information.

[0081] The graph construction means 13133 acquires graph data in which each of the two or more co-occurring words in each of the two or more pairs is a node, and the two nodes are connected by an edge whose weight is the number of times the two words co-occur.

[0082] When the co-occurrence relationship data (a, b, 10), (a, c, 1), (b, d, 10), and (c, e, 3) have been acquired, the graph construction means 13133 obtains a weighted undirected graph shown in FIG. 4 from the co-occurrence relationship between the two words and the number of co-occurrences.

[0083] The division means 13134 divides the graph data acquired by the graph construction means 13133 into two or more subgraphs using the edge connections between nodes and the edge weights. An image of the processing by the division means 13134 is shown in FIG. 5. In FIG. 5, the division means 13134 divides the graph data 501 acquired by the graph construction means 13133 into four subgraphs (502, 503, 504, 505). The division means 13134 constructs subgraphs by making two or more nodes connected to edges with weights equal to or greater than a threshold (dense edge connections) belong to the same subgraph. The processing by the division means 13134 is a well-known technique, and therefore a detailed description will be omitted. (See, for example, "Introduction to Large-Scale Graph Data Analysis," Internet URL: "https: / / www.ccs.tsukuba.ac.jp / wp-content / uploads / sites / 14 / 2016 / 10 / / shiokawa.pdf.")

[0084] It is preferable that the task information acquisition means 1313 acquires, for each of the two or more subgraphs acquired by the division means 13134, task information having words for each of the two or more nodes that the subgraph has.

[0085] The prompt composition unit 132 composes a prompt to be given to the generation AI using the task information and prompt template acquired by the task information acquisition unit 131. For example, the prompt composition unit 132 assigns the task information acquired by the task information acquisition unit 131 to a variable of the first prompt template to compose a first prompt to be given to the generation AI.

[0086] The prompt construction unit 132 constructs a prompt by, for example, substituting the task information acquired by the task information acquisition unit 131 into the variables of the prompt template.

[0087] It is preferable that the prompt composition unit 132 composes a prompt for each piece of summarized unit task information, which is task information for each of two or more summarized unit times.

[0088] It is preferable that the prompt construction unit 132 constructs a prompt by substituting the work information into a prompt template for each of two or more subgraphs.

[0089] It is preferable that the prompt composition unit 132 composes a prompt using task information and a prompt template for each of two or more summary unit times and each of two or more sub-graphs.

[0090] The prompt construction unit 132 constructs a second prompt, for example, by using all or part of two or more answers acquired from the generation AI and a second prompt template. The prompt construction unit 132 constructs the second prompt, for example, by substituting all or part of the two or more answers acquired from the generation AI into variables of the second prompt template.

[0091] The answer acquisition unit 133 passes the prompt constructed by the prompt construction unit 132 to the generation AI and acquires an answer from the generation AI. For example, the answer acquisition unit 133 assigns the prompt constructed by the prompt construction unit 132 to an argument of an API for accessing the generation AI, executes the API, and acquires an answer from the generation AI.

[0092] It is preferable that the answer acquisition unit 133 passes the first prompt constructed by the prompt construction unit 132 to the generation AI for each of the two or more subgraphs and acquires an answer from the generation AI. For example, the answer acquisition unit 133 passes the second prompt constructed by the prompt construction unit 132 to the generation AI and acquires an answer from the generation AI.

[0093] The summary acquisition unit 134 acquires summary information, which is information that explains the content of the work performed by the user and is information that uses the answer acquired by the answer acquisition unit 133. Note that the summary information may be the answer.

[0094] It is preferable that the summary acquisition unit 134 acquires summary information for each of two or more pieces of summarized unit work information.

[0095] The report destination acquisition unit 135 acquires, from the report destination management unit 112, a report destination identifier paired with a user identifier paired with the work information.

[0096] The output unit 14 outputs various types of information, such as summary information and captured images.

[0097] Here, output usually means transmission to terminal device 2, but it may also be a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to other external devices, storage on a recording medium, and handing over of processing results to other processing devices or other programs.

[0098] The summary output unit 141 outputs the summary information acquired by the summary acquisition unit 134. It is preferable that the summary output unit 141 transmits the summary information acquired by the summary acquisition unit 134 to a report destination identified by a report destination identifier acquired by the report destination acquisition unit 135, for example.

[0099] The summary output unit 141 outputs summary information for each of two or more summary unit times, for example. The summary output unit 141 outputs summary information that is information that summarizes the summary information for each of two or more summary unit times. The summary output unit 141 stores the summary information, for example, in pairs with the captured image.

[0100] Various types of information are stored in the terminal storage unit 21 included in the terminal device 2. The various types of information include, for example, a user identifier.

[0101] The terminal reception unit 22 receives various types of information, instructions, etc. Examples of the various types of information, instructions, etc. include an operation performed by a user, a login instruction, a logout instruction, and a start instruction that is an instruction from an administrator.

[0102] The login instruction is an instruction to log in to a server, application, etc. (not shown). The logout instruction is an instruction to log out from a server, application, etc. (not shown).

[0103] The means for inputting various information and instructions may be any means, such as a touch panel, keyboard, mouse, or menu screen.

[0104] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing to convert received information, instructions, etc. into information, instructions, etc. with a structure to be transmitted. For example, the various types of processing are processing to convert received information into information with a structure to be output.

[0105] The device processing unit 23 performs processing according to, for example, an operation received from a user.

[0106] The device processing unit 23 acquires raw information. The device processing unit 23, for example, constantly acquires raw information which is a captured image. The device processing unit 23, for example, acquires raw information which is operation information that identifies an operation accepted from a user. The device processing unit 23, for example, acquires raw information which is event information that has occurred in response to an operation accepted from a user. The device processing unit 23, for example, associates the acquired raw information with time information. The device processing unit 23, for example, acquires time information from a clock (not shown).

[0107] The terminal transmitting unit 24 transmits various types of information, instructions, etc. to the information processing device 1. The various types of information, instructions, etc. are, for example, source information and start instructions. It is preferable that the source information be associated with time information.

[0108] The terminal receiving unit 25 receives various types of information, such as summary information and captured images.

[0109] The terminal output unit 26 outputs various types of information, such as summary information and captured images.

[0110] The storage unit 11, template management unit 111, report destination management unit 112, and terminal storage unit 21 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0111] There is no restriction on the process by which information is stored in the storage unit 11 etc. For example, information may be stored in the storage unit 11 etc. via a recording medium, information transmitted via a communication line etc. may be stored in the storage unit 11 etc., or information input via an input device may be stored in the storage unit 11 etc.

[0112] The reception unit 12 is preferably realized by a wireless or wired communication means, but may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0113] The processing unit 13, task information acquisition unit 131, prompt construction unit 132, answer acquisition unit 133, summary acquisition unit 134, report destination acquisition unit 135, raw information acquisition means 1311, determination means 1312, task information acquisition means 1313, word acquisition means 13131, co-occurrence count acquisition means 13132, graph construction means 13133, division means 13134, and terminal processing unit 23 can typically be realized by a processor, memory, etc. The processing procedures of the processing unit 13, etc. are typically realized by software, and the software is recorded on a recording medium such as a ROM. However, they may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0114] The output unit 14 and the summary output unit 141 are usually realized by wireless or wired communication means, but may also be realized by driver software for an output device such as a display or speaker, or by a combination of driver software for an output device and the output device.

[0115] The terminal reception unit 22 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0116] The terminal transmitting unit 24 is usually realized by a wireless or wired communication means, but may also be realized by a broadcasting means.

[0117] The terminal receiving unit 25 is usually realized by a wireless or wired communication means, but may also be realized by a means for receiving broadcasts.

[0118] The terminal output unit 26 may be realized by, for example, driver software for an output device such as a display or speaker, or driver software for an output device and an output device.

[0119] Next, an example of the operation of the work summarizing device 1 will be described with reference to the flowchart of FIG.

[0120] (Step S601) The reception unit 12 determines whether or not a set of raw information has been received. If a set of raw information has been received, the process proceeds to step S602; if not, the process returns to step S601. The set of raw information includes two or more pieces of raw information. Each of the two or more pieces of raw information is usually associated with time information. Furthermore, a user identifier is associated with the set of raw information. Furthermore, the reception unit 12 receives the set of raw information, for example, after the user has completed their work. It is preferable that the reception unit 12 receive the set of raw information immediately after the user has completed their work.

[0121] (Step S602) The processing unit 13 acquires a user identifier.

[0122] (Step S603) The processing unit 13 stores each of the two or more pieces of raw information in association with the time information and the user identifier.

[0123] (Step S604) The processing unit 13 assigns 1 to the counter i.

[0124] (Step S605) The task information acquisition unit 131 performs processing to acquire task information using raw information for the i-th summary time unit. An example of such unit time task information acquisition processing will be described with reference to the flowchart in FIG.

[0125] (Step S606) The answer acquisition unit 133 etc. acquires an answer using the task information for the i-th summary time unit. An example of such answer acquisition processing will be described with reference to the flowchart of FIG.

[0126] (Step S607) The summary obtaining unit 134 obtains summary information using the answer obtained in step S606.

[0127] (Step S608) The summary obtaining unit 134 stores the summary information in association with the i-th summary time unit and the user identifier.

[0128] (Step S609) The processing unit 13 increments the counter i by one.

[0129] (Step S610) The processing unit 13 determines whether or not the i-th summarization time unit exists. If the i-th summarization time unit exists, the process returns to step S604, and if not, the process proceeds to step S611.

[0130] (Step S611) The summary obtaining unit 134 obtains output summary information. An example of such output summary information obtaining processing will be described with reference to the flowchart of FIG.

[0131] (Step S612) The summary obtaining unit 134 stores the output summary information in association with the user identifier and the original information.

[0132] (Step S613) The report destination acquisition unit 135 acquires, from the report destination management unit 112, a report destination identifier that pairs with the user identifier.

[0133] (Step S614) The summary acquisition unit 134 composes a report to be output. The report includes output summary information. The report includes, for example, summary information for each summary time unit. The report includes, for example, original information or a link to the original information (for example, a URL).

[0134] (Step S615) The summary output unit 141 transmits the report to the report destination indicated by the report destination identifier. The process returns to step S601.

[0135] In the flowchart of FIG. 6, the process ends when the power is turned off or an interrupt occurs to end the process.

[0136] Next, an example of the unit time task information acquisition process in step S605 will be described with reference to the flowchart in FIG.

[0137] (Step S701) The raw information acquiring means 1311 assigns 1 to a counter i.

[0138] (Step S702) The raw information acquisition means 1311 acquires the i-th piece of raw information from the set of acquired pieces of raw information.

[0139] (Step S703) The determination means 1312 determines whether the original information or the time information paired with the original information satisfies the switching condition. If the switching condition is satisfied, the process proceeds to step S712, and if not, the process proceeds to step S704.

[0140] (Step S704) The work information acquisition means 1313 acquires title information from the i-th raw information.

[0141] (Step S705) The word acquisition means 13131 divides the title information into words and acquires two or more words. In this case, it is preferable that the word acquisition means 13131 does not acquire words that match an exclusion condition. The exclusion condition is, for example, "the word is only one character," "the word is only numbers," or "the word is only symbols." The process of dividing the title information into words is, for example, a morphological analysis process, but is not limited to this.

[0142] (Step S706) The co-occurrence count acquiring unit 13132 acquires one or more word sets (word A, word B), which are combinations of two words, using the two or more words acquired in step S705.

[0143] (Step S707) The co-occurrence count obtaining unit 13132 assigns 1 to the counter j.

[0144] (Step S708) The co-occurrence count acquiring means 13132 determines whether the jth word set exists among the word sets acquired in step S706. If the jth word set exists, the process proceeds to step S709; if not, the process proceeds to step S711.

[0145] (Step S709) The co-occurrence count obtaining unit 13132 increments the co-occurrence count of the j-th word set by 1. Note that the initial value of the co-occurrence count of a word set is 0.

[0146] (Step S710) The co-occurrence count obtaining means 13132 increments the counter j by 1. The process returns to step S708.

[0147] (Step S711) The raw information acquiring means 1311 increments the counter i by 1. The process returns to step S702.

[0148] (Step S712) The graph construction unit 13133 creates graph data using a set of co-occurrence relationship data (for example, (word A, word B, number of co-occurrences)).

[0149] (Step S713) The dividing means 13134 divides the graph data obtained by the graph configuration means 13133 to obtain two or more subgraphs. The subgraphs are also graph data having nodes and edges.

[0150] (Step S714) The work information acquisition means 1313 assigns 1 to the counter i.

[0151] (Step S715) The work information acquisition means 1313 determines whether or not the i-th subgraph exists. If the i-th subgraph exists, the process proceeds to step S716, and if not, the process returns to the upper level process.

[0152] (Step S716) The work information acquiring means 1313 acquires the words of each of the two or more nodes that the i-th subgraph has.

[0153] (Step S717) The work information acquisition means 1313 acquires the work information having two or more words acquired in step S716 and temporarily stores it in a buffer (not shown). The work information is, for example, (word A, word B, word C, ...).

[0154] (Step S718) The work information acquisition means 1313 increments the counter i by 1. The process returns to step S715.

[0155] In the flowchart of FIG. 7, it is preferable that the co-occurrence count acquiring means 13132 accumulates one or more pieces of co-occurrence relationship data.

[0156] Next, an example of the response acquisition process in step S606 will be described with reference to the flowchart in FIG.

[0157] (Step S801) The prompt construction unit 132 acquires a first prompt template from the template management unit 111.

[0158] (Step S802) The prompt construction unit 132 assigns 1 to a counter i.

[0159] (Step S803) The prompt composition unit 132 determines whether the i-th task information exists in a buffer (not shown). If the i-th task information exists, the process proceeds to step S804; if not, the process returns to the upper process.

[0160] (Step S804) The prompt construction unit 132 constructs a first prompt by substituting the ith task information for the variables of the first prompt template.

[0161] (Step S805) The answer acquisition unit 133 passes the first prompt to the generation AI.

[0162] (Step S806) The answer acquisition unit 133 determines whether or not an answer has been acquired from the generation AI. If an answer has been acquired, the process proceeds to step S807, and if an answer has not been acquired, the process returns to step S806.

[0163] (Step S807) The answer acquiring unit 133 temporarily stores the answer acquired in step S806 in a buffer (not shown).

[0164] (Step S808) The prompt composition unit 132 increments the counter i by 1. The process returns to step S803.

[0165] Next, an example of the output summary information acquisition process in step S611 will be described with reference to the flowchart in FIG.

[0166] (Step S901) The prompt construction unit 132 acquires a second prompt template from the template management unit 111.

[0167] (Step S902) The prompt construction unit 132 acquires one or more pieces of summary information.

[0168] (Step S903) The prompt construction unit 132 constructs a second prompt by substituting one or more pieces of summary information for the variables of the second prompt template.

[0169] (Step S904) The answer acquisition unit 133 passes the second prompt to the generation AI.

[0170] (Step S905) The answer acquisition unit 133 determines whether or not an answer has been acquired from the generation AI. If an answer has been acquired, the process proceeds to step S906, and if an answer has not been acquired, the process returns to step S905.

[0171] (Step S906) The summary obtaining unit 134 obtains output summary information based on the response obtained in step S905, and returns to the upper-level processing.

[0172] Next, an example of the operation of the terminal device 2 will be described with reference to the flowchart of FIG.

[0173] (Step S1001) The terminal reception unit 22 determines whether or not a login instruction has been received. If a login instruction has been received, the process proceeds to step S1002, and if not, the process proceeds to step S1005.

[0174] (Step S1002) The terminal processing unit 23 performs a login process. With this login process, the user can start work using the terminal device 2.

[0175] (Step S1003) The device processing unit 23 acquires raw information, such as a captured image and an active window image.

[0176] (Step S1004) The device processing unit 23 acquires time information from a clock (not shown). The device processing unit 23 temporarily stores the original information in a buffer (not shown) in association with the time information. Return to step S1001. The buffer (not shown) may be the device storage unit 21.

[0177] (Step S1005) The terminal reception unit 22 determines whether or not an operation has been received from the user. If an operation has been received from the user, the process proceeds to step S1006, and if not, the process proceeds to step S1007.

[0178] (Step S1006) The device processing unit 23 and the like perform processing according to the operation. Go to step S1003.

[0179] (Step S1007) The terminal reception unit 22 determines whether or not a logout instruction has been received. If a logout instruction has been received, the process proceeds to step S1008, and if not, the process proceeds to step S1003.

[0180] (Step S1008) The terminal processing unit 23 acquires the raw information.

[0181] (Step S1009) The device processing unit 23 acquires the time information. The device processing unit 23 temporarily stores the original information in a buffer (not shown) in association with the time information.

[0182] (Step S1010) The device processing unit 23 acquires a set of raw information temporarily stored in a buffer (not shown). The raw information is associated with time information. The device processing unit 23 acquires a user identifier from the device storage unit 21 and associates the user identifier with the set of raw information. The terminal transmission unit 24 transmits the set of raw information associated with the user identifier and time information to the work summarizing device 1.

[0183] (Step S1011) The device processing unit 23 performs a logout process. Return to step S1001. It is assumed that the user's work using the terminal device 2 is completed by the logout process.

[0184] In the flowchart of FIG. 9, the process ends when the power is turned off or an interrupt occurs to end the process.

[0185] A specific example of the operation of the information system A in this embodiment will be described below.

[0186] The template management unit 111 of the work summarizing device 1 stores the first prompt template shown in Fig. 11, the second prompt template shown in Fig. 12, and the first prompt template shown in Fig. 13. In the prompt templates, character strings enclosed in "<" and ">" are variables.

[0187] The report destination management unit 112 stores a report destination management table shown in Fig. 14. The report destination management table is a table that manages one or more records each having an "ID," "user identifier," and "report destination identifier." In Fig. 14, the report destination is, for example, the user's superior or a manager.

[0188] In the above situation, the following two specific examples will be explained. Specific Example 1 is a case where the first prompt template of Fig. 11 and the second prompt template of Fig. 12 are used. Specific Example 2 is a case where the first prompt template of Fig. 13 and the second prompt template of Fig. 12 are used.

[0189] (Example 1) Let us assume that user X, identified by user identifier "U01," logs in to a server (not shown) using terminal device 2 at 9:00 on a certain day. After logging in, user X operates terminal device 2 to perform work on the server. Then, user X logs out of the server at 17:00 on the same day.

[0190] It is assumed that while user X is working (from login to logout), the device processing unit 23 of the terminal device 2 continuously acquires captured images (an example of original information) that are screen shots, pairs them with time information (here, the time), and stores them in the terminal storage unit 21 of the terminal device 2. It is assumed here that the device processing unit 23 stores the captured images paired with the time information in the terminal storage unit 21 at intervals of, for example, 10 seconds.

[0191] When the terminal receiving unit 22 receives a logout instruction, the device processing unit 23 acquires, from the terminal storage unit 21, photographic information that is two or more captured images and that is associated with time information. The two or more captured images are a collection of captured images. Next, the device processing unit 23 acquires the user identifier "U01" from the terminal storage unit 21 and associates the user identifier with the collection of captured images. Next, the terminal transmission unit 24 transmits the collection of captured images that are associated with the user identifier "U01" and the time information to the work summarizing device 1. The device processing unit 23 then performs logout processing. Note that the terminal device 2 may transmit one or more captured images that are associated with the user identifier "U01" and the time information to the work summarizing device 1 at all times or whenever it is possible to acquire them.

[0192] Next, the reception unit 12 of the work summarizing device 1 receives a set of captured images associated with the user identifier "U01" and time information from the terminal device 2. Next, the processing unit 13 acquires the user identifier "U01." Then, the processing unit 13 stores each of the two or more captured images in the storage unit 11, associated with the time information and the user identifier "U01."

[0193] Here, it is assumed that the switching condition is "30 minutes have passed." That is, the word acquisition means 13131 of the task information acquisition unit 131 performs image recognition processing using, for example, captured images during a 30-minute summary time unit, extracts an active window from the captured images, and acquires title information from the image of the active window through character recognition processing on the image of the active window. Then, it is assumed that the word acquisition means 13131 acquires, for example, word information shown in FIG. 15 from the title information for each captured image. Then, the word acquisition means 13131 stores, for example, the word information shown in FIG. 15 in association with a set of captured images during the summary time unit. Note that the word information shown in FIG. 15 is associated with time.

[0194] Next, the co-occurrence count acquiring means 13132 acquires co-occurrence relationship data "for example, (Google, Chrome, 2), (ABC-KINTAI, Cloud, 1), (email, Outlook, 1), ..." having a structure of (target word A, target word B, co-occurrence count) for each summarization time unit (30 minutes) from the information in Fig. 15. Note that it is preferable that the co-occurrence count acquiring means 13132 accumulates two or more pieces of co-occurrence relationship data in association with the captured image for the summarization time unit.

[0195] Next, the graph constructing means 13133 constructs graph data (see, for example, FIG. 4) for each summary time unit (30 minutes) using a large number of pieces of co-occurrence relationship data.

[0196] Next, the dividing means 13134 performs clustering processing on the graph data acquired by the graph configuration means 13133, divides the graph data, and acquires two or more subgraphs (see, for example, FIG. 5).

[0197] Next, the work information acquisition means 1313 acquires, for each of the two or more subgraphs, the words of each of the two or more nodes that the subgraph has, and constructs work information (for example, (word A, word B, word C, . . .)).

[0198] Next, the prompt construction unit 132 acquires a first prompt template (FIG. 11) from the template management unit 111. Next, the prompt construction unit 132 constructs a first prompt by substituting the task information acquired by the task information acquisition unit 1313 into the variable <task information> of the first prompt template for each summary time unit (30 minutes) and for each subgraph.

[0199] Next, the answer acquisition unit 133 passes a first prompt to the generation AI for each summary time unit (30 minutes) and for each subgraph, and acquires an answer from the generation AI. Next, the summary acquisition unit 134 acquires summary information including one or more answers from the generation AI for each summary time unit (30 minutes). The summary acquisition unit 134 associates the acquired summary information for each summary time unit with time zone information and temporarily stores it in a buffer (not shown). A large number of pieces of summary information associated with time zone information are called a summary set.

[0200] Next, the prompt construction unit 132 acquires the second prompt template of Fig. 12. The prompt construction unit 132 assigns the summary set acquired by the summary acquisition unit 134 to the variable <summary set> of the second prompt template to construct a second prompt.

[0201] Next, the answer acquisition unit 133 passes the second prompt to the generation AI and acquires an answer from the generation AI. The answer includes a summary of the morning work, a summary of the afternoon work, and information on User X's main work.

[0202] Next, the summary acquisition unit 134 uses the response from the generation AI to acquire output summary information for user X. The output summary information includes a summary of the morning work, a summary of the afternoon work, and information on user X's main work.

[0203] Next, the summary output unit 141 stores the output summary information acquired by the summary acquisition unit 134 in the storage unit 11 in association with the user identifier "U01", the set of co-occurrence relationship data, and the set of photographed images.

[0204] Next, the report destination acquisition unit 135 acquires the report destination identifier "ab@x.com" that pairs with the user identifier "U01" from the report destination management table (FIG. 14). Next, the summary acquisition unit 134 composes a report to be output. The report includes output summary information. The report includes, for example, summary information for each summary time unit. The report includes, for example, original information or a link to the original information (e.g., a URL). The summary output unit 141 sends the report by email to the report destination indicated by the report destination identifier "ab@x.com". An example of the report is shown in FIG. 16.

[0205] As a result of the above, a summary of User X's work status for the day is sent to User X's boss, identified by "ab@x.com."

[0206] (Example 2) As in Specific Example 1, assume that User X logs in to a server (not shown) using terminal device 2 at 9:00 on a certain day. After logging in, User X operates terminal device 2 to perform work on the server. Then, User X logs out of the server at 17:00 on the same day.

[0207] The reception unit 12 of the work summarizing device 1 then receives a set of captured images associated with the user identifier "U01" and time information from the terminal device 2. Next, the processing unit 13 acquires the user identifier "U01." The processing unit 13 then stores each of the two or more captured images in the storage unit 11 in association with the time information and the user identifier "U01."

[0208] Here, the switching condition is assumed to be "30 minutes have passed." That is, every 30 minutes, the work information acquisition unit 131 acquires an image of the active window from among the captured images included in that 30-minute period. That is, in this case, the image of the active window is the work information.

[0209] Next, the prompt construction unit 132 acquires the first prompt template (FIG. 13) from the template management unit 111.

[0210] Next, prompt composition unit 132 acquires two or more images of the active window for each summary time unit (30 minutes) that have changed from the previous image (for example, FIG. 17). For each summary time unit (30 minutes) and for each image, prompt composition unit 132 assigns the image of the active window acquired by work information acquisition unit 131 to the variable <image> of the first prompt template (FIG. 13) to compose a first prompt.

[0211] Next, the answer acquisition unit 133 passes the first prompt to the generation AI for each summary time unit (30 minutes) and for each subgraph, and acquires an answer from the generation AI (for example, "This is an image of browsing the "Azure AI" web page and learning about Web APIs.").

[0212] Next, the summary acquisition unit 134 acquires summary information including one or more answers from the generation AI for each summary time unit (30 minutes). The summary acquisition unit 134 associates the acquired summary information for each summary time unit with time zone information and temporarily stores the acquired summary information in a buffer (not shown).

[0213] Thereafter, the prompt construction unit 132 acquires the second prompt template of FIG. 12, and processing is carried out in the same manner as in the first specific example.

[0214] As described above, according to this embodiment, by using the generation AI to obtain a summary of the work content that the user has performed on the device, it becomes easier to understand the work content that the user has performed on the device.

[0215] Furthermore, according to this embodiment, by using the generation AI to obtain a summary of the work content for each summary unit time, it becomes easier to understand the work content that the user has performed on the device.

[0216] Furthermore, according to this embodiment, by using the generation AI to obtain a summary of the work content for each unit time, it becomes easier to understand the work content that the user has performed on the device.

[0217] Furthermore, according to this embodiment, by using the generation AI to obtain a summary of the work content for each time period of a series of cohesive tasks, it becomes easier to understand the work content that the user performed on the device.

[0218] Furthermore, according to this embodiment, a summary of the work performed by a user on a device can be reported to a reporter corresponding to the user.

[0219] Furthermore, according to this embodiment, a summary of the work performed by the user on the device is obtained using information displayed on the screen on which the user performs the work, making it easier to understand the work content.

[0220] Furthermore, according to this embodiment, a summary of the work performed by the user on the device is obtained using the title information of the window displayed on the screen on which the user is performing the work, making it easier to understand the work content.

[0221] Furthermore, according to this embodiment, a summary of the work performed by the user on the device can be obtained using a screenshot of the window displayed on the screen on which the user is performing the work, making it easier to understand the work content.

[0222] Furthermore, according to this embodiment, by obtaining a summary of the work that the user performed on the device for each word group that indicates the characteristics of the work, it becomes easier to understand the work that the user performed on the device.

[0223] In this embodiment, as shown in FIG. 18, summary information and captured images or window images are associated with each other, stored, and displayed, thereby making it possible to obtain a report that makes it easier for the user to understand the work they have done.

[0224] Furthermore, in this embodiment, as shown in FIG. 19, by linking the summary information with the title information that is the source of the summary information or one or more words and time information obtained from the title information, the time required for the task corresponding to the summary information can be determined.

[0225] Furthermore, in this embodiment, the processing in this embodiment may be realized by software. This software may be distributed by software download, etc. Furthermore, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software realizing the work summarizing device 1 in this embodiment is the following program. That is, this program causes a computer that can access a template management unit that stores prompt templates, which are information for acquiring information describing the content of work performed by a user on a device from a generation AI and are the basis for constructing prompts to be given to the generation AI, to function as: a work information acquisition unit that continuously acquires work information related to work performed by a user on a device; a prompt construction unit that constructs a prompt to be given to the generation AI using the work information acquired by the work information acquisition unit and the prompt template; an answer acquisition unit that passes the prompt constructed by the prompt construction unit to the generation AI and acquires an answer from the generation AI; a summary acquisition unit that acquires summary information, which is information using the answer acquired by the answer acquisition unit and describes the content of the work performed by the user; and a summary output unit that outputs the summary information.

[0226] FIG. 20 is a block diagram of a computer system 300 that executes the programs described in this specification to realize the work summarizing device 1 and the like according to the various embodiments described above.

[0227] In FIG. 20, a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0228] 20, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0229] A program that causes the computer system 300 to execute the functions of the work summarizing device 1 of the above-described embodiment and the like may be stored on a CD-ROM 3101, inserted into the CD-ROM drive 3012, and then transferred to the hard disk 3017. Alternatively, the program may be transmitted to the computer 301 via a network (not shown) and stored on the hard disk 3017. The program is loaded into the RAM 3016 when executed. The program may also be loaded directly from the CD-ROM 3101 or the network.

[0230] The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the work summarizing device 1 of the above-described embodiment. The program only needs to include instructions that call appropriate functions (modules) in a controlled manner to achieve the desired results. How the computer system 300 operates is well known, and a detailed description thereof will be omitted.

[0231] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0232] The computer that executes the program may be a single computer or a plurality of computers, that is, it may perform centralized processing or distributed processing.

[0233] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0234] Furthermore, in each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0235] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0236] As described above, the work summarizing device 1 according to the present invention has the effect of making it easier for a user to understand the work content performed on the device, and is useful as a server or the like that receives original information and transmits work summary information. [Explanation of symbols]

[0237] A. Information Systems 1. Work summary device 1. Information processing equipment 2. Terminal Device 3 Generation AI device 11 Storage area 12 Reception 13 Processing section 14 Output section 21 Terminal storage section 22 Terminal Reception 23 Terminal processing section 24 Terminal transmitter 25 Terminal receiving section 26 Terminal Output Unit 111 Template Management Department 112 Reportee Management Department 131 Work information acquisition unit 132 Prompt Component 133 Answer acquisition part 134 Summary Acquisition Unit 135 Report acquisition department 141 Summary Output Section 1311 Source information acquisition means 1312 Judgment means 1313 Work information acquisition means 13131 Word Acquisition Method 13132 Co-occurrence count acquisition method 13133 Graph Construction Method 13134 Division means

Claims

1. a template management unit that stores prompt templates, which are information for obtaining information from the generation AI that explains the content of the work performed by the user on the device and are the basis for constructing prompts to be given to the generation AI; a work information acquisition unit that acquires work information related to work performed by a user on the device; a prompt composition unit that composes a prompt to be given to a generation AI using the task information acquired by the task information acquisition unit and the prompt template; an answer acquisition unit that passes the prompt constructed by the prompt construction unit to the generation AI and acquires an answer from the generation AI; a summary acquisition unit that acquires summary information, which is information using the answer acquired by the answer acquisition unit and is information explaining the content of the work performed by the user; and a summary output unit that outputs the summary information.

2. The work information acquisition unit Acquire task information for each of two or more summary unit times, which are a portion of the total time during which the user has performed a series of tasks; The prompt composition unit A prompt is configured for each of the summary unit task information, which is task information for each of the two or more summary unit times; The summary acquisition unit Acquire the summary information for each of the two or more summarized unit work information; The summary output unit 2. The work summarizing device according to claim 1, wherein the summary information is output for each of the two or more summary unit times.

3. 3. The task summarizing device according to claim 2, wherein the summarization unit time is a predetermined fixed time.

4. The work information acquisition unit a determination means for determining whether or not the operation performed by the user on the device satisfies a switching condition; 3. The task summarizing device according to claim 2, further comprising task information acquisition means for acquiring task information for a summarization unit time up to the time before the switching condition is satisfied, and for acquiring task information for a summarization unit time after the switching condition is satisfied.

5. The work information acquisition unit Acquire task information paired with a user identifier that identifies the user; The system further includes a report destination acquisition unit that acquires a report destination identifier paired with a user identifier paired with the work information from a report destination management unit that stores report destination identifiers that identify report destinations to which the summary information is to be sent, in pairs with two or more user identifiers; The summary output unit The work summarizing device according to claim 1 , wherein the summary information is transmitted to a report destination identified by the report destination identifier acquired by the report destination acquisition unit.

6. The work information acquisition unit The task summarizing device according to claim 1 , wherein task information displayed on a screen on which the user performs a task is acquired.

7. 7. The task summarizing device according to claim 6, wherein the task information includes title information that is a character string indicating the title of a window on the screen.

8. 7. The task summarizing device according to claim 6, wherein the task information includes a window image that is an image of a window on the screen.

9. The work information acquisition unit a word acquisition means for acquiring words that identify the work at each of two or more times; a co-occurrence count acquiring means for acquiring the number of co-occurrences of two words occurring at the same time for each of two or more times using the words acquired by the word acquiring means; a graph construction means for acquiring graph data in which each of the two or more co-occurring words in each of the two or more pairs is a node and the two nodes are connected by an edge whose weight is the number of times the two words co-occur; a division means for dividing the graph data acquired by the graph construction means into two or more subgraphs using edge connections between nodes and edge weights; and a task information acquisition means for acquiring task information having words of each of the two or more nodes included in the subgraph for each of the two or more subgraphs, The prompt composition unit Substituting the work information into the prompt template for each of the two or more subgraphs to construct a prompt; The answer acquisition unit 9. The work summarizing device according to claim 1, wherein the prompt constructed by the prompt construction unit is passed to a generation AI for each of the two or more subgraphs, and an answer is obtained from the generation AI.

10. The work information acquisition unit raw information acquisition means for acquiring raw information, which is a captured image of the screen of the device, at each of two or more times; 10. The task summarizing device according to claim 1, further comprising task information acquisition means for acquiring the task information from the raw information acquired by the raw information acquisition means at each of two or more times.

11. The summary output unit 11. The work summarizing device according to claim 10, wherein the summary information is stored in pairs with the photographed image.

12. A task summarization method implemented by a template management unit that stores prompt templates, which are information for obtaining from a generation AI information explaining the content of a task performed by a user on a device and are information that serves as the basis for constructing prompts to be given to the generation AI, a task information acquisition unit, a prompt construction unit, an answer acquisition unit, a summary acquisition unit, and a summary output unit, a work information acquisition step in which the work information acquisition unit acquires work information related to work performed by a user on a device; a prompt configuration step in which the prompt configuration unit configures a prompt to be provided to a generation AI using the work information acquired by the work information acquisition unit and the prompt template; an answer acquisition step in which the answer acquisition unit passes the prompt constructed by the prompt construction unit to the generation AI and acquires an answer from the generation AI; a summary acquisition step in which the summary acquisition unit acquires summary information, which is information using the answer acquired by the answer acquisition unit and is information explaining the content of the work performed by the user; a summary output step in which the summary output unit outputs the summary information.

13. a computer that can access a template management unit that stores prompt templates, which are information for obtaining from the generation AI information explaining the content of the work performed by the user on the device and are information that serves as the basis for constructing prompts to be given to the generation AI; a work information acquisition unit that acquires work information related to work performed by a user on the device; a prompt composition unit that composes a prompt to be given to a generation AI using the task information acquired by the task information acquisition unit and the prompt template; an answer acquisition unit that passes the prompt constructed by the prompt construction unit to the generation AI and acquires an answer from the generation AI; a summary acquisition unit that acquires summary information, which is information using the answer acquired by the answer acquisition unit and is information explaining the content of the work performed by the user; A program for causing the summary output unit to function as a summary output unit that outputs the summary information.

Images