Information processing device and program

Abstract

To provide an information processing device and a program for improving the accuracy of retrieval of information related to a specific business theme as compared with a case of estimating the business theme using only information used during work.SOLUTION: In a user terminal consisting of a main body of a computer, a display, a keyboard, and a mouse, the main body 100 of the computer includes a business theme estimation part 111 having a function of displaying, on a screen, a graph which expresses the transition relationship between active windows identified from an operation log 121 and the strength of the connection between the active windows having a transition relationship through program execution, and a function for displaying an estimated result of business using the active window corresponding to each node of the graph.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] There are times when you want to check the content displayed in the window used during work (hereinafter referred to as the "active window") at another time. For example, there are times when you want to check the information referred to during the previous procedure during the next procedure. If the information referred to during the previous procedure can be displayed without omission, the labor of information collection can be saved, and an improvement in work efficiency is expected. For the purpose of supporting this user request, a technique has been developed to display a screenshot of the active window displayed in relation to a specific task by estimating and recording the tasks related to the information displayed in the active window. For example, Patent Document 1 describes a technique for estimating the task of the content displayed in the active window based only on the information of the content.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when the content such as the file name of the content or the title appearing in the content is abstract or the relevance between these contents and the task is low, it is difficult to estimate the task, and there is a possibility that an incorrect task may be recorded in association with the content. If an incorrect task is associated with the content, not only will information unintended by the user be presented and work efficiency will decrease, but there is also a problem that the required information cannot be found.

[0005] The present invention aims to improve the accuracy of retrieving information related to a specific task compared to estimating a task using only the information used during the task. [Means for solving the problem]

[0006] The invention described in claim 1 relates to a computer, A function to read operation logs from a storage device, identify transitions from an active window to an inactive window that appear in the read operation logs as relationships between windows, and a function to identify the strength of the connection between windows with such transition relationships from the operation logs, with the active window as a node and the active window as an inactive window. Relationship of transitions Let be an edge, and the relationship of the transitions described above Strength of connections between windows in the display manner of the edge A function to display the graph on the screen, and an active window corresponding to each node of the graph. Using the strength of the connection between windows that have the aforementioned transition relationship, the active window Estimate the workload The function of the above estimation This is a program that implements the function of displaying the results. The invention described in claim 2 is the program described in claim 1, wherein the strength of the bond between active windows is determined by recording the number of transitions between windows having the transition relationship. The invention described in claim 3 is the program described in claim 1, wherein, in the estimation of the business, the business theme of the active window is estimated from the operation log of the active window, and if the business theme cannot be estimated, the business theme of the active window is estimated using the strength of the connections between windows that have the transition relationship. The invention described in claim 4 is the program described in claim 3, which obtains the title and / or content of the active window from the operation log and estimates the business theme. Claim 5 The invention described above Estimated The function for displaying results is the program according to claim 1, which displays the similarities and differences of the estimated tasks in an identifiable manner on the graph. Claim 6 The invention described above Estimated The function for displaying results expresses the differences in estimated tasks by differences in the display manner of the corresponding nodes, claim 5 This is the program described in [the document]. Claim 7 The invention described above Estimated The function for displaying results represents the range of nodes estimated to be in the same business using a graphic representation, claim 5 This is the program described in [the document]. Claim 8 The invention described is The function for estimating the aforementioned tasks estimates the tasks and their confidence levels using values ​​calculated based on rule-based methods, title similarity, keyword similarity, or confidence levels calculated for active windows with transition relationships. The aforementioned Estimated The function for displaying results is to represent the differences in the confidence levels of the estimated tasks in an identifiable manner on the graph, according to the claim. 1 or 5 This is the program described in [the document]. Claim 9 The invention described above EstimatedThe function of displaying the results is a program according to claim 8 which expresses the difference in the confidence level of the estimated operation by the difference in the display mode of the corresponding node. The invention according to claim 10 further has a function of correcting the results according to the content of the received correction when receiving a correction for the estimated operation, and is a program according to claim Estimated The function of displaying the results is a program according to claim 8 which expresses the difference in the confidence level of the estimated operation by a figure. The invention according to claim 11 The function of displaying the graph on the screen is a program according to claim 1, which represents the multiple active windows displayed multiple times by one node and represents the strength of the connection between the active windows by the length of the edge connecting each node. 9 The invention according to claim The invention according to claim 12 has a processor, and the processor uses the relationship of window transitions to estimate the operation of the active window, and is an information processing device. Claim 13 The invention according to claim The operation log is read from the storage device, the transition from the active window to the inactive window that appears in the read operation log is identified as a transition relationship between windows, the strength of the connection between the windows with the transition relationship is identified from the operation log, the active window is represented as a node, the transition relationship from the active window to the inactive window is represented as an edge, and a graph is displayed on the screen that represents the strength of the connection between the windows with the transition relationship in the display form of the edge, and the active window corresponding to each node in the graph is displayed. window and the above transition relationship The strength of the connections between windows. to estimate the operation of the active window, and is an information processing device. of estimate the operation The results of the above estimation are displayed. using the relationship of window transitions, and is an information processing device. 。

Advantages of the Invention

[0007] According to the invention described in claim 1, it is possible to improve the accuracy of searching for information related to a specific operation as compared with the case of estimating the operation estimated using only the information used during the operation. According to the invention described in claim 2, compared to estimating tasks using only the information used during the work, it is possible to improve the accuracy of searching for information related to a specific task. According to the invention described in claim 3, compared to estimating tasks using only the information used during the work, it is possible to improve the accuracy of searching for information related to a specific task. According to the invention described in claim 4, compared to estimating tasks using only the information used during the work, it is possible to improve the accuracy of searching for information related to a specific task. According to the invention described in claim 5 it is possible to facilitate the confirmation of the similarities and differences of the estimated operations. According to the invention described in claim 6 it is possible to facilitate the confirmation of the similarities and differences of the estimated operations. According to the invention described in claim7 According to the described invention, it is possible to easily verify the similarities and differences between presumed business operations. Claim 8 According to the described invention, it is possible to easily confirm differences in the degree of confidence of estimated business operations. Claim 9 According to the described invention, it is possible to easily identify nodes with a low degree of confidence in the estimated operations. Claim 10 According to the described invention, it is possible to easily identify nodes with a low degree of confidence in the estimated operations. Claim 11 According to the described invention, it is possible to avoid the inclusion or omission of information unrelated to specific tasks. Claim 12 According to the described invention, the relationships between tasks can be confirmed by the length of the edges. Claim 13 According to the described invention, compared to estimating tasks using only the information used during the work, it is possible to improve the accuracy of searching for information related to specific tasks. 。 [Brief explanation of the drawing]

[0008] [Figure 1] This diagram illustrates an example configuration of a user terminal operated by a user. (A) shows an example of the external appearance of the user terminal, and (B) shows an example of the hardware configuration of the user terminal. [Figure 2] This diagram illustrates an example of a functional configuration achieved through program execution. [Figure 3] This diagram illustrates an example of a record of transitions between active windows. (A) is a diagram illustrating the transitions between active windows, and (B) is a diagram illustrating the number of transitions. [Figure 4] This diagram illustrates an example of the data structure of a business theme dictionary. [Figure 5] This figure shows an example of a graph illustrating the relationship between transitions between active windows. [Figure 6] This diagram illustrates the relationship between the estimated results before and after modification. [Figure 7]This flowchart illustrates an example of the estimation process for business themes. [Figure 8] This flowchart illustrates an example of the process performed in step 31. [Figure 9] This is a flowchart illustrating an example of the process performed in step 33. [Figure 10] This is a flowchart illustrating an example of the process performed in step 35. [Figure 11] This flowchart illustrates an example of the process performed in Step 5. [Figure 12] This diagram illustrates the process in Step 53. (A) shows a list of active windows related to active window Xj (j=1, 2…) whose business theme has not yet been estimated, and (B) shows the confidence level for each business theme estimated from the related active windows. [Figure 13] This is a flowchart illustrating an example of the process performed in Step 6. [Figure 14] This flowchart illustrates another example of the process performed in Step 6. [Figure 15] This diagram illustrates an example of how to display the active window used in related tasks. (A) is a graph showing the relationship between transitions between active windows, (B) shows an example of how to display the active window corresponding to the node specified by the user with cursor K, and (C) shows an example of how to display information about the specified node. [Figure 16] This diagram illustrates other examples of how the active window used in related tasks can be displayed. (A) is a diagram showing the active window in timeline format, (B) shows an example of how the active window can be displayed corresponding to the node specified by the user with cursor K, and (C) shows an example of how information about the specified node can be displayed. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. <Terminology> First, let's explain the terminology used in this embodiment. The "active window" refers to the window that the user is currently using or has used during their work. Even if multiple windows are displayed on the screen, only one window is used for work at a time. Therefore, there is only one active window at any given moment. For example, when multiple windows are displayed overlapping, the window closest to the viewer is the active window. Also, when multiple windows are displayed side-by-side on the screen, the window with a higher brightness in its frame or title row compared to the others is the active window.

[0010] The active window corresponds to an application program (hereinafter referred to as "app" or "program"). Examples of apps include browsers, email clients, office software, and drawing software. In this embodiment, windows other than the active window are referred to as inactive windows. For example, if five windows are displayed on the screen, there is one active window and four inactive windows.

[0011] An "operation log" is a record of user actions on a computer and the active window that was the target of those actions. The operation log includes, for example, the date and time of the operation, the content of the operation, the user who instructed the operation, and other recorded information. The "business theme" is the objective of the work. In this embodiment, the business theme is defined in the business theme dictionary. Business themes vary depending on the industry and job type.

[0012] "Business" refers to work performed by multiple people, while "tasks" refer to work performed by an individual. Businesses are often performed by a department or section. Tasks include research, document creation, and coding. Furthermore, the term "business" in the patent claims is used to mean "business theme," that is, the purpose of the business. In addition, the term "business" in the patent claims includes not only business themes involving multiple people, but also themes of work performed by individuals.

[0013] A "graph" consists of nodes that represent the active window and edges that connect the nodes. Nodes are represented, for example, by a circle. However, the shapes used to represent nodes are not limited to circles. For example, triangles, quadrilaterals, or other polygons may also be used. In this embodiment, nodes with the same business theme are represented by the same color. Furthermore, in this embodiment, the confidence level of the estimated business theme is represented by differences in brightness or gradation.

[0014] An edge is a line segment that represents the transition relationship between nodes. In this embodiment, the distance of the edge connecting strongly connected nodes is short, and the distance of the edge connecting weakly connected nodes is long. The strength of a connection can be determined, for example, by the number of transitions. Nodes with many transitions have strong connections, while those with few transitions have weak connections. Alternatively, the thickness or color of edges can also be used to represent the strength of a connection.

[0015] <Device configuration> Figure 1 illustrates an example configuration of a user terminal (hereinafter referred to as "user terminal") 1. (A) is an example of the external appearance of user terminal 1, and (B) is an example of the hardware configuration of user terminal 1. The user terminal 1 shown in Figure 1(A) consists of a main unit 100, a display 110, a keyboard 120, and a mouse 130.

[0016] The main unit 100 is the main body of the computer. The main unit 100 shown in Figure 1(B) includes a processor 101 that controls the operation of the entire terminal, a ROM (Read Only Memory) 102 in which the BIOS (Basic Input Output System) and other data are stored, a RAM (Random Access Memory) 103 used as the work area of ​​the processor 101, an auxiliary storage device 104, and a communication device 105. The processor 101 and each device are connected via buses and other signal lines.

[0017] The processor 101 is a device that performs various functions through the execution of a program. In this embodiment, the processor 101 implements, for example, functions to estimate the relationship between the active window and the business theme, functions to visualize the estimated results, and other functions through program execution. The auxiliary storage device 104 is, for example, a hard disk drive or semiconductor storage. Programs, operation logs, and the like are stored in the auxiliary storage device 104. The term "program" is used as a general term for the OS (Operating System) and application programs.

[0018] The communication device 105 is a device that enables communication with external devices and terminals. The communication device 105 conforms to, for example, serial cables, HDMI (=High-Definition Multimedia Interface) (registered trademark), Bluetooth (registered trademark), WiFi (registered trademark), and Ethernet (registered trademark). The display 110 is a device that displays information and is an example of an output device. The display 110 is, for example, a liquid crystal display or an organic EL (=Electro Luminescence) display. Keyboard 120 and mouse 130 are devices for inputting information, and are examples of input devices.

[0019] <Functional Configuration> The following section describes the functional configuration of user terminal 1 using Figures 2 to 6. Figure 2 illustrates an example of a functional configuration realized through program execution. The functions shown in Figure 2 represent the parts of the functions realized by processor 101 that relate to the estimation of business themes related to the active window and the visualization of the estimated results. The business theme estimation unit 111 is a functional unit that calculates the business theme and confidence level of the active window.

[0020] The business theme estimation unit 111 uses the operation log 121 and the business theme dictionary 122 to calculate the business theme and its confidence level. The operation log 121 retrieves information such as the title of the active window, the content contained in the active window, the application used to display the active window, the source active window, the destination active window, the date and time of the transition, the date and time the active window was used (hereinafter also referred to as "usage date and time"), the duration the active window was used (hereinafter also referred to as "usage duration"), the user's account, a screenshot of the active window, and information received through the keyboard 120 (see Figure 1) and mouse 130 (see Figure 1).

[0021] If a screenshot is obtained from the operation log 121, OCR (Optical Character Recognition / Reader) processing is used to extract text and accompanying information contained in the screenshot. The extracted text is used by the work estimation unit 112. The accompanying information includes details such as the layout of text and images, and the size of the font.

[0022] Figure 3 illustrates an example of a record of transitions between active windows. (A) is a diagram illustrating the transitions between active windows, and (B) is a diagram illustrating the number of transitions. Furthermore, if the active window is used for a short period of time, the corresponding transition may be considered noise and excluded from the recorded number of transitions. For example, the active window may switch due to a user error or misunderstanding. In Figure 3(A), the active window before the transition is "0001", and the active window after the transition is "0002". As shown in Figure 3(A), the window located behind the active window is an inactive window.

[0023] Figure 3(B) shows the numbers in each row that identify the screen corresponding to the active window before the transition, and the numbers in each column that identify the screen corresponding to the active window after the transition. In Figure 3(B), the numbers at the coordinates where each row and each column intersect represent the number of transitions. For example, the number of transitions from active window "0001" to active window "0002" is 3, and the number of transitions from active window "0002" to active window "0001" is 2.

[0024] The business theme dictionary 122 retrieves keywords used to determine similarity to pre-defined business themes. Figure 4 illustrates an example of the data structure of the business theme dictionary 122. The business theme dictionary 122 contains keywords used for rule-based judgment (hereinafter referred to as "rule-based judgment keywords") 122B and keywords used for similarity judgment (hereinafter referred to as "similarity judgment keywords") 122C, associated with business theme 122A.

[0025] In Figure 4, the business themes are divided into two categories: major and minor. For example, the major category "Business Development" includes two minor categories: "Business Process Improvement Support" and "Satellite Office Business." In Figure 4, two or three subcategories are registered for one major category, but it is also acceptable for only one subcategory to be registered for one major category.

[0026] Keyword 122B for rule-based determination and keyword 122C for similarity determination are both keywords related to the business theme. Rule-based determination keyword 122B is a keyword used when determining related business themes using a rule-based approach.

[0027] Rule-based determination refers to a process that determines relationships based on, for example, whether or not they match keywords. Keywords of this type include, for example, the application that displays the active window, the URL (Uniform Resource Locator) that indicates the network location within the active window, and the title.

[0028] The similarity determination keyword 122C is used when determining a relationship using a confidence score calculated from the similarity between keywords. Similarity-based determination refers to a process that calculates a confidence score from the similarity between keywords contained in the active window's title and keywords registered in the dictionary, and determines the relationship based on whether the calculated confidence score is above a threshold α.

[0029] The similarity between two keywords is calculated, for example, as the cosine distance between the two vectors. Note that, as a prerequisite for calculating similarity, the strings contained in the active window are converted into vectors using natural language processing. This type of keyword database includes keywords that frequently appear in business-related topics, for example. Keywords are not limited to single words; they can also include strings of characters. Figure 4 shows examples of keywords, including words such as "proposal," "quote," and "sales," as well as strings such as "business process improvement support" and "satellite office business."

[0030] The business theme estimation unit 111 is equipped with a function to generate a graph that represents the strength of the relationships between active windows using information on transitions between active windows. Figure 5 shows an example of a graph illustrating the relationship between transitions between active windows. The circles in the diagram represent nodes, each representing an active window. Each circle represents one active window. There is a one-to-one correspondence between nodes and active windows. Therefore, even active windows that are displayed multiple times are represented by a single node. The line segments connecting two circles are called edges, and their length represents the strength of the connection.

[0031] The strength of a bond is expressed, for example, by the length, thickness, and color of the edges. For instance, a short edge indicates a strong bond, while a long edge indicates a weak bond. Similarly, a thick edge indicates a strong bond, while a thin edge indicates a weak bond. Finally, red edges indicate a strong bond, while blue edges indicate a weak bond.

[0032] In this embodiment, the strength of the connection is evaluated, for example, by the number of transitions. The number of transitions can be obtained from the operation log 121. Furthermore, a strong connection can also be described as a "strong relationship" or "high level of relationship," while a weak connection can be described as a "weak relationship" or "low level of relationship." The business theme estimation unit 111 is also equipped with a function to correct the business theme estimation result when the confidence level of the estimated business theme is low, by using information from an active window that has a high relationship with the target active window and also has a high confidence level for its own business theme.

[0033] Figure 6 is a diagram illustrating the relationship before and after the correction of the estimation results. The chart shown in Figure 6 consists of a column for the log ID that identifies the active window, a column for the estimated business theme, a column for the confidence level calculated for the estimated business theme, a column for the business theme after correcting the estimation result, a column for the confidence level calculated for the business theme after correcting the estimation result, and a column for the actual business theme. In the case of Figure 6, there are three types of business themes: A, B, and C.

[0034] Incidentally, the "-" symbol attached to log IDs "0002" and "0012" indicates that the business theme is undecided. In Figure 6, the "-" symbol is attached to the column for the business theme corresponding to the active window where the confidence level of the estimated business theme is less than 0.2. 0.2 is an example of a threshold α. In this embodiment, the business themes of the active window whose confidence level calculated in the initial estimation is less than 0.6 are targeted for modification. 0.6 is an example of a threshold β.

[0035] In the example in Figure 6, the initial confidence score calculated for log IDs "0002", "0006", "0007", "0010", "0012", "0013", "0014", "0015", and "0020" is less than 0.6. Therefore, a second estimation of the business theme is performed for these active windows. For example, for log ID "0002," the business theme has been changed from "undetermined" to "A." The confidence level has increased from 0.10 to 0.50. For log ID "0010," the business theme has been changed from C to A. The confidence level has increased from 0.35 to 0.50. For log ID "0006," the business theme has been changed from C to B. However, the confidence level remains at 0.55.

[0036] The business theme estimation unit 111 is also equipped with a function to display the results of the business theme estimation as a graph. The graph in question is the one shown in Figure 5. The graph clearly displays the similarities and differences in business themes corresponding to each active window. The differences in estimated business themes are represented by differences in the display of the corresponding nodes. For example, they are represented by differences in node color. In Figure 5, color differences are represented by differences in intensity. Therefore, sets of nodes represented by the same intensity are related to the same business theme. In addition, differences in business themes may be represented by differences in the shapes used to represent the nodes. Furthermore, differences in business themes may be distinguished by differences in the size and brightness of the shapes representing the nodes. They may also be represented by differences in the borders or background colors indicating the range of nodes corresponding to the same business theme. These representations using shapes and borders are examples of differences in display characteristics.

[0037] In addition, differences in the confidence level of the calculated business themes are indicated by differences in the display of the corresponding nodes. For example, nodes in the active window with a confidence level higher than the standard are represented by circles, while nodes in the active window with a confidence level lower than the standard are represented by rectangles. Alternatively, differences in confidence levels may be represented by differences in node color or the size of the shapes representing the nodes. Furthermore, the brightness of nodes in active windows with a calculated confidence level lower than the standard may be set higher than the brightness of nodes in active windows with a confidence level higher than the standard. The difference in brightness will be set so that it can be distinguished by the user. These differences in shape and brightness are examples of differences in display characteristics. The display of the estimation results may also be performed using the functions of the estimation result visualization unit 114.

[0038] In addition, the business theme estimation unit 111 also has a function to accept modifications to business themes based on user input. User input to modify a business theme is executed when a modification instruction is received for a specific active window in the graph shown in Figure 5 or the chart shown in Figure 6. When a user enters a revised business theme, the estimation results shown in Figure 6 will be overwritten with the user's input.

[0039] The task estimation unit 112 is a functional unit that estimates the content of the task to be performed through the active window, the target of the task, and the tools to be used for the task. The tasks include, for example, data entry, viewing, printing, and uploading. The objects of the work include, for example, electronic documents, web pages, and people. Tools include, for example, the names of application programs and systems. The work estimation unit 112 uses the operation log 121 and the work content dictionary 123 to estimate the content of the work, etc. The work content dictionary 123 registers the relationship between the information included as a record in the operation log and the content of the work.

[0040] The task estimation unit 113 is a functional unit that displays a set of active windows with the same or similar business themes in an identifiable manner. The graph mentioned above is an example of an identifiable display. Furthermore, charts and graphs displaying data chronologically in sets are also examples of identifiable displays. The estimation result visualization unit 114 is a functional unit that presents the results estimated by the business theme estimation unit 111 and the task estimation unit 113 to the user.

[0041] <Estimation process for business themes> The following sections will explain the process of estimating business themes using Figures 7 to 14. <Overview> Figure 7 is a flowchart illustrating an example of the estimation process for business themes. The symbol S shown in the diagram stands for step. The process shown in Figure 7 is implemented by processor 101 (see Figure 1) through program execution. First, processor 101 obtains the operation log of the active window (Step 1). In the case of Figure 7, the total number of active windows to be processed is Nmax.

[0042] Next, processor 101 determines whether n ≤ Nmax or not (step 2). n is the number of active windows on which the process in step 3 was executed. Therefore, the initial value of n is 0. If a positive result is obtained in step 2, processor 101 estimates the business theme of the active window to be processed (step 3).

[0043] When n > Nmax, a negative result is obtained in step 2. In this case, processor 101 determines whether or not a business theme has been estimated for all active windows (step 4). In other words, for all active windows, it is determined whether or not the confidence level of the estimated business theme is greater than or equal to the threshold β. The threshold β here is 0.6. If a negative result is obtained in step 4, i.e., if there is an active window where the confidence level of the estimated business theme is less than the threshold β, the processor 101 analyzes the relevant active window and re-estimates the business theme (step 5). On the other hand, if a positive result is obtained in step 4, or after step 5 is executed, the processor 101 updates the business theme dictionary 122 (step 6).

[0044] <Details of each process> The following describes the details of the processes performed at each step. <Step 3> Step 3 consists of substeps 31 through 38. As mentioned above, each substep performs processing on the operation log of the active window that is the target of processing among the Nmax active windows.

[0045] <Steps 31-32> The processor 101 determines the business theme and confidence level based on rules (step 31), and determines whether the confidence level is greater than the threshold α (<β) (step 32). In this embodiment, the threshold α is set to 0.2. Figure 8 is a flowchart illustrating an example of the process performed in step 31. First, the processor 101 determines whether the title contains a specific keyword (step 311). The specific keyword is read from the rule-based determination keyword 122B (see Figure 4) of the business theme dictionary 122 (see Figure 2).

[0046] If a positive result is obtained in step 311, the processor 101 determines the work theme and sets the confidence level to 1 (step 312). After step 312 is completed, processor 101 updates n to n+1 and proceeds to step 32. In contrast, if the title does not contain a specific keyword, the processor 101 obtains a negative result in step 311. In this case, the processor 101 determines whether or not there is a record of access to a specific URL (step 313). The specific URL is read from the rule-based determination keyword 122B of the business theme dictionary 122.

[0047] If a positive result is obtained in step 313, processor 101 determines the work theme and sets the confidence level to 1 (step 314). In this case as well, after step 314 is executed, the processor 101 updates n to n+1 and proceeds to step 32. In contrast, if there is no record of access to a URL registered in the business theme dictionary 122, the processor 101 obtains a negative result in step 313. In this case, the processor 101 determines whether or not there is a record of using a specific application (step 315). The specific application is read from the rule-based determination keyword 122B of the business theme dictionary 122.

[0048] If a positive result is obtained in step 315, processor 101 determines the work theme and sets the confidence level to 1 (step 316). In this case as well, after step 316 is executed, processor 101 updates n to n+1 and proceeds to step 32. In contrast, if there is no record of using an application registered in the business theme dictionary 122, the processor 101 obtains a negative result in step 315. In this case, processor 101 sets the confidence level to 0 (step 317) and proceeds to step 32.

[0049] The processor 101, having proceeded to step 32, determines whether the set confidence level is greater than the threshold α. If the business theme is determined in step 31, the confidence level is 1, and therefore a positive result is obtained in step 32. In this case, processor 101 proceeds to step 38. On the other hand, if the business theme is not determined in step 31, the confidence level is 0, and a negative result is obtained in step 32. In this case, processor 101 proceeds to step 33.

[0050] <Steps 33-34> Next, the processor 101 determines the business theme and confidence level based on the similarity of the keywords (step 33), and determines whether the confidence level is greater than the threshold α (step 34). Figure 9 is a flowchart illustrating an example of the process performed in step 33. First, processor 101 preprocesses the title of the active window to be processed (step 331). In doing so, processor 101 removes noisy strings from the title.

[0051] Next, processor 101 performs morphological analysis on the title and calculates the title vector (step 332). Next, the processor 101 calculates the similarity between the title vector and the vector in the business theme dictionary 122 (see Figure 2), and selects the business theme with the highest similarity (step 333). The similarity is calculated, for example, as cosine similarity. Subsequently, the processor 101 calculates a confidence score based on the selected similarity (step 334). The confidence score is given as a value between 0 and 1, inclusive. Alternatively, the selected similarity score may be used directly as the confidence score. After this, processor 101 proceeds to step 34.

[0052] The processor 101, having proceeded to step 34, determines whether the set confidence level is greater than the threshold α. As mentioned earlier, the business theme selected in step 33 is assigned a confidence level between 0 and 1. If a positive result is obtained in step 34, processor 101 proceeds to step 38. On the other hand, if a negative result is obtained in step 34, the processor 101 proceeds to step 35.

[0053] <Steps 35-36> Next, processor 101 uses the screenshot to determine the business theme and confidence level (step 35), and then determines whether the confidence level is greater than threshold α (step 36). Figure 10 is a flowchart illustrating an example of the process performed in step 35. First, processor 101 extracts text and attribute information from the screenshot of the active window to be processed (step 351). In doing so, processor 101 removes noisy strings from the text. Next, the processor 101 determines whether the title contains a specific keyword (step 352). The specific keyword is read from the rule-based determination keyword 122B (see Figure 4) of the business theme dictionary 122 (see Figure 2), similar to step 311 (see Figure 8).

[0054] If a positive result is obtained in step 352, processor 101 determines the work theme and sets the confidence level to 1 (step 353). After step 353 is completed, processor 101 proceeds to step 36. In contrast, if the title does not contain a specific keyword, the processor 101 obtains a negative result in step 352. In this case, processor 101 generates a screenshot vector (step 354). Specifically, processor 101 vectorizes the text extracted from the screenshot by weighting it based on attribute information. The generated vector is called the screenshot vector.

[0055] Next, the processor 101 calculates the similarity between the screenshot vectors and the vectors in the business theme dictionary 122 (see Figure 2), and selects the business theme with the highest similarity (step 355). Next, the processor 101 calculates a confidence score based on the selected similarity (step 356). The confidence score here is also given as a value between 0 and 1, inclusive. Note that the selected similarity may be used directly as the confidence score. After this, processor 101 proceeds to step 36.

[0056] Processor 101, having proceeded to step 36, determines whether the set confidence level is greater than the threshold α. As mentioned earlier, the business theme selected in step 35 is assigned a confidence level between 0 and 1. If a positive result is obtained in step 36, processor 101 proceeds to step 38. On the other hand, if a negative result is obtained in step 36, the processor 101 proceeds to step 37.

[0057] <Steps 37-38> If a negative result is obtained in step 36, the processor 101 performs the processing for when the estimation fails (step 37) and returns to step 2. Specifically, processor 101 assigns a label indicating a failed estimation to the estimated business theme. In Figure 6, two log IDs, "0002" and "00012," with a confidence level less than 0.2, are labeled with "-". On the other hand, if a positive result is obtained in any of steps 32, 34, or 36, the processor 101 outputs the estimated result (step 38) and returns to step 2. An example of the output here is shown in the second and third columns of the table in Figure 6.

[0058] <Step 5> Figure 11 is a flowchart illustrating an example of the process performed in Step 5. Incidentally, Step 5 is performed when a negative result is obtained in Step 4, that is, when there is an active window where the confidence level of the estimated business theme is less than the threshold β. Here, the threshold β is 0.6.

[0059] First, the processor 101 calculates the strength of the connections between active windows based on the transition information (step 51). In this embodiment, the strength of the connections is calculated for all active windows. Next, the processor 101 extracts active windows whose business themes are not yet estimated (step 52). After executing step 52, step 51 may be executed to calculate the connections between the extracted active windows and active windows that have transition relationships with them.

[0060] Next, processor 101 extracts all associated active windows for each active window for which a business theme has not been estimated, and determines weights according to the confidence level of the associated active window and the strength of the association with that active window (step 53). Next, the processor 101 calculates the weights for each business theme, determines the business theme with the largest total weight value to be the active window business theme, and sets the confidence level (step 54).

[0061] Figure 12 is a diagram illustrating the process in step 53. (A) shows a list of active windows related to active window Xj (j=1, 2…) whose business theme has not yet been estimated, and (B) shows the confidence level for each business theme estimated from the related active windows. In Figure 12(A), the six active windows A through F are related to the target active window Xj.

[0062] The business theme of Active Window A is "Business Theme A," and its confidence level is calculated to be 0.8. Furthermore, the strength of the connection between Active Window Xj and Active Window A is 10. Therefore, the weight is calculated to be 8 (= 0.8 × 10). In other words, the weight is calculated by multiplying the confidence level of the business theme in Active Window A (the transition partner) by the strength of the connection. The business theme for Active Window B is "Business Theme B," and its confidence level is calculated to be 0.7. Furthermore, the strength of the connection between Active Window Xj and Active Window B is 6. Therefore, the weight is calculated to be 4.2 (=0.7 × 6).

[0063] Weights are calculated similarly for other business themes, but for example, the business theme for active window F is "Business Theme B," and its confidence level is calculated as 0.8. However, the strength of the connection between active window Xj and active window F is 1. Therefore, the weight is calculated as 0.8 (=0.8 × 1).

[0064] On the other hand, the business theme of Active Window D is "Business Theme C," and its confidence level is 0.8. However, the strength of the connection between Active Window Xj and Active Window D is 3. Therefore, the weight is calculated to be 2.4 (=0.8 × 3). In other words, although the confidence levels of Active Window A, Active Window D, and Active Window F are all 0.8, the difference in the magnitude of their weights arises from the difference in the strength of their connection to Active Window Xj.

[0065] Figure 12(B) shows the sum of the weights calculated for each of the three business themes, along with the calculated confidence levels. For example, the total weight for business theme A is calculated as 12.0, which is calculated from Active Window A's 8.0, Active Window C's 2.8, and Active Window E's 1.2. Similarly, the total weight for business theme B is calculated as 5.0, which is the sum of 4.2 for active window B and 0.8 for active window F. The total weight for business theme C is 2.4 for active window D.

[0066] By the way, the sum of all the weights is 19.4 (=12.0 + 5.0 + 2.4). In this embodiment, the sum of all the weights is divided by the sum of the values ​​calculated for each business theme, and the business theme corresponding to the largest value is estimated to be the business theme of the active window Xj. In Figure 12(B), the business theme of the active window Xj is set to "Business Theme A". Its confidence level is calculated as 0.62 (=12.0 / 19.4).

[0067] <Step 6> Figure 13 is a flowchart illustrating an example of the process performed in step 6. Incidentally, step 6 is performed if a positive result is obtained in step 4, or after step 5 has been performed. Figure 13 shows the processing when the estimated business theme has a high degree of confidence. Whether a business theme has a high degree of confidence is determined by comparing the degree of confidence with a threshold. The threshold used to determine whether to update the business theme dictionary may be set independently of the other thresholds mentioned above. However, this does not prevent it from being the same value as the other thresholds.

[0068] If there are active windows that are determined to have a high degree of confidence for the estimated business theme, the processor 101 extracts the active windows with high confidence and groups them by estimated business theme (step 61). Next, processor 101 extracts words from the active window title or screenshot (step 62). Next, processor 101 extracts as feature words words that appear frequently in a specific business theme and infrequently in other business themes (step 63). Finally, processor 101 adds a feature word that is not in the business theme dictionary 122 (see Figure 2) (step 64).

[0069] Figure 14 is a flowchart illustrating another example of the process performed in Step 6. Figure 14 shows the process when the confidence level of the estimated business theme is low. Whether or not a business theme has a low confidence level is determined by comparing the confidence level with a threshold. The threshold here may be the same as or different from the threshold used to determine whether a business theme has a high confidence level. In any case, if there are active windows that are determined to have a low degree of confidence in the estimated business theme, the processor 101 extracts the active windows with low confidence and presents the estimated business theme to the user (step 71).

[0070] Next, the processor 101 determines whether the user has given the correct instruction (step 72). If a positive result is obtained in step 72, the processor 101 terminates the process. On the other hand, if a negative result is obtained in step 72, the processor 101 accepts a correction from the user (step 73). The correction here refers to a correction of the business theme. The processor 101 treats the business theme indicated by the user as the correct answer.

[0071] Next, processor 101 extracts the title of the active window to be processed and words from the screenshot as feature words (step 74). Subsequently, the processor 101 adds feature words from the extracted feature words that are not present in the business theme dictionary 122 (see Figure 2) (step 75). This addition increases the likelihood of more accurate business themes being estimated in subsequent estimations.

[0072] <Example of a screen presented to the user> Users sometimes want to search for the active window they used for their work or tasks by entering keywords. Additionally, as a function of processor 101 (see Figure 1), active windows related to the user's tasks and work may be displayed. The following describes an example screen used to display the active window.

[0073] <Example 1> Figure 15 illustrates an example of the display of active windows used in related tasks. (A) is a graph showing the relationship between transitions between active windows, (B) shows an example of the display of the active window corresponding to the node specified by the user with cursor K, and (C) shows an example of the display of information about the specified node. The screen 110A shown in Figure 15(A) displays a graph in which the active window is represented as a node and the transition relationship between active windows is represented as an edge.

[0074] Incidentally, in Figure 15(A), nodes with low confidence levels are displayed with higher brightness compared to other nodes. In Figure 15(A), nodes with low confidence levels are indicated by dashed lines. Displaying them with higher brightness attracts the user's attention. Therefore, the user can click on the corresponding node to check the contents of the active window and verify whether the estimated business theme is correct or incorrect. Furthermore, if the estimated business theme is incorrect, the error can be corrected with a simple operation.

[0075] Furthermore, in this graph, nodes estimated to belong to the same business theme are represented in the same form. For example, nodes estimated to belong to the same business theme are represented in the same color. In this case, a user viewing the graph shown in Figure 15(A) can identify the range of active windows belonging to the same business theme by the difference in color. The graph shown in Figure 15(A) shows that it represents a collection of nodes belonging to the same business theme, but it does not reveal the content of the business theme itself.

[0076] Therefore, in the example shown in Figure 15(B), the screenshot corresponding to the node at the position where cursor K is hovered over or clicked is displayed. In the case of screen 110B in Figure 15(B), two screenshots are displayed. In the graph shown in Figure 15(A), if the active window is the same, it is grouped into a single node. However, even if the active window is the same, the contents of the windows displayed behind it may differ. In Figure 15(B), it is possible to select any of the screenshots and check their contents.

[0077] Furthermore, in the graph display shown in Figure 15(A), hovering the cursor K over a specific node or clicking on a specific node may display the title of the active window on the screen. Additionally, the active window associated with the hovered or clicked node may be displayed as a pop-up, or information about other active windows referenced by that node's active window, or information about other active windows referencing that node, may be displayed. Furthermore, to represent the importance of a specified node, the color and size of the node on screen 110A may be changed according to the number of characters entered in the corresponding active window.

[0078] Figure 15(C) shows an example of how information is displayed for a node that cursor K is hovering over or clicking. In Figure 15(C), "Date and Time of Use," "Title," "Estimated Business Theme," "Usage Time," and "Number of Uses" are displayed. In Figure 15(C), the recorded usage dates and times are "2021-09-09 14:23", "2021-09-09 14:26", and "2021-09-09 16:54". These three usage dates and times correspond to three uses.

[0079] Additionally, the title recorded was "How to use the △△ Library," the estimated work theme was recorded as "OX technology development," and the usage time was recorded as 250 seconds. In this embodiment, clicking a specific node displays the corresponding screenshot as screen 110B, but it may also be displayed on the screen accessing content such as a document or website corresponding to the active window.

[0080] <Example 2> Figure 16 illustrates other examples of how the active window used in related tasks can be displayed. (A) shows the active window in timeline format, (B) shows an example of how the active window can be displayed corresponding to a node specified by the user with cursor K, and (C) shows an example of how information about the specified node can be displayed. In the screen 110C shown in Figure 16(A), the vertical axis represents the business theme, and the horizontal axis represents time. Figure 16(A) shows the time when the active window was displayed for each of the four business themes w1, w2, w3, and w4.

[0081] In Figure 16(A), time periods with low confidence are displayed with higher brightness compared to other nodes. Therefore, in Figure 16(A), time periods with low confidence are enclosed by dashed lines. By displaying time periods with low confidence levels in high brightness, the user's attention is drawn. This allows the user to click on the corresponding time period to check the contents of the active window and verify the accuracy of the estimated work theme. Furthermore, if the estimated work theme is incorrect, the error can be corrected with a simple operation.

[0082] Furthermore, in the timeline shown in Figure 16(A), each time period is represented by a density corresponding to the number of characters entered per minute. For example, the more characters entered per minute, the lighter the density, and the fewer characters entered, the darker the density. In this case, users looking at the graph in Figure 16(A) can check the time periods with a high and low number of characters entered per minute. In the case of the timeline shown in Figure 16(A), it is clear that an active window belonging to the same business theme was used, but the contents of the active window cannot be determined.

[0083] Therefore, in the example shown in Figure 16(B), the screenshot displayed corresponds to the time period when the cursor K was hovered over or clicked. In the case of screen 110D in Figure 16(B), one screenshot is displayed. By reviewing the displayed screenshots, it is possible to easily confirm the relevance of the task to the user's work.

[0084] In the case of the timeline shown in Figure 16(A), the title of the active window may be displayed on the screen when the cursor K is hovered over or clicked at a specific time period. Additionally, the active window associated with the hovered or clicked node may be displayed as a pop-up, or information about other active windows referenced by the active window during the same time period, or information about other active windows referencing the active window during the same time period may be displayed.

[0085] Figure 16(C) shows an example of displaying information for the time period when cursor K is hovered over or clicked. In Figure 16(C), "Date and Time of Use," "Title," "Estimated Business Theme," "Usage Time," and "Number of Uses" are displayed. In Figure 16(C), the recorded usage dates and times are "2021-09-09 14:23", "2021-09-09 14:26", and "2021-09-09 16:54". These three usage dates and times correspond to three uses.

[0086] Additionally, the title recorded was "How to use the △△ Library," the estimated work theme was recorded as "OX technology development," and the usage time was recorded as 250 seconds. In this embodiment, clicking on a specific time period displays the corresponding screenshot as screen 110D, but it may also be displayed on the screen accessing content such as a document or website corresponding to the active window. In this embodiment, the screen 110A described in Example 1 (see Figure 15) and the screen 110C described in Example 2 (see Figure 16) can be switched between by the user.

[0087] <Summary> In this embodiment, the relationship between the transitions of active windows used by the user and the strength of the connections between active windows are quantified and used to estimate the business themes of the active windows. As a result, the accuracy of the business theme estimation is improved. Furthermore, since the user is presented with quantified information on the relationships and strength of connections between transitions, tasks such as reviewing work, reusing viewed active windows, and searching for information become more efficient.

[0088] Furthermore, in this embodiment, a function is provided to draw the user's attention to active windows where the confidence level of the estimated business theme is low. This also streamlines the process of correcting the estimated business theme. Furthermore, by correcting the relationship between the active window and the business theme, the accuracy of estimating the business theme corresponding to the newly viewed active window will also improve. Furthermore, by correctly correcting the relationship between the active window and the work theme, it becomes easier to review and reuse the active window and other information using the presented graphs and timelines.

[0089] <Other Embodiments> (1) Although embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the embodiments described above. It is clear from the claims that embodiments with various modifications or improvements made to those described above are also included in the technical scope of the present invention.

[0090] (2) In the above-described embodiment, the user terminal 1 is provided with a function to estimate the business theme using the active window, using the transition relationship between active windows identified from the operation log and the strength of the connection between active windows that have a transition relationship. However, this function may also be provided on a server or other information processing device on the network. Furthermore, servers and other information processing devices may record not only the operation logs of a specific user, but also the operation logs of multiple users. In this case, the information processing device can calculate the transition relationships between active windows for multiple users, as well as the strength of the connections between active windows that have transition relationships. This functionality enables highly accurate sharing of knowledge and experience among multiple users.

[0091] (3) In the above-described embodiment, the user terminal 1 was shown as a desktop terminal, but it may also be a notebook terminal, a tablet terminal, a smartphone, or smart glasses. Smart glasses are a device that displays a virtual image in front of the user's line of sight.

[0092] (4) In the embodiments described above, examples were given of a process for determining the business theme and confidence level using a rule-based method (step 31), a process for determining the business theme and confidence level using keyword similarity (step 32), and a process for determining the business theme and confidence level using screenshots (step 33). However, the business theme and confidence level may be determined using only one or two of these methods. Alternatively, the business theme and confidence level may be estimated using other processes.

[0093] (5) The processor in each of the embodiments described above refers to a processor in a broad sense, and includes not only general-purpose processors (e.g., CPUs) but also specialized processors (e.g., GPUs (=Graphical Processing Units), ASICs (=Application Specific Integrated Circuits), FPGAs (=Field Programmable Gate Arrays), programmable logic devices, etc.). Furthermore, the processor operations in each of the embodiments described above may be performed by a single processor alone, or by multiple processors located in physically separate locations working together. Also, the order in which each operation is executed by the processor is not limited to the order described in each of the embodiments described above, but may be changed individually. [Explanation of Symbols]

[0094] 1...User terminal, 100...Main unit, 101...Processor, 104...Auxiliary storage device, 105...Communication device, 110...Display, 111...Business theme estimation unit, 112...Work estimation unit, 113...Task estimation unit, 114...Estimation result visualization unit, 120...Keyboard, 121...Operation log, 122...Business theme dictionary, 122A...Business theme, 122B...Keywords for rule-based judgment, 122C...Keywords for similarity judgment, 123...Work content dictionary, 130...Mouse

Claims

1. On the computer, A function that reads operation logs from a storage device and identifies transitions from an active window to an inactive window that appear in the read operation logs as relationships between window transitions, A function to identify the strength of the connection between windows that have the aforementioned transition relationship from the operation log, A function to display a graph on the screen in which the active window is represented as a node, the transition relationship from the active window to the inactive window is represented as an edge, and the strength of the connection between windows with such a transition relationship is represented by the display manner of the edges. A function to estimate the work of the active window using the strength of the connection between the active window corresponding to each node in the graph and the windows with the aforementioned transition relationship, A function to display the results of the estimation, A program to achieve this.

2. The strength of the connection between active windows is determined by recording the number of transitions between windows that have the transition relationship. The program according to claim 1.

3. In the estimation of the business, The active window's operation log allows us to estimate the active window's business theme. If the aforementioned business theme cannot be estimated, The business theme of the active window is estimated using the strength of the connections between windows that have the aforementioned transition relationship. The program according to claim 1.

4. Obtain the title and / or content of the active window from the operation log to estimate the business theme, The program according to claim 3.

5. The function that displays the results of the estimation is: The differences and similarities in the estimated tasks are represented in an identifiable way on the graph. The program according to claim 1.

6. The function that displays the results of the estimation is: The differences in estimated tasks are represented by differences in the display of the corresponding nodes. The program according to claim 5.

7. The function that displays the results of the estimation is: The range of nodes estimated to be related to the same task is represented by a graphic. The program according to claim 5.

8. The function for estimating the business is: The confidence level of the business is estimated using a value calculated based on rule-based methods, title similarity, keyword similarity, or the confidence level of the business calculated for active windows with transition relationships. The function that displays the results of the estimation is: The differences in the confidence levels of the estimated tasks are clearly represented on the graph. The program according to claim 1 or 5.

9. The function that displays the results of the estimation is: The difference in the confidence level of the estimated tasks is represented by the difference in the display appearance of the corresponding nodes. The program according to claim 8.

10. The function that displays the results of the estimation is: The difference in the confidence level of the estimated tasks is represented by a diagram. The program according to claim 8.

11. The system further has a function to modify the results according to the content of the modifications received when it accepts corrections to the estimated tasks. The program according to claim 9.

12. The function that displays the aforementioned graph on the screen is Active windows that have been displayed multiple times are represented by a single node, and the strength of the connection between active windows is represented by the length of the edges connecting each node. The program according to claim 1.

13. It has a processor, The aforementioned processor, The operation log is read from the storage device, and the transition from the active window to the inactive window that appears in the read operation log is identified as the transition relationship between windows. From the operation log, the strength of the connection between the windows that have the aforementioned transition relationship is identified. A graph is displayed on the screen in which the active window is represented as a node, the transition relationship from the active window to the inactive window is represented as an edge, and the strength of the connection between windows with such a transition relationship is represented by the display configuration of the edges. Using the strength of the connections between the active window corresponding to each node in the graph and the windows with the aforementioned transition relationship, the tasks of the active window are estimated. The results of the above estimation are shown below. Information processing device.

Images