Assistance device, operating system, display data creation device, assistance method, display data creation method, recording medium

Abstract

A display data creation apparatus (10) has: an acquisition section (11) that acquires operation log information (121) indicating a log of operations on a user interface (201), operation component configuration information (202) indicating a configuration of operation components displayed on the user interface (201) at the time of the operations; an extraction section (13) that extracts an operation pattern that repeatedly occurs in the operation log information (121); and a creation section (14) that creates display data for displaying the operation pattern extracted by the extraction section (13) together with the configuration of the operation components shown by the operation component configuration information (202).

Description

TECHNICAL FIELD

[0001] The present application relates to an assisting apparatus, an operation system, a display data creating apparatus, an assisting method, a display data creating method, a non-transitory computer-readable recording medium having an assisting program recorded thereon, and a non-transitory computer-readable recording medium having a display data creating program recorded thereon. BACKGROUND

[0002] In a field of a factory as a representative of FA (Factory Automation), a control system that realizes various processes by controlling a large number of devices by a control apparatus typified by a programmable controller is used. The processes realized by the control system are managed by a user in the field via a display as a UI (User Interface) terminal connected to the control apparatus, for example. As for the contents of the operation screen of such a display, it is not limited to directly using the contents set by the manufacturer of the display, but is appropriately designed by the user according to the needs of the field.

[0003] The design of the operation screen needs to consider the convenience of use for a plurality of operators including a person in charge of the design and a user other than the person in charge. Here, a technology for assisting the person in charge of the design by analyzing a history of operations for the operation screen is proposed (for example, refer to Patent Literature 1). In Patent Literature 1, an event analysis apparatus that analyzes event log data that indicates a history of operations by an operator and estimates the configuration of the operation sequence is described. According to the apparatus, support for the compilation and improvement of the operation guide can be provided.

[0004] Patent Literature 1: Japanese Patent Application Publication No. 2017-091178 SUMMARY

[0005] In the technology of Patent Literature 1, as a graph in which nodes that indicate unit operations are connected, the operation sequence is output to an output screen. However, in the display of such a graph, it is required for the person in charge of the design to have knowledge of what kind of operation is associated with the node. In particular, in a case where the number of objects that should be managed using the operation screen is large, it is required to have extensive knowledge related to the operation screen, and confirmation work is frequently generated. Therefore, the operation screen is a large burden on the person in charge of the design, and it can be difficult to design an appropriate operation screen.

[0006] The present application has been made in view of the above circumstances, and an object thereof is to make it easy to design an appropriate operation screen.

[0007] To achieve the above object, the display data creation apparatus of the present application has: an acquisition unit which acquires operation log information indicating a log of operations on a user interface, operation component configuration information indicating a configuration of operation components displayed on the user interface at the time of the operations; an extraction unit which extracts a plurality of operations in succession which occur repeatedly in the operation log information, that is, an operation pattern; and a creation unit which creates display data for displaying the operation pattern extracted by the extraction unit together with the configuration of the operation components indicated by the operation component configuration information.

[0008] Effects of the Invention

[0009] According to the present application, the creation unit creates display data for displaying the operation pattern together with the configuration of the operation components. If the operation pattern is displayed together with the configuration of the operation components by the display data, the user can easily recognize the operation components corresponding to the operation pattern. Thus, it becomes easy to design an appropriate operation screen. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 FIG. 1 is a diagram showing a functional structure of a display data creation apparatus according to Embodiment 1.

[0011] Figure 2 FIG. 2 is a diagram showing an example of a display of a user interface according to Embodiment 1.

[0012] Figure 3 FIG. 3 is a diagram showing an example of operation component configuration information according to Embodiment 1.

[0013] Figure 4 FIG. 4 is a diagram showing an example of component information according to Embodiment 1.

[0014] Figure 5 FIG. 5 is a diagram showing a hardware structure of a display data creation apparatus according to Embodiment 1.

[0015] Figure 6 FIG. 6 is a diagram showing an example of operation log information according to Embodiment 1.

[0016] Figure 7 FIG. 7 is a flowchart showing a display data creation process according to Embodiment 1.

[0017] Figure 8 FIG. 8 is a diagram for explaining segmentation from an operation log to an operation sequence according to Embodiment 1.

[0018] Figure 9 FIG. 9 is a diagram for explaining determination of a threshold according to Embodiment 1.

[0019] Figure 10is a diagram showing a calculation example of a distance between two operation sequences to which Embodiment 1 is related.

[0020] Figure 11 is a diagram showing one example of classifying an operation mode from operation sequences to which Embodiment 1 is related.

[0021] Figure 12 is a diagram showing one example of first operation mode information to which Embodiment 1 is related.

[0022] Figure 13 is a diagram showing one example of second operation mode information to which Embodiment 1 is related.

[0023] Figure 14 is a diagram showing a first example of a display screen based on display data to which Embodiment 1 is related.

[0024] Figure 15 is a diagram showing a second example of a display screen based on display data to which Embodiment 1 is related.

[0025] Figure 16 is a diagram showing a third example of a display screen based on display data to which Embodiment 1 is related.

[0026] Figure 17 is a diagram for explaining association information to which Embodiment 1 is related.

[0027] Figure 18 is a diagram showing a functional structure of a display data creation apparatus to which Embodiment 2 is related.

[0028] Figure 19 is a first diagram showing one example of a suggestion of a new arrangement of operation components to which Embodiment 2 is related.

[0029] Figure 20 is a diagram for explaining accumulation of distances between operation components to which Embodiment 2 is related.

[0030] Figure 21 is a second diagram showing one example of a suggestion of a new arrangement of operation components to which Embodiment 2 is related.

[0031] Figure 22 is a diagram showing a modification example of page transition to which Embodiment 2 is related.

[0032] Figure 23 is a third diagram showing one example of a suggestion of a new arrangement of operation components to which Embodiment 2 is related.

[0033] Figure 24 is a diagram showing an apparatus information storage apparatus to which a modification example is related.

[0034] Figure 25 is a view showing a device information acquisition section to which a modification pertains. DETAILED DESCRIPTION

[0035] Hereinafter, a display data creation device 10 to which an embodiment of the present application pertains will be described in detail with reference to the drawings.

[0036] Embodiment 1

[0037] The display data creation device 10 to which the present embodiment pertains is as shown in Figure 1 a display 20 corresponding to a UI terminal and an operated device 21 that acts in accordance with the operation content of the display 20 by a user, and constitutes an operation system 100. The operation system 100 is used for improving the operation screen of the display 20 in facilities typified by factories.

[0038] The operated device 21 is, for example, a PLC (Programmable Logic Controller), an I / O (Input / Output) unit used in connection with the PLC, a work machine, other FA equipment. The operated device 21 is connected to the display 20 via a communication line such as a USB (Universal Serial Bus) cable or an industrial network, and communicates with the display 20.

[0039] The display 20 is a computer having a user interface 201. The user interface 201 can be a touch panel constituted integrally with an input device typified by an LCD (Liquid Crystal Display), or can be a combination of a display device constituted separately, and a pointing device typified by a mouse and an input device such as a keyboard. The structure of the user interface 201 is not limited to this, as long as the display of information to a user and the input of information from the user can be realized.

[0040] The operation components are configured and displayed in the user interface 201 based on the preset operation component configuration information 202. The operation component refers to, for example, an image such as a button, a text input field, and a switch that becomes an element of an operation screen displayed on a screen. The operation component is operated by the user of the display 20. The operation of the operation component refers to, for example, selection of the operation component by input of the user represented by pressing and tapping, movement of the operation component by dragging, or input of a value or a character string to the operation component using a keyboard of hardware or software. For example, if the operation component to which the operation of starting the operation of the operated device 21 is assigned in advance is operated, the display 20 transmits an instruction of starting the operation to the operated device 21, and the operated device 21 operates in accordance with the instruction. The operation of the operation component is not limited to the above-described example, but is input of information associated with the operated device 21 to the displayed operation component by the user.

[0041] The operation screen of the user interface 201 is appropriately changed in accordance with the setting environment of the operated device 21 and the display 20. That is, the operation component configuration information 202 is set in accordance with the situation on the spot. For example, in a case where a new operated device 21 is set as an operation target, an operation screen for operating the new operated device 21 is designed.

[0042] An image or a window including the operation component displayed on the operation screen is hereinafter referred to as a page. The operation component included in the page and the jump of the page are specified by the operation component configuration information 202. In detail, as exemplified in Figure 2 In the page PG01 displayed on the main screen, the operation components PT01, PT02, PT03, PT04, and PT05 are configured. Further, if the operation component PT02 as a button is operated, the displayed page is jumped to the page PG02 as a device D1 operation screen, and the operation components PT11 and PT12 are configured on the page PG02. In addition, if the other operation components on the page PG01 are operated, the page PG03 is displayed, and if any of the operation components on the page PG02 is operated, the page PG04 is displayed.

[0043] The operation component configuration information 202 includes, as exemplified in Figure 3 the page information 203 related to the page and the component information 204 related to the operation component configured in the page. The page information 203 is a table form of data in which a page ID (Identifier) for identifying the page, a page name, a kind of the page, a position on the screen indicating a coordinate of the upper left end point of the page, and a size indicating the width and the height of the page are associated for each page.

[0044] The component information 204 is, as exemplified in Figure 4 Figure 4 In the component information 204, the behavior associated with the operation of the operation component is described as a jump to another page. However, information related to the jump to another page can be defined as information different from the component information 204. Further, components defined to have the same behavior can be arranged in different pages. The component arranged in the component information 204 can include components other than the operation component, such as an image displayed regardless of the user's operation.

[0045] The display 20 is connected to the display data creation apparatus 10 via a communication line or an industrial network, and communicates with the display data creation apparatus 10. Further, the operated apparatus 21 can be connected to the display data creation apparatus 10 as shown in Figure 1

[0046] The display data creation apparatus 10 is a computer that assists a designer who designs the arrangement of the operation components displayed on the operation screen, by providing the result of the analysis of the user's operation on the operation screen of the display 20. Figure 5 The hardware structure of the display data creation apparatus 10 is shown. The display data creation apparatus 10 has a processor 31, a main storage unit 32, an auxiliary storage unit 33, an input unit 34, an output unit 35, and a communication unit 36, as shown in Figure 5

[0047] The processor 31 includes a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The processor 31 functions as described later by executing a program PI stored in the auxiliary storage unit 33. The program PI corresponds to one example of a display data creation program.

[0048] ​​​The main storage section 32 includes a RAM (Random Access Memory). The program PI is loaded from the auxiliary storage section 33 to the main storage section 32. Also, the main storage section 32 is used as a work area of the processor 31.

[0049] The auxiliary storage section 33 includes a nonvolatile memory typified by an EEPROM (Electrically Erasable Programmable Read-Only Memory). In addition to the program PI, the auxiliary storage section 33 stores various data used for processing by the processor 31. The auxiliary storage section 33 supplies data used by the processor 31 to the processor 31 and stores data supplied from the processor 31, in accordance with an instruction from the processor 31.

[0050] The input section 34 includes an input device typified by an input key and a keyboard. The input section 34 acquires information input by a user of the display data creation device 10 and notifies the acquired information to the processor 31.

[0051] The output section 35 includes a display device typified by an LED (Light Emitting Diode) and an LCD (Liquid Crystal Display) and an output device such as a speaker. The output section 35 prompts various information to a user, in accordance with an instruction from the processor 31.

[0052] The communication section 36 includes a communication interface circuit for communicating with an external device. The communication section 36 receives a signal from the outside and notifies information shown by the signal to the processor 31. Also, the communication section 36 transmits a signal showing information output from the processor 31 to the external device.

[0053] The display data creation device 10 functions to display a result obtained from a log extraction operation mode for a log of an operation by a user with respect to the user interface 201 of the display 20, by cooperative action of the above-described hardware structure. In detail, the display data creation device 10 has, as its function, an acquisition section 11 that acquires operation log information 121 showing a log of an operation with respect to the user interface 201 and the operation component configuration information 202 stored in the storage section 12 from the display 20, a storage section 12 that stores various information, an extraction section 13 that extracts an operation mode repeatedly appearing in a log of an operation from the log of the operation, a creation section 14 that creates display data for displaying the operation mode together with a configuration of an operation component, and a display section 15 that displays the operation mode and the configuration of the operation component to a user of the display data creation device 10 based on the display data. Hereinafter, the user of the display data creation device 10 is appropriately referred to as a designer. Figure 1 The display data creation device 10 functions to display a result obtained from a log extraction operation mode for a log of an operation by a user with respect to the user interface 201 of the display 20, by cooperative action of the above-described hardware structure. In detail, the display data creation device 10 has, as its function, an acquisition section 11 that acquires operation log information 121 showing a log of an operation with respect to the user interface 201 and the operation component configuration information 202 stored in the storage section 12 from the display 20, a storage section 12 that stores various information, an extraction section 13 that extracts an operation mode repeatedly appearing in a log of an operation from the log of the operation, a creation section 14 that creates display data for displaying the operation mode together with a configuration of an operation component, and a display section 15 that displays the operation mode and the configuration of the operation component to a user of the display data creation device 10 based on the display data. Hereinafter, the user of the display data creation device 10 is appropriately referred to as a designer.

[0054] The acquisition section 11 is realized mainly by the communication section 36. The acquisition section 11 acquires and writes to the storage section 12 by requesting the display 20 to transmit the operation log information 121 and the operation component configuration information 202. The acquisition section 11 corresponds to one example of an acquisition unit that acquires operation log information indicating a log of operations on a user interface and operation component configuration information indicating a configuration of operation components displayed on the user interface at the time of the operations in the display data creation apparatus 10. The storage section 12 is realized mainly by the auxiliary storage section 33.

[0055] The operation log information 121 is data in a table form in which records are arranged in time series, as exemplified in Figure 6 The operation log information 121 is data in a table form in which records are arranged in time series, as exemplified in

[0056] The extraction section 13 is realized mainly by the processor 31. In the log of operations recorded over a sufficiently long period, a sequence of operations is repeatedly performed. For example, in a case where a first process of causing one of the operated devices 21 to operate after a plurality of parameters are set using the operation section and a second process of causing another of the operated devices 21 to operate after another parameter is set are alternately performed, a sequence of operations performed at the start of the first process and a sequence of operations performed at the start of the second process are alternately repeated. Further, even in different sequences of operations, the operated devices 21 are sometimes caused to perform the same operation. For example, in a case where the order of input of the parameters is different, in a case where the operator is different, and in a case where an unnecessary operation or an erroneous operation is performed due to inexperience of the operator, sequences of operations that are different from each other but similar to each other are repeatedly performed. The extraction section 13 classifies the sequences of operations that are repeatedly performed in the log of operations into groups and determines operation patterns that represent each group. Further, the extraction section 13 writes operation pattern information 122 that represents the operation patterns to the storage section 12. The extraction section 13 corresponds to an example of an extraction unit that extracts a plurality of the operations, i.e., operation patterns, that are repeatedly performed in the log of operations in the display data creation device 10.

[0057] The operation pattern is a series of operations on the user interface 201 and refers to operations that are frequently input. The operation pattern can be a series of operations that are the same or can be a series of operations that are similar to each other as described later.

[0058] Here, the operation refers to the minimum unit of input of information to the user interface 201. For example, in a case where input of certain information can be divided into a first operation and a second operation performed after the first operation, and in a case where a third operation different from the first operation and the second operation can be performed after the second operation, the input of the information corresponds to two or more operations. In a case where the second operation is necessarily performed after the first operation, the input of the information including the first operation and the second operation is treated as one operation. In a case where input of a string to an input field like a frame, the input of the string as a whole corresponds to one operation in a case where the string as a whole becomes an object of processing in the display 20 after the input of the string as a whole is completed, and the input of a single character corresponds to one operation in a case where processing based on the character is generated in the display 20 each time the single character is input.

[0059] The creating section 14 is realized mainly by the processor 31. The creating section 14 reads out the operation mode information 122 and the operation member configuration information 202 from the storage section 12, creates display data for displaying the configuration of the operation member corresponding to the operation included in the operation mode together with the operation mode to the designer, and outputs to the display section 15. Details of the display data will be described later. The creating section 14 corresponds to an example of a creating means for creating display data for displaying the operation mode extracted by the extracting means together with the configuration of the operation member indicated by the operation member configuration information in the display data creating apparatus 10.

[0060] The display section 15 is realized mainly by the output section 35. The display section 15 displays the operation mode and the configuration of the operation member to the designer based on the display data, thereby assisting the reconfiguration of the operation member by the designer. The display data can be data directly showing an image to be displayed, or data indicating the structure of the above image. In the case where the display data shows the structure of the image, the display section 15 generates and displays the image based on the display data.

[0061] Next, the use of the display data creating apparatus 10 will be described. Figures 7-16 The operation mode display processing performed by the display data creating apparatus 10 will be described. Figure 7 The display data creating processing illustrated in FIG. 8 is started by, for example, the activation of a specific application software of the display data creating apparatus 10, or a specific input by the designer. The display data creating processing corresponds to an example of a display data creating method performed by the display data creating apparatus 10. Further, the configuration of the display data creating apparatus 10 is an example, and can be arbitrarily changed. Figure 7 The execution order and the execution contents of the steps of the display data creating processing illustrated in FIG. 8 are examples, and can be arbitrarily changed.

[0062] In the display data creating processing, the acquiring section 11 acquires the operation log information 121 from the display 20 (step S1). The acquiring section 11 can acquire not one file as the operation log information 121, but data different according to each operator, each model manufactured by the operated apparatus 21, or each state of the apparatus indicated by the apparatus information, and finally acquire the operation log information 121 illustrated in FIG. 9. Figure 6 In the display data creating processing, the acquiring section 11 acquires the operation log information 121 from the display 20 (step S1). The acquiring section 11 can acquire not one file as the operation log information 121, but data different according to each operator, each model manufactured by the operated apparatus 21, or each state of the apparatus indicated by the apparatus information, and finally acquire the operation log information 121 illustrated in FIG. 9.

[0063] Next, the extraction unit 13 divides the operation log represented by the operation log information 121 into an operation sequence based on the time interval of the operation (step S3). An operation sequence is a portion obtained by dividing the operation log into several parts, equivalent to a sequence of operations. The input required by the user is not limited to being completed through the operation of a single operating component. For example, when an error occurs in the operated device 21 or when the user confirms information on the operation screen, the user ends the input to the user interface 201 by sequentially operating multiple operating components. Each time an operating component is operated, a record is added to the operation log information 121; therefore, a single input of information by the user typically corresponds to a series of consecutive operations, i.e., an operation sequence.

[0064] Here, as Figure 8 As shown, when the records of operation log information 121 are arranged according to their corresponding times, the time length from the start of one operation to the next operation in the operation sequence is short, while the time length from the end of one operation sequence to the start of the next operation sequence is relatively long. Therefore, the extraction unit 13 determines that these operations belong to the same operation sequence if the time length from the start of one operation to the next operation is less than a threshold, and determines that these operations belong to different operation sequences if the time length exceeds the threshold. For example, if there are records for 13:00:00, 1 second, 2 seconds, 13 seconds, and 15 seconds, with a threshold of 10 seconds, the operation log is divided into operation sequences containing operations performed at 0 seconds, 1 second, and 2 seconds, and the next operation sequence containing operations performed at 13 seconds and 15 seconds.

[0065] The threshold used to obtain the operation sequence can also be preset to a fixed value. However, for the display 20 where the operation screen is designed on-site, it is difficult to determine an appropriate threshold in advance, and a fixed value may be inappropriate as the threshold for segmenting the operation sequence. Furthermore, since the operation screen is designed on-site, a method for segmenting the operation sequence based on a predefined structure of the operation screen cannot be used.

[0066] Here, the extraction unit 13 determines the threshold by analyzing past records. Specifically, the extraction unit 13 sequentially calculates the time length from the start of each past operation to the next operation, as shown in the operation log information 121. Figure 9 The process involves creating a histogram of occurrence frequencies for that time period. Then, the extraction unit 13 determines a threshold based on the created histogram. Specifically, the threshold is determined by approximating the histogram as a probability distribution represented by a normal or log-normal distribution. Alternatively, the extraction unit 13 can apply a Gaussian mixture model to the histogram and calculate the threshold through optimal estimation. Furthermore, as... Figure 9As shown, the value of the standard deviation σ at the time of approximation of the histogram to a normal distribution can also be used as the threshold value. As described above, if the threshold value is determined by statistical analysis of past records, an appropriate threshold value is expected to be determined for the display 20 having an arbitrary operation screen set in various environments.

[0067] In addition, it is sometimes preferable to use different threshold values in a page in which the amount of information displayed is large and the operation takes time because confirmation work is required, and in a page in which the operation ends in a short time. Also, the time taken for the operation differs between an operation member in which a button is pressed to end the operation and an operation member in which input of a character string is required, and thus it is preferable to use different threshold values. Therefore, the extraction unit 13 can also determine the threshold value by extracting records in the operation log information 121 for each page or each operation member, analyzing the time interval of the operations before and after the records, and taking into account the case where the operation time differs depending on the page or the operation member. For example, the threshold value used for the length of time until the operation of an operation member that is a button is shorter than the threshold value used for the length of time until the operation of an operation member that is an input field of a character string.

[0068] The records that become the target of the analysis of the length of time between the records by the extraction unit 13 can be all of the operation logs shown by the operation log information 121, or can be a part of the operation logs shown by the operation log information 121. For example, the extraction unit 13 can also extract records of operations related to a specific page from the operation log information 121, and analyze the length of time between the records. In addition, the extraction unit 13 can also extract records of operations including jumps between pages, such as a jump from a specific page to an arbitrary other page X, and a jump from the page X to the specific page, and analyze the length of time between the records. The extraction unit 13 can also extract records of operations related to jumps between a specific two pages, such as a jump from the page PG01 to the page PG02, and a jump from the page PG02 to the page PG01, and analyze the length of time between the records. Although an example of extraction of records related to a page is described, the length of time can also be analyzed after the records are extracted for an operation member.

[0069] In addition, the operation time can differ depending on the operator. For example, it is considered that the time interval of the operation by an unskilled operator is longer than the time interval of the operation by a skilled operator. Here, the extraction unit 13 can also determine a more appropriate threshold value corresponding to the operator by extracting and analyzing past records in the operation log information 121 for each operator.

[0070] Further, the extraction unit 13 can extract the record for each state of the operated device 21 and determine the threshold value corresponding to the state. The determination of the threshold value associated with the page, the operation member, the operator, and the state of the operated device 21 described above can be arbitrarily combined.

[0071] Further, in Figure 7 The operation sequence split from the operation log in step S3 of the present embodiment can also include an operation sequence that is not an appropriate operation sequence. For example, since the time interval from the last operation in a sequence of 1 operation to the first operation of the next sequence of operations is short, it is possible that a sequence of 2 operations is split into a sequence of 1 operation. Further, since the user is not concentrating on the operation but is thinking, the operation takes time, and it is possible that a sequence of 1 operation is split into a sequence of 2 operations. Further, since the operation error or the operation order differs depending on each user, it is possible that a plurality of operation sequences that are the same kind of operation sequence but have several different orders of operations occur.

[0072] Here, as Figure 7 indicated, the extraction unit 13 classifies the operation sequences obtained in step S3 into groups and extracts an operation pattern corresponding to the group (step S4). The same or similar operation sequences belong to each group. The operation pattern is a representative operation sequence of the group and can be any one of the operation sequences actually belonging to the group or an operation sequence newly created by the extraction unit 13 as an operation sequence representing the characteristics of the operation sequences belonging to the group.

[0073] Whether the operation sequences are similar to each other is determined, for example, based on the distance between the operation sequences calculated by DP (Dynamic Programming) matching. Figure 10 An example of the calculation of the distance between two operation sequences is shown. Here, the more similar the operation sequences are, the smaller the distance between the operation sequences is, and thus the distance between the operation sequences corresponds to the degree of similarity indicating how similar the operation sequences are to each other.

[0074] Figure 10 The operation sequence SQ1 is a sequence from the first operation named "SQ1-1" to the ninth operation named "SQ1-9", and the operation sequence SQ2 is a sequence from the first operation named "SQ2-1" to the eleventh operation named "SQ2-11". Each operation corresponds to a combination of a page on which an operation member is displayed and the operation member operated. For example, the operation named "SQ1-1" means that the operation member PT01 is operated on the page PG01. Further, the combination of the page, the operation member, and the value input in association with the operation can also be handled as one operation.

[0075] As Figure 10As shown, a table is created by arranging the operations that constitute two operation sequences along the row and column directions. The distance between the operation sequences is calculated by evaluating the values ​​within each mass of this table. First, if the operation "SQ1-1" shown directly above the top-left mass is identical to the operation "SQ2-1" shown to the left of that mass, the value of that mass is zero. If these operations are different, the value of that mass is a non-discrepancy penalty of 3. Figure 10 In the example, because the operation is consistent, the value of the top left block is zero.

[0076] Next, the extraction unit 13 calculates the values ​​of the blocks adjacent to the right, the blocks adjacent to the bottom, and the blocks adjacent to the bottom right of the block whose value has been calculated. Specifically, if the comparison of the blocks determined by the column of operation "SQ1-2" and the row of operation "SQ2-1" is consistent, the block to be calculated is moved from left to right. Therefore, a rightward movement penalty of 1 is added to the value of the left block (zero), and the value of the block determined by "SQ1-2" and "SQ2-1" is set to 1. If the operations are inconsistent, an inconsistency penalty of 3 is added. For the blocks determined by "SQ1-1" and "SQ2-2", a downward movement penalty is also added to calculate the value.

[0077] For the block determined by "SQ1-2" and "SQ2-2", the direction with the smallest value among the three directions of movement: from top, from the upper left, and from the left. Figure 10 In the example, the movement penalty is minimal (zero) when moving from the top left, the operation is consistent, and no inconsistency penalty is added, so the value of this block is zero. However, an inconsistency penalty is added when the operations differ. For example, for the block defined by columns "SQ1-3" and rows "SQ2-3", the movement penalty is minimal (zero) when moving from the top left. The value is calculated as 3 by adding the zero movement penalty and the 3 inconsistency penalty to the value of the top left block (zero). Subsequently, the values ​​of blocks are calculated similarly from approximately the top left to approximately the bottom right. The value of the bottom right block in this way represents the distance between the operation sequences. Furthermore, the inconsistency penalty can also be zero when both the page and the operation component are consistent, 1 when either is different, and 2 when both are different.

[0078] Extraction unit 13 is for use in Figure 3 Calculate the two combinations of the operation sequence obtained in step S3. Figure 10 The distance between the illustrated operation sequences. Furthermore, the extraction unit 13 uses hierarchical clustering to merge operation sequences that are close to each other into the same group, thereby obtaining...Figure 11 The tree structure shown classifies operation sequences similar to each other into groups. The extraction unit 13 retains operation sequences that are somewhat numerous among the classified operation sequences as operation patterns, and excludes operation sequences that are few in number from the subsequent processing targets. For example, the extraction unit 13 can discard groups whose number of operation sequences is less than 10, or can set the first 50% of groups whose number of operation sequences is large as the subsequent processing targets.

[0079] Further, the method of classifying similar operation sequences is not limited to the example described above, and any method can be employed. For example, the extraction unit 13 can replace each operation with a numerical value, group vectors of operation sequences that have numerical values of operations as components by hierarchical clustering, or can group by a method such as k-means, decision tree, or random forest.

[0080] As described above, the extraction unit 13 divides the operation log into operation sequences using the threshold value decided based on past records, but in such division, sometimes operation sequences that should be classified into different operation patterns are divided into one operation sequence, and sometimes operation sequences that should be classified into one operation pattern are divided into multiple operation sequences. Then, the extraction unit 13 detects the occurrence of operation patterns from the operation log to the operation sequences extracted in step S4 of the operation pattern extraction processing (step S5). In detail, the extraction unit 13 sequentially performs comparison of operation sequences appearing in the operation log with operation patterns respectively. The comparison can be calculation of the distance shown above, or calculation of other similarity. Figure 7 Figure 10 Further, the extraction unit 13 can detect the occurrence of operation sequences similar to operation patterns based on the threshold value. At this time, the processing by the extraction unit 13 can be processing of searching for operations consistent with the first operation of the operation pattern from the operation log, increasing the records from there to, for example, twice the length of the operation pattern, and detecting the occurrence of operation sequences similar to the operation pattern in the middle where the similarity becomes less than or equal to the threshold value. Step S5 can be said to be division of the operation log into operation sequences based on the operation patterns extracted in step S4 instead of the time interval in step S3.

[0081] Further, the extraction unit 13 can omit steps S3 and S4 and perform step S5 using the operation patterns extracted in the past at the start time point of the display data creation processing. Figure 7

[0082] ​​Next, the extraction section 13 updates the operation pattern by further classifying the operation pattern detected in step S5 into a new operation sequence (step S6). Thus, the classification of the operation sequence is updated, and a more appropriate operation pattern is expected. Further, in consideration of the calculation load due to the reclassification of the operation pattern, step S6 can be omitted.

[0083] The operation pattern extracted in step S6, or the operation pattern extracted in step S4 in the case where step S6 is omitted, is associated with the time on the operation log in step S5, and stored in the storage section 12 as operation pattern information 122.

[0084] The operation pattern information 122 includes Figure 12 the first operation pattern information 122a shown in Figure 13 the second operation pattern information 122b shown in. The first operation pattern information 122a is a list of the categories of the extracted operation patterns, and is table-form data in which a pattern ID for identifying each operation pattern, a sub-group ID for identifying a sub-group equivalent to the same operation sequence as the group to which the operation pattern corresponds, one operation representing the sub-group, a start page displayed at the start of the operation sequence of the sub-group, a first and a second and subsequent operations constituting the operation sequence are associated. Here, the first and the second and subsequent operations are each represented by a combination of a page ID and a component ID.

[0085] The second operation pattern information 122b, as shown in Figure 13 , is data showing the operation pattern appearing on the operation log in association with the time of appearance. In detail, the second operation pattern information 122b is table-form data in which the start time and the end time of the appearance of the operation pattern, a sub-group ID representing the operation sequence of the appearance, a model of the product manufactured by the operated device 21, and an operator are associated. For example, the operation sequence of the sub-group ID "SUB01-01-01" appearing from 13:00:00 to 7 seconds on September 28, 2020, if referring to Figure 12 , it is known that the operation sequence belongs to the operation pattern of the pattern ID "PTN01-02". In the detection of the operation pattern from the operation log in step S5 described above, in the case where a similar pattern is detected rather than a completely identical pattern, the distance or the degree of similarity is also recorded in the second operation pattern information 122b in Figure 13 . Also, the second operation pattern information 122b can further show the first and the second and subsequent operations constituting the operation sequence of the appearance.

[0086] Returning to Figure 7After the step S6, the creation section 14 creates display data for displaying the operation mode together with the configuration of the operation section (step S7), and the display section 15 displays the configuration of the operation section and the operation mode based on the created display data (step S8). Then, the display data creation processing ends.

[0087] Figure 14 One example of a display screen of the display data creation apparatus 10 is shown. In the display screen 401 of the display data creation apparatus 10, Figure 14 after the designation of the period of the operation log, the designation of the model, the designation of the operator, and the selection of the state of the operated apparatus 21 as the operation mode extraction condition, the designer presses the display button 402. Thereby, the operation log satisfying the designated condition is extracted from the operation log information 121, and the operation mode is extracted from the extracted operation log. Further, the display data creation apparatus 10 can call the operation mode satisfying the designated condition from the created operation mode information 122.

[0088] If the display button 402 is pressed, the extraction result of the operation mode is displayed in the lower part of the display screen 401. The extraction result of the operation mode is shown in a table, and the number of occurrences and the average time taken for the input of the operation mode are displayed for each mode ID. Thereby, the designer can easily judge the operation mode with high frequency and the operation mode taking time for the input, and research the structure of the operation screen based on the judgment. Further, the table can show the sub-group ID of the operation sequence appearing instead of the mode ID, or the sub-group ID can be shown together with the mode ID. In addition, the representative operation of the operation mode can be further shown in the table, and the line corresponding to the operation mode can be sorted in the order of the number of occurrences from many to few or the order of the average time from long to short.

[0089] In addition, for the operation mode of a predetermined number of operation modes with high frequency of occurrence shown in the table, the number of occurrences per one hour can be counted, and the progress of the number of occurrences can be displayed by a line graph. In addition, the model manufactured by the operated apparatus 21 or the operator can be displayed together with the line graph Figure 1 and displayed like a Gantt chart.

[0090] If the button 403 of "Detail" included in any line of the table configuring Figure 14 the display screen 401 is pressed, information indicating the detail of the operation mode is displayed in the display screen 501 as shown in Figure 15 . The display screen 501 shows the detailed content of the operation mode, and the operation sequence of the sub-group included in the operation mode is displayed at a glance. In the display screen 501, Figure 15In the example of FIG. 9, the operation mode includes three kinds of operation sequences, and the display data 120 shows the number of times of appearance in each of the operation logs of the three kinds of operation sequences and the average time of operation cost, and the sequence of operations constituting the operation sequences. Thus, the designer can easily judge the operation sequences with high frequency and the operation sequences that take time to input, and study the structure of the operation screen based on the judgment. In addition, the designer can easily find abnormal operation sequences that can include operation errors by comparing the typical operation sequences as the operation mode with abnormal operation sequences different from the typical operation sequences.

[0091] If the designer presses the display switching button 502, the sequence of operations shown by the page ID and the component ID is replaced with the sequence of the reduced images identical to the operation screen displayed in the user interface 201 of the display 20, as shown in FIG. 10. In each of the reduced images, the operation component that becomes the operation target is emphasized and shown. In the example of FIG. 10, the operation component is emphasized and shown by being blackened. Thus, the designer can recognize the operation screen of the actual user interface 201 and easily recognize in which configuration of the operation components the abnormal operation sequence is generated. In addition, the designer can easily confirm the operations shown by the page ID and the component ID and the operation screen at the time of the operations by pressing the display switching button 502. In order to make the above-mentioned association easy, it is preferable to display the page ID and the component ID in the same position as the reduced images of the operation screen. Figure 16 Figure 16 In the example of FIG. 10, the operation component is emphasized and shown by being blackened. Thus, the designer can recognize the operation screen of the actual user interface 201 and easily recognize in which configuration of the operation components the abnormal operation sequence is generated. In addition, the designer can easily confirm the operations shown by the page ID and the component ID and the operation screen at the time of the operations by pressing the display switching button 502. In order to make the above-mentioned association easy, it is preferable to display the page ID and the component ID in the same position as the reduced images of the operation screen.

[0092] As described above, the creation section 14 creates display data for displaying the operation mode together with the configuration of the operation components. If the operation mode is displayed together with the configuration of the operation components by the display data, the designer can easily recognize the operation components corresponding to the operation mode. Thus, the design of the appropriate operation screen can be easily performed.

[0093] In addition, the operation log information 121 shows the operation components used in the operations, and the display data is data for emphasizing and displaying the operation components used in the operations constituting the operation mode. Thus, in the image that reproduces the structure of the operation screen, the designer can visually recognize the execution order of the operations constituting the operation mode. By comparing the operation mode with the configuration of the operation components, the designer can study the configuration of the operation components that is easy to operate for the user, and easily study the configuration of the operation components. In addition, the designer can easily obtain a figure that can be used for the operation guide of the operation screen.

[0094] ​Further, the operation log information 121 shows the operations performed sequentially in association with the time of performing the operations, and the extraction section 13 extracts the operation patterns by classifying the operation sequences obtained by dividing the logs of the operations shown in the operation log information 121 into operation sequences based on the length of time from the time of each operation to the time of the next operation. If the operation screen is fixed in advance, it is easy to divide the operation logs into operation sequences, but in the case where the operation member configuration information 202 of the display 20 is changed according to the environment of the site, it is difficult to obtain appropriate operation sequences. In this regard, according to the extraction section 13, since the operation sequences are obtained based on the time interval between the operations, it is possible to obtain appropriate operation sequences to some extent with respect to various operation screens in various sites.

[0095] Further, the extraction section 13 extracts the operation patterns by classifying the operation sequences, and detects the occurrence of the extracted operation patterns in the logs of the operations, and the display data is data for displaying the detection result of the extraction section 13. Thus, even if the time interval between the operations does not correspond to the operation pattern, it is possible to detect the appropriate operation pattern.

[0096] Further, as shown in Figure 14 the operator is specified as a condition for the logs of the operations to be extracted as the operation patterns. Thus, it is possible to analyze the operation patterns and the average time taken for the operations for each operator, and to estimate the familiarity with the operations. For example, with respect to the operation patterns in which the time difference is large depending on the operator, the ease of the operations of the experienced person and the novice person can be different, and it is conceivable to create an operation guide for the operation that takes time, thereby achieving improvement such as shortening the operation time.

[0097] Further, the state of the operated device 21 in which an error is included is specified as a condition. Thus, it is possible to determine the operation pattern input to the user interface 201 when the error occurs, and to create an operation guide based on the determined operation pattern. Specifically, it is possible to assist the designer who performs the work of grasping when the recovery procedure is not easy to know in which error, and studying whether or not it is possible to achieve improvement by, for example, creating an operation guide or a shortcut function.

[0098] Further, as shown in Figure 17As shown in the graph, objects 601 representing associated information related to the operated device 21 can also be displayed on a timeline. Associated information includes, for example, memos entered into a smartphone by an operator performing the operation on the operated device 21, or images captured by the operated device 21. Specifically, associated information is text information such as "September 28, 2020, 13:15:20, Device D1, 'XX occurred near Device D1, image.jpg'," and image information named "image.jpg". If the acquisition unit 11 acquires the associated information as described above, and the creation unit 14 creates display data containing the associated information, the associated information can be displayed in association with the timeline on the graph. Figure 17 In the example, hovering, clicking, or tapping the mouse on object 601 displays details of associated information and the contents of the image file. In this way, by collaborating with external associated information, the designer can be prompted to correlate the number of operations with the events occurring at that moment.

[0099] When displaying associated information, the acquisition unit 11 is an example of an acquisition unit that acquires associated information related to the user interface in association with time. Additionally, the display data is an example of data used to display the occurrence of operation patterns in the operation log, associated information, and time.

[0100] Implementation Method 2

[0101] Next, regarding Embodiment 2, the description will focus on the differences from Embodiment 1 described above. Furthermore, the same reference numerals will be used for structures that are the same as or equivalent to those in Embodiment 1, and their descriptions will be omitted or simplified. The difference between this embodiment and Embodiment 1 is that the display data creation device 10 has a suggestion unit 16 that suggests new configurations for the operating components.

[0102] like Figure 18 As shown, the display data creation device 10 includes a suggestion unit 16 that suggests new configurations for operating components based on operation log information 121 and operation mode information 122. The suggestion unit 16 is primarily implemented by the processor 31. The content of the suggestions made by the suggestion unit 16 is, for example, expressed in... Figure 15 The suggestion display button (not shown) is pressed on the display screen 501 showing the details. Suggestions from the suggestion unit 16 may include, for example, the creation of shortcut buttons or the reordering of operation components. Furthermore, the suggestion content can be customized by the designer, and with the designer's permission, the suggestion unit 16 can also generate new operation component configuration information 202 using the suggestions and set it on the display 20. This allows for easy improvement of the configuration of operation components displayed on the user interface 201.

[0103] Next, specific examples of the suggestion content suggested by the suggestion section 16 are described.

[0104] Figure 19 An example of suggesting the creation of a shortcut button is shown. The shortcut button is an operation member for collectively inputting a plurality of operations. For example, the suggestion section 16 determines an operation pattern including continuous operations greater than or equal to a threshold value in the operation pattern information 122, calculates the number of operations and the average time taken for the operations, and selects an operation pattern for which the calculation result is greater than or equal to a predetermined threshold value, or an operation pattern for which the calculation result is greater than that of another operation pattern. For example, an operation pattern satisfying the conditions of being greater than or equal to N times in 1 hour, being greater than or equal to M times in 1 day, the average time taken for the operations being greater than or equal to S seconds, the number of operations per 1 hour being greater than or equal to P times compared to another operation pattern, or the number of operations per 1 hour being greater than or equal to Q% compared to another operation pattern is selected. Furthermore, a suggestion of adding a new shortcut button corresponding to the collective input of the selected operation pattern is made.

[0105] The operations collected by the shortcut button are not limited to sequential operations of operation members displayed in a single page, and can include operations of operation members in a page after a jump. In addition, the suggestion section 16 can use the operation member arrangement information 202. For example, the suggestion section 16 can suggest the creation of a shortcut button in a case where the continuous operations are sequential operations of operation members arranged in a queue. In addition, the suggestion section 16 can calculate the distance between operation members, and suggest the creation of a shortcut button in a case where the position of an operation member on the screen deviates from a threshold value.

[0106] Figure 20 The order of operation members subjected to operations constituting an operation pattern is shown. In the order shown in the upper part of Figure 20 In the order shown in the upper part of

[0107] Specifically, the suggestion section 16 determines an operation mode including continuous operations greater than or equal to a threshold value from the operation mode information 122, and selects an operation mode in which the number of operations or the length of time of operations is large from the determined operation modes. Also, the suggestion section 16 calculates the coordinates of the central positions of the respective operation components based on the operation component configuration information 202. For example, if the position of the upper left end point of an operation component is (100, 100) and the size is (200, 50), the coordinates of the central position of the operation component are (200, 125).

[0108] The suggestion section 16 obtains the cumulative value of the distances in the current operation mode by calculating the coordinates of the central positions of the respective operation components corresponding to the operations constituting the selected operation mode, and sequentially adding the distances up to the operation component next to perform the operation. Also, the suggestion section 16 attempts to change the configuration of the operation components, and similarly obtains the cumulative value of the distances in the case where the configuration is changed. If the difference in the distances before and after the change in the configuration of the operation components is greater than or equal to a predetermined threshold value, the suggestion section 16 suggests the configuration of the operation components after the change as shown in Figure 21 .

[0109] In addition, as shown in Figure 22 , in the case where the page temporarily jumps to another page in the operation mode and then returns to the original page, it becomes easy for the user to perform the same operation in the same page. Therefore, the suggestion section 16 determines an operation mode in which the operation in the same page is interrupted only by an operation in another page during the operation in the same page from the operation mode information 122. In the case where the number of occurrences of the operation mode is greater than a threshold value or the operation time taken is greater than a certain time, the suggestion section 16 suggests the duplication or movement of the operation component to another page as shown in Figure 23 . Here, the suggestion section 16 can also suggest the new configuration of the duplicated or moved operation component based on the operation component configuration information 202.

[0110] As described above, if the suggestion by the suggestion section 16 is made, the research of the new configuration of the operation components by the designer can be assisted. The suggestion section 16 corresponds to an example of a suggestion unit that suggests the new configuration of the operation components in the display data creation apparatus 10.

[0111] Further, the suggestion section 16 can suggest the new configuration of the operation components based on the operation log information 121 without using the operation mode information 122. For example, the suggestion section 16 can suggest the deletion of an operation component that is not used even once in the operation log information 121. In addition, the suggestion section 16 can compare the logs of the operations before and after the application of the new configuration of the operation components, and prompt the result of the evaluation of the new configuration based on the comparison to the user. For example, it is assumed that the operation component is moved to another page, and the operation time is reduced.Figure 20 In the case of the above-described configuration, if errors in the operated device 21 frequently occur, it can be said that the above-described replacement of the configuration is not preferable. Therefore, the suggestion section 16 can also suggest restoration of the configuration of the operation section.

[0112] The above describes an embodiment of the present application, but the present application is not limited to the above-described embodiment.

[0113] For example, although the example in which the display data creation device 10 has the display section 15 is described, it is not limited thereto. For example, it can be that an operation mode extraction device as a cloud server transmits display data to an external display device. The operation mode extraction device corresponds to a device having a transmission section that transmits display data to a display device instead of the display section 15 in the above-described display data creation device 10. In addition, the operation mode extraction device can output display data to a display device via a connected cable without via a network.

[0114] In addition, although the example in which the display data creation device 10 and the display 20 are configured as separate devices is described, the display data creation device 10 can be integrally configured with the display 20.

[0115] In addition, although the example in which the acquisition section 11 directly acquires information from the display 20 is described, the acquisition section 11 can acquire by reading out information saved in an external device from the display 20, or can acquire by reading out information from a recording medium such as a memory card that is plugged by a user.

[0116] In addition, the display 20 can be a UI terminal that is temporarily connected to the operated device 21 as needed, or can be always connected to the operated device 21. The display 20 can have a UI function that uses a computing resource of the operated device 21, or can be a UI section that is integrally configured with the operated device 21.

[0117] In addition, the manner in which the operated device 21 provides device information to the display data creation device 10 is arbitrary. For example, as illustrated in Figure 24 In addition, as illustrated in Figure 25 In addition, as illustrated in

[0118] In addition, although an example in which the time interval between operations is focused on in order to separate the operation sequence from the operation log is described, the present application is not limited thereto. For example, the extraction unit 13 can also cut the operation sequence at the timing at which the state of the operated device 21 is changed, and can cut the operation screen of the user interface 201 from the home page to the home page as one operation sequence.

[0119] In addition, although an example in which the operation of the user interface 201 is performed using the operation member is described, an operation without using the operation member is also conceivable. For example, in the case where the previous menu is returned if a region of a button and a window that is not configured is tapped or clicked, the operation member such as the button and the window is not used. However, with respect to such an operation, it is sufficient to regard that information is input to the operation member corresponding to the entire background.

[0120] In addition, the function of the display data creation device 10 can be realized by a dedicated hardware or a general computer system.

[0121] For example, by distributing the program P1 executed by the processor 31 to a nonvolatile recording medium that can be read by a computer, installing the program P1 to the computer, it is possible to configure a device that performs the above-described processing. As such a recording medium, a floppy disk, a CD-ROM (Compact Disc Read-Only Memory), a DVD, an MO (Magneto-Optical Disc), and the like are conceivable.

[0122] In addition, the program P1 can be stored in a storage device of a server device on a communication network typified by the Internet, and downloaded to a computer, for example, by being superimposed on a carrier wave.

[0123] In addition, by executing the start while forwarding the program P1 via a communication network, it is also possible to realize the above-described processing.

[0124] Furthermore, by causing all or a part of the program P1 to be executed on a server device, and executing the program while transmitting and receiving information related to the processing by a computer via a communication network, it is also possible to realize the above-described processing.

[0125] In addition, in the case where the above-described function is realized by an OS (Operating System) or by the cooperative operation of the OS and an application, only a part other than the OS can be distributed to a medium, and the computer can be downloaded.

[0126] In addition, the means for realizing the function of the display data creation device 10 is not limited to software, and a part or all thereof can be realized by a dedicated hardware including a circuit.

[0127] The present application can be variously embodied and modified without departing from the broad spirit and scope of the present application. In addition, the above-described embodiments are merely illustrative of the present application and do not limit the scope of the present application. That is, the scope of the present application is not represented by the embodiments, but by the claims. Also, various modifications made within the scope of the claims and the meaning equivalent thereto are to be considered to fall within the scope of the present application.

[0128] Industrial applicability

[0129] The present application is suitable for improvement of work of a user interface.

[0130] Explanation of reference numerals

[0131] 100 operation system, 10 display data creation device, 11 acquisition section, 12 storage section, 121 operation log information, 122 operation mode information, 122a first operation mode information, 122b second operation mode information, 13 extraction section, 14 creation section, 15 display section, 16 suggestion section, 20 display, 201 user interface, 202 operation component configuration information, 203 page information, 204 component information, 21 operated device, 22 device information storage device, 23 device information acquisition section, 31 processor, 32 main storage section, 33 auxiliary storage section, 34 input section, 35 output section, 36 communication section, 37 internal bus, 401, 501 display screen, 402 display button, 403 button, 502 display switch button, 601 object, P1 program.

Claims

1. An assistance device having: an acquisition unit that acquires operation log information indicating a log of operations on a user interface, operation component configuration information indicating a configuration of operation components displayed on the user interface at the time of the operations; an extraction unit that extracts a plurality of operations, i.e., an operation pattern, that occur consecutively and repeatedly in the operation log information; and a display unit that displays the operation pattern extracted by the extraction unit together with the configuration of the operation components shown by the operation component configuration information, the operation log information showing the operation components used in the operations, the display unit emphasizing one of the operation components used in the operations that constitute the operation pattern and displaying a sequence of images showing operation screens.

2. The assistance device according to claim 1, wherein the operation log information shows the operations performed sequentially in association with the time of the operations, the extraction unit classifies operation sequences based on the length of time from the time of each operation until the time of the next operation to extract the operation pattern.

3. The assistance device according to claim 2, wherein the extraction unit extracts the operation pattern by classifying the operation sequences and detects the occurrence of the extracted operation pattern in the operation log information without being based on the length of time, the display unit displays the result of the detection by the extraction unit.

4. The assistance device according to claim 2, wherein the operation pattern corresponds to a group of the operation sequences, the extraction unit extracts the operation pattern by classifying the operation sequences and updates the operation pattern by further classifying the operation pattern that occurs in the operation log information into a new operation sequence.

5. An operation system including: the assistance device according to any one of claims 1 to 4; an operated device that operates in accordance with the operations on the user interface; and the user interface.

6. A display data creation device having: an acquisition unit that acquires operation log information indicating a log of operations on a user interface, operation component configuration information indicating a configuration of operation components displayed on the user interface at the time of the operations; an extraction unit that extracts a plurality of operations, i.e., an operation pattern, that occur consecutively and repeatedly in the operation log information; and a creation unit that creates display data for displaying the operation pattern extracted by the extraction unit together with the configuration of the operation components shown by the operation component configuration information, the acquisition unit acquiring association information associated with the user interface in a manner corresponding to the time, the operation log information showing the operations performed sequentially in association with the time of the operations, the display data being data for displaying the occurrence of the operation pattern in the operation log information and the association information in a manner corresponding to the time. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 7. A display data creation apparatus having: an acquisition unit that acquires operation log information indicating a log of operations on a user interface, and operation component configuration information indicating a configuration of operation components displayed on the user interface at the time of the operations; an extraction unit that extracts a plurality of operations, i.e., an operation pattern, that repeatedly occur in succession in the operation log information; a creation unit that creates display data for displaying the operation pattern extracted by the extraction unit together with the configuration of the operation components indicated by the operation component configuration information; and a suggestion unit that suggests a new configuration of the operation components based on at least one of the operation log information and the operation pattern.

8. The display data creation apparatus according to claim 7, wherein the suggestion made by the suggestion unit includes at least one of adding a new operation component corresponding to an input of the operation pattern, changing the configuration of a plurality of the operation components based on a total of distances on the user interface from the operation component corresponding to each of the operations constituting the operation pattern to the operation component corresponding to a next operation, and copying or moving the operation component to another page in a case where a page displayed with the operation component included therein jumps.

9. The display data creation apparatus according to claim 7, wherein the suggestion unit outputs a result obtained by evaluating the new configuration based on a comparison of the operation log information before and after application of the new configuration.

10. An operating system including: the display data creation apparatus according to any one of claims 6 to 9; an operated apparatus that operates in accordance with the operations on the user interface; and the user interface.

11. An assistance method including the following processes: an acquisition unit acquires operation log information indicating a log of operations, the acquisition unit acquires operation component configuration information indicating a configuration of operation components displayed on a user interface at the time of the operations, an extraction unit extracts a plurality of operations, i.e., an operation pattern, that repeatedly occur in succession in the operation log information, a display unit displays the operation pattern extracted by the extraction unit together with the configuration of the operation components indicated by the operation component configuration information, the operation log information indicates the operation components used in the operations, the display unit emphasizes one of the operation components used in the operations constituting the operation pattern, and displays a sequence of images showing operation screens.

12. A display data creation method including the following processes: an acquisition unit acquires operation log information indicating a log of operations, the acquisition unit acquires operation component configuration information indicating a configuration of operation components displayed on a user interface at the time of the operations, an extraction unit extracts a plurality of operations, i.e., an operation pattern, that repeatedly occur in succession in the operation log information, a display unit displays the operation pattern extracted by the extraction unit together with the configuration of the operation components indicated by the operation component configuration information, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The creating unit creates display data for displaying the operation pattern extracted by the extracting unit together with the configuration of the operation component shown by the operation component configuration information, The acquiring unit acquires the associated information associated with the user interface in a manner corresponding to the time, The operation log information shows the operations performed in sequence in association with the time at which the operations were performed, The display data is data for displaying the occurrence of the operation pattern in the operation log information and the associated information in a manner corresponding to the time.

13. A display data creating method including the following processes: An acquiring unit acquires operation log information indicating a log of operations, The acquiring unit acquires operation component configuration information indicating a configuration of an operation component displayed on a user interface at the time of performing the operation, An extracting unit extracts an operation pattern that is a plurality of operations that occur consecutively in the operation log information, A creating unit creates display data for displaying the operation pattern extracted by the extracting unit together with the configuration of the operation component shown by the operation component configuration information, A suggesting unit suggests a new configuration of the operation component based on at least one of the operation log information and the operation pattern.

14. A recording medium that is a non-transitory computer-readable recording medium that records an assistance program for causing a computer to function as: An acquiring unit that acquires operation log information indicating a log of operations and operation component configuration information indicating a configuration of an operation component displayed on a user interface at the time of performing the operation, An extracting unit that extracts an operation pattern that is a plurality of operations that occur consecutively in the operation log information, and A display unit that displays the operation pattern extracted by the extracting unit together with the configuration of the operation component shown by the operation component configuration information, The operation log information shows the operation component used in the operation, The display unit emphasizes one operation component used in the operation that constitutes the operation pattern and displays a sequence of images showing operation screens.

15. A recording medium that is a non-transitory computer-readable recording medium that records a display data creating program for causing a computer to function as: An acquiring unit that acquires operation log information indicating a log of operations and operation component configuration information indicating a configuration of an operation component displayed on a user interface at the time of performing the operation, An extracting unit that extracts an operation pattern that is a plurality of operations that occur consecutively in the operation log information, and A creating unit that creates display data for displaying the operation pattern extracted by the extracting unit together with the configuration of the operation component shown by the operation component configuration information, The acquiring unit acquires associated information associated with the user interface in a manner corresponding to the time, The operation log information shows the operations performed in sequence in association with the time at which the operations were performed, The display data is data for displaying the occurrence of the operation mode in the operation log information and the association information in correspondence with a time.

16. A non-transitory computer-readable recording medium recording a display data creation program for causing a computer to function as: an acquisition unit that acquires operation log information indicating a log of an operation, and operation component configuration information indicating a configuration of an operation component displayed on a user interface at the time of the operation; an extraction unit that extracts a plurality of consecutive operations, that is, an operation mode, repeatedly occurring in the operation log information; a creation unit that creates display data for displaying the operation mode extracted by the extraction unit together with the configuration of the operation component indicated by the operation component configuration information; and a suggestion unit that suggests a new configuration of the operation component based on at least one of the operation log information and the operation mode.

Images