Information processing apparatus, information processing method, program, and display system

Abstract

To display a display component related to an expected operation content with respect to a state change of an information processor.SOLUTION: According to an aspect of the present disclosure, an information processing apparatus 20 includes a history managing unit 22 that manages an operation history of an operation received from a user to the information processing apparatus and a state change history of the information processing apparatus, a history analyzing unit 23 that analyzes the operation history and the state change history and predicts an operation content corresponding to a state of the information processing apparatus, and a screen display control unit 24 that causes a display unit to display a display component related to the operation content predicted by the history analyzing unit.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and a display system. [Background technology]

[0002] When a user uses an information processing device such as an image processing device, the information processing device may display a screen that serves as a user interface. A technique is known in which the configuration of the screen displayed by the information processing device is changed according to the user's operation history (see, for example, Patent Document 1). Because display components such as buttons that the user frequently uses are displayed, the operational burden when operating the information processing device can be reduced and ease of use can be improved.

[0003] Patent Document 1 discloses a technology that manages the operation history of image processing functions when a user executes a job, counts the number of operations performed on processing function units such as printing and their combinations from the operation history, predicts image processing operations by prioritizing them based on the count values, and displays candidate functions, allowing the user to easily indicate image processing functions that they use frequently. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology disclosed in Patent Document 1 merely predicts the operation content to be performed by the user based only on the number of operations, and does not take into consideration predicting the operation content to be performed by the user including changes in the state of the information processing device. Therefore, when the user performs an appropriate operation in response to a change in the state of the information processing device, it is difficult to display a display component related to the predicted operation content.

[0005] In view of the above-described problems, the present invention provides a technique for displaying display components relating to expected operation details in response to a change in the state of an information processing device. [Means for solving the problem]

[0006] In view of the above problems, the present invention is an information processing device comprising: a history management unit that manages an operation history of operations received from a user on the information processing device and a status change history of the information processing device; a history analysis unit that analyzes the operation history and the status change history to predict operation content according to the status of the information processing device; and a screen display control unit that displays display components related to the operation content predicted by the history analysis unit on a display means. [Effects of the Invention]

[0007] In response to a change in the state of the information processing device, a display component relating to the expected operation content can be displayed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic perspective view of an example of an image processing device. [Figure 2] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image processing apparatus. [Figure 3] FIG. 1 is a diagram illustrating an example of a functional configuration of an image processing apparatus. [Figure 4] 10 is a sequence diagram illustrating an example of a process in which an image processing apparatus saves a user's operation history and a status change history of the image processing apparatus. [Figure 5] 10A and 10B are diagrams illustrating an example of an operation history and a state change history stored in a history information storage unit. [Figure 6] 10 is a sequence diagram illustrating an example of a process in which the image processing apparatus displays a predictive menu on an initial screen of an application. [Figure 7] FIG. 2 is a diagram illustrating an example of a home screen displayed on a display unit. [Figure 8] FIG. 10 is a diagram illustrating an example of an initial screen of an application on which a prediction menu is displayed. [Figure 9] FIG. 10 is a diagram illustrating an example of an operation reception screen that is displayed when a user presses a display component. [Figure 10] FIG. 10 is a flowchart illustrating an example of a process for generating a predicted menu by an image processing apparatus. [Figure 11] 10A and 10B are diagrams illustrating an example of an operation history and a state change history in which only the user name of a logged-in user is recorded. [Figure 12] FIG. 10 is a diagram illustrating an example of an occurrence rate. [Figure 13] FIG. 10 is a diagram showing an example of the number of operations listed for each operation content. [Figure 14] FIG. 10 is a diagram illustrating an example of a menu setting screen. [Figure 15] 10 is a sequence diagram illustrating an example of a process in which the image processing apparatus displays a favorites menu on an initial screen of an application. [Figure 16] FIG. 10 is a diagram illustrating an example of an initial screen of an application on which a favorites menu is displayed. [Figure 17] FIG. 1 is a diagram illustrating an example of a system configuration of a display system. [Figure 18] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server device. [Figure 19] FIG. 2 is an example of a functional block diagram illustrating functions of the display system in blocks. [Figure 20] 10 is an example of a sequence diagram illustrating a process in which a server device saves a user's operation history and status change history. [Figure 21] 10 is a sequence diagram illustrating an example of a process in which a server device determines a predicted menu and an image processing device displays an initial screen of an application. DETAILED DESCRIPTION OF THE INVENTION

[0009] An information processing apparatus and an information processing method provided by the information processing apparatus will be described below as an example of an embodiment of the present invention. In this embodiment, an image processing apparatus will be described as an example of the information processing apparatus.

[0010] The image processing device of this embodiment predicts the functions of the image processing device to be used by the user based on not only the operation history but also the state changes of the image processing device, thereby displaying a predictive menu that accepts operations that are frequently performed in the user's usage situation. The functions of the image processing device to be used by the user are, from the user's perspective, display components (such as buttons) for performing operations.

[0011] Specifically, the image processing device analyzes whether there is an operation that is likely to be performed after the state change of the image processing device, and if there is, raises the priority of the function that will be performed after the state change of the current image processing device. In other words, the image processing device creates a predicted menu by raising the priority of the function that will be performed after the state change of the current image processing device over the function predicted based only on the number of operations in the operation history.

[0012] The predictive menu is displayed on the first screen (hereinafter referred to as the initial screen) that appears after the application (basic functions of the image processing device, such as fax, scanner, copy, and printer) is launched. Display components are also displayed within the predictive menu, and when a display component is selected, the screen transitions directly to a screen that accepts execution of a function with a higher priority. This allows the user to execute the functions of the image processing device that they actually want to use with fewer operation steps. This makes it easier to display the functions that the user wants to use, improving operability.

[0013] <Terminology> A status change is a change in the status of an image processing device. The status change may also identify the current status of the image processing device.

[0014] A display component is a screen configuration that accepts operations from a user to use a function of the image processing device. In this embodiment, a button (for example, a button in the prediction menu 215 in FIG. 8) is used as an example of a display component, but the display component may also be an input field, operation guide, etc.

[0015] The display component related to the operation content may be any display component that supports the reception of the operation content. For example, it may be a display component that allows the user to perform the operation with fewer steps than if the display component were not present. The display component related to the operation content may be, for example, a button for transitioning to a screen that receives the operation content, but it may also be the screen that receives the operation content itself.

[0016] [First embodiment] The system configuration will be described with reference to FIG. 1. FIG. 1 shows a schematic perspective view of an image processing device 20 according to this embodiment. The image processing device 20 is an example of various information processing devices used by a user. In this embodiment, the image processing device 20 is described as an information processing device, but the image processing device 20 is merely an example of an information processing device in which display components are changed according to the user's operation history and status change history. Changing display components according to the user is sometimes called "personalization." Note that changing according to the user refers to optimizing for ease of use by the user, but it does not necessarily have to be optimal; it is sufficient if the user finds it easy to use.

[0017] The image processing device 20 is, for example, an MFP (Multifunction Peripheral). The image processing device 20 may also be called a printing device, an image forming device, a scanner device, a fax machine, a copier, etc. The image processing device 20 may also be a PC (Personal Computer), a projector, an electronic whiteboard, a digital camera, etc. The image processing device 20 may also be an AV (Audio Visual) device, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, a medical device, a network home appliance, a mobile phone, a smartphone, a tablet terminal, a game console, a wearable PC such as a VR (Virtual Reality) device, etc.

[0018] 1, the image processing device 20 is shown as a standalone device, but as will be described later, the image processing device 20 may be configured to communicate with a server device via a network. In this case, the server device can execute the processing related to personalization that the image processing device 20 executes. Furthermore, even if the server device does not execute the processing related to personalization, the operation history and status change history of one image processing device 20 can be shared with another image processing device 20.

[0019] <Hardware configuration example> The hardware configuration of the image processing device 20 will be described with reference to Fig. 2. As shown in Fig. 2, the image processing device 20 includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network I / F 950.

[0020] Of these, the controller 910 has a CPU 901, which is the main part of the computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, a local memory (MEM-C) 907, which is a storage unit, an HDD controller 908, and an HD 909, which is also a storage unit, and is configured such that the NB 903 and the ASIC 906 are connected by an AGP (Accelerated Graphics Port) bus 921.

[0021] Of these, the CPU 901 is a control unit that performs overall control of the image processing device 20. The NB 903 is a bridge that connects the CPU 901 with the MEM-P 902, the SB 904, and the AGP bus 921, and includes a memory controller that controls reading and writing to the MEM-P 902, a PCI (Peripheral Component Interconnect) master, and an AGP target.

[0022] The MEM-P 902 comprises a ROM 902a, which is memory for storing programs and data that realize the functions of the controller 910, and a RAM 902b, which is used for expanding the programs and data and as a drawing memory during memory printing. The programs stored in the RAM 902b may be provided by being recorded in an installable or executable file format on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0023] The SB 904 is a bridge for connecting the NB 903 with PCI devices and peripheral devices. The ASIC 906 is an integrated circuit (IC) for image processing applications that has hardware elements for image processing and serves as a bridge connecting the AGP bus 921, PCI bus 922, HDD controller 908, and MEM-C 907. The ASIC 906 includes a PCI target and AGP master, an arbiter (ARB) that forms the core of the ASIC 906, a memory controller that controls the MEM-C 907, multiple direct memory access controllers (DMACs) that perform image data rotation using hardware logic, and a PCI unit that transfers data between the scanner unit 931, printer unit 932, and facsimile unit via the PCI bus 922. A Universal Serial Bus (USB) interface or an Institute of Electrical and Electronics Engineers 1394 (IEEE 1394) interface may be connected to the ASIC 906.

[0024] The MEM-C907 is a local memory used as an image buffer for copying and a code buffer. The HD909 is a storage for storing image data, font data used during printing, and forms. The HD909 controls the reading and writing of data from and to the HD909 under the control of the CPU901. The AGP bus 921 is a bus interface for a graphics accelerator card proposed to speed up graphics processing, and direct high-throughput access to the MEM-P902 enables the graphics accelerator card to operate at high speed.

[0025] Further, the short-range communication circuit 920 is provided with a short-range communication circuit antenna 920a. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth (registered trademark).

[0026] The engine control unit 930 further includes a scanner unit 931, a printer unit 932, and a facsimile unit 933. The operation panel 940 includes a panel display unit 940a, such as a touch panel, that displays current setting values, selection screens, etc., and receives input from the operator. The operation panel 940 includes hard keys 940b, such as a numeric keypad that receives setting values ​​for image formation conditions, such as density setting conditions, and a start key that receives a copy start command. The controller 910 controls the entire image forming apparatus, and controls, for example, drawing, communication, and input from the operation panel 940. The scanner unit 931 or the printer unit 932 includes an image processing unit that performs error diffusion, gamma conversion, etc.

[0027] The image processing device 20 can sequentially switch among the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 940. The image forming device enters document box mode when the document box function is selected, copy mode when the copy function is selected, printer mode when the printer function is selected, and facsimile mode when the facsimile mode is selected.

[0028] The network I / F 950 is an interface for performing data communication using the communication network N2. The short-range communication circuit 920 and the network I / F 950 are electrically connected to the ASIC 906 via a PCI bus 922.

[0029] <About the function> Next, the functional configuration of the image processing device 20 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the functional configuration of the image processing device 20 according to this embodiment.

[0030] The image processing device 20 has an operation unit 21, a history management unit 22, a history analysis unit 23, a screen display control unit 24, a device state change detection unit 25, and a history information storage unit 29. Each of these functional units of the image processing device 20 is a function or means realized by the CPU 901 shown in Fig. 2 executing instructions contained in one or more programs installed in the image processing device 20. Not all of these functional units of the image processing device 20 are essential, and some functions may be omitted as long as the processing performed in this embodiment can be achieved.

[0031] The history information storage unit 29 is formed in the HD 909 or RAM 902b shown in Fig. 2. The history information storage unit 29 may be present on a network that the image processing device 20 can access.

[0032] The operation unit 21 is realized by an operation panel 940 or the like. The operation unit 21 has a display unit 26. The display unit 26 (an example of a display means) displays various screens created by the screen display control unit 24. The operation unit 21 can also be configured by a touch panel or the like, and accepts operations on the screens displayed by the display unit 26.

[0033] The history management unit 22 acquires the details of operations performed by the user from the operation unit 21, associates them with the user, and stores them as an operation history. The history management unit 22 also stores a state change history (described later in FIG. 5) of the image processing device 20 detected by the device state change detection unit 25.

[0034] The history analysis unit 23 analyzes the operation history and the state change history to predict the operation content according to the state of the image processing device 20. More specifically, the history analysis unit 23 predicts the operation content that the user will perform next based on the operation content that was performed in the past immediately after a state change that occurred in the image processing device 20. When making the prediction, the history analysis unit 23 counts the number of operations for each operation content and sets a priority order for the operation content.

[0035] The device state change detection unit 25 monitors the state of the image processing device 20. When the state of the image processing device 20 changes, the device state change detection unit 25 notifies the history management unit 22. The history management unit 22 stores the state change history.

[0036] The screen display control unit 24 creates a predicted menu to be displayed by the operation unit 21 based on the priorities set for the operation contents by the history analysis unit 23. The predicted menu includes display components related to the operation contents predicted to be performed by the user. The display components related to the operation contents are buttons and the like for transitioning to a screen that accepts the operation contents. The screen display control unit 24 notifies the operation unit 21 of the predicted menu and causes it to be displayed.

[0037] The history information storage unit 29 stores the operation history and status change history of each user, details of which will be explained with reference to FIG.

[0038] <Saving operation history> The process of saving the operation history and the status change history will be described with reference to Fig. 4. Fig. 4 is a sequence diagram illustrating the process of the image processing device 20 saving the user's operation history and the status change history of the image processing device 20.

[0039] S1: The user inputs authentication information into the operation unit 21 and logs in. The image processing device 20 performs an existing authentication process based on the authentication information and determines whether the authentication is successful or unsuccessful. Authentication may be performed by an external device. If authentication is successful, the user name is identified. The authentication information may be the user's identification information and password, an IC card, or biometric information.

[0040] The user operates any application such as a fax application, a scanner application, a copy application, or a printer application. The operation uses the functions of the image processing device 20, and is therefore an image processing function such as printing, faxing, copying, or scanning. The operation unit 21 accepts the user's operation, and the operation content is identified.

[0041] S2: The operation unit 21 requests image processing according to the operation content from the controller 910. The controller 910 executes the requested image processing.

[0042] S3: The history management unit 22 acquires the requested image processing. The history management unit 22 saves the operation history shown in FIG. 5. The operation history includes, for example, a timestamp, a user name, and operation details. Note that the operation details are either the same as the requested image processing, or can at least be identified from the image processing.

[0043] S4: The device status change detection unit 25, which monitors the status of the image processing device 20, detects any change in the status of the image processing device 20 and notifies the history management unit 22 of the details of the status change. The details of the status change are the current status of the image processing device 20. In FIG. 4, the status change of the image processing device 20 occurs after image processing, but the status change of the image processing device 20 is not limited to those caused by user operations. There are many status changes of the image processing device 20 that occur regardless of user operations.

[0044] S5: The history management unit 22 stores the status change history as shown in Fig. 5. The status change history includes a timestamp, a status change name, and the like.

[0045] <Operation history and status change history> The operation history and status change history will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the operation history and status change history stored in the history information storage unit 29. In Fig. 5, the number, timestamp, user name, and operation content constitute the operation history, and the number, timestamp, and device status changes constitute the status change history.

[0046] The number is identification information attached to the operation history and status change history, and is a consecutive number in Figure 5. The number does not have to be consecutive as long as it can be uniquely identified.

[0047] The timestamp is the date and time when the user operated the image processing device 20 (date and time when the operation was accepted) in the case of operation history, and is the date and time when a state change occurred in the image processing device 20 in the case of status change history.

[0048] The user name is identification information of the user who operates (logs in to) the image processing device 20.

[0049] The operation content is information specifying the operation performed by the user on the operation unit 21. The operation content can also be said to be the function of the image processing device 20 used by the user (for example, an image processing function). The operation content varies depending on the functions possessed by the image processing device 20, but in FIG. 5, "print," "stored document print," "received document settings," "Bcc send," "display transmission result," "communication result report," etc. are saved. The operation content may be represented by identification information or a code that identifies it.

[0050] The device status change indicates the current status of the image processing device 20 due to a change in status. In FIG. 5, the statuses stored are "Scanning original", "Stored document", "Memory capacity limit", "Direct transmission in progress", etc. In other words, "Scanning original" indicates a change from a state where an original is not being read to a state where it is being read. "Stored document" indicates a change from a state where there are no stored documents to a state where a new document has been stored. "Memory capacity limit" indicates a change from a state where there is sufficient memory capacity to a state where there is not sufficient memory capacity. "Direct transmission in progress" indicates a change from a state where a fax is not being sent to a state where a fax is being sent.

[0051] In this way, the operation content and the state change of the device are recorded in association with the timestamp, so the operation history and the state change of the device are arranged in chronological order.

[0052] <Prediction menu display process> Next, a process for displaying a predicted menu will be described with reference to Fig. 6. Fig. 6 is a sequence diagram illustrating a process in which the image processing device 20 displays a predicted menu on the initial screen of an application.

[0053] S11: First, it is assumed that the image processing device 20, in which the user has logged in, displays the home screen 200 on the display unit 26. An example of the home screen 200 is shown in FIG. 7. The user presses an application displayed on the home screen 200 to start the application. The application may be a fax application, a scanner application, a copy application, a printer application, or the like. The operation unit 21 accepts the application being pressed.

[0054] S12: The operation unit 21 designates the pressed application and requests the controller 910 to start the application. The controller 910 starts the designated application.

[0055] S13: Having acquired the request for the startup process, the history analysis unit 23 requests the history management unit 22 to acquire the operation history and the state change history. This is to display a predicted menu on the initial screen of the application.

[0056] S14: In response to a request, the history management unit 22 provides one week's worth of operation history and state change history to the history analysis unit 23. One week's worth is just an example, and ten days' worth, one month's worth, the entire past period, etc. may also be provided.

[0057] S15: The history analysis unit 23 analyzes the operation history and the state change history. Specific details of the analysis will be described in FIG. 10. The history analysis unit 23 determines, through the analysis, the priority of the operation contents (an example of the first operation contents) associated with the display components to be displayed in the prediction menu. Therefore, there may be a plurality of operation contents for which priorities are set, but there may also be only one operation content. Details of the processing of step S15 will be described in FIG. 10.

[0058] S16: The history analysis unit 23 notifies the screen display control unit 24 of the priority of the operation contents.

[0059] S17: The screen display control unit 24 creates a prediction menu including display components related to the operation contents arranged in order of priority of the operation contents. When the display component related to the operation contents is selected, the screen transitions to a screen that accepts the operation contents.

[0060] S18: The screen display control unit 24 provides the operation unit 21 with a predicted menu.

[0061] S19: The display unit 26 displays the initial screen of the application reflecting the predicted menu. An example of the initial screen of the application is shown in FIG.

[0062] 6, the request for the startup process is acquired by the history analysis unit 23, but the request for the startup process may be acquired by the history management unit 22 instead of the history analysis unit 23. In other words, the history management unit 22 acquires the request for the startup process from the operation unit 21 and manages the operation history. The history management unit 22 provides the operation history and the state change history to the history analysis unit 23. The history analysis unit 23 analyzes the operation history and the state change history and notifies the screen display control unit 24 of the analysis result (priority of the operation content).

[0063] Fig. 7 is an example of a home screen 200 displayed by the display unit 26. The home screen 200 displays application buttons 201 for launching one or more applications that are installed in the image processing device 20 and for which the user has a license. Note that the types of applications shown in Fig. 7 are merely examples and may vary depending on the user. The user presses a button or icon corresponding to the application (function) that the user wishes to execute from among the application buttons 201 displayed on the home screen 200.

[0064] Fig. 8 shows an initial screen 210 of an application on which a prediction menu 215 is displayed. Fig. 8 shows a fax application as an example of the application. Since it is a fax application, the initial screen 210 has a destination field 211, a destination list 212, a keypad 213, and a start button 214. The destination field 211 displays the fax destination (user name or fax number). The destination list 212 displays a list of frequently used destination user names or destination user names registered by the user. The keypad 213 is used when the user inputs a fax number (when accepting a fax number). The start button 214 is a button for accepting a request from the user to start fax transmission (for the user to start fax transmission).

[0065] The initial screen 210 further includes a prediction menu 215. In the prediction menu 215, display components 215a to 215e relating to operation details determined based on the operation history and the state change history are displayed in order of priority. As an example, the display components 215a to 215e are buttons that are pressed by the user. The user can select the display component 215a to 215e relating to the operation details that the user wishes to use from the prediction menu 215. In this way, displaying the prediction menu 215 reduces the operational burden and improves ease of use.

[0066] Additionally, the predicted menu 215 displays a past period 217 that was used to create the predicted menu. In Fig. 8, the period 217 is one week. The period 217 allows the user to understand that the display components 215a to 215e are displayed based on the changes in the device status and operation history that occurred during that period.

[0067] 9 shows the operation acceptance screen 220 that is displayed when the user presses one of the display components 215a to 215e. The operation acceptance screen 220 is a screen for accepting from the user an operation corresponding to the display component 215a to 215e selected by the user. In other words, the operation acceptance screen 220 is a screen for accepting the execution of an image processing function to be used by the user. In this way, a direct transition (shortcut) from the initial screen 210 to the operation acceptance screen 220 that executes an image processing function with a high priority can be made, so that the image processing function that the user actually wants to use can be executed in fewer operation steps.

[0068] <Details on creating a forecast menu> Next, a method for creating the predicted menu 215 will be described with reference to Fig. 10. Fig. 10 is a flowchart illustrating the process of creating a predicted menu by the image processing device 20. The process in Fig. 10 is executed in step S15 in Fig. 6.

[0069] First, the history analysis unit 23 leaves only the logged-in user name and deletes information about other user names from the operation history and status change history acquired from the history management unit 22 (S21). The operation history and status change history with only the logged-in user name remaining are shown in Fig. 11.

[0070] Next, for each type of "device status change", the history analysis unit 23 counts the number of operations performed immediately after the "device status change" for each operation content, and calculates the occurrence rate (S22).

[0071] 12 shows an example of the calculated occurrence rate. As explained in step S22, the state change of the device is associated with the operation content of the user performed immediately after the state change of the device and the number of operations for each operation content. The history analysis unit 23 calculates the occurrence rate of each operation count with respect to the total number of operations for the same state change, and sets priorities for the operation contents performed in the current state of the image processing device 20 in descending order of occurrence rate.

[0072] The occurrence rate is calculated as follows: For example, if the device status change indicates "Stored received document exists," the operations performed by the user immediately after this are "Stored received document print" 8 times and "Received document settings" 2 times. Therefore, the occurrence rate is 80% = 8 / 10 for "Stored received document print" and 20% = 2 / 10 for "Received document settings."

[0073] Returning to FIG. 10, the explanation will be given. The history analysis unit 23 determines up to a specified number (for example, a maximum of five) of user operation contents in descending order of occurrence rate, where the occurrence rate is equal to or greater than a threshold (for example, 30%) and the device status change matches the current status of the image processing device 20. The history analysis unit 23 sets priorities 1 to 5 for the operation contents in descending order of occurrence rate (S23). The maximum of five is just an example, and the number may be less than five or six or more. For example, according to FIG. 12, if the current device status change is "fax line disconnection," the priority of "time-specified transmission," which has the highest occurrence rate, is set to 1, and the priority of "reception mode switching," which has the next highest occurrence rate, is set to 2. Therefore, priorities 3 to 5 are left vacant.

[0074] Next, the history analysis unit 23 determines whether or not there is an empty priority order number (S24). If the determination in step S24 is Yes, the process in FIG. 10 ends, and if No, the process proceeds to step S25.

[0075] In step S25, the history analysis unit 23 counts and lists the number of operations for each operation content of the user without taking into account changes in the state of the device. The list of the number of operations for each operation content is shown in FIG.

[0076] The history analysis unit 23 sets the operation contents to vacant numbers in the order of the number of operations in descending order of frequency (S26). Thus, the priorities of 1 to 5 for the operation contents are determined. According to FIG. 13, the top three operation contents with the most operations are "print stored received document" 23 times, "set received document" 12 times, and "security" 10 times. Therefore, priority 3 is "print stored received document", priority 4 is "set received document", and priority 5 is "security". The history analysis unit 23 notifies the screen display control unit 24 of the operation contents and their priorities. The screen display control unit 24 determines the display components 215a to 215e to be displayed on the prediction menu 215 according to the priorities of 1 to 5 for the operation contents. That is, the display components 215a to 215e transition to screens that accept the operation contents with priorities 1 to 5, respectively.

[0077] In addition, when there is an overlap between operation contents set in consideration of the current device state change and operation contents set without considering the change in the device state among the operation contents associated with the priority order, the history analysis unit 23 raises the priority of operation contents with fewer operations so as to avoid overlap.

[0078] <Modification> In addition to the image processing device 20 determining the predicted menu based on the operation history and the state change history, the user may manually register in advance the display components to be displayed on the initial screen 210. A menu that displays display components related to the operation content (an example of the second operation content) manually registered by the user is called a favorite menu to distinguish it from the predicted menu 215. The user can set in advance in the image processing device 20 whether the menu to be displayed on the initial screen 210 is the predicted menu 215 or the favorite menu.

[0079] 14 is an example of the menu setting screen 230. The menu setting screen 230 has radio buttons 231 and 232 associated with a predicted menu and a favorite menu. By selecting radio button 231 or 232, the user can set whether the menu to be displayed on the initial screen 210 is the predicted menu 215 or the favorite menu. The setting of which menu to display is held for each user by the screen display control unit 24. Note that the user may also be able to set the image processing device 20 to display both.

[0080] Fig. 15 is a sequence diagram illustrating the process in which the image processing device 20 displays a favorites menu on the initial screen 210 of an application in this modification. Fig. 15 mainly explains the differences from Fig. 6. Steps S31, S32, and S33 have been added in Fig. 15.

[0081] S31: The screen display control unit 24 receives a request to start an application. The screen display control unit 24 determines whether the predicted menu 215 or the favorites menu is set as the setting for which menu to display.

[0082] S32: If the setting to display the favorites menu is enabled, the screen display control unit 24 creates a favorites menu based on the operation contents set by the user, and provides the favorites menu to the operation unit 21.

[0083] S33: If the setting to display the predicted menu is enabled, the screen display control unit 24 notifies the history management unit 22 of a request to start an application. After that, the same processing as in FIG.

[0084] FIG. 16 shows an initial screen 210 of the application on which a favorites menu 219 is displayed. The explanation of FIG. 16 will mainly focus on the differences from FIG. 8. The favorites menu 219 displays display components 219a to 219e related to operation details that the user has manually set in advance. The user can select display components 219a to 219e related to operation details that the user wishes to use from the favorites menu 219. While the predicted menu 215 is displayed according to the current state of the image processing device 20, the favorites menu 219 displays operation details that the user has manually registered. This makes it possible to display operation details that are less frequently operated but require more operation steps, thereby reducing the operational burden and improving ease of use.

[0085] Note that the image processing device 20 may switch from the state shown in FIG. 16 to the prediction menu 215 by user operation.

[0086] <Major Effects> The image processing device 20 of this information processing device can display a predictive menu 215 that uses not only the operation history but also the state change history of the image processing device 20 as a basis for judgment. The predictive menu 215 gives priority to displaying display components 215a to 215e related to the operation content to be performed after a state change of the image processing device 20, so that it can display display components 215a to 215e related to the operation content that the user frequently performs for the state of the image processing device 20. When a display component 215a to 215e in the predictive menu is selected, the screen directly transitions to a screen that accepts the operation, so that the function of the image processing device 20 that the user actually wants to use can be executed with fewer operation steps. This makes it easier to display the function that the user wants to use, improving operability.

[0087] [Second embodiment] In this embodiment, a display system will be described in which a server device on a network determines a predicted menu, and an image processing device 20 displays the predicted menu.

[0088] In the description of this embodiment, components or contents of the figures that are given the same symbols as in the first embodiment perform the same functions, so the description of components that have already been described may be omitted or only the differences may be described.

[0089] <System configuration example> 17 is an example of a system configuration diagram of a display system 100. The display system 100 has an image processing device 20 (an example of a first information processing device) and a server device 70 (an example of a second information processing device) that are capable of communicating via a network N.

[0090] The image processing device 20 is located in a facility such as a company, and is connected to a LAN or Wi-Fi installed within the facility. The server device 70 is located, for example, in a data center. The image processing device 20 is connected to the Internet i via a firewall 8, and the server device 70 is also connected to the Internet i via a high-speed LAN or the like within the data center.

[0091] The image processing device 20 may be connected to the Internet i using wireless communication such as a mobile phone network. In this case, the wireless communication may be 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), 6G (6th Generation), LTE (Long Term Evolution), WiMAX (Worldwide Interoperability for Microwave Access), or the like.

[0092] The server device 70 has one or more information processing devices, which act as a server to provide services to the image processing device 20. A server is a computer or software that functions to provide information and processing results in response to requests from clients. As will be described later, the server device receives and stores operation details and status change notifications from the information processing devices, and transmits a predictive menu to the image processing device 20 when the user displays the initial screen 210 of the application.

[0093] The configuration of the image processing device 20 may be the same as in the first embodiment, but in this embodiment, the server device executes the processing related to personalization. Note that multiple image processing devices 20 may be able to communicate with the server device. This allows the user to display the initial screen 210 of the personalized application no matter which image processing device 20 they use.

[0094] The communication method may be a general-purpose communication protocol such as HTTP, HTTPs, or WebSocket, or a dedicated communication protocol.

[0095] The server device 70 may be realized by cloud computing or by a single information processing device. Cloud computing refers to a form in which resources on a network are used without being aware of specific hardware resources. The server device 70 may exist on the Internet or on-premise.

[0096] <Hardware configuration example> The hardware configuration of the image processing device 20 may be the same as that shown in Fig. 2. In this embodiment, an example of the hardware configuration of the server device 70 will be described.

[0097] Fig. 18 is a hardware configuration diagram of the server device 70. As shown in Fig. 18, the server device 70 is constructed by a computer, and includes a CPU 601, a ROM 602, a RAM 603, an HD 604, an HDD (Hard Disk Drive) controller 605, an external device connection I / F (Interface) 608, a network I / F 609, a bus line 610, and a media I / F 616.

[0098] Of these, the CPU 601 controls the operation of the entire computer. The ROM 602 stores programs used to drive the CPU 601, such as the IPL. The RAM 603 is used as a work area for the CPU 601. The HD 604 stores various data, such as programs. The HDD controller 605 controls the reading and writing of various data from and to the HD 604 under the control of the CPU 601. The external device connection I / F 608 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories and printers. The network I / F 609 is an interface for data communication using a communication network. The bus line 610 is an address bus, a data bus, or the like for electrically connecting the components, such as the CPU 601, shown in FIG. 18.

[0099] A media I / F 616 controls reading and writing (storing) of data from and to a recording medium 615 such as a flash memory.

[0100] <Display system functions> Next, functions of the display system 100 will be described with reference to Fig. 19. Fig. 19 is an example of a functional block diagram showing functions of the display system 100 in blocks. Note that the description of Fig. 19 will mainly focus on the differences from Fig. 3.

[0101] In this embodiment, the image processing device 20 has a first communication unit 31, an operation unit 21, and a device status change detection unit 25. The functions of the operation unit 21 and the device status change detection unit 25 may be similar to those of the first embodiment. However, the operation unit 21 and the device status change detection unit 25 notify the server device 70 of the operation content and the device status change via the first communication unit 31.

[0102] The first communication unit 31 transmits and receives various information to and from the server device 70. The first communication unit 31 transmits to the server device 70 the operation content accepted by the operation unit 21 and the device state change detected by the device state change detection unit 25. In addition, the first communication unit 31 receives a predicted menu 215 from the server device 70 when the user displays an initial screen 210 of an application.

[0103] <<Server device functions>> The server device 70 has a second communication unit 32, a history management unit 22, a history analysis unit 23, and a screen display control unit 24. Each function of the server device 70 is a function or means realized when any of the components shown in Fig. 18 operates in response to an instruction from the CPU 601 in accordance with a program loaded from the HD 604 onto the RAM 603.

[0104] The second communication unit 32 transmits and receives various information to and from the image processing device 20. The second communication unit 32 receives operation details and changes in the device status from the image processing device 20. Furthermore, when the user displays the initial screen 210 of the application, the second communication unit 32 transmits a prediction menu 215 to the image processing device 20. Other functions are the same as those of the first embodiment, or, even if they are different, they will not hinder the explanation of this embodiment.

[0105] <Saving operation history> The process of saving the operation history and the status change history will be described with reference to Fig. 20. Fig. 20 is a sequence diagram illustrating the process of the server device 70 saving the operation history and the status change history of the user. Note that the explanation of Fig. 20 may include explanations of differences from Fig. 4.

[0106] S51: The user inputs authentication information into the operation unit 21 and logs in. The user operates any application such as a fax application, a scanner application, a copy application, or a printer application. The operation unit 21 accepts the user's operation.

[0107] S52: The first communication unit 31 of the image processing device 20 transmits to the server device 70 the operation content relating to the image processing performed by the user.

[0108] S53: The second communication unit 32 of the server device 70 receives the operation content related to the image processing. The history management unit 22 stores the operation history shown in FIG.

[0109] S54: The device status change detection unit 25, which monitors the status of the image processing device 20, detects any change in the status of the image processing device 20. The first communication unit 31 of the image processing device 20 transmits the details of the status change to the server device 70.

[0110] S55: The second communication unit 32 of the server device 70 receives the content of the status change, and the history management unit 22 stores the status change history as shown in FIG.

[0111] <Prediction menu display process> Next, a display process of the predicted menu 215 will be described with reference to Fig. 21. Fig. 21 is a sequence diagram illustrating a process in which the server device 70 determines the predicted menu 215 and the image processing device 20 displays the initial screen 210 of the application. Note that the description of Fig. 21 may include differences from Fig. 6.

[0112] S61: First, it is assumed that the image processing device 20, in which the user has logged in, displays the home screen 200. The user presses an application displayed on the home screen 200 to start the application. The operation unit 21 accepts the application being pressed.

[0113] S62: The first communication unit 31 of the image processing device 20 transmits to the server device 70 information that an application has been designated and that an operation to start the application has been performed, as well as the current status of the image processing device 20.

[0114] S63: The second communication unit 32 of the server device 70 receives the information that a startup operation has been performed. As a result, the history analysis unit 23 acquires the operation history and the status change history from the history management unit 22. The history analysis unit 23 analyzes the operation history and the status change history, determines the priority of the operation contents, and the screen display control unit 24 creates the predicted menu 215.

[0115] S64: The second communication unit 32 of the server device 70 transmits information constituting the predicted menu 215 (information including the predicted operation content) to the image processing device 20.

[0116] S65: The first communication unit 31 of the image processing device 20 receives the predicted menu 215, and the display unit 26 displays the initial screen 210 of the application on which the predicted menu 215 is reflected.

[0117] It should be noted that a favorite menu can also be displayed in this embodiment. The user registers the favorite menu in advance in the server device 70. The user also sets in the server device 70 whether to display the predicted menu 215 or the favorite menu 219.

[0118] <Major Effects> According to this embodiment, in addition to the effects of the first embodiment, the server device 70 executes the processing related to personalization that is performed by the image processing device 20, thereby reducing the processing load on the image processing device 20. Furthermore, the server device 70 manages the operation history and status change history, so that the operation history and status change history of multiple image processing devices 20 can be accumulated, and the predicted menu 215 can be displayed regardless of which image processing device 20 the user operates.

[0119] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0120] For example, the history analysis unit 23 may determine the priority of operation contents using a machine-learned model. For example, when the model is configured using a neural network, the model learns the correspondence between the change in device status and the operation contents, inputting the change in device status and outputting the operation contents performed by the user immediately after the status change. When the current status of the image processing device 20 is input, the model outputs the probability of the operation contents predicted to be performed by the user. For example, the five operation contents with the highest probability correspond to the priorities 1 to 5.

[0121] In addition, the configuration examples shown in Fig. 3 and the like are divided according to main functions to facilitate understanding of the processing by the image processing device 20. The method of dividing the processing units and their names do not limit the present invention. The processing by the image processing device 20 can be divided into even more processing units depending on the processing content. Furthermore, it can also be divided so that one processing unit includes even more processes.

[0122] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0123] <Additional Notes> [Claim 1] An information processing device, a history management unit that manages an operation history of operations received from a user on the information processing device and a status change history of the information processing device; a history analysis unit that analyzes the operation history and the state change history to predict operation content according to the state of the information processing device; a screen display control unit that displays, on a display means, a display component related to the operation content predicted by the history analysis unit; An information processing device having the above. [Claim 2] the display component is a button or an icon for displaying a screen for accepting the operation content predicted by the history analysis unit; The information processing device according to claim 1. [Claim 3] the history management unit manages the operation history and the state change history in chronological order; the history analysis unit predicts the operation content that the user will perform on the current state of the information processing device based on the operation content that was performed immediately after a state change that occurred in the information processing device. 3. The information processing device according to claim 1 or 2. [Claim 4] the history analysis unit counts the number of operations of the operation content performed immediately after a state change of the information processing device for each type of the state change; Calculating the ratio of the number of times each operation occurred to the total number of times each operation occurred for the same state change; Among the operation contents performed in the current state of the information processing device, a priority is set to the operation contents in descending order of the proportion; predicting the operation content having the priority equal to or higher than a threshold as the operation content according to the state of the information processing device; 4. The information processing device according to claim 1. [Claim 5] If the number of operation contents predicted based on the ratio does not reach a specified number, and therefore the number of operation contents to which the priority order is set does not reach the specified number, the history analysis unit sets priorities for the operation contents that do not reach the specified number in descending order of the number of times of operation; The information processing device according to claim 4. [Claim 6] a setting for displaying either a first operation content predicted by the history analysis unit or a second operation content preset by a user is stored; the screen display control unit causes a display unit to display the display component related to the first operation content or the second operation content based on the setting. 6. The information processing device according to claim 1. [Claim 7] the information processing device accepts a selection of one of a plurality of applications; the screen display control unit displays the display component on an initial screen of the application that has received the selection. 7. The information processing device according to claim 1. [Claim 8] When the pressing of the display component is accepted, the screen display control unit transitions to a screen that accepts operation content according to the state of the information processing device. 8. The information processing device according to claim 1. [Explanation of symbols]

[0124] 20 Image processing device 70 Server equipment [Prior art documents] [Patent documents]

[0125] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-236028

Claims

1. An information processing device, a history management unit that manages an operation history of operations received from a user on the information processing device and a status change history of the information processing device; a history analysis unit that analyzes the operation history and the state change history to predict operation content according to the state of the information processing device; a screen display control unit that displays, on a display means, a display component related to the operation content predicted by the history analysis unit; An information processing device having the above.

2. the display component is a button or an icon for displaying a screen for accepting the operation content predicted by the history analysis unit; The information processing device according to claim 1 .

3. the history management unit manages the operation history and the state change history in chronological order; the history analysis unit predicts the operation content that the user will perform on the current state of the information processing device based on the operation content that was performed immediately after a state change that occurred in the information processing device.

3. The information processing device according to claim 1 or 2.

4. the history analysis unit counts the number of operations of the operation content performed immediately after a state change of the information processing device for each type of the state change; Calculating the ratio of the number of times each operation occurred to the total number of times each operation occurred for the same state change; Among the operation contents performed in the current state of the information processing device, a priority is set to the operation contents in descending order of the proportion; predicting the operation content having the priority equal to or higher than a threshold as the operation content according to the state of the information processing device; The information processing device according to claim 1 .

5. If the number of operation contents predicted based on the ratio does not reach a specified number, and therefore the number of operation contents to which the priority order is set does not reach the specified number, the history analysis unit sets priorities for the operation contents that do not reach the specified number in descending order of the number of times of operation; The information processing device according to claim 4 .

6. a setting for displaying either a first operation content predicted by the history analysis unit or a second operation content preset by a user is stored; the screen display control unit causes a display unit to display the display component related to the first operation content or the second operation content based on the setting. The information processing device according to claim 1 .

7. the information processing device accepts a selection of one of a plurality of applications; the screen display control unit displays the display component on an initial screen of the application that has received the selection. The information processing device according to claim 1 .

8. When the pressing of the display component is accepted, the screen display control unit transitions to a screen that accepts operation content according to the state of the information processing device. The information processing device according to claim 1 .

9. An information processing method performed by an information processing device, a process in which a history management unit manages an operation history of operations received from a user on the information processing device and a status change history of the information processing device; a process in which a history analysis unit analyzes the operation history and the state change history to predict operation content according to the state of the information processing device; a process in which a screen display control unit displays, on a display means, a display component relating to the operation content predicted by the history analysis unit; An information processing method that performs the above.

10. An information processing device a history management unit that manages an operation history of operations received from a user on the information processing device and state changes of the information processing device; a history analysis unit that analyzes the operation history and the state change to predict operation content according to the state of the information processing device; a screen display control unit that displays, on a display means, a display component related to the operation content predicted by the history analysis unit; A program to function as a

11. A display system in which a first information processing device and a second information processing device communicate with each other via a network, The first information processing device an operation unit that receives an operation from a user on the first information processing device; a device state change detection unit that detects a state change of the first information processing device; a first communication unit that transmits the operation content received by the operation unit and the state change detected by the device state change detection unit to the second information processing device, The second information processing device a history management unit that manages an operation history based on the operation content received from the first information processing device and a status change history based on the status change; a history analysis unit that analyzes the operation history and the state change history to predict operation content according to the state of the first information processing device; a screen display control unit that creates a display element related to the operation content predicted by the history analysis unit; a second communication unit that transmits information that configures the display component to the first information processing device; A display system having:

Images