Image determination method, information processing device, and program

Abstract

To properly determine an item to be an object to be operated if a user is confused with operation.SOLUTION: A determination method for an image includes: acquiring operation information associated with an operation done by a first user in a state in which an image including a plurality of item images is displayed; and determining, on detecting a state in which the first user cannot determine on which one of the plurality of item images the operation is to be performed based upon the operation information, at least one item image to be an object for which the first user is urged to operate among the plurality of item images based upon operation history information associated with an item image being an object to be operated by at least one of the first user in the state in which the image including the plurality of item images is displayed and a second user different from the first user.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an image determination method, an information processing apparatus, and a program.

Background Art

[0002] A user may be confused about operations of a device, application software, etc. Possible reasons for the user's confusion include, for example, unclear operation methods, complex operation methods, or the user being unfamiliar with the operations. When the user is thus confused about operations, technologies for assisting the user's operations have been developed. For example, Patent Document 1 discloses an operation support system that determines whether the user is confused based on operation information of software collected using cloud computing and operation information of the software by the user. When it is determined that the user is confused, the operation support system presents help information stored in a help information storage unit to the user.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 does not disclose a method for determining help information presented to the user. For this reason, the information presented to the user may not be appropriately determined. Therefore, even when the user checks the presented help information, the user may not be able to grasp the operation method, and as a result, the confused state may not be resolved.

Means for Solving the Problems

[0005] One aspect of the image determination method according to the present invention includes: acquiring operation information relating to the operation of a first user in a state in which an image containing multiple item images is displayed; and, when a state is detected based on the operation information in which the first user is unable to decide which of the multiple item images to operate on, determining at least one item image from among the multiple item images that will prompt the first user to operate on, based on operation history information relating to the item image that was the target of operation by at least one of the first user and a second user different from the first user in a state in which an image containing multiple item images is displayed.

[0006] One aspect of the information processing apparatus according to the present invention comprises a processing apparatus, which performs the following actions: acquiring operation information relating to the actions of a first user in a state in which an image including a plurality of item images is displayed; and, based on the operation information, detecting a state in which the first user is unable to decide which of the plurality of item images to perform an action on, determining at least one item image among the plurality of item images to prompt the first user to perform an action on, based on operation history information relating to the item images that were the target of the actions of at least one of the first user and a second user different from the first user in a state in which an image including a plurality of item images is displayed.

[0007] One aspect of the program according to the present invention involves causing a processing device to perform the following actions: acquire operation information relating to the actions of a first user when an image containing multiple item images is displayed; and, based on the operation information, detect a state in which the first user cannot decide which of the multiple item images to perform an action on, determine at least one item image from the multiple item images that the first user should be prompted to act on, based on operation history information relating to the item images that were the target of operations by at least one of the first user and a second user different from the first user when the image containing multiple item images is displayed. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram illustrating the notification system Sys according to the first embodiment. [Figure 2] This is a schematic diagram illustrating an example of an operation image GW displayed on the touch panel 36a. [Figure 3] This is a schematic diagram illustrating an example of an operation image GW displayed on the touch panel 36b. [Figure 4] This is a block diagram showing the configuration of the notification system Sys according to the first embodiment. [Figure 5] This is a block diagram showing the configuration of the storage device 10 according to the first embodiment. [Figure 6] This is a flowchart illustrating the operation of Server 1 according to the first embodiment. [Figure 7] This is a flowchart to explain the process of determining the target of the operation. [Figure 8] This block diagram shows the configuration of smartphone 1A according to the second embodiment. [Figure 9] This is a block diagram showing the configuration of the storage device 10A according to the second embodiment. [Figure 10] This is a flowchart illustrating the operation of smartphone 1A according to the second embodiment. [Figure 11] This is a schematic diagram to explain the notification image GN1. [Figure 12] This is a schematic diagram to explain the notification image GN2. [Figure 13] This is a schematic diagram to explain the manipulated image GWa2. [Figure 14] This is a schematic diagram to explain the notification image GN3. [Figure 15] This is a schematic diagram to explain the operation image GWa3. [Modes for carrying out the invention]

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the attached drawings. Note that the dimensions and scale of parts in the drawings may differ from those of the actual parts, and some parts are shown schematically for ease of understanding. Furthermore, the scope of the present invention is not limited to these embodiments unless otherwise stated in the following description.

[0010] 1. First Embodiment In the first embodiment, a notification system comprising a smartphone operated by a user and a server that acquires information about the user's operation from the smartphone and determines whether or not the user is confused about the operation based on that information will be described using this example as an illustration of the image determination method, information processing device, and program according to the present invention. When the server according to this embodiment detects that the user is confused, it determines notification content to support the user's operation based on the operation history of one or more users, including the user in question. The smartphone according to this embodiment supports the user's operation by displaying an image based on the notification content determined by the server.

[0011] 1.1. Overview of the Notification System Figure 1 is a schematic diagram illustrating the notification system Sys according to the first embodiment. The notification system Sys comprises a server 1, a smartphone 3a, and a smartphone 3b. Server 1 stores operation history information 102. Smartphone 3a is equipped with a touch panel 36a. Smartphone 3a also displays an operation image GW containing multiple item images GC on the touch panel 36a. Each of the multiple item images GC is an image representing a button, checkbox, radio button, scroll bar, etc., for executing a predetermined function. User Ua performs an operation such as pressing an item image GC or swiping an operation image GW by touching the touch panel 36a. Smartphone 3b is equipped with a touch panel 36b. Smartphone 3b also displays an operation image GW containing multiple item images GC on the touch panel 36b, similar to smartphone 3a. User Ub performs an operation such as pressing an item image GC or swiping an operation image GW by touching the touch panel 36b, similar to user Ua. Note that the display mode of the operation image GW shown on touch panel 36a and the display mode of the operation image GW shown on touch panel 36b may differ from each other. Also, the display mode of the operation image GW may change or switch to other operation image GWs depending on the operation stage.

[0012] Server 1 acquires operation information JMa from smartphone 3a via network NW. The operation information JMa is information regarding an operation performed by user Ua on smartphone 3a when an operation image GW is displayed on the touch panel 36a. The operation information JMa includes, for example, information indicating the operation image GW displayed on the touch panel 36a when the operation was executed, information indicating the position on the touch panel 36a where the operation was received, information indicating the item image GC that was the target of the operation, information indicating the number of operations performed from a certain point, and information indicating the time taken from a certain operation to the next operation, etc. Also, when the operation image GW is a GUI (Graphical User Interface) used when operating a specific device, the operation information JMa includes information regarding the type and model number of the device. Also, when the operation image GW is a GUI used when operating a specific application software, the operation information JMa includes information indicating the version of the application software. The operation information JMa may be transmitted to server 1 every predetermined number of times the operation performed by user Ua on smartphone 3a, or may be transmitted at a predetermined time interval. In the following, the information indicating the item image GC that was the target of the operation may be referred to as "target information". Also, the information indicating the number of operations performed from a certain point may be referred to as "count information". Also, the information indicating the time taken from a certain operation to the next operation may be referred to as "time information".

[0013] Also, server 1 acquires operation information JMb from smartphone 3b via network NW. The operation information JMb is information regarding an operation performed by user Ub on smartphone 3b when an operation image GW is displayed on the touch panel 36b. The operation information JMb includes the same information as the operation information JMa. For example, the operation information JMb includes target information.

[0014] Server 1 determines whether user Ua is confused about the operation based on the operation information JMa. Specifically, Server 1 determines whether it is in a state where it cannot determine which item image among the plurality of item images GC included in the operation image GW the user Ua will perform an operation on.

[0015] If it is determined that user Ua is confused about the operation, that is, if it is determined that user Ua is in a state where it cannot determine which item image among the plurality of item images GC to perform an operation on, Server 1 determines the notification content for assisting user Ua's operation based on the operation history information 102. Specifically, Server 1 determines at least one item image GC among the plurality of item images GC that is the target to prompt user Ua to perform an operation based on the operation history information 102.

[0016] The operation history information 102 is information regarding the item image GC that was the target of the operation of at least one of user Ua and user Ub when the operation image GW was displayed. Server 1 updates the operation history information 102 based on the operation information JMa and the operation information JMb. That is, the operation history information 102 includes various information included in the operation information JMa and the operation information JMb. For example, the operation history information 102 includes target information.

[0017] When at least one item image GC among the plurality of item images GC that is the target to prompt user Ua to perform an operation is determined, Server 1 transmits the instruction information JDa to the smartphone 3a. The instruction information JDa is information for causing the smartphone 3a to notify user Ua of the notification content for assisting user Ua's operation, that is, the determined at least one item image GC. The smartphone 3a displays an image for notifying user Ua of the determined at least one item image GC on the touch panel 36a based on the instruction information JDa. By checking the item image GC indicated by the image displayed based on the instruction information JDa, user Ua can grasp the operation method and eliminate confusion.

[0018] Figure 2 is a schematic diagram illustrating an example of an operation image GW displayed on the touch panel 36a. Operation image GWa1 is displayed on the touch panel 36a as the operation image GW. Figure 3 is a schematic diagram illustrating an example of an operation image GW displayed on the touch panel 36b. Operation image GWb1 is displayed on the touch panel 36b as the operation image GW. Operation image GWa1 and operation image GWb1 each include multiple item images GC, namely item images GC1 to GC9. The display modes of operation image GWa1 and operation image GWb1 are different from each other.

[0019] 1.2. Configuration and Function of the Notification System The configuration and functions of the notification system Sys according to the first embodiment will be described below with reference to Figures 4 and 5.

[0020] Figure 4 is a block diagram showing the configuration of the notification system Sys according to the first embodiment. Server 1 comprises a storage device 10 for storing various information, a processing device 12 for controlling the operation of Server 1, and a communication device 14 for performing communication with terminal devices including smartphones 3a and 3b, as well as external storage devices, etc. The processing device 12 has the functions of a communication control unit 120, a confusion detection unit 121, a decision unit 122, a data management unit 123, and an instruction unit 124.

[0021] The storage device 10 is composed of, for example, volatile memory such as RAM and non-volatile memory such as ROM. Here, RAM is an abbreviation for Random Access Memory, and ROM is an abbreviation for Read Only Memory.

[0022] Figure 5 is a block diagram showing the configuration of the storage device 10 according to the first embodiment. The non-volatile memory of the storage device 10 stores a program 100 that defines the operation of the server 1, designation information 101 that specifies a predetermined item image GC from among a plurality of item image GCs, and operation history information 102. The operation history information 102 includes frequency information 103 that indicates the frequency of operations performed on each of the plurality of item image GCs by at least one of user Ua and user Ub. That is, the frequency information 103 includes information that indicates the number of times an operation has been performed on each of the plurality of item image GCs by at least one of user Ua and user Ub.

[0023] Furthermore, the volatile memory of the storage device 10 is used by the processing unit 12 as a work area when executing the program 100.

[0024] Furthermore, part or all of the storage device 10 may be located on an external storage device or an external server. Also, part or all of the various information stored in the storage device 10 may be stored in the storage device 10 in advance, or it may be obtained from an external storage device or an external server.

[0025] The processing unit 12 is comprised of one or more CPUs. However, the processing unit 12 may also include a programmable logic device such as an FPGA instead of a CPU, or in addition to a CPU. Here, CPU is an abbreviation for Central Processing Unit, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0026] The processing unit 12 functions as a communication control unit 120, confusion detection unit 121, decision unit 122, data management unit 123, and instruction unit 124, as shown in Figure 4, when the CPU and other components of the processing unit 12 execute the program 100.

[0027] The communication control unit 120 acquires various information from terminal devices, external storage devices, and external servers that are connected to the server 1 in a communicative manner by controlling the communication device 14. The communication control unit 120 also stores the acquired information in the storage device 10. Furthermore, the communication control unit 120 transmits various information to terminal devices, external storage devices, and external servers by controlling the communication device 14. For example, the communication control unit 120 acquires operation information JMa from a smartphone 3a, which is a terminal device. The communication control unit 120 also acquires operation information JMb from a smartphone 3b, which is a terminal device. The communication control unit 120 also transmits instruction information JDa to the smartphone 3a.

[0028] The confusion detection unit 121 detects a state in which user Ua is confused about the operation based on the operation information JMa. That is, the confusion detection unit 121 determines whether or not user Ua is confused about the operation based on various information contained in the operation information JMa. More specifically, the confusion detection unit 121 determines whether or not user Ua is in a state in which they cannot decide which of the multiple item images GC contained in the operation image GW to perform an operation on, based on various information contained in the operation information JMa.

[0029] For example, the confusion detection unit 121 determines whether the time indicated by the time information is above a threshold. If the time indicated by the time information is above a threshold, the confusion detection unit 121 determines that the user Ua is in a state where they cannot decide which of the multiple item images GC included in the operation image GW to perform an operation on.

[0030] Furthermore, the confusion detection unit 121 determines whether the number of operations indicated by the count information is equal to or greater than a threshold. If the number of operations indicated by the count information is equal to or greater than a threshold, the confusion detection unit 121 determines that the user Ua is in a state where they cannot decide which of the multiple item images GC included in the operation image GW to perform the operation on.

[0031] Furthermore, the confusion detection unit 121 determines whether the same operation has been performed repeatedly multiple times. If the same operation has been performed repeatedly multiple times, for example, if the user repeatedly presses a specific item image GC, or repeatedly swipes the operation image GW up and down, the confusion detection unit 121 determines that the user Ua is in a state where they cannot decide which of the multiple item images GC contained in the operation image GW to perform the operation on.

[0032] Furthermore, various methods can be employed to detect when a user (Ua) is confused or unsure of the operation, in addition to the methods described above.

[0033] The decision unit 122 determines the content of the notification to support the user Ua's actions. Specifically, the decision unit 122 determines at least one item image GC from among multiple item image GCs that will prompt the user Ua to take action.

[0034] For example, based on the frequency information 103, the determination unit 122 determines the item image GC that has been most frequently targeted by an operation of at least one of user Ua and user Ub as the item image GC to which user Ua will be prompted to perform an operation. The item image GC that has been most frequently targeted by an operation of at least one of user Ua and user Ub is sometimes referred to as the "first priority item image".

[0035] Furthermore, based on the frequency information 103, the determination unit 122 determines the item image GC that is the next most frequently targeted by an operation of at least one of user Ua and user Ub, after the first priority item image, as the item image GC to prompt user Ua to perform an operation on. Note that the item image GC that is the next most frequently targeted by an operation of at least one of user Ua and user Ub, after the first priority item image, may be referred to as the "second priority item image".

[0036] If, based on frequency information 103, it is not possible to determine at least one item image GC to prompt user Ua for an action, for example, if the number of operation information JMa and operation information JMb acquired by server 1 is insufficient and the reliability of the operation frequency indicated by frequency information 103 is low, the determination unit 122 determines at least one item image GC to prompt user Ua for an action based on designation information 101. That is, the determination unit 122 determines a predetermined item image GC specified in designation information 101 as the item image GC to prompt user Ua for an action.

[0037] If, based on the specified information 101, it is not possible to determine at least one item image GC to prompt user Ua for an action, for example, if the specified information 101 is not stored in the storage device 10, or if the predetermined item image GC specified by the specified information 101 is not included among the multiple item image GCs included in the operation image GW displayed on the touch panel 36a when the operation was performed, the determination unit 122 determines at least one item image GC to prompt user Ua for an action based on the operation history information 102. Specifically, the determination unit 122 determines the item image GC indicated by the target information included in the operation history information 102 as the item image GC to prompt user Ua for an action.

[0038] Hereinafter, we assume that the first priority item images are item images GC3, GC4, GC5, and GC9 as shown in Figures 2 and 3. That is, item images GC3, GC4, GC5, and GC9 are the item images GC that have been the target of operations by at least one of the users Ua and Ub most frequently, and each of them has been the target of an operation equally frequently.

[0039] Furthermore, we assume that the second-priority item image is item image GC2, as shown in Figures 2 and 3. That is, item image GC2 is the item image GC that is targeted by at least one of user Ua and user Ub most frequently, second only to the first-priority item image.

[0040] Furthermore, it is assumed that the predetermined item image GC specified in the specified information 101 is item image GC2, item image GC3, item image GC4, item image GC5, and item image GC9 as shown in Figures 2 and 3.

[0041] The data management unit 123 manages various types of information. For example, the data management unit 123 updates the operation history information 102 based on the operation information JMa. Similarly, the data management unit 123 updates the operation history information 102 based on the operation information JMb. Specifically, the data management unit 123 updates the frequency information 103 based on the target information contained in the operation information JMa. Furthermore, the data management unit 123 updates the frequency information 103 based on the target information contained in the operation information JMb. More specifically, the data management unit 123 updates the information in the frequency information 103 that indicates the number of operations performed on the item image GC indicated by the target information.

[0042] The instruction unit 124 generates instruction information JDa based on the notification content determined by the decision unit 122. Specifically, the instruction unit 124 generates instruction information JDa to notify the user Ua of at least one item image GC that the decision unit 122 has determined to prompt the user Ua to take action on.

[0043] The communication device 14 includes, for example, an interface board having a connector and an interface circuit, and has the function of receiving various information from terminal devices, external storage devices, and external servers, and the function of transmitting various information to terminal devices, external storage devices, and external servers, etc. The communication device 14 may send and receive various information using wired communication, or it may send and receive various information using wireless communication. When using wireless communication, the communication device 14 is configured to include an antenna that supports wireless communication compliant with a predetermined communication standard. In this embodiment, the communication device 14 is connected to smartphones 3a and 3b via a network NW so as to be able to communicate, and sends and receives various information between smartphones 3a and 3b.

[0044] The touch panel 36a is a device that integrates a display device for displaying images and an input device for receiving input operations from the user Ua. The display device is a so-called display panel and is composed of, for example, a liquid crystal panel or an organic EL panel. Here, EL is an abbreviation for Electro-Luminescence. The display device displays various images under the control of a control device (not shown) provided by the smartphone 3a. The input device is composed of, for example, a transparent sheet-like contact sensor. The input device is provided so as to cover the display device. The input device detects the touch position using an object in contact with the input device and the capacitance identified by the input device, and outputs data indicating the detected touch position to the control device provided by the smartphone 3a.

[0045] The touch panel 36b has the same configuration as the touch panel 36a. The display device of the touch panel 36b displays various images under the control of the control device (not shown) of the smartphone 3b. The input device of the touch panel 36b receives input operations from the user Ub and outputs data indicating the detected touch position to the control device of the smartphone 3b.

[0046] When the smartphone 3a displays the operation image GW on the touch panel 36a, it generates operation information JMa based on the operation received from the user Ua. The smartphone 3a also sends the operation information JMa to the server 1.

[0047] When the smartphone 3b displays the operation image GW on the touch panel 36b, it generates operation information JMb based on the operation received from the user Ub. The smartphone 3b also sends the operation information JMb to the server 1.

[0048] 1.3. Server Operation The operation of Server 1 according to the first embodiment will be described below with reference to Figures 6 and 7.

[0049] Figure 6 is a flowchart illustrating the operation of Server 1 according to the first embodiment. The series of operations shown in the flowchart begins, for example, when Server 1 is powered on and receives operation information JMa from smartphone 3a.

[0050] In step S101, the communication control unit 120 acquires operation information JMa from the smartphone 3a by controlling the communication device 14.

[0051] In step S102, the confusion detection unit 121 determines, based on the operation information JMa, whether or not user Ua is confused about the operation. That is, the confusion detection unit 121 determines, based on the operation information JMa, whether or not user Ua is in a state where they cannot decide which of the multiple item images GC included in the operation image GW to operate on. If user Ua is confused about the operation, in other words, if user Ua is in a state where they cannot decide which of the multiple item images GC included in the operation image GW to operate on, that is, if the result in step S102 is YES, the confusion detection unit 121 proceeds to step S200. If user Ua is not confused about the operation, in other words, if user Ua is not in a state where they cannot decide which of the multiple item images GC included in the operation image GW to operate on, that is, if the result in step S102 is NO, the confusion detection unit 121 proceeds to step S105.

[0052] If the processing unit 12 determines, based on the operation information JMa, that user Ua is unable to decide which of the multiple item images GC included in the operation image GW to perform an operation on, in other words, if it detects that user Ua is unable to decide which of the multiple item images GC included in the operation image GW to perform an operation on, the processing unit 12 determines in step S200 the content of a notification to support user Ua's operation. That is, in the operation target determination process executed in step S200, the processing unit 12 determines at least one item image GC from the multiple item images GC that will prompt user Ua to perform an operation. If the process in step S200 is executed, the processing unit 12 proceeds to step S103.

[0053] Figure 7 is a flowchart illustrating the process of determining the target of the operation. This flowchart shows a series of operations in the process of determining the target of the operation in step S200.

[0054] In step S201, the determination unit 122 determines, based on the frequency information 103, whether it can determine at least one item image GC that will prompt user Ua to take action. If it can determine at least one item image GC that will prompt user Ua to take action based on the frequency information 103, i.e., if the result in step S201 is YES, the determination unit 122 proceeds to step S202. If it cannot determine at least one item image GC that will prompt user Ua to take action based on the frequency information 103, i.e., if the result in step S201 is NO, the determination unit 122 proceeds to step S203.

[0055] In step S202, the decision unit 122 determines, based on the frequency information 103, at least one item image GC from among the multiple item image GCs to prompt user Ua to take action. Specifically, the decision unit 122 determines the first priority item image and the second priority item image as item image GCs to prompt user Ua to take action. More specifically, the decision unit 122 determines item image GC2, item image GC3, item image GC4, item image GC5, and item image GC9 as item image GCs to prompt user Ua to take action.

[0056] In step S203, the determination unit 122 determines, based on the specified information 101, whether it can determine at least one item image GC to prompt user Ua for an action. If it can determine at least one item image GC to prompt user Ua for an action based on the specified information 101, i.e., if the result in step S203 is YES, the determination unit 122 proceeds to step S204. If it cannot determine at least one item image GC to prompt user Ua for an action based on the specified information 101, i.e., if the result in step S203 is NO, the determination unit 122 proceeds to step S205.

[0057] In step S204, the determination unit 122 determines, based on the specified information 101, at least one item image GC from among a plurality of item images GC to prompt user Ua to take action. Specifically, the determination unit 122 determines a predetermined item image GC specified in the specified information 101 as the item image GC to prompt user Ua to take action. More specifically, the determination unit 122 determines item image GC2, item image GC3, item image GC4, item image GC5, and item image GC9 as the item image GC to prompt user Ua to take action.

[0058] In step S205, the determination unit 122 determines, based on the operation history information 102, at least one item image GC from among multiple item image GCs to prompt user Ua to perform an action. Specifically, the determination unit 122 determines the item image GC indicated by the target information included in the operation history information 102 as the item image GC to prompt user Ua to perform an action.

[0059] After the processing in step S202, step S204, or step S205 is performed, the processing unit 12 terminates the operation target determination process shown in the flowchart of Figure 7.

[0060] When the process for determining the target of the operation, as shown in the flowchart of Figure 7, is completed, the processing unit 12 executes the process in step S103, as shown in the flowchart of Figure 6.

[0061] In step S103, the instruction unit 124 generates instruction information JDa based on the notification content determined by the decision unit 122.

[0062] For example, if the determination unit 122 determines, based on the frequency information 103, at least one item image GC that the user Ua is prompted to act upon, the instruction information JDa is information to notify the smartphone 3a that the at least one item image GC is item image GC2, item image GC3, item image GC4, item image GC5, and item image GC9.

[0063] In step S104, the communication control unit 120 controls the communication device 14 to send instruction information JDa to the smartphone 3a.

[0064] Based on the instruction information JDa, the smartphone 3a notifies the user Ua of the notification content determined by the decision unit 122. Specifically, based on the instruction information JDa, the smartphone 3a displays an image on the touch panel 36a to notify the user Ua of at least one item image GC that prompts the user to take action.

[0065] In step S105, the data management unit 123 updates the operation history information 102 based on the operation information JMa. Specifically, the data management unit 123 updates the frequency information 103 based on the target information included in the operation information JMa.

[0066] After the processing in step S105 is completed, the processing unit 12 completes the series of operations shown in the flowchart of Figure 6.

[0067] Based on the above, according to the first embodiment, the server 1 can determine, based on the operation history information 102, at least one item image GC from among a plurality of item image GCs to prompt user Ua for an operation. That is, the server 1 can designate an item image GC that has been operated on by at least one of user Ua and user Ub as the item image GC to prompt user Ua for an operation.

[0068] Furthermore, according to the first embodiment, the server 1 can determine, based on frequency information 103, a first priority item image and a second priority item image as item image GCs to which user Ua should be prompted to perform an action. That is, the server 1 can select item image GCs with a high operation frequency from among item image GCs that have been operated by at least one of user Ua and user Ub, and set them as item image GCs to which user Ua should be prompted to perform an action.

[0069] Furthermore, according to the first embodiment, the server 1 can determine a predetermined item image GC as the item image GC to which the user Ua will be prompted to perform an action, based on the designated information 101. In other words, even if the server 1 cannot determine at least one item image GC to which the user Ua will be prompted to perform an action, based on the frequency information 103, it can provide highly accurate operation support by, for example, setting an item image GC that is expected to be frequently operated in advance as the item image GC specified in the designated information 101.

[0070] As described above, the image determination method according to the first embodiment includes: acquiring operation information JMa regarding the operation of user Ua in a state where an operation image GW including a plurality of item images GC is displayed; and, when a state is detected based on the operation information JMa in which user Ua cannot determine which of the plurality of item images GC to perform an operation on, determining at least one item image GC from the plurality of item images GC that will prompt user Ua to perform an operation, based on operation history information 102 regarding the item image GC that was the target of the operation of user Ua and at least one of user Ub other than user Ua in a state where the operation image GW is displayed.

[0071] Furthermore, the server 1 according to the first embodiment includes a processing unit 12, which performs the following: acquires operation information JMa regarding the operation of user Ua when an operation image GW including multiple item images GC is displayed; and, based on the operation information JMa, detects a state in which user Ua cannot decide which of the multiple item images GC to operate on, and determines at least one item image GC from the multiple item images GC that will prompt user Ua to operate on, based on operation history information 102 regarding the item image GC that was the target of the operation of user Ua and at least one other user Ub when the operation image GW was displayed.

[0072] Furthermore, the program 100 according to the first embodiment causes the processing unit 12 to perform the following actions: acquire operation information JMa regarding the operation of user Ua when an operation image GW containing multiple item images GC is displayed; and, if it detects a state in which user Ua cannot decide which of the multiple item images GC to operate on based on the operation information JMa, determine at least one item image GC from the multiple item images GC that will prompt user Ua to operate on, based on operation history information 102 regarding the item image GC that was the target of the operation of at least one of user Ua and user Ub other than user Ua when the operation image GW was displayed.

[0073] In other words, Server 1 designates any item image GC that at least one of User Ua and User Ub has previously operated on as the item image GC that User Ua will be prompted to operate on. This allows User Ua to understand how to operate on it, thus eliminating any confusion.

[0074] In the first embodiment, item image GC is an example of an "item image", operation image GW is an example of an "image containing multiple item images", user Ua is an example of a "first user", user Ub is an example of a "second user", operation information JMa is an example of "operation information relating to the operation of the first user", operation history information 102 is an example of "operation history information", server 1 is an example of an "information processing device", processing device 12 is an example of a "processing device", and program 100 is an example of a "program".

[0075] Furthermore, in the image determination method according to the first embodiment, the operation history information 102 includes frequency information 103 indicating the frequency of operations on each of a plurality of item image GCs of at least one of user Ua and user Ub, and determining at least one item image GC includes determining at least one first priority item image based on the frequency information 103, the at least one first priority item image is the item image GC that has been the target of operations most frequently by at least one of user Ua and user Ub.

[0076] In other words, Server 1 selects the item image GC with the highest operation frequency from among the item image GCs that at least one of User Ua and User Ub has previously operated on, and designates it as the item image GC to prompt User Ua to operate on. This allows User Ua to understand a reliable method of operation and reliably resolve any confusion.

[0077] In the first embodiment, frequency information 103 is an example of "frequency information". Also, "at least one first priority item image" is an example of item image GC3, item image GC4, item image GC5, and item image GC9.

[0078] Furthermore, in the image determination method according to the first embodiment, determining at least one item image GC includes determining at least one second priority item image based on frequency information 103, wherein the at least one second priority item image is an item image GC that is the second most frequently targeted by an operation of at least one of user Ua and user Ub after the at least one first priority item image.

[0079] In other words, Server 1 can select frequently used item image GCs from among the item image GCs that at least one of User Ua and User Ub has previously operated on, thereby increasing the number of item image GCs that User Ua can be prompted to operate on. As a result, User Ua can understand multiple reliable operating methods, and confusion can be resolved more reliably.

[0080] In the first embodiment, "at least one second priority item image" is, for example, item image GC2.

[0081] Furthermore, the image determination method according to the first embodiment further includes determining at least one item image GC based on designation information 101 that designates a predetermined item image GC from among the multiple item image GCs, if it is not possible to determine at least one item image GC based on frequency information 103 that indicates the frequency of operations on each of the multiple item image GCs of at least one of user Ua and user Ub.

[0082] As a result, even if Server 1 cannot determine at least one item image GC that prompts user Ua to perform an action based on frequency information 103, it can provide highly accurate operation support by, for example, setting an item image GC that is expected to be frequently operated in advance as the item image GC specified in specification information 101.

[0083] In the first embodiment, the designated information 101 is an example of "designated information." Furthermore, "predetermined item images" include, for example, item images GC2, GC3, GC4, GC5, and GC9.

[0084] Furthermore, in the image determination method according to the first embodiment, the display mode of the operation image GW displayed to user Ua and the display mode of the operation image GW displayed to user Ub are different, and determining at least one item image GC includes determining at least one item image GC that prompts user Ua to perform an operation, based on frequency information 103 indicating the frequency of operations for each of the multiple item images GC in the display mode of the operation image GW displayed to user Ua.

[0085] For example, if the operation image GW is a GUI used when operating a specific application software, the version of the application software may be updated, and the display manner of the operation image GW shown on some users' terminals may change. Even in such cases, server 1 determines the item image GC that prompts user Ua to perform an action based on frequency information 103 derived from operation information acquired from each user's terminal. In other words, server 1 becomes able to acquire operation information from a wide range of users, improving the reliability of the frequency indicated by frequency information 103. As a result, user Ua can grasp a reliable method of operation, and confusion can be reliably eliminated compared to the case where operation information is acquired only from terminals that display an operation image GW with a display manner similar to the operation image GW displayed to user Ua.

[0086] 2. Second Embodiment A second embodiment of the present invention will be described below. In each of the embodiments illustrated below, for elements whose operation and function are the same as in the first embodiment, the reference numerals used in the description of the first embodiment will be reused, and detailed descriptions of each will be omitted as appropriate.

[0087] In the second embodiment, a smartphone that detects when a user is having difficulty operating the device and determines the content of a notification to assist the user based on the user's operation history will be used as an example to explain the image determination method, information processing device, and program according to the present invention.

[0088] The configuration and functions of the smartphone 1A according to the second embodiment will be described below with reference to Figures 8 and 9. Figure 8 is a block diagram showing the configuration of the smartphone 1A according to the second embodiment. Figure 9 is a block diagram showing the configuration of the storage device 10A according to the second embodiment.

[0089] Smartphone 1A differs from server 1 according to the first embodiment in that it has a storage device 10A instead of storage device 10, a processing device 12A instead of processing device 12, and a touch panel 16A. Processing device 12A differs from processing device 12 in that it has the function of an operation information acquisition unit 125, an image generation unit 126 instead of an instruction unit 124, and a display control unit 127.

[0090] The storage device 10A differs from the storage device 10 in that it stores program 100A instead of program 100, operation history information 102A instead of operation history information 102, and operation image information 104. The operation history information 102A differs from the operation history information 102 in that it includes frequency information 103A instead of frequency information 103. The operation history information 102A is information about the item image GC that was the target of the user's operation on the smartphone 1A when the operation image GW is displayed. The frequency information 103A is information indicating the frequency of the user's operation on each of the multiple item image GCs on the smartphone 1A. The operation image information 104 is information representing the operation image GW. The operation image information 104 includes first operation image information 105 and second operation image information 106. The first operation image information 105 is information representing the operation image GWa1. The second operation image information 106 represents an image in which an image for assisting the user of the smartphone 1A is superimposed on the operation image GWa1.

[0091] In this embodiment, the item image GC that is most frequently accessed by the user of smartphone 1A may be referred to as the "first priority item image." Furthermore, the item image GC that is second most frequently accessed by the user of smartphone 1A may be referred to as the "second priority item image."

[0092] The processing unit 12A functions as the operation information acquisition unit 125, confusion detection unit 121, decision unit 122, data management unit 123, image generation unit 126, and display control unit 127 shown in Figure 8, when the CPU and other components of the processing unit 12A execute program 100A.

[0093] The touch panel 16A has the same configuration as the touch panels 36a and 36b. The display device of the touch panel 16A displays various images under the control of the display control unit 127. The input device of the touch panel 16A receives input operations from the user of the smartphone 1A and outputs data indicating the detected touch position to the processing unit 12A.

[0094] The display control unit 127 controls the touch panel 16A to display various images. For example, the display control unit 127 displays the operation image GW on the touch panel 16A based on the operation image information 104. Specifically, the display control unit 127 displays the operation image GWa1 on the touch panel 16A based on the first operation image information 105.

[0095] The operation information acquisition unit 125 acquires data related to an operation from the touch panel 16A when the touch panel 16A receives an operation from the user. This data includes information indicating the position on the touch panel 16A where the operation was received. In other words, the operation information acquisition unit 125 acquires operation information from the touch panel 16A. The operation information acquisition unit 125 also adds various information to the acquired operation information, such as information indicating the operation image GW that was displayed on the touch panel 16A when the operation was performed, target information, count information, and time information, etc., related to the operation performed by the user on the smartphone 1A. Therefore, the operation information to which various information has been added includes the same information as operation information JMa and operation information JMb.

[0096] The image generation unit 126 generates a notification image GN based on the notification content determined by the decision unit 122. The notification image GN is an image to assist user actions. Specifically, the notification image GN is an image to notify the user of the notification content determined by the decision unit 122. More specifically, the notification image GN is an image to notify the user of at least one item image GC determined by the decision unit 122 that prompts the user to take action.

[0097] Furthermore, the image generation unit 126 superimposes the notification image GN onto the operation image GW. For example, the image generation unit 126 superimposes the notification image GN onto the operation image GWa1. In other words, the image generation unit 126 edits the operation image GWa1 based on the notification content determined by the determination unit 122. To put it another way, the image generation unit 126 generates second operation image information 106 based on first operation image information 105 representing the operation image GWa1 and the notification content determined by the determination unit 122. That is, the second operation image information 106 is information representing an image in which the notification image GN is superimposed on the operation image GWa1.

[0098] Figure 10 is a flowchart illustrating the operation of smartphone 1A according to the second embodiment. The flowchart shown in Figure 10 is the same as the flowchart shown in Figure 6, except that steps S111 and S112 are executed instead of step S101, step S113 is executed instead of step S103, step S114 is executed instead of step S104, and step S115 is executed.

[0099] In step S111, the display control unit 127 displays the operation image GWa1 on the touch panel 16A based on the first operation image information 105.

[0100] In step S112, the operation information acquisition unit 125 acquires operation information from the touch panel 16A. The operation information acquisition unit 125 also adds various information to the acquired operation information.

[0101] In step S113, the image generation unit 126 edits the operation image GWa1 based on the notification content determined by the decision unit 122. That is, the image generation unit 126 generates a notification image GN based on the notification content determined by the decision unit 122. The image generation unit 126 also superimposes the notification image GN onto the operation image GWa1.

[0102] Figure 11 is a schematic diagram illustrating notification image GN1. Figure 12 is a schematic diagram illustrating notification image GN2. Notification images GN1 and GN2 are examples of notification images GN. Notification images GN1 and GN2 are generated in step S113 if the answer to step S201 is YES, as shown in the flowchart of Figure 7.

[0103] Notification image GN1 is an image used to notify the user of the first priority item image GC, which was determined in step S202 to be the item image GC that the user is prompted to act upon. Notification image GN1 includes instruction image GF1 and instruction image GF2. Instruction image GF1 is an image used to point to item image GC3, item image GC4, and item image GC5. Instruction image GF2 is an image used to point to item image GC9.

[0104] Notification image GN2 is an image used to notify the user of the second priority item image GC, which was determined in step S202 to be the item image GC that prompts the user to take action. Notification image GN2 includes instruction image GF3. Instruction image GF3 is an image used to indicate item image GC2.

[0105] Figure 13 is a schematic diagram illustrating the operation image GWa2. Operation image GWa2 is an image in which notification image GN1 and notification image GN2 are superimposed on operation image GWa1. Operation image GWa2 may be displayed on the touch panel 16A in step S114, which will be described later. As shown in Figure 13, the first priority item image is indicated by instruction images GF1 and GF2. The second priority item image is indicated by instruction image GF3. In other words, the user can understand how to operate by checking the item image GC notified by notification images GN1 and GN2, thereby eliminating confusion.

[0106] The display modes of notification image GN1 and notification image GN2 may differ from each other. For example, the border lines shown as instruction image GF1 and instruction image GF2 may be thicker than the border lines shown as instruction image GF3. In other words, the user can understand the relationship between the frequency of operation of item image GC indicated by notification image GN1 and the frequency of operation of item image GC indicated by notification image GN2 by looking at the difference in the display modes of notification image GN1 and notification image GN2.

[0107] Figure 14 is a schematic diagram illustrating notification image GN3. Notification image GN3 is an example of notification image GN. Notification image GN3 is generated in step S113 if the answer in step S203, as shown in the flowchart of Figure 7, is YES.

[0108] Notification image GN3 is an image used to notify the user of a predetermined item image GC specified in the designation information 101, which was determined in step S204 as the item image GC to prompt the user to take action on. Notification image GN3 includes instruction image GF4, instruction image GF5, and instruction image GF6. Instruction image GF4 is an image used to indicate item image GC2. Instruction image GF5 is an image used to indicate item image GC3, item image GC4, and item image GC5. Instruction image GF6 is an image used to indicate item image GC9.

[0109] Figure 15 is a schematic diagram illustrating the operation image GWa3. Operation image GWa3 is an image in which the notification image GN3 is superimposed on the operation image GWa1. Operation image GWa3 may be displayed on the touch panel 16A in step S114, which will be described later. As shown in Figure 15, the predetermined item image GC specified by the designation information 101 is indicated by instruction images GF4, GF5, and GF6. In other words, the user can understand how to operate the device by checking the item image GC notified by the notification image GN3, thereby eliminating any confusion.

[0110] The instruction images GF4 to GF6 included in the notification image GN3 may each contain a number to indicate the order of operations to the user. For example, instruction image GF4 contains "1" to indicate that it is the target of the first operation. Instruction image GF5 contains "2" to indicate that it is the target of the second operation. Instruction image GF6 contains "3" to indicate that it is the target of the third operation. In other words, the user can understand the order of operations by checking the numbers contained in each of the instruction images GF4 to GF6. As a result, the user can complete the series of operations by pressing the item images GC indicated by each of the instruction images GF4 to GF6 in the order of the numbers contained in each of the instruction images GF4 to GF6.

[0111] In step S114, the display control unit 127 causes the touch panel 16A to display an image on which the notification image GN is superimposed on the operation image GWa1, based on the second operation image information 106. For example, the display control unit 127 causes the touch panel 16A to display operation image GWa2 or operation image GWa3.

[0112] In step S115, the data management unit 123 determines, based on the operation information, whether or not the user has performed an operation termination operation. If the user has performed an operation termination operation, i.e., if the result in step S115 is YES, the processing unit 12A, which includes the data management unit 123, terminates the series of operations shown in the flowchart of Figure 10. If the user has not performed an operation termination operation, i.e., if the result in step S115 is NO, the data management unit 123 proceeds to step S112.

[0113] For example, if the pressed item image GC is a button that instructs smartphone 1A to end the operation, or a button that instructs smartphone 1A to transition from the operation image GW displayed on touch panel 16A to another operation image GW, the processing unit 12 terminates the series of operations shown in the flowchart of Figure 10.

[0114] Based on the above, according to the second embodiment, the smartphone 1A can determine a first priority item image and a second priority item image as item images GC that prompt the user to take action, based on frequency information 103A. Furthermore, the smartphone 1A can display a notification image GN1 for notifying the user of the first priority item image and a notification image GN2 for notifying the user of the second priority item image on the touch panel 16A. In other words, the smartphone 1A can notify the user of the matters determined to support user operation with a simpler configuration compared to the first embodiment.

[0115] As described above, the image determination method according to the second embodiment further includes displaying on the touch panel 16A a notification image GN1 for notifying the user of the smartphone 1A of at least one first priority item image, and a notification image GN2 for notifying the user of the smartphone 1A of at least one second priority item image.

[0116] In other words, the smartphone 1A according to this embodiment visually notifies the user of the item image GC that prompts the user to perform an action. This allows the user to easily understand, for example, which item image GC is the target item image GC and where the target item image GC is located.

[0117] In the second embodiment, the user of the smartphone 1A is an example of the "first user," notification image GN1 is an example of the "first notification image," notification image GN2 is an example of the "second notification image," and touch panel 16A is an example of the "display device." Furthermore, "at least one first priority item image" is an example of item images GC3, GC4, GC5, and GC9. Furthermore, "at least one second priority item image" is an example of item image GC2.

[0118] 3. Variant The above embodiments can be modified in various ways. Specific examples of modifications are given below. Furthermore, two or more embodiments arbitrarily selected from the following examples can be combined as appropriate, within the bounds of mutual consistency. In the modified examples given below, for elements whose operation and function are equivalent to those in the above embodiments, the same reference numerals used in the above description will be reused, and detailed explanations of each will be omitted as appropriate.

[0119] 3.1. Variation 1 In the embodiments described above, an example was given in which the determined notification content is notified using an image, but the present invention is not limited to such embodiments. For example, the determined notification content may be notified using voice. When notification is made using voice, the notification image is not superimposed on the operation image, so even if the area of ​​the image display portion on the display device is small, for example, notification can be made without degrading the visibility of the operation image.

[0120] 3.2. Variation 2 In the embodiments and modifications described above, a server 1 and a smartphone 1A were exemplified as information processing devices according to the present invention, but the present invention is not limited to these embodiments. For example, a personal computer having similar functions to the smartphone 1A may be used.

[0121] When using a personal computer, it is assumed that a pointer for pressing item image GC is displayed on the display device built into the personal computer, or on a display device connected to the personal computer. In such a case, information about the pointer's coordinates may be used to determine whether the user is confused with the operation. Specifically, if the pointer's coordinates repeatedly move back and forth between two specific areas, it may be determined that the user is confused with the operation. In other words, by setting an appropriate determination method according to the device implementing the present invention, it is possible to accurately detect whether the user is confused with the operation.

[0122] 3.3. Modification Example 3 In the embodiments and modifications described above, an example was given in which at least one item image GC to prompt the user for action is determined based on frequency information indicating the frequency of operations on each of the multiple item image GCs performed by one or more users. However, the present invention is not limited to such embodiments. For example, at least one item image GC to prompt the user for action may be determined based on success frequency information indicating the frequency of operations that satisfy predetermined conditions among the operations on each of the multiple item image GCs performed by one or more users. The success frequency information indicates, for example, the frequency with which one or more users press a predetermined item image GC among the multiple item image GCs, or the frequency with which they press a predetermined item image GC in a predetermined order. Furthermore, in step S201 of the operation target determination process, the determination unit 122 may determine whether or not it can determine at least one item image GC to prompt the user for action based on the success frequency information. This makes it possible to limit the item image GCs that may be notified to the user in advance, and enables highly accurate notifications.

[0123] 3.4. Variation 4 In the embodiments and modifications described above, when determining at least one item image GC that prompts the user to take action based on frequency information, the first priority item image and the second priority item image are exemplified as being determined as the item image GC that prompts the user to take action. However, the present invention is not limited to such embodiments. For example, only the first priority item image may be determined as the item image GC that prompts the user to take action. Also, if the item image GC that is the second most frequently interacted with by one or more users is referred to as the "third priority item image," then the first priority item image, the second priority item image, and the third priority item image may all be determined as the item image GC that prompts the user to take action.

[0124] Furthermore, the instruction images that point to the first priority item image, the second priority item image, and the third priority item image may include numbers to indicate the frequency order to the user, similar to the instruction images that point to the predetermined item image GC specified in the designation information 101. In other words, the user can grasp the priority of operations by checking the numbers included in each of the multiple instruction images.

[0125] 3.5. Modified Example 5 In the embodiments and modifications described above, the smartphone 1A is shown as determining at least one item image GC that the user is prompted to act upon based on the operation history information 102A. However, the present invention is not limited to such embodiments. For example, the smartphone 1A may obtain operation history information from a server that collects operation information from multiple users, and determine at least one item image GC that the user is prompted to act upon based on the operation history information. That is, the smartphone 1A may determine at least one item image GC that the user is prompted to act upon based on information regarding item image GCs that have been operated on by multiple users.

[0126] 4. Addendum The following is a summary of the present invention as an addendum.

[0127] 4.1. Appendix 1 A method for determining an image, comprising: acquiring operation information relating to the operation of a first user in a state in which an image containing multiple item images is displayed; and, when a state is detected based on the operation information in which the first user is unable to decide which of the multiple item images to operate on, determining at least one item image from the multiple item images that will prompt the first user to operate on, based on operation history information relating to the item image that was the target of operation by at least one of the first user and a second user different from the first user in a state in which the image containing the multiple item images is displayed.

[0128] In other words, the information processing device that implements the image determination method described in Appendix 1 uses an item image that has been operated by at least one of the first and second users as the item image to prompt the first user to operate. This allows the first user to understand how to operate the item and eliminate any confusion.

[0129] 4.2. Appendix 2 The method for determining an image according to Appendix 1, wherein the operation history information includes frequency information indicating the frequency of operations performed on each of the plurality of item images by at least one of the first user and the second user, and determining the at least one item image includes determining at least one first priority item image based on the frequency information, wherein the at least one first priority item image is the item image that has been the target of operations most frequently by at least one of the first user and the second user.

[0130] In other words, the information processing device that implements the image determination method described in Appendix 2 selects the item image with the highest operation frequency from among the item images that have been operated by at least one of the first user and the second user, and makes it the item image to prompt the first user to operate. As a result, the first user can grasp a reliable operating method and reliably resolve any confusion.

[0131] 4.3. Appendix 3 The method for determining an image according to Appendix 2, wherein determining the at least one item image includes determining at least one second priority item image based on the frequency information, the at least one second priority item image being the next most frequently manipulated item image after the at least one first priority item image, which is the target of operations by at least one of the first user and the second user.

[0132] In other words, the information processing device that implements the image determination method described in Appendix 3 can select frequently used item images from among item images that have been operated by at least one of the first and second users, while increasing the number of item images to which the first user can be prompted to operate. As a result, the first user can grasp multiple reliable operating methods and more reliably resolve any confusion.

[0133] 4.4. Appendix 4 The method for determining an image according to Appendix 3, further comprising displaying on a display device a first notification image for notifying the first user of at least one first priority item image, and a second notification image for notifying the first user of at least one second priority item image.

[0134] In other words, the information processing device that implements the image determination method described in Appendix 4 visually notifies the first user of the item image that prompts the user to perform an action. This allows the first user to easily understand, for example, which item image is the target item image and where the target item image is located.

[0135] 4.5. Appendix 5 The method for determining an image according to any one of the appendices 1 to 4, further comprising determining the at least one item image based on designation information that designates a predetermined item image among the plurality of item images, if it is not possible to determine the at least one item image based on frequency information indicating the frequency of operations performed on each of the plurality of item images by at least one of the first user and the second user.

[0136] As a result, even if the information processing device that implements the image determination method described in Appendix 5 is unable to determine at least one item image that prompts the first user to perform an action based on frequency information, it can, for example, set an item image that is expected to be frequently operated in advance as the item image specified in the specified information to provide highly accurate operation support.

[0137] 4.6. Appendix 6 The display mode of the image including the plurality of item images displayed to the first user and the display mode of the image including the plurality of item images displayed to the second user are different, and determining the at least one item image includes determining the at least one item image that prompts the first user to perform an action on, based on frequency information indicating the frequency of operations on each of the plurality of item images in the display mode of the image including the plurality of item images displayed to the first user. This is the method for determining an image according to any one of the appendices 1 to 5.

[0138] For example, if an image containing multiple item images is a GUI used when operating a specific application software, the display format of the image containing multiple item images displayed on some users' terminals may change when the version of the application software is updated. Even in such cases, the information processing device that implements the image determination method described in Appendix 6 determines the item image to prompt the first user to operate based on frequency information derived from operation information acquired from each user's terminal. In other words, the information processing device that implements the image determination method described in Appendix 6 can acquire operation information from a wide range of users, improving the reliability of the frequency indicated by the frequency information. As a result, the first user can grasp a highly reliable operating method, and confusion can be reliably eliminated compared to the case where operation information is acquired only from terminals that display an image containing multiple item images in the same display format as the image containing multiple item images displayed to the first user.

[0139] 4.7. Appendix 7 An information processing device comprising: an information processing device that performs the following actions: acquiring operation information relating to an operation of a first user in a state in which an image including a plurality of item images is displayed; and, when the information processing device detects a state in which the first user is unable to decide which of the plurality of item images to operate on, it determines, based on operation history information relating to an item image that was the target of an operation by at least one of the plurality of item images, either the first user or a second user different from the first user, in a state in which the image including the plurality of item images is displayed, at least one item image among the plurality of item images that should prompt the first user to operate on.

[0140] In other words, the information processing device described in Appendix 7 uses an item image that has been operated by at least one of the first and second users as the item image that prompts the first user to operate. This allows the first user to understand how to operate the item and eliminates any confusion.

[0141] 4.8. Appendix 8 A program that causes a processing unit to perform the following actions: acquire operation information relating to the actions of a first user when an image containing multiple item images is displayed; and, when the program detects a state in which the first user is unable to decide which of the multiple item images to perform an action on, based on the operation information, determine at least one item image from the multiple item images that the first user should be prompted to act on, based on operation history information relating to the item images that were the target of operations by at least one of the first user and a second user different from the first user when the image containing the multiple item images was displayed.

[0142] In other words, the information processing device that executes the program described in Appendix 8 will use an item image that has been operated by at least one of the first and second users as the item image that prompts the first user to operate. This allows the first user to understand how to operate the item and eliminate any confusion. [Explanation of Symbols]

[0143] 1...Server, 1A...Smartphone, 3a...Smartphone, 10...Storage device, 12...Processing device, 14...Communication device, 16A...Touch panel, 36a...Touch panel, 100...Program, 101...Specified information, 102...Operation history information, 103...Frequency information, 104...Operation image information, 105...First operation image information, 106...Second operation image information, 120...Communication control unit, 121...Confusion detection unit, 122...Decision unit, 123...Data management unit, 125...Operation information acquisition unit, 126...Image generation unit, 127...Display control unit, GW...Operation image, GC...Item image, GN1...Notification image, GF1...Instruction image, JMa...Operation information, JDa...Instruction information, NW...Network, Ua...User, Sys...Notification system.

Claims

1. To obtain operation information regarding the actions of the first user when an image containing multiple item images is displayed, If, based on the operation information, the system detects that the first user is unable to determine which of the multiple item images to operate on, it determines, based on the operation history information relating to the item image that was operated on by a second user (different from the first user) in the state in which the image containing the multiple item images is displayed, at least one item image among the multiple item images that the system will prompt the first user to operate on. Includes, The aforementioned operation information includes information indicating an image containing the multiple item images that were displayed when the operation was performed, Determining the at least one item image is performed based on the operation history information of the second user corresponding to the information included in the operation information. How to determine the image.

2. The operation history information includes frequency information indicating the frequency of operations performed on each of the multiple item images by a second user, who is different from the first user. Determining the at least one item image includes determining at least one first priority item image based on the frequency information. The at least one first priority item image is the item image that is most frequently manipulated by a second user different from the first user. The method for determining an image according to claim 1.

3. Determining the at least one item image includes determining at least one second priority item image based on the frequency information. The at least one second priority item image is an item image that is second only to the at least one first priority item image in frequency of being operated by a second user different from the first user. The method for determining an image according to claim 2.

4. The method further includes displaying on a display device a first notification image for notifying the first user of at least one first priority item image, and a second notification image for notifying the first user of at least one second priority item image. The method for determining an image according to claim 3.

5. If it is not possible to determine the at least one item image based on frequency information indicating the frequency of operations on each of the plurality of item images by a second user different from the first user, the at least one item image is further determined based on designation information that designates a predetermined item image from the plurality of item images. The method for determining an image according to claim 1.

6. The display mode of the image including the multiple item images shown to the first user and the display mode of the image including the multiple item images shown to the second user are different. Determining the at least one item image includes determining the at least one item image that the first user is prompted to act upon, based on frequency information indicating the frequency of operations on each of the multiple item images in a display mode of an image including the multiple item images displayed to the first user. A method for determining an image according to any one of claims 1 to 5.

7. Equipped with a processing device, The aforementioned processing apparatus is To obtain operation information regarding the actions of the first user when an image containing multiple item images is displayed, If, based on the operation information, the system detects that the first user is unable to determine which of the multiple item images to operate on, it determines, based on the operation history information relating to the item image that was operated on by a second user (different from the first user) in the state in which the image containing the multiple item images is displayed, at least one item image among the multiple item images that the system will prompt the first user to operate on. Based on the operation history information corresponding to the information in the operation information that shows an image including the multiple item images that were displayed when the operation was performed, the at least one item image is determined. Execute Information processing device.

8. In the processing unit, To obtain operation information regarding the actions of the first user when an image containing multiple item images is displayed, If, based on the operation information, the system detects that the first user is unable to determine which of the multiple item images to operate on, it determines, based on the operation history information relating to the item image that was operated on by a second user (different from the first user) in the state in which the image containing the multiple item images is displayed, at least one item image among the multiple item images that the system will prompt the first user to operate on. Based on the operation history information corresponding to the information in the operation information that shows an image including the multiple item images that were displayed when the operation was performed, the at least one item image is determined. To execute program.

Images