Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device capable of accurately predicting operation contents of a user and assisting the operation of the user, an information processing method, and an information processing program.SOLUTION: An information processing device comprises: a display control unit which controls the display of a user terminal including processing for displaying a plurality of target contents to be operated on the user terminal; an acquisition unit which acquires operation information indicative of an operation for a target content of the user; a prediction unit which predicts the operation contents of the user for the target content on the basis of the operation information; and a generation unit which generates proposal information assisting the operation of the user on the basis of the operation information and the operation contents of the user predicted by the prediction unit. The display control unit displays the proposal information generated by the generation unit on the user terminal.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program. [Background Art]

[0002] With the advent of wearable devices such as smartwatches and smart glasses, inconveniences such as selecting incorrect content have sometimes occurred when operating content in a GUI (Graphical User Interface) that relies on image display to mediate interaction between a user and these devices.

[0003] For example, Patent Document 1 below discloses an operation display device that can easily select one object when a plurality of objects are displayed in a touch area for a touch operation. When there are a plurality of objects in the touch area, the operation display device described in Patent Document 1 associates each object with a pressure level, and selects an object that is associated with the pressure level corresponding to the pressing pressure detected by the pressure detection unit. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2018-197898 [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] However, the operation display device described in Patent Document 1 requires the provision of a pressure sensor that detects the pressing pressure by detecting a change in capacitance of a piezoelectric ceramic element when pressure is applied to the piezoelectric ceramic element such as lead zirconate titanate (PZT: Pb(Zr,Ti)O₃), and thus cannot be applied to general devices such as smartwatches and smartphones.

[0006] In light of the above issues, this disclosure aims to provide an information processing device, an information processing method, and an information processing program that can accurately predict the user's actions and assist the user's actions. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objectives, the information processing device according to this disclosure comprises: a display control unit that performs a process of controlling the display of a user terminal, including a process of displaying a plurality of target contents that are the subject of operation processing on the user terminal; an acquisition unit that acquires operation information indicating the user's operation on the target contents; a prediction unit that predicts the user's operation content on the target contents based on the operation information; and a generation unit that generates suggestion information to assist the user's operation based on the operation information and the user's operation content predicted by the prediction unit, wherein the display control unit displays the suggestion information generated by the generation unit on the user terminal. [Effects of the Invention]

[0008] According to this disclosure, it is possible to provide an information processing device, an information processing method, and an information processing program that can accurately predict the user's actions and assist the user's actions. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of information processing according to the embodiment. [Figure 2] Figure 2 shows an example of proposed information generated by the generation unit of the information processing apparatus according to the embodiment. [Figure 3] Figure 3 shows an example of the configuration of an information processing system according to the embodiment. [Figure 4] Figure 4 shows an example of the configuration of an information processing device according to the present invention. [Figure 5] Figure 5 shows an example of information stored in the operation information storage unit of the information processing apparatus according to the embodiment. [Figure 6]Figure 6 shows an example of information stored in the content storage unit of the information processing device according to this embodiment. [Figure 7] Figure 7 shows an example of the configuration of a user terminal according to this embodiment. [Figure 8] Figure 8 is a flowchart showing an example of information processing according to the embodiment. [Figure 9] Figure 9 is a hardware configuration diagram showing an example of a computer that implements the functions of an information processing device. [Modes for carrying out the invention]

[0010] The following describes in detail, with reference to the drawings, the embodiments for implementing the information processing device, information processing method, and information processing program relating to this application (hereinafter referred to as "embodiments"). However, these embodiments do not limit the information processing device, information processing method, and information processing program relating to this application.

[0011] (Embodiment) [1-1. An example of information processing according to the embodiment] First, an example of information processing according to the embodiment will be described using Figure 1. Figure 1 is a diagram showing an example of information processing according to the embodiment. In Figure 1, an example is shown in which the information processing according to the embodiment is executed by the information processing device 100 and the user terminal 200. Hereinafter, the information processing according to the embodiment will be described step by step.

[0012] First, the information processing device 100 displays multiple target content items to be operated on the user terminal 200 (step S1). For example, the information processing device 100 displays multiple target content items to be operated on the user terminal 200, including e-book pages, text boxes, video scroll bars, browser web page scroll bars and links, and CAD (Computer-Aided Design) objects, and receives operation information from the user. The state in which multiple target content items to be operated are displayed refers to, for example, a state in which multiple scroll bars that accept swipe operations are displayed, a state in which scroll bars and page screens that accept swipe operations are displayed on top of each other, a state in which multiple links to transition to other pages are displayed side by side, or a state in which a keyboard screen for text input is displayed with multiple input keys lined up. For example, multiple target content items as shown in display screen G1 in Figure 1 may be displayed.

[0013] Next, the information processing device 100 acquires user operation information from the user terminal 200 (step S2). Here, operation information refers to information indicating the user's operation on the user terminal 200, and may be, for example, information on touch operations on a touch panel or information on the gaze point of an eye-tracking device. For example, the information processing device 100 acquires user U's operation information each time user U performs an operation on the user terminal 200.

[0014] Next, the information processing device 100 predicts the user's actions toward the target content based on the operation information (step S3). For example, the information processing device 100 may use a model that models the relationship between the events indicated by the operation information as prior events and the subsequent events indicating the user's actions using a Bayesian network, and predict the event with the highest probability of being a subsequent event predicted from the prior events as the user's actions.

[0015] Next, the information processing apparatus 100 generates proposal information for assisting the user's operation based on the operation information and the predicted content of the user's operation (step S4). For example, the information processing apparatus 100 may determine the user's operation status from the operation information, and generate proposal information for assisting the user's operation based on the determined user's operation status and the predicted content of the user's operation. In other words, the information processing apparatus 100 generates proposal information for assisting the user's operation in accordance with the user's operation status and the prediction of the user's operation content. Here, the user's operation status refers to the status of a plurality of target contents when the user performs an operation, and means, for example, a state where a plurality of icons are displayed densely or a state where a plurality of scroll bars are displayed.

[0016] Next, the information processing apparatus 100 causes the user terminal 200 to display the generated proposal information (step S5). For example, the information processing apparatus 100 may cause the user terminal 200 to display the proposal information generated in step S4 by a mode of displaying the proposal information overlapping the plurality of target contents displayed in step S1. It is preferable that the information processing apparatus 100 displays the proposal information at a position close to the touch position or the gaze point indicated by the user's operation information, which is considered to be where the user's attention is concentrated. For example, proposal information P1 as shown in the display screen G2 illustrated in FIG. 1 may be displayed.

[0017] Accordingly, the information processing apparatus 100 can predict the user's operation content based on the operation information, and generate proposal information that assists the user's operation in accordance with the predicted operation content. Therefore, it is possible to provide the information processing apparatus 100 capable of accurately predicting the user's operation content and assisting the user's operation.

[0018] [1-2. Another Example 1 of Information Processing According to Embodiment] When the operation information indicates the position of a boundary between a plurality of target contents as an operation point, the information processing apparatus 100 generates proposal information including the target content near the operation point.

[0019] This information processing will be explained step by step. First, the information processing device 100 performs the same processing as steps S1 to S3 shown in Figure 1. Since the processing from steps S1 to S3 is the same as the processing described above, the explanation will be omitted.

[0020] Next, when the operation information indicates the location of the boundary between multiple target content as the operation point, the information processing device 100 generates suggestion information that includes target content near the operation point (step S4-1). For example, suppose user U performs an operation on the user terminal 200, on the display screen G1 where multiple icons displayed in step S1 are shown, and the operation point is the location of the boundary of icon I1, which is shown as Target in Figure 1, meaning the target of the user's operation. In this case, the information processing device 100 generates suggestion information P1 shown on the display screen G2 in S5 of Figure 1. The suggestion information P1 shown in Figure 1 is information that includes a program that executes the process of executing the program associated with each icon for icons I2 and I3, which are icons near icon I1. Note that "near the operation point" means the target content included in the top three positions when the target content is arranged in order of closestness to the operation point or point of focus indicated by the operation information. By displaying such suggestion information on the user terminal 200, user U can accurately operate the target content they want to operate, even when multiple target content is displayed densely.

[0021] Next, the information processing device 100 performs the same process as in step S5 shown in Figure 1. Since the process in step S5 is the same as the process described above, its explanation will be omitted.

[0022] This allows the system to display suggestion information to assist users, even when multiple target content items are displayed in close proximity, preventing them from selecting the wrong content. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's actions and assist them in their operation.

[0023] [1-3. Another example of information processing according to the embodiment 2] The information processing device 100, based on the target content selected based on the operation information, transitions the display mode of the target content, and then, when the operation information indicates the position of the boundary between multiple target content as the operation point, generates suggestion information that includes target content near the operation point.

[0024] This information processing will be explained step by step. First, the information processing device 100 performs the same processing as steps S1 to S3 shown in Figure 1. Since the processing from steps S1 to S3 is the same as the processing described above, the explanation will be omitted.

[0025] Next, the information processing device 100 transitions the display mode of the target content based on the target content selected based on the operation information, and then, if the operation information indicates the position of the boundary of multiple target content as the operation point, it generates suggestion information that includes target content near the operation point (step S4-2). For example, suppose user U performs an operation to select icon I1 on the display screen G1 on the user terminal 200 where multiple icons displayed in step S1 are shown. In this case, the information processing device 100 performs a process to execute the program associated with icon I1, for example, transitioning the display mode of the multiple target content.

[0026] Here, the suggestion information generated by the information processing device 100 in such a case will be explained using Figure 2. Figure 2 is a diagram showing an example of suggestion information generated by the generation unit of the information processing device according to the embodiment. Figure 2 shows that the display mode has transitioned from display screen G1 to display screen G3. The suggestion information P2 displayed on display screen G3 in Figure 2 is information that includes a program that executes a process to execute the program associated with each icon for icons I2 and I3, which are icons near icon I1. Note that "near the operation point" means the target content included in the top three positions when the target content is arranged in order of closestness to the operation point or gaze point indicated by the operation information.

[0027] By displaying such suggested information on the user terminal 200 after transitioning through the display modes of multiple target content, user U can operate on the target content they intended, even if they have operated on the wrong target content.

[0028] Next, the information processing device 100 performs the same process as in step S5 shown in Figure 1. Since the process in step S5 is the same as the process described above, its explanation will be omitted.

[0029] This allows the system to display suggestion information to assist the user even when multiple target content items are displayed in close proximity, enabling the user to operate on the intended target content. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's actions and assist the user's operations.

[0030] [1-4. Other Examples of Information Processing According to the Embodiment 3] The information processing device 100 executes display control to hide the suggested information displayed on the user terminal 200 after the user has selected the target content.

[0031] This information processing will be explained step by step. First, the information processing device 100 performs the same processing as steps S1 to S5 shown in Figure 1. Since the processing from steps S1 to S3 is the same as the processing described above, its explanation will be omitted.

[0032] Next, after the user selects the target content, the information processing device 100 executes display control to hide the suggested information displayed on the user terminal 200 (step S6). For example, suppose that with the display screen G1 shown in Figure 1 displayed, user U performs an operation to indicate the position of the boundary between icons I1, I2, and I3 as an operation point, and the information processing device 100 displays the suggested information P1 shown on the display screen G2 in step S3. Then, suppose user U checks the display screen G2 displayed on the user terminal 200 and performs an operation to select icon I1. In this case, the information processing device 100 executes display control to hide the suggested information P1 on the display screen G2 displayed on the user terminal 200.

[0033] This allows the information processing device 100 to accurately predict the user's actions and assist them, and then dismiss the displayed suggestion information once the user has performed their intended action.

[0034] [2. Configuration of the Information Processing System] Next, the configuration of the information processing system according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example configuration of the information processing system according to the embodiment. As shown in Figure 3, the information processing system 1 includes an information processing device 100, a user terminal 200, and a network N. Note that the information processing system 1 shown in Figure 3 may be configured to include multiple user terminals 200.

[0035] The information processing device 100 may be, for example, a personal computer (PC), a workstation (WS), or a computer equipped with server functions. The information processing device 100 processes information transmitted from the user terminal 200 via the network N.

[0036] The user terminal 200 is an information processing device used by the user. The user terminal 200 may be, for example, a smartphone, smartwatch, smartphone glasses, tablet device, desktop PC, notebook PC, mobile phone, PDA (Personal Digital Assistant), or other information processing device. In the example shown in Figure 1, the user terminal 200 is a smartphone.

[0037] Network N connects the information processing device 100 and the user terminal 200 so that they can communicate with each other via wired or wireless connection. If Network N is wired, it may be implemented using Ethernet (registered trademark) as defined in IEEE 802.3. If Network N is wireless, it may be implemented using a wireless LAN (Local Area Network) as defined in IEEE 802.11.

[0038] [3. Configuration of the Information Processing Device] Next, the configuration of the information processing device 100 according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example of the configuration of the information processing device according to the embodiment. As shown in Figure 3, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in Figure 3, the information processing device 100 may also have an input unit (for example, a keyboard or mouse) that receives various operations from the administrator of the information processing device 100, and a display unit (for example, a liquid crystal display) for displaying various information.

[0039] (Regarding Communications Unit 110) The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wireless connection and transmits and receives information with the user terminal 200.

[0040] (Regarding memory unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks, SSDs (Solid State Drives), and optical discs. As shown in Figure 3, the storage unit 120 has an operation information storage unit 121 and a content storage unit 122.

[0041] (Regarding the operation information storage unit 121) The operation information storage unit 121 stores user operation information obtained from the user terminal 200. Here, an example of the information stored in the operation information storage unit 121 will be explained using Figure 5. Figure 5 is a diagram showing an example of the information stored in the operation information storage unit of the information processing device according to this embodiment.

[0042] In the example shown in Figure 5, the operation information storage unit 121 stores information related to the items "User ID" and "Operation Information" in association with each other.

[0043] The "User ID" is an identifier that identifies a user and is represented by a string of characters or a number. The "Operation Information" is the user's operation information linked to the "User ID". When operation information is received via a touch panel, the operation information may include the touch duration for each finger of the user, touch movement speed (X axis, Y axis), touchdown position (X axis, Y axis), touchup position (X axis, Y axis), contact area, and the difference between the touchdown and touchup positions. Furthermore, when operation information is received via an eye-tracking device, the operation information may include the gaze direction of the left and right eyeballs, and the position of the gaze points of the left and right eyeballs.

[0044] In other words, Figure 5 shows that the user operation information identified by the user ID "UID#1" is operation information "operation information #1".

[0045] Furthermore, the information stored in the operation information storage unit 121 is not limited to information related to the items "User ID" and "Operation Information," but may also store any other information related to the user's operations.

[0046] (Regarding the content storage unit 122) The content storage unit 122 stores multiple pieces of content to be displayed on the user terminal 200. Figure 6 is a diagram showing an example of information stored in the content storage unit of the information processing device according to this embodiment.

[0047] In the example shown in Figure 6, the content storage unit 122 stores information related to the items "Content ID" and "Content Data" in an associated manner.

[0048] The "Content ID" is an identifier that identifies the content and is represented by a string of characters, a number, or other means. The "Content Data" is the data of the content to be displayed on the user terminal 200.

[0049] In other words, Figure 6 shows that the content data indicated by content data "CD#1" is stored as content identified by content ID "CT#1".

[0050] Furthermore, the content storage unit 122 is not limited to information relating to the items "Content ID" and "Content Data," but may store any other information related to the content.

[0051] (Regarding the control unit 130) Next, returning to Figure 3, the control unit 130 will be described. The control unit 130 is implemented by a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc., which executes various programs stored in the memory of the information processing device 100 using RAM as the working area. Alternatively, the control unit 130 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0052] As shown in Figure 3, the control unit 130 includes an acquisition unit 131, a prediction unit 132, a determination unit 133, a generation unit 134, and a display control unit 135.

[0053] (Regarding acquisition unit 131) The acquisition unit 131 acquires various types of information from inside or outside the information processing device 100.

[0054] For example, the acquisition unit 131 acquires operation information indicating the user's actions on the target content. If the user terminal 200 accepts user operations via a touch panel, the acquisition unit 131 acquires operation information including the touch duration, touch movement speed (X axis, Y axis), touch-down position (X axis, Y axis), touch-up position (X axis, Y axis), contact area, and the difference between the touch-down and touch-up positions. Furthermore, if the user terminal 200 accepts user operations via an eye-tracking device, the acquisition unit 131 acquires operation information including the gaze direction of the user's left and right eyes, and the positions of the points of fixation of the left and right eyes.

[0055] (Regarding prediction unit 132) The prediction unit 132 predicts the user's actions toward the target content based on the operation information. The prediction unit 132 may predict the user's actions by predicting the posterior probability of the user's actions toward the target content based on the events indicated by the operation information. That is, the prediction unit 132 may use a Bayesian network based on Bayes' theorem to represent the interdependence of the selection probabilities of multiple target content and predict the user's actions based on the operation information. A Bayesian network is a model that determines the strength of causal relationships between events from the magnitude of conditional probabilities, which are the probability that other events occur when one event occurs, and organizes the causal relationships between many events using a directed graph. If the target content is an icon, the position of the operation point indicated by the user's operation information may be the prior event, and the probability that the user selects a specific icon from among multiple icons may be the posterior probability. That is, it may be predicted that the user will select the target content with the highest posterior probability among multiple target content.

[0056] (Regarding the determination unit 133) The determination unit 133 determines the user's operation status. Specifically, the determination unit 133 determines the user's operation status from the operation information. Here, operation status refers to the state of the target content when the user is performing an operation on the target content. For example, suppose that multiple icons are displayed side by side as multiple target content on the user terminal 200, and the position of the operation point indicated by the user's operation information indicates the position of the boundary between the multiple icons. In this case, the determination unit 133 determines that the user's operation status is one in which, in a state where multiple target content is displayed densely, the user is operating at the boundary between the multiple target content as the operation point.

[0057] (Regarding the generation unit 134) The generation unit 134 generates suggestion information to assist the user's operations based on the operation information and the user's operations predicted by the prediction unit 132. In other words, the generation unit 134 generates suggestion information to assist the user's operations according to the operation information, the user's operation status, and the prediction of the user's operations. The following describes specific examples of the suggestion information generated by the generation unit 134.

[0058] The generation unit 134 generates suggestion information that includes target content near the operation point when the operation information indicates the position of the boundary between multiple target content. For example, suppose user U performs an operation on the display screen G1 shown in Figure 1, which displays multiple icons on the user terminal 200, and the operation point is the position of the boundary of icon I1, which is shown as Target in Figure 1, meaning the target of the user's operation. In this case, the generation unit 134 generates suggestion information P1 shown in the display screen G2 shown in Figure 1. The suggestion information P1 shown in Figure 1 is information that includes a program that executes the process of executing the program associated with each icon for icons I2 and I3, which are icons near icon I1.

[0059] The generation unit 134 generates suggestion information that includes target content near the operation point when the operation information indicates the location of the boundary between multiple target content items as an operation point, after the display control unit 135 has transitioned pages based on the target content selected based on the operation information. For example, suppose user U performs an operation to select icon I1 on display screen G1, which displays multiple icons on user terminal 200. Then, suppose the process of executing the program associated with icon I1 is performed, and for example, the display mode of multiple target content items is transitioned. In this case, the generation unit 134 generates suggestion information such as suggestion information P2 shown on display screen G3 in Figure 2. The suggestion information P2 shown in Figure 2 is information that includes a program that executes the process of executing the program associated with each icon for icons I2 and I3, which are icons near icon I1. Note that "near the operation point" means the target content included in the top three positions when the target content is arranged in order of closestness to the operation point or point of focus indicated by the operation information.

[0060] (Regarding the display control unit 135) The display control unit 135 controls the display on the output unit 240 of the user terminal 200.

[0061] For example, the display control unit 135 displays multiple target content items on the user terminal 200 that are the subject of the operation process. The display control unit 135 displays multiple content items, such as ebook pages, text boxes, video scroll bars, browser web page scroll bars and links, and CAD objects. However, the content displayed by the display control unit 135 is not limited to these, and may also include content utilizing VR (Virtual Reality) or MR (Mixed Reality) technologies.

[0062] Furthermore, the display control unit 135 transitions the display modes of multiple target content when the user selects one of the multiple target content. For example, the display control unit 135 may transition the display modes of multiple target content by executing a program associated with the target content selected by the user.

[0063] Furthermore, the display control unit 135 executes display control to hide the suggested information displayed on the user terminal 200 after the user has selected the target content. For example, suppose that with the display screen G1 shown in Figure 1 displayed, user U performs an operation to indicate the boundary positions of icons I1, I2, and I3 as operation points, and displays the suggested information P1 shown on display screen G2. Then, suppose user U checks the display screen G2 displayed on the user terminal 200 and performs an operation to select icon I1. In this case, the display control unit 135 executes display control to hide the suggested information P1 on the display screen G2 displayed on the user terminal 200.

[0064] [4. User terminal configuration] Next, the configuration of the user terminal 200 according to the embodiment will be described using Figure 7. Figure 7 is a diagram showing an example of the configuration of a user terminal according to the embodiment. As shown in Figure 7, the user terminal 200 has a communication unit 210, a storage unit 220, an input unit 230, an output unit 240, and a control unit 250.

[0065] The communication unit 210 is implemented, for example, by a NIC (Network Interface Card). The communication unit 210 is connected to the network N by wire or wireless connection and transmits and receives various types of information to and from the information processing device 100 via the network N.

[0066] The memory unit 220 is a storage device that stores various types of information. The memory unit 220 comprises a main memory and an auxiliary storage device. The main memory may be implemented using semiconductor memory elements such as RAM, ROM, or flash memory. The auxiliary storage device may be implemented using a hard disk, SSD, or optical disc, for example.

[0067] The input unit 230 receives various operation information from the user. For example, the input unit 230 accepts various operations from the user via a touch panel on a display surface (e.g., the output unit 240). Alternatively, the input unit 230 may be an eye-tracking device. An eye-tracking device analyzes the user's eye movements based on image information of the user's eyeballs captured by a stereo camera, determines the direction of the user's gaze, and accepts the user's point of fixation as operation information.

[0068] The output unit 240 is a display screen for a tablet terminal, for example, which is implemented using a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. In other words, if the input unit 230 of the user terminal 200 is a touch panel, the output unit 240, which is the display screen, accepts user input and also outputs to the user. The output unit 240 may also be a speaker and output sound through the speaker.

[0069] The control unit 250 is implemented, for example, by a CPU or MPU executing various programs stored in the user terminal 200 using RAM as a working area. Alternatively, the control unit 250 may be implemented by an integrated circuit such as an ASIC or FPGA.

[0070] As shown in Figure 7, the control unit 250 includes a reception unit 251 and a display control unit 252. These components will be described in detail below.

[0071] The reception unit 251 receives operation information for target content from the user. Alternatively, for example, the reception unit 251 can display multiple target content items on the output unit 240 and receive operation information for multiple target content items from the user. If the output unit 240 is a touch panel, the reception unit 251 may receive operation information for target content from the user by sensing information such as the center point coordinates of the touch coordinates for each of the user's fingers, movement speed, movement direction, and contact area. If the input unit 230 is an eye-tracking device, the reception unit 251 may also receive information such as the gaze direction and fixation point of the user's left and right eyes as operation information.

[0072] The display control unit 252 controls the information displayed on the output unit 240. For example, the display control unit 252 controls the output unit 240 to display multiple target contents according to the control commands of the display control unit 135 of the information processing device 100. The display control unit 252 also controls the output unit 240 to display suggestion information generated by the generation unit 134 of the information processing device 100. In this case, the display control unit 252 may display the suggestion information overlaid on the original display.

[0073] [5. Information Processing Flow] Next, the information processing procedure by the information processing device 100 according to the embodiment will be described using Figure 8. Figure 8 is a flowchart of an example of information processing according to the embodiment. The information processing procedure according to the embodiment will be described below in accordance with the flowchart shown in Figure 8.

[0074] For example, the information processing device 100 displays multiple target content items that are the subject of the operation process on the user terminal 200 (step S101). The information processing device 100 then acquires operation information indicating the user's operation on the target content (step S102). The information processing device 100 then predicts the user's operation on the target content based on the operation information (step S103). The information processing device 100 then generates suggestion information to assist the user's operation based on the operation information and the predicted user operation (step S104). The information processing device 100 then displays the generated suggestion information on the user terminal 200 (step S105).

[0075] According to this, it is possible to predict the user's actions based on operation information and generate suggestion information to assist the user's actions according to the predicted actions. Therefore, it is possible to provide an information processing method and an information processing program that can accurately predict the user's actions and assist the user's actions.

[0076] [6. Hardware Configuration] Furthermore, the information processing device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration such as that shown in Figure 9. Figure 9 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090.

[0077] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, as well as programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as RAM, that temporarily stores data used by the arithmetic unit 1030 for various calculations. The secondary storage device 1050 is a storage device that stores data used by the arithmetic unit 1030 for various calculations and various databases, and is implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc.

[0078] Output IF1060 is an interface for transmitting information to be output to output devices 1010, which output various types of information such as monitors and printers. It is implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), and HDMI (High Definition Multimedia Interface). Input IF1070 is an interface for receiving information from various input devices 1020, such as mice, keyboards, and scanners. It is implemented using, for example, USB.

[0079] The input device 1020 may also be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), tape media, magnetic recording media, or semiconductor memory. Furthermore, the input device 1020 may also be an external storage medium such as a USB memory stick.

[0080] Network IF1080 receives data from other devices via network N and sends it to the arithmetic unit 1030, and also transmits data generated by the arithmetic unit 1030 to other devices via network N.

[0081] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0082] For example, when a computer 1000 functions as an information processing device 100, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040, thereby realizing the functions of the control unit 130 of the information processing device 100.

[0083] [7. Structure and Effects] The information processing device 100 according to this disclosure includes a display control unit 135 that performs processing to control the display of a user terminal 200, including processing to display a plurality of target contents that are the subject of operation processing on the user terminal 200; an acquisition unit 131 that acquires operation information indicating the user's operation on the target contents; a prediction unit 132 that predicts the user's operation content on the target contents based on the operation information; and a generation unit 134 that generates suggestion information to assist the user's operation based on the operation information and the user's operation content predicted by the prediction unit 132. The display control unit 135 causes the suggestion information generated by the generation unit 134 to be displayed on the user terminal 200.

[0084] This configuration allows the system to predict the user's actions based on operation information and generate suggestion information to assist the user's actions according to the predicted actions. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's actions and assist the user's actions.

[0085] The generation unit 134 of the information processing device 100 according to this disclosure generates proposed information including target content near the operation point when the operation information indicates the position of the boundary between multiple target content as the operation point.

[0086] This configuration allows for the display of suggestion information to assist the user, even when multiple target content items are displayed in close proximity, preventing the user from selecting the wrong content. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's actions and assist them.

[0087] The generation unit 134 of the information processing device 100 according to this disclosure generates proposed information including target content near the operation point when the operation information indicates the position of the boundary between multiple target content items as an operation point, after the display control unit 135 has transitioned the display mode of the target content based on the target content selected based on the operation information.

[0088] With this configuration, even if a user selects the wrong content when multiple target content items are displayed in close proximity, the system can display suggestion information to assist the user, allowing them to operate the content they intended. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's actions and assist them.

[0089] The display control unit 135 of the information processing device 100 relating to this disclosure executes display control to erase the display of suggested information on the user terminal 200 after the user has selected target content.

[0090] With this configuration, when suggestion information to assist the user's operation is displayed, the displayed suggestion information can be dismissed when the user operates on the intended target content. Therefore, it is possible to provide an information processing device 100 that can accurately predict the user's operation and assist the user's operation.

[0091] Furthermore, the information processing method relating to this disclosure includes the steps of: displaying a plurality of target contents that are the subject of operation processing on the user terminal 200; acquiring operation information indicating the user's operation on the target contents; predicting the user's operation on the target contents based on the operation information; generating suggestion information to assist the user's operation based on the operation information and the predicted user operation; and displaying the generated suggestion information on the user terminal 200.

[0092] This configuration allows for the prediction of user actions based on operational information, and the generation of suggestion information to assist the user's actions according to the predicted actions. Therefore, it is possible to provide an information processing method that can accurately predict user actions and assist the user's actions.

[0093] Furthermore, the information processing program relating to this disclosure causes a computer to execute the following steps: displaying multiple target contents that are the subject of operation processing on the user terminal 200; acquiring operation information indicating operations on the target contents using the user's gaze information; predicting the user's operations on the target contents based on the operation information; generating suggestion information to assist the user's operations based on the operation information and the predicted user operations; and displaying the generated suggestion information on the user terminal 200.

[0094] This configuration allows for the prediction of user actions based on operational information, and the generation of suggestion information to assist the user's actions according to the predicted actions. Therefore, it is possible to provide an information processing program that can accurately predict user actions and assist the user's actions.

[0095] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms, including those described in the disclosure section of the invention, based on the knowledge of those skilled in the art.

[0096] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit." For example, the acquisition unit 131 can be replaced with acquisition means or acquisition circuit. [Explanation of Symbols]

[0097] 1. Information Processing System 100 Information Processing Devices 110 Communications Department 120 Storage section 121 Operation information storage unit 122 Content Storage Unit 130 Control Unit 131 Acquisition Department 132 Prediction Section 133 Judgment section 134 Generation part 135 Display Control Unit 200 user terminals 210 Communications Department 220 Storage section 230 Input section 240 Output section 250 Control Unit 251 Reception Department 252 Display Control Unit N Network

Claims

1. A display control unit that performs processing to control the display of a user terminal, including processing to display multiple target content items that are the target of operation processing on the user terminal, An acquisition unit that acquires operation information indicating the user's actions on the target content, Based on the events indicated by the aforementioned operation information, a prediction unit predicts the posterior probability of the user's actions toward the target content and predicts the user's actions accordingly. The system includes the aforementioned operation information and a generation unit that generates suggestion information to assist the user's operation based on the user's operation content predicted by the prediction unit, The display control unit causes the proposed information generated by the generation unit to be displayed on the user terminal. Information processing device.

2. The generation unit generates suggestion information that includes target content near the operation point when the operation information indicates the location of the boundary between multiple target content. The information processing apparatus according to claim 1.

3. The generation unit generates suggested information including the multiple target content near the operation point when, after the display control unit has transitioned the display mode of the target content based on the target content selected based on the operation information, the newly acquired operation information indicates the position of the boundary between multiple target content as the operation point. The information processing apparatus according to claim 1 or claim 2.

4. The display control unit executes a display control to turn off the display of the suggested information on the user's terminal after the user has selected the target content. The information processing apparatus according to claim 3.

5. A method of information processing performed by a computer, The steps include displaying multiple target content items that are the subject of the operation process on the user's terminal, A step to obtain operation information indicating the user's actions on the target content, Based on the events indicated by the aforementioned operation information, the steps include predicting the posterior probability of the user's actions toward the target content and predicting the user's actions; The steps include generating suggested information to assist the user's operations based on the aforementioned operation information and the predicted user's operations, The steps include displaying the generated proposed information on the user's terminal, Information processing methods including

6. The steps include displaying multiple target content items that are the subject of the operation process on the user's terminal, A step of obtaining operation information indicating the operation on the target content using the user's eye-tracking information, Based on the events indicated by the aforementioned operation information, the steps include predicting the posterior probability of the user's actions toward the target content and predicting the user's actions; The steps include generating suggested information to assist the user's operations based on the aforementioned operation information and the predicted user's operations, The steps include displaying the generated proposed information on the user's terminal, An information processing program that causes a computer to execute something.

Images