Information processing device, information processing method, and information processing program

Abstract

To provide an information processing apparatus, an information processing method, and an information processing program, which allow a plurality of displayed content items to be properly operated.SOLUTION: An information processing apparatus includes: a display control unit which causes a user terminal to display a plurality of content items to be operated; an acquisition unit which acquires operation information including position coordinates of touch operation of a user, moving speed, and contact area; and a determination unit which determines a content item to be operated by the touch operation of the user, on the basis of operation information obtained after starting a touch of the user in addition to operation information obtained at the start of the touch. The display control unit causes the user terminal to display the operation target content determined by the determination unit while varying display modes.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] When displaying a plurality of contents and accepting operations on the contents, depending on the positional relationship of the plurality of contents, there are cases where the user cannot identify the content intended as the operation target and accepts an operation on a content different from the user's intention. In that case, the user has to redo the operation and input it, which has been inconvenient for the user.

[0003] For example, Patent Document 1 below discloses an information processing apparatus capable of easily selecting crowded operation targets. When it is determined that an operating body such as a hand is close to a first area, a display control unit that causes the display content of the first area to be displayed in a second area different from the first area, and an operation control unit that displays an operation on the first area as an operation on the display content displayed in the second area.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the information processing apparatus described in Patent Document 1 above cannot accurately grasp the operation target intended by the user and change and display the display mode of the content in accordance with the user's intention when a plurality of contents are displayed.

[0006] In view of the above issues, this disclosure aims to provide an information processing device, an information processing method, and an information processing program that can accurately manipulate content when multiple content items are displayed. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objectives, the information processing device according to this disclosure comprises a display control unit that displays multiple contents to be operated on a user terminal, an acquisition unit that acquires operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation, and a discrimination unit that determines the content to be operated on by the user's touch operation based on operation information at the start of the user's touch operation as well as operation information after the start of the touch operation, and the display control unit changes the display mode of the content to be operated on determined by the discrimination unit and displays it on the user terminal. [Effects of the Invention]

[0008] According to this disclosure, it is possible to provide an information processing device, an information processing method, and an information processing program that can accurately manipulate content when multiple contents are displayed. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of information processing according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of an information processing system according to the embodiment. [Figure 3] Figure 3 shows an example of the configuration of an information processing device according to the embodiment. [Figure 4] Figure 4 shows an example of information stored in the operation information storage unit of the information processing apparatus according to the embodiment. [Figure 5] Figure 5 shows an example of information stored in the model data storage unit of the information processing device according to the embodiment. [Figure 6] Figure 6 shows an example of information stored in the content storage unit of the information processing device according to this embodiment. [Figure 7] Figure 7 shows an example of content that the information processing device according to this embodiment displays on a user terminal. [Figure 8] Figure 8 shows an example of the configuration of a user terminal according to the embodiment. [Figure 9] Figure 9 is a flowchart showing an example of information processing according to the embodiment. [Figure 10] Figure 10 is a hardware configuration diagram showing an example of a computer that implements the functions of an information processing device. [Modes for carrying out the invention]

[0010] The following describes in detail, with reference to the drawings, the embodiments for implementing the information processing device, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). However, these embodiments do not limit the information processing device, information processing method, and information processing program according to the present application.

[0011] (Embodiment) [1-1. An example of information processing according to the embodiment] First, an example of information processing according to the embodiment will be explained using Figure 1. Figure 1 is a diagram showing an example of information processing according to the embodiment. First, an overview of the information processing according to the embodiment will be explained, and then each process will be explained in detail.

[0012] Figure 1 shows a process in which the information processing device 100 displays multiple content items to be operated on the user terminal 200, obtains user operation information from the user terminal 200, identifies the content to be operated on based on the operation information, determines the type of operation based on the operation information, and displays the content by changing the display mode of the content to be operated on based on the determined type of operation. Below, an example of the information processing according to the embodiment will be described step by step with reference to Figure 1.

[0013] First, the information processing device 100 causes a plurality of contents to be operated on to be displayed (step S1). For example, the information processing device 100 causes the user terminal 200 to display a plurality of contents to be operated on, including an electronic book page screen, a text box, a video scroll bar, a web page scroll bar, CAD (Computer-Aided Design) objects, etc., and receives the operation information of the user. The state in which a plurality of contents to be operated on are displayed means, for example, a case where a plurality of scroll bars that receive a swipe operation are displayed, or a state where a page screen that receives a scroll bar and a swipe operation are displayed overlappingly.

[0014] Next, the information processing device 100 acquires the operation information of the user from the user terminal 200 (step S2). For example, the information processing device 100 acquires operation information including the touch time required for each finger of the user U touching the touch panel of the user terminal 200, the touch movement speed (X-axis, Y-axis), the touch-down position (X-axis, Y-axis), the touch-up position (X-axis, Y-axis), the contact area, and the difference in the touch-down / touch-up position.

[0015] Next, the information processing device 100 discriminates the content to be operated on based on the operation information (step S3). For example, the information processing device 100 discriminates the content to be operated on using a model that has learned the relationship between the operation information when the user performs an operation on a predetermined content and the predetermined content. That is, the information processing device 100 discriminates the content that is the operation target of the user's touch operation by inputting the operation information of the user's touch operation into the learned model.

[0016] Next, the information processing device 100 determines the operation type based on the operation information (step S4). For example, according to the operation information, when it is detected that the user U starts a touch with two fingers and then performs an operation of expanding the interval between the two fingers, the information processing device 100 determines that the operation type is a pinch-out operation.

[0017] Next, the information processing apparatus 100 changes the display mode of the operation target content based on the operation type and displays it (step S5). For example, when the information processing apparatus 100 determines that the content CT1 shown in FIG. 1 is the operation target content and determines that the operation type of the user is a pinch-out operation, the information processing apparatus 100 changes to a display mode in which the content CT1 is enlarged and displayed, and displays the content CT1.

[0018] Thereby, even when a plurality of contents that are operation targets are displayed, the information processing apparatus 100 can grasp the content that is the operation target of the user and change and display the display mode of the content in accordance with the operation intention of the user.

[0019] [1-2. Another Example 1 of Information Processing According to the Embodiment] The information processing apparatus 100 discriminates the content that is the operation target of the touch operation of the user using a different model for each content.

[0020] This information processing will be described step by step. First, the information processing apparatus 100 executes the same processing as steps S1 to S2 shown in FIG. 1. The processing from steps S1 to S2 is the same as the above-described processing, so the description is omitted.

[0021] Next, the information processing apparatus 100 discriminates the operation target content based on the operation information (step S3). In this case, the information processing apparatus 100 discriminates the content that is the operation target using a different model for each content. For example, when the plurality of displayed contents are scroll bars, the information processing apparatus 100 uses a model that has learned the relationship between the operation information when an operation is performed on the scroll bar and the scroll bar to discriminate the content that is the operation target of the touch operation of the user.

[0022] Next, the information processing apparatus 100 executes the same processing as steps S4 to S5 shown in FIG. 1. The processing from steps S4 to S5 is the same as the above-described processing, so the description is omitted.

[0023] As a result, even when multiple pieces of content to be operated on are displayed, the information processing device 100 can identify the content to be operated on according to the user's intent and change the display of the content accordingly to accurately display it.

[0024] [1-3. Another example of information processing according to the embodiment 2] The information processing device 100 displays multiple contents, each with a different operation direction accepted by the user, and identifies the content to be operated on based on the operation information and the operation direction accepted by each content.

[0025] This information processing will be explained step by step. First, the information processing device 100 displays multiple target contents, each accepting different directions of operation from the user (step S1). In this case, the information processing device 100 displays, for example, a scroll bar that accepts vertical swipe operations and a scroll bar that accepts horizontal swipe operations on the user terminal 200. Alternatively, the information processing device 100 may overlay a page screen that accepts horizontal swipe operations and a page screen that accepts vertical swipe operations and display them on the user terminal 200.

[0026] Next, the information processing device 100 performs the same process as in step S2 shown in Figure 1. Since the process in step S2 is the same as the process described above, its explanation will be omitted.

[0027] Next, the information processing device 100 identifies the content to be operated on based on the operation information (step S3). In this case, the information processing device 100 identifies the content to be operated on based on the operation information and the operation direction accepted by each content. For example, if the user terminal 200 displays a scroll bar that accepts vertical swipe operations and a scroll bar that accepts horizontal swipe operations, and the operation information obtained from the user terminal 200 indicates a horizontal finger movement, the information processing device 100 identifies the content to be operated on as a scroll bar that accepts horizontal swipe operations.

[0028] Next, the information processing device 100 executes the same process as steps S4 to S5 shown in Figure 1. Since the process in steps S4 to S5 is the same as the process described above, its explanation will be omitted.

[0029] As a result, the information processing device 100 can identify the content to be operated on according to the user's intent and display the content by changing the display mode according to the user's intent.

[0030] [2. Configuration of the Information Processing System] Next, the configuration of the information processing system according to the embodiment will be described using Figure 2. Figure 2 is a diagram showing an example of the configuration of the information processing system according to the embodiment. As shown in Figure 2, the information processing system 1 includes an information processing device 100 and a user terminal 200. Note that the information processing system 1 shown in Figure 2 may include multiple information processing devices 100 and multiple user terminals 200. The information processing device 100 and the user terminal 200 are connected to each other via a predetermined communication network (network N) by wired or wireless means.

[0031] The information processing device 100 may be, for example, a personal computer (PC), a workstation (WS), or a computer equipped with server functions. The information processing device 100 processes information transmitted from the user terminal 200 via the network N.

[0032] The user terminal 200 is an information processing device used by the user. The user terminal 200 may be, for example, a smartphone, tablet, desktop PC, notebook PC, mobile phone, or PDA (Personal Digital Assistant). In the example shown in Figure 1, the user terminal 200 is a smartphone.

[0033] [3. Configuration of the Information Processing Device] Next, the configuration of the information processing device 100 according to the embodiment will be described using Figure 3. Figure 3 is a diagram showing an example of the configuration of the information processing device according to the embodiment. As shown in Figure 3, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Although not shown in Figure 3, the information processing device 100 may also have an input unit (for example, a keyboard or mouse) that receives various operations from the administrator of the information processing device 100, and a display unit (for example, a liquid crystal display) for displaying various information.

[0034] (Regarding Communications Unit 110) The communication unit 110 is implemented, for example, by a NIC (Network Interface Card). The communication unit 110 is connected to the network N by wire or wireless connection and transmits and receives information with the user terminal 200.

[0035] (Regarding memory unit 120) The storage unit 120 is implemented by, for example, semiconductor memory elements such as RAM (Random Access Memory) and flash memory, or by storage devices such as hard disks, SSDs (Solid State Drives), and optical discs. As shown in Figure 3, the storage unit 120 includes an operation information storage unit 121, a model storage unit 122, and a content storage unit 123.

[0036] (Regarding the operation information storage unit 121) The operation information storage unit 121 stores user operation information obtained from the user terminal 200. Here, an example of the information stored in the operation information storage unit 121 will be explained using Figure 4. Figure 4 is a diagram showing an example of the information stored in the operation information storage unit of the information processing device according to this embodiment.

[0037] In the example shown in Figure 4, the operation information storage unit 121 stores information related to the items "User ID" and "Operation Information" in an associated manner.

[0038] "User ID" is an identifier that identifies a user and is represented by a string of characters or a number. "Operation Information" is operation information of the user linked to the "User ID" and may include the touch time for each finger of the user, touch movement speed (X axis, Y axis), touchdown position (X axis, Y axis), touchup position (X axis, Y axis), contact area, difference between touchdown and touchup positions, etc.

[0039] In other words, Figure 4 shows that the user operation information identified by the user ID "UID#1" is operation information "operation information #1".

[0040] Furthermore, the information stored in the operation information storage unit 121 is not limited to information related to the items "User ID" and "Operation Information," but may also store any other information related to the user's operations.

[0041] (Regarding the model memory unit 122) The model storage unit 122 stores operation information when a user performs an operation on a predetermined content, and a trained model that has learned the relationship with the predetermined content. Figure 5 shows an example of the information stored in the model storage unit of the information processing device according to this embodiment.

[0042] In the example shown in Figure 5, the model storage unit 122 stores information related to the items "Model ID" and "Model Data" in an associated manner.

[0043] The "Model ID" is an identifier that identifies a machine learning model and is represented by a string, number, or other format. "Model Data" refers to the model data of the machine learning model. For example, the machine learning model may be a neural network.

[0044] In other words, in Figure 5, the model identified by model ID "M#1" represents the machine learning model M#1. Furthermore, the model data "MDT#1" represents the model data for machine learning model M#1.

[0045] In this case, if the machine learning model is a neural network, the model data "MDT#1" includes various information such as connection information, which describes how the nodes in each of the multiple layers that make up the neural network are connected to each other, and connection coefficients, which are multiplied by the numerical values ​​that are input and output between the connected nodes.

[0046] Furthermore, the model storage unit 122 is not limited to information related to the items "model ID" and "model data," but may store any other information related to the machine learning model.

[0047] (Regarding content storage unit 123) The content storage unit 123 stores multiple pieces of content to be displayed on the user terminal 200. Figure 6 shows an example of information stored in the content storage unit of the information processing device according to this embodiment.

[0048] In the example shown in Figure 6, the content storage unit 123 stores information related to the items "Content ID" and "Content Data" in an associated manner.

[0049] The "Content ID" is an identifier that identifies the content and is represented by a string of characters, a number, or other means. The "Content Data" is the data of the content to be displayed on the user terminal 200.

[0050] In other words, Figure 6 shows that the content data indicated by content data "CD#1" is stored as content identified by content ID "CT#1".

[0051] Furthermore, the content storage unit 123 is not limited to information related to the items "Content ID" and "Content Data," but may store any other information related to the content.

[0052] (Regarding the control unit 130) Next, returning to Figure 3, the control unit 130 will be described. The control unit 130 is implemented by a CPU (Central Processing Unit) or MPU (Micro Processing Unit), etc., which executes various programs stored in the memory of the information processing device 100 using RAM as the working area. Alternatively, the control unit 130 may be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0053] As shown in Figure 3, the control unit 130 includes an acquisition unit 131, a discrimination unit 132, a determination unit 133, and a display control unit 134.

[0054] (Regarding acquisition unit 131) The acquisition unit 131 acquires operation information from the user terminal 200, including the position coordinates, movement speed, and contact area of ​​the user's touch operation. For example, the acquisition unit 131 acquires information as operation information of the touch operation received from the user by the user terminal 200, including the touch duration for each finger of the user, touch movement speed (X axis, Y axis), touch-down position (X axis, Y axis), touch-up position (X axis, Y axis), contact area, and the difference between the touch-down and touch-up positions. Once the acquisition unit 131 has acquired the operation information, it stores the acquired operation information in the operation information storage unit 121.

[0055] (Regarding the discrimination unit 132) The discrimination unit 132 determines the content targeted by the user's touch operation based on the operation information at the start of the touch operation as well as the operation information after the start of the touch operation. In other words, the discrimination unit 132 determines the content targeted by the user's touch operation based not only on the touch-down position (X axis, Y axis) which indicates the operation information at the start of the touch operation, but also on the touch duration, touch movement speed (X axis, Y axis), touch-up position (X axis, Y axis), contact area, and the difference between the touch-down and touch-up positions which indicate the operation information after the start of the touch operation.

[0056] Specifically, the discrimination unit 132 uses a model that has learned the relationship between the operation information when the user operates on a predetermined content and the predetermined content to determine the content that the user is touching. The discrimination unit 132 reads the trained model from the model storage unit 122 and inputs the operation information of the user's touch operation into the trained model to determine the content that the user is touching. In this case, for example, a deep neural network may be used as the training model.

[0057] The discrimination unit 132 may use a different model for each piece of content to determine which content is the target of the user's touch operation. For example, if a scroll bar and a page screen are displayed as multiple pieces of content to be operated on, the discrimination unit 132 uses the user's touch operation information on the scroll bar and the page screen, and a model that has learned the relationship between the scroll bar and the page screen, to determine which content is the target of the user's touch operation. In other words, the discrimination unit 132 uses a different model for each combination of multiple pieces of content displayed to determine which content is the target of the user's touch operation.

[0058] The discrimination unit 132 determines the content to be operated on by the user's touch operation based on the operation information and the operation direction accepted by each content. For example, if the discrimination unit 132 displays a scroll bar that accepts vertical swipes and a scroll bar that accepts horizontal swipes, and the operation information of the user's touch operation indicates a horizontal finger movement, the discrimination unit 132 determines the target content to be operated on as a scroll bar that accepts horizontal swipes. Also, if the operation information of the user's touch operation indicates a vertical finger movement, the discrimination unit 132 determines the target content to be operated on as a scroll bar that accepts vertical swipes.

[0059] (Regarding the determination unit 133) The determination unit 133 determines the type of touch operation performed by the user based on the operation information at the start of the touch operation as well as the operation information after the start of the touch operation. The determination unit 133 determines the type of operation, including flick, swipe, pinch-in, and pinch-out, based on the change in position coordinates included in the operation information. Here, flick refers to an operation in which the user touches the touch panel, moves it quickly a certain distance, and then releases it. Swipe refers to stroking the touch panel up, down, left, or right while keeping the finger in contact with it. Pinch-in refers to a touch operation in which the user touches multiple locations (for example, two points) together and brings the touch positions closer together. Pinch-out refers to a touch operation in which the user touches multiple locations (for example, two points) together and moves the touch positions further apart.

[0060] In other words, the determination unit 133 determines which of the following types of touch operations the user performed—flick, swipe, pinch-in, or pinch-out—is based on information such as the touch duration, touch movement speed (X-axis, Y-axis), touch-down position (X-axis, Y-axis), touch-up position (X-axis, Y-axis), contact area, and the difference between the touch-down and touch-up positions, as included in the user's touch operation information.

[0061] (Regarding the display control unit 134) The display control unit 134 displays multiple pieces of content that are the target of the operation process on the user terminal 200. The display control unit 134 displays multiple pieces of content, such as ebook pages, text boxes, video scroll bars, web page scroll bars, and CAD (Computer-Aided Design) objects. However, the content displayed by the display control unit 134 is not limited to these, and may also include content utilizing VR (Virtual Reality) or MR (Mixed Reality) technologies.

[0062] The display control unit 134 changes the display mode of the target content determined by the determination unit 133 and displays it on the user terminal 200. Specifically, the display control unit 134 changes the display mode of the target content based on the type of operation determined by the determination unit 133 and displays it. This process of the display control unit 134 will be explained with reference to Figure 7.

[0063] Figure 7 shows an example of content displayed on a user terminal by an information processing device according to the embodiment. In Figure 7, content CT1, which shows a page screen that accepts swipe operations, and content CT2, which shows a scroll bar that accepts swipe operations, are displayed superimposed. For example, if the discrimination unit 132 identifies content CT1 as the content to be operated on, and the determination unit 133 determines that the user's operation type is a swipe operation, content CT1 is moved horizontally to display the content of the next page screen.

[0064] Furthermore, the display control unit 134 displays multiple content items that accept different user operation directions. For example, the display control unit 134 displays a scroll bar that accepts vertical swipes and a scroll bar that accepts horizontal swipes on the user terminal 200. The display control unit 134 may also overlay a page screen that accepts horizontal swipes and a page screen that accepts vertical swipes on the user terminal 200.

[0065] [4. User terminal configuration] Next, the configuration of the user terminal 200 according to the embodiment will be described using Figure 8. Figure 8 is a diagram showing an example of the configuration of a user terminal according to the embodiment. As shown in Figure 8, the user terminal 200 has a communication unit 210, an input unit 220, an output unit 230, and a control unit 240.

[0066] The communication unit 210 is implemented, for example, by a NIC (Network Interface Card). The communication unit 210 is connected to the network N by wire or wireless connection and transmits and receives various types of information to and from the information processing device 100 via the network N.

[0067] The input unit 220 receives various operation information from the user. For example, the input unit 220 accepts various operations from the user via a touch panel on the display surface (e.g., the output unit 230). The touch panel may be of various types, such as resistive, capacitive, surface acoustic wave, infrared, or electromagnetic induction. In the resistive method, a transparent electrode is formed from a resistive metal thin film, and a voltage is applied to one of two opposing resistive films. The contact position is detected by detecting the voltage corresponding to the operated position. In the capacitive method, the contact position is detected by capturing the change in capacitance between the fingertip and the conductive film. In the surface acoustic wave method, piezoelectric elements are attached to multiple corners of the substrate to generate vibrations, and the contact position is detected by capturing the rebound of the vibration waves when a finger touches the substrate. In the infrared method, the contact position is detected by capturing the blocking of infrared rays. In the electromagnetic induction method, an EMR (Electro-Magnetic Resonance) sensor is placed below the screen to capture the magnetic field emitted from the EMR pen and detect the contact position.

[0068] The output unit 230 is a display screen for a tablet terminal, for example, which is implemented using a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information. In other words, if the input unit 220 of the user terminal 200 is a touch panel, the output unit 230, which is the display screen, accepts user input and also outputs to the user. The output unit 230 may also be a speaker and output sound through the speaker.

[0069] The control unit 240 is implemented, for example, by a CPU or MPU executing various programs stored in the user terminal 200 using RAM as a working area. Alternatively, the control unit 240 may be implemented by an integrated circuit such as an ASIC or FPGA.

[0070] As shown in Figure 8, the control unit 240 includes a reception unit 241 and a display control unit 242.

[0071] The reception unit 241 receives operation information for content from the user. Alternatively, for example, the reception unit 241 may display multiple content items to be operated on the output unit 230 and receive operation information for multiple content items from the user. If the output unit 230 is a touch panel, the reception unit 241 may receive operation information for content from the user by sensing information such as the center point coordinates of the touch coordinates for each of the user's fingers, movement speed, movement direction, and contact area.

[0072] The display control unit 242 controls the information displayed on the output unit 230. For example, if the display control unit 242 receives an operation from the user on a page screen that accepts swipe operations, it issues a control command to the output unit 230 to display the next page screen after the currently displayed page screen, according to the user's operation. The display control unit 242 may issue a control command to the output unit 230 based on a control command transmitted from the display control unit 136 of the information processing device 100, or the display control unit 242 may issue a control command to the output unit 230 independently.

[0073] [5. Information Processing Flow] Next, the procedure for information processing by the information processing device 100 according to the embodiment will be explained using Figure 9. Figure 9 is a flowchart of an example of information processing according to the embodiment. For example, the information processing device 100 displays multiple contents that are the target of operation processing on the user terminal 200 (step S101). The information processing device 100 then acquires operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation (step S102). The information processing device 100 then determines the content that is the target of the user's touch operation based on the operation information (step S103). The information processing device 100 then changes the display mode of the determined target content and displays it (step S104).

[0074] [6. Hardware Configuration] Furthermore, the information processing device 100 according to the above-described embodiment is realized by a computer 1000 having a configuration such as that shown in Figure 10. Figure 10 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which an arithmetic unit 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090.

[0075] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, as well as programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as RAM, that temporarily stores data used by the arithmetic unit 1030 for various calculations. The secondary storage device 1050 is a storage device that stores data used by the arithmetic unit 1030 for various calculations and various databases, and is implemented using ROM (Read Only Memory), HDD (Hard Disk Drive), flash memory, etc.

[0076] Output IF1060 is an interface for transmitting information to be output to output devices 1010, which output various types of information such as monitors and printers. It is implemented using connectors of standards such as USB (Universal Serial Bus), DVI (Digital Visual Interface), and HDMI (High Definition Multimedia Interface). Input IF1070 is an interface for receiving information from various input devices 1020, such as mice, keyboards, and scanners. It is implemented using, for example, USB.

[0077] The input device 1020 may also be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), DVD (Digital Versatile Disc), or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), tape media, magnetic recording media, or semiconductor memory. Furthermore, the input device 1020 may also be an external storage medium such as a USB memory stick.

[0078] Network IF1080 receives data from other devices via network N and sends it to the arithmetic unit 1030, and also transmits data generated by the arithmetic unit 1030 to other devices via network N.

[0079] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0080] For example, when a computer 1000 functions as an information processing device 100, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040, thereby realizing the functions of the control unit 130 of the information processing device 100.

[0081] [7. Structure and Effects] The information processing device 100 according to this disclosure includes a display control unit 134 that displays multiple contents that are the subject of operation processing on a user terminal 200, an acquisition unit 131 that acquires operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation, and a discrimination unit 132 that determines the content that is the subject of the user's touch operation based on operation information at the start of the user's touch operation as well as operation information after the start of the touch operation, and the display control unit 134 changes the display mode of the content to be operated determined by the discrimination unit 132 and displays it on the user terminal 200.

[0082] With this configuration, even when multiple pieces of content are displayed, the system can identify the content the user is interacting with and change the display method accordingly.

[0083] The information processing device 100 according to this disclosure further includes a determination unit 133 that determines the user's intent to touch based on operation information at the start of the user's touch operation as well as operation information after the start of the touch operation, and a display control unit 134 changes the display mode of the content to be operated on in accordance with the intent to touch determined by the determination unit 133 and displays it.

[0084] With this configuration, even when multiple content items to be manipulated are displayed, the type of operation performed by the user can be determined, and the display mode of the content to be manipulated can be changed based on the determined operation type.

[0085] The discrimination unit 132 of the information processing device 100 relating to this disclosure uses a model that has learned the relationship between operation information when a user operates on a predetermined content and the predetermined content to determine the content that the user is touching.

[0086] With this configuration, even when multiple content items to be manipulated are displayed, the trained model can identify the content to be manipulated and change its display mode accordingly.

[0087] The discrimination unit 132 of the information processing device 100 according to this disclosure uses a different model for each piece of content to determine which content is the target of the user's touch operation.

[0088] With this configuration, even when multiple content items to be manipulated are displayed, different models can be used depending on the type of content to identify the content to be manipulated and change the way the content is displayed.

[0089] The display control unit 134 of the information processing device 100 according to this disclosure displays multiple contents, each with a different operation direction accepted by the user, and the discrimination unit 132 determines the content to be operated on based on the operation information and the operation direction accepted by each content.

[0090] With this configuration, even when multiple pieces of content are displayed, each with a different user interaction direction, the system can identify the content to be interacted with and change its display mode accordingly.

[0091] Furthermore, the information processing method relating to this disclosure includes the steps of: displaying multiple contents subject to operation processing on the user terminal 200; acquiring operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation; determining the content subject to the user's touch operation based on operation information at the start of the user's touch operation as well as operation information after the start of the touch operation; and displaying the determined content subject to operation on the user terminal 200 in a different display manner.

[0092] With this configuration, even when multiple pieces of content are displayed, the system can identify the content the user is interacting with and change the display method accordingly.

[0093] Furthermore, the information processing program relating to this disclosure causes a computer to perform the following steps: display multiple contents that are the subject of operation processing on the user terminal 200; acquire operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation; determine the content that is the subject of the user's touch operation based on the operation information at the start of the user's touch operation as well as the operation information after the start of the touch operation; and change the display mode of the determined content to be operated on and display it on the user terminal.

[0094] With this configuration, even when multiple pieces of content are displayed, the system can identify the content the user is interacting with and change the display method accordingly.

[0095] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms, including those described in the disclosure section of the invention, based on the knowledge of those skilled in the art.

[0096] Furthermore, the terms "section, module, unit" mentioned above can be replaced with "means" or "circuit." For example, the acquisition unit 131 can be replaced with acquisition means or acquisition circuit. [Explanation of symbols]

[0097] 1. Information Processing System 100 Information Processing Devices 110 Communications Department 120 Storage section 121 Operation information storage unit 122 Model Memory Unit 123 Content Storage Unit 130 Control Unit 131 Acquisition Department 132 Discrimination part 133 Judgment section 134 Display Control Unit 200 user terminals 210 Communications Department 220 Input section 230 Output section 240 Control Unit 241 Reception Department 242 Display Control Unit N Network

Claims

1. A display control unit that displays multiple content items that are the target of the operation process on the user terminal, An acquisition unit that acquires operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation, It includes a discrimination unit that determines the content to be operated on by the user's touch operation based on the user's operation information at the start of the touch operation as well as operation information after the start of the touch operation, The display control unit changes the display mode of the content to be operated on, as determined by the discrimination unit, to a display mode corresponding to the type of operation performed by the user as indicated by the operation information, and displays it on the user terminal. Information processing device.

2. The system further includes a determination unit that determines the user's intent to perform a touch operation based on the user's operation information at the start of the touch operation as well as operation information after the start of the touch operation. The display control unit changes the display mode of the target content to be operated on in accordance with the operation intent determined by the determination unit. The information processing apparatus according to claim 1.

3. The discrimination unit uses a model that has learned the relationship between operation information when a user performs an operation on content of a predetermined operation type and the content of that predetermined operation type to determine the content that the user is touching. The information processing apparatus according to claim 1.

4. The discrimination unit uses a different model for each type of content operation to determine the content that the user is touching. The information processing apparatus according to claim 3.

5. The display control unit displays multiple target contents, each with a different operation direction accepted by the user. The discrimination unit determines the content to be operated on based on the operation information and the operation direction accepted by each target content. The information processing apparatus according to claim 1.

6. The steps include displaying multiple pieces of content that are the target of the operation process on the user's terminal, The steps include obtaining operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation, The process involves determining the content to be touched by the user based on the user's initial touch information as well as subsequent touch information, and The step includes changing the display mode of the identified target content to a display mode corresponding to the type of operation performed by the user as indicated by the operation information, and displaying it on the user's terminal, Information processing methods.

7. The steps include displaying multiple pieces of content that are the target of the operation process on the user's terminal, The steps include obtaining operation information including the position coordinates, movement speed, and contact area of ​​the user's touch operation, The process involves determining the content to be touched by the user based on the user's initial touch information as well as subsequent touch information, and The steps include: changing the display mode of the identified target content to a display mode corresponding to the user's operation type indicated by the operation information and displaying it on the user's terminal; An information processing program that causes a computer to execute something.

Images