Information processing device, information processing system, and program

Abstract

To provide an information processing device and the like, for making hard to impede operations to objects of users even if a virtual pointing device is displayed on a display screen.SOLUTION: Provided is an information processing device 10 comprising a processor. The processor performs: control to display an object to be operated on a display screen 21; control to receive operations to the object by a first contact representing a contact to the object by a user and a second contact representing a contact to a virtual pointing device; and control to display the virtual pointing device in a prescribed position of the display screen 21 in a prescribed size, wherein if the virtual pointing device and the object are overlapped and displayed on the display screen 21, at least one alteration from the prescribed position, the prescribed size, the first contact or the second contact is performed.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] There are computer devices that allow objects displayed on a display screen to be operated by using a pointing device as well as by touching the display screen. In this case, it may be difficult to perform an intuitive and quick operation using a pointing device. Also, it may be difficult to perform a fine operation using a touch operation. When both operations are used together, the user's hand movement may increase, which may increase the burden on the user. Therefore, a virtual pointing device may be displayed on the display screen and operated to reduce the burden on the user.

[0003] Patent Document 1 describes simultaneously displaying a virtual touchpad and a graphical user interface of an operating system on the same display. In this case, touch input software converts touch packets from a user's touch into data packets appropriate for the screen and display resolution of the display or part of the display on which the OS is displayed. Gesture recognition software applies rules to the converted packets to determine the action the user means by the touch. An application controlling the mouse cursor then manipulates the mouse cursor according to the mouse action. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-241139 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is desirable that even if a virtual pointing device is displayed on the display screen, the object to be operated is not hidden, and the user can operate the object. An object of the present invention is to provide an information processing device and the like that does not impede a user's operation on an object even when a virtual pointing device is displayed on a display screen. [Means for solving the problem]

[0006] The invention described in claim 1 includes a processor, which performs control to display an object to be operated on a display screen, control to accept an operation on the object by a first contact by a user on the object and a second contact on a virtual pointing device, and control to display the virtual pointing device at a predetermined position on the display screen with a predetermined size, and the virtual pointing device and the object are displayed superimposed on the display screen. attitude In this case, the predetermined position Change the position so that it does not overlap with the object. , the predetermined size The object position is changed to be smaller so as not to overlap with the object position. , the first contact and The second contact and a change in which an operation on the object by the first contact is accepted when the first contact and the second contact are the same, while an operation on the object by the second contact is not accepted when the first contact and the second contact are the same. Second Contact a change in which the object is operated on by the first contact while not being operated on by the second contact; At least one of t It is an information processing device that performs 。 Claim 2 In the invention described in Do not overlap with the position of the object. Make it smaller Make the above changes Claims that display an animation that transitions from the previous size to the new size when 1 The present invention relates to an information processing device. Claim 3 the predetermined position is in a first region that is relatively closer to the right and lower side of the display screen when the user views the display screen, The predetermined position is set to a position that does not overlap with the object. The change is to change the position of the first region to a position different from the predetermined position. 1The present invention relates to an information processing device. Claim 4 The invention described in the above is directed to the predetermined position and The above The information processing device according to claim 1, wherein the predetermined size can be set by a user. 。 Claim 5 The invention described in The at least one The above changes Change 2. The information processing device according to claim 1, wherein the first and second values ​​are varied depending on the type of the object. Claim 6 The invention described in The at least one The above changes Change The claim that the operation required of the user varies depending on the type of the object. 5 The present invention relates to an information processing device. Claim 7 The invention described in the above relates to an information processing device that creates image information, and a display means having a display screen that displays a screen based on the image information created by the information processing device, the information processing device having a processor, the processor performing control to display an object to be operated on the display screen, control to accept an operation on the object by a first contact by a user on the object and a second contact on a virtual pointing device, and control to display the virtual pointing device at a predetermined position on the display screen with a predetermined size, and the virtual pointing device and the object are displayed superimposed on the display screen. attitude In this case, the predetermined position Change the position so that it does not overlap with the object. , the predetermined size The object position is changed to be smaller so as not to overlap with the object position. , If the first contact and the second contact are the same, First Contact a change in which the object is operated on by the first contact while not being operated on by the second contact, and a change in which the object is operated on by the first contact while not being operated on by the second contact when the first contact and the second contact are the same Second Contact a change in which the object is operated on by the first contact while not being operated on by the second contact; At least one of t Do Information Processing System It is. The invention described in claim 8 isA function of displaying an object to be operated on a display screen, a function of accepting an operation on the object by a first contact by a user on the object and a second contact on a virtual pointing device, and a function of displaying the virtual pointing device at a predetermined position on the display screen with a predetermined size, the virtual pointing device and the object being displayed superimposed on the display screen. attitude In this case, the predetermined position Change the position so that it does not overlap with the object. , the predetermined size The object position is changed to be smaller so as not to overlap with the object position. , If the first contact and the second contact are the same, First Contact a change in which the object is operated on by the first contact while not being operated on by the second contact, and a change in which the object is operated on by the first contact while not being operated on by the second contact when the first contact and the second contact are the same Second Contact a change in which the object is operated on by the first contact while not being operated on by the second contact; At least one of t This is a program that: Effect of the Invention

[0007] According to the invention of claim 1, even if a virtual pointing device is displayed, it is difficult for the user to interfere with the operation of the object. 。 Claim 2 According to the invention, when the size of the virtual pointing device is changed, the user can be notified of the movement. Claim 3 According to the invention, the virtual pointing device can be displayed at a position that allows the user to easily operate it. Claim 4 According to the invention, the position of the virtual pointing device can be changed according to the user's dominant hand and preferences. 。 Claim 5 According to the invention, a method that is less likely to impede user operations can be selected depending on the object. Claim 6 According to the invention, a method that is less likely to impede a user's operation can be selected in accordance with the operation on the object. Claim 7 According to the invention, even if a virtual pointing device is displayed, it is unlikely to hinder the user's operation on an object. Claim 8 According to the invention, a function that does not hinder a user's operation on an object even when a virtual pointing device is displayed can be realized by a computer. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of an information processing device. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of a display device. [Figure 4] 4 is a flowchart illustrating an operation of the information processing apparatus according to the first embodiment. [Diagram 5] 5(a) and 5(b) are diagrams showing the processing performed in steps 103 to 105 in FIG. [Figure 6] 5(a) and 5(b) are diagrams showing the process performed in step 106 in FIG. [Figure 7] 6(a) and 6(b) are diagrams showing another example of the first embodiment. [Figure 8] 13(a) to 13(c) are diagrams showing a case where the position where the virtual track pad is displayed and the size of the virtual track pad are set by the user. [Figure 9] 10 is a flowchart illustrating an operation of an information processing apparatus according to a second embodiment. [Figure 10] 10(a) to 10(c) are diagrams showing the process performed in step 204 of FIG. [Figure 11] 10(a) and 10(b) are diagrams showing the animation performed in step 207 of FIG. [Figure 12] 13 is a flowchart illustrating an operation of an information processing apparatus according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Configuration of information processing device 10> Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0010] <Explanation of the overall information processing system> FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to the present embodiment. As shown in the figure, the information processing system 1 of this embodiment includes an information processing device 10 that creates and controls the display of image information for displaying an image on a display device 20, the display device 20 that receives the image information created by the information processing device 10 and displays an image based on this image information, and an input device 30 that allows a user to input various information to the information processing device 10.

[0011] The information processing device 10 is, for example, a so-called desktop personal computer (PC). The information processing device 10 runs various application software under the control of an OS (Operating System) to create image information and perform other operations.

[0012] The display device 20 is an example of a display means having a display screen 21 for displaying an image. The display device 20 is configured with a function for displaying an image by additive color mixing, such as a liquid crystal display for a PC, a liquid crystal television, or a projector. Therefore, the display method in the display device 20 is not limited to the liquid crystal method. In the example shown in FIG. 1, the display screen 21 is provided inside the display device 20, but when a projector is used as the display device 20, the display screen 21 is a screen or the like provided outside the display device 20. The display device 20 of this embodiment is a touch monitor. This uses a touch panel as the display screen 21. In this case, image information is created by the information processing device 10, and an image is displayed on the touch monitor based on this image information. Then, the user inputs an instruction to perform a desired process by touching the touch monitor or the like.

[0013] The input device 30 is composed of a keyboard, a mouse, a trackball, a touch pad, a track pad, etc. The input device 30 is used by the user to input instructions to the information processing device 10 for performing processing desired by the user when starting and ending application software for performing image processing, and performing image processing, which will be described in detail later. That is, in the information processing system 1 of this embodiment, the user can input instructions both by touching the display screen 21, etc., and by operating the input device 30.

[0014] The information processing device 10 and the display device 20 are connected via a Digital Visual Interface (DVI). Instead of DVI, the information processing device 10 and the display device 20 may be connected via a High-Definition Multimedia Interface (HDMI) (registered trademark), a DisplayPort, or the like. The information processing device 10 and the input device 30 are connected to each other via, for example, a Universal Serial Bus (USB). Instead of the USB, the information processing device 10 and the input device 30 may be connected to each other via IEEE1394, RS-232C, or the like.

[0015] The information processing system 1 in this embodiment is not limited to the form shown in Fig. 1. For example, a notebook PC can be exemplified as the information processing system 1. In this case, the information processing device 10, the display device 20, and the input device 30 are integrated into one PC. Moreover, examples of the information processing system 1 include a smartphone and a tablet terminal. In this case, the tablet terminal includes a touch panel that displays images and inputs user instructions. That is, the touch panel functions as the display device 20 and the input device 30.

[0016] <Configuration of information processing device 10> FIG. 2 is a diagram showing the hardware configuration of the information processing device 10. As shown in FIG. The information processing device 10 shown in the figure has a CPU (Central Processing Unit) 101 that controls each part through the execution of a program, a communication module 102 used for communication with external devices, an internal memory 103 in which system data and internal data are stored, and an external memory 104 as an auxiliary storage device.

[0017] The CPU 101 is an example of a processor, and executes programs such as an OS (operating system) and application software. In this embodiment, the internal memory 103 and the external memory 104 are semiconductor memories. The internal memory 103 has a ROM (Read Only Memory) in which a BIOS (Basic Input Output System) and the like are stored, and a RAM (Random Access Memory) used as a main storage device. The CPU 101 and the internal memory 103 constitute a computer. The CPU 101 uses the RAM as a work space for programs. The external memory 104 is a storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores firmware, application software, and the like. The communication module 102 is a communication interface for communicating with the outside.

[0018] <Configuration of the display device 20> FIG. 3 is a diagram showing the hardware configuration of the display device 20. As shown in FIG. The display device 20 shown in the figure has a control module 201 that controls the display operation of the display device 20, a communication interface 202 that receives image signals from the information processing device 10, a display 203 that displays images, and a film sensor 204 that detects touch operations on the display 203.

[0019] The control module 201 generates a drive signal for driving the display 203 based on the image signal received by the communication interface 202, and causes the display 203 to display an image. The communication interface 202 not only receives image signals from the information processing device 10, but also transmits coordinate information of the position and the like to the information processing device 10 when a user operation is detected by the film sensor 204. The display 203 is configured, for example, by an organic EL (Electro Luminescent) display or a liquid crystal display. In this embodiment, images and other information are displayed on the surface of the display 203. The film sensor 204 is disposed on the surface of the display 203. The film sensor 204 has a characteristic of not interfering with the observation of information displayed on the display 203, and detects the position where the user operates the film sensor 204 by a change in capacitance.

[0020] The display screen 21 is composed of a display 203 and a film sensor 204. With this configuration, the display screen 21 of the present embodiment has a function as a touch panel, and both touch operation and pointing operation are possible. That is, the display screen 21 allows the user to operate objects, etc. displayed on the display screen 21 by touching the display screen 21, etc. Also, the user can perform the same operation by using the input device 30 to move a cursor, etc. displayed on the display screen 21 to the position of the object, etc., and clicking or dragging, etc. Note that here, an "object" is something that is displayed on the display screen 21 and is an object (operation object) that the user operates. Examples of objects include check boxes, menus, icons, windows, dialog boxes, buttons, etc. The OS running on the information processing device 10 is also configured to realize a so-called hybrid UI that supports both touch operations and pointing operations.

[0021] In this case, fine operations are difficult with touch operations, and quick operations are difficult with pointing operations. Therefore, a user may want to use these operations in combination to perform a task. However, using these operations in combination requires the user to move their hand more between the display screen 21 and the input device 30. As a result, the user's hand has to move a larger distance, which can easily lead to inefficiency in the task. Furthermore, this places a greater burden on the user, which can lead to the user feeling tired.

[0022] To solve this problem, it is conceivable to use a part of the display screen 21 as a trackpad area and use this area for pointing operations. That is, a virtual trackpad is displayed on the display screen 21, and when a user wishes to perform a pointing operation, the user operates this virtual trackpad. Also, when a user wishes to perform a touch operation, the user touches the display screen 21, etc. This virtual trackpad Tp is an example of a virtual pointing device. Here, the "virtual pointing device" is an input device that specifies an input position or coordinates on the display screen 21, and this function is realized by virtually displaying this input device on the display screen 21.

[0023] In this configuration, touch operation and pointing operation can be used together on the display screen 21. As a result, the moving distance of the user's hand is reduced, and work efficiency is improved. Furthermore, the burden on the user is reduced, and the user is less likely to become fatigued. At this time, the information processing device 10 performs control to display a virtual trackpad that occupies a predetermined area on the display screen 21, and objects that can be operated by both the user's touch and the virtual trackpad on the display screen 21.

[0024] However, this method does not allow the user to perform a touch operation on an object present in the trackpad area. That is, when the virtual trackpad and the object are displayed overlapping each other on the display screen 21, the object is hidden by the virtual trackpad, and the user cannot perform a touch operation on the object. In this embodiment, the information processing device 10 is configured as follows to solve this problem. Specifically, when the track pad area and the object are displayed superimposed on the display screen 21 and the user is unable to operate the object, a change is made so that the object can be operated but not in an unoperable state. This matter will be described in detail below.

[0025] [First embodiment] Here, first, a description will be given of a first embodiment of the information processing device 10. In the first embodiment, the CPU 101 changes the position at which the virtual track pad is displayed to a position that does not overlap with an object.

[0026] FIG. 4 is a flowchart illustrating the operation of the information processing device 10 of the first embodiment. First, the CPU 101 starts up the application software in response to a user instruction (step 101). Next, CPU 101 reads setting information on the position and size of the virtual track pad to be displayed and setting information on the control method to be applied from external memory 107 (step 102).

[0027] Then, CPU 101 determines whether or not the object and the virtual track pad are overlapped on the screen of the application software (step 103). As a result, if there is overlapping (Yes in step 103), CPU 101 changes the setting information of the position and size of the virtual track pad so as to avoid the object (step 104). In this case, for example, the size of the virtual track pad Tp is reduced so as not to overlap with the position of the object. Then, the virtual track pad is displayed on the display screen 21 based on the changed setting information (step 105). When reducing the size of the virtual track pad, the CPU 101 also displays an animation of the transition from the size before the change to the size after the change (step 106).

[0028] On the other hand, if there is no overlap (No in step 103), the track pad is displayed on the display screen 21 according to the default setting information read in step 102 (step 107). Next, CPU 101 determines whether the screen of the application software has been changed as a result of a user operation or the like (step 108). As a result, if there has been a change (Yes in step 108), the process returns to step 102. On the other hand, if there has been no change (No in step 108), the CPU 101 waits (step 109) and returns to step 108.

[0029] 5(a) and 5(b) are diagrams showing the processes carried out in steps 103 to 105 in FIG. FIG. 5(a) shows a state in which the object and the virtual track pad are superimposed on each other. In this case, the virtual track pad Tp and the check boxes Cb3 and Cb4, which are objects, are superimposed and displayed on the display screen 21. Then, by touching the area where the virtual track pad Tp is displayed, the same operation as that of the actual track pad is realized. In this case, the check boxes Cb1 and Cb2 are not superimposed on the virtual track pad Tp, so the user can operate them by touching them, but the check boxes Cb3 and Cb4 are hidden by the virtual track pad Tp, so they cannot be operated by touching them.

[0030] On the other hand, FIG. 5(b) is a diagram showing a case where the size of the virtual track pad Tp is reduced in step 104 of FIG. In this case, the positions of the virtual track pad Tp and the check boxes Cb3 and Cb4 are not overlapped, and the check boxes Cb3 and Cb4 are not hidden by the virtual track pad Tp, so the user can operate them by touch.

[0031] 6(a) and 6(b) are diagrams showing the process performed in step 106 in FIG. Here, a case is shown in which an animation is displayed that transitions from the state in FIG. 6(a) to the state in FIG. 6(b). Of these, Fig. 6(a) shows the virtual trackpad Tp before the size is changed. And Fig. 6(b) shows the virtual trackpad Tp after the size is changed. Then, by displaying an animation of transition from the state of Fig. 6(a) to the state of Fig. 6(b), the user is notified that the position of the virtual trackpad Tp has changed. In this case, an animation of the virtual trackpad Tp shrinking from the state of Fig. 6(a) to the state of Fig. 6(b) is displayed.

[0032] In the above example, the area in which the virtual track pad Tp is displayed is reduced to an area that does not overlap with the position of the object, but the present invention is not limited to this. 7(a) and (b) are diagrams showing another example of the first embodiment. Here, as shown in FIG. 7(a), first, the virtual track pad Tp is displayed in a first area relatively closer to the right and lower side of the display screen 21 when the user sees the display screen 21. That is, the virtual track pad Tp is displayed in a position closer to the lower right side than the center of the display screen 21. This is because, since the user is often right-handed, displaying the virtual track pad Tp on the right side of the display screen 21 is more convenient for the user during operation, and displaying the virtual track pad Tp on the lower side of the display screen 21 is more convenient for the user during operation. Then, when the area of ​​the virtual track pad Tp and the position of the object overlap, the CPU 101 changes the virtual track pad Tp to a second area different from the first area. 7(b) is a diagram showing the virtual track pad Tp after it has been moved to the second area. Here, the virtual track pad Tp has been moved to a position where no object is displayed, for example, to the upper left of the display screen 21. The virtual track pad Tp may also be moved within the first area.

[0033] However, the position where the virtual track pad Tp is displayed and the size of the virtual track pad Tp may be set by the user. 8(a) to 8(c) are diagrams showing a case where the position where the virtual track pad Tp is displayed and the size of the virtual track pad Tp are set by the user. Of these, FIG. 8(a) is a diagram showing the default position of the virtual track pad Tp, which is displayed relatively to the right and lower side of the display screen 21, similar to FIG. 7(a). At this time, if the user wants to enlarge the virtual track pad Tp, as shown in FIG. 8(b), the user touches the upper left corner of the rectangular virtual track pad Tp and moves it to the upper left. This allows the virtual track pad Tp to be set to be larger. It is to be noted that the virtual track pad Tp can be set to be smaller by touching the upper left corner of the virtual track pad Tp and moving it to the lower right. The same operation can also be achieved by using other corners of the virtual track pad Tp.

[0034] Also, for example, there may be cases where the user is left-handed and wants the virtual track pad Tp to be displayed at a position relatively closer to the left and lower side on the display screen 21. In this case, as shown in FIG. 8(c), the user touches one point on the rectangular virtual track pad Tp and moves it to the left on the display screen 21. This allows the setting to move the position where the virtual track pad Tp is displayed. Note that the position where the virtual track pad Tp is displayed may be moved to any position on the display screen 21 by a similar operation. Thus, in the first embodiment, the CPU 101 performs control to display a virtual pointing device with a predetermined size at a predetermined position on the display screen 21. At this time, when the virtual pointing device and an object are displayed superimposed on the display screen 21, the CPU 101 changes the position and size at which the virtual pointing device Tp is displayed.

[0035] [Second embodiment] Next, a description will be given of a second embodiment of the information processing device 10. In the second embodiment, the CPU 101 changes the operation on the virtual track pad Tp and the operation on the object so as to be different from each other.

[0036] FIG. 9 is a flowchart illustrating the operation of the information processing device 10 of the second embodiment. In FIG. 9, steps 201 to 203 and steps 208 to 209 are similar to steps 101 to 103 and steps 108 to 109 in FIG. 4, and therefore their explanation will be omitted. In the second embodiment, if the virtual trackpad Tp and the object are overlapped in step 203 (Yes in step 203), the operation on the virtual trackpad Tp and the operation on the object are changed to be different (step 204). That is, the CPU 101 changes the operation method performed by the user to be different between the virtual trackpad Tp and the object. Then, the virtual trackpad is displayed on the display screen 21 according to the setting information read in step 202 (step 205). On the other hand, if there is no overlap (No in step 203), no such change is made (step 206), and a virtual track pad is displayed on the display screen 21 according to the setting information read in step 202 (step 205). Then, the CPU 101 displays an animation showing how to operate the virtual track pad Tp (step 207).

[0037] 10(a) to (c) are diagrams showing the process performed in step 204 of FIG. Of these, Fig. 10(a) is a diagram similar to Fig. 5(a), and shows a case where check boxes Cb3 and Cb4, which are objects, are superimposed on the virtual track pad Tp and displayed on the display screen 21. In this case, the check boxes Cb3 and Cb4 are hidden by the virtual track pad Tp, and therefore cannot be operated in this state.

[0038] In this case, the operation on the virtual track pad Tp and the operation on the object are different touch operations. Specifically, the operation method on the virtual track pad Tp is changed from the default setting.

[0039] As shown in Fig. 10(b), the operation on the object remains the default single touch. That is, when there is only one touch in the area where the virtual track pad Tp and the object overlap, the CPU 101 determines that it is an operation on the object. As a result, the user can perform a touch operation on the check boxes Cb3 and Cb4, which are objects, by single touch.

[0040] On the other hand, as shown in FIG. 10(c), the operation on the virtual trackpad Tp is changed from single touch to multi-touch. That is, when there is a touch at two or more points in the area where the virtual trackpad Tp and the object overlap, the CPU 101 determines that it is an operation on the virtual trackpad Tp. That is, the operation method on the virtual trackpad Tp is changed from the default single touch to multi-touch. As a result, the user can perform a pointing operation using the virtual trackpad Tp by multi-touch.

[0041] 11(a) and 11(b) are diagrams showing the animation performed in step 207 of FIG. 11(a) shows an animation of icon I1 displayed when the operation on the virtual trackpad Tp is changed from single touch to multi-touch. This icon I1 shows the index finger and middle finger stretched out, and informs the user that the virtual trackpad Tp needs to be operated with two fingers. 11(b) shows an animation of the virtual trackpad Tp being operated as a single touch, in which icon I2 is displayed. This icon I2 shows the index finger being extended, and informs the user that the virtual trackpad Tp can be operated with one finger. In the above example, the operation method for the virtual trackpad Tp is changed from the default setting, but the operation method for the object may be changed from the default setting. Furthermore, the operation method such as touching an object by a user is an example of a first contact that is a contact with an object. Furthermore, the operation method such as touching a virtual trackpad Tp by a user is an example of a second contact that is a contact with the virtual trackpad Tp. Thus, in the second embodiment, CPU 101 performs control to display a virtual pointing device with a predetermined size at a predetermined position on display screen 21. At this time, when the virtual pointing device and the object are displayed superimposed on display screen 21, CPU 101 changes either the first contact or the second contact if they are the same.

[0042] [Third embodiment] Next, a third embodiment of the information processing device 10 will be described. In the third embodiment, the first embodiment and the second embodiment are used in combination. Here, the first embodiment and the second embodiment are used depending on the type of object. Specifically, the virtual track pad Tp is changed to be different depending on the operation required of the user depending on the type of object.

[0043] FIG. 12 is a flowchart illustrating the operation of the information processing device 10 of the third embodiment. In FIG. 12, steps 301 to 303 and steps 312 to 313 are similar to steps 101 to 103 and steps 108 to 109 in FIG. 4, and therefore their explanation will be omitted. In the third embodiment, if there is overlapping in step 303 (Yes in step 303), CPU 101 determines whether or not the overlapping object is a target to which the second embodiment should be applied (step 304).

[0044] In this case, the target object may occupy a large area on the display screen 21, for example. In this case, it may be difficult to avoid overlapping between the virtual track pad Tp and the object, as in the first embodiment. In the case of such an object, the CPU 101 determines that the second embodiment should be applied. On the other hand, in the case of an object that occupies only a small area on the display screen 21, it determines that the second embodiment should not be applied, but that the first embodiment described below should be applied. Also, when there are a plurality of objects and these are arranged across the entire display screen 21, it may be difficult to avoid overlapping between the virtual track pad Tp and the objects. The same can be said in this case.

[0045] Furthermore, the user can also set which of the first embodiment and the second embodiment should be used. In this case, if the user sets the first embodiment to be applied and the second embodiment not to be applied, the CPU 101 determines that the second embodiment should be applied. On the other hand, if the user sets the second embodiment to be applied and the first embodiment not to be applied, the CPU 101 determines that the second embodiment should not be applied.

[0046] Then, in step 304, if the CPU 101 determines that the second embodiment should be applied (Yes in step 304), the operation on the virtual trackpad Tp and the operation on the object are changed to be different (step 305). In this case, the operation on the virtual trackpad Tp is changed. Specifically, for example, as described above, the setting of the operation on the virtual trackpad Tp is changed from single touch to multi-touch. And the operation on the object is left as single touch. Then, the virtual track pad Tp is displayed based on the setting information read in step 302 (step 306). After that, the process proceeds to step 310, where an animation is displayed (step 310). This animation is as shown in FIG. 11. After that, the process proceeds to step 312.

[0047] On the other hand, if the CPU 101 determines that the second embodiment is not to be applied (No in step 304), such a change is not made, and the operation setting for the virtual track pad Tp is set to the default operation method (step 307), i.e., the single touch is kept as it is.

[0048] Then, the CPU 101 determines that the first embodiment should be applied, and changes the setting information (step 308). Specifically, as described above, the setting is made to reduce the area in which the virtual track pad Tp is displayed to an area that does not overlap with the position of the object. Then, the virtual track pad Tp is displayed on the display screen 21 with the changed setting information (step 309). That is, the virtual track pad Tp is displayed on the display screen 21 with the changed size. The CPU 101 also displays an animation of the change from the size before the change to the size after the change (step 310). This animation is as shown in FIG. 5. After that, the process proceeds to step 312.

[0049] If there is no overlap in step 303 (No in step 303), a virtual track pad Tp is displayed on the display screen 21 according to the default setting information read in step 302 (step 311).

[0050] According to the embodiment described above, touch operation and pointing operation can be used together on the display screen 21, and the moving distance of the user's hand is reduced. As a result, the effect of making work more efficient and reducing the burden on the user can be maintained. Even in this case, the user is not prevented from performing a touch operation on an object present in the area where the virtual trackpad Tp is displayed.

[0051] In the above example, the virtual track pad Tp is shown as the virtual pointing device, but the present invention is not limited to this, and any type or name may be used as long as a pointing operation can be performed. Also, a server device such as a cloud server arranged on a cloud may be used instead of the information processing device 10, and the display device 20 may display image information based on information sent from the server device.

[0052] <Program Description> Here, the processes performed by the information processing device 10 in the present embodiment described above are performed by a program such as application software, for example.

[0053] Therefore, the program that realizes the processing performed by the information processing device 10 in this embodiment can be understood as a program that provides a computer with the following functions: displaying an object to be operated on on a display screen; accepting operations on the object through a first contact by the user on the object and a second contact on the virtual pointing device; and displaying a virtual pointing device at a predetermined position and size on the display screen; and when the virtual pointing device and object are displayed superimposed on the display screen, changing at least one of the predetermined position, the predetermined size, the first contact, and the second contact.

[0054] The program for implementing the present embodiment can be provided not only by communication means but also by being stored in a recording medium such as a CD-ROM.

[0055] Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the claims that various modifications and improvements to the above embodiment are also included in the technical scope of the present invention. [Explanation of symbols]

[0056] 10: information processing device, 20: display device, 21: display screen, 30: input device, 101: CPU, Cb1, Cb2, Cb3, Cb4: checkboxes, Tp: virtual track pad

Claims

1. A processor is provided. The processor, A control for displaying an object to be operated on a display screen; A control for accepting an operation on the object by a first contact by a user on the object and a second contact by a user on a virtual pointing device; and controlling the display of the virtual pointing device at a predetermined position on the display screen with a predetermined size; When the virtual pointing device and the object are displayed superimposed on the display screen, the information processing device performs at least one of the following changes: changing the predetermined position to a position that does not overlap with the object; changing the predetermined size to a smaller position so that it does not overlap with the position of the object; changing the object size so that it does not overlap with the position of the object; changing the object size so that it accepts an operation on the object by a first contact when the first contact and the second contact are the same, while not accepting an operation on the object by the second contact; and changing the object size so that it does not overlap with the position of the object by the first contact when the first contact and the second contact are the same, while not accepting an operation on the object by the first contact.

2. The information processing device according to claim 1 , wherein the processor displays an animation of a transition from a size before the change to a size after the change when the predetermined size is reduced so as not to overlap with the position of the object.

3. the predetermined position is in a first region that is relatively closer to the right and lower side of the display screen when the user views the display screen, The information processing apparatus according to claim 1 , wherein the change to change the predetermined position to a position that does not overlap with the object is a change to a position different from the predetermined position in the first region.

4. An information processing device as described in claim 1, wherein the predetermined position and the predetermined size can be set by a user.

5. An information processing device as described in claim 1, wherein the at least one change is made different depending on the type of the object.

6. The information processing apparatus according to claim 5 , wherein the at least one of the changes is made different depending on an operation required of the user depending on a type of the object.

7. An information processing device that creates image information; a display means having a display screen for displaying a screen based on the image information created by the information processing device; Equipped with The information processing device includes a processor, The processor, A control for displaying an object to be operated on a display screen; A control for accepting an operation on the object by a first contact by a user on the object and a second contact by a user on a virtual pointing device; and controlling the display of the virtual pointing device at a predetermined position on the display screen with a predetermined size; When the virtual pointing device and the object are displayed superimposed on the display screen, the information processing system performs at least one of the following changes: changing the predetermined position to a position that does not overlap with the object; changing the predetermined size to a smaller position so that it does not overlap with the position of the object; changing the object so that, when the first contact and the second contact are the same, an operation on the object by the first contact is accepted while not accepting an operation on the object by the second contact; and changing the object so that, when the first contact and the second contact are the same, an operation on the object by the second contact is accepted while not accepting an operation on the object by the first contact.

8. A computer comprising: A function of displaying an object to be operated on a display screen; a function of accepting an operation on the object by a first contact by a user on the object and a second contact by a user on a virtual pointing device; a function of displaying the virtual pointing device at a predetermined position on the display screen with a predetermined size; Realize this, When the virtual pointing device and the object are displayed superimposed on the display screen, the program performs at least one of the following changes: changing the predetermined position to a position that does not overlap with the object; changing the predetermined size to a smaller position so that it does not overlap with the position of the object; changing the object so that, when the first contact and the second contact are the same, an operation on the object by the first contact is accepted while not accepting an operation on the object by the second contact; and changing the object so that, when the first contact and the second contact are the same, an operation on the object by the second contact is accepted while not accepting an operation on the object by the first contact.

Images