Information processing apparatus, display control method and display control program

Abstract

To provide an information processing apparatus, a display control method, and a display control program for making a hidden display of an external monitor visible.SOLUTION: An information processing apparatus 50 has a display and is connected to an external monitor installed on the back side of the display so that there is an invisible area where the display is blocked by the display. The information processing apparatus is provided with a determination unit 54 for determining whether or not the window of the external monitor is active and a part of the active window on the external monitor is included in the invisible area, and a display control unit 55 for, when the determination unit 54 determines that the window on the external monitor is active and a part of the active window on the external monitor is included in the invisible area, displaying the part of the active window within the invisible area on the external monitor on the display.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, a display control method, and a display control program. [Background technology]

[0002] When using a PC (personal computer) or tablet, an external monitor may be connected and operations may be performed on two or more screens: the PC display or tablet and the external monitor. This type of use is called multi-display. The external monitor is controlled by the PC or tablet, and information from the PC or tablet is displayed on it. Summary of the Invention [Problem to be solved by the invention]

[0003] However, when using multiple displays, if an external monitor is placed behind the PC display or tablet, depending on the usage environment, part of the display on the external monitor may be hidden (blocked) by the display or tablet, causing the user to have to change their posture or line of sight in order to see the hidden display on the external monitor.

[0004] The present disclosure has been made in consideration of the above circumstances, and has an object to provide an information processing device, a display control method, and a display control program that enable a hidden display on an external monitor to be viewed. [Means for solving the problem]

[0005] In order to solve the above problems, the information processing device, the display control method, and the display control program disclosed herein employ the following means. The information processing device disclosed herein is an information processing device having a display, and includes a determination unit that determines whether a window displayed on an external monitor installed behind the display so that an invisible range, which is at least a part of the display area, is active and whether a part of the active window is included in the invisible range, and a display control unit that displays a part of the active window within the invisible range on the display when the determination unit determines that the window is active and that a part of the active window is included in the invisible range.

[0006] The display control method disclosed herein is a display control method for an information processing device having a display, the display control method including a determination step of determining whether a window displayed on an external monitor installed behind the display such that an invisible range, which is at least a part of the display area, is active and whether a part of the active window is included in the invisible range, and a display control step of displaying, on the display, the part of the active window within the invisible range when it is determined in the determination step that the window is active and that a part of the active window is included in the invisible range.

[0007] A display control program according to the present disclosure causes a computer to function as the information processing device described above. Effect of the Invention

[0008] According to the present disclosure, even if the display on the external monitor is obstructed by the display, the user can view the display on the external monitor without changing his / her posture or line of sight. Furthermore, since the display on the external monitor is displayed on the display, the user can perform operations without feeling uncomfortable. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram showing the installation of an information processing device and an external monitor as a conventional example. [Diagram 2] 1A and 1B are diagrams illustrating a user's field of view and non-visible range when an obstacle is present. [Diagram 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device according to some embodiments of the present disclosure. [Figure 4] FIG. 2 is a diagram illustrating an example of functions of an information processing device according to some embodiments of the present disclosure. [Diagram 5] FIG. 2 is a side view of an information processing device and an external monitor according to some embodiments of the present disclosure. [Figure 6] FIG. 11 is a diagram illustrating a processing flow of an information processing device according to some embodiments of the present disclosure. [Figure 7] 11 is a diagram illustrating a display process flow of an invisible range by an information processing device according to some embodiments of the present disclosure. [Figure 8] 1A and 1B are diagrams illustrating an installation state of an information processing device and an external monitor according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of an information processing device, a display control method, and a display control program according to the present disclosure will be described with reference to the drawings. FIG. 1 is a diagram showing the installation of an information processing device and an external monitor as a conventional example. 1 shows an information processing device 50 having a display 100, and an external monitor 200. The information processing device 50 and the external monitor 200 are connected by a cable (e.g., an HDMI (registered trademark) cable). The information processing device 50 has a multi-display, and for example, the display 100 is set as "display 1" and the external monitor 200 is set as "display 2." Such display settings may be made by the information processing device 50 or by the user.

[0011] 1, depending on the installation state of the information processing device 50 and the external monitor 200, a part of the external monitor 200 may be hidden by the display 100 of the information processing device 50. In this embodiment, the range in which the display of the external monitor 200 is blocked by the display 100 and cannot be viewed is defined as the invisible range.

[0012] FIG. 2 is a diagram showing the user's field of view and non-visible areas when an obstacle is present. 2 is a diagram showing a user and an obstacle 10 viewed from above. The field of view of the user's left eye is indicated by a dashed line, and the field of view of the user's right eye is indicated by a dashed line. When the user looks in the direction of the obstacle 10, there is an invisible area 20 in the user's field of vision that is blocked by the obstacle 10 and cannot be seen.

[0013] FIG. 3 is a diagram illustrating an example of a hardware configuration of an information processing device according to some embodiments of the present disclosure. 3, the information processing device (Controller) 50 is a computer system, and includes, for example, a CPU (Central Processing Unit: Processor) 1100, a secondary storage device (ROM, Secondary storage: Memory) 1200, a main storage device (RAM, Main Memory) 1300, a hard disk drive (HDD) 1400 as a large-capacity storage device, and a communication unit 1500 for connecting to a network or the like. Note that a solid-state drive (SSD) may be used as the large-capacity storage device. These units are connected via a bus 1800.

[0014] The CPU 1100 controls the entire information processing device 50 using, for example, an OS (Operating System) stored in a secondary storage device 1200 connected via a bus 1800, and executes various processes by executing various programs stored in the secondary storage device 1200. One or more CPUs 1100 may be provided, and may cooperate with each other to realize processes.

[0015] The main memory device 1300 is composed of writable memory such as cache memory or RAM (Random Access Memory), and is used as a working area for reading out programs executed by the CPU 1100 and writing data processed by the programs.

[0016] The secondary storage device 1200 is a non-transitory computer readable storage medium. The secondary storage device 1200 is, for example, a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Examples of the secondary storage device 1200 include a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a SSD (Solid State Drive) flash memory. The secondary storage device 1200 stores, for example, an OS for controlling the entire information processing device such as Windows (registered trademark), iOS (registered trademark), Android (registered trademark), a BIOS (Basic Input / Output System), various device drivers for operating peripheral devices as hardware, various application software, and various data and files. In addition, the secondary storage device 1200 stores programs for implementing various processes and various data required for implementing various processes. A plurality of secondary storage devices 1200 may be provided, and the above-mentioned programs and data may be divided and stored in each secondary storage device 1200.

[0017] The information processing device 50 may also include an input unit such as a keyboard and a mouse, a display unit such as a liquid crystal display device that displays data, etc. Also, the information processing device 50 may include a notification unit such as a speaker that includes a display unit and outputs a lamp, sound, and in particular an alarm sound.

[0018] FIG. 4 is a diagram illustrating an example of functions of an information processing device according to some embodiments of the present disclosure. As shown in FIG. 4, the information processing device 50 includes an acquisition unit 51, a calculation unit 52, a correction unit 53, a determination unit 54, and a display control unit 55.

[0019] A series of processes for realizing the functions of the information processing device 50 is stored in the secondary storage device 1200 (see FIG. 3) in the form of a program, for example, and the CPU (processor) 1100 (see FIG. 3) reads this program into the main storage device 1300 (see FIG. 3) and executes information processing and arithmetic processing to realize various functions. The program may be pre-installed in the secondary storage device 1200, provided in a state stored in another non-transitory computer-readable storage medium, or distributed via wired or wireless communication means. Examples of non-transitory computer-readable storage media include magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, and semiconductor memories.

[0020] The acquisition unit 51 shown in Fig. 4 acquires an image captured by a rear camera (camera) 70 (see Fig. 5), which will be described later. In particular, in this embodiment, the rear camera 70 captures an image to be output to the external monitor 200, and the acquisition unit 51 acquires this image.

[0021] The calculation unit 52 calculates and derives the distance between the information processing device 50 and the external monitor 200 or the inclination of the information processing device 50 with respect to the external monitor 200, or both, based on the image acquired by the acquisition unit 51. The calculation unit 52 performs calculations based on the outer shape of the external monitor 200 or a predetermined image for adjustment displayed on the external monitor 200. Since the external monitor 200 is connected to the information processing device 50, the outer shape of the external monitor 200 and the size of the display screen are known to the information processing device 50. The predetermined image for adjustment may be a predetermined image that is easy to perform calculations on, such as a marker or a pattern, or may be an icon such as a task bar displayed on the external monitor 200.

[0022] The correction unit 53 corrects data corresponding to the image output to the external monitor 200 based on the distance between the information processing device 50 and the external monitor 200 derived by the calculation unit 52, or the inclination of the information processing device 50 with respect to the external monitor 200, or both, and sets the invisible range of the external monitor 200. Information on the invisible range can be used by the display control unit 55, which will be described later.

[0023] The determination unit 54 determines whether or not a window displayed on the external monitor 200 is active and whether or not a part of the active window displayed on the external monitor 200 is included in the invisible range. In addition, the determination unit 54 may determine whether or not a movement of the position or a change in the angle of the display 100 of the information processing device 50 has been detected.

[0024] When the determination unit 54 determines that a window displayed on the external monitor 200 is active and that a portion of the active window displayed on the external monitor 200 is included in the invisible range, the display control unit 55 displays the portion of the active window displayed on the external monitor 200 that is within the invisible range on the display 100. For an inactive window (inactive window) displayed on the external monitor 200, even if a portion of the inactive window is included in the invisible range, the display control unit 55 does not display the portion of the inactive window on the display 100. The display control unit 55 displays, at an appropriate position on the display 100, an image that is the same as the image displayed in the invisible range of the external monitor 200, and is data corresponding to the image output to the external monitor 200 corrected by the correction unit 53. Displaying at an appropriate position on the display 100 means that the invisible range of an active window displayed on the external monitor 200 is displayed on the display 100, and the image is displayed at a position where the user can recognize it as one active window based on the display of the external monitor 200 and the display 100. The information processing device 50 controls the external monitor 200, and therefore controls data related to the image output to the external monitor 200. Therefore, instead of an image captured by the rear camera 70, a part of the image displayed on the external monitor 200 controlled by the information processing device 50 itself may be displayed. Furthermore, when the determination unit 54 determines that the information processing device 50 or the display 100 has moved or the angle has changed, the display control unit 55 performs calibration using the calculation unit 52 and the correction unit 53, and resets the invisible range. The information processing device 50 can be equipped with a sensor that detects its movement and a sensor that detects the angle of the display 100, and the determination unit 54 determines the movement of the information processing device 50 or the display 100 or the angle change based on the outputs of these sensors.

[0025] FIG. 5 shows a side view of an information processing device and an external monitor according to some embodiments of the present disclosure. As shown in FIG. 5, depending on the installation state of the information processing device 50 and the external monitor 200 connected to the information processing device 50, a part of the external monitor 200 may be hidden by the information processing device 50, and there may be a range that cannot be seen.

[0026] The information processing device 50 has a built-in rear camera 70, and the rear camera 70 can capture an image of the external monitor 200 side within a rear camera capturing range 75.

[0027] When the calculation unit 52 performs a calculation for grasping the positional relationship between the external monitor 200 and the information processing device 50, the information processing device 50 outputs a predetermined image for adjustment to the external monitor 200. The output image may be a predetermined image that is easy to perform the calculation, such as a marker or a pattern.

[0028] FIG. 6 shows a processing flow of an information processing device according to some embodiments of the present disclosure. In step S101, when the calculation unit 52 performs a calculation for grasping the positional relationship between the external monitor 200 and the information processing device 50, an image for adjustment is output to the external monitor 200.

[0029] When a predetermined image is output to the external monitor 200, the rear camera 70 captures it in step S102, and the acquisition unit 51 acquires the image captured by the rear camera 70.

[0030] In step S103, the calculation unit 52 compares the predetermined image captured by the acquisition unit 51 with the image of the external monitor 200 output by the information processing device 50, and calculates and derives the distance between the display 100 and the external monitor 200 and the inclination of the display 100 with respect to the external monitor 200. When an external display and a sensor for detecting the inclination of the display are provided, the inclination sensor may be used to acquire the inclination of the display 100 with respect to the external monitor 200.

[0031] In step S104, the correction unit 53 sets the invisible range of the external monitor 200 based on the distance between the information processing device 50 and the external monitor 200 and the inclination of the display 100 with respect to the external monitor 200 derived by the calculation unit 52. Information on the invisible range becomes available to the display control unit 55.

[0032] The display control unit 55 displays, on the display 100 under specific conditions, a part of the display object that overlaps with the invisible range among the display objects displayed on the external monitor 200 (steps S105 and S106, STEP A surrounded by a dashed line in FIG. 6).

[0033] The information processing device 50 may be equipped with a sensor that detects its movement and a sensor that detects the angle of the display 100. Based on the outputs of these sensors, the determination unit 54 determines the movement and angle change of the information processing device 50 and the display 100. When the determination unit 54 determines the movement and angle change of the information processing device 50 and the display 100, the display control unit 55 performs calibration by the calculation unit 52 and the correction unit 53, and resets the invisible range.

[0034] FIG. 7 illustrates a process flow for displaying the invisible range by an information processing device according to some embodiments of the present disclosure. By transitioning from step S104 in FIG. 6 and performing the process of STEP A in FIG. 7, the process for displaying the invisible range is executed.

[0035] In step S201, the determination unit 54 determines whether a window (e.g., window 210) displayed on the external monitor 200 is active and whether a part of the active window 210 displayed on the external monitor 200 is included in the invisible range.

[0036] When the determination unit 54 determines that the window 210 displayed on the external monitor 200 is active and that a portion of the active window 210 displayed on the external monitor 200 is included in the invisible range (YES in S201), the display control unit 55 displays the active window 210 within the invisible range displayed on the external monitor 200 on the display 100 (S202).

[0037] FIG. 8 is a diagram showing an installation state of an information processing device and an external monitor according to some embodiments of the present disclosure. As shown in FIG. 8, in this embodiment, a part of the display on the external monitor 200 is blocked by the display 100 of the information processing device 50, and an unviewable range exists.

[0038] The process of displaying the invisible range of the external monitor 200 on the display 100 is executed when the user selects to execute the process. An example of a method of selecting to execute the process is when the user presses a predetermined shortcut key. The process may also be executed when the determination unit 54 determines that the cursor 215 is located in the invisible range. A part of the active window 210 displayed on the external monitor 200, which is included in the invisible range and is displayed on the display 100, is shown as range 110 in FIG. 8.

[0039] On display 100, the image originally displayed on display 100 is displayed, and when window 210 on external monitor 200 becomes active, a part of active window 210 on external monitor 200, indicated by range 110, is displayed in front of the image displayed on display 100. An area included in the invisible range that is part of inactive window 220, which is an inactive window displayed on external monitor 200, is not displayed on display 100.

[0040] Next, in the display processing flow of FIG. 7, while the range 110 is displayed on the display 100, the judgment unit 54 judges whether or not a location on the display 100 other than the active window 210 displayed on the external monitor 200 has been clicked, i.e., whether or not any window or desktop on the display 100 has become active (S203), and whether or not it has been detected that an active window displayed on the external monitor 200 has become inactive, i.e., whether or not another window on the external monitor 200 has become active (S204), or whether or not the desktop of the external monitor 200 has become active (S205).

[0041] When the determination unit 54 determines that any window or desktop of the display 100 has become active while the range 110 is displayed on the display 100 (YES in S203), the display control unit 55 ends the display of the range 110 on the display 100 (step S206) and transitions to step S201. On the other hand, when the determination unit 54 determines that none of the windows and desktops of the display 100 have become active (NO in step S203), the process returns to step S202 and continues to display the range 110 on the display 100.

[0042] When another window of the external monitor 200 becomes active while the range 110 is displayed on the display 100 (YES in S204), the display control unit 55 ends the display of the range 110 on the display 100 (S206) and transitions to step S201. The determination unit 54 determines whether or not a part of another active window displayed on the external monitor 200 is included in the invisible range (S201). If it is determined that a part of the other active window is included in the invisible range (YES in S201), the display control unit 55 displays the other active window displayed on the external monitor 200 within the invisible range on the display 100 in step S202. On the other hand, if it is determined that a part of the other active window is not included in the invisible range (NO in S201), the other active window of the external monitor 200 is not blocked by the display 100 and is visible, so the display control unit 55 does not display the other active window of the external monitor 200 on the display 100.

[0043] Moreover, when the determination unit 54 determines that another window on the external monitor 200 is not active (NO in S204), the process returns to step S202, and the display control unit 55 continues to display the range 110 on the display 100.

[0044] When the desktop of the external monitor 200 becomes active while the area 110 is being displayed on the display 100 (YES in S205), the display control unit 55 ends the display of the area 110 on the display 100 (S206) and transitions to step S201. On the other hand, if the determination unit 54 determines that the desktop of the external monitor 200 is not active (NO in S205), the process returns to step S202, and the display control unit 55 continues to display the range 110 on the display 100.

[0045] When a cursor 215 is present on an active window 210 on the external monitor 200 and the cursor 215 moves from outside the non-visible range into the non-visible range, the cursor 215 is displayed on the range 110.

[0046] As described above, the information processing device 50, the display control method, and the display control program according to the present embodiment provide the following advantages. According to the information processing device 50 of the present embodiment, the user can view the display on the external monitor 200 without changing his / her line of sight. In addition, since the display on the external monitor 200 is displayed on the display 100, the user can operate it without feeling uncomfortable.

[0047] According to the information processing device 50 of this embodiment, display is performed via the display 100 only when a window displayed on the external monitor 200 is active, so that the display on the display 100 and the display on the external monitor 200 can be used selectively depending on the situation.

[0048] According to the information processing device 50 of this embodiment, the user can bring to the front a window that the user wants to handle by performing a click operation.

[0049] According to the information processing device 50 of this embodiment, when moving the cursor, the cursor 215 warps from the window of the external monitor 200 to the window displayed via the display 100, and since the cursor 215 appears to move on the active window 210, the user can perform the operation without feeling uncomfortable.

[0050] Although the present disclosure has been described above using the embodiments, the technical scope of the present disclosure is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments without departing from the gist of the disclosure, and the forms in which such modifications or improvements are made are also included in the technical scope of the present disclosure. In addition, the above embodiments may be appropriately combined. Furthermore, the processing flow described in the above embodiment is also one example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the present disclosure.

[0051] For example, in the embodiment of the present disclosure, the calculation unit 52 calculates and derives the distance between the information processing device 50 and the external monitor 200 and the inclination of the information processing device 50 relative to the external monitor 200, but in addition to this, it may also calculate and derive the position of the user's eyes or the distance between the position of the user's eyes and the information processing device 50, or both.

[0052] In this case, the correction unit 53 may correct data corresponding to the image output to the external monitor 200 based on the distance between the information processing device 50 and the external monitor 200 or the inclination of the information processing device 50 with respect to the external monitor 200 derived by the calculation unit 52, or both, and further the position of the user's eyes or the distance between the user's eye position and the information processing device 50, or both. By using the position of the user's eyes or the distance between the user's eye position and the information processing device 50, it is possible to derive the invisible range in more detail than when only the distance between the information processing device 50 and the external monitor 200 or the inclination of the information processing device 50 with respect to the external monitor 200 is used. In addition, even if the position or posture of the user changes, the invisible range can be corrected according to the position of the user's eyes to perform appropriate display by the display control unit 55.

[0053] Also, for example, in the embodiment of the present disclosure, the display control unit 55 displays a part of an active window displayed on the external monitor 200 within the invisible range on the display 100, but when the determination unit 54 determines that any window or desktop on the external monitor 200 is active, the display control unit 55 may display all displays of the external monitor 200 within the invisible range on the display 100. When any window or desktop on the external monitor 200 is active, all displays within the invisible range are displayed via the display 100, allowing the user to perform operations without being aware of the invisible range.

[0054] In this case, the display control unit 55 may cause the display 100 to display the external monitor 200 in the invisible range including the task bar of the external monitor 200. This allows the user to perform operations on the external monitor 200, including operations on the task bar, without being aware of the invisible range.

[0055] Furthermore, the display control unit 55 may perform calibration and set the invisible range using a task bar displayed on the external monitor 200. When performing calibration, it is necessary to display a predetermined adjustment image for calibration on the full screen on the external monitor 200, but the calibration can be easily performed by using the display of the task bar.

[0056] Also, for example, in an embodiment of the present disclosure, when a window on external monitor 200 is active and part of the active window displayed on external monitor 200 is included in the invisible range, the active window on external monitor 200 within the invisible range is displayed on display 100; however, the invisible range of a window over which the mouse is hovered on external monitor 200 may be displayed on display 100. [Explanation of symbols]

[0057] 10 Obstacles 20. Invisible Zone 50 Information processing device 51 Acquisition Department 52 Arithmetic section 53 Correction section 54 Judgment section 55 Display control unit 70 Rear Camera 75 Rear camera shooting range 100 Display 110 Range 200 External Monitor 210 Active Window 215 Cursor 220 Inactive Windows 1100 CPU 1200 Secondary storage 1300 Main storage 1500 Communications Department 1800 Bus

Claims

1. An information processing device having a display, a determination unit that determines whether a window displayed on an external monitor installed behind the display such that an unviewable range, which is at least a part of a display area, is blocked by the display, and that a part of the active window is included in the unviewable range; a display control unit that displays, when the determination unit determines that the window is active and that a portion of the active window is included in the non-visible range, on the display; and An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein when the determination unit detects that the active window on the external monitor has become inactive, the display control unit terminates display of the part of the active window on the display.

3. 2 . The information processing device according to claim 1 , wherein when the determination unit determines that a location on the display other than the active window of the external monitor has been clicked, the display control unit terminates display of the active window on the display.

4. 2. The information processing device according to claim 1, wherein when the determination unit determines that the cursor has moved from outside the invisible range into the invisible range on the active window of the external monitor, the display control unit displays the cursor on the active window displayed on the display.

5. the determination unit determines whether or not at least a portion of a task bar displayed on the external monitor is included in the invisible range; 2 . The information processing device according to claim 1 , wherein when the determination unit determines that at least a portion of a task bar displayed on the external monitor is included in the invisible range, the display control unit causes the display to display at least a portion of the task bar in the invisible range.

6. The information processing apparatus according to claim 1 , wherein the display control unit performs calibration using a task bar displayed on the external monitor, and sets the invisible range.

7. A display control method for an information processing device having a display, comprising: a determination step of determining whether a window displayed on an external monitor installed behind the display such that an unviewable range, which is at least a part of a display area, is blocked by the display, and whether a part of the active window is included in the unviewable range; a display control method in which, when it is determined in the determination step that the window is active and that a portion of the active window is included in the invisible range, a display control step is executed by a computer to display, on the display, the portion of the active window that is within the invisible range.

8. A display control program for causing a computer to function as the information processing device according to claim 1.

Images