Virtual display arrangement system, virtual display arrangement method, and computer program

Abstract

The present invention reduces the time and effort required to adjust the position at which a virtual display is placed using a smart glass. In the present invention, a position calculation unit of a smart glass calculates the position in a virtual space at which a virtual display is to be placed, as virtual screen position information. An acquisition unit of a computer device acquires and stores the virtual screen position information of the virtual display in a storage device. A provision unit provides the virtual screen position information of the virtual display stored in the storage device to the smart glass. The smart glass places the virtual display at the position in the virtual space as represented by the virtual screen position information provided by the computer device.

Description

Virtual display layout system, virtual display layout method, and program storage medium

[0001] The present disclosure relates to technology related to the placement of virtual displays and the display of information using smart glasses.

[0002] When working with a computer, a display device with a large screen or multiple display devices may be used as a display device to display information processed by the computer, in order to improve ease and efficiency of work.

[0003] As remote work becomes more common, work environments are becoming more diverse. As a result, some people work from home, and when these people perform computer-based tasks (hereinafter referred to as PC (Personal Computer) tasks), it can be difficult to use a display device with a large screen or multiple display devices due to workspace limitations. In such cases, smart glasses can be used to improve the work environment. Specifically, smart glasses refer to eyeglass-type wearable devices. Smart glasses include augmented reality (AR) types that superimpose images and text on camera footage, mixed reality (MR) types that extend from AR and recognize a real three-dimensional space to present the wearer with a world that blends reality and virtuality, and virtual reality (VR) types that present the wearer with a completely virtual world. By using such smart glasses, it is possible to place a display device in a virtual space created by the smart glasses, even in places where it is difficult to place a large display device or multiple display devices in the real world, thereby improving the work environment for PC tasks. Here, real space refers to a real physical space, and virtual space refers to a space in a coordinate system determined by the smart glasses or a space generated by the smart glasses.

[0004] Patent Document 1 (Japanese Patent Laid-Open Publication No. 4-287094) discloses a configuration that stores information about the layout positions of windows displayed on a display screen, thereby enabling the layout of those windows to be reproduced.

[0005] Japanese Patent Application Publication No. 4-287094

[0006] The position of the display device (hereinafter also referred to as the virtual display) in the virtual space created by the smart glasses can be changed by, for example, the operation of the smart glasses wearer. In other words, the smart glasses wearer can create a PC work environment that suits them by moving the virtual display to a position that suits their preferences.

[0007] However, every time the smart glasses are started (in other words, every time an application program (hereinafter also referred to as an app) that controls the operation of the smart glasses is started), the position of the virtual display returns to the initial setting position. Therefore, in order to perform PC work using the virtual display in the position of his / her preference, the wearer of the smart glasses (a PC worker) needs to adjust the position of the virtual display every time the smart glasses are started, which is a hassle.

[0008] The present disclosure has been devised to solve the above-mentioned problems. That is, a main objective of the present disclosure is to provide a technology that reduces the effort required to adjust the position of a virtual display using smart glasses.

[0009] In order to achieve the above object, one aspect of the virtual display placement system according to the present disclosure comprises: smart glasses having a position calculation unit that calculates the placement position of the virtual display in a virtual space as virtual screen position information; and a computer device having an acquisition unit that acquires the virtual screen position information of the virtual display and stores it in a storage device, and a provision unit that provides the virtual screen position information of the virtual display stored in the storage device to the smart glasses; and the smart glasses place the virtual display at the placement position in the virtual space represented by the virtual screen position information provided by the computer device.

[0010] In addition, one aspect of the virtual display arrangement method according to the present disclosure is to use a computer device to acquire virtual screen position information of the virtual display from smart glasses having a position calculation unit that calculates the arrangement position of the virtual display in a virtual space as virtual screen position information, store the acquired information in a storage device, provide the virtual screen position information of the virtual display stored in the storage device to the smart glasses, and cause the smart glasses to arrange the virtual display at the arrangement position in the virtual space represented by the virtual screen position information.

[0011] Furthermore, in one aspect, the program storage medium according to the present disclosure stores a computer program that causes a computer to execute the following processes: a process of acquiring virtual screen position information of a virtual display from smart glasses having a position calculation unit that calculates the placement position of the virtual display in virtual space as virtual screen position information, and storing the information in a storage device; and a process of providing the virtual screen position information of the virtual display stored in the storage device to the smart glasses, so that the smart glasses can place the virtual display at the placement position in virtual space represented by the virtual screen position information.

[0012] According to the present disclosure, it is possible to reduce the effort involved in adjusting the placement of a virtual display using smart glasses.

[0013] FIG. 1 is a diagram illustrating a configuration of an embodiment of a virtual display arrangement system according to the present disclosure. FIG. 2 is a diagram illustrating an example of a virtual display using smart glasses. FIG. 3 is a diagram illustrating attention area history information. FIG. 4 is a diagram illustrating an example of highlighting using smart glasses. FIG. 5 is a sequence diagram illustrating an example of operation in the virtual display arrangement system. FIG. 6 is a sequence diagram illustrating another example of operation in the virtual display arrangement system. FIG. 7 is a diagram illustrating an example of a mode in which a line of sight trajectory is represented by smart glasses. FIG. 8 is a diagram illustrating another embodiment of the virtual display arrangement system. FIG. 9 is a sequence diagram illustrating yet another example of operation in the virtual display arrangement system.

[0014] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0015] First Embodiment A virtual display arrangement system according to a first embodiment of the present disclosure includes a computer device 2 and smart glasses 3 as shown in FIG.

[0016] The smart glasses 3 are glasses-type wearable terminals that are worn by a wearer 8 as shown in FIG. 2 and can place a virtual display 10 in a virtual space that is visible to the wearer 8. The virtual display is a display generated in the virtual space by the smart glasses 3. The smart glasses 3 are connected to a computer device 2 and operate in cooperation with the computer device 2, thereby displaying information processed by the computer device 2 on the virtual display 10. For example, when the wearer (in other words, a PC operator) 8 is working on writing a computer program (hereinafter also referred to as a program for short) using the computer device 2, the smart glasses 3 can display the code of the program on the virtual display 10.

[0017] 1, the smart glasses 3 include a control device 4, an eye camera 5, and an ambient capturing camera 6. The eye camera 5 is a camera used for gaze detection to detect the gaze of the wearer 8, and is provided on the frame of the smart glasses 3 so as to capture, for example, the eyeballs of the wearer 8. Information on the gaze of the wearer 8 detected by the eye camera 5 is compiled into gaze trajectory information representing the movement of the gaze and output from the smart glasses 3 to the computer device 2 at a predetermined timing. This gaze trajectory information is associated with user identification information that identifies the corresponding wearer 8 and time information.

[0018] The surroundings capturing camera 6 is a camera that captures images of the surroundings of the smart glasses 3. Here, the surroundings capturing camera 6 is a camera used to detect the movement of the fingers of the wearer 8 and the location of a display device (hereinafter also referred to as a physical display) 7 in the real space connected to the computer device 2.

[0019] The control device 4 includes a calculation device 40 and a storage device 60. The storage device 60 includes a storage medium for storing data and a computer program (program) 61. A specific example of the storage device 60 is a semiconductor memory, but the type and number of the storage devices 60 are not limited here, and a description thereof will be omitted. Furthermore, when multiple types of storage devices are provided in the smart glasses 3, they will be collectively referred to as the storage device 60.

[0020] The arithmetic device 40 is configured with a processor such as a CPU (Central Processing Unit). The arithmetic device 40 can have functions based on a program 61 by reading and executing the program 61 stored in a storage device 60. Here, the arithmetic device 40 has a virtual display control unit 41, a position calculation unit 42, an attention area detection unit 43, a wearing detection unit 44, and a highlighting unit 45.

[0021] The virtual display control unit 41 performs operation control related to the arrangement (display) of the virtual displays 10. For example, the virtual display control unit 41 arranges the virtual displays 10 in the virtual space created by the smart glasses 3 in response to an instruction from the wearer 8. The arrangement positions of these virtual displays 10 are arrangement positions in the virtual space represented by the virtual screen position information received from the computer device 2. Furthermore, information on the number of virtual displays 10 by the virtual display control unit 41 is, for example, input by the wearer 8 to the computer device 2 using an input device such as a keyboard, and transmitted from the computer device 2 to the smart glasses 3. The virtual display control unit 41 arranges the number of virtual displays 10 in the virtual space according to the number received from the computer device 2. Furthermore, when an instruction to change the number of virtual displays 10 is issued by the wearer 8, the virtual display control unit 41 deletes or adds virtual displays 10 in response to the instruction.

[0022] Furthermore, the virtual display control unit 41 detects the movement of the hands and fingers of the wearer 8 by performing image analysis on the images captured by the surrounding imaging camera 6. When the virtual display control unit 41 detects an operation by the wearer 8 to change the position of the virtual display 10 from the detection result of the movement of the hands and fingers of the wearer 8, the virtual display control unit 41 changes the position (display position) of the virtual display 10 in accordance with the operation of the wearer 8. Furthermore, when the virtual display control unit 41 detects an operation by the wearer 8 to change the size of the virtual display 10 from the detection result of the movement of the hands and fingers of the wearer 8, the virtual display control unit 41 changes the size of the virtual display 10 in accordance with the operation of the wearer 8.

[0023] Furthermore, the virtual display control unit 41 also controls the operation of the screen display of the virtual display 10. For example, the virtual display control unit 41 changes the resolution of the screen display on the virtual display 10 in response to an instruction from the wearer 8. Furthermore, when information on a screen display target to be displayed on the virtual display 10 is transmitted from the computer device 2 to the smart glasses 3, the virtual display control unit 41 controls the screen display of the transmitted information on the screen display target on the virtual display 10. When there are multiple virtual displays 10, the virtual display control unit 41 displays the information on the screen display target on a virtual display 10 specified by display screen information associated with the information on the screen display target. Specifically, for example, it is assumed that three virtual displays 10 are virtually arranged by the smart glasses 3. To distinguish between the three virtual displays 10, the three virtual displays 10 are referred to as virtual screen A, virtual screen B, and virtual screen C. Furthermore, it is assumed that the information to be displayed on the screen is generated when the wearer 8 executes an application program (hereinafter also referred to as an app) called "XXXX" on the computer device 2. Furthermore, it is assumed that the wearer 8 has specified that information on the screen display target related to the app "XXXX" be displayed on the virtual screen A. In such a case, the information on the screen display target related to the app "XXXX" is transmitted from the computer device 2 to the smart glasses 3, and the information on the screen display target is associated with information indicating that the information is to be displayed on the virtual screen A as display screen information. The virtual display control unit 41 displays the information on the screen display target related to the app "XXXX" on the virtual display 10 of the virtual screen A by referring to such display screen information. Note that the method for specifying the screen display destination when there are multiple virtual displays 10 is not limited to the method described above. For example, a screen display destination specification method may be adopted in which a different communication path (such as a socket) is set for each virtual display 10 between the computer device 2 and the smart glasses 3, and the smart glasses 3 determines the virtual display 10 to be the display destination based on information on the communication path.

[0024] The position calculation unit 42 calculates the placement position of the virtual display 10 in the virtual space as virtual screen position information. Here, the virtual screen position information is information that represents the placement position of the virtual display 10 using coordinates in a coordinate space that is set as follows. For example, in a workspace (real space) where a PC worker who is the wearer 8 of the smart glasses 3 performs PC work, a base point that serves as the origin of the coordinate space is determined in advance. The position calculation unit 42 sets a coordinate space with the base point as the origin. The base point may be a mark or the like placed on a desk in the PC worker's workspace, but here, one of the four vertices (e.g., the lower left vertex) of the rectangular screen of the physical display 7 connected to the computer device 2 is set as the base point.

[0025] That is, in this example, the position calculation unit 42 first transmits a request to the computer device 2 to display a two-dimensional code on the physical display 7 in order to recognize the physical display 7. When the computer device 2 detects the two-dimensional code displayed on the physical display 7 in response to this request from an image captured by the surrounding imaging camera 6, the position calculation unit 42 recognizes the screen on which the detected two-dimensional code is displayed as the physical display 7. The position calculation unit 42 then sets a coordinate space (e.g., a three-dimensional Cartesian coordinate system) with the origin set to a base point (e.g., the lower left vertex of the screen) determined by the recognized physical display 7. Furthermore, the position calculation unit 42 calculates (detects) coordinates representing the placement position of the virtual display 10 in the virtual space in the set coordinate space as virtual screen position information. The virtual screen position information of the virtual display 10 includes, for example, the coordinates of the four vertices (four corners) of the rectangular screen of the virtual display 10. In this case, the virtual screen position information includes not only the placement position of the virtual display 10 but also information on the tilt and screen size of the virtual display 10.

[0026] The position calculation unit 42 outputs the calculated virtual screen position information of the virtual display 10 to the computer device 2. The virtual screen position information of the virtual display 10 is associated with virtual screen identification information that identifies the corresponding virtual display 10. In addition, the virtual screen position information is also associated with resolution information that indicates the resolution of the corresponding virtual display 10.

[0027] The timing for the calculation of the virtual screen position information by the position calculation unit 42 and the output of the information to the computer device 2 as described above may be, for example, when the wearer 8 adjusts the position of the virtual display 10 and inputs information indicating the completion of the position adjustment (adjustment completion notification) into the smart glasses 3. The position adjustment of the virtual display 10 may be performed multiple times. In such a case, each time the adjustment completion notification is input by the wearer 8, the position calculation unit 42 calculates the virtual screen position information and outputs it to the computer device 2. The resolution of the virtual display 10 may also be changed. Each time the resolution is changed, resolution information indicating the changed resolution is output to the computer device 2 in association with virtual screen identification information. Another example of the timing for the calculation of the virtual screen position information by the position calculation unit 42 and the output of the information to the computer device 2 may be when the wearer 8 moves his / her hand to adjust the position of the virtual display 10 and then releases his / her hand from the virtual display 10. Another example of the timing is when it is detected that the layout position of the virtual display 10 has been changed and a set time (for example, one second) has elapsed without the next change.

[0028] The virtual screen position information of the virtual display 10 output from the position calculation unit 42 of the smart glasses 3 to the computer device 2 is stored in a storage device by the computer device 2. The virtual screen position information of the virtual display 10 stored in the storage device is associated with virtual screen identification information of the corresponding virtual display 10. Furthermore, this virtual screen position information is associated with user identification information that identifies the wearer 8 using the smart glasses 3 (virtual display 10). In other words, it is conceivable that the smart glasses 3 may be used by multiple users. It is also conceivable that the computer device 2 is a device shared by multiple users, and separate smart glasses 3 for different users are connected to the computer device 2. Since it is expected that users will adjust the placement position of the virtual display 10 according to their own preferences, user identification information is associated with the virtual screen position information of the virtual display 10.

[0029] After setting the coordinate space as described above, the position calculation unit 42 further calculates physical screen position information that indicates the layout position of the physical display 7 in the set coordinate space. The position calculation unit 42 also outputs the calculated physical screen position information of the physical display 7 to the computer device 2. The physical screen position information of the physical display 7 is associated with physical screen identification information that identifies the physical display 7.

[0030] In the virtual display arrangement system 1 of the first embodiment, during a start-up period after an app that controls the smart glasses 3 is launched, the smart glasses 3 are configured to request setting information, including virtual screen position information of the virtual display 10, from the computer device 2. The setting information is information used to arrange the virtual display 10 and includes virtual screen position information and resolution information. User identification information is associated with the setting information request that requests such setting information from the computer device 2. For example, a user authentication process is performed when the smart glasses 3 or the computer device 2 to which the smart glasses 3 are connected starts to be used, allowing the smart glasses 3 and the computer device 2 to acquire user identification information. The user identification information acquired in this manner is associated with the virtual screen position information of the virtual display 10. Note that if user authentication is not performed, the wearer 8 may input user identification information.

[0031] Furthermore, in the first embodiment, when the smart glasses 3 receive setting information for the virtual display 10 from the computer device 2, the virtual display control unit 41 of the smart glasses 3 uses information included in the received setting information to place the virtual display 10 in the virtual space created by the smart glasses 3. That is, the virtual display control unit 41 places the virtual display 10 at a position in the coordinate space represented by the virtual screen position information included in the setting information. Furthermore, the virtual display control unit 41 displays information on the virtual display 10 using characters and images at the resolution indicated by the resolution information included in the setting information.

[0032] The attention area detection unit 43 in the smart glasses 3 detects an attention area focused on by the wearer 8 of the smart glasses 3, using the gaze detection result by the eye camera 5. For example, if the gaze stays within an area narrower than a predetermined size for a period of time equal to or longer than a threshold, the attention area detection unit 43 detects the area where the gaze stayed as the attention area. Note that the threshold used to detect whether the gaze has stayed may be set appropriately taking into consideration the content of the PC work performed by the wearer 8, and the numerical value is not limited.

[0033] When the attention area detection unit 43 detects an attention area, it outputs attention area information representing the attention area to the computer device 2. The attention area information includes, for example, information on the time when attention started (start time), information on the length of time attention was maintained (attention duration), and information representing the position and size of the attention area (attention area range). The attention area information is associated with, for example, user identification information. When the attention area information is output, scan path information is also output to the computer device 2. The computer device 2 that has received the attention area information stores the attention area information in the attention area history information in the storage device, as shown in the image diagram of FIG. 3. The scan path information is also stored in the storage device by the computer device 2.

[0034] The wearing detection unit 44 detects that the smart glasses 3 are being worn by a person. There are multiple methods for this wearing detection, such as a method using a sensor or a method using an image captured by the eye camera 5, and any of these methods may be adopted, so a description thereof will be omitted.

[0035] When the wearing detection unit 44 detects that the smart glasses 3 are worn, it outputs a wearing detection notification to the computer device 2. Furthermore, when the wearing detection unit 44 no longer detects that the smart glasses 3 are worn, it outputs a removal detection notification to the computer device 2.

[0036] In the first embodiment, the computer device 2 has a configuration (function) for detecting that the wearer 8 has removed the smart glasses 3 to stop working and then put the smart glasses 3 back on and resumed the work, using the removal detection notification and the wearing detection notification output from the wearing detection unit 44. Furthermore, when detecting the resumption of such work or when an app that controls the smart glasses 3 is launched, the computer device 2 has a configuration for reading out the most recent attention area information from the attention area history information stored in the storage device and outputting it to the smart glasses 3.

[0037] When the highlighting unit 45 receives attention area information from the computer device 2 upon starting up the smart glasses 3 or resuming work, the highlighting unit 45 highlights the attention area represented by the received attention area information on the smart glasses 3. Examples of highlighting methods include a method of clarifying the attention area by coloring or by using a frame 11 as shown in FIG. 4. When coloring, it is preferable to use a color with high transparency so that the information displayed on the physical display 7 or the virtual display 10 can be seen.

[0038] The portion highlighted by the highlighting unit 45 when the smart glasses 3 are turned on or when work is resumed is the portion that the wearer (PC worker) 8 focused on the last time the smart glasses 3 were used, that is, before the work was finished or interrupted. This allows the PC worker to smoothly continue from the previous work by focusing on the highlighted portion when starting or resuming work, thereby improving work efficiency. In other words, the virtual display arrangement system 1 of the first embodiment can be said to support the work of the wearer (PC worker) 8. Furthermore, because the highlighting by the highlighting unit 45 is performed by the virtual display arrangement system 1, this can be achieved without modifying the app whose information is displayed on the physical display 7 or the virtual display 10.

[0039] 1, the computer device 2 is connected to a physical display 7. There are different types of computer devices 2, such as desktop and notebook types, and either type of computer device 2 may be used here. If the computer device 2 is a notebook type, the computer device 2 and the physical display 7 are integrated together.

[0040] The computer device 2 includes an arithmetic unit 20 and a storage device 50. The storage device 50 includes a storage medium for storing data (information) and a program 51. There are multiple types of storage devices, such as magnetic disk devices and semiconductor memory elements, and the type and number of storage devices 50 are not limited here, so a description thereof will be omitted. Furthermore, when the computer device 2 includes multiple storage devices, they will be collectively referred to as storage devices 50, and no distinction will be made between them.

[0041] In the first embodiment, the program 51 stored in the storage device 50 includes a program related to the virtual display arrangement system 1. The types of data (information) stored in the storage device 50 and related to the virtual display arrangement system 1 are roughly divided into setting files and history files. The setting file is information used to arrange the virtual display 10 in a virtual space using the smart glasses 3, and includes the virtual screen position information, resolution information of the virtual display 10, and physical screen position information as described above. The history file is history information related to the wearer 8 of the smart glasses 3, and includes gaze path information representing the movement of the wearer 8's line of sight, attention area history information, execution history information of apps used by the wearer 8 for PC work, and website browsing history information.

[0042] The arithmetic device 20 is configured with processors such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The arithmetic device 20 can have various functions based on a program 51 by reading and executing the program 51 stored in the storage device 50. Here, the arithmetic device 20 has an acquisition unit 21, a provision unit 22, an analysis unit 23, a work unit 24, and a display control unit 25 as functional units related to the virtual display arrangement system 1.

[0043] The work unit 24 executes apps that are used for PC work by the wearer (PC operator) 8 of the smart glasses 3. The work unit 24 also stores information representing the history of the executed apps in the storage device 50 as app execution history information, which is one of the history files.

[0044] The display control unit 25 performs display control to display information to be displayed on the physical display 7 or the virtual display 10 by an app being executed by the working unit 24. Here, as described above, the PC operator (wearer 8) operates the computer device 2 to specify the display on which the information to be displayed on the screen is to be displayed. For example, the PC operator (wearer 8) specifies the display on which the information to be displayed on the screen is to be displayed, such that information to be displayed on the screen related to app "XXXX" is displayed on the virtual screen A, which is the virtual display 10, and information to be displayed on the screen related to app "ZZZ" is displayed on the physical display 7. The display control unit 25 references such display specification information and displays the information to be displayed on the specified display by controlling the display operation of the physical display 7 and controlling the display of the virtual display 10 in cooperation with the virtual display control unit 41 of the smart glasses 3. Display history information related to the information to be displayed on the screen by the display control unit 25 is stored in the storage device 50 as one of history files. The display that displays the information to be displayed on the screen may be changed depending on the operation of the computer device 2, or the information to be displayed on the screen may be changed when the application "XXXX" is terminated and a new application "XABX" is started. When the relationship between the information to be displayed on the screen and the display is changed in this way, the display history information is updated in accordance with the change.

[0045] The acquisition unit 21 acquires information output from the smart glasses 3 and stores the acquired information in the storage device 50. In the first embodiment, as described above, virtual screen position information, physical screen position information, gaze trajectory information, and attention area information are output from the smart glasses 3 to the computer device 2. The virtual screen position information and physical screen position information are stored in the storage device 50 as setting files. In addition, the gaze trajectory information and attention area information are stored in the storage device 50 as gaze trajectory information and attention area history information, which are history files.

[0046] The analysis unit 23 analyzes the display portion that the wearer 8 of the smart glasses 3 focused on, using the attention area information, virtual screen position information, physical screen position information, and display history information. That is, the analysis unit 23 identifies the app related to the information displayed on the physical display 7 or the virtual display 10 that the wearer 8 focused on, or the focused information. As a specific example, the analysis unit 23 detects that the wearer 8 focused on the "MMM area" of the program in the app "XXXX" currently being displayed on the virtual screen A, which is the virtual display 10. The analysis result by the analysis unit 23 is stored in the storage device 50, for example, as information on the focus content, in association with the attention area information.

[0047] When the smart glasses 3 request setting information during a start-up period after activation of the smart glasses 3, the providing unit 22 returns (provides) to the smart glasses 3 the setting information (information including virtual screen position information and resolution information) read from the storage device 50 in response to this request. As described above, here, user authentication processing for the wearer 8 is performed when starting to use the computer device 2 or when starting up the smart glasses 3, and therefore user identification information corresponding to the wearer 8 is acquired. The providing unit 22 reads from the storage device 50 the virtual screen position information and resolution information associated with the user identification information corresponding to the wearer 8, and provides the read information to the smart glasses 3. Note that when the virtual screen position information and resolution information associated with the user identification information of the wearer 8 are not stored in the storage device 50, for example, the providing unit 22 provides the smart glasses 3 with pre-set initial setting virtual screen position information and resolution information.

[0048] The setting information provided to the smart glasses 3 by the providing unit 22 is used, and the virtual display control unit 41 of the smart glasses 3 virtually places the virtual display 10 in real space.

[0049] Furthermore, during a start-up period after the smart glasses 3 are activated, the providing unit 22 reads, from the display history information in the storage device 50, history information related to the information of the screen display target that the wearer 8 had displayed on the physical display 7 or the virtual display 10 when the wearer 8 finished the previous task. Furthermore, based on the read history information, the providing unit 22 starts, via the working unit 24, an app related to the task that the wearer 8 was performing before finishing the previous task, and further outputs information about the task content that the wearer 8 was performing before finishing the previous task using the app to the display control unit 25. As a result, information related to the task that the wearer 8 was performing before finishing the previous task is displayed on the physical display 7 or the virtual display 10. In other words, the screen display of the app related to the task that the wearer 8 was performing before finishing the previous task is restored on the physical display 7 or the virtual display 10.

[0050] Furthermore, during a start-up period after the smart glasses 3 are activated, the providing unit 22 extracts the most recent attention area information from the attention area history information associated with the user identification information corresponding to the wearer 8, and provides the information to the smart glasses 3. As a result, the highlighting unit 45 highlights on the smart glasses 3 the area that the wearer 8 was paying attention to before the end of the previous task.

[0051] Furthermore, the providing unit 22 detects that the wearer 8 has removed the smart glasses 3 to interrupt their work and then put the smart glasses 3 back on and resumed the work, using the wearing detection notification and the removal detection notification output by the wearing detection unit 44 of the smart glasses 3. When detecting that the wearer 8 has resumed their work in this way, the providing unit 22 also extracts the most recent attention area information from the attention area history information associated with the user identification information corresponding to the wearer 8, as described above, and provides the information to the smart glasses 3. As a result, the highlighting unit 45 highlights on the smart glasses 3 the area that the wearer 8 was focusing on before the interruption.

[0052] Furthermore, when the providing unit 22 receives a request from the smart glasses 3 to display a two-dimensional code on the physical display 7, it generates a two-dimensional code including screen identification information, resolution information, and size information. Furthermore, the providing unit 22 causes the display control unit 25 to display the generated two-dimensional code on the physical display 7. As a result, the position calculation unit 42 of the smart glasses 3 recognizes the physical display 7 and sets a coordinate space, and calculates the coordinates of the physical display 7 and the virtual display 10 in the set coordinate space.

[0053] The virtual display arrangement system 1 has the configuration described above. Next, an example of the operation of the virtual display arrangement system 1 will be described with reference to FIGS. 5 and 6. FIG. 5 is a sequence diagram showing an example of the operation of the virtual display arrangement system 1 when the smart glasses 3 are started up. Here, it is assumed that the storage device 50 of the computer device 2 has already stored therein setting files such as virtual screen position information, resolution information, and physical screen position information related to the wearer 8, as well as history files such as gaze trajectory information, attention area history information, and application execution history information. Note that if the storage device 50 has not yet stored therein setting files or history files related to the wearer 8, pre-set initial settings such as virtual screen position information and resolution information may be applied.

[0054] For example, the computer device 2 starts executing an app that controls the smart glasses 3, and similarly starts communication with a smart glasses app launched on the smart glasses 3 (step 101). Furthermore, based on a setting file and a history file stored in the storage device 50 and associated with user identification information corresponding to the wearer 8, the computer device 2 generates the number of virtual displays used the previous time at the resolution used the previous time. Furthermore, the computer device 2 starts an app corresponding to the information displayed on the physical display 7 or the virtual display 10 at the time the previous task was completed, and restores the display state of the physical display 7 or the virtual display 10 at the time the previous task was completed. The providing unit 22 then provides the smart glasses 3 with setting information including virtual screen position information and resolution associated with user identification information corresponding to the wearer 8 of the smart glasses 3 (step 102). Upon receiving the provided information, the smart glasses 3 store the received setting information including the virtual screen position information and resolution in the storage device 60 (step 103).

[0055] Thereafter, the position calculation unit 42 in the smart glasses 3 requests the computer device 2 to display a two-dimensional code (step 104). In response, the provision unit 22 and the display control unit 25 of the computer device 2 perform display control to display a two-dimensional code for physical display recognition on the physical display 7 (step 105). The position calculation unit 42 of the smart glasses 3 recognizes the physical display 7 starting from the display of the two-dimensional code on the physical display 7 by image analysis of the image captured by the surrounding imaging camera 6 (step 106).

[0056] As a result, the position calculation unit 42 identifies the position of the physical display 7 in the virtual space, and a coordinate space is set with the origin defined by the physical display 7. In the first embodiment, the computer device 2 is described as being connected to one physical display 7, but multiple physical displays 7 may be connected to the computer device 2. When multiple physical displays 7 are connected to the computer device 2, the above-described two-dimensional code is displayed on each of the multiple physical displays 7. The displayed two-dimensional code contains information for identifying and recognizing the physical display 7 displaying the two-dimensional code. As a result, the position calculation unit 42 sets a coordinate space with the origin defined by the main physical display 7, as described above, based on the display of the two-dimensional code of the main physical display 7 recognized from the information from the two-dimensional code. Furthermore, with respect to the sub-physical display 7 (i.e., a physical display 7 other than the main physical display 7) recognized from the information from the two-dimensional code, the position calculation unit 42 determines physical screen position information representing the layout position of the sub-physical display 7 using coordinates in the set coordinate space, and outputs the information to the computer device 2 (step 107). Upon receiving the physical screen position information, the computer device 2 updates the physical screen position information in the storage device 50 (in other words, updates the setting file (step 108)).

[0057] Meanwhile, the virtual display control unit 41 of the smart glasses 3 arranges the virtual display 10 in the set coordinate space (step 109) after the coordinate space has been set. The arrangement and display of the virtual display 10 uses the information provided to the smart glasses 3 from the computer device 2 and stored in the storage device 60 by the operations of steps 102 and 103.

[0058] Furthermore, the providing unit 22 of the computer device 2 provides the smart glasses 3 with information related to the restored display on the virtual display 10 as described above. As a result, the information that the wearer 8 of the smart glasses 3 displayed on the virtual display 10 at the time of completing the previous work is displayed (restored) on the virtual display 10. Furthermore, the providing unit 22 extracts the most recent attention area information (i.e., information on the area that was focused on before the previous work was completed) from the attention area history information in the storage device 50. Then, the providing unit 22 provides the attention area information that was focused on before the previous work was completed to the smart glasses 3 (step 110). As a result, the highlighting unit 45 highlights the attention area represented by the received attention area information (step 111).

[0059] After that, for example, suppose the wearer 8 adjusts (changes) the position, size, resolution, etc. of the virtual display 10 and inputs a notification of the completion of this adjustment to the smart glasses 3. In this case, the position calculation unit 42 of the smart glasses 3 detects the completion of adjustment of the virtual display 10 based on the notification of the completion of adjustment (step 112). Then, the position calculation unit 42 outputs setting information of the adjusted virtual display (information including virtual screen position information and resolution information) to the computer device 2 (step 113). Upon receiving the setting information from the smart glasses 3, the computer device 2 updates the setting file, such as the virtual screen position information, in the storage device 50 (step 114). Through such operations of the computer device 2 and the smart glasses 3, information related to the display of the virtual display 10, such as when the smart glasses 3 are started up, is set and updated.

[0060] 6 is a sequence diagram illustrating an example of an operation related to highlighting an attention area in the virtual display arrangement system 1. For example, assume that information is displayed on the physical display 7 by the display control unit 25 of the computer device 2, and information is also displayed on the virtual display 10 through cooperation between the display control unit 25 of the computer device 2 and the virtual display control unit 41 of the smart glasses 3. In this case, assume that the attention area detection unit 43 detects an attention area focused on by the wearer 8 using line-of-sight information detected by the eye camera 5 (step 201). As a result, the attention area detection unit 43 outputs attention area information representing the detected attention area to the computer device 2 (step 202). Upon receiving the attention area information, the computer device 2 stores the received attention area information as attention area history information, which is a history file in the storage device 50 (i.e., updates the history file (step 203)). Similarly, thereafter, when the attention area detection unit 43 detects an attention area (step 204), it outputs attention area information representing the detected attention area to the computer device 2 (step 205). As a result, the computer device 2 stores the received attention area information as attention area history information, which is a history file in the storage device 50 (updates the history file (step 206)).

[0061] After that, when the wearing detection unit 44 no longer detects that the wearer 8 is wearing the smart glasses 3, that is, when it detects that the smart glasses 3 have been removed (step 207), it outputs a removal detection notification to the computer device 2 (step 208). After that, when the wearing detection unit 44 detects that the wearer 8 is wearing the smart glasses 3 (step 209), it outputs a wearing detection notification to the computer device 2 (step 210).

[0062] The providing unit 22 in the computer device 2 detects that the wearer 8 has resumed work after interrupting it by receiving the removal detection notification and the wearing detection notification, and extracts the most recent attention area information from the attention area history information in the storage device 50. In other words, the providing unit 22 extracts information indicating the attention area that the wearer 8 was focusing on before the work was interrupted. Then, the providing unit 22 provides the extracted attention area information to the smart glasses 3 (step 211).

[0063] The highlighting unit 45 of the smart glasses 3 highlights the attention area represented by the attention area information provided by the providing unit 22 (step 212).

[0064] The virtual display arrangement system 1 in the first embodiment is configured as described above. As a result, the virtual display arrangement system 1 can achieve the following effects. That is, the virtual display arrangement system 1 stores virtual screen position information, which indicates the arrangement position of the virtual display 10 in the virtual space by the smart glasses 3, in the computer device 2. Then, when the smart glasses 3 are started up, the virtual display arrangement system 1 arranges (displays) the virtual display 10 at a position using the virtual screen position information. As a result, when the smart glasses 3 are started up, the virtual display 10 is arranged at the position where it was arranged the last time the smart glasses 3 were used. In other words, this eliminates the need to adjust the arrangement position of the virtual display 10 to the same position as the last time the smart glasses 3 were used when the smart glasses 3 are started up. In other words, the virtual display arrangement system 1 can reduce the effort required to adjust the arrangement position of the virtual display 10 by the smart glasses 3.

[0065] Furthermore, in the virtual display arrangement system 1 according to the first embodiment, history information relating to the information to be displayed on the physical display 7 or the virtual display 10 is stored in the storage device 50 of the computer device 2. Furthermore, when the smart glasses 3 are started up, the computer device 2 starts up an app related to the task that was being performed before the previous task was completed based on the history information, and further uses the app to display information about the task that the wearer 8 was performing before the previous task was completed on the physical display 7 or the virtual display 10. That is, the screen display of the app related to the task that the wearer 8 was performing before the previous task was completed is restored on the physical display 7 or the virtual display 10. This reduces the effort required of the wearer 8 to restore the screen display to the same state as when the smart glasses 3 were last used when they were started up.

[0066] Furthermore, in the first embodiment, the virtual display arrangement system 1 detects an attention area focused by the wearer 8 of the smart glasses 3 and stores attention area information representing the attention area in the computer device 2. Then, when the smart glasses 3 are started up or resumed after an interruption, the attention area information is used to highlight the area on the smart glasses 3 that the wearer 8 focused on before the previous work was completed or interrupted. This allows the wearer 8 to smoothly recall the content of the PC work they were performing before the previous work was completed or interrupted, and to start working more quickly. Therefore, the virtual display arrangement system 1 can improve the work efficiency of the wearer 8. Furthermore, because the attention area is highlighted in the smart glasses 3, the attention area can be highlighted without changing the app whose processing content is being displayed on the physical display 7 or the virtual display 10.

[0067] Furthermore, in the first embodiment, the information related to the smart glasses 3 stored in the storage device 50 by the computer device 2 is associated with user identification information. Furthermore, information such as virtual screen position information that the computer device 2 reads from the storage device 50 and outputs to the smart glasses 3 is information associated with user identification information corresponding to the wearer 8 of the smart glasses 3. The virtual display 10 is arranged using such information such as virtual screen position information. In other words, even if it is assumed that multiple users will wear the smart glasses 3 and use the computer device 2, information related to the smart glasses 3, such as virtual screen position information, is stored for each user. Therefore, the arrangement position of the virtual display 10 when the smart glasses 3 are started up and the highlighting of the attention area when resuming work after an interruption can be controlled according to the user.

[0068] (Variations of the First Embodiment) In the above example, examples of the manner in which the highlighting unit 45 highlights the region of interest include coloring the region of interest and surrounding it with the frame line 11. Alternatively, for example, if the content displayed in the region of interest is program code or text, the highlighting unit 45 may highlight the program code or text in the region of interest by underlining it.

[0069] In the above example, when the smart glasses 3 are turned on and when the wearer 8 resumes work after an interruption, the highlighting unit 45 highlights the focus area before the previous work was completed or the focus area before the interruption. In addition, the highlighting unit 45 may also perform a highlighting operation in the following cases. Examples of the additional highlighting timing include when the wearer 8 shifts their gaze from the virtual display 10 to another virtual display 10 or a physical display 7 and then returns their gaze to the virtual display 10, or when the wearer 8 shifts their gaze from the physical display 7 to the virtual display 10 and then returns their gaze to the physical display 7. In such cases, the highlighting unit 45 highlights the focus area that the wearer 8 focused on before shifting their gaze on the virtual display 10 or the physical display 7 to which the wearer 8 returned their gaze. Note that if the time between shifting their gaze to another display and returning their gaze to the original display is shorter than a predetermined highlighting determination time (e.g., several seconds), it cannot be said that the wearer 8 intentionally switched their gaze to another display, and therefore the highlighting unit 45 does not perform highlighting.

[0070] Furthermore, the wearer 8 may be able to select between a mode in which highlighting is performed by the highlighting unit 45 and a mode in which highlighting is not performed.

[0071] Second Embodiment A second embodiment according to the present disclosure will be described below. In the description of the second embodiment, components having the same names as those of the virtual display arrangement system in the first embodiment will be assigned the same reference numerals, and a duplicate description of the common components will be omitted.

[0072] The virtual display arrangement system 1 of the second embodiment has the following configuration in addition to the configuration of the first embodiment. That is, it is expected that some users will change the number and arrangement positions of virtual displays depending on the work they are doing. In consideration of this, the position calculation unit 42 is configured to accept, through an operation by the wearer 8, the assignment of a file name to the setting information indicating the number, arrangement positions, and resolution of the virtual displays before outputting the setting information to the computer device 2. As a result, the computer device 2 stores, in the storage device 50, the setting information related to the virtual display 10 to which the file name has been assigned by the wearer 8. That is, the storage device 50 stores multiple pieces of virtual screen position information for the virtual displays, which are different from one another.

[0073] Furthermore, in the second embodiment, the providing unit 22 has the following configuration in addition to the configuration of the first embodiment. That is, when the smart glasses 3 are started up or when the wearer 8 requests to read the setting information, the providing unit 22 reads out the setting information (information including virtual screen position information) to which a file name has been assigned by the wearer 8 from the storage device 50. Furthermore, the providing unit 22 causes the display control unit 25 or the virtual display control unit 41 to display a list of file names of the read setting information on the physical display 7 or the virtual display 10. This display also displays a message prompting the wearer 8 to select a file name to be used for setting the placement position of the smart glasses 3, etc. Furthermore, when the providing unit 22 detects that a file name has been selected by the wearer 8, it provides the smart glasses 3 with setting information corresponding to the file name selected by the wearer 8. That is, the providing unit 22 provides the smart glasses 3 with virtual screen position information specified by the wearer 8 from among multiple pieces of virtual screen position information stored in the storage device 50. As a result, the virtual display 10 is arranged by the smart glasses 3 with settings according to the request of the wearer 8.

[0074] The virtual display arrangement system 1 of the second embodiment has the configuration described above, and can, for example, use smart glasses 3 to arrange virtual displays 10 in positions and numbers that suit the wearer 8's needs, without causing any effort to the wearer 8.

[0075] <Other Embodiments> The present disclosure is not limited to the first and second embodiments and may employ various embodiments. For example, in the first and second embodiments, application execution history and website browsing history information executed by the wearer 8 (PC operator) are stored in the storage device 50 of the computer device 2. In addition, the computer device 2 may be connected to a database 9 as represented by the dotted line in FIG. 1 , and application execution history and website browsing history information executed by the wearer 8 (PC operator) may be stored in the database 9. The database 9 may be readable and writable by multiple computers. For example, the database 9 may enable a business manager of the wearer 8 to access application execution history and website browsing history information related to the wearer 8. In other words, the business manager of the wearer 8 can view application execution history and website browsing history information related to the wearer 8 and be aware of the work status of the wearer 8.

[0076] Furthermore, the computer device 2 may transmit application execution history and website browsing history information related to the wearer 8 to a predetermined destination (e.g., a business manager of the wearer 8) when the application that controls the smart glasses 3 is closed. Such a configuration is effective as a means for the business manager to grasp the work status of the wearer 8 (PC operator).

[0077] Furthermore, in the first and second embodiments, the computer device 2 is connected to the physical display 7, and the wearer 8 (PC worker) can perform work using not only the virtual display 10 of the smart glasses 3 but also the physical display 7. On the other hand, for example, in a situation where it is difficult to arrange the physical display 7 in real space, the computer device 2 does not need to be connected to the physical display 7. Even in such an environment where it is difficult to install the physical display 7, the wearer 8 (PC worker) can perform PC work using the virtual display 10 of the smart glasses 3, with the virtual display 10 at a desired position, size, and number.

[0078] In such a case, since there is no physical display, the position calculation unit 42 in the first and second embodiments cannot obtain a base point determined by the physical display used to set the coordinate space. In such a case, for example, a mark or the like that serves as a base point in the real space used to set the coordinate space is determined in advance, and a sticker with such a mark is affixed to, for example, a desk. The position calculation unit 42 extracts such a mark as a base point from the image captured by the surrounding image capturing camera 6 and sets a coordinate space with the base point as the origin.

[0079] Furthermore, in addition to the configuration of the smart glasses 3 in the first and second embodiments, for example, the virtual display control unit 41 may display a line of sight (line of sight trajectory) 13 as represented by the dotted line in Figure 7 on the smart glasses 3 in response to a request from the wearer 8.

[0080] Furthermore, in the first and second embodiments, an example is shown in which the computer device 2 and the smart glasses 3 are separate entities. Alternatively, the computer device 2 may be integrated with the smart glasses 3. In this case, the smart glasses 3 may have a function to receive information such as setting information from the outside via a wireless communication function or the like. Furthermore, the storage device is not limited to storage devices built into the computer device 2 and the smart glasses 3, and may be, for example, an external storage device, a database on an external server, a database on the cloud, or the like. Furthermore, a combination of two or more of these may be used as a storage device. Furthermore, the storage devices that store programs, setting information, history information, and the like may be the same or different.

[0081] Furthermore, in the first and second embodiments, the smart glasses 3 are provided with an eye camera 5 to acquire gaze information, but the eye camera 5 in the smart glasses 3 may be omitted if gaze information of the wearer 8 can be acquired by other means or if gaze information is not used by omitting the highlighting function, for example. Also, with regard to the surroundings capturing camera 6, the surroundings capturing camera 6 in the smart glasses 3 may be omitted if the surrounding situation can be acquired by other means, for example. Furthermore, multiple physical displays 7 may be connected to the computer device 2. Specifically, there is an example in which a sub-display is connected to a notebook computer device.

[0082] Furthermore, the position calculation unit 42 may acquire the coordinates of the physical display 7 as follows. For example, a two-dimensional code including display size information is displayed at the center of the screen of the physical display 7. The position calculation unit 42 may acquire the coordinates of the physical display 7 by calculating the coordinates of the four corners of the screen together with the coordinates of the two-dimensional code. Alternatively, the smart glasses 3 may read a base point displayed at the end point of a diagonal line of the screen of the physical display 7 using, for example, the surrounding imaging camera 6, and the position calculation unit 42 may acquire the coordinates of the physical display 7 by calculating the coordinates of the four corners of the screen of the physical display 7 using the read information. Furthermore, instead of displaying a two-dimensional code on the screen, the position calculation unit 42 may recognize the physical display 7 by recognizing a specific object (such as a sticker) attached to a corner of the physical display 7 and calculate the coordinates of the physical display 7. Note that the method of calculating the coordinates of the physical display 7 is not limited to these. As a result, the physical display 7 can be added to the target for acquiring the attention area, similar to the virtual display 10, and the calculated coordinates can be used to acquire the information. In a VR environment, by displaying a base point on the display of a virtual laptop computer, it can be used in combination with a virtual display in the first or second virtual display arrangement system. In this case, the display of the virtual laptop computer having the base point can be used as the target for obtaining attention area information, but since these coordinates are already determined, coordinate calculation is not required.

[0083] Furthermore, the first and second virtual display arrangement systems can also be applied to the following environments. For example, in a free address office where individuals are not assigned fixed seats, if there are multiple desks of the same shape, by placing the same mark dedicated to that area (i.e., a marker that serves as a base point when setting coordinates in the virtual space) at the same position on each desk, the virtual display can be displayed in the same arrangement regardless of which desk the wearer 8 is at and reads the mark with the smart glasses 3. Also, for example, if there are multiple desk shapes in a free address office, such as an area with long desks and an area with round desks, by varying the marks depending on the pattern, the virtual display can be displayed in an arrangement linked to the shape of each desk.

[0084] Furthermore, the virtual display arrangement system according to the present disclosure may also have a configuration as shown in Fig. 8. That is, one embodiment of the virtual display arrangement system shown in Fig. 8 includes a computer device 80 and smart glasses 90. The smart glasses 90 are glasses-type wearable terminals that are capable of arranging a virtual display in a virtual space created by the smart glasses 90. The smart glasses 90 include a position calculation unit 91. The position calculation unit 91 calculates the arrangement position of the virtual display in the virtual space as virtual screen position information.

[0085] The computer device 80 is connected to the smart glasses 90. The computer device 80 includes an acquisition unit 81 and a provision unit 82. The acquisition unit 81 acquires virtual screen position information of the virtual display and stores it in a storage device. The storage device may be a storage device built into the computer device, or may be a storage device such as a database connected to the computer device. The provision unit 82 provides the virtual screen position information of the virtual display stored in the storage device to the smart glasses. The smart glasses 90 place the virtual display at a placement position in the virtual space represented by the virtual screen position information provided from the computer device 80.

[0086] An example of the operation of such a virtual display arrangement system will be described with reference to FIG. 9 . For example, suppose that a wearer of smart glasses adjusts the arrangement position of a virtual display virtually arranged by the smart glasses. In such a case, the position calculation unit 91 of the smart glasses 90 calculates the arrangement position of the virtual display in the virtual space as virtual screen position information (step 501). This calculated virtual screen position information is output from the smart glasses 90 to the computer device 2, for example (step 502). The acquisition unit 81 of the computer device 2 acquires the output virtual screen position information (step 503). Then, the acquisition unit 81 acquires the virtual screen position information and stores (preserves) it in a storage device (step 503).

[0087] After that (for example, the next day), when the smart glasses 90 are started, the providing unit 82 of the computer device 80 provides the virtual screen position information stored in the storage device to the smart glasses 90 (step 505). As a result, the smart glasses 90 uses the virtual screen position information to position the virtual display (step 506).

[0088] In the virtual display arrangement system shown in FIG. 8 , virtual screen position information, which is information about the arrangement position of the virtual display in the virtual space by the smart glasses 90, is acquired by an acquisition unit 81 of a computer device 80 and stored in a storage device. Furthermore, for example, when the smart glasses 90 are started up, the provision unit 82 provides the virtual screen position information from the computer device 80 to the smart glasses 90, so that the smart glasses 90 can arrange and display the virtual display at the arrangement position in the virtual space represented by the virtual screen position information. In other words, when the smart glasses 90 are started up, the virtual display can be arranged without the effort of the smart glasses wearer adjusting the position of the virtual display to the arrangement position in the virtual space represented by the virtual screen position information. Therefore, this virtual display arrangement system can achieve the effect of reducing the effort required to adjust the arrangement position of the virtual display by the smart glasses.

[0089] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes: (Supplementary Note 1) A virtual display arrangement system comprising: smart glasses having a position calculation unit that calculates a placement position of a virtual display in a virtual space as virtual screen position information; and a computer device having: an acquisition unit that acquires the virtual screen position information of the virtual display and stores it in a storage device; and a provision unit that provides the virtual screen position information of the virtual display stored in the storage device to the smart glasses, wherein the smart glasses arrange a virtual display at a placement position in the virtual space represented by the virtual screen position information provided by the computer device. (Supplementary Note 2) The smart glasses further include a wearing detection unit that detects that the smart glasses are worn by a wearer, an attention area detection unit that detects an attention area that the wearer focuses on using the detection result of the wearer's line of sight, and a highlighting unit that, when the wearer wears the smart glasses, highlights the attention area that the wearer focused on the last time they were used, and the acquisition unit of the computer device further acquires attention area information representing the attention area of ​​the wearer from the smart glasses and stores it in the storage device, and the provision unit further provides the attention area information stored in the storage device to the smart glasses when the wearer wears the smart glasses. (Supplementary Note 3) The virtual display arrangement system of Supplementary Note 1, wherein the position calculation unit of the smart glasses detects a predetermined base point in real space, sets a coordinate space with the base point as an origin, and calculates virtual screen position information representing the arrangement position of the virtual display using coordinates in the coordinate space.(Supplementary Note 4) The virtual display arrangement system according to Supplementary Note 3, wherein the computer device is connected to a physical display which is a display device in real space, the providing unit of the computer device further displays a two-dimensional code including information for recognizing the physical display on the physical display, and the position calculation unit of the smart glasses calculates virtual screen position information of the virtual display by detecting a base point determined by the physical display recognized using the two-dimensional code displayed on the physical display and setting a coordinate space with the base point as an origin. (Supplementary Note 5) The virtual screen position information acquired from the smart glasses by the acquiring unit of the computer device and stored in the storage device is associated with user identification information which identifies a user who is a wearer of the smart glasses related to the virtual screen position information, and the providing unit reads out the virtual screen position information from the storage device using the user identification information corresponding to the wearer of the smart glasses and provides it to the smart glasses. (Supplementary Note 6) The virtual display arrangement system according to Supplementary Note 1, wherein the storage device stores a plurality of different virtual screen position information of the virtual display, and the providing unit of the computer device provides the smart glasses with virtual screen position information specified by the wearer from among the plurality of virtual screen position information stored in the storage device. (Supplementary Note 7) The virtual display arrangement system according to Supplementary Note 5, wherein the storage device stores different virtual screen position information for each user, and the providing unit of the computer device provides the smart glasses with virtual screen position information associated with a combination of a predetermined base point and user identification information of the user wearing the smart glasses. (Supplementary Note 8) The virtual display arrangement system according to Supplementary Note 4, wherein the smart glasses read one or more marks displayed or posted on a physical display connected to the computer device, and the smart glasses acquire coordinates of four corners of the screen of the physical display based on at least one of the coordinates of the read marks and information included in the marks, thereby making the physical display a target for acquiring attention area information.(Supplementary Note 9) A virtual display arrangement method, in which a computer device acquires virtual screen position information of a virtual display from smart glasses having a position calculation unit that calculates the arrangement position of the virtual display in a virtual space as virtual screen position information, and stores the acquired virtual screen position information in a storage device, provides the virtual screen position information of the virtual display stored in the storage device to the smart glasses, and causes the smart glasses to arrange the virtual display at the arrangement position in the virtual space represented by the virtual screen position information. (Supplementary Note 10) The storage device stores different virtual screen position information for each user, and the computer device reads out virtual screen position information from the storage device, which is linked to a combination of a predetermined base point and user identification information of a user who is wearing the smart glasses, and provides the readout to the smart glasses. (Supplementary Note 11) The method for arranging a virtual display according to Supplementary Note 9, wherein the smart glasses read one or more marks displayed or posted on a physical display connected to the computer device, and the smart glasses acquire the coordinates of the four corners of the screen of the physical display based on at least one of the coordinates of the read mark and information contained in the mark, thereby making the physical display the target for acquiring attention area information. (Supplementary Note 12) A program storage medium storing a computer program that causes a computer to execute the following processes: a process of acquiring virtual screen position information of a virtual display from smart glasses having a position calculation unit that calculates the placement position of the virtual display in a virtual space as virtual screen position information, and storing the acquired virtual screen position information in a storage device; and a process of providing the virtual screen position information of the virtual display stored in the storage device to the smart glasses, in order to place the virtual display at the placement position in the virtual space represented by the virtual screen position information by the smart glasses.

[0090] The present invention has been described above using the above-described embodiments as exemplary examples. However, the present invention is not limited to the above-described embodiments. In other words, the present invention can be applied in various aspects that can be understood by a person skilled in the art within the scope of the present invention.

[0091] This application claims priority based on Japanese Patent Application No. 2023-057036, filed on March 31, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0092] REFERENCE SIGNS LIST 1 Virtual display arrangement system 2, 80 Computer device 3, 90 Smart glasses 7 Physical display 10 Virtual display 21, 81 Acquisition unit 22, 82 Provision unit 23 Analysis unit 42 Position calculation unit 43 Attention area detection unit 44 Wearing detection unit 45 Highlighting unit

Claims

1. smart glasses having a position calculation means for calculating the placement position of the virtual display in a virtual space as virtual screen position information; a computer device having an acquisition means for acquiring virtual screen position information of a virtual display and storing the acquired information in a storage device, and a provision means for providing the virtual screen position information of the virtual display stored in the storage device to the smart glasses; Equipped with The smart glasses are a virtual display placement system that places a virtual display at a placement position in a virtual space represented by virtual screen position information provided by the computer device.

2. The smart glasses further include a wear detection means for detecting that the smart glasses are worn by a wearer, an attention area detection means for detecting an attention area that the wearer is paying attention to using a detection result of the wearer's line of sight, and a highlighting means for highlighting an attention area that the wearer was paying attention to the last time the smart glasses were used when the wearer puts on the smart glasses, The acquisition means of the computer device further acquires attention area information representing an attention area of ​​the wearer from the smart glasses and stores the information in the storage device, and the provision means further provides the attention area information stored in the storage device to the smart glasses when the wearer wears the smart glasses. The virtual display arrangement system according to claim 1 .

3. The position calculation means of the smart glasses detects a predetermined base point in real space, sets a coordinate space with the base point as the origin, and calculates virtual screen position information representing the arrangement position of a virtual display using coordinates in the coordinate space. The virtual display arrangement system according to claim 1 .

4. the computer device is connected to a physical display that is a display device in real space; The providing means of the computer device further displays a two-dimensional code including information for recognizing the physical display on the physical display; The position calculation means of the smart glasses detects a base point determined by the physical display recognized using a two-dimensional code displayed on the physical display, and calculates virtual screen position information of the virtual display by setting a coordinate space with the base point as the origin. The virtual display arrangement system according to claim 3 .

5. The virtual screen position information that the acquisition means of the computer device acquires from the smart glasses and stores in the storage device is associated with user identification information that identifies a user who is wearing the smart glasses related to the virtual screen position information, and the providing means reads out the virtual screen position information from the storage device using the user identification information corresponding to the wearer of the smart glasses and provides the information to the smart glasses. The virtual display arrangement system according to claim 1 .

6. The storage device stores a plurality of different virtual screen position information of the virtual display, and the providing means of the computer device provides the smart glasses with virtual screen position information designated by the wearer from among the plurality of virtual screen position information stored in the storage device. The virtual display arrangement system according to claim 1 .

7. The storage device stores different virtual screen position information for each user, The providing means of the computer device provides the smart glasses with virtual screen position information associated with a combination of a predetermined base point and user identification information of a user who is wearing the smart glasses. The virtual display arrangement system according to claim 5 .

8. The smart glasses read one or more marks displayed or displayed on a physical display connected to the computing device; The virtual display arrangement system of claim 4, wherein the smart glasses acquire the coordinates of the four corners of the screen of the physical display based on at least one of the coordinates of the read mark and information contained in the mark, thereby making the physical display the target for acquiring attention area information.

9. By a computer device, acquiring virtual screen position information of the virtual display from smart glasses having a position calculation means for calculating the placement position of the virtual display in the virtual space as virtual screen position information, and storing the acquired information in a storage device; A method for arranging a virtual display, which provides virtual screen position information of the virtual display stored in the storage device to the smart glasses, and causes the smart glasses to arrange the virtual display at a position in the virtual space represented by the virtual screen position information.

10. A process of acquiring virtual screen position information of the virtual display from smart glasses having a position calculation means that calculates the placement position of the virtual display in a virtual space as virtual screen position information and storing the acquired information in a storage device; a process of providing the smart glasses with virtual screen position information of the virtual display stored in the storage device so that the smart glasses can place the virtual display at a position in the virtual space represented by the virtual screen position information; A computer program that causes a computer to execute the above.