Screen sharing control device, screen sharing control system, screen sharing control method and program

Abstract

To start screen sharing at appropriate timing.SOLUTION: A screen sharing control device comprises an acquisition unit, a determination unit, and a transmission unit. The acquisition unit acquires information indicating display timing of a shared object screen. Or, the acquisition unit acquires work information indicating the work state of a user of a sharing destination of the shared object screen. The determination unit determines timing to start sharing of the shared object screen, according to the information indicating the display timing or the work information. The transmission unit transmits a control signal for displaying the shared object screen, to at least one terminal device which is the sharing destination of the shared object screen, according to the determined timing.SELECTED DRAWING: Figure 26

Description

[Technical field]

[0001] The present disclosure relates to a screen sharing control device, a screen sharing control system, a screen sharing control method, and a program. [Background technology]

[0002] A technique for sharing a user's screen is disclosed in Patent Document 1. The online conference support device of Patent Document 1 starts sharing a screen when a user who wishes to share the screen selects a "screen sharing" button. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 091516 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 starts sharing at a timing determined by a user who will share the screen. However, the timing determined by a user who will share the screen is not necessarily an appropriate timing to start sharing.

[0005] An object of the present disclosure is to provide a screen sharing control device, a screen sharing control system, a screen sharing control method, and a program that start screen sharing at appropriate timing. [Means for solving the problem]

[0006] A screen sharing control device of one embodiment of the present disclosure includes an acquisition means for acquiring information indicating the display timing of a screen to be shared, a determination means for determining the timing to start sharing the screen to be shared in accordance with the information indicating the display timing, and a transmission means for transmitting a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared in accordance with the determined timing.

[0007] A screen sharing control device according to one embodiment of the present disclosure includes an acquisition means for acquiring work information indicating the work status of a user with whom the shared screen is to be shared, a determination means for estimating when the user with whom the shared screen is to be shared is free based on the work information and determining the estimated timing as the timing for starting sharing of the shared screen, and a transmission means for transmitting a control signal for displaying the shared screen to at least one terminal device with which the shared screen is to be shared, in accordance with the determined timing.

[0008] A screen sharing control method according to one embodiment of the present disclosure involves a computer acquiring information indicating the display timing of a screen to be shared, determining the timing to start sharing the screen to be shared according to the information indicating the display timing, and transmitting a control signal to at least one terminal device with which the screen to be shared is to be shared according to the determined timing, causing the screen to be displayed.

[0009] In one embodiment of the screen sharing control method of the present disclosure, a computer acquires work information indicating the work status of a user with whom a shared screen is to be shared, estimates when the user with whom the shared screen is to be shared is free based on the work information, determines the estimated timing as the timing to start sharing the shared screen, and transmits a control signal to at least one terminal device with which the shared screen is to be shared, in accordance with the determined timing, to cause the shared screen to be displayed.

[0010] A program of one embodiment of the present disclosure causes a computer to perform the following processes: acquiring information indicating the display timing of a screen to be shared; determining the timing to start sharing the screen to be shared according to the information indicating the display timing; and transmitting a control signal to at least one terminal device with which the screen to be shared is to be shared according to the determined timing, causing the screen to be shared to be displayed.

[0011] A program of one embodiment of the present disclosure causes a computer to perform the following processes: acquiring work information indicating the work status of a user with whom the shared screen is to be shared; estimating when the shared user is free based on the work information and determining the estimated timing as the timing to start sharing the shared screen; and transmitting a control signal to at least one terminal device with which the shared screen is to be shared, in accordance with the determined timing, to cause the shared screen to be displayed. Effect of the Invention

[0012] According to the present disclosure, it is possible to provide a screen sharing control device, a screen sharing control system, a screen sharing control method, and a program that start screen sharing at an appropriate timing. [Brief description of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram for explaining a screen sharing control system according to the present disclosure. [Diagram 2] 1 is a block diagram showing an example of a configuration of a screen sharing control system according to the present disclosure. [Diagram 3] 11 is a table showing an example of user schedule information. [Figure 4] 11 is a table showing an example of scenario information. [Diagram 5] FIG. 13 is a conceptual diagram illustrating an example of a scenario information registration screen. [Figure 6] FIG. 11 is a diagram illustrating an example of event tag related information. [Figure 7] FIG. 13 is a conceptual diagram illustrating an example of an event tag related information input screen. [Figure 8] 13 is a table showing an example of a control information confirmation screen. [Figure 9] 11 is a conceptual diagram illustrating an example of an input screen for receiving an input as to whether or not to share a window. FIG. [Figure 10] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 11]10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 12] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 13] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 14] 1 is a conceptual diagram for explaining a screen sharing control system according to the present disclosure. [Figure 15] 1 is a block diagram showing an example of a configuration of a screen sharing control system according to the present disclosure. [Figure 16] FIG. 13 is a conceptual diagram illustrating an example of a display screen for requesting a sharing destination user to share a screen. [Figure 17] FIG. 13 is a conceptual diagram showing an example of a display screen for requesting an input from a sharing source user, including information for identifying a screen to be shared and information for identifying a sharing destination user U. [Figure 18] 13 is a conceptual diagram illustrating an example of a display screen for notifying a sharing source user that the sharing source user is waiting for sharing permission from a sharing destination user. FIG. [Figure 19] FIG. 13 is a conceptual diagram illustrating an example of a screen request notification screen. [Figure 20] 13 is a flowchart illustrating an example of a pre-processing operation of the screen sharing control device according to the present disclosure. [Figure 21] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 22] 1 is a conceptual diagram for explaining a screen sharing control system according to the present disclosure. [Diagram 23] 1 is a block diagram showing an example of a configuration of a screen sharing control system according to the present disclosure. [Figure 24] 13 is a flowchart illustrating an example of a pre-processing operation of the screen sharing control device according to the present disclosure. [Diagram 25] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 26] 1 is a block diagram showing an example of a configuration of a screen sharing control device according to the present disclosure. [Figure 27] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Figure 28] 1 is a block diagram showing an example of a configuration of a screen sharing control device according to the present disclosure. [Figure 29] 10 is a flowchart illustrating an example of an operation of the screen sharing control device according to the present disclosure. [Diagram 30] 1 is a block diagram showing an example of a configuration of a screen sharing control system according to the present disclosure. [Diagram 31] 1 is a block diagram showing an example of a configuration of a screen sharing control system according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The embodiments described below are limited in a manner that is technically preferable for implementing the present disclosure, but the scope of the invention is not limited to the following. In all drawings used to explain the following embodiments, the same reference numerals are used for similar parts unless otherwise specified. In the following embodiments, repeated explanations of similar configurations and operations may be omitted.

[0015] In the following embodiments, when there is no need to distinguish between users U1 and U2, they will be referred to as users U. Similarly, when there is no need to distinguish between windows W1 to W6, they will be referred to as windows W. Furthermore, when there is no need to distinguish between display screens S1 and S2, they will be referred to as display screens S.

[0016] [First embodiment]

[0017] (Configuration of Screen Sharing Control System 1) FIG. 1 is a diagram for explaining a screen sharing control system 1 according to the present disclosure. The screen sharing control system 1 is a system for a user U of a screen sharing source to share a screen with a user U of a screen sharing destination. A window Wn (n is a natural number) displayed on a display screen S of a terminal device of a sharing source is shared to a terminal device of a sharing destination via a screen sharing control device 10. The screen sharing control system 1 may share some of the windows displayed on the display screen S, or may share all of the windows. In the example of FIG. 1, a user U1 is a sharing source user, and a user U2 is a sharing destination user. In addition, in the example of FIG. 1, a window W1 is shared among the windows included in the display screen S1 of the terminal device 70 of the sharing source. In the example of FIG. 1, it is assumed that there are two users U (user U1, user U2), but this is an example. The user U may be one or more. In addition, the number of users of the sharing destination and the sharing source is not limited to the example of FIG. 1. For example, there may be more than one user of the sharing destination and the sharing source.

[0018] Fig. 2 is a block diagram showing an example of a configuration of a screen sharing control system 1 according to the present disclosure. As shown in Fig. 2, the screen sharing control system 1 includes a screen sharing control device 10, a terminal device 70, and a terminal device 80. The screen sharing control device 10 is connected to each of the terminal device 70 and the terminal device 80 so as to be able to communicate with each other.

[0019] The terminal device 70 and the terminal device 80 are information processing devices. In FIG. 2, the screen sharing control system 1 includes two terminal devices, the terminal device 70 and the terminal device 80, but this is just an example. The number of terminal devices may be any number equal to or greater than two. The terminal device 70 and the terminal device 80 are realized by information processing devices such as, for example, a PC (Personal Computer), a tablet terminal, or a smartphone. The terminal device 70 and the terminal device 80 may also be realized by a wearable terminal such as smart glasses.

[0020] The terminal device 70 and the terminal device 80 each include at least an output unit, a storage unit, and an input unit. The output unit displays a display screen S on a display unit of the terminal device 70 or the terminal device 80. The display unit is realized by, for example, a display. The storage unit stores information required for processing by the screen sharing control device 10. Specifically, the storage unit stores at least one of the user's schedule information and scenario information. The storage unit is realized by, for example, a storage device such as a hard disk or a semiconductor memory. The input unit is realized by, for example, an input / output interface connected to an input device such as a mouse or a keyboard. Information input by the user U via the input unit is transmitted to the screen sharing control device 10.

[0021] (Configuration of the screen sharing control device 10) Next, the screen sharing control device 10 will be described with reference to FIG. 2. The screen sharing control device 10 is realized by a general information processing device. For example, the screen sharing control device 10 is realized by a server device. For example, the screen sharing control device 10 is realized by a server device that employs cloud computing to provide services to users via a network. The screen sharing control device 10 may be realized by an application installed on the server device or the cloud. As shown in FIG. 2, the screen sharing control device 10 includes an acquisition unit 11, a determination unit 12, a transmission unit 13, and a storage unit 14. The information stored in the storage unit 14 may be stored in the storage unit of the terminal device 70.

[0022] The acquisition unit 11 acquires information indicating the display timing of the screen to be shared. The information indicating the display timing of the screen to be shared is acquired from the storage unit 14 or the storage unit of the terminal device. In this embodiment, the information indicating the display timing of the screen to be shared is schedule information including a starting point for starting sharing of the screen to be shared. For example, the schedule information is schedule information of the user U or scenario information of a meeting. The schedule information is a list of planned actions of the user U. The schedule information includes planned actions of the user U and the scheduled times of the actions. The scenario information is a progress schedule of the meeting that is predetermined for each time. The display timing of the screen to be shared is a time point after the information indicating the display timing is acquired. The effective display timing of the screen to be shared is a time point after the time that is the starting point for starting sharing of the screen to be shared. The display timing of the screen to be shared may be a time point after the information indicating the display timing is acquired.

[0023] The schedule information of the user U is obtained, for example, from an application such as a scheduler installed in a terminal device owned by the user U. The schedule information of the user U may be obtained from the storage unit 14.

[0024] 3 is a diagram showing an example of schedule information of a user U. The schedule information includes identification information for identifying the user U, an event date and time, an event title, event tag information, and shared information. In this embodiment, the schedule information does not need to include both the event tag information and the shared information. The schedule information only needs to include either the event tag information or the shared information. The reason for this will be described later.

[0025] Various information included in the schedule information will be described below. The identification information for identifying the user U is identification information for uniquely identifying the user U. For example, the identification information includes information such as the name, sex, age, and employee number of the user U. The event date and time refers to the date and time when an event included in the schedule information of the user U is performed. In the example of FIG. 3, the event date and time is a date and time. The event title is the title of the event determined by the user U. The event tag information is information for identifying the type of the event. The event tag information is, for example, information indicating whether the event is a personal task or a meeting. As shown in FIG. 3, the event tag information may be the name of a meeting. The event tag information may be given in advance by the user, or may be automatically given by analyzing the event title using known natural language processing. The shared information is information including window information for identifying a window to be shared and information for identifying the user U of the sharing destination. The window information is, for example, identification information of the window W. For example, the window information is coordinate information indicating the position of the window W on the display screen of the terminal device of the user who is the sharing source. For example, the window information is application identification information indicating a specific application. The application identification information may include information on a file opened in the application. For example, the window information is window tag information associated with each window. The window tag information is tag information for identifying attributes and features of the window W. For example, the window tag information includes a project name and a task name. The window tag information may be assigned in advance by a user or may be assigned automatically. The window tag information is automatically assigned by analyzing the window title or text information included in the window using, for example, known natural language processing. The window tag information associated with each window is stored in the storage unit 14 or the storage unit of the terminal device. In addition, the window information may be identification information of one window W, or identification information of a group of windows obtained by grouping multiple windows W.

[0026] FIG. 4 is a diagram showing an example of scenario information. The scenario information includes information for identifying a sharing source user U, information for identifying a sharing destination user U, window information for identifying a window to be shared, and a date and time when the screen is shared. For example, the acquisition unit 11 acquires the scenario information stored in the storage unit 14. The acquisition unit 11 may accept input of information included in the scenario information from a terminal device and generate scenario information using the input information. For example, the acquisition unit 11 causes an output unit of the terminal device to display a scenario information registration screen that accepts input of information. The acquisition unit 11 generates scenario information using input information input via the scenario information registration screen displayed on the output unit of the terminal device.

[0027] FIG. 5 is an example of a scenario information registration screen. The scenario information registration screen may be provided in any format, such as a dedicated application screen or a web page by the screen sharing control device 10. A user U who is an operator of a terminal device inputs the sharing date and time (D1, D2), information for identifying the sharing source user U (D3), information for identifying the sharing destination user U (D4), and window information (D5) via an input unit. The GUI shown in FIG. 5 is an example, and any GUI (Graphical User Interface) may be used for the scenario information registration screen. For example, the scenario information registration screen may include any GUI, such as a push button, a radio button, or a text input field. When the user U finishes inputting various information, the user U presses the button B1. When the user U presses the button B1, the various information inputted via the input unit of the terminal device is transmitted to the acquisition unit 11.

[0028] The scenario information of the conference may be automatically generated using a sharing history of a conference held in the past. For example, the scenario information of the conference is generated using a trained model that has been machine-learned. For example, when the acquisition unit 11 inputs the title of the conference to the trained model, the trained model outputs the scenario information. The acquisition unit 11 acquires the scenario information output from the trained model. For example, the trained model is a model that has been trained using the title of the conference, information for identifying a user U who is the sharer, information for identifying a user U who is the share destination, and window information for identifying a window to be shared as teacher data. The acquisition unit 11 may output the scenario information output by the trained model to a scenario information registration screen as an input candidate. The automatic generation of the scenario information of the conference is effective, for example, in a regular conference. The reason is that in a regular conference, the timing of starting screen sharing and the screen to be shared may be the same as in a conference held in the past.

[0029] The determination unit 12 is an example of a determination means for determining the timing to start sharing a screen to be shared according to information indicating the display timing acquired by the acquisition unit 11. The determination unit 12 determines the timing to start sharing a screen to be shared according to schedule information. Alternatively, the determination unit 12 determines the timing to start sharing a screen to be shared according to scenario information. Alternatively, the determination unit 12 determines the timing to start sharing based on the content of a conversation between users.

[0030] First, a method for determining the timing to start sharing according to schedule information will be described. When the schedule information includes event tag information, the determination unit 12 refers to the event tag information. The determination unit 12 identifies the time of an event, which is information indicating that the event tag information is a meeting. When event tag related information associated with the event tag information is stored in the storage unit 14, the determination unit 12 refers to the event tag related information to determine a screen to be shared.

[0031] FIG. 6 is an example of the event tag-related information. As shown in FIG. 6, the event tag-related information includes window information for identifying a screen to be shared and information for identifying a user who is a sharing destination. When the event tag-related information is not stored in the storage unit 14, the acquisition unit 11 requests the user associated with the schedule information, i.e., the terminal device of the sharing source user, to input the event tag-related information. For example, the acquisition unit 11 causes the output unit of the terminal device to display an event tag-related information input screen. The acquisition unit 11 acquires the input information from the terminal device as the event tag-related information.

[0032] FIG. 7 is an example of an event tag related information input screen. The event tag related information input screen includes at least information D6 requesting an input for identifying a screen to be shared, and information D8 requesting an input for identifying a user to be shared. The event tag related information input screen may include a search box D7 for searching for a user to be shared. The GUI used in the input screen of FIG. 7 is an example, and is not limited thereto. The determination unit 12 determines the timing to start sharing using the input information about the event tag related information acquired by the acquisition unit 11. The sharing source user may specify himself / herself as the sharing destination user. When the user himself / herself is selected as the sharing destination user, the screen to be shared specified by the user is automatically launched at the event start time. By specifying himself / herself as the sharing destination user, the desired screen can be launched at the specified time, and therefore the work efficiency of the sharing source user can be improved.

[0033] A specific example of determining the timing to start sharing when schedule information includes event tag information will be described with reference to Figs. 3, 6, and 7. In this specific example, it is assumed that the acquisition unit 11 has acquired the schedule information shown in Fig. 3. First, the determination unit 12 identifies the time "10:00-12:00" for which the event tag information is "Measure X Meeting". Similarly, the determination unit 12 identifies the time "16:30-17:00" for which the event tag information is "Project C Progress Meeting".

[0034] When the event tag related information shown in FIG. 6 is stored in the storage unit 14, the determination unit 12 refers to the sharing information to identify the screen to be shared and the sharing destination. Then, the determination unit 12 determines that the window W1 is to be shared with users U2, U3, and U4 at 10:00. Similarly, the determination unit 12 determines that the window associated with the window tag information "Project C" is to be shared with user U2 at 16:30. The determination unit 12 uses the determined information to generate control information as shown in FIG. 8. The control information includes the date and time of sharing, information for identifying the sharing source user U, information for identifying the sharing destination user U, and window information of the window W to be shared.

[0035] When the event tag related information shown in FIG. 6 is not stored in the storage unit 14, the determination unit 12 displays an event tag related information input screen as shown in FIG. 7 on the output unit of the terminal device 70 operated by the user U1 who is the sharing source user. The user U1 inputs information into each item via the input unit of the terminal device 70. In this specific example, it is assumed that the user U1 inputs the information shaded in FIG. 7. The user U1 presses the button B2 when inputting is completed for all items. When the button B2 is pressed, the terminal device 70 transmits the input information to the screen sharing control device 10. The acquisition unit 11 receives the transmitted input information. The determination unit 12 determines the timing to start sharing based on the received information. Furthermore, the determination unit 12 generates control information as shown in FIG. 8. It is desirable that the determination unit 12 displays the event tag related information input screen before the start time of the event. For example, the determination unit 12 displays the event tag related information input screen 5 minutes before the start time of the event. When this method is adopted, the screen to be shared cannot be shared until user U1 has completed input. Therefore, if the event tag related information input screen is displayed at the same time as the event start time, there is a possibility that screen sharing cannot start at the appropriate time. In contrast, if the event tag related information input screen is displayed before the event start time, screen sharing can start at the appropriate time.

[0036] For an event where it is known in advance that a plurality of people will share a screen, an event tag related information input screen as shown in FIG. 7 is displayed on a terminal device operated by each sharing source user. The sharing source user registers a window or file to be shared via the event tag related information input screen. The acquisition unit 11 receives input information transmitted from a terminal device operated by each sharing source user. The determination unit 12 determines the timing to start sharing according to the received information and a predetermined sharing order. For example, the determination unit 12 determines the timing when the presenter's presentation ends as the timing to start screen sharing by the next presenter. The event tag related information input screen may include information for accepting input of a scheduled time to start screen sharing. When the scheduled time is received from a terminal device operated by a sharing source user, the determination unit 12 may be configured to send a confirmation message to the sharing source user as to whether or not to start sharing when the received scheduled time approaches.

[0037] When the schedule information includes the sharing information, the determination unit 12 refers to the sharing information. The determination unit 12 identifies an event having the sharing information. For example, in the example of FIG. 3, the determination unit 12 identifies an event having a time of "10:00 to 12:00" and an event having a time of "16:30 to 17:00". The determination unit 12 shares the shared screen included in the sharing information with the sharing destination included in the sharing information at the start time of the identified event. For example, in the example of FIG. 3, the determination unit 12 determines that the window W1 is shared with the user U2 at 10:00, and that the windows W3 and W4 are shared with the user U2 at 16:30. Furthermore, the determination unit 12 generates control information as shown in FIG. 8. Before starting sharing, the determination unit 12 may display a screen on the output unit of the terminal device operated by the user U, which allows the sharing source user U to confirm whether or not sharing can be started at the determined timing. For example, the determination unit 12 displays a control information confirmation screen as shown in FIG. 8 on an output unit of a terminal device operated by the sharing source user U. The control information confirmation screen includes the generated control information (D16) and an item (D17) for inputting whether or not to perform screen sharing according to the control information. The control information may be modified by the sharing source user U via the control information confirmation screen. When inputting has been completed for all items, the sharing source user U presses button B7. When button B7 is pressed, the terminal device transmits the input information to the screen sharing control device 10.

[0038] Next, a method of determining the timing to start sharing according to the scenario information will be described. The determination unit 12 determines the timing to start sharing of the screen to be shared by referring to information for identifying the user U who shares the screen, the window information of the window to be shared, and the sharing date and time, which are included in the scenario information. A specific example of determining the timing to start sharing according to the scenario information will be described with reference to FIG. 4. The determination unit 12 determines that the window W1 is shared by the user U1 at 10:00 by referring to the scenario information shown in FIG. 4. Similarly, the determination unit 12 determines that the window W3 is shared by the user U1 at 10:15. Thereafter, the determination unit 12 similarly determines the user U to share, the window information of the window W to be shared, and the timing to start sharing by referring to the scenario information. Furthermore, the determination unit 12 generates control information as shown in FIG. 8.

[0039] Next, a method of determining the timing to start sharing based on the user's conversation information will be described. First, the acquisition unit 11 acquires the user's conversation information. The conversation information includes text conversation information and voice conversation information. The text conversation information is acquired from various communication applications such as a chat application. The voice conversation information is acquired through a voice acquisition unit such as a microphone. The voice acquisition unit may be provided in a terminal device operated by the user U, or may be realized by an external device communicably connected to the screen sharing control device 10. The acquisition unit 11 converts the acquired voice into text information by a known voice recognition technology. When the text information acquired by the acquisition unit 11 satisfies a predetermined condition (also referred to as a sharing start condition) related to the event tag information, the determination unit 12 presents the user U with an input screen for deciding whether or not to share a window. For example, the sharing start condition is that the number of occurrences of the event tag information included in the text information exceeds a predetermined threshold. In another example, the sharing start condition may be the frequency of occurrence of the event tag information within a predetermined period. In yet another example, the sharing start condition may be that the time when the event tag information appears is a predetermined time. In yet another example, the sharing start information may be information indicating that the elapsed time since the appearance of certain event tag information reaches a predetermined time. For example, when the event tag information "Project C" appears more than a predetermined threshold in a conversation between users, the determination unit 12 presents an input screen as shown in FIG. 9 to the user U. As shown in FIG. 9, the input screen includes information (B8 and B9) for receiving an input of whether or not to share a window, and an item (D17) for receiving an input of the timing of starting sharing. The acquisition unit 11 receives information input by the user U via the input screen. The determination unit 12 determines the timing of starting sharing included in the input information as the timing of starting sharing of the screen to be shared. When the above-mentioned predetermined condition (sharing start condition) is satisfied, the determination unit 12 may immediately start sharing without presenting an input screen as shown in FIG. 9. For example, the determination unit 12 may immediately start sharing when the text information includes a predetermined keyword.For example, when the predetermined keyword is "share" and a sentence "share the screen of project C" appears in the text information, the determination unit 12 determines a window related to project C as a screen to be shared. In addition, the determination unit 12 determines the sharing start timing to be immediate.

[0040] The transmission unit 13 is an example of a transmission means for transmitting a control signal for displaying the shared screen to at least one terminal device that is a sharing destination of the shared screen according to the timing determined by the determination unit 12. The transmission unit 13 generates a control signal for displaying the shared screen by using the control information generated by the determination unit 12. The control signal includes information for identifying the sharing source user, window information of the shared screen, information for identifying the sharing destination user, and information on the sharing date and time. The control signal may include a signal for displaying a marker representing the cursor position of the sharing source terminal device superimposed on the shared screen. The transmission unit 13 transmits the generated control signal to a terminal device operated by the sharing destination user U included in the control information. The terminal device that receives the control information controls the display of the screen according to the control signal. The transmission unit 13 may also transmit a signal for displaying a marker representing the cursor position of the sharing destination user superimposed on the shared screen to the sharing source terminal device. By displaying the cursor position of the sharing destination user on the shared screen, an effect is obtained in which it is easier for the sharing source user and the sharing destination user to communicate with each other. For example, when a sharing source user receives guidance from a sharing destination user, a marker indicating the cursor position on the shared screen of the sharing destination user serves as a guidance pointer on the shared screen of the sharing source user.

[0041] The transmission unit 13 may transmit a control signal for opening the screen to be shared to at least one terminal device that is a sharing source of the screen to be shared, in accordance with the timing of starting sharing determined by the determination unit 12. Specifically, the transmission unit 23 transmits a control signal for displaying the screen to be shared if the screen to be shared is not open, at a timing prior to the timing of starting sharing. This allows the screen to be shared to be shared smoothly, since the screen to be shared is automatically displayed before sharing starts.

[0042] The storage unit 14 stores at least one of window tag information associated with each window W, schedule information of the user U, scenario information of the conference, and event tag related information. As described above, the information stored in the storage unit 14 may be stored in a storage unit of a terminal device. In that case, the screen sharing control device 10 does not need to include the storage unit 14.

[0043] (Operation of the screen sharing control device 10) Next, an example of the operation of the screen sharing control device 10 in this embodiment will be described with reference to Figs. 10 to 13. Figs. 10 and 11 are diagrams showing an example of the operation when the information indicating the display timing is schedule information. Fig. 12 is a diagram showing an example of the operation when the information indicating the display timing is scenario information. Fig. 13 is an example of the operation when the information indicating the display timing is event information included in the contents of a conversation between users.

[0044] <Example of operation when the information indicating the display timing is schedule information> First, an example of an operation in a case where the information indicating the display timing is schedule information will be described with reference to Fig. 10 and Fig. 11. The acquisition unit 11 acquires schedule information of a user U by referring to the storage unit or the storage unit 14 of the terminal device (step S101).

[0045] Next, the determination unit 12 checks whether the acquired schedule information includes event tag information (step S102). If the schedule information includes event tag information (Yes in step S102), the first process shown in FIG. 11 is performed (step S103). Details of the first process will be described later. After completing the first process, the transmission unit 13 transmits a control signal to the sharing destination terminal device to display the screen to be shared (step S106).

[0046] If the schedule information does not include the event tag information (No in step S102), the determination unit 12 checks whether the schedule information includes the shared information (step S104). Note that the order of the checking process in step S102 and the checking process in step S104 may be interchanged.

[0047] If the schedule information includes sharing information (Yes in step S104), the determination unit 12 determines the timing to start sharing the screen to be shared (step S105). If the schedule information includes sharing information (No in step S104), the screen sharing control device 10 ends the process. Alternatively, instead of ending the process, the screen sharing control device may request the terminal device to input event tag information or sharing information. Next, the transmission unit 13 transmits a control signal to the terminal device of the sharing destination to display the screen to be shared (step S106).

[0048] After completing the process of step S106, the screen sharing control device 10 ends the series of processes.

[0049] Next, the first process of step S103 will be described with reference to Fig. 11. If the schedule information includes event tag information (Yes in step S102), the determination unit 12 checks whether or not event tag-related information is stored in the storage unit 14 or the storage unit of the terminal device (step S1031). If the event tag-related information is stored (Yes in step S1031), the determination unit 12 refers to the schedule information and the event tag-related information and determines the timing to start sharing the screen to be shared (step S1032).

[0050] If the event tag related information is not stored (No in step S1031), the determination unit 12 accepts an input of the event tag related information from the terminal device (step S1033). The determination unit 12 refers to the schedule information and the accepted input information to determine the timing to start sharing the screen to be shared (step S1034). When the process of step S1032 or step S1034 ends, the first process shown in FIG. 11 ends.

[0051] <Example of operation when the information indicating the display timing is scenario information> Next, an example of the operation when the information indicating the display timing is scenario information will be described with reference to Fig. 12. First, the acquisition unit 11 acquires scenario information of the conference by referring to the storage unit or the storage unit 14 of the terminal device (step S111).

[0052] Next, the determination unit 12 determines the timing to start sharing the screen to be shared, according to the acquired scenario information (step S112).

[0053] Next, the transmission unit 13 transmits a control signal for displaying the screen to be shared to the terminal device as the sharing destination (step S113). After completing the process of step S113, the screen sharing control device 10 ends the series of processes described above.

[0054] <Example of operation when the information indicating the display timing is event information included in the conversation between users> Next, an example of the operation when the information indicating the display timing is event information included in the contents of the conversation between users will be described with reference to Fig. 13. First, the acquisition unit 11 acquires conversation information between users (step S121). Specifically, the acquisition unit 11 acquires voice conversation information between users via a voice information acquisition unit provided in a terminal device or an external device. Alternatively, the acquisition unit 11 acquires text conversation information from various communication applications such as a chat application.

[0055] Next, the acquisition unit 11 converts the acquired conversation information into text information (step S122). If the information acquired by the acquisition unit 11 is text conversation information, the process of step S122 is omitted since the information is already text information.

[0056] Next, the determination unit 12 judges whether or not the number of occurrences of the event tag information included in the text information exceeds a predetermined threshold (step S123). If the number of occurrences of the event tag information does not exceed the predetermined threshold (No in step S123), the screen sharing control device 10 ends a series of processes. On the other hand, if the number of occurrences of the event tag information exceeds the predetermined threshold (Yes in step S123), the determination unit 12 accepts an input of whether or not to share a window (step S124). Specifically, the determination unit 12 causes an input screen including information for accepting an input of whether or not to share a window and information for accepting an input of the timing of starting sharing to be displayed on an output unit of the terminal device operated by the user.

[0057] Next, the determination unit 12 determines the screen to be shared and the timing to start sharing the screen to be shared, in accordance with the information input via the input screen (step S125).

[0058] Next, the transmission unit 13 transmits a control signal for displaying the screen to be shared to the terminal device of the sharing destination (step S126). After completing the process of step S126, the screen sharing control device 10 ends the series of processes described above.

[0059] As described above, the screen sharing control device of the present embodiment includes an acquisition unit, a determination unit, and a transmission unit. The determination unit determines the timing to start sharing the screen to be shared according to information indicating the display timing. The information indicating the display timing is schedule information including a starting point to start sharing the screen to be shared. For example, the schedule information includes at least one of user schedule information and meeting scenario information. The determination unit detects a starting point to start sharing from the schedule information acquired by the acquisition unit, and determines the detected starting point as the timing to start sharing the screen to be shared. In the present embodiment, the determination unit detects the scheduled time as a starting point to start sharing. This allows the screen sharing control device of the present embodiment to determine an appropriate timing to perform screen sharing. The transmission unit transmits a control signal to display the screen to be shared to the terminal device of the sharing destination according to the determined timing. The control signal includes information for identifying the user of the sharing source, window information of the screen to be shared, information for identifying the user of the sharing destination, and information on the sharing date and time. This allows the screen sharing control device of the present embodiment to start screen sharing at an appropriate timing. Furthermore, according to the screen sharing control device of the present embodiment, the sharing source user does not need to perform a sharing operation in advance, which improves operability.

[0060] In addition, the determination unit of this embodiment determines the screen to be shared. As a result, the screen sharing control device of this embodiment can determine the screen to be shared even if the user has not specified the screen to be shared in advance. For example, the screen sharing control device of this embodiment determines the screen to be shared by referring to the user's scheduler. In other words, the screen sharing control device of this embodiment can perform processing without requiring pre-processing by the user.

[0061] Moreover, the acquisition unit of this embodiment acquires input of information included in the scenario information from the terminal device of the sharing source or the terminal device of the sharing destination, and generates scenario information using the acquired information. Moreover, the acquisition unit of this embodiment requests input of event tag related information from the terminal device of the user associated with the schedule information, i.e., the user of the sharing source. With the above configuration, the screen sharing control device of this embodiment can perform processing even if the scenario information or the event tag related information has not been registered in advance. Furthermore, when requesting input from the user, the screen sharing control device of this embodiment displays the input screen shown in FIG. 5 or FIG. 7 on the terminal device. As a result, the screen sharing control device of this embodiment can improve the operability for the user.

[0062] Further, the acquisition unit of this embodiment acquires user conversation information as information indicating the display timing. When the conversation information satisfies a sharing start condition related to a predetermined keyword, the determination unit of this embodiment determines the timing to start sharing the shared screen related to the keyword. For example, the keyword indicates event tag information. For example, the sharing start condition is that the number of occurrences of the event tag information included in the text information exceeds a predetermined threshold. In another example, the sharing start condition may be the frequency of occurrence of the event tag information within a predetermined period. In yet another example, the sharing start condition may be that the time when the event tag information appears is a predetermined time. Thereby, the screen sharing control device of this embodiment can start screen sharing at an appropriate timing during a conversation between users.

[0063] A typical screen sharing control device starts screen sharing at a timing determined by a user who is a screen share source. However, the timing determined by a user who is a screen share source is not necessarily the appropriate timing to start sharing. For example, if a user who is a screen share source takes time to perform a sharing operation, sharing cannot start at an appropriate timing. In an online conference, if a user who is a screen share source takes time to perform a sharing operation, screen sharing cannot be performed at an appropriate timing determined by the progress schedule. In contrast, the screen sharing control device of this embodiment can start screen sharing at an appropriate timing without the user who is a screen share source taking time to perform a sharing operation.

[0064] [Second embodiment]

[0065] (Configuration of the screen sharing control device 20) Next, a screen sharing control system 2 and a screen sharing control device 20 according to this embodiment will be described with reference to the drawings. In the following description, the same parts as those in the first embodiment will be omitted as appropriate. The screen sharing control system 1 of the first embodiment determines the scheduled time, which is the starting point for starting sharing and is included in the schedule information, as the timing to start sharing the screen to be shared. The screen sharing control system 2 of this embodiment differs from the screen sharing control system 1 of the first embodiment in that it determines the timing to start sharing the screen to be shared as the timing when it is estimated that the user at the sharing destination has free time.

[0066] First, the screen sharing control system 2 in the present disclosure will be described with reference to FIG. 14. The screen sharing control system 2 of this embodiment is a system that starts screen sharing when the user at the sharing destination is free. When the user U1 at the sharing source desires to share a screen, the user U1 transmits a screen sharing request to the screen sharing control device 20 via the terminal device 70. The sharing request includes information on the screen to be shared and information for identifying the user at the sharing destination. The screen sharing control device 20 acquires information from the terminal device 80 operated by the user U2 at the sharing destination, and estimates the work status of the user U2. When the user U2 is working, the screen sharing control device 20 does not notify the terminal device 80 that a screen sharing request has been made. When the user U2 has finished working, the screen sharing control device 20 notifies the terminal device 80 that a screen sharing request has been made. When the user U2 who has received the notification permits the sharing request, the screen of the user U1 at the sharing source is shared.

[0067] FIG. 15 is a block diagram showing an example of the configuration of the screen sharing control system 2. As shown in FIG. 15, the screen sharing control system 2 includes a screen sharing control device 20, a terminal device 70, and a terminal device 80. The terminal device 70 and the terminal device 80 include an output unit, a storage unit, an input unit, and a photographing unit. The photographing unit is realized by an imaging device such as a digital camera. The photographing unit may be an external device connected to the terminal device. In FIG. 15, the screen sharing control system 2 includes two terminal devices, the terminal device 70 and the terminal device 80, but this is an example. The number of terminal devices may be any number as long as it is two or more. The screen sharing control device 20 includes an acquisition unit 21, a determination unit 22, a transmission unit 23, and a storage unit 24.

[0068] The acquisition unit 21 acquires work information indicating the work status of a user who is a sharing destination of the screen to be shared. The work information is acquired from the storage unit 24 or a storage unit of a terminal device operated by the sharing destination user U. Specifically, the work information includes operation log information of the terminal device operated by the sharing destination user U, line of sight information of the user U, a facial photograph of the user U, and the like. The line of sight information of the user U and the facial photograph of the user U are acquired from a photographing unit provided in the terminal device.

[0069] The contents of the various information included in the work information are described below. The operation log information of the terminal device is a log of operations performed by the user U on the terminal device. The operation log information includes at least the date and time of the work, the operation content, and the status of the work. The status of the work is information indicating whether the work has started or been completed. The gaze information of the user U is log information of the gaze position of the user U and the time when the user U gazed at the position. The operation log information and the gaze information are updated as new information is acquired. A face photo of the user U is acquired in real time by the photographing unit of the terminal device.

[0070] The determination unit 22 determines the timing to start sharing the screen to be shared with the sharing destination user according to the information acquired by the acquisition unit 21. The determination unit 22 determines the timing to start sharing the screen using the task information acquired by the acquisition unit 21. Specifically, the determination unit 22 estimates the timing when the sharing destination user U will be free. The determination unit 22 determines the estimated timing as the timing to start sharing the screen.

[0071] A method of estimating the timing when user U is free by using operation log information of a terminal device operated by a sharing destination user U will be described below. For example, the determination unit 22 refers to the operation log information and estimates the timing when a file, application, or window W is closed as the timing when user U is free. In another example, the determination unit 22 estimates the timing when user U saves a file he or she is working on as the timing when user U is free. In another example, when no operation is performed for a certain period of time, the determination unit 22 estimates the timing as the timing when user U is free.

[0072] Next, a method of estimating the timing when the user U has freed his / her hands using the gaze information of the user U will be described below. For example, the determination unit 22 refers to the gaze information, and when the gaze position of the user remains outside the display screen of the terminal device for a certain period of time or more, estimates the timing as the timing when the user U has freed his / her hands.

[0073] Next, a method for estimating the timing when the user U's hands are free using a facial photograph of the user U will be described below. The determination unit 22 estimates the degree of concentration on the task of the user U based on the acquired facial photograph using a known concentration estimation technique. When the degree of concentration of the user falls below a certain value, the determination unit 22 estimates the timing as the timing when the user U's hands are free. The known concentration estimation technique is, for example, a technique for estimating the degree of concentration based on a change in the movement of the facial muscles of the user U.

[0074] The determination unit 22 further determines the screen to be shared and the user U who is to be shared. Hereinafter, a procedure in which the determination unit 22 determines information for identifying the screen to be shared and the user U who is to be shared will be described with reference to Figs. 16 to 18.

[0075] First, as shown in FIG. 16, the acquisition unit 21 displays a button B3 for requesting screen sharing from the sharing destination user U on the display screen S1 of the terminal device operated by the sharing source user U. When the button B3 is pressed by the user U, the acquisition unit 21 displays an input screen as shown in FIG. 17 on the display screen S1 of the terminal device operated by the sharing source user U. The input screen includes at least information (D9) for identifying the screen to be shared and information (D11) for identifying the sharing destination user U. The input screen may also include a search box (D10) for searching for the sharing destination user, a text box (D12) for inputting a message to the sharing destination user, and a priority flag (D15) for specifying the priority of the sharing request. The priority flag is a flag for determining whether or not to start sharing regardless of the timing of starting sharing determined by the determination unit 22. When the priority flag "high" is specified, the transmission unit 23 transmits a control signal for starting sharing regardless of the timing of starting sharing determined by the determination unit 22. On the other hand, when the priority flag "low" is specified, the transmission unit 23 transmits a control signal to start sharing at the timing of starting sharing determined by the determination unit 22. When the sharing source user U inputs various information displayed on the input screen, the user U presses the button B4. When the sharing source user U presses the button B4, various information input via the input unit of the terminal device is transmitted to the acquisition unit 11. The determination unit 22 uses the information acquired by the acquisition unit 11 to determine window information of the window to be shared and information for identifying the sharing destination user U. Until the sharing destination user allows sharing, the display screen of the sharing source user U may display information (D13) indicating that sharing is awaited from the sharing destination user, as shown in FIG. 18, for example. Note that the GUIs used by the display screens of FIG. 16 to FIG. 18 are merely examples, and may be realized by other GUIs.

[0076] The transmission unit 23 transmits a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared, in accordance with the timing of starting sharing determined by the determination unit 22, the screen to be shared, and information on the user U with whom the screen to be shared is to be shared. Hereinafter, a procedure for transmitting the control signal by the transmission unit 23 will be described with reference to FIG.

[0077] First, when the timing of the start of sharing determined by the determination unit 22 arrives, the acquisition unit 21 displays a screen request notification screen (D14) as shown in FIG. 19 on the display screen S2 of the terminal device operated by the user U of the share destination determined by the determination unit 22. When the priority flag of the sharing request is received from the terminal device of the share source and the received priority flag is "high", the acquisition unit 21 displays the screen request notification screen regardless of the timing of the start of sharing determined by the determination unit 22. In order to attract the attention of the user U of the share destination, the acquisition unit 21 may output a notification sound simultaneously with the display of the sharing request notification screen. The notification sound is output through a speaker provided in the terminal device of the share destination or a speaker connected to the terminal device of the share destination. In addition, in order to attract the attention of the user of the share destination, the acquisition unit 21 may cause the sharing request screen to flash. The screen request notification screen is displayed, for example, as a pop-up on the display screen S2 as shown in FIG. 19. The screen request notification screen includes information for identifying the sharing source user and information (B5, B6) requesting input as to whether or not to permit the screen sharing request. When message information from the sharing source user U has been received, the screen request notification screen may include message information from the sharing source user U. When the sharing destination user U inputs that the screen sharing request is permitted, the acquisition unit 21 acquires the input information. When the transmission unit 23 acquires the input permitting the screen sharing request, the transmission unit 23 transmits a control signal for displaying the screen to be shared determined by the determination unit 22 to the terminal device possessed by the sharing destination user U.

[0078] In addition to the information described in the first embodiment, the storage unit 24 may store work information related to the work status of the sharing destination user U. The work information may be stored in the storage unit of the terminal device.

[0079] (Operation of the screen sharing control device 20) Next, an example of the operation of the screen sharing controller 20 in this embodiment will be described with reference to Fig. 20 and Fig. 21. The screen sharing controller 20 in this embodiment requires a pre-processing step shown in Fig. 20.

[0080] <Pretreatment> First, an example of the pre-processing operation of the screen sharing control device 20 will be described with reference to Fig. 20. The acquisition unit 21 acquires at least one of operation log information of the terminal device operated by the user, the user's line of sight information, and the user's facial photograph as work information from the terminal device (step S201). The acquisition unit 21 stores the work information acquired in step S201 in the storage unit 24 or the storage unit of the terminal device (step S202). The acquisition process in step S201 is performed in real time.

[0081] <This action> Next, an example of the operation of the screen sharing control device 20 in the present disclosure will be described with reference to Fig. 21. First, the acquisition unit 21 acquires a screen sharing request from a terminal device operated by a sharing source user U (step S211). The screen sharing request includes at least information for identifying the screen to be shared and information for identifying a sharing destination user U.

[0082] Next, the determination unit 22 determines the timing to start sharing the screen to be shared, the screen to be shared, and the user to be shared (step S212). Specifically, the determination unit 22 determines the timing to start sharing the screen according to the work information stored in the storage unit 24 or the storage unit of the terminal device. The determination unit 22 determines the screen to be shared and the user to be shared using input information from the terminal device.

[0083] Next, the acquisition unit 21 receives input information on whether or not to permit the screen sharing request from the terminal device operated by the user U of the sharing destination. If screen sharing is not permitted (No in step S213), the process returns to step 212. The screen sharing control device 20 repeats the processes of steps S212 to S213 until information permitting the screen sharing request is received from the terminal device operated by the user U of the sharing destination. When information permitting the screen sharing request is received from the terminal device operated by the user U of the sharing destination (Yes in step S213), the transmission unit 23 transmits a control signal for displaying the screen to be shared to the terminal device of the sharing destination (step S214). When the screen sharing control device 20 finishes the process of step S214, the process ends the series of processes described above.

[0084] As described above, the screen sharing control device of the present embodiment includes an acquisition unit, a determination unit, and a transmission unit. The acquisition unit acquires work information indicating the work status of the user at the sharing destination of the screen to be shared. The determination unit estimates that the user at the sharing destination has free time according to the work information acquired by the acquisition unit, and determines the estimated timing as the timing to start sharing the screen to be shared. The transmission unit transmits a control signal to display the screen to be shared to at least one terminal device that is the sharing destination according to the determined timing. With the above configuration, the screen sharing control device of the present embodiment can start screen sharing at a timing when it is estimated that the user at the sharing destination has free time. In other words, the screen sharing control device of the present embodiment can start screen sharing at a timing convenient for the user at the sharing destination. In other words, according to the screen sharing control device of the present embodiment, screen sharing can be started at an appropriate timing.

[0085] The acquisition unit included in the screen sharing control device of this embodiment accepts input from the terminal device by displaying the screens shown in Figs. 16 to 19 on the terminal device. The acquisition unit acquires a sharing request including information on the screen to be shared and the terminal device to be shared from the terminal device that is the sharing source by displaying the screen shown in Fig. 16. The transmission unit transmits an input screen for whether or not to permit screen sharing as shown in Fig. 19 to at least one terminal device that is the sharing destination of the screen to be shared according to the timing determined by the determination unit. The acquisition unit acquires input information for whether or not to permit the sharing request from at least one terminal device that is the sharing destination of the screen to be shared. When the input information indicating that the sharing request is permitted is acquired, the transmission unit transmits a control signal for displaying the screen to be shared to at least one terminal device that is the sharing destination of the screen to be shared. By displaying these screens, the user can intuitively perform input. In other words, the screen sharing control device of this embodiment can improve the operability of the user.

[0086] A typical screen sharing control device starts sharing at a timing determined by a user who is the source of screen sharing. However, the timing determined by the user who is the source of screen sharing is not necessarily the appropriate timing to start sharing. This is because the timing determined by the user who is the source of screen sharing does not necessarily match the timing that is convenient for the user who is the destination of screen sharing. In contrast, the screen sharing control device of this embodiment makes it possible to start screen sharing at an appropriate timing.

[0087] [Third embodiment]

[0088] (Configuration of screen sharing control system 3 and screen sharing control device 30) Next, a screen sharing control system 3 and a screen sharing control device 30 according to this embodiment will be described with reference to the drawings. The screen sharing control system 3 according to this embodiment is different from the first and second embodiments in that it includes a terminal device realized by an information processing device such as a PC, a tablet terminal, or a smartphone, and a terminal device realized by an information processing device capable of realizing MR (Mixed reality). In the following description, the same parts as those in the first or second embodiment will not be described as appropriate. The method of this embodiment can also be applied to the first and second embodiments.

[0089] As shown in FIG. 22, the screen sharing control system 3 of the present embodiment includes a terminal device 70-1 and a terminal device 70-2. The terminal device 70-1 is realized by an information processing device such as a PC, a tablet terminal, or a smartphone. The terminal device 70-2 is realized by an information processing device capable of realizing MR, such as smart glasses. As is well known, smart glasses display an image of a virtual space by superimposing it on a real space. The terminal device 70-2 is worn by a user U1. V1 is the field of view of the user through the terminal device 70-2. The field of view V1 includes a view of the real space transmitted through the lens of the terminal device 70-2 and an image of the virtual space displayed on the lens of the terminal device 70-2. Images of windows W1 to W5 operated by the user U1 are displayed in the virtual space.

[0090] FIG. 23 is a block diagram showing an example of the configuration of a screen sharing control system in the present disclosure. As shown in FIG. 23, the screen sharing control system 3 includes a screen sharing control device 30, a terminal device 70-1, a terminal device 70-2, a terminal device 80-1, and a terminal device 80-2. The terminal device 70-2 and the terminal device 80-2 are realized by an information processing device capable of realizing MR, such as smart glasses. The terminal device 70-2 and the terminal device 80-2 include a gaze information acquisition unit that acquires gaze information of a user U. The gaze information is log information of the gaze position of the user U and the time when the user U gazes at the position. The gaze information acquisition unit calculates the gaze information by a known gaze measurement technique. The gaze information acquisition unit is realized by, for example, a gaze meter device including an infrared light emitting element, a gaze detection sensor, and a detection unit. The infrared light emitting element irradiates infrared rays to the user's eyeball. The gaze detection sensor captures the reflected light of the eyeball. The detection unit detects the movement of the gaze and the gaze point on the display device based on the position of the pupil. Furthermore, the detection unit calculates gaze information based on the movement of the gaze and the information of the gaze point on the display device. The above-mentioned gaze measurement technology is an example, and the gaze information can be calculated by various other methods that a person skilled in the art can think of.

[0091] As shown in FIG. 23, the screen sharing control device 30 includes an acquisition unit 31, a determination unit 32, a transmission unit 33, and a storage unit .

[0092] The acquisition unit 31 acquires information indicating the display timing of the screen to be shared. Alternatively, the acquisition unit 31 acquires work information indicating the work status of the sharing destination user U. The acquisition unit 31 further acquires line-of-sight information of the user U from a line-of-sight information acquisition unit of the terminal device 70-2 or the terminal device 80-2.

[0093] The determination unit 32 determines the timing to start sharing the screen to be shared, according to the information acquired by the acquisition unit 31. The procedure for determining the timing to start sharing is as described in the first or second embodiment.

[0094] The transmission unit 33 transmits, in accordance with the timing determined by the determination unit 32, a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared.

[0095] Furthermore, the transmission unit 33 transmits a control signal to at least one terminal device that is a sharing destination of the screen to be shared, the marker P representing the gaze position of the user U of the sharing source, received from the terminal device that is the sharing source, superimposed on the screen to be shared. The transmission unit 33 also transmits a control signal to at least one terminal device that is a sharing source of the screen to be shared, the marker P representing the gaze position of the user of the sharing destination, received from the terminal device that is the sharing destination, superimposed on the screen to be shared. The transmission unit 33 transmits a control signal to display the marker P representing the gaze position as follows. First, the transmission unit 33 specifies window information of the window W that the user U is gazing at, from the gaze position of the user U acquired by the acquisition unit 31. Next, the transmission unit 33 specifies position information indicating the position (x, y) of the user U on the two-dimensional coordinates in the display area of ​​the window W, from the gaze position of the user U. The transmission unit 33 transmits a control signal to display the marker P at the specified position information. When there are multiple users U, the transmission unit 33 displays a marker P (P-1, P-2) at the gaze position for each user U, as shown in Fig. 22. The transmission unit 33 may change the GUI of the marker P for each user U. The transmission unit 33 may also transmit a control signal for displaying information for identifying the user U (e.g., name) together with the marker P. The transmission unit 33 may also transmit a control signal for displaying a marker representing the cursor position of the sharing source terminal device, superimposed on the shared screen, together with the marker P.

[0096] In addition, when there are multiple screens to be shared, the transmission unit 33 updates the screen to be shared that the sharing source user U is currently gazing at in real time, and updates the other screens to be shared at a preset interval. The transmission unit 33 uses the gaze position of the sharing source user U to identify the window that the sharing source user is currently gazing at. For example, in the case shown in FIG. 22, it is assumed that the sharing source user U1 shares windows W1 to W5 with the sharing destination user U2. It is assumed that the user U1 is gazing at the window W1. In this case, the transmission unit 33 transmits a control signal to display the windows W1 to W5 to the terminal device operated by the user U2. At this time, the transmission unit 33 incorporates in the control signal the window W1 to be updated in real time. The transmission unit 33 incorporates in the control signal the other windows W2 to W5 to be updated, for example, once per second.

[0097] (Operation of the screen sharing control device 30) Next, an example of the operation of the screen sharing controller 30 in this embodiment will be described with reference to Fig. 24 and Fig. 25. The screen sharing controller 30 in this embodiment requires a pre-processing step shown in Fig. 24.

[0098] <Pretreatment> First, an example of the pre-processing operation of the screen sharing control device 30 will be described with reference to Fig. 24. The acquisition unit 31 acquires user gaze information from a gaze information acquisition unit of the terminal device (step S301). The user gaze information is log information of the gaze position of the user U and the time during which the user U gazes at the position. The acquisition unit 31 stores the gaze information acquired in step S301 in the storage unit 34 or the storage unit of the terminal device (step S302). The acquisition process in step S301 is performed in real time.

[0099] <This action> Next, an example of the operation of the screen sharing control device 30 in this disclosure will be described with reference to FIG.

[0100] First, the acquisition unit 31 acquires information indicating the display timing of a screen to be shared or task information indicating a task status of a user who is a sharing destination (step S311).

[0101] Next, the determination unit 32 determines the timing to start sharing the screen to be shared, according to the information acquired by the acquisition unit 31 (step S312).

[0102] Next, the transmission unit 33 incorporates into the control signal the timing for updating the shared screen using the gaze position of the sharing source user U (step S313). Specifically, the transmission unit 33 incorporates into the control signal a command to update the shared screen that the sharing source user U is gazing at in real time, and to update the other shared screens at a preset interval.

[0103] Next, the transmission unit 33 transmits a control signal for displaying the screen to be shared to the terminal device as the sharing destination (step S314).

[0104] Next, the transmission unit 33 transmits to the sharing destination terminal device a control signal for superimposing and displaying the marker P, which is received from the sharing destination terminal device and indicates the gaze position of the sharing destination user, on the screen to be shared (step S315). The transmission unit 33 transmits to the sharing source terminal device a control signal for superimposing and displaying the marker P, which is received from the sharing destination terminal device and indicates the gaze position of the sharing destination user, on the screen to be shared (step S316). After completing the process of step S316, the screen sharing control device 30 terminates the series of processes described above.

[0105] The screen sharing control system of the present embodiment includes smart glasses. The smart glasses include a gaze information acquisition unit that acquires gaze information including the gaze position of the user who is the source of sharing. The transmission unit of the screen sharing control device of the present embodiment incorporates the timing of updating the shared target screen into the control signal using the gaze position received from the smart glasses worn by the user who is the source of sharing. Specifically, the transmission unit incorporates an instruction into the control signal to update the shared target screen that the user who is the source of sharing is gazing at in real time, and to update the other shared target screens at a preset interval. When there are multiple shared target screens, updating all the shared target screens in real time increases the load on the screen sharing control device. The screen sharing control device of the present embodiment updates the shared target screen that the user who is the source of sharing is gazing at in real time, and does not update the other shared target screens in real time. The screen sharing control device of the present embodiment can reduce the load on the screen sharing control device by the above configuration.

[0106] The transmission unit included in the screen sharing control device of this embodiment transmits a control signal to the terminal device of the sharing destination, which transmits a marker representing the gaze position of the sharing source user received from the terminal device of the sharing source, superimposed on the screen to be shared. This allows the user of the sharing destination to intuitively grasp the position at which the sharing source user is currently gazing. For example, when a voice call is made at the same time as screen sharing, the marker serves as a pointer. By using the marker as a pointer, the sharing source user does not need to verbally explain which part of the screen to be shared should be referred to. Therefore, the sharing source user can smoothly share the screen. In other words, the screen sharing control device of this embodiment can improve the operability of the user.

[0107] The transmission unit included in the screen sharing control device of this embodiment transmits a control signal to the terminal device of the sharing source to superimpose and display a marker representing the gaze position of the user of the sharing destination received from the terminal device of the sharing destination on the screen to be shared. This allows the user of the sharing destination to intuitively grasp the position at which the user of the sharing source is currently gazing. If the gaze position of the user of the sharing destination does not match the position expected by the user of the sharing source, the user of the sharing source can grasp this fact. Furthermore, since the gaze position of the user of the sharing destination can be grasped, the user of the sharing destination can be guided to the user of the sharing destination. For example, the user of the sharing source can guide the gaze of the user of the sharing destination by saying, "Please check the description a little higher." This allows the user of the sharing source to smoothly share the screen. In other words, the screen sharing control device of this embodiment can improve the operability of the user.

[0108] [Fourth embodiment] Next, the screen sharing control device 40 according to the present disclosure will be described with reference to FIGS.

[0109] (Configuration of the screen sharing control device 40) As shown in FIG. 26, the screen sharing control device 40 of this embodiment includes an acquisition unit 41, a determination unit 42, and a transmission unit 43.

[0110] The acquisition unit 41 is an example of an acquisition means for acquiring information indicating the display timing of a screen to be shared. Specifically, the information indicating the display timing of a screen to be shared is information including at least one of schedule information of a user and scenario information of a meeting.

[0111] The determination unit 42 is an example of a determination means that determines the timing to start sharing the screen to be shared, in accordance with the information indicating the display timing acquired by the acquisition unit 41.

[0112] The transmission unit 43 is an example of a transmission means that transmits a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the timing determined by the determination unit .

[0113] (Operation of the screen sharing control device 40) Next, an example of the operation of the screen sharing control device 40 in this embodiment will be described with reference to FIG.

[0114] First, the acquisition unit 41 acquires information indicating the display timing of the screen to be shared (step S41). Next, the determination unit 42 determines the timing to start sharing the screen to be shared according to the information indicating the display timing acquired by the acquisition unit 41 (step S42). Next, the transmission unit 43 transmits a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared according to the timing determined by the determination unit 42 (step S43).

[0115] According to the screen sharing control device of the fourth embodiment, it is possible to obtain the same advantageous effects as those of the screen sharing control device of the first and third embodiments.

[0116] Next, the screen sharing control device 50 according to the present disclosure will be described with reference to FIGS.

[0117] (Configuration of the screen sharing control device 50) As shown in FIG. 28, the screen sharing control device 50 of this embodiment includes an acquisition unit 51, a determination unit 52, and a transmission unit 53.

[0118] The acquisition unit 51 is an example of an acquisition means for acquiring work information indicating the work status of a user with whom the screen to be shared is to be shared.

[0119] The determination unit 52 is an example of a determination means that estimates when the sharing destination user has free time based on the task information, and determines the estimated timing as the timing to start sharing the screen to be shared.

[0120] The transmitting unit 53 is an example of a transmitting means that transmits, in accordance with the timing determined by the determining unit 52, a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared.

[0121] (Operation of the screen sharing control device 50) Next, an example of the operation of the screen sharing control device 50 in this embodiment will be described with reference to FIG.

[0122] First, the acquisition unit 51 acquires task information indicating the task status of the user at the sharing destination of the screen to be shared (step S51). Next, the determination unit 52 estimates when the user at the sharing destination becomes free according to the task information, and determines the estimated timing as the timing to start sharing the screen to be shared (step S52). Next, the transmission unit 53 transmits a control signal for displaying the screen to be shared to at least one terminal device at the sharing destination of the screen to be shared, according to the timing determined by the determination unit 52 (step S53).

[0123] According to the screen sharing control device of the fifth embodiment, it is possible to obtain the same advantageous effects as those of the screen sharing control device of the second and third embodiments.

[0124] [Sixth embodiment] Next, a screen sharing control system 6 according to the present disclosure will be described with reference to FIG.

[0125] The screen sharing control system 6 of this embodiment includes the screen sharing control device according to any one of the first to fifth embodiments, and at least one smart glass 60. In Fig. 30, as an example, the screen sharing control system 6 is illustrated as including the screen sharing control device 40 of the fourth embodiment.

[0126] At least one or more pieces of smart glasses are worn by at least one of a sharing destination user U and a sharing source user U.

[0127] The screen sharing control system 6 of this embodiment provides the following advantages in addition to the advantages provided by the screen sharing control device of the first, second, third and fourth embodiments.

[0128] The image of the virtual space displayed in the field of view of the user wearing the smart glasses is not displayed in the real world. The screen displayed on a PC display can be seen by other users, for example, by peering in from the side, but the screen displayed on the smart glasses cannot be seen by other users even if they peer in. In other words, there was an issue that the screen of the virtual space displayed in the field of view of the user wearing the smart glasses could not be shared as easily as the screen displayed on the PC display.

[0129] In contrast, the screen sharing control system of the present embodiment includes smart glasses and the screen sharing control device described in any one of the first to fifth embodiments. The screen sharing control device performs screen sharing when a user wearing the smart glasses makes a screen sharing request. That is, according to the screen sharing control system of the present embodiment, the screen of a user wearing the smart glasses can be easily shared.

[0130] [Computer hardware configuration] The functions of each of the components in each of the embodiments of the present disclosure described above can be realized not only by hardware but also by a computer device and firmware based on program control.

[0131] Fig. 31 is a diagram showing an example of a hardware configuration in which the screen sharing control device in the present disclosure is realized by a computer device 90 including a processor. The screen sharing control device in each embodiment is realized by the computer device 90. As shown in Fig. 31, the computer device 90 includes a CPU (Central Processing Unit) 91, a memory 92, a storage device 93 such as a hard disk for storing programs, an input / output interface 94 for connecting input devices and output devices, and a communication interface 95 for network connection. The input / output interface 94 and the communication interface 95 may be integrated as an interface with the outside.

[0132] The CPU 91 runs an operating system to control the screen sharing control device of the present disclosure. For example, the CPU 91 reads out programs and data from a storage medium attached to a drive device or the like into the memory 92. The CPU 91 also functions as, for example, a part of the acquisition unit 11, decision unit 12, or transmission unit 13 in the screen sharing control device 10 of the present disclosure, and executes processes or commands based on the programs.

[0133] The storage device 93 is, for example, an optical disk, a flexible disk, a magneto-optical disk, an external hard disk, or a semiconductor memory. A part of the storage device is a storage medium that is a non-volatile storage device, and the program is recorded therein. The program may also be downloaded from an external computer (not shown) connected to a communication network.

[0134] The input device connected to the input / output interface 94 is realized by, for example, a mouse, a keyboard, etc., and is used for input operations. Similarly, the output device connected to the input / output interface 94 is realized by, for example, a display, etc., and is used for displaying and checking output results.

[0135] Although the present disclosure has been described above with reference to each embodiment, the present disclosure is not limited to the above embodiment. Various changes that a person skilled in the art can understand within the scope of the present disclosure can be made to the configuration and details of the present disclosure. For example, although multiple operations are described in order in the form of a flowchart, the order of description does not limit the order in which the multiple operations are performed. Therefore, when implementing each embodiment, the order of the multiple operations can be changed to the extent that the content is not affected.

[0136] Some or all of the above embodiments can be described as follows:

[0137] (Appendix 1) An acquisition means for acquiring information indicating a display timing of a screen to be shared; a determination means for determining a timing to start sharing the screen to be shared according to the information indicating the display timing; a transmission means for transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the determined timing; A screen sharing control device comprising: (Appendix 2) the information indicating the display timing is schedule information including a start point of sharing the screen to be shared, The screen sharing control device according to claim 1, wherein the plan information includes at least one of a user's schedule information and a conference scenario information. (Appendix 3) The screen sharing control device described in Appendix 2, wherein the determination means detects a start point for starting sharing of the screen to be shared from the acquired schedule information, and determines the detected start point as the timing for starting sharing of the screen to be shared. (Appendix 4) the acquiring means acquires user conversation information as information indicating the display timing; The screen sharing control device according to claim 1, wherein the determination means determines the timing to start sharing of a shared screen related to a predetermined keyword when the conversation information satisfies a sharing start condition related to the keyword. (Appendix 5) when the conversation information satisfies the sharing start condition, the determination means transmits, to a terminal device operated by a user, an input screen requesting an input as to whether or not to share a shared target screen related to the keyword and an input of a timing to start sharing; The acquiring means acquires input information input via the input screen displayed on the terminal device, The screen sharing control device according to claim 4, wherein, when the input information includes information indicating that a shared screen related to the keyword is to be shared, the determination means determines a timing to start sharing the shared screen related to the keyword. (Appendix 6) An acquisition means for acquiring work information indicating a work status of a user who is a sharing destination of the screen to be shared; a determination means for determining when the user at the sharing destination has free time according to the task information, and determining the estimated time as a time to start sharing the screen to be shared; a transmission means for transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the determined timing; A screen sharing control device comprising: (Appendix 7) The acquisition means further acquires a sharing request including information about the screen to be shared and a sharing destination terminal device from a sharing source terminal device; the transmission means transmits, in accordance with the determined timing, an input screen for prompting the user to accept or decline the sharing request to at least one terminal device that is a sharing destination of the screen to be shared; The acquiring means further acquires input information indicating whether or not to permit the sharing request from at least one terminal device that is a sharing destination of the screen to be shared; The screen sharing control device according to claim 6, wherein when input information indicating that the sharing request is permitted is obtained, the transmission means transmits a control signal to at least one terminal device with which the screen to be shared is to be displayed. (Appendix 8) A screen sharing control device according to any one of Supplementary Notes 1 to 7; One or more smart glasses worn by at least one of a sharing destination user and a sharing source user; A screen sharing control system comprising: (Appendix 9) The smart glasses include a gaze information acquisition means for acquiring gaze information including a user's gaze position, The screen sharing control system of claim 8, wherein the transmission means incorporates the timing for updating the screen to be shared into the control signal using a gaze position received from the smart glasses worn by the user who is the source of sharing. (Appendix 10) The smart glasses include a gaze information acquisition means for acquiring gaze information including a user's gaze position, The screen sharing control system according to claim 8, wherein the transmission means further transmits to the destination terminal device a control signal for superimposing on the screen to be shared a marker representing the gaze position of the source user received from the source terminal device. (Appendix 11) The computer Acquire information indicating the display timing of the screen to be shared, determining a timing to start sharing the screen to be shared according to the information indicating the display timing; a control signal for displaying the screen to be shared, to at least one terminal device with which the screen to be shared is shared, in accordance with the determined timing; (Appendix 12) The computer Acquire work information indicating the work status of a user to whom the shared screen is to be shared; Inferring when the user at the sharing destination has free time according to the task information, and determining the estimated time as a time to start sharing the screen to be shared; a control signal for displaying the screen to be shared, to at least one terminal device with which the screen to be shared is shared, in accordance with the determined timing; (Appendix 13) On the computer, A process of acquiring information indicating a display timing of a screen to be shared; A process of determining a timing to start sharing the screen to be shared according to the information indicating the display timing; a process of transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, according to the determined timing; A program to execute. (Appendix 14) On the computer, A process of acquiring work information indicating a work status of a user who is a sharing destination of the shared screen; A process of estimating when the user at the sharing destination has free time according to the task information, and determining the estimated time as a time to start sharing the screen to be shared; a process of transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, according to the determined timing; A program to execute. [Explanation of symbols]

[0138] 1, 2, 3, 6 Screen Sharing Control System 10, 20, 30, 40, 50 screen sharing controller 11, 21, 31, 41, 51 Acquisition section 12, 22, 32, 42, 52 Decision section 13, 23, 33, 43, 53 Transmitter 14, 24, 34 storage section 60 Smart Glasses 70, 80 Terminal equipment 90 Computer Equipment 91 CPU(Central Processing Unit) 92 Memory 93 Storage device 94 Input / Output Interface 95 Communication Interface

Claims

1. An acquisition means for acquiring information indicating a display timing of a screen to be shared; a determination means for determining a timing to start sharing the screen to be shared according to the information indicating the display timing; a transmission means for transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the determined timing; A screen sharing control device comprising:

2. An acquisition means for acquiring work information indicating a work status of a user who is a sharing destination of the screen to be shared; a determination means for determining when the user at the sharing destination has free time according to the task information, and determining the estimated time as a time to start sharing the screen to be shared; a transmission means for transmitting a control signal for displaying the screen to be shared to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the determined timing; A screen sharing control device comprising:

3. The acquisition means further acquires a sharing request including information about the screen to be shared and a sharing destination terminal device from a sharing source terminal device; the transmission means transmits, in accordance with the determined timing, an input screen for prompting the user to accept or decline the sharing request to at least one terminal device that is a sharing destination of the screen to be shared; The acquiring means further acquires input information indicating whether or not to permit the sharing request from at least one terminal device that is a sharing destination of the screen to be shared; The screen sharing control device according to claim 2, wherein when input information indicating that the sharing request is permitted is acquired, the transmission means transmits a control signal to at least one terminal device with which the screen to be shared is to be displayed.

4. the acquiring means acquires user conversation information as information indicating the display timing; The screen sharing control device according to claim 1 , wherein, when the conversation information satisfies a sharing start condition related to a predetermined keyword, the determination means determines a timing to start sharing of a screen to be shared related to the keyword.

5. A screen sharing control device according to any one of claims 1 to 4, One or more smart glasses worn by at least one of a sharing destination user and a sharing source user; A screen sharing control system comprising:

6. The smart glasses include a gaze information acquisition means for acquiring gaze information including a user's gaze position, The screen sharing control system according to claim 5 , wherein the transmission means incorporates the timing of updating the screen to be shared into the control signal using a gaze position received from the smart glasses worn by the user who is the source of sharing.

7. The smart glasses include a gaze information acquisition means for acquiring gaze information including a user's gaze position, The screen sharing control system according to claim 5, wherein the transmission means further transmits to the sharing destination terminal device a control signal for superimposing on the screen to be shared a marker representing the gaze position of the sharing source user received from the sharing source terminal device.

8. The computer Acquire information indicating the display timing of the screen to be shared, determining a timing to start sharing the screen to be shared according to the information indicating the display timing; A screen sharing control method, comprising: transmitting, in accordance with the determined timing, a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared.

9. The computer Acquire work information indicating the work status of a user to whom the shared screen is to be shared; Inferring when the user at the sharing destination has free time according to the task information, and determining the estimated time as a timing to start sharing the screen to be shared; A screen sharing control method, comprising: transmitting, in accordance with the determined timing, a control signal for displaying the screen to be shared to at least one terminal device with which the screen to be shared is to be shared.

10. On the computer, A process of acquiring information indicating a display timing of a screen to be shared; A process of determining a timing to start sharing the screen to be shared according to the information indicating the display timing; a process of transmitting a control signal to at least one terminal device that is a sharing destination of the screen to be shared, in accordance with the determined timing, to cause the screen to be shared to be displayed; A program to execute.

Images