Information processing device, information processing method, and program

Abstract

To provide an information processing device capable of maintaining the convenience of other devices even when a request is made from one screen sharing destination device (smartphone, etc.), an information processing method, and a program.SOLUTION: An information processing device 10 comprises: an output unit 142, which is a first display unit; a screen generation unit 115 which generates a screen displayed in the first display unit; and a transmission and reception control unit 114 which transmits the screen to external terminal 20a and 20b via a network. The information processing device further comprises: an operation detection unit 111 which detects an operation of a user for the screen displayed in a second display unit of the external terminal; a display control unit 117 which performs control to cause a selection screen for selecting whether or not to permit the execution of an event based on the operation to be displayed to at least one device among the information processing device and the external terminals according to the screen and the operation; and an event execution determination unit 116 which determines whether or not to permit the execution of the event based on the operation on the basis of selection results of the user for the selection screen.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In a mirroring system that shares screens between multiple devices, there are cases where the screen of a screen sharing source device (such as an in-vehicle device) is uniformly controlled based on a request from one screen sharing destination device (such as a smartphone).

[0003] Patent Document 1 discloses an information processing device including a wireless communication unit and a control unit. This wireless communication unit performs data communication with other information processing devices. Also, it is assumed that data transmission is newly performed between the information processing device and a first information processing device that plays the role of a group owner of a network formed by at least the information processing device itself and the first information processing device. In this case, the control unit controls the exchange of information for updating the roles for performing data transmission between the first information processing device and the second information processing device. Patent Document 2 discloses a 5G or pre-5G communication system provided to support higher data transmission rates after 4G communication systems such as LTE. This is a method for using a service in which a plurality of electronic devices are linked to a first electronic device, and includes a step of connecting the first electronic device to a central display device and mirroring the service on the central display device based on a playlist of the service stored in the first electronic device. It also includes a step of receiving a request to change the playlist from the second electronic device and updating the playlist in response to the change request. It also includes a step of transmitting updated playlist-related information to the central display device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2016 / 163181 [Patent Document 2] Special table 2019-511798 publication Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the display screen is controlled by one screen sharing destination based on the request, it is not convenient for devices other than the device that is the one screen sharing destination. The present invention aims to provide an information processing device, an information processing method, and a program that can maintain the convenience of other devices even when a request is made from one screen sharing device. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides an information processing device comprising a first display unit, a screen generation unit which generates a screen to be displayed on the first display unit, and a transmission / reception control unit which transmits the screen to a terminal device via a network, the information processing device also comprising: an operation detection unit which detects a user's operation performed on a screen displayed on a second display unit provided on the terminal device; a display control unit which controls the display, on at least one of the information processing device and the terminal device, of a selection screen which allows the user to select whether or not to allow execution of an event based on the operation, depending on the screen and the operation; and an event execution determination unit which determines whether or not to allow execution of an event based on the operation, based on the user's selection on the selection screen. Effect of the Invention

[0007] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that can maintain the convenience of other devices even when a request is made from one screen sharing device. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an image control system to which the present invention is applied. [Diagram 2] 1A and 1B are diagrams showing a state in which a multimedia playback screen is displayed on an information processing device and an external terminal. [Diagram 3] 13A to 13C are diagrams showing screens displayed on each device when a user of an external terminal performs a pause operation. [Figure 4] 4 is a sequence diagram showing processes performed by an information processing device and an external terminal in the first embodiment. FIG. [Diagram 5] 11A and 11B are diagrams illustrating a state in which a map screen is displayed on the information processing device and the external terminal. [Figure 6] 13A to 13C are diagrams showing screens displayed on each device when a user of an external terminal performs a zoom-out operation on a map screen. [Figure 7] FIG. 11 is a sequence diagram showing processes performed by an information processing device and an external terminal in the second embodiment. [Figure 8] 1A and 1B are diagrams illustrating a state in which a display screen of a web page is displayed on an information processing device and an external terminal. [Figure 9] 13A to 13C are diagrams showing screens displayed on each device when a user of an external terminal performs an operation to display a linked screen. [Figure 10] FIG. 13 is a sequence diagram showing processes performed by an information processing device and an external terminal in the third embodiment. [Figure 11] FIG. 11 is a diagram showing notification information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. <Outline of image control system 1> FIG. 1 is a block diagram showing the configuration of an image control system 1 to which the present invention is applied. The image control system 1 is mounted on a vehicle, which is an example of a moving body. The image control system 1 includes an information processing device 10 and external terminals 20a and 20b. The information processing device 10 and the external terminals 20a and 20b are connected to each other via a network 30.

[0010] The information processing device 10 generates image information of a screen to be shared with the external terminals 20a and 20b, and displays this screen. The information processing device 10 is a computer device. More specifically, the information processing device 10 is a car navigation device, a car audio device, an automobile control system, or the like. As shown in the figure, the information processing device 10 includes a processor 11 such as a CPU (Central Processing Unit), a memory 12 as a storage means, and a persistent storage device 13 such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The processor 11 executes various software such as an OS (operating system software) and an application program (application software). The memory 12 is a storage area that stores various software and data used for executing the software, and the persistent storage device 13 is a storage area that stores input data for various software and output data from various software. The information processing device 10 also includes a user interface (hereinafter referred to as "user IF") 14 for exchanging information with the user, and a communication interface (hereinafter referred to as "communication IF") 15 for communicating with the outside.

[0011] The processor 11 performs processing to realize the functions of an operation detection unit 111, an event analysis unit 112, an application execution unit 113, a transmission / reception control unit 114, a screen generation unit 115, an event execution determination unit 116, a display control unit 117, and a notification information selection unit 118.

[0012] The operation detection unit 111 detects user operations performed on the screens displayed on the external terminals 20a and 20b. The event analysis unit 112 analyzes the contents of an event based on the operation of the user of the external terminal 20a, 20b. That is, the event analysis unit 112 analyzes the contents of the operation intended by the user who operates the external terminal 20a, 20b. In addition, the event analysis unit 112 judges whether the user operation of the external terminal 20a, 20b is a predetermined operation.

[0013] The application execution unit 113 executes applications that run on the information processing device 10. The applications to be executed will be described later. The transmission / reception control unit 114 controls transmission and reception of information between the external terminals 20a and 20b via the communication IF 15. It can also be said that the transmission / reception control unit 114 transmits the same screen as that displayed on the output unit 142 to the external terminals 20a and 20b via the network 30.

[0014] The screen generating unit 115 generates a screen to be displayed on the output unit 142 in response to the execution of an application. The event execution determination unit 116 determines whether or not to permit execution of an event based on a user's operation, based on the result of the user's selection on a selection screen, which will be described later. A specific method of this determination will be described later. The display control unit 117 controls the display of the screen generated by the screen generation unit 115 on the output unit 142. The display control unit 117 also controls the display of a selection screen for allowing the user to select whether or not to permit execution of an event based on the operation result, on at least one of the information processing device 10 and the external terminals 20a and 20b, depending on the screen being displayed on the output unit 142 and the user's operation result on the screen. The screen being displayed on the output unit 142 is a screen that is displayed in conjunction with the execution of an application. In this case, it can be said that the display control unit 117 selects a device that displays the selection screen from the information processing device 10 and the external terminals 20a and 20b, depending on the screen displayed by the application and the contents of the operation intended by the user, and controls the display of the selection screen on the selected device. The specific contents of this will be described later. The notification information selection unit 118 selects notification information to be transmitted to the information processing device 10 and the external terminals 20a and 20b. The notification information is selected based on the results of operations performed by each user on the screens displayed on the information processing device 10 and the external terminals 20a and 20b. Details of the notification information will be described later with reference to FIG. 11.

[0015] The persistent storage device 13 stores notification information 131 selected by the persistent storage device 13. Here, the notification information 131 stores notification information regarding whether permission is required, whether approval is pending, whether rejection is required, and whether an operation is required. The user IF 14 includes an input unit 141 and an output unit 142. The input unit 141 is used when the user makes an input to the information processing device 10. The output unit 142 is an example of a first display unit, and displays a screen or information for the user. In the present embodiment, the input unit 141 and the output unit 142 are touch panels having these functions. However, the present invention is not limited to this, and the input unit 141 may be a keyboard, a mouse, a touch pad, or the like. The output unit 142 may be a display device such as a liquid crystal display that does not have a touch panel function.

[0016] The external terminals 20a and 20b are examples of terminal devices that share a screen with the information processing device 10. The external terminals 20a and 20b are also computer devices similar to the information processing device 10, and are composed of a processor, a memory, and the like. The external terminals 20a and 20b are provided with a second display unit that can display the same screen as the screen transferred from the information processing device 10 and displayed on the output unit 142. The second display unit is, for example, a touch panel. The external terminals 20a and 20b may also have a configuration similar to that of the information processing device 10 as shown in FIG. 1. The external terminals 20a and 20b are, for example, portable terminal devices carried by a user, specifically, smartphones, tablets, and the like. Alternatively, they may be display devices for the rear seats, and the like. Here, two external terminals 20a and 20b are displayed as the external terminals 20a and 20b, but the number of the external terminals may be any number.

[0017] The network 30 may use wireless communication lines such as a mobile phone line, Wi-Fi (Wireless Fidelity, registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), or UWB (Ultra-Wide Band). The network 30 may also use wired communication lines such as a wired LAN (Local Area Network) line, USB (Universal Serial Bus), optical communication lines, or public telephone lines. The network 30 may use both wireless communication lines and wired communication lines, or may also use the Internet, etc.

[0018] <Operation of the information processing device 10> Three embodiments of the operation of the information processing device 10 will be described.

[0019] [First embodiment] In the first embodiment, a case will be described in which the application execution unit 113 executes video playback software for playing multimedia as an application. In this case, the screen generation unit 115 generates a multimedia playback screen. Then, the display control unit 117 displays the multimedia playback screen on the output unit 142, and transmits an instruction to the external terminals 20a and 20b to display the multimedia playback screen transferred from the information processing device 10.

[0020] FIG. 2 is a diagram showing a state in which a multimedia playback screen is displayed on the information processing device 10 and the external terminals 20a and 20b. Here, the screen of the information processing device 10 is displayed in the front seat of the vehicle. Also, the screen of the external terminal 20a is displayed in the rear seat (right), and the screen of the external terminal 20b is displayed in the rear seat (left). In this case, the user of the information processing device 10 is, for example, the driver of the vehicle. And the users of the external terminals 20a and 20b are, for example, people who watch the playback screen of multimedia on their own smartphones in the rear seat. In this case, the multimedia image information generated by the information processing device 10 is transferred to the external terminals 20a and 20b, and a common screen is displayed on the information processing device 10 and the external terminals 20a and 20b. In addition, the case where the user of the external terminal 20a touches the pause button Bt2 for pausing as an operation related to the screen is shown.

[0021] The event analysis unit 112 judges whether the operation of the user of the external terminal 20a is a predetermined operation, and if the operation is not a predetermined operation, judges the event based on the operation to be invalid. The process to be executed with the screen operation as a trigger is predefined in the application that realizes the screen sharing of the information processing device 10. In this case, the operation target to be analyzed can be limited. In the example of FIG. 2, a previous track skip button Bt1, a pause button Bt2, and a next track skip button Bt3 are provided, and events based on operations other than the corresponding skip previous track, pause, and skip next track are disabled.

[0022] 3 is a diagram showing a screen displayed on each device when the user of the external terminal 20a performs a pause operation. The fact that the user of the external terminal 20a performs a pause operation is an analysis result obtained by the event analysis unit 112. Then, based on the analysis result of the event analysis unit 112, the notification information selection unit 118 selects notification information to be notified to the information processing device 10 and the external terminals 20a and 20b. In this case, the display control unit 117 controls the information processing device 10 and an external terminal other than the external terminal where the operation is performed to display a dialog A as a selection screen for selecting whether or not to permit execution of an event based on the operation (here, a pause). Specifically, the display control unit 117 controls the information processing device 10 to display the dialog A, transmits a dialog display request (notification information 131) to the external terminal 20b, and causes the external terminal 20b to display the dialog A. Here, the dialog A is displayed based on the selection result of the notification information selection unit 118. FIG. 3 shows a case where the dialog A including the message "There is an operation request from the rear seat (right). Do you permit the operation?" is displayed to the information processing device 10 and the external terminal 20b. The users of the information processing device 10 and the external terminal 20b touch the "Yes" button in the dialog A if they permit it, and touch the "No" button if they do not permit it. The figure shows that the user of the information processing device 10 touches the "Yes" button, and the user of the external terminal 20b touches the "No" button. Dialog A is deleted after the user's touch operation. Furthermore, while the user of the information processing device 10 or the external terminal 20b is selecting whether to permit or not permit, a dialog B including a message "operation request in progress" is displayed on the external terminal 20a where the operation was performed.

[0023] Possible conditions for the event execution determination unit 116 to permit the execution of an event based on an operation include, for example, permission by a majority vote, permission for a specific seat (e.g., the driver's seat), permission from the source of an image, etc. It is preferable to set a time limit for allowing the user to select whether or not to permit the execution of an event based on an operation. When this time limit is exceeded (timeout occurs), the information processing device 10 tallies the replies obtained at that time.

[0024] The selection screen is displayed on the device that is affected by the execution of the event based on the operation, among the information processing device 10 and the external terminals 20a and 20b. When the information processing device 10 shares a multimedia playback screen with the external terminals 20a and 20b, for example, if the user of the external terminal 20a performs a pause operation, this operation affects everyone who watches the multimedia playback screen. Specifically, the screen may suddenly change or a pop-up may appear. Therefore, a selection screen that allows users other than the user who performed the operation to select whether or not to permit the execution of the event based on the operation (pause in this case) is displayed on the device of the user who performed the operation, and the users are prompted to select whether or not to permit the execution of the event based on the operation. This gives the users other than the user who performed the operation the authority to not permit the execution of the event based on the operation, and allows them to anticipate in advance the screen changing or the pop-up appearing after the permission of the execution of the event based on the operation. In this case, the display control unit 117 can also be said to control the information processing device 10 and the external terminal 20b to display dialog A, which is a selection screen that allows the user to select whether or not to allow an event based on the operation (here, pause), depending on the screen displayed by the video playback software and the operation performed by the user.

[0025] When the information processing device 10 permits the execution of an event based on an operation according to the selection result from each user on the selection screen, the information processing device 10 executes processing based on the permitted event. In this case, the event based on the pause operation performed by the user of the external terminal 20a is executed, and the multimedia playback screen is paused. On the other hand, when the execution of an event based on an operation is not permitted, the event based on the disallowed operation is not executed. In this case, the event based on the pause operation performed by the user of the external terminal 20a is rejected, and the multimedia playback screen in each device is not paused. In addition, a dialogue C including a message "The operation has been rejected" is displayed on the external terminal 20a. The dialogue C is deleted when a predetermined time has elapsed since it was displayed.

[0026] FIG. 4 is a sequence diagram showing processes performed by the information processing device 10 and the external terminals 20a and 20b in the first embodiment. The information processing device 10 generates a multimedia playback screen as a screen to be transferred to be shared with the external terminals 20a and 20b (S101). The information processing device 10 displays a multimedia playback screen on the information processing device 10, and transmits image information of this screen to the external terminals 20a and 20b (S102). The external terminals 20a and 20b receive the image information sent from the information processing device 10, and display the same multimedia playback screen as that of the information processing device 10 on each of the external terminals 20a and 20b (S103, S104).

[0027] When the user of the external terminal 20a touches the screen while the multimedia playback screen is being displayed, an event based on the operation is sent to the information processing device 10 (S105). The information processing device 10 analyzes the content of the event based on the operation of the user of the external terminal 20a (S106). S105 to S106 can be considered as an operation phase in which screen operations from the user of the external terminal 20a are accepted.

[0028] If the operation is a predetermined operation, the information processing device 10 displays a dialog A as a selection screen for allowing the user to select whether or not to permit execution of an event based on the operation by the user of the external terminal 20a (S107). Meanwhile, the information processing device 10 transmits a dialog display request (notification information 131) to the external terminals 20a and 20b (S108). This can be realized by the information processing device 10 sending the dialog display request (notification information 131) to the external terminals 20a and 20b. As a result, the external terminal 20a displays a dialog B including a message "operation request in progress" (S109). Furthermore, the external terminal 20b displays a dialog A similar to that of the information processing device 10 (S110). The information processing device 10 accepts an operation for dialog A from a user of the information processing device 10 (S111). The information processing device 10 also accepts an operation for dialog A from a user of the external terminal 20b (S112). That is, the information processing device 10 accepts the selection result of each user, "Yes" or "No" in the selection in dialog A. S107 to S112 can be considered as an approval phase for approving a screen operation from the user of the external terminal 20a.

[0029] The information processing device 10 tallies the operation receptions and determines whether or not an event based on an operation from the user of the external terminal 20a can be executed (S113). The information processing device 10 transmits a processing request corresponding to the determination result of S113 (S114). As a result, the information processing device 10 performs a process corresponding to the result of the determination (S115). In this case, this process is either to reflect an event based on the operation or to do nothing. The external terminal 20a also performs processing corresponding to the judgment result based on an instruction from the information processing device 10 (S116). In this case, this processing is either to delete the dialog B and display a screen displayed on the information processing device 10 reflecting the event based on the operation, or to delete the dialog B and display a dialog C including a message "The operation has been rejected." This is the request result for the operation performed by the user of the external terminal 20a. Note that the dialog C is deleted when a predetermined time has elapsed since it was displayed. The external terminal 20b also performs processing corresponding to the determination result based on an instruction from the information processing device 10, but in this case, does nothing (S117). That is, the external terminal 20b shares the screen displayed on the information processing device 10 in S115. Specifically, the external terminal 20b displays a screen displayed on the information processing device 10 reflecting an event based on an operation by the user of the external terminal 20a, or a screen that was displayed on the information processing device 10 before the operation by the user of the external terminal 20a. S113 to S117 can be considered as an execution phase in which a screen operation by the user of the external terminal 20a is executed.

[0030] Second Embodiment In the second embodiment, a case will be described where the application execution unit 113 executes navigation software as an application. In this case, the screen generation unit 115 generates a map screen. Then, the display control unit 117 displays the map screen on the output unit 142, and transmits an instruction to the external terminals 20a and 20b to display the map screen transferred from the information processing device 10.

[0031] FIG. 5 is a diagram showing a state in which map screens are displayed on the information processing device 10 and the external terminals 20a and 20b. In this case, image information of the map screen generated by the information processing device 10 is transferred to the external terminals 20a and 20b, and a common map screen is displayed on the information processing device 10 and the external terminals 20a and 20b. In addition, a case is shown in which the user of the external terminal 20a touches the zoom out button Bt5 for zooming out the map screen as an operation related to the screen.

[0032] In the example of FIG. 5, a zoom-in button Bt4 and a zoom-out button Bt5 are provided, and events based on operations other than the zoom-in and zoom-out operations corresponding to these buttons are disabled.

[0033] FIG. 6 shows the screens displayed on each device when the user of the external terminal 20a performs a zoom-out operation on the map screen. The display control unit 117 controls the information processing device 10 to display the dialog A as a selection screen for selecting whether or not to permit the execution of an event based on an operation (zooming out the map in this case). On the other hand, the display control unit 117 does not control the external terminals 20a and 20b to display a selection screen, and the dialog A is not displayed on the external terminals 20a and 20b. Fig. 6 shows a case in which the dialog A including the message "There is an operation request from the back seat (right). Do you want to permit the operation?" is displayed on the information processing device 10. While the user of the information processing device 10 is selecting whether to permit or not, the external terminal 20a displays a dialog B including a message "operation request in progress." Here, the reason why the external terminal 20a displays the dialog B is because the display control unit 117 transmits a dialog display request (notification information 131) to the external terminal 20a based on the selection result of the notification information selection unit 118. Note that no dialog is displayed on the external terminal 20b.

[0034] When a map screen is displayed as a shared screen, for example, if the user of the external terminal 20a performs an operation to zoom out the map, the event based on this operation mainly affects the driver who is driving in the front seat. That is, if an unintended operation is performed on the map screen displayed on the information processing device 10 while driving, it may affect driving. On the other hand, the impact on the user of the external terminal 20b is small. Therefore, the display control unit 117 controls the information processing device 10 to display a selection screen that allows the user to select whether or not to permit the execution of the event based on this operation (here, zooming out the map). On the other hand, the display control unit 117 does not control the external terminal 20b to display a selection screen. That is, the dialogue A is not displayed on the external terminal 20b. In this case, it can be said that the display control unit 117 controls the information processing device 10 to display the dialogue A, which is a selection screen that allows the user to select whether or not to permit the execution of the event based on the operation, according to the screen displayed by the navigation software and the operation performed by the user (here, zooming out the map).

[0035] When the information processing device 10 permits the execution of an event based on an operation in response to a selection result from the user of the information processing device 10 on the selection screen, the information processing device 10 executes processing based on the permitted event. In this case, an event based on the operation of zooming out the map performed by the user of the external terminal 20a is executed, and the map zooms out. On the other hand, when the event based on the operation is not permitted, the event based on the disallowed operation is not executed. In this case, the event based on the operation of zooming out the map performed by the user of the external terminal 20a is rejected, and the map screen does not zoom out. In this case, a dialogue C including a message "The operation has been rejected" is displayed on the external terminal 20a. The dialogue C is deleted when a predetermined time has elapsed since it was displayed.

[0036] FIG. 7 is a sequence diagram showing processes performed by the information processing device 10 and the external terminals 20a and 20b in the second embodiment. S201 to S207 in Fig. 7 are the same as S101 to S107 in Fig. 4. However, the operation by the user of the external terminal 20a is changed from the operation of pausing to an operation of zooming out the map screen. The process after S208 in FIG. 7 is as follows. The information processing device 10 transmits a dialog display request (notification information 131) to the external terminal 20a (S208). As a result, the external terminal 20a displays a dialog B including a message "operation request in progress" (S209). Unlike the case of FIG. 4, the dialog display request (notification information 131) is sent only to the external terminal 20a, and no dialog is displayed on the external terminal 20b. The information processing device 10 accepts an operation for the dialog A from the user of the information processing device 10 (S210). Unlike the case of Fig. 4, since the dialog is not displayed on the external terminal 20b, the information processing device 10 does not accept an operation for the dialog A from the user of the external terminal 20b. 7 are similar to S113 to S117 in FIG.

[0037] In the second embodiment, S205 to S206 can be considered as an operation phase for accepting screen operations from the user of the external terminal 20a. S207 to S210 can be considered as an approval phase for approving the screen operations from the user of the external terminal 20a. S211 to S215 can be considered as an execution phase for executing the screen operations from the user of the external terminal 20a.

[0038] Third embodiment In the third embodiment, a case will be described in which the application execution unit 113 executes browsing software such as a browser as an application. In this case, the screen generation unit 115 generates a display screen of a web page. Then, the display control unit 117 displays the display screen of the web page on the output unit 142, and transmits an instruction to the external terminals 20a and 20b to display the display screen of the web page transferred from the information processing device 10.

[0039] FIG. 8 is a diagram showing a state in which a web page display screen is displayed on the information processing device 10 and the external terminals 20a and 20b. In this case, image information of the display screen of the web page generated by the information processing device 10 is transferred to the external terminals 20a and 20b, and a display screen of the common web page is displayed on the information processing device 10 and the external terminals 20a and 20b. In addition, the case where the user of the external terminal 20a touches the link button Bt6 for transitioning to another web page as an operation related to the screen is shown. In this case, the event based on this operation is to transition the screen to the link destination.

[0040] In the example of FIG. 8, a link button Bt6 is provided, and events based on operations other than those for transitioning to another web page, which is the corresponding operation, are disabled.

[0041] FIG. 9 shows screens displayed on each device when the user of the external terminal 20a performs an operation to display a screen at a link destination. The display control unit 117 controls the display of dialog D on the external terminal 20a on which the operation was performed as a selection screen for selecting whether or not to permit execution of an event based on the operation. On the other hand, the display control unit 117 does not control the display of the selection screen on other devices. In other words, no dialog is displayed on devices other than the external terminal 20a on which the operation was performed. FIG. 9 shows a case in which dialog D containing a message "Do you want to open this page in a browser?" is displayed on the external terminal 20a.

[0042] When a web page display screen is displayed as a shared screen and an operation is performed to transition to another web page, the event based on this operation only affects the user of the external terminal 20a. On the other hand, the information processing device 10 and the external terminal 20b display a conventional web page, so there is no effect on these users. Therefore, the display control unit 117 controls the external terminal 20a to display a selection screen for selecting whether or not to permit the execution of the event based on this operation. On the other hand, the display control unit 117 does not control the display of the selection screen on devices other than the external terminal 20a. In other words, the dialogue D is not displayed on devices other than the external terminal 20a. In this case, it can be said that the display control unit 117 controls the external terminal 20a to display the dialogue D, which is a selection screen for selecting whether or not to permit the execution of the event based on the operation, according to the screen displayed by the browsing software and the operation performed by the user (here, screen transition to a link destination).

[0043] When the user of the external terminal 20a permits an event based on the operation, that is, when the "Yes" button in the dialogue D is selected, the screen sharing between the external terminal 20a and the information processing device 10 is released, and the linked screen is displayed by the browsing software included in the external terminal 20a where the operation was performed. In this case, releasing the screen sharing means that the information processing device 10 stops transferring (sending) the shared screen to the external terminal 20a. In this way, the shared screen is not controlled by the external terminal 20a that performed the screen operation, and the information processing device 10 and the external terminal 20b can maintain the screen before the operation by the user of the external terminal 20a. In addition, the external terminal 20a can transition to another web page as the user operates. That is, the convenience of each device is improved.

[0044] FIG. 10 is a sequence diagram showing processes performed by the information processing device 10 and the external terminals 20a and 20b in the third embodiment. S301 to S306 in Fig. 10 are the same as S201 to S206 in Fig. 7. However, the operation by the user of the external terminal 20a is changed from the operation of zooming out the map screen to the operation of displaying the linked screen. The process after S307 in FIG. 10 is as follows. The information processing device 10 transmits a dialog display request (notification information 131) to the external terminal 20a (S307). As a result, the external terminal 20a displays a dialog D including a message "Do you want to open this page in your browser?" (S308). The information processing device 10 accepts an operation for the dialogue D from the user of the external terminal 20a (S309). That is, the information processing device 10 accepts an operation to select "Yes" or "No" in the dialogue D. 10 are similar to S211 and S212 in FIG. In the subsequent step S312, the information processing device 10 performs processing corresponding to the determination result, but in this case, nothing is done (S312). The external terminal 20a performs processing corresponding to the determination result based on an instruction from the information processing device 10 (S313). In this case, this processing is either to display the linked screen in the browser provided in the external terminal 20a or to do nothing. The external terminal 20b performs processing corresponding to the determination result based on an instruction from the information processing device 10, but in this case, does nothing (S314).

[0045] In the third embodiment, S305 to S306 can be considered as an operation phase for accepting screen operations from the user of the external terminal 20a. S307 to S309 can be considered as an approval phase for approving the screen operations from the user of the external terminal 20a. S310 to S314 can be considered as an execution phase for executing the screen operations from the user of the external terminal 20a.

[0046] <Explanation of Notification Information 131> FIG. 11 is a diagram showing the notification information 131. As shown in FIG. The notification information 131 is a processing request or notification content that the display control unit 117 transmits to the information processing device 10 and the external terminals 20a and 20b based on the selection result of the notification information selection unit 118. Here, the notification information 131 shown by IDs 1 to 5 is shown. Among them, IDs 1 to 4 correspond to permission required / not required, approval waiting, rejection, and operation required / not required shown in FIG. 1. Furthermore, the notification information of IDs 1 to 4 represents information of the dialogues A to D shown in FIGS. 4 to 10. Note that the notification content is not limited to this, and the specific content of the event based on the user's operation may be displayed on the dialogue, for example, "There is a request for operation (pause) from the back seat (right). Do you want to allow the operation?"

[0047] According to the embodiment described above, it is possible to provide the information processing device 10 and the image control system 1 that can maintain the convenience of other devices even when a request is made from one screen sharing destination device. As a result, it is possible to provide a safe and easy-to-use screen sharing function.

[0048] <Explanation of information processing method> The processing performed by the information processing device 10 in this embodiment can be regarded as an information processing method performed by the information processing device 10, which is equipped with a first display unit, a screen generation unit 115 that generates a screen to be displayed on the first display unit, the output unit 142, and a transmission / reception control unit 114 that transmits the screen to the external terminals 20a, 20b via the network 30, and which detects a user's operation performed on a screen displayed on a second display unit equipped on the external terminals 20a, 20b, and controls at least one of the information processing device 10 and the external terminals 20a, 20b to display a selection screen that allows the user to select whether or not to allow execution of an event based on the operation, depending on the screen and the operation, and determines whether or not to allow execution of an event based on the operation, based on the user's selection on the selection screen.

[0049] <Program Description> The processing performed by the information processing device 10 in this embodiment is prepared as a program such as application software. This processing is realized by the cooperation of software and hardware resources. That is, the processor 11 in the computer provided in the information processing device 10 executes a program that realizes each of the above-mentioned functions, and realizes each of these functions.

[0050] In this embodiment, the processing performed by the information processing device 10 can be understood as a program executed by a computer provided in the information processing device 10, which has a first display unit, a screen generation unit 115 that generates a screen to be displayed on the first display unit, the output unit 142, and a transmission / reception control unit 114 that transmits the screen to the external terminals 20a, 20b via the network 30, and which realizes the following functions: detecting a user's operation performed on a screen displayed on a second display unit provided in the external terminals 20a, 20b; controlling the display, on at least one of the information processing device 10 and the external terminals 20a, 20b, of a selection screen that allows the user to select whether or not to allow execution of an event based on the operation, depending on the screen and the operation; and determining whether or not to allow execution of an event based on the operation, based on the user's selection on the selection screen.

[0051] The program for implementing the present embodiment can be provided not only by communication means but also by being stored in a recording medium such as a CD-ROM.

[0052] Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the claims that various modifications and improvements to the above embodiment are also included in the technical scope of the present invention.

[0053] For example, in the above embodiment, the information processing device 10 is the screen sharing source, but in reality, the roles of the screen sharing source and the screen sharing destination can be reversed. That is, at a certain point in time, the external terminal 20a or the external terminal 20b is the screen sharing source, and the information processing device 10 is the screen sharing destination. In addition, for example, the operation phase, approval phase, and execution phase are provided to make the contents of the invention easier to understand, and are not necessarily clearly divided into these three phases. There is no particular limitation on the names, and for example, the approval phase may be called an inquiry phase. [Explanation of symbols]

[0054] 1...image control system, 10...information processing device, 11...processor, 12...memory, 20a, 20b...external terminal, 111...operation detection unit, 112...event analysis unit, 113...application execution unit, 114...transmission / reception control unit, 115...screen generation unit, 116...event execution determination unit, 117...display control unit, 118...notification information selection unit, 142...output unit, A to D...dialog

Claims

1. An information processing device comprising: a first display unit; a screen generating unit that generates a screen to be displayed on the first display unit; and a transmission / reception control unit that transmits the screen to a terminal device via a network, an operation detection unit that detects an operation performed by a user on the screen displayed on a second display unit included in the terminal device; a display control unit that performs control to display, in at least one of the information processing device and the terminal device, a selection screen that allows the user to select whether or not to permit execution of an event based on the operation, in accordance with the screen and the operation; and an event execution determination unit that determines whether or not to permit execution of an event based on the operation, based on a user's selection result on the selection screen; An information processing device comprising:

2. the screen is a multimedia playback screen, The information processing device according to claim 1 , wherein the display control unit performs control to display the selection screen on the information processing device and on the terminal device other than the terminal device on which the operation is performed.

3. the screen is a map screen, The information processing device according to claim 1 , wherein the display control unit controls the information processing device to display the selection screen, but does not control the terminal device to display the selection screen.

4. the screen is a display screen of a web page, The information processing device according to claim 1 , wherein the display control unit performs control to cause the terminal device on which the operation has been performed to display the selection screen, and does not perform control to cause other devices to display the selection screen.

5. The event based on the operation causes a screen transition to a link destination, The information processing device according to claim 4, wherein the display control unit, when execution of an event based on the operation is permitted, stops transmitting the screen to the terminal device that performed the operation, and controls the display of the linked screen using viewing software provided in the terminal device on which the operation was performed.

6. An information processing method performed by an information processing device including a first display unit, a screen generating unit that generates a screen to be displayed on the first display unit, and a transmission / reception control unit that transmits the screen to a terminal device via a network, Detecting a user operation performed on the screen displayed on a second display unit included in the terminal device; performing control to display, in at least one of the information processing device and the terminal device, a selection screen that allows the user to select whether or not to permit execution of an event based on the operation, in accordance with the screen and the operation; determining whether to permit execution of the event based on the operation based on a result of the user's selection on the selection screen; Information processing methods.

7. A program executed by a computer included in an information processing device including a first display unit, a screen generating unit that generates a screen to be displayed on the first display unit, and a transmission / reception control unit that transmits the screen to a terminal device via a network, a function of detecting a user's operation performed on the screen displayed on a second display unit included in the terminal device; a function of controlling at least one of the information processing device and the terminal device to display a selection screen that allows the user to select whether or not to permit execution of an event based on the operation, in accordance with the screen and the operation; and a function of determining whether or not to permit execution of the event based on the operation, based on a user's selection result on the selection screen; A program to achieve this.

Images