Information processing apparatus, information processing method, and storage medium

Abstract

An information processing apparatus, an information processing method, and a storage medium. The information processing apparatus (10) can maintain convenience of other apparatuses even when a request is made by one screen sharing target apparatus (e.g., a smartphone). A screen generation unit (115) generates a screen to be displayed on a first display (output unit 142). The transmission / reception control unit (114) transmits the screen to external terminals (20a, 20b) via a network. An operation detection unit (111) detects a user operation performed on a screen displayed on a second display unit provided by the external terminal. A display control unit (117) controls display of a selection screen to at least one device selected from the information processing device and the external terminal in response to the screen and the operation so as to allow the user to select whether to allow execution of an event based on the operation. An event execution determination unit (116) determines whether or not execution of an operation-based event is permitted in response to a selection result on the selection screen by a user.

Description

Technical Field

[0001] The present invention relates to an information processing device, an information processing method and a storage medium. Background Art

[0002] In a mirroring system that performs screen sharing between a plurality of devices, a screen of a screen sharing source device (eg, a car-mounted device) may be uniformly controlled based on a request from a single screen sharing target device (eg, a smartphone).

[0003] Patent document 1 discloses an information processing device provided with a wireless communication unit and a control unit. The wireless communication unit performs data communication with other information processing devices. In addition, it is assumed that new data is transmitted to and from a first information processing device that plays the role of a group owner of a network formed by at least the device and the first information processing device. In this case, the control unit controls the exchange of information to update the role of data transmission to each of the first information processing device and the second information processing device.

[0004] Patent document 2 discloses a 5G or quasi-5G communication system that is configured to support a higher data transmission rate after a 4G communication system (such as LTE). This is a method of using a service in which multiple electronic devices are linked to a first electronic device and includes the following steps: connecting to a central display device and causing the first electronic device to mirror the service to the central display device based on a stored playlist of the service. It also includes the following steps: receiving a playback list change request from a second electronic device and updating the playback list in response to the change request. It also includes the following steps: transmitting updated playlist related information to the central display device.

[0005] [Prior art literature]

[0006] Patent document 1: WO2016 / 163181A1 (equivalent to US20180069761 A1 and EP3282810A1)

[0007] Patent Document 2: WO2017 / 142336A1 Summary of the invention

[0008] [Problems to be Solved by the Invention]

[0009] However, based on the request, the display screen is controlled by one screen sharing target, which is inconvenient for devices other than the one screen sharing target.

[0010] An object of the present invention is to provide an information processing device, an information processing method, and a program capable of maintaining convenience for other devices even when a request is made by one screen sharing target device.

[0011] [Methods of solving the problem]

[0012] In order to solve the above-mentioned problems, an information processing apparatus of the present invention has the following configuration.

[0013] The screen generation unit generates a screen to be displayed on the first display unit. The transmission / reception control unit transmits the screen to the external terminal via the network. The operation detection unit detects a user operation performed on the screen displayed on the second display unit provided by the external terminal. The display control unit controls the display of a selection screen to at least one device selected from the information processing device and the external terminal in response to the screen and the operation, so as to allow the user to select whether to allow the execution of an event based on the operation. The event execution determination unit determines whether to permit the execution of an event based on the operation in response to a selection result by the user on the selection screen.

[0014] [Effects of the invention]

[0015] The present invention can provide an information processing device, an information processing method, and a program that can maintain convenience for other devices even when a request is made by one screen sharing target device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram depicting the configuration of a display control system suitable for use with the present invention.

[0017] Figure 2 Describes the multimedia playback screen displayed on the information processing device and external terminals.

[0018] Figure 3 Describes the screen displayed on each device when the user of the external terminal performs a pause operation.

[0019] Figure 4 is a sequence diagram depicting a process executed by each of the information processing apparatus and the external terminal according to the first embodiment.

[0020] Figure 5 Describes the map screen displayed on the information processing device and the external terminal.

[0021] Figure 6 Describes the screen displayed on each device when the user of the external terminal performs a map screen zoom-out operation.

[0022] Figure 7 is a sequence diagram depicting a process executed by each of the information processing apparatus and the external terminal according to the second embodiment.

[0023] Figure 8Describes web page screens displayed on information processing devices and external terminals.

[0024] Fig. 9 Describes a screen displayed on each device when a user of an external terminal performs an operation to display a link screen.

[0025] Fig.10 is a sequence diagram depicting a process executed by each of the information processing apparatus and the external terminal according to the third embodiment.

[0026] Fig.11 Describes notification information. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described below with reference to the accompanying schematic drawings.

[0028] <Overview of Image Control System 1>

[0029] Figure 1 : is a block diagram depicting the configuration of a display control system 1 applicable to the present invention.

[0030] The image control system 1 is installed on a vehicle as an example of a mobile body. The image control system 1 is equipped with an information processing unit 10 and external terminals 20a and 20b. The information processing unit 10 and the external terminals 20a and 20b are connected via a network 30.

[0031] The information processing unit 10 generates image information for a screen to be shared with the external terminals 20A and 20B and also displays the screen.

[0032] The information processing unit 10 is a computer device. More specifically, the information processing unit 10 is a car navigation device, a car audio device, a car control system, etc. As depicted in the figure, the information processing unit 10 is equipped with a processor 11 (such as a central processing unit (CPU)), a memory 12 as a storage device, and a permanent storage device 13 (such as a hard disk drive (HDD) or a solid state drive (SSD)). The processor 11 executes various software, such as an operating system (basic software) and an application program (application software), etc. The memory 12 is a storage area for storing the following: various software and data for executing the software, etc. The permanent storage 13 is a storage area for storing input data for various software and output data from various software.

[0033] Furthermore, the information processing apparatus 10 is also provided with a user interface (hereinafter referred to as "user IF") 14 for exchanging information with a user and a communication interface (hereinafter referred to as "communication IF") 15 for communicating with the outside.

[0034] The processor 11 performs processing to realize the functions of the operation detection unit 111, the event analysis unit 112, the application execution unit 113, the transmission / reception control unit 114, the screen generation unit 115, the event execution determination unit 116, the display control unit 117 and the notification information selection unit 118.

[0035] The operation detection unit 111 detects a user operation performed on screens displayed on the external terminals 20 a and 20 b .

[0036] The event analysis unit 112 analyzes the content of the event based on the operation of the user of the external terminals 20A and 20B. In other words, the event analysis unit 112 analyzes the content of the operation expected by the user operating the external terminals 20A and 20B. In addition, the event analysis unit 112 also determines whether the operation of the user operating the external terminals 20A and 20B is a predetermined operation.

[0037] The application program execution unit 113 executes an application program running on the information processing device 10. The following describes an application program to be executed.

[0038] The transmission / reception control unit 114 controls transmission and reception of information between the external terminals 20A and 20B via the communication IF 15. Specifically, the transmission / reception control unit 114 transmits the same screen as the screen displayed on the output unit 142 to the external terminals 20A and 20B via the network 30.

[0039] The screen generation unit 115 generates a screen to be displayed on the output unit 142 in response to the execution of the application program.

[0040] The event execution determination unit 116 determines whether to permit execution of an event based on a user operation in response to a selection result by the user on a selection screen to be described later. A specific determination method is described below.

[0041] The display control unit 117 controls the display of the screen generated by the screen generation unit 115 on the output unit 142. The display control unit 117 controls the display of the selection screen to at least one device selected from the information processing device 10 and the external terminals 20a and 20b in response to the screen displayed on the external unit 142 and the user's operation result on the screen, so as to allow the user to select whether to allow the execution of the event based on the operation result. The screen displayed on the output unit 142 is a screen displayed while the application is executed. In this case, the display control unit 117 selects the device to display the selection screen from the information processing device 10 and the external terminals 20a and 20b based on the screen displayed by the application and the content of the operation expected by the user, and controls the display of the selection screen on the selected device. The specific content is described below.

[0042] The notification information selection unit 118 selects notification information to be sent to the information processing device 10 and the external terminals 20a and 20b. The notification information is selected based on the operation results of each user on the screen currently displayed on the information processing device 10 and the external terminals 20a and 20b. Fig.11 To describe the details of the notification information.

[0043] The non-transitory storage device 13 stores notification information 131 to be selected by the non-transitory storage device 13. Here, the notification information 131 includes notification information on permission required, approval awaited, rejected, and required operation.

[0044] The user IF 14 has an input unit 141 and an output unit 142. The input unit 141 is used by the user to provide input to the information processing device 10. The output unit 142 is an example of a first display unit, and displays a screen and information to the user. In this embodiment, the input unit 141 and the output unit 142 are touch panels including these functions. However, this is not a limitation, and the input unit 141 can be a keyboard, a mouse, a touch pad, etc. The output unit 142 can be a single-function display device, for example, a liquid crystal display without touch panel functionality.

[0045] The external terminals 20a and 20b are examples of terminal devices that share a screen with the information processing device 10. Similar to the information processing device 10, the external terminals 20a and 20b are also computer devices and include a processor, a memory, etc. The external terminals 20a and 20b are equipped with a second display unit that can display the same screen as the screen transmitted from the information processing device 10 and displayed on the output unit 142. The second display unit is, for example, a touch panel. In addition, the external terminals 20a and 20b may have a Figure 1 The external terminals 20a and 20b are, for example, mobile terminal devices carried by users, specifically smartphones and tablet computers. Alternatively, a display device for a rear seat is possible. Although two external terminals 20A and 20B are shown here, any number of external terminals is possible.

[0046] The network 30 may use wireless communication including, for example, cellular phone lines, Wi-Fi (Wireless Fidelity, registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), and UWB (Ultra Wideband). The network 30 may use wired communication such as a wired LAN (Local Area Network) circuit, USB (Universal Serial Bus), an optical communication line, a public telephone line, etc. In addition, the network 30 may be a combination of a wireless communication line and a wired communication line, or may use the Internet, etc. in combination.

[0047] <Description of Operation of Information Processing Device 10>

[0048] Three embodiments of the operation of the information processing device 10 are described below.

[0049] [First embodiment]

[0050] In the first embodiment, the application execution unit 113 executes the video playback software that plays multimedia as an application. In this case, the screen generation unit 115 generates the multimedia playback screen. In addition, the display control unit 117 displays the multimedia playback screen on the output unit 142, and sends instructions to the external terminals 20a and 20b to display the multimedia playback screen transmitted from the aforementioned information processing unit 10.

[0051] Figure 2 Multimedia playback screens displayed on the information processing device 10 and the external terminals 20a and 20b are depicted.

[0052] Here, the screen of the information processing device 10 is displayed at the front seat of the vehicle. The screen of the external terminal 20a is displayed on the back seat (right), and the screen of the external terminal 20b is displayed on the back seat (left). In this case, the user of the information processing device 10 is, for example, the driver of the vehicle. In addition, the users of the external terminals 20a and 20b are, for example, people who watch the multimedia playback screen on their own smart phones in the back seat.

[0053] In this case, the multimedia image information generated by the information processing device 10 is transmitted to the external terminals 20a and 20b, and a common screen is displayed on the information processing device 10 and the external terminals 20a and 20b. In addition, as an operation related to the screen, the figure depicts a situation in which the user of the external terminal 20a touches the pause button Bt2 to pause.

[0054] The event analysis unit 112 determines whether the operation of the user of the external terminal 20a is a predefined operation, and if the operation is not a predefined operation, the event based on the operation is considered invalid. The process to be executed by triggering the screen operation is predefined in the application that realizes the screen sharing on the information processing device 10. In this case, the operation target to be analyzed can be limited.

[0055] exist Figure 2 In the example of , a previous track skip button Bt1, a pause button Bt2, and a next track skip button Bt3 are provided, and events based on operations other than corresponding operations for these buttons (previous track skip, pause, and next track skip) are disabled.

[0056] Figure 3The screen displayed on each device when the user of the external terminal 20a performs the pause operation is depicted. Note that the fact that the user of the external terminal 20a performs the pause operation is an analysis result obtained by the event analysis unit 112. In addition, based on the analysis result of the event analysis unit 112, the notification information selection unit 118 selects notification information to be provided to the information processing device 10 and the external terminals 20a and 20b.

[0057] In this case, the display control unit 117 controls the display of the dialog box A as a selection screen that allows the user to select whether to allow the execution of an event based on the operation on the information processing device 10 and the external terminal other than the external terminal in which the operation (in this case, pause) is performed. Specifically, the display control unit 117 controls the display of the dialog box A on the information processing device 10, sends a dialog box display request (notification information 131) to the external terminal 20b, and displays the dialog box A on the external terminal 20b. Here, the display of the dialog box A is based on the selection result made by the notification information selection unit 118. Figure 3 Depicts a situation in which a dialog box A containing a message "There is an operation request from the rear seat (right). Should the operation be allowed?" is displayed on the information processing device 10 and the external terminal 20b. The users of the information processing device 10 and the external terminal 20b touch the "Yes" button in the dialog box A to grant permission, or touch the "No" button to deny permission. The figure depicts the user of the information processing device 10 touching the "Yes" button and the user of the external terminal 20b touching the "No" button. The dialog box A is deleted after the user touch operation.

[0058] Furthermore, while the user of the information processing apparatus 10 or the external terminal 20b selects permission or non-permission of the operation, the external terminal 20a on which the operation is performed displays a dialog box B with a message "Operation request in progress".

[0059] Possible conditions under which the event execution determination unit 116 permits the execution of an operation-based event include, for example, majority permission by majority voting, permission for a specific seat (e.g., the driver's seat), and permission from an image publisher. It is preferable to set a deadline that allows the user to select whether to permit the occurrence of an operation-based event. When the deadline is exceeded (timeout), the information processing device 10 counts the replies obtained at that point.

[0060] The selection screen is displayed to the information processing device 10 and the external terminals 20a and 20b that will be affected by the execution of the event based on the operation. When the information processing device 10 shares the multimedia playback screen with the external terminals 20a and 20b, for example, if the user of the external terminal 20a performs the pause operation, the operation will affect everyone who watches the multimedia playback screen. Specifically, the screen will switch suddenly or a pop-up window will appear. Therefore, the selection screen is displayed to allow the device of the user other than the user who performs the operation to select whether to allow the execution of the event based on the operation (in this case, pause). This provides the authorization that does not allow the execution of the event based on the operation for the user other than the user who performs the operation, and also allows the user to anticipate in advance whether the screen will be switched or a pop-up window will appear after the execution of the event based on the operation is allowed. In this case, the display control unit 117 controls the information processing unit 10 and the external terminal 20b to display the dialog box A as a selection screen that allows the user to select whether to allow the execution of the event based on the operation based on the screen displayed by the video playback software and the operation performed by the user (here, pause).

[0061] When the information processing device 10 permits the execution of an operation-based event according to the selection result made by each user on the selection screen, the process is performed based on the permitted event. In this case, an event based on the pause operation performed by the user of the external terminal 20a is executed, and the multimedia playback screen is paused. In contrast, when the execution of an operation-based event is not allowed, an event based on the disallowed operation is not executed. In this case, an event based on the pause operation performed by the user of the external terminal 20a is rejected, and the multimedia playback screen on each device is not paused. The external terminal 20a displays a dialog box C containing a message "This operation has been rejected". The dialog box C is deleted after a predetermined amount of time has passed after the display.

[0062] Figure 4 is a sequence diagram depicting a process executed by each of the information processing apparatus 10 and the external terminals 20a, 20b according to the first embodiment.

[0063] The information processing unit 10 generates a multimedia playback screen as a screen to be transmitted to be shared with the external terminals 20 a and 20 b ( S101 ).

[0064] The information processing device 10 displays a multimedia playback screen on the information processing device 10 and transmits image information of the screen to the external terminals 20a and 20b (S102).

[0065] The external terminals 20a and 20b receive the image information transmitted from the information processing device 10, and display the same multimedia playback screen as that of the information processing device 10 at each of the external terminals 20a and 20b (S103, S104).

[0066] When the user of the external terminal 20 a performs a screen operation by touching the screen while the multimedia playback screen is displayed, an event based on the operation is transmitted to the information processing device 10 ( S105 ).

[0067] The information processing device 10 analyzes the content of the event based on the operation of the user of the external terminal 20a (S106).

[0068] Steps S105 and S106 are an operation phase for accepting screen operations from the user of the external terminal 20a.

[0069] If the operation is a predetermined operation, the information processing device 10 displays a dialog box A as a selection screen that allows the information processing device 10 to select whether to permit execution of an event based on the operation by the user of the external terminal 20a (S107).

[0070] At the same time, the information processing device 10 sends a dialog display request (notification information 131) to the external terminals 20a and 20b (S108). This can be achieved by the information processing device 10 sending a dialog display request (notification information 131) to the external terminals 20a and 20b. As a result, the external terminal 20a displays a dialog box B containing a message "Operation request in progress" (S109). In addition, the external terminal 20b displays a dialog box A similar to the dialog box of the information processing device 10 (S110).

[0071] The information processing device 10 accepts an operation on the dialog box A from the user of the information processing device 10 (S111). In addition, the information processing device 10 accepts an operation on the dialog box A from the user of the external terminal 20b (S112). In other words, the information processing device 10 accepts the result of each user's selection of "yes" or "no" in the selection in the dialog box A.

[0072] Steps S107 to S112 are an approval phase for approving screen operations from the user of the external terminal 20a.

[0073] The information processing device 10 counts the operation reception, and determines whether an event based on the operation from the user of the external terminal 20 a is executable ( S113 ).

[0074] The information processing apparatus 10 transmits a processing request corresponding to the determination result of S113 ( S114 ).

[0075] Therefore, the information processing device 10 performs processing corresponding to the determination result (S115). In this case, the process either reflects an event based on the operation or does not perform any operation.

[0076] The external terminal 20a also performs processing corresponding to the determination result based on the instruction from the information processing device 10 (S116). In this case, the process either deletes the dialog box B and displays a screen reflecting an event based on the operation on the information processing device 10, or deletes the dialog box B and displays a dialog box C with a message "The operation has been rejected". This is the result of a request for an operation performed by the user of the external terminal 20a. The dialog box C is deleted after a predetermined amount of time has passed after being displayed.

[0077] The external terminal 20a also performs processing corresponding to the determination result based on the instruction from the information processing device 10 (S117). In other words, the external terminal 20b shares the screen displayed on the information processing device 10 in S115. Specifically, the external terminal 20b displays a screen displayed on the information processing device 10 reflecting an event based on the operation of the user of the external terminal 20a or a screen displayed on the information processing device 10 before the operation of the user of the external terminal 20a.

[0078] S113 to S117 are in an execution phase in which screen operations from the user of the external terminal 20 a are performed.

[0079] [Second embodiment]

[0080] The second embodiment describes a case where the application program execution unit 113 executes navigation software as an application program. In this case, the screen generation unit 115 generates a map screen. In addition, the display control unit 117 displays the map screen on the output unit 142, and sends an instruction to the external terminals 20a and 20b to display the map screen transmitted from the aforementioned information processing unit 10.

[0081] Figure 5 Map screens displayed on the information processing device 10 and the external terminals 20a and 20b are depicted.

[0082] In this case, the image information of the map screen generated by the information processing device 10 is transmitted to the external terminals 20a and 20b, and a common map screen is displayed on the information processing device 10 and the external terminals 20a and 20b. In addition, as a screen-related operation performed by the user of the external terminal 20a, a case is shown in which the user touches the zoom-out button Bt5 to zoom out on the map screen.

[0083] exist Figure 5In the example of , a zoom-in button Bt4 and a zoom-out button Bt5 are provided, and events based on operations other than zoom-in and zoom-out as operations corresponding to these buttons are disabled.

[0084] Figure 6 Depicted is a screen displayed on each device when the user of the external terminal 20a performs a map screen zoom-out operation.

[0085] The display control unit 117 controls the display of a dialog box A as a selection screen that allows the information processing device 10 to select whether to allow the execution of an event based on an operation (in this case, zooming out on the map). On the other hand, the display control unit 117 does not control the display of the selection screen to the external terminals 20a and 20b, and the dialog box A is not displayed on the external terminals 20a and 20b. Figure 6 Depicted is a case where a dialog box A containing a message “There is an operation request from the rear seat (right). Should the operation be permitted?” is displayed on the information processing device 10 .

[0086] While the user of the information processing device 10 selects to allow or not allow the operation, the external terminal 20a displays a dialog box B with a message “Operation request in progress”. The external terminal 20a displays the dialog box B as a result of the dialog box display request (notification information 131) sent to the external terminal 20a by the display control unit 117 based on the selection result of the notification information selection unit 118. Note that the dialog box is not displayed on the external terminal 20b.

[0087] When the map screen is displayed as a shared screen, if, for example, an operation of reducing the map is performed by a user of the external terminal 20a, the event based on the operation will mainly affect the driver who is driving in the front seat. In other words, if an unexpected operation is performed on the map screen displayed by the information processing unit 10 while driving, driving may be affected. On the other hand, the impact on the user of the external terminal 20b is small. Therefore, the display control unit 117 controls the display of a selection screen that allows the information processing device 10 to select whether to allow the execution of an event based on the operation (in this case, map reduction). On the other hand, the display control unit 117 does not control the display of the selection screen on the external terminal 20b. In other words, the dialog box A is not displayed on the external terminal 20b. In this case, the display control unit 117 controls the information processing unit 10 and the external terminal to display a dialog box A as a selection screen that allows the user to select whether to allow the execution of an event based on the operation in response to the screen displayed by the navigation software and the operation performed by the user (here, map reduction).

[0088] When the information processing device 10 permits the execution of an operation-based event according to the result of the selection made by the user of the information processing device 10 on the selection screen, the process is executed based on the permitted event. In this case, an event based on a map zoom-out operation performed by the user of the external terminal 20a is executed, and the map is zoomed out. On the other hand, when an event based on an operation is not permitted, an event based on an operation that is not permitted is not executed. In this case, an event based on a map zoom-out operation performed by the user of the external terminal 20A is rejected, and the map screen is not zoomed out. In this case, the external terminal 20a displays a dialog box C containing a message "This operation has been rejected". The dialog box C is deleted after a predetermined amount of time has passed after being displayed.

[0089] Figure 7 is a sequence diagram depicting a process executed by each of the information processing apparatus 10 and the external terminals 20a, 20b according to the second embodiment.

[0090] Figure 7 S201 to S207 in are similar to Figure 4 However, the operation performed by the user of the external terminal 20a is changed from the pause operation to the reduction of the map screen.

[0091] exist Figure 7 After S208 in , the process is as follows.

[0092] The information processing device 10 sends a dialog display request (notification information 131) to the external terminal 20a (S208). As a result, the external terminal 20a displays a dialog box B containing a message "Operation request in progress" (S209). Figure 4 Unlike the case in , a dialog display request (notification information 131) is made only to the external terminal 20a, and the dialog is not displayed on the external terminal 20b.

[0093] The information processing device 10 accepts an operation on the dialog box A from the user of the information processing device 10 (S210). Figure 4 Unlike the case in , the dialog box is not displayed on the external terminal 20b, so the information processing device 10 does not accept the operation of the dialog box A from the user of the external terminal 20b.

[0094] Figure 7 S211 to S215 in are similar to Figure 4 S113 to S117 in.

[0095] In the second embodiment, steps S205 and S206 are an operation phase for accepting screen operations from the user of the external terminal 20a. Steps S207 to S210 are an approval phase for approving screen operations from the user of the external terminal 20a. S211 to S215 are an execution phase in which screen operations from the user of the external terminal 20a are executed.

[0096] [Third embodiment]

[0097] The third embodiment describes a case where the application program execution unit 113 executes browsing software such as a browser as an application program. In this case, the screen generation unit 115 generates a display screen of a web page. The display control unit 117 displays the web page display screen on the output unit 142, and sends an instruction to the external terminals 20a and 20b to display the web page display screen transmitted from the aforementioned information processing unit 10.

[0098] Figure 8 The web page screens displayed on the information processing device 10 and the external terminals 20 a and 20 b are depicted.

[0099] In this case, the image information of the web page display screen generated by the information processing device 10 is transmitted to the external terminals 20a and 20b, and a common web page display screen is displayed on the information processing device 10 and the external terminals 20a and 20b. As a screen-related operation, the figure depicts a case where the user of the external terminal 20a touches the link button Bt6 to switch to another web page. In this case, the event based on the aforementioned operation is a screen switch to a link target.

[0100] exist Figure 8 In the example in , a link button Bt6 is provided, and an event based on an operation other than transition to another web page as a corresponding operation is disabled.

[0101] Fig. 9 A screen displayed on each device when the user of the external terminal 20 a performs an operation of displaying a link screen is depicted.

[0102] The display control unit 117 controls the display of a dialog box D as a selection screen that allows the external terminal 20a on which the operation is performed to select whether to allow the execution of an event based on the operation. On the other hand, the display control unit 117 does not control the display of the selection screen to other devices. In other words, the dialog box is not displayed on any device other than the external terminal 20a on which the operation is performed. Fig. 9 Depicted is a case in which a dialog box D containing a message “Do you want to open this page in a browser?” is displayed to the external terminal 20 a .

[0103] When the display screen of the web page is displayed as a shared screen, if the operation of switching to another web page is performed, only the user of the external terminal 20a is affected by the event based on the operation. On the other hand, the conventional web page is displayed on the information processing device 10 and the external terminal 20b, so there is no effect on these users. Therefore, the display control unit 117 controls the display of the selection screen, which allows the external terminal 20a to select whether to allow the execution of the event based on the operation. On the other hand, the display control unit 117 does not control the display of the selection screen on the device other than the external terminal 20a. In other words, the dialog box D is not displayed on any other device other than the external terminal 20a. In this case, the display control unit 117 controls the information processing unit and the external terminal to display the dialog box D as a selection screen that allows the user to select whether to allow the execution of the event based on the operation based on the screen displayed by the browser software and the operation performed by the user (here, the screen is switched to the link target).

[0104] When the user of the external terminal 20a allows the event based on the operation, in other words, when the "Yes" button in the dialog box D is selected, the screen sharing between the external terminal 20a and the information processing device 10 is released, and the link screen is displayed by the viewing software provided by the external terminal 20a in which the operation is performed. In this case, the release of screen sharing means that the information processing device 10 stops transmitting (sending) the shared screen to the external terminal 20a. In this way, the shared screen is not controlled by the external terminal 20a that performs the screen operation, and the information processing equipment 10 and the external terminal 20b can remain in the screen before the user of the external terminal 20a performs the operation. The external terminal 20a can also be converted to other web pages according to the operation performed by the user. In other words, the convenience of each device is improved.

[0105] Fig.10 is a sequence diagram depicting a process executed by each of the information processing apparatus 10 and the external terminals 20a, 20b according to the third embodiment.

[0106] Fig.10 S301 to S306 in are similar to Figure 7 However, the operation performed by the user of the external terminal 20a is changed from the map screen reduction operation to the manipulation of the display of the link screen.

[0107] exist Fig.10 After S307 in , the process is as follows.

[0108] The information processing device 10 sends a dialog display request (notification information 131) to the external terminal 20a (S307). As a result, the external terminal 20a displays a dialog box D including a message "Do you want to open this page in your browser?" (S308).

[0109] The information processing device 10 accepts an operation of the dialog box D from the user of the external terminal 20a (S309). In other words, the information processing device 10 accepts a "yes" operation or a "no" operation to be selected in the dialog box D.

[0110] Fig.10 S310 to S311 in are similar to Figure 7 S211 to S212 in.

[0111] In subsequent S312 , the information processing device 10 executes processing corresponding to the determination result, but in this case, it does not perform any operation ( S312 ).

[0112] The external terminal 20a executes processing corresponding to the determination result (S313) based on the instruction from the information processing device 10. In this case, the process displays a link screen in the browser provided by the external terminal 20a or does not perform any operation.

[0113] The external terminal 20a executes a process corresponding to the determination result based on the instruction from the information processing device 10 (S314).

[0114] In the third embodiment, steps S305 and S306 are an operation phase for accepting screen operations from the user of the external terminal 20a. Steps S307 to S309 are an approval phase for approving screen operations from the user of the external terminal 20a. S310 to S314 are an execution phase in which screen operations from the user of the external terminal 20a are executed.

[0115] <Description of Notification Information 131>

[0116] Fig.11 The notification information 131 is depicted.

[0117] The notification information 131 is a processing request or notification content sent by the display control unit 117 to the information processing device 10 and the external terminals 20a and 20b based on the selection result of the notification information selection unit 118. Here, the notification information 131 indicated by IDs 1 to 5 is depicted. Among them, IDs 1 to 4 are Figure 1 The required permission, pending approval, rejection, and required operation are described in FIG. In addition, the notification information for IDs 1 to 4 indicates Figures 4 to 10 The information in the dialog boxes A to D depicted in . The notification content is not limited to this case, and for example, the specific content of the event based on the operation may be displayed in the dialog box, such as "There is an operation (pause) request from the rear seat (right). Do you want to allow the operation?"

[0118] The aforementioned embodiment can provide the information processing device 10 and the image control system 1 that can maintain convenience for other devices even when a request is made by one screen sharing target device. The result is a safe and easy-to-use screen sharing feature.

[0119] <Description of Information Processing Method>

[0120] The process executed by the information processing device 10 of the present embodiment is an information processing method executed by the information processing device 10, the information processing device including a first display unit, a screen generation unit 115 for generating a screen to be displayed on the first display unit as an output unit 142, and a transmission / reception control unit 114 for transmitting the screen to the external terminals 20a, 20b via the network 30, the information processing method comprising the following steps:

[0121] detecting a user operation performed on the screen displayed on the second display unit of the external terminal device 20a, 20b;

[0122] In response to the screen and the operation, controlling display of a selection screen to at least one of the information processing device 10 and the external terminals 20a, 20b, the selection screen allowing the user to select whether to allow execution of an event based on the operation; and

[0123] Based on the selection result of the user on the selection screen, it is determined whether to permit the execution of the event based on the operation.

[0124] <Description of the program>

[0125] The processing performed by the information processing equipment 10 in the present embodiment is prepared as a program, such as application software, for example. The process is implemented by software resources and hardware resources working together. That is, the processor 11 inside the computer in the information processing device 10 executes the program that implements each of the aforementioned functions and enables each of these functions to be implemented.

[0126] In the present embodiment, the processing performed by the information processing device 10 is a program executed by a computer of the information processing device 10, which is equipped with a first display unit, a screen generation unit 115 that generates a screen to be displayed on the output unit 142, the first display unit, and a transmission / reception control unit 114 that transmits the screen to the external terminals 20a and 20b via the network 30. The program enables the information processing device 10 and the external terminals 20a and 20b to implement: a function for detecting a user operation performed on a screen displayed on a second display provided by the external terminals 20a and 20b; a function for controlling the display of a selection screen in response to the screen and the operation, the selection screen allowing the user to select whether to allow the execution of an event based on the operation; and a function for determining whether to allow the execution of an event based on the operation based on the result of the user's selection on the selection screen.

[0127] The program for realizing the present embodiment may be provided using communications or by storing the program in a recording medium such as a CD-ROM.

[0128] The technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear from the claims that various changes or improvements to the above embodiments are also included in the technical scope of the present invention.

[0129] For example, in the above-mentioned embodiment, the information processing device 10 is the screen sharing source, but in the implementation process, the roles of the screen sharing source and the screen sharing target are interchangeable. In other words, at a certain point in time, the external terminal 20a or 20b becomes the screen sharing source, and the information processing device 10 becomes the screen sharing target.

[0130] For example, the operation phase, approval phase, and execution phase are provided for the purpose of explaining the content of the present invention in an easy-to-understand manner, but the process may not be clearly divided into these three phases. The names are not limited to any specific names. For example, the approval phase may be called the inquiry phase.

[0131] Description of Reference Numerals

[0132] 1...image control system, 10...information processing device, 11...processor, 12...memory, 20a, 20b...external terminal, 111...operation detection unit, 112...event analyzer, 113...application executor, 114...transmission / reception control unit, 115...screen generation unit, 116...event execution determination unit, 117...display control unit, 118...notification information selection unit, 142...output unit, A to D...dialog box.

Claims

1. An information processing device for sharing a screen with an external terminal device, the information processing device comprising: a first display unit; a screen generating unit, the screen generating unit generating a screen to be displayed on the first display unit; A transmission / reception control unit, which transmits the picture to the terminal device via a network; an operation detection unit that detects a user operation performed on the screen displayed on a second display unit provided by the terminal device; a display control unit that controls display of a selection screen to at least one of the information processing device and the terminal device in response to the screen and the operation, the selection screen allowing a user to select whether to allow execution of an event based on the operation; as well as An event execution determination unit determines whether to permit execution of an event based on the operation according to a selection result of the user on the selection screen.

2. The information processing device according to claim 1, wherein The picture is a multimedia playback picture; and The display control unit controls display of the selection screen to the information processing device and a terminal device other than the terminal device on which the operation is performed.

3. The information processing device according to claim 1, wherein The screen is a map screen; and The display control unit controls display of the selection screen to the information processing device, and does not control display of the selection screen to a terminal device.

4. The information processing device according to claim 1, wherein The screen is a display screen of a web page; and The display control unit controls display of the selection screen to the terminal device that performs the operation, and does not control display of the selection screen to other devices.

5. The information processing device according to claim 4, wherein The event based on the aforementioned operation is a screen transition to a link target; and When the execution of the event based on the operation is permitted, the display control unit stops transmission of the screen to and from the terminal device executing the operation, and controls display of a link screen by viewing software provided by the terminal device at which the operation is executed.

6. An information processing method performed by an information processing device, the information processing device comprising a first display unit, a screen generation unit for generating a screen to be displayed on the first display unit, and a transmission / reception control unit for transmitting the screen to a terminal device via a network, the information processing method comprising the following steps: detecting a user operation performed on the screen displayed on the second display unit of the terminal device; In response to the screen and the operation, controlling display of a selection screen to at least one of the information processing device and the terminal device, the selection screen allowing a user to select whether to allow execution of an event based on the operation; as well as Whether to permit execution of an event based on the operation is determined based on a selection result of the user on the selection screen.

7. A non-transitory storage medium storing a program, wherein the program enables a computer to implement: generating a screen to be displayed on a first display unit of an information processing device; Transmitting the image to an external terminal device via a network; detecting a user operation performed on the screen displayed on the second display unit of the terminal device; In response to the screen and the operation, controlling display of a selection screen to at least one of the information processing device and the terminal device, the selection screen allowing a user to select whether to allow execution of an event based on the operation; as well as Whether to permit execution of an event based on the operation is determined based on a selection result of the user on the selection screen.

Images