Information processing methods

Abstract

Depending on the model of the information processing device and the type of operating system running on it, problems may occur when outputting image data to external devices such as projectors. [Solution] The information processing method includes: establishing a communication connection with a projection device; causing an application program to transmit control signals to the projection device via a signal communication path to control the projection device; and causing a web browser extension program that functions on a web browser to acquire output data including at least one of video data and audio data.

Description

Technical Field

[0001] This disclosure relates to an information processing method.

Background Art

[0002] A system including a display device and a control device is known. The system described in Patent Document 1 includes a projector, which is an example of a display device, and an information processing device, which is an example of a control device. The projector and the information processing device are communicatively connected via a network. The projector receives image data and the like from the information processing device and projects the image data. The information processing device operates on an operating system such as Windows, Android, or iOS. Windows, Android, and iOS are registered trademarks, respectively.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Problems may occur when outputting image data to an external device such as a projector depending on the model of the information processing device or the type of operating system operating on the information processing device.

Means for Solving the Problems

[0005] The information processing method of this disclosure includes communicatively connecting with a projection device, causing an application program to transmit a control signal for controlling the projection device to the projection device via a signal communication path, and causing a web browser extension program that functions on a web browser to acquire output data including at least one of video data and audio data.

Brief Description of the Drawings

[0006] [Figure 1] A diagram showing the schematic configuration of the projection system. [Figure 2] A diagram showing the block configuration of the projection system. [Figure 3] A diagram showing the general configuration of the projection unit. [Figure 4] A diagram showing the system configuration of the projection system. [Figure 5] A diagram showing an example of the display screen. [Figure 6] A diagram showing the control flow executed by the control unit. [Figure 7] A diagram showing the control flow executed by the control unit. [Figure 8] An example of an installation instruction image is shown in the diagram. [Figure 9] A diagram showing an example of an image to be acquired. [Figure 10] A diagram showing the system configuration of the projection system. [Figure 11] A diagram showing the control flow executed by the control unit. [Modes for carrying out the invention]

[0007] Figure 1 shows a schematic configuration of projection system 1. Projection system 1 comprises a control device 10, a projector 100, a router 200, an application provision server 300, and a web server 400. Projector 100 projects a projection image PG onto the projection surface SC. The application provision server 300 and the web server 400 communicate with the control device 10 via a communication network NW. The projection system 1 shown in Figure 1 comprises one control device 10 and one projector 100, but is not limited to this. Projection system 1 may comprise multiple control devices 10 and multiple projectors 100.

[0008] The control device 10 is connected to the projector 100 in a communicative manner. The control device 10 transmits projected image data to the projector 100. The control device 10 performs mirroring, which causes the display screen DS shown on the control device 10's display 15 to be projected onto the projector 100. The control device 10 may also have a function to adjust the image shape, etc., of the projected image PG that the projector 100 projects onto the projection surface SC. The control device 10 can be a tablet terminal, a mobile computer, a desktop PC, etc. Figure 1 shows a mobile computer equipped with a display 15 as the control device 10. The control device 10 corresponds to an example of an information processing device.

[0009] The control device 10 executes various programs, such as a web browser 80, on the dedicated operating system 70. The dedicated operating system 70 is, for example, ChromeOS. Chrome is a registered trademark. The dedicated operating system 70 can create a virtual environment. In the virtual environment, programs that can run on an operating system different from the dedicated operating system 70 can run on the dedicated operating system 70. An example of an operating system different from the dedicated operating system 70 is AndroidOS. An example of a program that can run on an operating system different from the dedicated operating system 70 is an Android application.

[0010] The display 15 displays a screen containing various videos and images. The display 15 is composed of a liquid crystal panel, an organic EL (electro-luminescence) panel, etc. The display 15 may also have a touch input function that accepts input operations from the user.

[0011] The projector 100 is positioned opposite the projection surface SC. The projector 100 is communicatively connected to the control device 10. The projector 100 corresponds to an example of a projection device. The projector 100 may also be communicatively connected to an external device different from the control device 10. The projector 100 receives projection image data from the control device 10. The projector 100 projects the projection image PG onto the projection surface SC based on the projection image data. The projector 100 may perform mirroring of the display screen DS displayed on the display 15 of the control device 10 based on the projection image data. The projector 100 may project the projection image PG onto the projection surface SC based on pre-stored OSD data. OSD stands for On-Screen Display.

[0012] The projection surface SC displays the projected image PG projected from the projector 100. The projection surface SC shown in Figure 1 is composed of a screen, but is not limited to this. The projection surface SC may be a wall or ceiling in a room, or an exterior wall of a building, etc. The surface shape of the projection surface SC is not limited to a flat surface, but may be a three-dimensional shape such as a curved surface, a surface with irregularities, or a sphere.

[0013] Router 200 is a communication device that interconnects the control unit 10, projector 100, application server 300, and web server 400. Router 200 communicates with the control unit 10 and the application server 300 via the communication network NW. Router 200 communicates with the control unit 10 and the web server 400 via the communication network NW. Router 200 communicates with the control unit 10 and the projector 100 using a predetermined communication protocol. Router 200 may also communicate with the projector 100 and the web server 400. The control unit 10 and the projector 100 may communicate directly without going through Router 200.

[0014] The communication network NW is a line network that communicatively connects the application providing server 300, the web server 400, and the control device 10. As an example, the communication network NW is a WAN (Wide Area Network).

[0015] The application providing server 300 provides various applications to the control device 10, function extension applications added to the web browser 80, and the like. As an example, the application providing server 300 is managed by a system provider that provides a dedicated operating system 70. The application providing server 300 may be composed of a cloud server.

[0016] The web server 400 provides the display of HTML and objects to the web browser 80 of the client software. The web server 400 may provide a web application to the web browser 80 of the control device 10 based on an instruction from the control device 10. As an example, the web server 400 is managed by an application provider that provides a projection application 91 described later.

[0017] FIG. 2 shows the block configuration of the projection system 1. FIG. 2 shows the block configurations of the control device 10, the projector 100, the application providing server 300, and the web server 400 included in the projection system 1. FIG. 2 shows the projection system 1 with the router 200 omitted.

[0018] The control device 10 includes a display 15, an output unit 20, a control unit 30, a storage unit 40, and a communication unit 50. The display 15 is not shown in FIG. 2. The control device 10 may include an input unit and the like not shown.

[0019] The output unit 20 outputs various generated data generated by the control unit 30 to the display 15 and the like. The output unit 20 is composed of various circuits, output terminals, and the like. The output unit 20 corresponds to an example of an output unit. The output unit 20 has a display data output unit 21 and an audio output unit 23.

[0020] The display data output unit 21 outputs video data for displaying video to the display 15. The video data includes image data for displaying still images. The video data is included in the generated data. The display data output unit 21 may also output video data to a display device attached externally to the control device 10.

[0021] The audio output unit 23 outputs audio data related to sound. The audio data is included in the generated data. The audio output unit 23 is connected to a speaker or the like (not shown). The speaker or the like may be built into the control device 10 or attached externally to the control device 10. The audio output unit 23 outputs audio data via wired or wireless connection.

[0022] The control unit 30 is a device control controller that controls each part of the control device 10. The control unit 30 is, for example, a processor having a CPU (Central Processing Unit). The control unit 30 is composed of one or more processors. A dedicated operating system 70 runs on the control unit 30. The control unit 30 executes various programs on the dedicated operating system 70. By executing the device control program, the control unit 30 functions as various functional units. For example, the control unit 30 functions as a browser control unit 31, an application control unit 33, and an acquisition control unit 35. The control unit 30 may also function as functional units other than the browser control unit 31, the application control unit 33, and the acquisition control unit 35.

[0023] The browser control unit 31 operates the web browser 80 on the dedicated operating system 70 and displays web pages, etc. The browser control unit 31 acquires extension applications and extends the functionality of the web browser 80 by running the extension applications. The browser control unit 31 may also run web applications on the web browser 80 and execute the functions of the web applications. As an example, the browser control unit 31 operates the capture function extension application 81, which will be described later, and captures at least one of the video data and audio data.

[0024] The application control unit 33 runs applications on the dedicated operating system 70 or in a virtual environment. The application control unit 33 runs applications with specifications compatible with the dedicated operating system 70 on the dedicated operating system 70. The application control unit 33 also runs applications with specifications different from those compatible with the dedicated operating system 70 in a virtual environment. The virtual environment consists of a virtual device 90 that runs on the dedicated operating system 70. The virtual device 90 will be described later.

[0025] The acquisition control unit 35 executes various operations when acquiring extension applications, web applications, etc. The acquisition control unit 35 displays a message on the display 15 prompting the acquisition of extension applications, etc. The acquisition control unit 35 may also generate notification voice data to output the message, etc., as voice. Based on various instructions from the user of the control device 10, the acquisition control unit 35 sends an acquisition request to the application provision server 300, etc., requesting the acquisition of extension applications, etc. The acquisition control unit 35 installs the acquired extension applications, etc., into the control unit 30 and executes various functions.

[0026] The storage unit 40 stores various programs and various data. The storage unit 40 stores device control programs, various function extension applications, various web applications, and various other applications. The storage unit 40 stores various data generated by the control unit 30. The storage unit 40 is composed of volatile semiconductor memory such as RAM (Random Access Memory), non-volatile memory such as ROM (Read Only Memory) and flash memory. The storage unit 40 may also function as a work area for the control unit 30. The storage unit 40 corresponds to an example of a storage unit.

[0027] The communication unit 50 is a communication interface circuit that communicates with the projector 100, the application server 300, etc. The communication unit 50 connects wirelessly to the projector 100, etc., according to a predetermined communication protocol. The communication unit 50 includes a wireless communication port. The wireless communication port is a wireless LAN (Local Area Network) communication port, etc. The communication unit 50 may also be connected to the projector 100, etc., by wire. The communication unit 50 has a wired connector such as USB (Universal Serial Bus). The communication unit 50 transmits projected image data, etc., to the projector 100, etc. The projected image data includes video data, audio data, etc. The communication unit 50 receives various data from the application server 300, etc.

[0028] The projector 100 comprises a projection unit 110, a projector-integrated speaker 120, a projector control unit 130, a projector memory unit 140, and a projector communication unit 150.

[0029] The projection unit 110 projects image light based on the projected image data onto the projection surface SC. The projection unit 110 projects the projected image PG onto the projection surface SC based on the control of the projector control unit 130. The projection unit 110 projects video, still images, OSD images, etc., as the projected image PG based on the video data contained in the projected image data. The OSD image is an image that shows various settings and operation information related to the projector 100.

[0030] Figure 3 shows a schematic configuration of the projection unit 110. Figure 3 shows an example of the projection unit 110. The projection unit 110 comprises a light source 111, three liquid crystal light bulbs 113, a light bulb drive unit 115, and a projection lens 117.

[0031] The light source 111 emits light into the liquid crystal light bulb 113. The light source 111 comprises a light source unit 111a, a reflector 111b, an integrator optical system (not shown), and a color separation optical system (not shown). The light source unit 111a emits light. The light source unit 111a is composed of a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, etc. The light source unit 111a emits light based on the control of the projector control unit 130. The reflector 111b reduces variations in the emission direction of the light emitted by the light source unit 111a. The integrator optical system reduces variations in the brightness distribution of the light emitted from the light source unit 111a. The light that has passed through the reflector 111b is incident on the color separation optical system. The color separation optical system separates the incident light into red, green, and blue color light components.

[0032] The liquid crystal light bulb 113 modulates the light emitted from the light source 111. By modulating the light, the liquid crystal light bulb 113 generates a projected image PG, etc. The liquid crystal light bulb 113 is composed of a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal light bulb 113 has a rectangular pixel area 113a that includes a plurality of pixels PL arranged in a matrix. In the liquid crystal light bulb 113, a driving voltage is applied to each pixel PL of the liquid crystal. The projector 100 shown in Figure 3 has three liquid crystal light bulbs 113. The projector 100 is configured to use a transmissive liquid crystal panel equipped with liquid crystal light bulbs 113, but is not limited to this. The projector 100 may also be equipped with a reflective liquid crystal panel or an LCOS (Liquid Crystal On Silicon) liquid crystal panel. The projector 100 may also be configured to be equipped with one or more DMDs (Digital Mirror Devices).

[0033] The three liquid crystal light bulbs 113 are a red light liquid crystal light bulb 113R, a green light liquid crystal light bulb 113G, and a blue light liquid crystal light bulb 113B. The red color light component separated by the color separation optical system is incident on the red light liquid crystal light bulb 113R. The green color light component separated by the color separation optical system is incident on the green light liquid crystal light bulb 113G. The blue color light component separated by the color separation optical system is incident on the blue light liquid crystal light bulb 113B.

[0034] The light bulb drive unit 115 applies a drive voltage to each pixel PL based on the projected image data received from the projector control unit 130. The light bulb drive unit 115 is, for example, a control circuit. The drive voltage is supplied by a drive source (not shown). The light bulb drive unit 115 may also apply the drive voltage to each pixel PL based on the projected image data corrected by the projection control unit 131, which will be described later. When the light bulb drive unit 115 applies the drive voltage to each pixel PL, each pixel PL is set to a light transmittance based on the projected image data. The light emitted from the light source 111 is modulated by passing through the pixel region 113a. The three liquid crystal light bulbs 113 form a color component image for each color of light.

[0035] The projection lens 117 synthesizes the color component images formed by the liquid crystal light bulb 113 and projects them at an enlarged size. The projection lens 117 projects the projected image PG onto the projection surface SC. The projected image PG is a multi-color image formed by synthesizing the individual color component images.

[0036] The projector-integrated speaker 120 shown in Figure 2 outputs sound based on audio data contained in the projected image data. The projector-integrated speaker 120 receives audio data from the projector control unit 130 and outputs sound corresponding to the audio data. The projector 100 shown in Figure 2 is equipped with a projector-integrated speaker 120, but is not limited thereto. The projector 100 may also be equipped with an external speaker. The external speaker outputs sound based on audio data.

[0037] The projector control unit 130 is a projector control controller that controls each unit of the projector 100. The projector control unit 130 is, for example, a processor having a CPU. The projector control unit 130 consists of one or more processors. By executing firmware, the projector control unit 130 functions as a projection control unit 131 and a communication control unit 133. The projector control unit 130 may also function as a functional unit other than the projection control unit 131 and the communication control unit 133.

[0038] The projection control unit 131 controls projection by the projection unit 110. Based on the video data contained in the projection image data transmitted from the control device 10, the projection control unit 131 causes the projection unit 110 to project the projection image PG. Based on the audio data contained in the projection image data, the projection control unit 131 causes the projector's built-in speaker 120 to output audio. The projection control unit 131 synchronizes the projection image PG projected from the projection unit 110 with the audio output from the projector's built-in speaker 120. The projection control unit 131 performs various correction processes on the video data, such as keystone correction.

[0039] The communication control unit 133 controls communication with the control device 10. The communication control unit 133 causes the projector communication unit 150 to perform various communications. The communication control unit 133 controls communication according to the function of the projector communication unit 150.

[0040] The projector memory unit 140 stores firmware and various data. The projector memory unit 140 also stores projection image data transmitted from the control device 10, correction data for correcting the video data contained in the projection image data, and so on. The projector memory unit 140 is composed of volatile semiconductor memory such as RAM, and non-volatile memory such as ROM and flash memory.

[0041] The projector communication unit 150 is a communication interface circuit that communicates with the control device 10, external devices, etc. The projector communication unit 150 connects wirelessly to the control device 10, etc., according to a predetermined communication protocol. The projector communication unit 150 may or may not have a function unit that communicates using WebRTC (Web Real-Time Communication) technology. The projector communication unit 150 includes a wireless communication port. The wireless communication port is a wireless LAN communication port, etc. The projector communication unit 150 may also be connected to the control device 10, etc., by wire. The projector communication unit 150 has a wired connector such as USB. The projector communication unit 150 receives projected image data from the control device 10. The projector communication unit 150 transmits various data such as projector setting information to the control device 10, etc.

[0042] The application provision server 300 stores multiple extension applications, multiple applications for dedicated operating systems, etc. The multiple extension applications include the capture function extension application 81. The capture function extension application 81 functions on the web browser 80. The capture function extension application 81 captures the generated data output from the output unit 20 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The capture function extension application 81 causes the web browser 80 to execute a function to acquire the generated data. The capture function extension application 81 corresponds to an example of a web browser extension program.

[0043] The web server 400 stores multiple web applications. These multiple web applications include the capture application 83. The capture application 83 runs on the web browser 80. The capture application 83 captures the generated data output from the output unit 20 as captured data. The capture application 83 has similar functionality to the capture function extension application 81. As an example, the capture application 83 transmits the generated data acquired through WebSocket communication processing. The capture application 83 corresponds to an example of a web application program.

[0044] First Embodiment The first embodiment shows a first projection system 1a, which is an example of projection system 1. The first projection system 1a includes a first projector 100a having a stream communication unit 151 that communicates using WebRTC technology. The first projector 100a is an example of projector 100.

[0045] Figure 4 shows the system configuration of projection system 1. Figure 4 schematically shows the system configuration of the first projection system 1a, which is an example of projection system 1. Figure 4 shows the control device 10 and the first projector 100a.

[0046] The control device 10 operates a dedicated operating system 70. In the first embodiment, the dedicated operating system 70 is ChromeOS. The control unit 30 of the control device 10 operates the dedicated operating system 70. Applications corresponding to the dedicated operating system 70 can run on the dedicated operating system 70. On the dedicated operating system 70 shown in Figure 4, a web browser 80 and a virtual device 90 run. Applications other than the web browser 80 and virtual device 90 may also run on the dedicated operating system 70.

[0047] The web browser 80 is software that displays various web pages. The web browser 80 connects to a web server 400, etc. The web browser 80 runs on a dedicated operating system 70. One example of the web browser 80 is Chrome. The web browser 80 launches various extension applications and various web applications. The web browser 80 shown in Figure 4 can run the capture function extension application 81. The web browser 80 may also run the capture application 83.

[0048] The capture function extension application 81 is launched on the web browser 80. The control device 10 obtains the capture function extension application 81 from the application provision server 300 and stores it in the storage unit 40. The web browser 80 operates the capture function extension application 81 based on operation requests from other applications. The capture function extension application 81 captures the generated data output from the output unit 20 of the control device 10 as capture data. The capture function extension application 81 transmits the capture data as shared data to the first projector 100a via the stream distribution path SP. The shared data includes at least one of video data and audio data. The shared data corresponds to an example of output data. The capture function extension application 81 may also transmit the capture data to the projection application 91.

[0049] The capture application 83 is launched on the web browser 80. The control device 10 obtains the capture application 83 from the web server 400 and stores it in the storage unit 40. The web browser 80 operates the capture application 83 based on a web application operation request from another application. The capture application 83 captures the generated data output from the output unit 20 of the control device 10 as capture data. The capture application 83 sends the capture data as shared data to the first projector 100a via the stream distribution path SP. The capture application 83 may also send the capture data to the projection application 91.

[0050] The virtual device 90 is a virtual device that runs on the dedicated operating system 70. The virtual device 90 is an example of an emulator that runs on the dedicated operating system 70. The virtual device 90 functions as a device that runs an operating system different from the dedicated operating system 70 on the dedicated operating system 70. The virtual device 90 provides a virtual environment on the dedicated operating system 70. The virtual device 90 is, as an example, a device on which the Android OS runs. Applications that run on an operating system different from the dedicated operating system 70 can be launched on the virtual device 90. Applications that run on an operating system different from the dedicated operating system 70 are Android apps. The projection app 91 shown in Figure 4 is an example of an Android app.

[0051] The projection application 91 is software that controls the projector 100, including the first projector 100a. The projection application 91 is stored in the storage unit 40 in advance. The projection application 91 shown in Figure 4 causes the first projector 100a to project various projection images PG. The projection application 91 sends various control signals to the first projector 100a. The projection application 91 runs the capture function extension application 81 on the web browser 80. The projection application 91 may also run the capture application 83 on the web browser 80. The projection application 91 runs the capture function extension application 81, etc., and causes the web browser 80 to acquire shared data. The projection application 91 sends the shared data to the first projector 100a. The projection application 91 receives various response signals, etc., from the first projector 100a. The projection application 91 corresponds to an example of an application program.

[0052] The projection application 91, running on the virtual device 90, sends and receives various control signals, etc., to and from the dedicated operating system 70 via various APIs (Application Programming Interfaces). The APIs vary depending on the version of the operating system running on the virtual device 90. Depending on the version of the operating system running on the virtual device 90, the projection application 91 may have difficulty controlling the first projector 100a. The projection application 91 uses the functions of the web browser 80 to obtain shared data.

[0053] Figure 5 shows an example of the display screen DS. The display screen DS is displayed on the display 15. The display screen DS shown in Figure 5 is the display screen DS when the control device 10 starts the projection application 91. The display screen DS includes the UI screen 170 of the projection application 91. UI stands for User Interface.

[0054] The UI screen 170 displays multiple icons, etc. The UI screen 170 includes a projector name display section 171, a mirroring request icon 172, a photo display request icon 173, a document display request icon 174, a camera image display request icon 175, a remote control function request icon 176, a multi-projection request icon 177, a disconnection request icon 178, and an end request icon 179.

[0055] The projector name display unit 171 displays the projector name of the first projector 100a connected to the control device 10. The projector name display unit 171 displays the projector name included in the model information of the first projector 100a obtained from the first projector 100a.

[0056] The mirroring reception icon 172 receives a mirroring instruction from the user. When the user performs a predetermined operation on the mirroring reception icon 172, the mirroring reception icon 172 receives the mirroring instruction. When the projection application 91 receives the mirroring instruction, it generates a mirroring control signal corresponding to the mirroring instruction. The mirroring reception icon 172 includes a switch icon 172a.

[0057] The switch icon 172a indicates whether the first projector 100a is performing mirroring. The user can determine whether the first projector 100a is performing mirroring by checking the display of the switch icon 172a. The switch icon 172a may also accept a mirroring command from the user.

[0058] The photo display reception icon 173 receives a photo display instruction from the user. When the user performs a predetermined operation on the photo display reception icon 173, the photo display reception icon 173 receives the photo display instruction. When the projection application 91 receives the photo display instruction, it performs a photo display process that causes the control device 10 to project the photo image stored on the first projector 100a. The projection application 91 sends the photo image data to be displayed to the first projector 100a, causing the first projector 100a to project the photo image as a projected image PG.

[0059] The document display reception icon 174 receives a document display instruction from the user. When the user performs a predetermined operation on the document display reception icon 174, the document display reception icon 174 receives the document display instruction. When the projection application 91 receives the document display instruction, it performs document display processing to project a PDF file or the like stored in the control device 10 onto the first projector 100a. For example, the projection application 91 sends PDF data that displays a PDF image to the first projector 100a, causing the first projector 100a to project the PDF image as a projected image PG.

[0060] The camera image display reception icon 175 receives a camera image display instruction from the user. If the control device 10 is equipped with an imaging device such as a camera, the camera image display reception icon 175 becomes operable. When the user performs a predetermined operation on the camera image display reception icon 175, the camera image display reception icon 175 receives the camera image display instruction. When the projection application 91 receives the camera image display instruction, it performs camera image display processing to project the image captured by the imaging device onto the first projector 100a. The projection application 91 sends the imaging data to display the captured image to the first projector 100a, causing the first projector 100a to project the captured image as a projected image PG.

[0061] The remote control function reception icon 176 receives remote control function operation instructions from the user. When the user performs a predetermined operation on the remote control function reception icon 176, the remote control function reception icon 176 receives the remote control function operation instruction. When the projection application 91 receives the remote control function operation instruction, it displays a remote control image on the projection image PG that shows the remote control for operating the first projector 100a. The user can perform an input operation on the remote control image to control the first projector 100a.

[0062] The multi-projection reception icon 177 receives a multi-projection instruction from the user. The multi-projection reception icon 177 becomes operable when multiple projectors 100, including the first projector 100a, are connected to the control device 10. When the user performs a predetermined operation on the multi-projection reception icon 177, the multi-projection reception icon 177 receives the multi-projection instruction. The projection application 91 then performs multi-projection, causing the multiple projectors 100 connected to the control device 10 to project the projected image PG onto the projection surface SC.

[0063] The disconnection notification icon 178 receives a disconnection instruction from the user. When the user performs a predetermined operation on the disconnection notification icon 178, the disconnection notification icon 178 receives the disconnection instruction. When the projection application 91 receives the disconnection instruction, it disconnects the communication connection between the control device 10 and the first projector 100a.

[0064] The termination notification icon 179 accepts termination instructions from the user. When the user performs a predetermined operation on the termination notification icon 179, the termination notification icon 179 accepts the termination instruction. Upon receiving the termination instruction, the projection application 91 stops operating. The projection application 91 hides the UI screen 170.

[0065] The stream distribution path SP shown in Figure 4 is a path for real-time communication between the web browser 80 and the first projector 100a. The stream distribution path SP communicates using WebRTC technology as an example. When the first projector 100a has a projector communication unit 150 that supports WebRTC technology, the web browser 80 can transmit shared data to the first projector 100a via the stream distribution path SP. By transmitting shared data to the first projector 100a via the stream distribution path SP, the control device 10 can transmit the shared data to the first projector 100a without degrading its quality. The stream distribution path SP corresponds to an example of a browser communication path.

[0066] The TCP / IP communication path TP is a path that communicates using TCP / IP (Transmission Control Protocol / Internet Protocol). The TCP / IP communication path TP and the stream distribution path SP are different communication paths. The projection application 91 sends various control signals to the first projector 100a via the TCP / IP communication path TP. The projection application 91 receives response signals, etc., from the first projector 100a via the TCP / IP communication path TP. When the first projector 100a does not have a projector communication unit 150 that supports WebRTC technology, the projection application 91 sends shared data to the first projector 100a via the TCP / IP communication path TP. When shared data is sent to the first projector 100a via the TCP / IP communication path TP, the quality of the shared data may be degraded depending on the status of the virtual device 90. It is preferable that the shared data is sent to the first projector 100a via the stream distribution path SP. The TCP / IP communication path TP corresponds to an example of a signal communication path.

[0067] The projector communication unit 150 of the first projector 100a has a stream communication unit 151 and a signal communication unit 153. The projector communication unit 150 shown in Figure 4 is compatible with WebRTC technology.

[0068] The stream communication unit 151 is a communication circuit compatible with WebRTC technology. The stream communication unit 151 communicates with the capture function extension application 81 via the stream distribution path SP. The stream communication unit 151 receives shared data transferred from the capture function extension application 81 in real time using WebRTC technology.

[0069] The signal communication unit 153 communicates with the projection application 91 via the TCP / IP communication path TP. The signal communication unit 153 receives control signals from the projection application 91. The signal communication unit 153 transmits response signals and the like generated by the first projector 100a to the projection application 91. By the signal communication unit 153 sending and receiving control signals and the like between the projection application 91 and the first projector 100a, the projection application 91 can control the first projector 100a.

[0070] Figure 6 shows the control flow performed in projection system 1. Figure 6 shows the control flow performed in first projection system 1a, which is an example of projection system 1. Figure 6 shows the control flow performed in control device 10 and first projector 100a. The control flow performed in control device 10 corresponds to an example of an information processing method. By executing the control flow, control device 10 performs mirroring, which projects the display screen DS displayed on display 15 onto the first projector 100a.

[0071] In step S101, the control device 10 starts the projection application 91. The control device 10 starts the projection application 91 when it receives a predetermined operation from the user. The projection application 91 starts on the virtual device 90. The projection application 91 runs in the virtual environment.

[0072] After starting the projection application 91, the control device 10 establishes a communication connection with the first projector 100a in step S102. The projection application 91 sends control signals such as connection requests to the first projector 100a via the TCP / IP communication path TP, thereby establishing a communication connection with the first projector 100a.

[0073] After the projection application 91 is launched, the first projector 100a establishes a communication connection with the projection application 91 in step S131. The first projector 100a receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The first projector 100a transmits various data such as model information and establishes a communication connection with the projection application 91. Once the first projector 100a has established a communication connection with the projection application 91, it can receive shared data transferred from the control device 10.

[0074] After establishing a communication connection with the first projector 100a, the control device 10 receives a mirroring instruction in step S103. When the projection application 91 is started, it displays the UI screen 170 on the display 15. When the user performs a predetermined operation on the mirroring reception icon 172 included in the UI screen 170 displayed on the display 15, the mirroring reception icon 172 receives the mirroring instruction. Upon receiving the mirroring instruction, the projection application 91 generates various control signals, such as a mirroring control signal, corresponding to the mirroring instruction.

[0075] After receiving a mirroring instruction, the projection application 91 sends a capture request corresponding to the mirroring instruction to the browser control unit 31 in step S104. The browser control unit 31 receives the capture request.

[0076] After the projection application 91 sends a mirroring request, the browser control unit 31 determines in step S111 whether or not the capture function extension application 81 is stored in the memory unit 40. If the browser control unit 31 determines that the capture function extension application 81 is not stored in the memory unit 40, it proceeds to step A (S111: NO). Step A will be described later. If the browser control unit 31 determines that the capture function extension application 81 is stored in the memory unit 40, it proceeds to step S112 (S111: YES).

[0077] In step S112, the browser control unit 31 starts the capture function extension application 81. When the browser control unit 31 receives a capture request, it determines whether or not the web browser 80 is running. If the web browser 80 is not running, the browser control unit 31 starts the web browser 80. After starting the web browser 80, the browser control unit 31 starts the capture function extension application 81. Once the capture function extension application 81 is started, the web browser 80 can acquire the generated data output from the output unit 20 as shared data through the functions of the capture function extension application 81.

[0078] The control device 10 executes the capture function extension application 81, and then in step S113, it acquires shared data. The web browser 80 instructs the capture function extension application 81 to acquire shared data. The capture function extension application 81 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. Capture data is an example of shared data that includes at least one of the video data and audio data.

[0079] When the control device 10 acquires shared data, in step S114, it sends the first SDP information from the web browser 80 to the projection application 91. When the projection application 91 acquires the first SDP information in step S105, it sends the first SDP information to the projector 100 in step S106. When the projector 100 acquires the first SDP information in step S132, it registers the first SDP information in the projector memory unit 140 in step S133. In step S134, the projector 100 sends the second SDP information to the projection application 91. When the projection application 91 acquires the second SDP information in step S107, it sends the second SDP information to the web browser 80 in step S108. When the control device 10 acquires the second SDP information in step S115, it registers the second SDP information in the memory unit 40 in step S116. The web browser 80 establishes WebRTC communication with the projector 100 in step S117. The projector 100 establishes WebRTC communication with the web browser 80 in step S135.

[0080] When the control device 10 establishes WebRTC communication with the projector 100, in step S118, it transfers the shared data to the first projector 100a. The capture function extension application 81, which runs on the web browser 80, transfers the shared data acquired via the stream distribution path SP to the projector communication unit 150 of the first projector 100a. The capture function extension application 81 transfers the shared data in real time via the stream distribution path SP to the stream communication unit 151 of the projector communication unit 150. By having the capture function extension application 81 transfer the shared data to the first projector 100a via the stream distribution path SP, the control device 10 can transfer the shared data to the first projector 100a while suppressing a decrease in the quality of the shared data. The projection application 91 can transfer the shared data to the first projector 100a using the functions of the capture function extension application 81.

[0081] When the control device 10 transfers the shared data, the first projector 100a receives the shared data in step S136. The stream communication unit 151 of the projector communication unit 150 receives the shared data via the stream distribution path SP.

[0082] After receiving the shared data, the first projector 100a performs mirroring in step S137. If the shared data includes video data, the first projector 100a acquires the video data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the video data to the projection unit 110. The projection control unit 131 causes the projection unit 110 to project the video data. The projection unit 110 projects the projected image PG based on the video data onto the projection surface SC. If the shared data includes audio data, the first projector 100a acquires the audio data included in the shared data. The projection control unit 131 transmits the audio data to the projector's built-in speaker 120. The projector's built-in speaker 120 outputs audio based on the audio data. Based on the shared data including video data and audio data, the first projector 100a displays the display screen DS shown on the display 15 of the control device 10 and outputs audio based on the audio data. When the shared data is audio data, the projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The first projector 100a outputs audio based on the audio data. The first projector 100a performs mirroring based on the shared data, which includes either video data or audio data.

[0083] The control flow executed by the control device 10 includes establishing a communication connection with the first projector 100a, causing the projection application 91 to send control signals to the first projector 100a via the TCP / IP communication path TP, and causing the capture function extension application 81, which operates on the web browser 80, to acquire shared data including at least one of video data and audio data. The control device 10 can suppress problems such as degradation of shared data quality by having the capture function extension application 81, which runs on the web browser 80, acquire the shared data. The projection application 91 can capture the shared data without having a function to acquire the shared data.

[0084] In the control flow, it is preferable to have the capture function extension application 81 transfer the shared data to the first projector 100a via a stream distribution path SP that is different from the TCP / IP communication path TP. By transferring the shared data via the stream distribution path SP, the control device 10 can suppress the degradation of the shared data when transmitting it to the first projector 100a.

[0085] Figure 7 shows the control flow performed in projection system 1. Figure 7 shows the control flow performed in first projection system 1a, which is an example of projection system 1. Figure 7 shows the control flow when the capture function extension application 81 is not stored in the storage unit 40 at step S111 in Figure 6 (step S111; NO). Figure 7 shows the control flow of process A shown in Figure 6.

[0086] In step S151, the control device 10 determines whether or not there are restrictions on the installation of extension applications. Restrictions on the installation of extension applications are set in advance by the administrator of the control device 10, and the set restriction information is stored in the storage unit 40. The browser control unit 31 of the control device 10 determines whether or not there are restrictions on the installation of extension applications based on the restriction information. If the browser control unit 31 determines that there are no restrictions on the installation of extension applications, it proceeds to step S152 (step S151: NO). If the browser control unit 31 determines that there are restrictions on the installation of extension applications, it proceeds to step S161 (step S151: YES).

[0087] If the control device 10 determines that there are no restrictions on the installation of extension apps, in step S152, it displays an installation instruction image 180 on the display 15. If there are no restrictions on the installation of extension apps, the user can install various extension apps, including the capture function extension app 81, on the control device 10. The control device 10 prompts the user to install the capture function extension app 81 by displaying the installation instruction image 180 on the display 15.

[0088] Figure 8 shows an example of an installation instruction image 180. The installation instruction image 180 is displayed on the display 15. The installation instruction image 180 is displayed within the display screen DS. The installation instruction image 180 shown in Figure 8 is displayed on the UI screen 170 by the web browser 80, but is not limited to this. The installation instruction image 180 may be displayed in a position that does not overlap with the UI screen 170. When the installation instruction image 180 is displayed within the display screen DS, the UI screen 170 may be hidden.

[0089] Installation instruction image 180, as an example, includes an installation instruction message 181, an installation license icon 183, and an installation cancellation icon 185. Installation instruction image 180 may also include messages other than the installation instruction message 181.

[0090] The installation instruction message 181 instructs the user to install the capture function extension application 81. The installation instruction message 181 shown in Figure 8 indicates that the capture function extension application 81 is a video / audio input function extension. The installation instruction message 181 corresponds to an example of an installation instruction display. By viewing the installation instruction message 181, the user can understand that they are being asked to install the capture function extension application 81.

[0091] The installation license icon 183 accepts installation instructions from the user. The installation instructions indicate a request to install the capture function extension application 81. When the user performs a predetermined operation on the installation license icon 183, the control device 10 accepts the installation instructions.

[0092] The installation cancellation icon 185 accepts an instruction from the user to deny installation. This instruction indicates that the user is not permitted to install the capture function extension application 81. When the user performs a predetermined operation on the installation cancellation icon 185, the control device 10 accepts the instruction to deny installation. The control device 10 does not install the capture function extension application 81 and terminates the process.

[0093] After displaying the installation instruction image 180 in step S152 as shown in Figure 7, the control device 10 accepts the installation instruction in step S153. When the user performs a predetermined operation on the installation permission icon 183, the control device 10 accepts the installation instruction.

[0094] After receiving the installation instruction, the control device 10 sends a download request to the application provision server 300 in step S154. The download request is a signal requesting the download of the capture function extension application 81. The control device 10 stores the server address of the application provision server 300 in advance. The acquisition control unit 35 of the control unit 30 sends the download request to the server address.

[0095] When the control device 10 sends a download request, the application provision server 300 receives the download request in step S171. The application provision server 300 receives the download request sent by the acquisition control unit 35.

[0096] When the application provider server 300 receives a download request, in step S172, it sends the capture function extension application 81 to the control unit 10. The application provider server 300 then sends the capture function extension application 81 to the communication unit 50 of the control unit 10 in response to the download request.

[0097] When the app provision server 300 sends the capture function extension app 81, the control device 10 receives the capture function extension app 81 in step S155. The communication unit 50 receives the capture function extension app 81. The acquisition control unit 35 acquires the capture function extension app 81 via the communication unit 50.

[0098] After receiving the capture function extension application 81, the control device 10 installs the capture function extension application 81 in step S156. By installing the capture function extension application 81, the control device 10 enables the web browser 80 to capture the generated data and acquire the generated data as shared data. After installing the capture function extension application 81, the control device 10 proceeds to step S112 shown in Figure 6.

[0099] If the control device 10 determines that the installation of extension apps is restricted, in step S161, it displays the acquisition instruction image 190 on the display 15. When the installation of extension apps is restricted, the user cannot install various extension apps, including the capture function extension app 81, on the control device 10. By displaying the acquisition instruction image 190 on the display 15, the control device 10 prompts the user to acquire the capture app 83, which has similar functionality to the capture function extension app 81.

[0100] Figure 9 shows an example of the acquisition instruction image 190. The acquisition instruction image 190 is displayed on the display 15. The acquisition instruction image 190 is displayed within the display screen DS. The acquisition instruction image 190 shown in Figure 9 is displayed on the UI screen 170, but is not limited to this. The acquisition instruction image 190 may be displayed in a position that does not overlap with the UI screen 170. When the acquisition instruction image 190 is displayed within the display screen DS, the UI screen 170 may be hidden.

[0101] The acquisition instruction image 190 includes, as an example, an acquisition instruction message 191, an acquisition permission icon 193, and an acquisition cancellation icon 195. The acquisition instruction image 190 may also include messages other than the acquisition instruction message 191.

[0102] The acquisition instruction message 191 instructs the user to acquire the capture application 83. In the acquisition instruction message 191 shown in Figure 9, the capture application 83 is identified as a video / audio input application. The acquisition instruction message 191 corresponds to an example of an acquisition instruction display. By viewing the acquisition instruction message 191, the user can understand that they are being asked to acquire the capture application 83.

[0103] The acquisition permission icon 193 accepts an acquisition instruction from the user. The acquisition instruction is an instruction indicating a request to acquire the capture application 83. When the user performs a predetermined operation on the acquisition permission icon 193, the control device 10 accepts the acquisition instruction.

[0104] The acquisition cancellation icon 195 accepts an instruction from the user to deny acquisition. An acquisition denial instruction indicates that the capture application 83 is not permitted to be acquired. When the user performs a predetermined operation on the acquisition cancellation icon 195, the control device 10 accepts the acquisition denial instruction. The control device 10 does not acquire the capture application 83 and terminates the process.

[0105] In step S161 shown in Figure 7, the acquisition instruction image 190 is displayed, and then in step S162, the acquisition instruction is received. When the user performs a predetermined operation on the acquisition permission icon 193, the control device 10 receives the acquisition instruction.

[0106] After receiving the acquisition instruction, the control device 10 sends an acquisition request to the web server 400 in step S163. The acquisition request is a signal requesting the acquisition of the capture application 83. The control device 10 has previously stored the address of the web page on which the capture application 83 operates in the storage unit 40. The address of the web page on which the capture application 83 operates is stored in the storage unit 40 when the projection application 91 is installed in the control device 10. The acquisition control unit 35 of the control unit 30 makes an acquisition request for the capture application 83 by accessing the address of the web page on which the capture application 83 operates.

[0107] When the control device 10 sends an acquisition request, the web server 400 receives the acquisition request in step S191. The web server 400 receives the acquisition request sent by the acquisition control unit 35.

[0108] When the web server 400 receives an acquisition request, in step S192 it sends the capture application 83 to the control unit 10. The web server 400 sends the web page on which the capture application 83 operates to the communication unit 50 of the control unit 10 in response to the acquisition request. The web server 400 sends the capture application 83 to the communication unit 50 by sending the web page on which the capture application 83 operates.

[0109] When the web server 400 sends the capture application 83, the control device 10 receives the capture application 83 in step S164. The communication unit 50 receives the capture application 83. The acquisition control unit 35 acquires the capture application 83 via the communication unit 50.

[0110] After receiving the capture application 83, the control device 10 instructs it to acquire shared data in step S165. The web browser 80 launches the capture application 83 and instructs it to acquire the shared data. The capture application 83 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The captured data is an example of shared data that includes at least one of the video data and audio data.

[0111] After acquiring the shared data, the control device 10 proceeds to step S112 shown in Figure 6.

[0112] The control flow executed by the control device 10 includes, if the capture function extension application 81 is not stored in the storage unit 40, displaying an installation instruction message 181 to install the capture function extension application 81, receiving an installation instruction to install the capture function extension application 81, obtaining the capture function extension application 81, and installing the capture function extension application 81. The control device 10 can activate the capture function extension application 81 in response to a capture request, even if the capture function extension application 81 is not pre-stored.

[0113] The control flow executed by the control device 10 includes, if the capture function extension application 81 is not stored in the memory unit 40, displaying an acquisition instruction message 191 prompting the acquisition of the capture application 83 which runs on the web browser 80; accepting the acquisition instruction for the capture application 83; launching the capture application 83 on the web browser 80; and causing the capture application 83 to acquire the shared data. The control device 10 can acquire shared data using the capture application 83 when there are restrictions on the installation of extension applications.

[0114] The control flow executed by the control device 10 preferably includes instructing the capture application 83 to transfer the shared data to the first projector 100a via a stream communication path SP that is different from the TCP / IP communication path TP. The control device 10 can transfer shared data to the first projector 100a using the capture application 83.

[0115] Second Embodiment The second embodiment shows a second projection system 1b, which is an example of projection system 1. The second projection system 1b includes a second projector 100b that does not have a stream communication unit 151 that communicates using WebRTC technology. The second projector 100b is an example of projector 100.

[0116] Figure 10 shows the system configuration of projection system 1. Figure 10 schematically shows the system configuration of second projection system 1b, which is an example of projection system 1. Figure 10 shows the control device 10 and the second projector 100b.

[0117] The control device 10 shown in Figure 10 has the same configuration as the control device 10 shown in Figure 4. The second projector 100b shown in Figure 10 does not have a stream communication unit 151. The control device 10 and the second projector 100b are not connected via the stream distribution path SP, but via the TCP / IP communication path TP. The control device 10 transmits shared data to the second projector 100b via the TCP / IP communication path TP.

[0118] Figure 11 shows the control flow performed in projection system 1. Figure 11 shows the control flow performed in second projection system 1b, which is an example of projection system 1. Figure 11 shows the control flow performed by control device 10 and second projector 100b. By executing the control flow, control device 10 performs mirroring, which projects the display screen DS displayed on display 15 onto the second projector 100b.

[0119] In step S201, the control device 10 starts the projection application 91. The control device 10 starts the projection application 91 when it receives a predetermined operation from the user. The projection application 91 starts on the virtual device 90. The projection application 91 runs in the virtual environment.

[0120] After starting the projection application 91, the control device 10 establishes a communication connection with the second projector 100b in step S202. The projection application 91 sends control signals such as connection requests to the second projector 100b via the TCP / IP communication path TP, thereby establishing a communication connection with the second projector 100b.

[0121] After the projection application 91 is launched, the second projector 100b establishes a communication connection with the projection application 91 in step S231. The second projector 100b receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The second projector 100b transmits various data such as model information and establishes a communication connection with the projection application 91. Once the second projector 100b has established a communication connection with the projection application 91, it can receive shared data transferred from the control device 10.

[0122] After establishing a communication connection with the second projector 100b, the control device 10 receives a mirroring instruction in step S203. When the projection application 91 is started, it displays the UI screen 170 on the display 15. When the user performs a predetermined operation on the mirroring reception icon 172 included in the UI screen 170 displayed on the display 15, the mirroring reception icon 172 receives the mirroring instruction. Upon receiving the mirroring instruction, the projection application 91 generates various control signals, such as a mirroring control signal, corresponding to the mirroring instruction.

[0123] After receiving a mirroring instruction, the projection application 91 sends a capture request corresponding to the mirroring instruction to the browser control unit 31 in step S204. The browser control unit 31 receives the capture request.

[0124] After the projection application 91 sends a mirroring request, the browser control unit 31 determines in step S211 whether or not the capture function extension application 81 is stored in the storage unit 40. If the browser control unit 31 determines that the capture function extension application 81 is not stored in the storage unit 40, it proceeds to step A shown in Figure 7 (S211: NO). If the browser control unit 31 determines that the capture function extension application 81 is stored in the storage unit 40, it proceeds to step S212 (S211: YES).

[0125] In step S212, the browser control unit 31 starts the capture function extension application 81. When the browser control unit 31 receives a capture request, it determines whether or not the web browser 80 is running. If the web browser 80 is not running, the browser control unit 31 starts the web browser 80. After starting the web browser 80, the browser control unit 31 starts the capture function extension application 81. Once the capture function extension application 81 is started, the web browser 80 can acquire the generated data output from the output unit 20 as shared data through the functions of the capture function extension application 81.

[0126] The control device 10 executes the capture function extension application 81, and then in step S213, it acquires shared data. The web browser 80 instructs the capture function extension application 81 to acquire shared data. The capture function extension application 81 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The captured data is an example of shared data that includes at least one of the video data and audio data.

[0127] When the control device 10 acquires shared data, in step S214, it sends the shared data to the projection application 91. Since the control device 10 and the second projector 100b do not form a stream distribution path SP, the capture function extension application 81 cannot directly transfer the shared data to the second projector 100b. The capture function extension application 81 sends the shared data to the projection application 91, causing the projection application 91 to transfer the shared data to the second projector 100b.

[0128] The projection application 91 receives the shared data in step S205 after the web browser 80 has acquired the shared data. The projection application 91 receives the shared data sent by the capture function extension application 81.

[0129] After receiving the shared data, the projection application 91 transfers the shared data to the second projector 100b in step S206. The projection application 91 transfers the shared data to the second projector 100b via the TCP / IP communication path TP.

[0130] When the control device 10 transfers the shared data, the second projector 100b receives the shared data in step S232. The signal communication unit 153 of the projector communication unit 150 receives the shared data via the TCP / IP communication path TP.

[0131] After receiving the shared data, the second projector 100b performs mirroring in step S233. If the shared data includes video data, the second projector 100b acquires the video data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the video data to the projection unit 110. The projection control unit 131 causes the projection unit 110 to project the video data. The projection unit 110 projects the projected image PG based on the video data onto the projection surface SC. If the shared data includes audio data, the second projector 100b acquires the audio data included in the shared data. The projection control unit 131 transmits the audio data to the projector's built-in speaker 120. The projector's built-in speaker 120 outputs audio based on the audio data. Based on the shared data including video data and audio data, the second projector 100b displays the display screen DS shown on the display 15 of the control device 10 and outputs audio based on the audio data. When the shared data is audio data, the projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The second projector 100b outputs audio based on the audio data. The second projector 100b performs mirroring based on the shared data, which includes either video data or audio data.

[0132] The control flow executed by the control device 10 includes causing the projection application 91 to acquire the shared data obtained by the capture function extension application 81, and causing the projection application 91 to transfer the shared data to the second projector 100b via the TCP / IP communication path TP. The control device 10 can transmit shared data to the second projector 100b, which does not support WebRTC technology.

[0133] A summary of this disclosure is provided below.

[0134] Note 1 An information processing method comprising: establishing a communication connection with a projection device; causing an application program to transmit control signals to the projection device via a signal communication path to control the projection device; and causing a web browser extension program that functions on a web browser to acquire output data including at least one of video data and audio data. The information processing device can minimize problems such as a decrease in output data quality by using a web browser extension program that runs on a web browser to acquire output data. Application programs can receive output data without needing to have their own output data acquisition function.

[0135] Note 2 The information processing method described in Appendix 1 includes causing the web browser extension program to transfer the output data to the projection device via a browser communication path different from the signal communication path. By transmitting output data via a browser communication path, the information processing device can minimize the degradation of output data when transmitting it to the projection device.

[0136] Note 3 The information processing method described in Appendix 1 includes causing the application program to acquire the output data obtained by the web browser extension program, and causing the application program to transfer the output data to the projection device via the signal communication path. The information processing device can transmit output data to a projection device that does not support WebRTC technology.

[0137] Note 4 The information processing method described in any one of the appendices 1 to 3 includes, if the web browser extension program is not stored in the storage unit, displaying an installation instruction message to install the web browser extension program, receiving an installation instruction to install the web browser extension program, obtaining the web browser extension program, and installing the web browser extension program. If the information processing device does not have a web browser extension program pre-stored, it can activate the web browser extension program in response to installation instructions.

[0138] Note 5 The information processing method described in any one of the appendices 1 to 3 includes, if the web browser extension program is not stored in the storage unit, displaying an acquisition instruction display prompting the acquisition of a web application program that runs on the web browser; accepting an acquisition instruction for the web application program; starting the web application program on the web browser; and causing the web application program to acquire the output data. When there are restrictions on the installation of web browser extension programs, the information processing device can retrieve shared data using a web application program.

[0139] Note 6 The information processing method described in Appendix 5 includes causing the web application program to transfer the output data to the projection device via a browser communication path different from the signal communication path. The information processing device can transfer output data to the projection device using a web application program. [Explanation of symbols]

[0140] 1…Projection system, 1a…First projection system, 1b…Second projection system, 10…Control device, 15…Display, 20…Output unit, 21…Display data output unit, 23…Audio output unit, 30…Control unit, 31…Browser control unit, 33…Application control unit, 35…Acquisition control unit, 40…Storage unit, 50…Communication unit, 70…Dedicated operating system, 80…Web browser, 81…Capture function extension application, 83…Capture application, 90…Virtual device, 91…Projection application, 100 ...Projector, 100a...First projector, 100b...Second projector, 110...Projection unit, 111...Light source, 111a...Light source section, 111b...Reflector, 113...Liquid crystal light bulb, 113a...Pixel area, 113B...Liquid crystal light bulb for blue light, 113G...Liquid crystal light bulb for green light, 113R...Liquid crystal light bulb for red light, 115...Light bulb drive unit, 117...Projection lens, 120...Projector built-in speaker, 130...Projector control unit, 131...Projection control section 133...Communication control unit, 140...Projector memory unit, 150...Projector communication unit, 151...Stream communication unit, 153...Signal communication unit, 170...UI screen, 171...Projector name display unit, 172...Mirroring reception icon, 173...Photo display reception icon, 174...Document display reception icon, 175...Camera image display reception icon, 176...Remote control function reception icon, 177...Multi-projection reception icon, 178...Disconnection reception icon, 179...End reception icon, 18 0...Installation instruction image, 181...Installation instruction message, 183...Installation license icon, 185...Installation cancel icon, 190...Acquisition instruction image, 191...Acquisition instruction message, 193...Acquisition license icon, 195...Acquisition cancel icon, 200...Router, 300...App provider server, 400...Web server, DS...Display screen, NW...Communication network, PG...Projected image, PL...Pixel, SC...Projection surface, SP...Stream distribution path, TP...TCP / IP communication path.

Claims

1. Connecting to the projection device via communication, The application program is instructed to transmit control signals to the projection device via a signal communication path, To cause a web browser extension program that functions on a web browser to acquire output data that includes at least one of video data and audio data, Information processing methods including

2. The web browser extension program is instructed to transfer the output data to the projection device via a browser communication path different from the signal communication path. The information processing method according to claim 1, including the following:

3. The application program is to acquire the output data obtained by the web browser extension program, The application program is to transfer the output data to the projection device via the signal communication path, The information processing method according to claim 1, including the following:

4. If the web browser extension program is not stored in the memory unit, To display an installation instruction message prompting the user to install the aforementioned web browser extension program, The system accepts installation instructions to install the aforementioned web browser extension program, Obtaining the aforementioned web browser extension program, Installing the aforementioned web browser extension program, The information processing method according to claim 1, including the following:

5. If the web browser extension program is not stored in the memory unit, To display an acquisition instruction prompting the acquisition of a web application program that runs on the aforementioned web browser, The system accepts instructions to obtain the aforementioned web application program, Launching the web application program on the aforementioned web browser, The web application program is to acquire the output data, The information processing method according to claim 1, including the following:

6. The web application program is instructed to transfer the output data to the projection device via a browser communication path different from the signal communication path. The information processing method according to claim 5, including the following:

Images