Information processing methods

Abstract

Depending on the model of the information processing device and the type of operating system running on it, problems may occur when outputting image data to external devices such as projectors. [Solution] The information processing method includes: communicating with a projection device; storing an application program that controls the projection device; executing the application program; instructing the application program to launch a web application program on a web browser; instructing the application program to transmit control signals to the projection device via a signal communication path; and instructing the web application program to acquire output data that includes at least one of the video data and audio data output from the output unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing method.

Background Art

[0002] A system including a display device and a control device is known. The system described in Patent Document 1 includes a projector, which is an example of a display device, and an information processing device, which is an example of a control device. The projector and the information processing device are communicatively connected via a network. The projector receives image data and the like from the information processing device and projects the image data. The information processing device operates on an operating system such as Windows, Android, or iOS. Windows, Android, and iOS are registered trademarks, respectively.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Depending on the model of the information processing device and the type of the operating system operating on the information processing device, problems may occur when outputting image data to an external device such as a projector.

Means for Solving the Problems

[0005] The information processing method of the present disclosure includes: communicating with a projection device; storing an application program for controlling the projection device; executing the application program; instructing the application program to launch a web application program on a web browser; instructing the application program to transmit control signals to the projection device via a signal communication path; and instructing the web application program to acquire output data including at least one of video data and audio data output from an output unit. [Brief explanation of the drawing]

[0006] [Figure 1] A diagram showing the schematic configuration of the projection system. [Figure 2] A diagram showing the block configuration of the projection system. [Figure 3] A diagram showing the general configuration of the projection unit. [Figure 4] A diagram showing the system configuration of the projection system. [Figure 5] A diagram showing an example of the display screen. [Figure 6] A diagram showing the control flow executed by the control unit. [Figure 7] A diagram showing the system configuration of the projection system. [Figure 8] A diagram showing the control flow executed by the control unit. [Figure 9] A diagram showing the system configuration of the projection system. [Figure 10] A diagram showing the control flow executed by the control unit. [Modes for carrying out the invention]

[0007] Figure 1 shows a schematic configuration of projection system 1. Projection system 1 comprises a control device 10, a projector 100, a router 200, and a cloud server 300. The projector 100 projects a projection image PG onto the projection surface SC. The cloud server 300 communicates with the control device 10 via a communication network NW. The projection system 1 shown in Figure 1 comprises one control device 10 and one projector 100, but is not limited to this. Projection system 1 may comprise multiple control devices 10 and multiple projectors 100.

[0008] The control device 10 communicates with the projector 100. The control device 10 transmits the projected image data to the projector 100. The control device 10 performs mirroring, causing the display screen DS shown on the display 15 to be projected onto the projector 100 as the projected image PG. The control device 10 may also have a function to adjust the image shape, etc., of the projected image PG that the projector 100 projects onto the projection surface SC. The control device 10 corresponds to an example of an information processing device. The control device 10 can be a tablet terminal, a mobile computer, a desktop PC, etc. Figure 1 shows a mobile PC equipped with a display 15 as the control device 10.

[0009] The control device 10 executes various programs, such as a web browser 80, on the dedicated operating system 70. The dedicated operating system 70 is, for example, ChromeOS. Chrome is a registered trademark. The dedicated operating system 70 corresponds to an example of an operating system. The dedicated operating system 70 can create a virtual environment. In the virtual environment, programs that run on an operating system different from the dedicated operating system 70 can run. An example of an operating system different from the dedicated operating system 70 is AndroidOS. An example of a program that runs on an operating system different from the dedicated operating system 70 is an Android application.

[0010] The display 15 displays a screen containing various videos and images. The display 15 is composed of a liquid crystal panel, an organic EL (electro-luminescence) panel, etc. The display 15 may also have a touch input function that accepts input operations from the user.

[0011] The projector 100 is positioned opposite the projection surface SC. The projector 100 communicates with the control device 10. The projector 100 corresponds to an example of a projection device. The projector 100 may also communicate with an external device different from the control device 10. The projector 100 receives projection image data from the control device 10. The projector 100 projects the projection image PG onto the projection surface SC based on the projection image data. The projector 100 may mirror the display screen DS displayed on the display 15 of the control device 10 based on the projection image data. The projector 100 may also project the projection image PG onto the projection surface SC based on pre-stored OSD data. OSD stands for On-Screen Display.

[0012] The projection surface SC displays the projected image PG projected from the projector 100. The projection surface SC shown in Figure 1 is composed of a screen, but is not limited to this. The projection surface SC may be a wall or ceiling in a room, or an exterior wall of a building, etc. The surface shape of the projection surface SC is not limited to a flat surface, but may be a three-dimensional shape such as a curved surface, a surface with irregularities, or a sphere.

[0013] Router 200 is a communication device that interconnects the control device 10, the projector 100, and the cloud server 300. Router 200 communicates with the control device 10 and the cloud server 300 via a communication network NW. Router 200 communicates with the control device 10 and the projector 100 using a predetermined communication protocol. Router 200 may also communicate with the projector 100 and the cloud server 300.

[0014] The communication network NW communicatively connects the cloud server 300 and the control device 10 or the like. As an example, the communication network NW is a WAN (Wide Area Network).

[0015] The cloud server 300 is a virtual server operating in a cloud computing environment. The cloud server 300 is a service that uses one or more virtualized servers via the Internet. The cloud server 300 shown in FIG. 1 includes a web server 310 and a WebSocket server 320.

[0016] The web server 310 provides the display of HTML and objects to the web browser 80 of the client software. FIG. 1 virtually shows the web server 310. The web server 310 provides a web application to the web browser 80 of the control device 10 based on an instruction from the control device 10. When the control device 10 or the like stores the web application in advance, the projection system 1 may not include the web server 310.

[0017] The WebSocket server 320 is a server that performs data communication with the web application on the web browser 80 according to the WebSocket standard. FIG. 1 virtually shows the WebSocket server 320. The WebSocket server 320 is used as a signaling server for SDP exchange when streaming and distributing shared data or the like using WebRTC (Web Real-Time Communication) technology. SDP is an abbreviation for Session Description Protocol.

[0018] FIG. 2 shows the block configuration of the projection system 1. FIG. 2 shows the block configurations of the control device 10, the projector 100, and the cloud server 300 included in the projection system 1. FIG. 2 shows the projection system 1 with the router 200 omitted.

[0019] The control device 10 includes an output unit 20, a control unit 30, a storage unit 40, and a communication unit 50. The control device 10 may include a display 15, an input unit, etc. not shown in FIG. 2.

[0020] The output unit 20 outputs various generated data generated by the control unit 30 etc. to the display 15 etc. The output unit 20 is composed of various circuits, output terminals, etc. The output unit 20 corresponds to an example of an output portion. The output unit 20 has a display data output section 21 and an audio output section 23.

[0021] The display data output section 21 outputs video data for displaying a video to the display 15. The video data includes image data for displaying a still image. The video data is included in the generated data. The video data is transferred from the WebSocket server 320 to the control device 10 using the WebRTC technology. The display data output section 21 may output video data etc. to a display device externally attached to the control device 10.

[0022] The audio output section 23 outputs audio data related to audio. The audio data is included in the generated data. The audio data is transferred from the WebSocket server 320 to the control device 10 using the WebRTC technology. The audio output section 23 connects to a speaker etc. not shown. The speaker etc. may be built into the control device 10 or externally attached to the control device 10. The audio output section 23 outputs audio data by wire or wirelessly. [[ID=<<MASK>>>13]] [[ID=<<MASK>>>14]]

[0023] [[ID=<<MASK>>>15]] The control unit 30 is a device control controller that controls each part of the control device 10. The control unit 30 is, for example, a processor having a CPU (Central Processing Unit). The control unit 30 is composed of one or more processors. A dedicated operating system 70 runs on the control unit 30. The control unit 30 executes various programs on the dedicated operating system 70. By executing various programs, the control unit 30 functions as various functional units. For example, the control unit 30 functions as a browser control unit 31 and an application control unit 33. The control unit 30 may also function as a server functional unit 35. The control unit 30 may also function as a functional unit other than the browser control unit 31, the application control unit 33, and the server functional unit 35.

[0024] The browser control unit 31 operates the web browser 80 on the dedicated operating system 70 and displays web pages, etc. The browser control unit 31 also operates web applications on the web browser 80 and executes the functions of the web applications. For example, the browser control unit 31 operates a capture application 81, which is a web application, on the web browser 80 and captures at least one of video data and audio data. Details of the capture application 81 will be described later.

[0025] The application control unit 33 runs applications on the dedicated operating system 70 or in a virtual environment. The application control unit 33 runs applications with specifications compatible with the dedicated operating system 70 on the dedicated operating system 70. The application control unit 33 also runs applications with specifications different from those compatible with the dedicated operating system 70 in a virtual environment. The virtual environment is configured on a virtual device 90 that runs on the dedicated operating system 70. The virtual device 90 will be described later.

[0026] The server function unit 35 constructs various server functions within the control device 10. The server function unit 35 operates as a virtual server. When the server function unit 35 operates as a virtual server, the control device 10 has server functionality. For example, when the projection application 91 instructs the control device 10 to construct a virtual server, such as a virtual web server 35a, the server function unit 35 operates as a virtual web server 35a within the control device 10. The projection application 91 and the virtual web server 35a will be described later.

[0027] The storage unit 40 stores various programs and various data. The storage unit 40 stores device control programs, various web applications, various applications, etc. The storage unit 40 stores various data generated by the control unit 30. The storage unit 40 is composed of volatile semiconductor memory such as RAM (Random Access Memory), non-volatile memory such as ROM (Read Only Memory), flash memory, etc. The storage unit 40 may also function as a work area for the control unit 30. The storage unit 40 corresponds to an example of a storage unit.

[0028] The communication unit 50 is a communication interface circuit that communicates with the projector 100, the cloud server 300, etc. The communication unit 50 connects wirelessly to the projector 100, etc., according to a predetermined communication protocol. The communication unit 50 includes a wireless communication port. The wireless communication port is a wireless LAN (Local Area Network) communication port, etc. The communication unit 50 may also be connected to the projector 100, etc., by wire. The communication unit 50 has a wired connector such as USB (Universal Serial Bus). The communication unit 50 transmits various control signals, projected image data, etc., to the projector 100. The projected image data includes video data, audio data, etc. The communication unit 50 receives various data, response signals, etc., from the cloud server 300, etc.

[0029] The projector 100 comprises a projection unit 110, a projector-integrated speaker 120, a projector control unit 130, a projector memory unit 140, and a projector communication unit 150.

[0030] The projection unit 110 projects image light based on the projected image data onto the projection surface SC. The projection unit 110 projects the projected image PG onto the projection surface SC based on the control of the projector control unit 130. The projection unit 110 projects video, still images, OSD images, etc., as the projected image PG based on the video data etc. included in the projected image data. The OSD image is an image that shows various settings and operation information related to the projector 100.

[0031] Figure 3 shows a schematic configuration of the projection unit 110. Figure 3 shows an example of the projection unit 110. The projection unit 110 comprises a light source 111, three liquid crystal light bulbs 113, a light bulb drive unit 115, and a projection lens 117.

[0032] The light source 111 emits light into the liquid crystal light bulb 113. The light source 111 comprises a light source unit 111a, a reflector 111b, an integrator optical system (not shown), and a color separation optical system (not shown). The light source unit 111a emits light. The light source unit 111a is composed of a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, etc. The light source unit 111a emits light based on the control of the projector control unit 130. The reflector 111b reduces variations in the emission direction of the light emitted by the light source unit 111a. The integrator optical system reduces variations in the brightness distribution of the light emitted from the light source unit 111a. The light that has passed through the reflector 111b is incident on the color separation optical system. The color separation optical system separates the incident light into red, green, and blue color light components.

[0033] The liquid crystal light bulb 113 modulates the light emitted from the light source 111. By modulating the light, the liquid crystal light bulb 113 generates a projected image PG, etc. The liquid crystal light bulb 113 is composed of a transmissive liquid crystal panel in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal light bulb 113 has a rectangular pixel area 113a that includes a plurality of pixels PL arranged in a matrix. In the liquid crystal light bulb 113, a driving voltage is applied to each pixel PL of the liquid crystal. The projector 100 shown in Figure 3 has three liquid crystal light bulbs 113. The projector 100 is configured to use a transmissive liquid crystal panel equipped with liquid crystal light bulbs 113, but is not limited to this. The projector 100 may also be equipped with a reflective liquid crystal panel or an LCOS (Liquid Crystal On Silicon) liquid crystal panel. The projector 100 may also be configured to be equipped with one or more DMDs (Digital Mirror Devices).

[0034] The three liquid crystal light bulbs 113 are a red light liquid crystal light bulb 113R, a green light liquid crystal light bulb 113G, and a blue light liquid crystal light bulb 113B. The red color light component separated by the color separation optical system is incident on the red light liquid crystal light bulb 113R. The green color light component separated by the color separation optical system is incident on the green light liquid crystal light bulb 113G. The blue color light component separated by the color separation optical system is incident on the blue light liquid crystal light bulb 113B.

[0035] The light bulb drive unit 115 applies a drive voltage to each pixel PL based on the projected image data received from the projector control unit 130. The light bulb drive unit 115 is, for example, a control circuit. The drive voltage is supplied by a drive source (not shown). The light bulb drive unit 115 may also apply the drive voltage to each pixel PL based on the projected image data corrected by the projection control unit 131, which will be described later. When the light bulb drive unit 115 applies the drive voltage to each pixel PL, each pixel PL is set to a light transmittance based on the projected image data. The light emitted from the light source 111 is modulated by passing through the pixel region 113a. The three liquid crystal light bulbs 113 form a color component image for each color of light.

[0036] The projection lens 117 synthesizes the color component images formed by the liquid crystal light bulb 113 and projects them at an enlarged size. The projection lens 117 projects the projected image PG onto the projection surface SC. The projected image PG is a multi-color image formed by synthesizing the individual color component images.

[0037] The projector-integrated speaker 120 shown in Figure 2 outputs sound based on audio data contained in the projected image data. The projector-integrated speaker 120 receives audio data from the projector control unit 130 and outputs sound corresponding to the audio data. The projector 100 shown in Figure 2 is equipped with a projector-integrated speaker 120, but is not limited thereto. The projector 100 may also be equipped with an external speaker. The external speaker outputs sound based on audio data.

[0038] The projector control unit 130 is a projector control controller that controls each unit of the projector 100. The projector control unit 130 is, for example, a processor having a CPU. The projector control unit 130 consists of one or more processors. The projector control unit 130 functions as a projection control unit 131 by executing firmware. The projector control unit 130 may also function as a projector server function unit 133 by executing firmware. The projector control unit 130 may also function as a function unit other than the projection control unit 131 or the projector server function unit 133.

[0039] The projection control unit 131 controls projection by the projection unit 110. Based on the video data contained in the projection image data transmitted from the control device 10, the projection control unit 131 causes the projection unit 110 to project the projection image PG. Based on the audio data contained in the projection image data, the projection control unit 131 causes the projector's built-in speaker 120 to output audio. The projection control unit 131 synchronizes the projection image PG projected from the projection unit 110 with the audio output from the projector's built-in speaker 120. The projection control unit 131 performs various correction processes on the video data, such as keystone correction.

[0040] The projector server function unit 133 implements various server functions within the projector 100. For example, if the firmware has the ability to execute server functions, the projector server function unit 133 operates as a virtual server within the projector 100. When the projector server function unit 133 operates as a virtual server, the projector 100 has server functions.

[0041] The projector memory unit 140 stores firmware and various data. The projector memory unit 140 also stores projection image data transmitted from the control device 10, correction data for correcting the video data contained in the projection image data, and so on. The projector memory unit 140 is composed of volatile semiconductor memory such as RAM, and non-volatile memory such as ROM and flash memory.

[0042] The projector communication unit 150 is a communication interface circuit that communicates with the control device 10, external devices, etc. The projector communication unit 150 connects wirelessly to the control device 10, etc., according to a predetermined communication protocol. The projector communication unit 150 includes a wireless communication port. The wireless communication port is a wireless LAN communication port, etc. The projector communication unit 150 may also be connected to the control device 10, etc., by wire. The projector communication unit 150 has a wired connector such as a USB. The projector communication unit 150 receives projected image data, control signals, etc., from the control device 10. The projector communication unit 150 transmits various data such as projector setting information to the control device 10, etc.

[0043] The cloud server 300 comprises a web server 310, a WebSocket server 320, a server storage device 330, and a server communication device 340. The web server 310, WebSocket server 320, server storage device 330, and server communication device 340 are configured within one or more devices.

[0044] The server storage device 330 stores various programs, various data, etc. The server storage device 330 stores web pages, web applications, etc., provided by various web servers 310. The server storage device 330 is composed of a magnetic recording medium such as an HDD (Hard Disk Drive). The server storage device 330 may be composed of volatile semiconductor memory such as RAM, or non-volatile memory such as ROM or flash memory.

[0045] The server communication device 340 is a communication interface circuit that communicates with the control device 10, etc. The server communication device 340 connects with the control device 10, etc. by wire or wireless connection according to a predetermined communication protocol. The server communication device 340 includes wired connectors, wireless communication ports, etc. Wired connectors include USB connectors, LAN connectors, etc. Wireless communication ports include Wi-Fi communication ports, Bluetooth communication ports, etc. Wi-Fi and Bluetooth are registered trademarks. The server communication device 340 transmits web applications to the control device 10. The server communication device 340 receives various signals transmitted from the control device 10.

[0046] First Embodiment The first embodiment shows the system configuration and control flow of a first projection system 1a, which is an example of projection system 1. In the first projection system 1a, the control device 10 constructs a system configuration that transmits shared data to the projector 100 using a cloud server 300. The shared data includes at least one of video data and audio data. The shared data corresponds to an example of output data. As an example, the control device 10 transmits the shared data to the projector 100 to perform mirroring, which projects the display screen DS displayed on the display 15 onto the projector 100. In the first projection system 1a, the web server 310 in the cloud server 300 provides a web application to the control device 10. The control device 10 uses the WebSocket server 320 in the cloud server 300 to send and receive various signals. In the first projection system 1a, the server function unit 35 and the projector server function unit 133 do not operate as virtual servers.

[0047] Figure 4 shows the system configuration of projection system 1. Figure 4 schematically shows the system configuration of the first projection system 1a, which is an example of projection system 1. Figure 4 shows the control device 10, the projector 100, and the cloud server 300.

[0048] The control device 10 operates a dedicated operating system 70. In the first embodiment, the dedicated operating system 70 is ChromeOS. The control unit 30 of the control device 10 operates the dedicated operating system 70. Applications corresponding to the dedicated operating system 70 can run on the dedicated operating system 70. On the dedicated operating system 70 shown in Figure 4, a web browser 80 and a virtual device 90 run. Applications other than the web browser 80 and virtual device 90 may also run on the dedicated operating system 70.

[0049] The web browser 80 is software that connects to the web server 310. The web browser 80 runs on a dedicated operating system 70. One example of a web browser 80 is Chrome. The web browser 80 can launch various web applications. The web browser 80 shown in Figure 4 can launch a capture application 81. The capture application 81 is an example of a web application.

[0050] The capture application 81 is launched on the web browser 80. The web browser 80 shown in Figure 4 obtains the capture application 81 from the web server 310. The capture application 81 captures the generated data output from the output unit 20 of the control device 10 as capture data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The capture data includes at least one of the video data and the audio data. The capture application 81 sends the capture data as shared data to the projector 100 via the stream distribution path SP. The capture application 81 may also send the capture data to the projection application 91. The capture application 81 corresponds to an example of a web application program.

[0051] The virtual device 90 is a virtual device that runs on the dedicated operating system 70. The virtual device 90 is an example of an emulator that runs on the dedicated operating system 70. The virtual device 90 functions as a device that runs an operating system different from the dedicated operating system 70 on the dedicated operating system 70. The virtual device 90 provides a virtual environment on the dedicated operating system 70. The virtual device 90 is, as an example, a device on which the Android OS runs. Applications that run on an operating system different from the dedicated operating system 70 can be launched on the virtual device 90. Applications that run on an operating system different from the dedicated operating system 70 are Android apps. The projection app 91 shown in Figure 4 is an example of an Android app.

[0052] The projection application 91 is software that controls the projector 100. The projection application 91 is stored in the memory unit 40 in advance. The projection application 91 causes the projector 100 to project various projection images PG. The projection application 91 sends various control signals to the projector 100, the cloud server 300, etc. The projection application 91 receives response signals, etc. from the projector 100, the cloud server 300, etc. The projection application 91 causes the capture application 81 to acquire shared data. The projection application 91 may also cause the capture application 81 to send the shared data to the projector 100. The projection application 91 corresponds to an example of an application program.

[0053] The projection application 91, which runs on the virtual device 90, sends and receives various control signals, etc., to and from the dedicated operating system 70 via various APIs (Application Programming Interfaces). The APIs vary depending on the version of the operating system running on the virtual device 90. Depending on the version of the operating system running on the virtual device 90, the projection application 91 may have difficulty controlling the projector 100. The projection application 91 controls the projector 100 using the functions of the web browser 80 and the cloud server 300.

[0054] Figure 5 shows an example of the display screen DS. The display screen DS is displayed on the display 15. The display screen DS shown in Figure 5 is the display screen DS when the control device 10 starts the projection application 91. The display screen DS includes the UI screen 170 of the projection application 91. UI stands for User Interface.

[0055] The UI screen 170 displays multiple icons, etc. The UI screen 170 includes a projector name display section 171, a mirroring request icon 172, a photo display request icon 173, a document display request icon 174, a camera image display request icon 175, a remote control function request icon 176, a multi-projection request icon 177, a disconnection request icon 178, and an end request icon 179.

[0056] The projector name display unit 171 displays the projector name of the projector 100 connected to the control device 10. The projector name display unit 171 displays the projector name based on the model information of the projector 100 obtained from the projector 100.

[0057] The mirroring reception icon 172 receives a mirroring instruction from the user. When the user performs a predetermined operation on the mirroring reception icon 172, the mirroring reception icon 172 receives the mirroring instruction. When the projection application 91 receives the mirroring instruction, it generates a mirroring control signal corresponding to the mirroring instruction. The mirroring reception icon 172 includes a switch icon 172a.

[0058] The switch icon 172a indicates whether the projector 100 is performing mirroring. The user can determine whether the projector 100 is performing mirroring by checking the display of the switch icon 172a. The switch icon 172a may also accept a mirroring command from the user.

[0059] The photo display reception icon 173 receives a photo display instruction from the user. When the user performs a predetermined operation on the photo display reception icon 173, the photo display reception icon 173 receives the photo display instruction. When the projection application 91 receives the photo display instruction, it performs a photo display process that causes the control device 10 to project the photo image stored on the projector 100. The projection application 91 sends the photo image data to be displayed to the projector 100 and causes the projector 100 to project the photo image as a projection image PG.

[0060] The document display reception icon 174 receives a document display instruction from the user. When the user performs a predetermined operation on the document display reception icon 174, the document display reception icon 174 receives the document display instruction. Upon receiving the document display instruction, the projection application 91 performs document display processing to project a PDF file or the like stored in the control device 10 onto the projector 100. For example, the projection application 91 sends PDF data to the projector 100 to display a PDF image, causing the projector 100 to project the PDF image as a projected image PG.

[0061] The camera image display reception icon 175 receives a camera image display instruction from the user. If the control device 10 is equipped with an imaging device such as a camera, the camera image display reception icon 175 becomes operable. When the user performs a predetermined operation on the camera image display reception icon 175, the camera image display reception icon 175 receives the camera image display instruction. When the projection application 91 receives the camera image display instruction, it performs camera image display processing to project the image captured by the imaging device onto the projector 100. The projection application 91 sends the imaging data to display the captured image to the projector 100, causing the projector 100 to project the captured image as a projected image PG.

[0062] The remote control function reception icon 176 receives remote control function operation instructions from the user. When the user performs a predetermined operation on the remote control function reception icon 176, the remote control function reception icon 176 receives the remote control function operation instruction. When the projection application 91 receives the remote control function operation instruction, it displays a remote control image on the projection image PG that shows the remote control for operating the projector 100. When the user performs an input operation on the remote control image, the projector 100 can perform the corresponding control operation.

[0063] The multi-projection reception icon 177 accepts multi-projection instructions from the user. The multi-projection reception icon 177 becomes operable when multiple projectors 100 are connected to the control device 10. When the user performs a predetermined operation on the multi-projection reception icon 177, the multi-projection reception icon 177 accepts the multi-projection instruction. The projection application 91 then performs multi-projection, causing the multiple projectors 100 connected to the control device 10 to project the projected image PG onto the projection surface SC.

[0064] The disconnection notification icon 178 receives a disconnection command from the user. When the user performs a predetermined operation on the disconnection notification icon 178, the disconnection notification icon 178 receives the disconnection command. When the projection application 91 receives the disconnection command, it disconnects the communication connection between the control device 10 and the projector 100.

[0065] The termination notification icon 179 accepts termination instructions from the user. When the user performs a predetermined operation on the termination notification icon 179, the termination notification icon 179 accepts the termination instruction. Upon receiving the termination instruction, the projection application 91 stops operating. The projection application 91 hides the UI screen 170.

[0066] The stream distribution path SP shown in Figure 4 is a path for real-time communication between the web browser 80 and the projector 100. The stream distribution path SP communicates using WebRTC technology, for example. The web browser 80 can send shared data to the projector 100 via the stream distribution path SP. By the web browser 80 sending shared data to the projector 100 via the stream distribution path SP, the control device 10 can send the shared data to the projector 100 without degrading its quality. The stream distribution path SP corresponds to an example of a browser communication path.

[0067] The TCP / IP communication path TP is a path that communicates using TCP / IP (Transmission Control Protocol / Internet Protocol). The TCP / IP communication path TP and the stream distribution path SP are different communication paths. The projection application 91 sends various control signals to the projector 100 via the TCP / IP communication path TP. The projection application 91 receives control signals from the projector 100 via the TCP / IP communication path TP. The projection application 91 may also send shared data to the projector 100 via the TCP / IP communication path TP. When shared data is sent to the projector 100 via the TCP / IP communication path TP, the quality of the shared data may be degraded depending on the status of the virtual device 90. It is preferable that the shared data is sent to the projector 100 via the stream distribution path SP. The TCP / IP communication path TP corresponds to an example of a signal communication path.

[0068] The projector communication unit 150 of the projector 100 has a stream communication unit 151 and a signal communication unit 153.

[0069] The stream communication unit 151 communicates with the capture application 81 via the stream distribution path SP. The stream communication unit 151 supports WebRTC technology. The stream communication unit 151 receives shared data transmitted from the capture application 81 in real time.

[0070] The signal communication unit 153 establishes a communication connection with the projection application 91 via the TCP / IP communication path TP. The signal communication unit 153 receives control signals from the projection application 91. The signal communication unit 153 transmits control signals generated by the projector 100 to the projection application 91. By the signal communication unit 153 sending and receiving control signals between the projection application 91 and the projector 100, the projection application 91 can control the projector 100.

[0071] The web server 310 included in the cloud server 300 has a server storage unit 330a. The server storage unit 330a is part of the server storage device 330. The server storage unit 330a stores various web applications, including the capture application 81. The web server 310 provides the web applications stored in the server storage unit 330a to the web browser 80.

[0072] The WebSocket server 320 receives various control signals transmitted from the projection application 91 via the TCP / IP communication path TP. The control signals transmitted from the projection application 91 include a web page request. A web page request is a signal instructing the web server 310 to send a web page containing the web application to the web browser 80. When the WebSocket server 320 receives a web page request, it sends a web page request to the web server 310. Based on the web page request, the web server 310 sends a web page containing the web application to the web browser 80.

[0073] Figure 6 shows the control flow performed in projection system 1. Figure 6 shows the control flow performed in first projection system 1a, which is an example of projection system 1. Figure 6 shows the control flow performed in control device 10, projector 100, and cloud server 300. The control flow performed in control device 10 corresponds to an example of an information processing method. By executing the control flow, control device 10 performs mirroring, which projects the display screen DS displayed on display 15 onto projector 100.

[0074] In step S101, the control device 10 starts the projection application 91. The control device 10 starts the projection application 91 when it receives a predetermined operation from the user. The projection application 91 starts on the virtual device 90. The projection application 91 runs in the virtual environment.

[0075] After starting the projection application 91, the control device 10 establishes a communication connection with the projector 100 in step S102. The projection application 91 sends control signals such as connection requests to the projector 100 via the TCP / IP communication path TP, thereby establishing a communication connection with the projector 100.

[0076] After the projection application 91 is launched, the projector 100 establishes a communication connection with the control device 10 in step S131. The projector 100 receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The projector 100 transmits various data such as model information and establishes a communication connection with the control device 10. Once the projector 100 has established a communication connection with the control device 10, it becomes possible to receive various data from the control device 10.

[0077] After establishing a communication connection with the projector 100, the control device 10 receives a mirroring instruction in step S103. When the projection application 91 is launched, it displays the UI screen 170 on the display 15. When the user performs a predetermined operation on the mirroring reception icon 172 included in the UI screen 170 displayed on the display 15, the mirroring reception icon 172 receives the mirroring instruction. Upon receiving the mirroring instruction, the projection application 91 generates various control signals, such as a mirroring control signal, corresponding to the mirroring instruction.

[0078] After receiving the mirroring instruction, the control device 10 sends a startup instruction to the control unit 30 in step S104. The projection application 91 sends the startup instruction to the browser control unit 31 of the control unit 30. The browser control unit 31 receives the startup instruction.

[0079] When the projection application 91 sends a startup command to the browser control unit 31, the control device 10 starts the web browser 80 in step S111. If the web browser 80 is not running, the browser control unit 31 of the control unit 30 starts the web browser 80 based on the startup command. The web browser 80 starts up based on the startup command and displays a web page tab. If the web browser 80 is running when the browser control unit 31 receives the startup command, it displays a new web page tab.

[0080] After receiving the mirroring instruction, the control device 10 sends a web page request to the cloud server 300 in step S105. The projection application 91 sends the web page request to the WebSocket server 320 in the cloud server 300 via the TCP / IP communication path TP. The web page request is a request to send the web page on which the capture application 81 operates to the web browser 80. The web page request includes the page address of the web page on which the capture application 81 operates. The page address is stored in the storage unit 40 when the projection application 91 is installed in the control device 10.

[0081] When the projection application 91 sends a web page request, the cloud server 300 receives the web page request in step S151. The WebSocket server 320 included in the cloud server 300 receives the web page request sent from the projection application 91 via the TCP / IP communication path TP. The WebSocket server 320 sends the web page request to the web server 310. The web server 310 receives the web page request.

[0082] When the cloud server 300 receives a web page request, in step S152 it sends web page information to the web browser 80. The web server 310 retrieves the web page information stored in the server storage unit 330a based on the page address included in the web page request. The web page information is information relating to the web page on which the capture application 81 is running. The web server 310 sends the web page information to the web browser 80.

[0083] When the cloud server 300 transmits web page information, the control device 10 receives the web page information in step S112. When the web browser 80 receives the web page information, it displays the web page on the web page tab in which the capture application 81 will run based on the web page information. The web browser 80 then makes the capture application 81 executable.

[0084] After receiving the web page information, the control device 10 executes the capture application 81 in step S113. The web browser 80 executes the capture application 81 on the web page tab.

[0085] The control device 10 executes the capture application 81 and then, in step S114, causes it to acquire shared data. The web browser 80 causes the capture application 81 to acquire shared data. The capture application 81 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The captured data is an example of shared data that includes at least one of the video data and audio data.

[0086] When the control device 10 acquires shared data, in step S115, it sends the first SDP information from the web browser 80 to the WebSocket server 320. When the cloud server 300 acquires the first SDP information, in step S154, it sends the first SDP information from the WebSocket server 320 to the projection application 91. When the projection application 91 acquires the first SDP information, it sends the first SDP information to the projector 100 in step S107. When the projector 100 acquires the first SDP information, it registers the first SDP information in the projector memory unit 140 in step S133. In step S134, the projector 100 sends the second SDP information to the projection application 91. When the projection application 91 acquires the second SDP information, it sends the second SDP information to the cloud server 300 in step S109. When the cloud server 300 obtains the second SDP information, in step S156, it sends the second SDP information from the WebSocket server 320 to the control unit 10. When the control unit 10 obtains the second SDP information, in step S117, it registers the second SDP information in the storage unit 40. The web browser 80 establishes WebRTC communication with the projector 100. When the control device 10 establishes WebRTC communication with the projector 100, in step S119, it sends the shared data to the projector 100. The control device 10 sends the shared data acquired by the capture application 81 via the stream distribution path SP to the projector communication unit 150 of the projector 100. By sending the shared data acquired by the capture application 81 to the projector 100 via the stream distribution path SP, the control device 10 can send the shared data to the projector 100 while suppressing a decrease in the quality of the shared data. The projection application 91 can send the shared data to the projector 100 using the functions of the capture application 81.

[0087] When the control device 10 transmits shared data, the projector 100 receives the shared data in step S136. The stream communication unit 151 of the projector communication unit 150 receives the shared data via the stream distribution path SP.

[0088] After receiving the shared data, the projector 100 performs mirroring in step S137. If the shared data includes video data, the projector 100 acquires the video data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the video data to the projection unit 110. The projection control unit 131 causes the projection unit 110 to project the video data. The projection unit 110 projects the projected image PG based on the video data onto the projection surface SC. If the shared data includes audio data, the projector 100 acquires the audio data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The projector's built-in speaker 120 outputs audio based on the audio data. Based on the shared data including video data and audio data, the projector 100 projects the display screen DS displayed on the display 15 of the control device 10 and outputs audio based on the audio data. When the shared data is audio data, the projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The projector 100 outputs audio based on the audio data. The projector 100 performs mirroring based on the shared data, which includes either video data or audio data.

[0089] The control flow executed by the control device 10 includes: establishing a communication connection with the projector 100; storing a projection application 91 for controlling the projector 100; executing the projection application 91; instructing the projection application 91 to launch the capture application 81 on the web browser 80; instructing the projection application 91 to send control signals to the projector 100 via the TCP / IP communication path TP; and instructing the capture application 81 to acquire shared data, which includes at least one of the video data and audio data output from the output unit 20. The control device 10 can suppress problems such as degradation of shared data quality by having the control device 10 acquire shared data using a capture application 81 that runs on a web browser 80. The projection application 91 can capture shared data without having a function to acquire shared data.

[0090] In the control flow, it is preferable to have the capture application 81 send the shared data to the projector 100 via a stream distribution path SP that is different from the TCP / IP communication path TP. By transmitting shared data via the stream distribution path SP, the control device 10 can suppress the degradation of shared data when transmitting it to the projector 100.

[0091] The shared data is captured data that includes at least one of the video data and audio data output from the output unit 20. The control device 10 can cause the projector 100 to perform mirroring based on at least one of the video data and audio data.

[0092] In the control flow shown in Figure 6, the capture application 81 sends the shared data to the projector 100 via the stream distribution path SP, but this is not limited to that. The projection application 91 may receive the shared data from the capture application 81 and send the shared data to the projector 100.

[0093] Projector 100 may not have the functionality to receive shared data using WebRTC technology. In this case, when the capture application 81 acquires the shared data, it sends the shared data to the projection application 91 via the WebSocket server 320. The projection application 91 receives the shared data. The projection application 91 sends the shared data to the projector communication unit 150 of projector 100 via the TCP / IP communication path TP. The projector communication unit 150 receives the shared data and sends it to the projector control unit 130. The projection control unit 131 of the projector control unit 130 sends the video data included in the shared data to the projection unit 110 and the audio data included in the shared data to the projector's built-in speaker 120. Projector 100 performs mirroring based on the shared data using at least one of the projection unit 110 and the projector's built-in speaker 120. The communication method used to send the shared data from the capture application 81 to the projection application 91 via the WebSocket server 320 may be either WebRTC communication or TCP / IP communication, but WebRTC communication is preferred. This is because WebRTC communication is faster than TCP / IP communication, which helps to suppress data transfer delays.

[0094] It is preferable to have the projection application 91 acquire the shared data from the capture application 81. The projection application 91 can acquire the shared data and then send it to the projector 100 via the TCP / IP communication path TP.

[0095] Second Embodiment The second embodiment shows the system configuration and control flow of the second projection system 1b, which is an example of projection system 1. The second projection system 1b is an example of projection system 1 in which the projector 100 has a projector server function unit 133. The projector server function unit 133 has server functions similar to the cloud server 300 shown in the first embodiment. The control device 10 performs mirroring by sending shared data to the projector 100, thereby projecting the display screen DS displayed on the display 15 onto the projector 100. In the second projection system 1b, the projector server function unit 133 in the projector 100 provides the control device 10 with a web application. In the second projection system 1b, the server function unit 35 of the control device 10 does not operate.

[0096] Figure 7 shows the system configuration of projection system 1. Figure 7 also shows a schematic representation of the system configuration of second projection system 1b, which is an example of projection system 1. Figure 7 shows the control device 10 and the projector 100. The configuration of the control device 10 shown in Figure 7 is the same as the configuration of the control device 10 shown in Figure 4.

[0097] Figure 7 shows a projector 100 having a projector control unit 130 and a projector communication unit 150. The configuration of the projector communication unit 150 shown in Figure 7 is the same as the configuration of the projector communication unit 150 shown in Figure 4. The projector control unit 130 has a projector server function unit 133.

[0098] The projector server function unit 133 performs operations equivalent to those of the cloud server 300 shown in Figure 4. The projector server function unit 133 includes a web server function unit 133a and a WebSocket server function unit 133b. The web server function unit 133a performs operations similar to those of the web server 310 shown in Figure 4. The WebSocket server function unit 133b performs operations similar to those of the WebSocket server 320 shown in Figure 4.

[0099] Figure 8 shows the control flow performed in projection system 1. Figure 8 shows the control flow performed in second projection system 1b, which is an example of projection system 1. Figure 8 shows the control flow performed in control device 10 and projector 100. By executing the control flow, control device 10 performs mirroring, which projects the display screen DS displayed on display 15 onto projector 100.

[0100] In step S201, the control device 10 starts the projection application 91. The control device 10 starts the projection application 91 when it receives a predetermined operation from the user. The projection application 91 starts on the virtual device 90. The projection application 91 runs in the virtual environment.

[0101] After starting the projection application 91, the control device 10 establishes a communication connection with the projector 100 in step S202. The projection application 91 sends control signals such as connection requests to the projector 100 via the TCP / IP communication path TP, thereby establishing a communication connection with the projector 100.

[0102] After the projection application 91 is launched, the projector 100 establishes a communication connection with the control device 10 in step S231. The projector 100 receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The projector 100 transmits various data such as model information and establishes a communication connection with the control device 10. Once the projector 100 has established a communication connection with the control device 10, it becomes possible to receive various data from the control device 10.

[0103] After establishing a communication connection with the projector 100, the control device 10 receives a mirroring instruction in step S203. When the projection application 91 is launched, it displays the UI screen 170 on the display 15. When the user performs a predetermined operation on the mirroring reception icon 172 included in the UI screen 170 displayed on the display 15, the mirroring reception icon 172 receives the mirroring instruction. Upon receiving the mirroring instruction, the projection application 91 generates various control signals, such as a mirroring control signal, corresponding to the mirroring instruction.

[0104] After receiving the mirroring instruction, the control device 10 sends a startup instruction to the control unit 30 in step S204. The projection application 91 sends the startup instruction to the browser control unit 31 of the control unit 30. The browser control unit 31 receives the startup instruction.

[0105] When the projection application 91 sends a startup command to the browser control unit 31, the control device 10 starts the web browser 80 in step S211. If the web browser 80 is not running, the browser control unit 31 of the control unit 30 starts the web browser 80 based on the startup command. The web browser 80 starts up based on the startup command and displays a web page tab. If the web browser 80 is running when the browser control unit 31 receives the startup command, it displays a new web page tab.

[0106] After receiving the mirroring instruction, the control device 10 sends a web page request to the projector 100 in step S205. The projection application 91 sends the web page request to the WebSocket server function unit 133b in the projector 100 via the TCP / IP communication path TP. The web page request includes the page address of the web page on which the capture application 81 operates. The page address is stored in the storage unit 40 when the projection application 91 is installed in the control device 10.

[0107] In step S232, the projector 100 receives a web page request when the projection application 91 sends a web page request. The WebSocket server function unit 133b included in the projector 100 receives the web page request sent from the projection application 91 via the TCP / IP communication path TP. The WebSocket server function unit 133b sends the web page request to the web server function unit 133a. The web server function unit 133a receives the web page request.

[0108] When the projector 100 receives a web page request, in step S233, it sends web page information to the web browser 80. The web server function unit 133a retrieves the web page information stored in the projector storage unit 140 based on the page address included in the web page request. The web page information is information relating to the web page on which the capture application 81 is running. The web server function unit 133a sends the web page information to the web browser 80.

[0109] When the projector 100 transmits web page information, the control device 10 receives the web page information in step S212. When the web browser 80 receives the web page information, it displays the web page on the web page tab in which the capture application 81 will run based on the web page information. The web browser 80 then makes the capture application 81 executable.

[0110] After receiving the web page information, the control device 10 executes the capture application 81 in step S213. The web browser 80 executes the capture application 81 on the web page tab.

[0111] The control device 10 executes the capture application 81 and then, in step S214, causes it to acquire shared data. The web browser 80 causes the capture application 81 to acquire shared data. The capture application 81 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. Capture data is an example of shared data that includes at least one of the video data and audio data.

[0112] When the control device 10 acquires shared data, in step S215, it sends the first SDP information from the web browser 80 to the WebSocket server function unit 133b. When the projector 100 acquires the first SDP information, in step S235, it registers the first SDP information in the projector memory unit 140. In step S236, the projector 100 sends the second SDP information to the capture application 81. When the control device 10 acquires the second SDP information, in step S217, it registers the second SDP information in the memory unit 40. The control device 10 establishes WebRTC communication with the projector 100. When the control device 10 establishes WebRTC communication with the projector 100, in step S219, it sends the shared data to the projector 100. The control device 10 sends the shared data acquired by the capture application 81 via the stream distribution path SP to the projector communication unit 150 of the projector 100. By sending the shared data acquired by the capture application 81 to the projector 100 via the stream distribution path SP, the control device 10 can send the shared data to the projector 100 while suppressing a decrease in the quality of the shared data. The projection application 91 can send the shared data to the projector 100 using the functions of the capture application 81.

[0113] When the control device 10 transmits shared data, the projector 100 receives the shared data in step S238. The stream communication unit 151 of the projector communication unit 150 receives the shared data via the stream distribution path SP.

[0114] After receiving the shared data, the projector 100 performs mirroring in step S239. If the shared data includes video data, the projector 100 acquires the video data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the video data to the projection unit 110. The projection control unit 131 causes the projection unit 110 to project the video data. The projection unit 110 projects the projected image PG based on the video data onto the projection surface SC. If the shared data includes audio data, the projector 100 acquires the audio data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The projector's built-in speaker 120 outputs audio based on the audio data. Based on the shared data which includes at least one of the video data and audio data, the projector 100 mirrors the display screen DS displayed on the display 15 of the control device 10.

[0115] If the projector 100 has server functionality, the control flow preferably includes the projector 100 storing the capture application 81 and retrieving the capture application 81 from the projector 100. The control device 10 can acquire the capture application 81 from the projector 100.

[0116] Third Embodiment The third embodiment shows the system configuration and control flow of the third projection system 1c, which is an example of projection system 1. The third projection system 1c is an example of projection system 1 in which the control device 10 has a server function unit 35. The server function unit 35 has server functions similar to the cloud server 300 shown in the first embodiment. The control device 10 performs mirroring by transmitting shared data to the projector 100, thereby projecting the display screen DS displayed on the display 15 onto the projector 100. In the third projection system 1c, the server function unit 35 provides the functions of the cloud server 300 shown in Figure 4. In the third projection system 1c, the projector 100 does not have a projector server function unit 133.

[0117] Figure 9 shows the system configuration of projection system 1. Figure 9 schematically shows the system configuration of the third projection system 1c, which is an example of projection system 1. Figure 9 shows the control device 10 and the projector 100. The configuration of the projector 100 shown in Figure 9 is the same as the configuration of the projector 100 shown in Figure 4.

[0118] In the control device 10 shown in Figure 9, the server function unit 35 functions as a virtual server. The server function unit 35 operates as a virtual web server 35a and a virtual WebSocket server 35b. The virtual web server 35a performs the same operations as the web server 310 shown in Figure 4. The virtual WebSocket server 35b performs the same operations as the WebSocket server 320 shown in Figure 4.

[0119] Figure 10 shows the control flow performed in projection system 1. Figure 10 shows the control flow performed in third projection system 1c, which is an example of projection system 1. Figure 10 shows the control flow performed in control device 10 and projector 100. By executing the control flow, control device 10 performs mirroring, which projects the display screen DS displayed on display 15 onto projector 100.

[0120] In step S301, the control device 10 starts the projection application 91. The control device 10 starts the projection application 91 when it receives a predetermined operation from the user. The projection application 91 starts on the virtual device 90. The projection application 91 runs in the virtual environment.

[0121] After starting the projection application 91, the control device 10 establishes a communication connection with the projector 100 in step S302. The projection application 91 sends control signals such as connection requests to the projector 100 via the TCP / IP communication path TP, thereby establishing a communication connection with the projector 100.

[0122] After the projection application 91 is launched, the projector 100 establishes a communication connection with the control device 10 in step S331. The projector 100 receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The projector 100 transmits various data such as model information and establishes a communication connection with the control device 10. Once the projector 100 has established a communication connection with the control device 10, it becomes possible to receive various data from the control device 10.

[0123] After establishing a communication connection with the projector 100, the control device 10 receives a mirroring instruction in step S303. When the projection application 91 is launched, it displays the UI screen 170 on the display 15. When the user performs a predetermined operation on the mirroring reception icon 172 included in the UI screen 170 displayed on the display 15, the mirroring reception icon 172 receives the mirroring instruction. Upon receiving the mirroring instruction, the projection application 91 generates various control signals, such as a mirroring control signal, corresponding to the mirroring instruction.

[0124] After receiving the mirroring instruction, the control device 10 sends a startup instruction to the control unit 30 in step S304. The projection application 91 sends the startup instruction to the browser control unit 31 of the control unit 30. The browser control unit 31 receives the startup instruction.

[0125] When the projection application 91 sends a startup command to the browser control unit 31, the control device 10 starts the web browser 80 in step S311. If the web browser 80 is not running, the browser control unit 31 of the control unit 30 starts the web browser 80 based on the startup command. The web browser 80 starts up based on the startup command and displays a web page tab. If the web browser 80 is running when the browser control unit 31 receives the startup command, it displays a new web page tab.

[0126] After receiving the mirroring instruction, the control device 10 sends a web page request to the server function unit 35 in step S305. The projection application 91 sends a web page request to the virtual web server 35a. The web page request includes the page address of the web page on which the capture application 81 operates. The page address is stored in the storage unit 40 when the projection application 91 is installed in the control device 10.

[0127] When the projection application 91 sends a web page request, the control device 10 receives the web page request in step S321. The virtual WebSocket server 35b operating within the control device 10 receives the web page request sent from the projection application 91. The virtual WebSocket server 35b sends the web page request to the virtual web server 35a. The virtual web server 35a receives the web page request.

[0128] When the control device 10 receives a web page request, in step S322, it sends web page information to the web browser 80. The virtual web server 35a retrieves the web page information stored in the storage unit 40 based on the page address included in the web page request. The web page information is information relating to the web page on which the capture application 81 is running. The virtual web server 35a sends the web page information to the web browser 80.

[0129] When the control device 10 transmits web page information, it receives the web page information in step S312. When the web browser 80 receives the web page information, it displays the web page on the web page tab in which the capture application 81 will run based on the web page information. The web browser 80 then makes the capture application 81 executable.

[0130] After receiving the web page information, the control device 10 executes the capture application 81 in step S313. The web browser 80 executes the capture application 81 on the web page tab.

[0131] The control device 10 executes the capture application 81 and then, in step S314, causes it to acquire shared data. The web browser 80 causes the capture application 81 to acquire shared data. The capture application 81 captures the generated data output from the output unit 20 of the control device 10 as captured data. The generated data includes at least one of the video data output from the display data output unit 21 and the audio data output from the audio output unit 23. The captured data is an example of shared data that includes at least one of the video data and audio data.

[0132] When the control device 10 acquires shared data, in step S315, it sends the first SDP information from the web browser 80 to the virtual WebSocket server 35b. When the server function unit 35 acquires the first SDP information, in step S324, it sends the first SDP information from the virtual WebSocket server 35b to the projection application 91. When the projection application 91 acquires the first SDP information, it sends the first SDP information to the projector 100 in step S307. When the projector 100 acquires the first SDP information, it registers the first SDP information in the projector memory unit 140 in step S333. In step S334, the projector 100 sends the second SDP information to the projection application 91. When the projection application 91 acquires the second SDP information, it sends the second SDP information to the virtual WebSocket server 35b in step S309. When the server function unit 35 acquires the second SDP information, in step S326, it sends the second SDP information from the virtual WebSocket server 35b to the web browser 80. When the web browser 80 acquires the second SDP information, in step S317, it registers the second SDP information in the storage unit 40. The web browser 80 then establishes WebRTC communication with the projector 100. When the control device 10 establishes WebRTC communication with the projector 100, in step S319, it sends the shared data to the projector 100. The control device 10 sends the shared data acquired by the capture application 81 via the stream distribution path SP to the projector communication unit 150 of the projector 100. The control device 10 sends the shared data to the projector communication unit 150 in real time via the stream distribution path SP. By sending the shared data acquired by the capture application 81 to the projector 100 via the stream distribution path SP, the control device 10 can send the shared data to the projector 100 while suppressing a decrease in the quality of the shared data. The projection application 91 can send the shared data to the projector 100 using the functions of the capture application 81.

[0133] When the control device 10 transmits shared data, the projector 100 receives the shared data in step S336. The stream communication unit 151 of the projector communication unit 150 receives the shared data via the stream distribution path SP.

[0134] After receiving the shared data, the projector 100 performs mirroring in step S337. If the shared data includes video data, the projector 100 acquires the video data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the video data to the projection unit 110. The projection control unit 131 causes the projection unit 110 to project the video data. The projection unit 110 projects the projected image PG based on the video data onto the projection surface SC. If the shared data includes audio data, the projector 100 acquires the audio data included in the shared data. The projection control unit 131 of the projector control unit 130 transmits the audio data to the projector's built-in speaker 120. The projector's built-in speaker 120 outputs audio based on the audio data. The projector 100 performs mirroring based on the shared data which includes at least one of the video data and the audio data.

[0135] If the control device 10 has server functionality, the control flow preferably includes storing the capture application 81 in the storage unit 40 and retrieving the capture application 81 from the storage unit 40. The control device 10 can acquire the capture application 81 without connecting to a cloud server 300 or the like.

[0136] A summary of this disclosure is provided below.

[0137] Note 1 An information processing method comprising: communicating with a projection device; storing an application program for controlling the projection device; executing the application program; instructing the application program to launch a web application program on a web browser; instructing the application program to transmit control signals to the projection device via a signal communication path; and instructing the web application program to acquire output data including at least one of video data and audio data output from an output unit. The information processing device can minimize problems such as degradation of shared data quality by using a web application program that runs on a web browser to acquire output data. The application program can receive output data without having its own function to acquire the output data.

[0138] Note 2 The information processing method described in Appendix 1 includes causing the web application program to transmit the output data to the projection device via a browser communication path different from the signal communication path. By transmitting output data via a browser communication path, the information processing device can minimize the degradation of output data when transmitting it to the projection device.

[0139] Note 3 The information processing method described in Appendix 1 includes causing the application program to acquire the output data from the web application program, and causing the application program to transfer the output data to the projection device via the signal communication path. The information processing device can transmit output data to a projection device that does not support WebRTC technology.

[0140] Note 4 The information processing method described in any one of the appendices 1 to 3 includes, if the projection device has a server function, storing the web application program in the projection device and retrieving the web application program from the projection device. The information processing device can retrieve web application programs from the projection device.

[0141] Note 5 The information processing method described in any one of the appendices 1 to 3 includes, if the server has a server function, storing the web application program in the storage unit and retrieving the web application program from the storage unit. The information processing device can acquire web application programs without connecting to an external server or similar device.

[0142] Note 6 The information processing method described in any one of the appendices 1 to 5, wherein the output data is capture data obtained by capturing at least one of the video data and audio data output from the output unit. The information processing device can cause the projection device to perform mirroring based on the output data. [Explanation of symbols]

[0143] 1…Projection system, 1a…First projection system, 1b…Second projection system, 1c…Third projection system, 10…Control device, 15…Display, 20…Output unit, 21…Display data output unit, 23…Audio output unit, 30…Control unit, 31…Browser control unit, 33…Application control unit, 35…Server function unit, 35a…Virtual web server, 35b…Virtual WebSocket server, 35a…Virtual web server, 40…Storage unit, 50…Communication unit, 70…Dedicated operating system TEM, 80…Web browser, 81…Capture app, 90…Virtual device, 91…Projection app, 100…Projector, 110…Projection unit, 111…Light source, 111a…Light source unit, 111b…Reflector, 113…Liquid crystal light bulb, 113a…Pixel area, 113B…Liquid crystal light bulb for blue light, 113G…Liquid crystal light bulb for green light, 113R…Liquid crystal light bulb for red light, 115…Light bulb drive unit, 117…Projection lens, 120…Projector built-in speaker, 13 0...Projector control unit, 131...Projection control unit, 133...Projector server function unit, 133a...Web server function unit, 133b...WebSocket server function unit, 140...Projector memory unit, 150...Projector communication unit, 151...Stream communication unit, 153...Signal communication unit, 170...UI screen, 171...Projector name display unit, 172...Mirroring reception icon, 173...Photo display reception icon, 174...Document display reception icon, 17 5...Camera image display request icon, 176...Remote control function request icon, 177...Multi-projection request icon, 178...Disconnection request icon, 179...End request icon, 200...Router, 300...Cloud server, 310...Web server, 320...WebSocket server, 330...Server storage device, 340...Server communication device, DS...Display screen, NW...Communication network, PG...Projected image, PL...Pixel, SC...Projection surface, SP...Stream distribution path, TP...TCP / IP communication path.

Claims

1. Connecting to the projection device via communication, The application program that controls the projection device is stored, Executing the aforementioned application program, The aforementioned application program is capable of launching a web application program on a web browser, The application program is to transmit a control signal to the projection device via a signal communication path, The web application program is instructed to acquire output data that includes at least one of the video data and audio data output from the output unit. Information processing methods including

2. The web application program is instructed to transmit the output data to the projection device via a browser communication path different from the signal communication path. The information processing method according to claim 1, including the following:

3. The application program is instructed to obtain the output data from the web application program. The application program is to transfer the output data to the projection device via the signal communication path, The information processing method according to claim 1, including the following:

4. If the projection device has server functionality, The projection device stores the web application program, To obtain the web application program from the projection device, The information processing method according to claim 1, including the following:

5. If it has server functionality, the web application program is stored in the memory unit, To retrieve the web application program from the storage unit, The information processing method according to claim 1, including the following:

6. The output data is capture data obtained by capturing at least one of the video data and audio data output from the output unit. The information processing method according to claim 1.

Images