Information processing method of information processing apparatus and display system

Abstract

Information processing method of information processing apparatus and display system. Processing of a display screen of a control device when the control device is mirrored by a projector is not considered. An information processing method of an information processing apparatus includes: receiving a display operation of causing a web browser to display a setting page that sets a projection device; based on the display operation, performing display processing of displaying a setting window that includes the setting page on a display screen in a first size; receiving, in the setting page, a projection instruction operation of causing the projection device to project the display screen; and based on the projection instruction operation, performing window display processing of displaying a retrieval window that retrieves display data for causing the projection device to project the display screen in a second size smaller than the first size, and data transmission processing of transmitting the display data to the projection device via the retrieval window.

Description

Technical Field

[0001] This disclosure relates to information processing methods and display systems for information processing devices. Background Technology

[0002] Image display devices that connect to control devices such as computers are known. The projector described in Patent Document 1 is an example of such an image display device. The projector displays an input image based on image information supplied via a connection terminal. The projector is capable of displaying using Miracast. Miracast is a registered trademark. Miracast is a display transmission technology based on wireless communication. The projector can mirror the display screen of the control device using Miracast. Miracast is an example of a mirroring technique.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2020-64131 Summary of the Invention

[0006] However, the processing of the control device's display screen when the projector mirrors the control device's display screen is not considered.

[0007] The information processing method of the information processing apparatus disclosed herein includes: establishing a communication connection with a projection device; receiving a display operation that causes a web browser to display a setting page for setting the projection device; performing display processing based on the display operation to display a setting window containing the setting page on a display screen at a first size; receiving a projection instruction operation in the setting page that causes the projection device to project the display screen; and performing window display processing and data transmission processing based on the projection instruction operation, wherein in the window display processing, an acquisition window for acquiring display data is displayed at a second size smaller than the first size, the display data causing the display screen to be displayed, and in the data transmission processing, the display data is transmitted to the projection device via the acquisition window.

[0008] The display system disclosed herein includes a projection device and an information processing device for operation by a web browser. The information processing device performs the following processes: establishing a communication connection with the projection device; receiving a display operation that causes the web browser to display a setting page for setting the projection device; based on the display operation, performing display processing to display a setting window containing the setting page at a first size on a display screen; receiving a projection instruction operation in the setting page that causes the projection device to project the display screen; performing window display processing and data transmission processing based on the projection instruction operation; in the window display processing, displaying a window for obtaining display data at a second size smaller than the first size, the display data being used to display the display screen; in the data transmission processing, sending the display data to the projection device via the window; and sending the display data to the projection device. The projection device performs the following processes: receiving the display data from the information processing device; and projecting the display screen based on the display data. Attached Figure Description

[0009] Figure 1 It is a diagram showing the general structure of the projection system.

[0010] Figure 2 This is a diagram showing the modular structure of the projection system.

[0011] Figure 3 It is a diagram showing the general structure of the projection unit.

[0012] Figure 4 It is a diagram showing the system structure of the projection system.

[0013] Figure 5 This is an example of a displayed screen.

[0014] Figure 6 It is a diagram that shows the control flow executed by the control device.

[0015] Figure 7 This is an example of a displayed screen.

[0016] Figure 8 This is an example of a displayed screen.

[0017] Figure 9 This is an example of a displayed screen.

[0018] Figure 10 It is a diagram representing a projected image projected onto a projection surface.

[0019] Figure 11 It is a diagram that shows the control flow executed by the control device.

[0020] Figure 12 This is an example of a displayed screen.

[0021] Figure 13 This is an example of a displayed screen.

[0022] Figure 14 This is an example of a displayed screen.

[0023] Figure 15 This is an example of a displayed screen.

[0024] Figure 16 It is a diagram representing a projected image projected onto a projection surface.

[0025] Explanation of reference numerals in the attached figures

[0026] 1… Projection system, 10… Control device, 15… Display, 17… Input unit, 20… Output unit, 21… Display data output unit, 23… Sound output unit, 30… Control unit, 31… Browser control unit, 33… Application control unit, 40… Storage unit, 50… Communication unit, 70… Dedicated operating system, 80… Web browser, 81… Projection setting page, 81a… Mirror start icon, 81w… Projection setting page window, 83… Data acquisition page, 83w… Data acquisition page window, 84… Capture application, 90…Virtual device, 91…Projection application, 100…Projector, 110…Projection unit, 111…Light source, 111a…Light source section, 111b…Reflector, 113…Liquid crystal light valve, 113a…Pixel area, 113B…Liquid crystal light valve for blue light, 113G…Liquid crystal light valve for green light, 113R…Liquid crystal light valve for red light, 115…Light valve drive section, 117…Projection lens, 120…Projector built-in speaker, 130…Projector control unit, 131…Projection control section, 140…Projector storage unit, 1 50…Projector communication unit, 151…Streaming communication unit, 153…Signal communication unit, 170…UI screen, 171…Projector name display unit, 172…Mirroring acceptance icon, 172a…Switch icon, 173…Photo display acceptance icon, 174…Document display acceptance icon, 175…Camera image display acceptance icon, 176…Remote control function acceptance icon, 177…Multiple projection acceptance icon, 178…Cut off acceptance icon, 179…End acceptance icon, 180…Select screen, 181…Select image, 181… a… First selected image, 183… Share icon, 185… Share cancel icon, 200… Router, 300… Cloud server, 310… Network server, 320… WebSocket server, 330… Server storage device, 330a… Server storage unit, 340… Server communication device, BW… Browser window, DS… Display screen, NW… Communication network, PG… Projected image, PL… Pixel, SC… Projection surface, SP… Streaming path, TP… TCP / IP communication path, TW… Tab window. Detailed Implementation

[0027] Figure 1 The schematic structure of projection system 1 is shown. Projection system 1 includes a control device 10, a projector 100, a router 200, and a cloud server 300. The projector 100 projects a projected image PG onto the projection surface SC. The cloud server 300 is communicatively connected to the control device 10 via a communication network NW. Figure 1 The projection system 1 shown has a control device 10 and a projector 100, but is not limited thereto. The projection system 1 may also have multiple control devices 10 and multiple projectors 100. The projection system 1 corresponds to an example of a display system.

[0028] The control device 10 is communicatively connected to the projector 100. The control device 10 sends projected image data to the projector 100. The control device 10 executes actions to cause the projector 100 to project a display screen DS onto the monitor 15 as a mirror image of the projected image PG. The control device 10 may have functions such as adjusting the shape of the projected image PG projected by the projector 100 onto the projection surface SC. The control device 10 corresponds to an example of an information processing device. The control device 10 is a tablet terminal, mobile computer, desktop computer, etc. Figure 1 The mobile computer equipped with a display 15 and an input unit 17 is referred to as a control device 10.

[0029] The control device 10 executes various programs, such as a web browser 80, on a dedicated operating system 70. The dedicated operating system 70 is, for example, ChromeOS. Chrome is a registered trademark. The dedicated operating system 70 is an example of an operating system. The dedicated operating system 70 can construct a virtual environment. In the virtual environment, programs that run on an operating system different from the dedicated operating system 70 can run. An operating system different from the dedicated operating system 70 is, for example, AndroidOS. Android is a registered trademark. Programs that run on an operating system different from the dedicated operating system 70 are, for example, Android applications.

[0030] The display 15 displays images and videos. The display 15 is composed of a liquid crystal panel, an organic EL (electro-luminescence) panel, etc. The display 15 may have a touch input function to accept user input operations.

[0031] Input unit 17 accepts user input operations. Figure 1 The input unit 17 shown represents a keyboard, but is not limited to it. The input unit 17 may consist of a mouse, touchpad, etc. When the display 15 has touch input functionality, the display 15 functions as the input unit 17.

[0032] The projector 100 is positioned opposite the projection surface SC. The projector 100 corresponds to an example of a projection device. The projector 100 is communicatively connected to the control device 10. Figure 1The projector 100 shown communicates with the control device 10 via router 200, but the projector 100 can also communicate directly with the control device 10. The projector 100 can also communicate with external devices other than the control device 10. The projector 100 receives projected image data from the control device 10. The projector 100 projects a projected image PG onto the projection surface SC based on the projected image data. The projector 100 can also mirror the display screen DS shown on the monitor 15 of the control device 10 based on the projected image data. The projector 100 can also project the projected image PG onto the projection surface SC based on pre-stored OSD data. OSD is an abbreviation for Screen Display.

[0033] The projection surface SC displays the projected image PG projected from the projector 100. Figure 1 The projection surface SC shown is formed by a screen, but is not limited to this. The projection surface SC can also be an indoor wall, ceiling, building exterior wall, etc. The shape of the projection surface SC is not limited to a plane, but can also be a curved surface, a surface with concave and convex features, a sphere, or other three-dimensional shapes.

[0034] Router 200 is a communication device that connects control device 10, projector 100, and cloud server 300 to each other. Router 200 connects control device 10 and cloud server 300 via communication network NW. Router 200 connects control device 10 and projector 100 according to a defined communication protocol. Router 200 can also connect projector 100 and cloud server 300.

[0035] The communication network NW connects the cloud server 300 to the control device 10 and other devices. As an example, the communication network NW is a WAN (Wide Area Network).

[0036] A cloud server 300 is a virtual server that operates in a cloud computing environment. A cloud server 300 is a service that utilizes one or more virtualized servers via the Internet. Figure 1 The cloud server 300 shown includes a network server 310 and a WebSocket server 320.

[0037] The network server 310 provides HTML and object display to the web browser 80 of the client software. Figure 1 A virtual network server 310 is displayed. Based on instructions from the control device 10, the network server 310 provides web applications to the web browser 80 of the control device 10. When the control device 10 or the like pre-stores web applications, the projection system 1 may not need to have a network server 310.

[0038] The WebSocket server 320 is a server that enables data communication between web applications on a web browser 80 according to the WebSocket standard. Figure 1 The WebSocket server 320 is virtually shown. The WebSocket server 320 is used as a signaling server for SDP switching when the controller 10 uses WebRTC (Web Real-Time Communication) technology to stream shared data, etc. SDP is an abbreviation for Session Description Protocol.

[0039] Figure 2 The module structure of projection system 1 is shown. Figure 2 The module structure of the projection system 1, including the control device 10, the projector 100, and the cloud server 300, is shown. Figure 2 The projection system 1 is shown with router 200 omitted.

[0040] The control device 10 includes a display 15, an input unit 17, an output unit 20, a control unit 30, a storage unit 40, and a communication unit 50. The control device 10 also includes a built-in speaker (not shown).

[0041] The display 15 displays the display screen DS based on the control of the control unit 30. The display 15 is connected to the display data output unit 21 of the output unit 20. The display data output unit 21 will be described later. Figure 2 The display 15 shown is disposed on the control device 10, but is not limited thereto. The display 15 may also be an external display panel or the like disposed on the control device 10.

[0042] The input unit 17 accepts various input operations from the user. The input unit 17 generates an input signal corresponding to the input operation and outputs the input signal to the control unit 30. Figure 2 The input unit 17 shown is provided in the control device 10, but is not limited thereto. The input unit 17 may also be an input device external to the control device 10.

[0043] The output unit 20 outputs various generated data generated by the control unit 30, etc., to the display 15, etc. The output unit 20 is composed of various circuits, output ports, etc. The output unit 20 corresponds to an example of an output section. The output unit 20 has a display data output section 21 and a sound output section 23.

[0044] The display data output unit 21 outputs image data of the displayed image to the display 15. The image data includes image data of the displayed still image. The image data is included in the generated data. The display data output unit 21 can also output image data to the display panel externally mounted on the control device 10.

[0045] The sound output unit 23 outputs sound data related to sound. The sound data is included in the generated data. The sound output unit 23 is connected to a built-in speaker or an external speaker (not shown). The sound output unit 23 outputs sound data via wired or wireless means.

[0046] The control unit 30 is a device control controller that controls the various parts of the control device 10. For example, the control unit 30 is a processor with a CPU (Central Processing Unit). The control unit 30 is composed of one or more processors. A dedicated operating system 70 runs in the control unit 30. The control unit 30 executes various programs on the dedicated operating system 70. The control unit 30 functions as various functional units by executing these programs. For example, the control unit 30 functions as a browser control unit 31 and an application control unit 33. The control unit 30 can also function as a functional unit other than the browser control unit 31 and the application control unit 33.

[0047] The browser control unit 31 enables the web browser 80 to run on the dedicated operating system 70 and display various web pages. The browser control unit 31 also enables web applications to run on the web browser 80 and execute their functions. For example, the browser control unit 31 enables a capture application 84, which is a web application, to run on the web browser 80 and acquire the generated data output by the output unit 20. Details of the capture application 84 will be described later.

[0048] Application control unit 33 enables applications to run on dedicated operating system 70 or in a virtual environment. Application control unit 33 enables applications with specifications corresponding to dedicated operating system 70 to run on dedicated operating system 70. Application control unit 33 also enables applications with specifications different from those corresponding to dedicated operating system 70 to run in a virtual environment. The virtual environment is built on virtual device 90 running on dedicated operating system 70. Virtual device 90 will be described later.

[0049] Storage unit 40 stores various programs and data. Storage unit 40 stores device control programs, various network applications, and various other applications. Storage unit 40 stores various data generated by control unit 30. Storage unit 40 is composed of volatile semiconductor memory such as RAM (Random Access Memory), non-volatile memory such as ROM (Read Only Memory), and flash memory. Storage unit 40 can also function as the working area of ​​control unit 30. Storage unit 40 corresponds to one example of a storage section.

[0050] The communication unit 50 is a communication interface circuit that communicates with the projector 100, cloud server 300, etc. The communication unit 50 wirelessly connects to the projector 100, etc., according to a specified communication protocol. The communication unit 50 includes a wireless communication port, such as a wireless LAN (Local Area Network) communication port. The communication unit 50 can also be wired to the projector 100, etc. The communication unit 50 has a wired connector such as a USB (Universal Serial Bus). The communication unit 50 sends various control signals and projected image data to the projector 100. The projected image data includes video data and audio data. The communication unit 50 receives various data and signals from the cloud server 300, etc.

[0051] The projector 100 includes a projection unit 110, a built-in speaker 120, a control unit 130, a storage unit 140, and a communication unit 150.

[0052] The projection unit 110 amplifies and projects an image based on the projected image data onto the projection surface SC. The projection unit 110 projects a projected image PG onto the projection surface SC under the control of the projector control unit 130. The projection unit 110 projects dynamic images, static images, OSD images, etc., as the projected image PG based on image data contained in the projected image data. OSD images are images representing various settings, operating information, etc., related to the projector 100.

[0053] Figure 3 This shows a schematic structure of the projection unit 110. Figure 3 An example of a projection unit 110 is shown. The projection unit 110 includes a light source 111, three liquid crystal light valves 113, a light valve driving unit 115, and a projection lens 117.

[0054] Light source 111 emits light into liquid crystal light valve 113. Light source 111 includes a light source unit 111a, a reflector 111b, an integrating optical system (not shown), and a dichroic optical system (not shown). Light source unit 111a emits light. Light source unit 111a is composed of a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, etc. Light source unit 111a emits light under the control of projector control unit 130. Reflector 111b reduces the deviation in the emission direction of the light emitted from light source unit 111a. The integrating optical system reduces the brightness distribution deviation of the light emitted from light source unit 111a. Light passing through reflector 111b is incident on the dichroic optical system. The dichroic optical system separates the incident light into red, green, and blue light components.

[0055] The liquid crystal light valve 113 modulates the light emitted from the light source 111. The liquid crystal light valve 113 generates a projected image PG, etc., by modulating the light. The liquid crystal light valve 113 is constructed from a transmissive liquid crystal panel, etc., in which liquid crystal is sealed between a pair of transparent substrates. The liquid crystal light valve 113 has a rectangular pixel region 113a containing a plurality of pixels PL arranged in a matrix. In the liquid crystal light valve 113, a driving voltage is applied to the liquid crystal according to each pixel PL. Figure 3 The projection unit 110 shown has three liquid crystal light valves 113. The projector 100 is configured to use a transmissive liquid crystal panel with liquid crystal light valves 113, but is not limited thereto. The projector 100 may have a reflective liquid crystal panel or an LCOS (Liquid Crystal On Silicon) liquid crystal panel. The projector 100 may also have a structure with more than one DMD (Digital Mirror Device).

[0056] The three liquid crystal light valves 113 are a red light liquid crystal light valve 113R, a green light liquid crystal light valve 113G, and a blue light liquid crystal light valve 113B. The red light component separated by the color separation optical system is incident on the red light liquid crystal light valve 113R. The green light component separated by the color separation optical system is incident on the green light liquid crystal light valve 113G. The blue light component separated by the color separation optical system is incident on the blue light liquid crystal light valve 113B.

[0057] The light valve driving unit 115 applies a driving voltage to each pixel PL based on the projected image data received from the projector control unit 130. The light valve driving unit 115 is, for example, a control circuit. The driving voltage is supplied by a driving source (not shown). The light valve driving unit 115 may also apply a driving voltage to each pixel PL based on the projected image data corrected by the projection control unit 131 described later. When the light valve driving unit 115 applies a driving voltage to each pixel PL, each pixel PL is set to a light transmittance based on the projected image data. Light emitted from the light source 111 is modulated through the pixel region 113a. The three liquid crystal light valves 113 form a color composition image of each color light.

[0058] The projection lens 117 synthesizes and magnifies the images of each color component formed by the liquid crystal light valve 113 and projects them. The projection lens 117 projects the projected image PG onto the projection surface SC. The projected image PG is a multicolor image synthesized from the images of each color component.

[0059] Figure 2 The projector's built-in speaker 120, as shown, outputs sound based on sound data contained in the projected image data. The projector's built-in speaker 120 receives sound data from the projector control unit 130 and outputs sound corresponding to the sound data. Figure 2 The projector 100 shown has a built-in speaker 120, but is not limited thereto. The projector 100 may also have an external speaker. The external speaker outputs sound based on sound data.

[0060] The projector control unit 130 is a projector control controller that controls the various units of the projector 100. As an example, the projector control unit 130 is a processor with a CPU. The projector control unit 130 is composed of one or more processors. The projector control unit 130 functions as a projection control unit 131 by executing firmware. The projector control unit 130 can also function as a functional unit other than the projection control unit 131.

[0061] The projection control unit 131 controls the projection of the projection unit 110. Based on the image data contained in the projection image data sent from the control device 10, the projection control unit 131 causes the projection unit 110 to project a projection image PG. Based on the sound data contained in the projection image data, the projection control unit 131 causes the projector's built-in speaker 120 to output sound. The projection control unit 131 synchronizes the projection image PG projected from the projection unit 110 with the sound output from the projector's built-in speaker 120. The projection control unit 131 performs various correction processes on the image data, such as keystone correction.

[0062] The projector storage unit 140 stores firmware and various data. It stores projected image data sent from the control device 10, as well as correction data for correcting image data contained within the projected image data. The projector storage unit 140 is composed of volatile semiconductor memory such as RAM, and non-volatile memory such as ROM and flash memory.

[0063] The projector communication unit 150 is a communication interface circuit for communicating with the control device 10, external devices, etc. The projector communication unit 150 wirelessly communicates with the control device 10 and other devices according to a specified communication protocol. The projector communication unit 150 includes a wireless communication port, such as a wireless LAN communication port. The projector communication unit 150 can also be wired to communicate with the control device 10 and other devices. The projector communication unit 150 has a wired connector such as a USB connector. The projector communication unit 150 receives projected image data, control signals, etc., from the control device 10. The projector communication unit 150 sends various data, such as projector setting information, to the control device 10 and other devices.

[0064] The cloud server 300 includes a network server 310, a WebSocket server 320, a server storage device 330, and a server communication device 340. The network server 310, the WebSocket server 320, the server storage device 330, and the server communication device 340 are configured in one or more devices.

[0065] Server storage device 330 stores various programs and data. Server storage device 330 stores web pages, network applications, etc., provided from various network servers 310 to control devices 10, etc. Server storage device 330 is composed of magnetic recording media such as HDD (Hard Disk Drive). Server storage device 330 may also be composed of volatile semiconductor memory such as RAM, or non-volatile memory such as ROM and flash memory.

[0066] Server communication device 340 is a communication interface circuit that communicates with control device 10, etc. Server communication device 340 communicates with control device 10, etc., via wired or wireless means according to a specified communication protocol. Server communication device 340 includes a wired connector, a wireless communication port, etc. The wired connector is a USB connector, a LAN connector, etc. The wireless communication port is a Wi-Fi communication port, a Bluetooth communication port, etc. Wi-Fi and Bluetooth are registered trademarks. Server communication device 340 sends network applications, etc., to control device 10. Server communication device 340 receives various signals sent from control device 10.

[0067] Figure 4 The system structure of projection system 1 is shown. Figure 4 The control device 10, projector 100, and cloud server 300 are shown.

[0068] A dedicated operating system 70 runs in the control device 10. The dedicated operating system 70 is, for example, ChromeOS. The control unit 30 of the control device 10 enables the dedicated operating system 70 to run. Applications corresponding to the dedicated operating system 70 can run on the dedicated operating system 70. Figure 4 On the dedicated operating system 70 shown, the web browser 80 and the virtual device 90 run. Applications other than the web browser 80 and the virtual device 90 can also run on the dedicated operating system 70.

[0069] Web browser 80 is software that connects to web server 310. Web browser 80 runs on dedicated operating system 70. Web browser 80 is, for example, Chrome. Web browser 80 displays various web pages in browser window BW, which includes tab window TW. Web browser 80 retrieves web pages from web server 310. Web pages may include web applications running within the web pages. Web browser 80 displays projection settings page 81 and data retrieval page 83 in browser window BW or tab window TW. Capture application 84 is launched within data retrieval page 83.

[0070] The projection settings page 81 accepts a mirror start instruction operation. When the user performs a mirror start instruction operation on the projection settings page 81, the projection settings page 81 configures the projector 100 to perform mirroring. The projection settings page 81 is an example of a webpage. The web browser 80 obtains projection settings page information related to the projection settings page 81 from the web server 310. Based on the projection settings page information, the web browser 80 displays the projection settings page 81 in the browser window (BW) or tab window (TW). The projection settings page 81 corresponds to an example of a settings page.

[0071] Data acquisition page 83 causes capture application 84 to run and acquire shared data. Data acquisition page 83 is an example of a webpage. Web browser 80 obtains data acquisition page information related to data acquisition page 83 from web server 310. Based on the data acquisition page information, web browser 80 displays data acquisition page 83 in browser window (BW) or tab window (TW). Capture application 84 is launched on data acquisition page 83. Data acquisition page 83 causes capture application 84 to start and acquire shared data.

[0072] The capture application 84 is launched on the data acquisition page 83 displayed on the web browser 80. The web browser 80 obtains the data acquisition page 83 containing the capture application 84 from the web server 310. The capture application 84 acquires generated data output from the output unit 20 of the control device 10 as shared data. The generated data includes image data output from the display data output unit 21. The image data is the data that causes the display screen DS to be displayed on the monitor 15. The image data corresponds to an example of display data. The shared data may also include sound data output from the sound output unit 23. The shared data is sent to the projector 100 as projected image data via the streaming path SP. The capture application 84 may also send the shared data to the projection application 91. The capture application 84 corresponds to an example of a web application.

[0073] Virtual device 90 is a virtual device running on dedicated operating system 70. Virtual device 90 is an example of an emulator running on dedicated operating system 70. Virtual device 90 functions as a device that enables the running of an operating system different from dedicated operating system 70 on dedicated operating system 70. Virtual device 90 provides a virtual environment on dedicated operating system 70. Virtual device 90 is, for example, a device running Android OS. Applications running on a different operating system than dedicated operating system 70 can be launched on virtual device 90. Applications running on a different operating system than dedicated operating system 70 are Android applications. Figure 4 The projection application 91 shown is an example of an Android application.

[0074] Projection application 91 is software that controls projector 100. Projection application 91 is pre-stored in storage unit 40. Projection application 91 enables projector 100 to project various projected images PG. Projection application 91 sends various control signals to projector 100, cloud server 300, etc. Projection application 91 receives response signals from projector 100, cloud server 300, etc. Projection application 91 enables capture application 84 to acquire shared data. Projection application 91 enables capture application 84 to send the shared data as projected image data to projector 100. Projection application 91 can also enable capture application 84 to send shared data to projection application 91.

[0075] The projection application 91 controls the display of the web browser 80. The projection application 91 controls the display mode of the browser window BW or tab window TW that displays the projection settings page 81 and the data acquisition page 83. By controlling the display mode of the browser window BW or tab window TW, the projection application 91 enables the projector 100 to project the desired display image DS. The projection application 91 sends setting information, such as the control settings for the display mode of the browser window BW or tab window TW, to the web browser 80. The projection application 91 sends setting information, either directly or via the web server 310, to the web browser 80. Based on the setting information, the web browser 80 changes the display mode of the browser window BW or tab window TW.

[0076] The projection application 91, running on the virtual device 90, sends and receives various control signals to and from the dedicated operating system 70 via various APIs (Application Programming Interfaces). The APIs vary depending on the version of the operating system running on the virtual device 90. Depending on the version of the operating system running on the virtual device 90, the projection application 91 may sometimes have difficulty controlling the projector 100. The projection application 91 uses the functions of the web browser 80 and the cloud server 300 to send shared data to the projector 100 as projected image data.

[0077] Figure 5 An example of a display screen DS is shown. The display screen DS is displayed on monitor 15. Figure 5 The display screen DS shown illustrates the screen displayed when the control device 10 starts the projection application 91. The display screen DS includes the UI screen 170 of the projection application 91. UI is short for User Interface.

[0078] UI screen 170 displays multiple icons, etc. UI screen 170 includes projector name display 171, mirror acceptance icon 172, photo display acceptance icon 173, document display acceptance icon 174, camera image display acceptance icon 175, remote control function acceptance icon 176, multi-projection acceptance icon 177, cut-off acceptance icon 178, and end acceptance icon 179.

[0079] The projector name display unit 171 displays the projector name of the projector 100 connected to the control device 10. The projector name display unit 171 displays the projector name based on the model information of the projector 100 obtained from the projector 100.

[0080] The mirror acceptance icon 172 accepts the user's mirror instruction operation. When the user performs a specified operation on the mirror acceptance icon 172, the mirror acceptance icon 172 accepts the mirror instruction operation. When accepting the mirror instruction operation, the projection application 91 generates a mirror control signal, etc., corresponding to the mirror instruction operation. The mirror acceptance icon 172 includes a switch icon 172a.

[0081] The switch icon 172a indicates whether the projector 100 is performing mirroring. By checking the display of the switch icon 172a, the user can determine whether the projector 100 is performing mirroring. Figure 5 The switch icon 172a shown indicates that mirroring has not been performed. Switch icon 172a can accept mirroring instructions from the user.

[0082] The photo display acceptance icon 173 accepts the user's photo display instruction. When the user performs a specified operation on the photo display acceptance icon 173, the photo display acceptance icon 173 accepts the photo display instruction. When the photo display instruction is accepted, the projection application 91 performs photo display processing on the photo image stored in the projection control device 10 of the projector 100. The projection application 91 sends the photo image data for displaying the photo image to the projector 100, causing the projector 100 to project the photo image as the projected image PG.

[0083] The document display acceptance icon 174 accepts the user's document display instruction. When the user performs a specified operation on the document display acceptance icon 174, the document display acceptance icon 174 accepts the document display instruction. When the document display instruction is accepted, the projection application 91 performs document display processing on PDF files, etc., stored in the projection control device 10 of the projector 100. As an example, the projection application 91 sends PDF data for displaying a PDF image to the projector 100, causing the projector 100 to project the PDF image as the projected image PG.

[0084] The camera image display acceptance icon 175 accepts the user's camera image display instruction. When the control device 10 has a camera or other imaging device, the camera image display acceptance icon 175 can be operated. When the user performs a specified operation on the camera image display acceptance icon 175, the camera image display acceptance icon 175 accepts the camera image display instruction. When the camera image display instruction is accepted, the projection application 91 performs camera image display processing to cause the projector 100 to project the captured image from the imaging device. The projection application 91 sends the captured data for displaying the captured image to the projector 100, causing the projector 100 to project the captured image as the projected image PG.

[0085] The remote control function acceptance icon 176 accepts the user's remote control function action instructions. When the user performs a specified operation on the remote control function acceptance icon 176, the remote control function acceptance icon 176 accepts the remote control function action instruction. When the remote control function action instruction is accepted, the projection application 91 displays the remote control image representing the remote control operating the projector 100 on the projected image PG. When the user inputs an operation on the remote control image, the projector 100 can be controlled according to the input operation.

[0086] The multi-projection acceptance icon 177 accepts multiple projection instructions from the user. When multiple projectors 100 are connected to the control device 10, the multi-projection acceptance icon 177 can be operated. When the user performs a specified operation on the multi-projection acceptance icon 177, the multi-projection acceptance icon 177 accepts the multiple projection instructions. The projection application 91 performs multiple projections, causing the multiple projectors 100 connected to the control device 10 to project the projected image PG onto the projection surface SC.

[0087] The cut-off acceptance icon 178 accepts the user's cut-off instruction. When the user performs a prescribed operation on the cut-off acceptance icon 178, the cut-off acceptance icon 178 accepts the cut-off instruction. When the cut-off instruction is accepted, the projection application 91 cuts off the communication connection between the control device 10 and the projector 100.

[0088] The End Acceptance icon 179 accepts the user's application termination request. When the user performs a specified operation on the End Acceptance icon 179, the End Acceptance icon 179 accepts the application termination request. When the application termination request is accepted, the projected application 91 stops running. The UI screen 170 becomes non-displayable.

[0089] Figure 4 The streaming path SP shown is the path for real-time communication between the web browser 80 and the projector 100. As an example, the streaming path SP uses WebRTC technology for communication. The web browser 80 can send shared data to the projector 100 via the streaming path SP. Because the web browser 80 sends shared data to the projector 100 via the streaming path SP, the control device 10 can send the shared data to the projector 100 without degrading its quality. The streaming path SP corresponds to an example of a browser communication path.

[0090] The TCP / IP communication path TP is a communication path that uses TCP / IP (Transmission Control Protocol / Internet Protocol). The TCP / IP communication path TP and the streaming path SP are different communication paths. Projection application 91 sends various control signals to projector 100 via the TCP / IP communication path TP. Projection application 91 also receives control signals from projector 100 via the TCP / IP communication path TP. Projection application 91 can also send shared data to projector 100 via the TCP / IP communication path TP. When sending shared data to projector 100 via the TCP / IP communication path TP, quality degradation may occur in the shared data depending on the condition of the virtual device 90. It is preferable to send shared data to projector 100 via the streaming path SP. The TCP / IP communication path TP corresponds to an example of a signal communication path.

[0091] The projector communication unit 150 of the projector 100 has a streaming communication unit 151 and a signal communication unit 153.

[0092] The streaming communication unit 151 is connected to the capture application 84 via the streaming path SP. The streaming communication unit 151 corresponds to WebRTC technology. The streaming communication unit 151 receives shared data sent from the capture application 84 through real-time communication.

[0093] The signal communication unit 153 is communicatively connected to the projection application 91 via the TCP / IP communication path TP. The signal communication unit 153 receives control signals from the projection application 91. The signal communication unit 153 sends control signals generated by the projector 100 to the projection application 91. By transmitting and receiving control signals between the projection application 91 and the projector 100 through the signal communication unit 153, the projection application 91 can control the projector 100.

[0094] The cloud server 300 includes a network server 310 with a server storage unit 330a. The server storage unit 330a is part of the server storage device 330. The server storage unit 330a stores web page information related to various web pages. This web page information includes network application information related to various web applications. The network server 310 provides the web page information stored in the server storage unit 330a to the web browser 80.

[0095] WebSocket server 320 receives various control signals sent from projection application 91 via TCP / IP communication path TP. These control signals include webpage requests. For example, a webpage request is a signal instructing web server 310 to send a webpage containing a webpage from web browser 80. Upon receiving a webpage request, WebSocket server 320 sends the webpage request to web server 310. Based on the webpage request, web server 310 sends webpage information to web browser 80.

[0096] exist Figure 4 In the projection system 1 shown, the cloud server 300 includes a network server 310 that provides various web pages, but is not limited to these. Projection setting page information related to the projection setting page 81 and data acquisition page information related to the data acquisition page 83 can be pre-stored in the storage unit 40 of the control device 10 or the projector storage unit 140 of the projector 100. Projection setting page information and data acquisition page information are examples of web page information. When the projection setting page information, etc., is stored in the storage unit 40, the web browser 80 retrieves the projection setting page information, etc., from the storage unit 40. In this case, the control unit 30 has virtual server functionality equivalent to the cloud server 300. When the projection setting page information, etc., is stored in the projector storage unit 140, the web browser 80 retrieves the projection setting page information, etc., from the projector 100. In this case, the projector control unit 130 has virtual server functionality equivalent to the cloud server 300. When the projection setting page information, etc., is pre-stored in the storage unit 40 or the projector storage unit 140, the projection system 1 may not have a cloud server 300.

[0097] First Implementation Method

[0098] The first embodiment illustrates the control flow and each display screen DS when the web browser 80 has not been started before the projection application 91 receives the user's mirroring instruction. In the first embodiment, the projection application 91 starts the web browser 80. After the web browser 80 starts, the projection application 91 uses the web browser 80 to cause the projector 100 to mirror the display screen DS.

[0099] Figure 6 The control flow executed by projection system 1 is shown. Figure 6 The control flow executed by the control device 10, the projector 100, and the cloud server 300 is shown. The control flow executed by the control device 10 corresponds to an example of an information processing method. The control device 10 executes the control flow to cause the projector 100 to project a mirror image of the display screen DS displayed on the monitor 15.

[0100] Projection application 91 is launched in step S101. When the input unit 17 of the control device 10 receives a specified operation from the user, the control device 10 launches projection application 91. Projection application 91 is launched on virtual device 90. Projection application 91 executes in the virtual environment.

[0101] After startup, the projection application 91 establishes a communication connection with the projector 100 in step S102. The projection application 91 sends control signals such as connection requests to the projector 100 via the TCP / IP communication path TP to establish a communication connection with the projector 100.

[0102] After the projection application 91 is started, the projector 100 establishes a communication connection with the control device 10 in step S141. The projector 100 receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The projector 100 sends various data such as model information to establish a communication connection with the control device 10. When a communication connection with the control device 10 is established, the projector 100 can receive various data from the control device 10.

[0103] After establishing a communication connection with the projector 100, the projection application 91 accepts a mirroring instruction operation in step S103. When the projection application 91 starts, a UI screen 170 is displayed on the monitor 15. When the user performs a mirroring instruction operation on the mirroring acceptance icon 172 included in the UI screen 170 displayed on the monitor 15, the mirroring acceptance icon 172 accepts the mirroring instruction operation.

[0104] Figure 7 An example of a display screen DS is shown. The display screen DS is displayed on monitor 15. Figure 7 The displayed screen DS includes the UI screen 170 of the projection application 91. Figure 7 The displayed screen DS shows the screen when the user performs a mirror instruction operation on the mirror acceptance icon 172.

[0105] When a user performs a mirror instruction operation on the mirror acceptance icon 172, such as Figure 7 As shown, the display method of switch icon 172a has changed. Switch icon 172a is slid to the right. By confirming the position of switch icon 172a, the user can confirm that mirror acceptance icon 172 has accepted the mirror instruction operation.

[0106] When a mirroring instruction operation is received, the projection application 91 starts the web browser 80 in the steps following step S104, and performs projection setting page display processing to display the projection setting page 81 in the browser window BW of the web browser 80. The mirroring instruction operation corresponds to an example of a display operation.

[0107] like Figure 6 As shown, when a mirroring instruction operation is received, the projection application 91 sends a browser launch instruction to the control unit 30 in step S104. Based on the mirroring instruction operation, the projection application 91 sends the browser launch instruction to the browser control unit 31 of the control unit 30. The browser control unit 31 receives the browser launch instruction.

[0108] When the projection application 91 sends a browser launch instruction, the web browser 80 launches in step S121. When the browser launch instruction is received from the projection application 91, the browser control unit 31 executes a browser launch process to launch the web browser 80. This browser launch process corresponds to one example of the launch process. The web browser 80 launches based on the mirror instruction operation.

[0109] After startup, web browser 80 sends a projection settings page request to cloud server 300 in step S122. The projection settings page request is a request to display projection settings page 81 on web browser 80. The projection settings page request includes the address of projection settings page 81. By sending the projection settings page request, projection application 91 causes a projection settings page window 81w containing projection settings page 81 to be displayed in browser window BW of web browser 80. Projection settings page window 81w corresponds to an example of a settings window. Projection application 91 displays projection settings page window 81w in web browser 80 based on a mirroring instruction operation. Projection application 91 can send the projection settings page request directly to cloud server 300, or it can send it to cloud server 300 via web browser 80.

[0110] When a browser launch instruction is sent, the projection application 91 sends the projection settings page settings to the cloud server 300 in step S105. The projection settings page settings are the display settings when the web browser 80 displays the projection settings page window 81w. By sending the projection settings page settings to the cloud server 300, the projection application 91 specifies the display size of the browser window BW when the web browser 80 displays the projection settings page window 81w. By sending the projection settings page settings to the cloud server 300, the projection application 91 causes the projection settings page window 81w to be displayed at a first display size. The first display size corresponds to an example of a first size.

[0111] The projection application 91 may also choose not to send the projection settings page to the cloud server 300 when displaying the projection settings page 81 in the browser window BW that appears when the web browser 80 starts. The projection settings page request may also include the projection settings page. When the projection settings page request includes the projection settings page, step S105 is omitted.

[0112] When web browser 80 sends a projection settings page request, cloud server 300 receives the request in step S161. Web server 310, included in cloud server 300, receives the projection settings page request. Web server 310 obtains projection settings page information based on the projection settings page address contained in the request. The projection settings page information is what causes the projection settings page 81 to be displayed on web browser 80.

[0113] When the projection application 91 sends the projection settings page, the cloud server 300 receives the projection settings page information in step S162.

[0114] After receiving the projection settings page request and the projection settings page settings, the cloud server 300 sends the projection settings page information to the web browser 80 in step S163. The web server 310 included in the cloud server 300 sends the projection settings page information to the web browser 80. When the projection settings page request includes projection settings page settings, the web server 310 sends the projection settings page settings to the web browser 80.

[0115] When the cloud server 300 sends the projection settings page information, the web browser 80 receives the projection settings page information in step S123.

[0116] When projection setting page information is received, the web browser 80 displays the projection setting page 81 in step S124. Based on the projection setting page information, the browser control unit 31 performs projection setting page display processing, which displays the projection setting page window 81w containing the projection setting page 81. This projection setting page display processing corresponds to one example of the display processing. The web browser 80 displays the projection setting page window 81w at a first display size.

[0117] Figure 8 This represents an example of a display screen DS. The display screen DS is shown on monitor 15. Figure 8 The displayed screen DS shows the screen shown when the browser control unit 31 performs projection setting page display processing based on projection setting page information and projection setting page settings. Figure 8 The displayed screen DS shows the browser window BW.

[0118] Figure 8 The browser window BW shown is the projection settings page window 81w, which displays the projection settings page 81. The projection settings page 81 displayed in the projection settings page window 81w accepts input operations related to mirroring the display screen DS. The projection settings page window 81w contains a mirroring start icon 81a.

[0119] The mirror start icon 81a accepts user input. When the user performs a specified operation on the mirror start icon 81a, the projection settings page 81 accepts the mirror start instruction operation. The mirror start instruction operation is an instruction operation that causes the projector 100 to project the display screen DS. The mirror start instruction operation corresponds to an example of the projection instruction operation.

[0120] The projection settings page window 81w is displayed at a first display size. For example, the first display size is the full screen size of the monitor 15. The first display size is not limited to the full screen size of the monitor 15. The first display size is only required to display the mirrored start icon 81a; its size is not limited.

[0121] like Figure 6 As shown, after displaying the projection settings page 81, the web browser 80 accepts the mirror start indication operation in step S125. When the user performs a mirror start indication operation on the mirror start icon 81a, the projection settings page 81 accepts the mirror start indication operation. The web browser 80 sends the signal corresponding to the mirror start indication operation to the projection application 91.

[0122] When the image start instruction is accepted, the web browser 80 sends the image start instruction to the cloud server 300 in step S126.

[0123] When the web browser 80 sends a mirror start instruction, the cloud server 300 receives the mirror start instruction in step S164.

[0124] When a mirroring start instruction is received, the cloud server 300 sends a mirroring start instruction to the projection application 91 in step S165.

[0125] When the cloud server 300 sends a mirror start instruction, the projection application 91 receives the mirror start instruction in step S106.

[0126] Upon receiving the mirror start instruction, the projection application 91 sends a data retrieval page request and data retrieval page settings to the cloud server 300 in step S107. The data retrieval page request is a request to cause the web browser 80 to display the data retrieval page 83. The data retrieval page request includes the data retrieval page address of data retrieval page 83. By sending the data retrieval page request, the projection application 91 causes the data retrieval page window 83w, which contains data retrieval page 83, to be displayed in the web browser 80. The data retrieval page window 83w corresponds to an example of a retrieval window. The projection application 91 displays the data retrieval page window 83w in the web browser 80 based on the mirror start instruction operation.

[0127] The data retrieval page settings are the display settings when the web browser 80 displays the data retrieval page window 83w. The projection application 91 specifies the display size of the browser window BW when the web browser 80 displays the data retrieval page window 83w by sending the data retrieval page settings to the cloud server 300. The projection application 91 also causes the data retrieval page window 83w to be displayed at a second display size smaller than the first display size by sending the data retrieval page settings to the web browser 80. The second display size corresponds to an example of the second size.

[0128] When the projection application 91 sends a data retrieval page request and data retrieval page settings, the cloud server 300 receives the data retrieval page request and data retrieval page settings in step S166. The network server 310 included in the cloud server 300 also receives the data retrieval page request and data retrieval page settings. Based on the data retrieval page address contained in the data retrieval page request, the network server 310 retrieves the data retrieval page information. This data retrieval page information is what causes the web browser 80 to display the data retrieval page 83.

[0129] After receiving the data retrieval page request and the data retrieval page settings, the cloud server 300 sends the data retrieval page information to the web browser 80 in step S167. The web server 310 included in the cloud server 300 sends the data retrieval page information to the web browser 80.

[0130] When the cloud server 300 sends data to obtain page information, the web browser 80 receives data to obtain page information in step S127.

[0131] When data retrieval page information is received, the web browser 80 displays the data retrieval page 83 in step S128. Based on the data retrieval page information, the browser control unit 31 performs retrieval page display processing to display the data retrieval page window 83w containing the data retrieval page 83. This retrieval page display processing corresponds to an example of window display processing. Through this retrieval page display processing, the web browser 80 displays the data retrieval page window 83w at a second display size smaller than the first display size.

[0132] After displaying the data acquisition page 83, the web browser 80 displays a selection screen 180 on the monitor 15 in step S129. The selection screen 180 allows the user to select the display screen DS to be projected by the projector 100. The web browser 80 can change the display size of the projection settings page window 81w when displaying the selection screen 180.

[0133] Figure 9 This represents an example of a display screen DS. The display screen DS is shown on monitor 15. Figure 9The displayed screen DS shows the screen DS when the web browser 80 displays the selection screen 180. Figure 9 The displayed screens are DS (Display UI screen 170), selection screen 180, and browser window BW.

[0134] Selection screen 180 includes a selection image 181, a share icon 183, and a share cancel icon 185. Selection screen 180 displays more than one selection image 181. Figure 9 The selection screen 180 shown includes a first selection image 181a as the selection image 181. When multiple selection images 181 are displayed in the selection screen 180, the multiple selection images 181 are arranged and displayed within the selection screen 180.

[0135] Image selection 181 accepts user input. Image selection 181 is a captured image of the display screen DS projected by projector 100. The first image selection 181a is... Figure 9 The image shown is a scaled-down version of the display screen DS. When the user performs a specified selection operation on the selection image 181, the selected image 181 is selected as the display screen DS projected by the projector 100.

[0136] The shared icon 183 accepts user input. When the user performs a specified operation on the shared icon 183, the projection application 91 accepts the selection instruction operation. The selected image 181, after the selection operation, is selected as the display screen DS projected by the projector 100.

[0137] The share cancel icon 185 accepts user input. When the user performs a specified operation on the share cancel icon 185, the selection screen 180 becomes inactive, and the mirroring is canceled. The control device 10 does not send sharing data to the projector 100.

[0138] When the selection screen 180 is displayed on the display screen DS, the UI screen 170 is displayed on the display screen DS. Figure 9 The selection screen 180 shown is displayed in a position that does not overlap with the UI screen 170, but it can also be displayed in a position that overlaps with the UI screen 170. The selection screen 180 is preferably displayed in a position that does not overlap with the UI screen 170.

[0139] like Figure 9 As shown, when the selection screen 180 is displayed on the display screen DS, it is preferable to shrink the display of the projection settings page window 81w. When the web browser 80 displays the selection screen 180 on the display screen DS, the web browser 80 is controlled to shrink the display of the projection settings page window 81w. By shrinking the display of the projection settings page window 81w, the visibility of the selection screen 180 is improved.

[0140] When selection screen 180 is displayed on display screen DS, the projection settings page window 81w can also be made invisible. After the web browser 80 displays data and obtains page 83, the browser control unit 31 performs a setting window non-display process to make the projection settings page window 81w invisible. The browser control unit 31 performs the setting window non-display process, making the projection settings page window 81w invisible. The setting window non-display process corresponds to one example of window non-display process. By making the projection settings page window 81w invisible, multiple operations and accidental operations by the user on the projection settings page window 81w are prevented.

[0141] like Figure 6 As shown, after displaying the selection screen 180, the web browser 80 accepts the selection instruction operation in step S130. When the user performs a specified operation on the share icon 183 within the selection screen 180, the web browser 80 accepts the selection instruction operation. When accepting the selection instruction operation, the web browser 80 sets the selection screen 180 to be hidden.

[0142] When the selection instruction is received, the web browser 80 acquires shared data in step S131. The web browser 80 acquires the generated data output from the output unit 20 as shared data through the function of the capture application 84 running within the data acquisition page 83. The shared data includes image data output from the display data output unit 21. The image data is data displayed on the display screen DS shown on the monitor 15. The shared data may also include sound data output from the sound output unit 23.

[0143] When retrieving shared data, the web browser 80 uses page display processing to display the data retrieval page window 83w at a second display size. By displaying the data retrieval page window 83w at the second display size, the web browser 80 can reduce the proportion of the data retrieval page window 83w occupied within the display screen DS. This allows the user to see other parts of the screen without being obstructed by the data retrieval page window 83w.

[0144] When shared data is acquired, web browser 80 sends the shared data to projector 100 in step S132. Based on the mirror start instruction operation, web browser 80 performs a shared data transmission process, sending the shared data as projected image data to projector 100. This shared data transmission process corresponds to an example of the data transmission process. Web browser 80 sends the shared data to projector 100 via the streaming path SP.

[0145] When the web browser 80 sends shared data to the projector 100, the projector 100 receives the shared data in step S142. The projector communication unit 150 of the projector 100 receives the shared data sent via the streaming path SP. The projector control unit 130 obtains the shared data via the projector communication unit 150.

[0146] When shared data is received, projector 100 performs mirroring in step S143. Projector 100 acquires the image data contained in the shared data. Projection control unit 131 of projector control unit 130 sends the image data to projection unit 110. Projection control unit 131 causes projection unit 110 to project the image data. Projection unit 110 projects a projection image PG based on the image data onto projection surface SC. When the shared data contains sound data, projector 100 acquires the sound data contained in the shared data. Projection control unit 131 sends the sound data to projector built-in speaker 120. Projector built-in speaker 120 outputs sound based on the sound data. Based on the shared data containing image data and sound data, projector 100 displays the display screen DS on the display 15 of control device 10 and outputs sound based on the sound data. Projector 100 performs mirroring based on the shared data containing image data. Figure 10 The image shown is a mirror image projected by DS.

[0147] Figure 10 The projected image PG is shown projected onto the projection surface SC. Figure 10 This shows the projected image PG projected onto the projection surface SC when the projector 100 performs mirroring. The projected image PG is the same image as the display screen DS displayed on the monitor 15. The display screen DS, which is the same as the projected image PG, is displayed on the monitor 15.

[0148] The projected image PG includes the UI screen 170 and the browser window BW. The browser window BW is a data acquisition page window 83w containing the data acquisition page 83. The data acquisition page window 83w is displayed at a second display size smaller than the first display size. By displaying the data acquisition page window 83w at the second display size, the user can easily see images outside the data acquisition page window 83w. Figure 10 In the projected image PG shown, the user can easily see the UI screen 170. This is displayed on monitor 15. Figure 10 The projected image PG shown is the same as the displayed screen DS. The user can easily see the UI screen 170 and easily perform input operations on the UI screen 170.

[0149] The control flow of the control device 10 includes: communicating with the projector 100; accepting a mirroring instruction operation that causes the web browser 80 to display a projection setting page 81 for setting the projector 100; based on the mirroring instruction operation, performing a projection setting page display process that displays a projection setting page window 81w containing the projection setting page 81 on the display screen DS at a first display size; accepting a mirroring start instruction operation in the projection setting page 81 that causes the projector 100 to project the display screen DS; and based on the mirroring start instruction operation, performing a page acquisition display process and a shared data transmission process, wherein in the page acquisition display process, a data acquisition page window 83w for acquiring image data for displaying the display screen DS is displayed at a second display size smaller than the first display size, and in the shared data transmission process, image data is transmitted to the projector 100 via the data acquisition page window 83w.

[0150] The data acquisition page window 83w is displayed at a second display size by the control device 10, making it easy for the user to see images outside the data acquisition page window 83w.

[0151] The control flow of the control device 10 includes: when the web browser 80 is not started before the mirror instruction operation is accepted, the browser startup process of starting the web browser 80 is executed based on the mirror instruction operation; the projection setting page display process is the process of displaying the projection setting page window 81w containing the projection setting page 81 in the started web browser 80 at a first display size.

[0152] The control device 10 can launch the web browser 80 based on the mirror instruction operation, so that the projection setting page 81 is displayed in the browser window BW.

[0153] The control flow of the control device 10 preferably includes setting window non-display processing, in which the operation is initiated based on the mirror and the projection setting page window 81w is closed.

[0154] By making the projection settings page window 81w invisible, multiple operations and accidental operations by the user on the projection settings page window 81w are prevented.

[0155] The projection system 1 includes a projector 100 and a web browser 80, and a control device 10 that operates accordingly. The control device 10 performs the following processes: establishing a communication connection with the projector 100; receiving a mirroring instruction operation that causes the web browser 80 to display a projection setting page 81 for setting the projector 100; based on the mirroring instruction operation, performing a projection setting page display process that displays a projection setting page window 81w containing the projection setting page 81 at a first display size on the display screen DS; receiving a mirroring start instruction operation in the projection setting page 81 that causes the projector 100 to project the display screen DS; based on the mirroring start instruction operation, performing a page acquisition display process and a shared data transmission process, wherein in the page acquisition display process, a data acquisition page window 83w for acquiring image data for displaying the display screen DS is displayed at a second display size smaller than the first display size; in the shared data transmission process, image data is transmitted to the projector 100 via the data acquisition page window 83w; and image data is transmitted to the projector 100. The projector 100 performs the following processes: receiving image data from the control device 10; and projecting the display screen DS based on the image data.

[0156] The data acquisition page window 83w is displayed at a second display size by the control device 10, and the user can easily see the image outside the data acquisition page window 83w on the display 15 or the projection surface SC.

[0157] Second Implementation Method

[0158] The second embodiment illustrates the control flow and each display screen DS when the web browser 80 is started before the projection application 91 accepts the user's mirroring instruction. In the second embodiment, after the web browser 80 is started, the projection application 91 uses the started web browser 80 to cause the projector 100 to mirror the display screen DS.

[0159] Figure 11 The control flow executed by projection system 1 is shown. Figure 11 The control flow executed by the control device 10, the projector 100, and the cloud server 300 is shown. By executing the control flow, the control device 10 executes a mirror image of the display screen DS projected by the projector 100 onto the display screen 15.

[0160] The web browser 80 is launched in step S221. When the user performs a specified input operation using the input unit 17, the browser control unit 31 receives a browser launch instruction and executes browser launch processing to launch the web browser 80. The web browser 80 is launched before the projection application 91 receives a mirror instruction operation.

[0161] Projection application 91 is launched in step S201. When the input unit 17 of the control device 10 receives a specified operation from the user, the control device 10 launches projection application 91. Projection application 91 is launched on virtual device 90. Projection application 91 runs in the virtual environment.

[0162] Figure 12 An example of a display screen DS is shown. The display screen DS is displayed on monitor 15. Figure 12 The display screen DS shown illustrates the screen displayed when the control device 10 starts the projection application 91. The display screen DS includes the browser window BW of the web browser 80 and the UI screen 170 of the projection application 91. When the projection application 91 is started, the UI screen 170 appears as follows... Figure 12 The image is displayed in the browser window BW.

[0163] like Figure 11 As shown, after startup, the projection application 91 establishes a communication connection with the projector 100 in step S202. The projection application 91 sends control signals such as connection requests to the projector 100 via the TCP / IP communication path TP to establish a communication connection with the projector 100.

[0164] After the projection application 91 is started, the projector 100 establishes a communication connection with the control device 10 in step S241. The projector 100 receives control signals such as connection requests from the projection application 91 via the TCP / IP communication path TP. The projector 100 sends various data such as model information to establish a communication connection with the control device 10.

[0165] After establishing a communication connection with the projector 100, the projection application 91 accepts the mirroring instruction operation in step S203. After the projection application 91 is started, as follows... Figure 12 As shown, a UI screen 170 is displayed on the monitor 15. When the user performs a mirroring instruction operation on the mirroring acceptance icon 172 included in the UI screen 170 displayed on the monitor 15, the mirroring acceptance icon 172 accepts the mirroring instruction operation.

[0166] Figure 13 An example of a display screen DS is shown. The display screen DS is displayed on monitor 15. Figure 13 The displayed screen DS includes the browser window BW of the web browser 80 and the UI screen 170 of the projection application 91. Figure 13 The displayed screen DS shows the screen when the user performs a mirror instruction operation on the mirror acceptance icon 172.

[0167] When a user performs a mirror instruction operation on the mirror acceptance icon 172, such as Figure 13As shown, the display method of switch icon 172a has been changed. Switch icon 172a is slid to the right.

[0168] like Figure 11 As shown, when a mirroring instruction operation is received, the projection application 91 sends a projection settings page request to the cloud server 300 in step S204. By sending the projection settings page request, the projection application 91 causes the projection settings page 81 to be displayed in the tab window TW of the web browser 80. The projection settings page window 81w, containing the projection settings page 81, is displayed as the tab window TW. The projection application 91 displays the projection settings page window 81w as the tab window TW based on the mirroring instruction operation. The projection application 91 can send the projection settings page request directly to the cloud server 300, or it can send it to the cloud server 300 via the web browser 80.

[0169] When the projection application 91 sends a projection settings page request, the cloud server 300 receives the projection settings page request in step S261. The network server 310 included in the cloud server 300 receives the projection settings page request. Based on the projection settings page address contained in the projection settings page request, the network server 310 obtains the projection settings page information.

[0170] After receiving the projection settings page request, cloud server 300 sends the projection settings page information to web browser 80 in step S262. Web server 310, included in cloud server 300, sends the projection settings page information to web browser 80.

[0171] When the cloud server 300 sends the projection settings page information, the web browser 80 receives the projection settings page information in step S222.

[0172] When the projection settings page information is received, the web browser 80 displays the projection settings page 81 in step S223. Based on the projection settings page information, the browser control unit 31 performs projection settings page display processing, which displays the projection settings page window 81w containing the projection settings page 81 as a tab window TW. The web browser 80 displays the projection settings page window 81w of the first display size as a tab window TW.

[0173] Figure 14 An example of a display screen DS is shown. The display screen DS is displayed on monitor 15. Figure 14 The displayed screen DS represents the screen that the browser control unit 31 displays when performing projection setting page display processing based on projection setting page information. Figure 14 The displayed screen DS represents the tab window TW.

[0174] Figure 14The web browser 80 displays the projection settings page window 81w, representing the projection settings page 81, as a tab window TW. The projection settings page 81w displayed in the projection settings page window 81w accepts input operations related to mirroring the display screen DS. The projection settings page window 81w contains a mirroring start icon 81a.

[0175] The projection settings page window 81w, displayed as the tab window TW, is shown at a first display size. For example, the first display size is the full-screen size of the monitor 15. The first display size is also the display size of the browser window BW when the web browser 80 is launched in step S221.

[0176] like Figure 11 As shown, after displaying the projection settings page 81, the web browser 80 accepts the mirror start indication operation in step S224. When the user performs a mirror start indication operation on the mirror start icon 81a, the projection settings page 81 accepts the mirror start indication operation.

[0177] When the image start instruction is accepted, the web browser 80 sends the image start instruction to the cloud server 300 in step S225.

[0178] When the web browser 80 sends a mirror start instruction, the cloud server 300 receives the mirror start instruction in step S263.

[0179] When a mirroring start instruction is received, the cloud server 300 sends a mirroring start instruction to the projection application 91 in step S264.

[0180] When the cloud server 300 sends a mirror start instruction, the projection application 91 receives the mirror start instruction in step S205.

[0181] Upon receiving the mirror start instruction, the projection application 91 sends a data retrieval page request and data retrieval page settings to the cloud server 300 in step S206. By sending the data retrieval page request and data retrieval page settings, the projection application 91 displays the data retrieval page window 83w, which contains the data retrieval page 83, as a tab window TW. The web browser 80, based on the mirror start instruction operation, causes the data retrieval page window 83w to be displayed as a tab window TW.

[0182] The data retrieval page settings are the display settings when the web browser 80 displays the data retrieval page window 83w as a tab window TW. The projection application 91 specifies the display size of the tab window TW when the web browser 80 displays the data retrieval page window 83w as a tab window TW by sending the data retrieval page settings to the cloud server 300. The projection application 91 also sends the data retrieval page settings to the cloud server 300, causing the data retrieval page window 83w, displayed as a tab window TW, to be displayed at a second display size smaller than the first display size.

[0183] When the projection application 91 sends a data retrieval page request and data retrieval page settings, the cloud server 300 receives the data retrieval page request and data retrieval page settings in step S265. The network server 310 included in the cloud server 300 receives the data retrieval page request and data retrieval page settings. The network server 310 retrieves the data retrieval page information based on the data retrieval page address contained in the data retrieval page request.

[0184] After receiving the data retrieval page request and the data retrieval page settings, the cloud server 300 sends the data retrieval page information to the web browser 80 in step S266. The web server 310 included in the cloud server 300 sends the data retrieval page information to the web browser 80.

[0185] When the cloud server 300 sends data to obtain page information, the web browser 80 receives data to obtain page information in step S226.

[0186] When the data retrieval page information is received, the web browser 80 displays the data retrieval page 83 as a tab window TW in step S227. Based on the data retrieval page information, the browser control unit 31 performs retrieval page display processing to display the data retrieval page window 83w containing the data retrieval page 83. Through the retrieval page display processing, the web browser 80 displays the data retrieval page window 83w as a tab window TW at a second display size smaller than the first display size.

[0187] When the data acquisition page window 83w is displayed as a tab window TW on the display screen DS, it is preferable to keep the projection settings page window 81w hidden. When the receiving mirror starts indicating operation, the web browser 80 causes the browser control unit 31 to perform a setting window tab-not-display process to close the projection settings page window 81w, which will be displayed as a tab window TW. The setting window tab-not-display process corresponds to an example of the tab window-not-display process. The browser control unit 31 performs the setting window tab-not-display process to close the projection settings page window 81w. By closing the projection settings page window 81w, multiple operations and accidental operations by the user on the projection settings page window 81w are prevented.

[0188] After displaying the data acquisition page 83, the web browser 80 displays the selection screen 180 on the monitor 15 in step S228. The web browser 80 can also change the display size of the projection settings page window 81w, which is displayed as the tab window TW, when displaying the selection screen 180.

[0189] Figure 15 This represents an example of a display screen DS. The display screen DS is shown on monitor 15. Figure 15 The displayed screen DS shows the screen DS when the web browser 80 displays the selection screen 180. Figure 15 The displayed screen DS shows the selection screen 180 and the browser window BW. Figure 15 The selection screen shown is 180 and Figure 9 The selection screen shown is the same as 180. Selection screen 180 is displayed on web browser 80.

[0190] When the selection screen 180 is displayed on the display screen DS, it is preferable to minimize the display of the projection settings page window 81w, which is displayed as a tab window TW. When the selection screen 180 is displayed on the display screen DS, the web browser 80 minimizes the display of the projection settings page window 81w. By minimizing the display of the projection settings page window 81w, the visibility of both the selection screen 180 and the browser window BW is improved.

[0191] like Figure 11 As shown, after displaying the selection screen 180, the web browser 80 accepts the selection instruction operation in step S229. When the user performs a specified operation on the share icon 183 within the selection screen 180, the web browser 80 accepts the selection instruction operation. When the selection instruction operation is accepted, the web browser 80 sets the selection screen 180 to be hidden.

[0192] When the selection instruction is received, the web browser 80 acquires shared data in step S230. The web browser 80 acquires the generated data output from the output unit 20 as shared data through the function of the capture application 84 running within the data acquisition page 83. The shared data includes image data output from the display data output unit 21. The shared data may also include audio data output from the audio output unit 23.

[0193] When retrieving shared data, the web browser 80 performs page display processing, causing the data retrieval page window 83w, which is currently displayed as a tab window TW, to be displayed at a second display size. By displaying the data retrieval page window 83w at a second display size, the web browser 80 can reduce the proportion of the data retrieval page window 83w occupied within the display screen DS. This allows the user to see other parts of the screen without being obstructed by the data retrieval page window 83w.

[0194] When shared data is acquired, web browser 80 sends the shared data to projector 100 in step S231. Based on the mirror start instruction operation, web browser 80 performs shared data sending processing and sends the shared data to projector 100. Web browser 80 sends the shared data to projector 100 via streaming path SP.

[0195] When the web browser 80 sends shared data to the projector 100, the projector 100 receives the shared data in step S242. The projector communication unit 150 of the projector 100 receives the shared data sent via the streaming path SP. The projector control unit 130 obtains the shared data via the projector communication unit 150.

[0196] When shared data is received, projector 100 performs mirroring in step S243. Projector 100 acquires the image data contained in the shared data. Projection control unit 131 of projector control unit 130 sends the image data to projection unit 110. Projection control unit 131 causes projection unit 110 to project the image data. Projection unit 110 projects a projection image PG based on the image data onto projection surface SC. When the shared data contains sound data, projector 100 acquires the sound data contained in the shared data. Projection control unit 131 sends the sound data to projector built-in speaker 120. Projector built-in speaker 120 outputs sound based on the sound data. Based on the shared data containing image data and sound data, projector 100 displays the display screen DS on the display 15 of control device 10 and outputs sound based on the sound data. Based on the shared data containing image data, projector 100 performs mirroring to project the display screen DS.

[0197] Figure 16 The projected image PG is shown projected onto the projection surface SC. Figure 16 The projected image PG is shown when the projector 100 performs mirroring and projects onto the projection surface SC. The projected image PG is the same image as the display screen DS shown on the monitor 15. The display screen DS, which is the same as the projected image PG, is displayed on the monitor 15.

[0198] The projected image PG includes a browser window BW and a tab window TW. The tab window TW is a data acquisition page window 83w containing data acquisition page 83. The data acquisition page window 83w, displayed as tab window TW, is shown at a second display size smaller than the first display size. By displaying the data acquisition page window 83w at the second display size, the user can easily see the browser window BW. This is displayed on monitor 15. Figure 16 The projected image PG shown is the same as the displayed screen DS. The user can easily see the browser window BW.

[0199] In the control flow of the control device 10, when the web browser 80 is started before the projection setting page display processing is executed, the projection setting page display processing is based on the mirror indication operation, and the projection setting page window 81w of the first display size is displayed as the tab window TW of the web browser 80.

[0200] By displaying the projection settings page window 81w as a tab window TW, the webpage display that was shown in the browser window BW before the projection settings page display process was performed is maintained.

[0201] The control flow of the control device 10 preferably includes a setting window label non-display process that closes the label window TW based on the mirror start indication operation.

[0202] By closing the projection settings page window 81w, the user is prevented from performing multiple operations or accidental operations on the projection settings page window 81w.

[0203] The following is a summary published in this note.

[0204] Postscript 1

[0205] An information processing method of an information processing device includes: establishing a connection with a projection device across a network for transmitting data; receiving a display operation that causes a web browser to display a setting page for setting the projection device; based on the display operation, performing display processing to display a setting window containing the setting page on a display screen at a first size; receiving a projection instruction operation in the setting page that causes the projection device to project the display screen; and based on the projection instruction operation, performing window display processing and data transmission processing, wherein in the window display processing, a window for obtaining display data is displayed at a second size smaller than the first size, the display data being used to cause the projection device to project the display screen, and in the data transmission processing, the display data is transmitted to the projection device via the window for obtaining the display data.

[0206] By displaying the acquisition window at a second size using an information processing device, the user can easily see images outside the acquisition window.

[0207] Appendix 2

[0208] The information processing method of the information processing device described in Appendix 1 includes: when the web browser has not been started before the display operation is accepted, performing a startup process to start the web browser based on the display operation, wherein in the display process, the setting window containing the setting page is displayed in the started web browser at the first size.

[0209] The information processing device can launch a web browser based on the display operation, so that the setting page is displayed in the web browser.

[0210] Appendix 3

[0211] The information processing method of the information processing device described in Appendix 1 or 2 includes: performing window non-display processing to close the set window based on the projection instruction operation.

[0212] By making the settings window invisible, multiple operations and accidental operations by the user are prevented.

[0213] Appendix 4

[0214] In the information processing method of the information processing device described in Appendix 1, when the web browser has been started before the display processing is performed, the display processing displays the set window of the first size as the tab window of the web browser based on the display operation.

[0215] By displaying the settings window as a tab, the webpage displayed in the web browser is maintained before display processing is performed.

[0216] Appendix 5

[0217] The information processing method of the information processing device described in Appendix 4 includes: performing a label window non-display processing to close the label window based on the projection instruction operation.

[0218] By closing the settings window, users are prevented from performing multiple operations or accidental operations on the settings window.

[0219] Appendix 6

[0220] A display system includes a projection device and an information processing device for operation by a web browser. The information processing device performs the following processes: establishing a connection with the projection device across a network for data transmission; receiving a display operation that causes the web browser to display a setting page for setting the projection device; based on the display operation, performing display processing to display a setting window containing the setting page at a first size on a display screen; receiving a projection instruction operation in the setting page that causes the projection device to project the display screen; performing window display processing and data transmission processing based on the projection instruction operation, wherein in the window display processing, a window for obtaining display data is displayed at a second size smaller than the first size, the display data being used to cause the projection device to project the display screen; in the data transmission processing, the display data is sent to the projection device via the window; and the display data is sent to the projection device. The projection device performs the following processes: receiving the display data from the information processing device; and projecting the display screen based on the display data.

[0221] The acquisition window is displayed at a second size by an information processing device, and the user can easily see the image outside the acquisition window on the monitor or projection surface.

Claims

1. An information processing method for an information processing device, wherein, The information processing method of the information processing device includes: In order to transmit data, a connection is established with the projection device across the network; The system accepts an operation to display a settings page for configuring the projection device in a web browser. Based on the aforementioned display operation, display processing is performed to display the setting window containing the setting page on the display screen at a first size; The settings page accepts projection instructions to make the projection device project the display image. as well as Based on the projection instruction operation, window display processing and data transmission processing are performed. In the window display processing, a window for obtaining display data is displayed at a second size smaller than the first size. This display data is used to enable the projection device to project the display image. In the data transmission processing, the display data is sent to the projection device via the window.

2. The information processing method of the information processing device according to claim 1, wherein, The information processing method of the information processing device includes: when the web browser has not been started before the display operation is accepted, performing a startup process to start the web browser based on the display operation. In the display process, the settings window containing the settings page is displayed at the first size in the launched web browser.

3. The information processing method of the information processing device according to claim 2, wherein, The information processing method of the information processing device includes: performing window non-display processing to close the set window based on the projection instruction operation.

4. The information processing method of the information processing device according to claim 1, wherein, If the web browser has already started before the display processing is performed, the display processing displays the set window of the first size as a tab window of the web browser based on the display operation.

5. The information processing method of the information processing device according to claim 4, wherein, The information processing method of the information processing device includes: performing a label window non-display processing to close the label window based on the projection instruction operation.

6. A display system, wherein, The display system includes a projection device and an information processing device for a web browser to run. The information processing device performs the following processing: In order to transmit data, a connection is established with the projection device across the network; The system accepts an operation that causes the web browser to display a settings page for configuring the projection device. Based on the display operation, display processing is performed to display the setting window containing the setting page on the display screen at a first size; The settings page accepts projection instructions to make the projection device project the display image; Based on the projection instruction operation, window display processing and data transmission processing are performed. In the window display processing, a window for obtaining display data is displayed at a second size smaller than the first size. This display data is used to enable the projection device to project the display screen. In the data transmission processing, the display data is sent to the projection device via the window. as well as The display data is sent to the projection device. The projection device performs the following processing: Receive the display data from the information processing device; and Based on the displayed data, the displayed image is projected.

Images