Display method, information processing apparatus and program

Abstract

To provide a display method, an information processing apparatus and a program that improve user convenience.SOLUTION: A display method includes: projecting, by a projection system 1, a first image including a first figure on a projection object 30 from a projection device (projector 100); displaying, by an information processing apparatus 200, a second image including a second figure corresponding to the first figure, on a display device different from the projection device (projector 100); making a moving display, showing that the first figure is moving in the first image, on the second image by the information processing apparatus 200 associatively with the second figure on receiving a movement operation to move the position of the first figure by the information processing apparatus 200; and projecting the first figure after the movement from the projection device (projector 100).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a display method, an information processing apparatus, and a program.

Background Art

[0002] A technique is known in which an image projected by a projection device is corrected by remote operation by an information processing apparatus.

[0003] For example, Patent Document 1 discloses a parameter generation device that generates deformation parameters and is used in a projection device having geometric correction means.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, when an image projected by a projection device is corrected by remote operation by an information processing apparatus, it may be difficult for the user to know whether the operation result of the user is reflected in the image projected by the projection device even if the user operates the information processing apparatus. For example, when a part of the image projected by the projection device is displayed outside the projection target screen or the like, it may be difficult for the user to recognize the change in the image displayed outside the screen.

Means for Solving the Problems

[0006] The present disclosure relates to a display method in a projection system including a projection device and an information processing device, including: projecting a first image including a first graphic from the projection device; displaying a second image including a second graphic corresponding to the first graphic on a display device different from the projection device by the information processing device; when the information processing device receives a movement operation for moving the position of the first graphic, displaying, by the information processing device, a movement display indicating that the first graphic is moving in the first image, in association with the second graphic, on the second image; and projecting the moved first graphic from the projection device.

[0007] The present disclosure relates to an information processing device including one or more processors for controlling a display device, wherein the one or more processors: display a second image including a second graphic corresponding to a first graphic included in a first image projected from a projection device on the display device; when receiving a movement operation for moving the position of the first graphic, display, in association with the second graphic corresponding to the first graphic selected by the movement operation, a movement display indicating that the first graphic is moving in the first image, on the second image; and transmit a projection instruction for the moved first graphic to the projection device.

[0008] The present disclosure relates to a program for causing a processor mounted on an information processing device for controlling a display device to: display a second image including a second graphic corresponding to a first graphic included in a first image projected from a projection device on the display device; when receiving a movement operation for moving the position of the first graphic, display, in association with the second graphic, a movement display indicating that the first graphic is moving in the first image, on the second image; and transmit a projection instruction for the moved first graphic to the projection device.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

[0010] [1. Configuration of the Projection System] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a diagram showing the system configuration of the projection system 1. The projection system 1 includes a projector 100 and an information processing apparatus 200. The information processing apparatus 200 and the projector 100 are connected by a cable 10. FIG. 1 shows an example in which the information processing apparatus 200 and the projector 100 are wired-connected by the cable 10, but the connection between the information processing apparatus 200 and the projector 100 may be wireless. Further, the information processing apparatus 200 and the projector 100 may be connected via a network. The projector 100 corresponds to an example of a projection device.

[0011] The projector 100 displays an image on the projection target 30 by projecting image light onto the projection target 30. The projection target 30 may be a screen in the shape of a curtain, or may be the wall surface of a building or the surface of a fixture.

[0012] The information processing apparatus 200 supplies a signal including image data to be displayed on the projector 100 to the projector 100. A signal including image data transmitted from the information processing apparatus 200 to the projector 100 is referred to as a display signal. As the information processing apparatus 200, for example, a personal computer such as a desktop personal computer or a notebook personal computer, or a mobile terminal such as a smartphone or a tablet personal computer is used.

[0013] [2. Configuration of Projector] FIG. 2 is a block diagram showing the configuration of the projector 100. The configuration of the projector 100 will be described with reference to FIG. 2. The projector 100 includes an infrared light receiving unit 110, a first communication interface 120, an image processing unit 130, a frame memory 135, a projection unit 140, an imaging unit 150, and a first control unit 160. Hereinafter, the interface is abbreviated as I / F.

[0014] The infrared light receiving unit 110 is a receiving device for infrared signals, and includes a light receiving element, a decoder, etc. (not shown). The infrared light receiving unit 110 receives and decodes the infrared signal transmitted from the remote controller 105. The infrared light receiving unit 110 decodes the infrared signal and outputs an instruction signal corresponding to the operated operation key or button of the remote controller 105 to the first control unit 160. The infrared light receiving unit 110 may be configured to receive a wireless signal such as Bluetooth from the remote controller 105. When receiving a wireless signal, the infrared light receiving unit 110 may include an antenna and a receiving circuit. Bluetooth is a registered trademark.

[0015] The first communication I / F 120 is connected to the information processing apparatus 200 via the cable 10 and performs data communication with the information processing apparatus 200 mutually. The first communication I / F 120 is a wired interface having, for example, connection terminals such as a USB (Universal Serial Bus) connector and an Ethernet connector, and an interface circuit. The first communication I / F 120 may be a wireless communication interface. The first communication I / F 120 receives a display signal and extracts the image data included in the received display signal. The first communication I / F 120 outputs the extracted image data to the image processing unit 130. Ethernet is a registered trademark.

[0016] A frame memory 135 is connected to the image processing unit 130. The frame memory 135 includes a plurality of banks. Each bank has a storage capacity capable of writing image data for one frame. The frame memory 135 is constituted by, for example, SDRAM (Synchronous Dynamic RAM). The image processing unit 130 expands the image data input from the first communication I / F 120 in the frame memory 135.

[0017] The image processing unit 130 expands the image data input from the first communication I / F 120 in the frame memory 135. The image processing unit 130 performs processing while synchronizing with the synchronization signal input from the first communication I / F 120 on the image data expanded in the frame memory 135.

[0018] The image processing performed by the image processing unit 130 includes, for example, resolution conversion processing or resizing processing, correction of distortion aberration, shape correction processing, digital zoom processing, adjustment of the color tone and luminance of the image, and the like. The image processing unit 130 executes the processing specified by the first control unit 160 and performs processing using the parameters input from the first control unit 160 as necessary. Also, the image processing unit 130 can of course execute a combination of a plurality of the above image processes.

[0019] The image processing unit 130 and the frame memory 135 are constituted by, for example, an integrated circuit. The integrated circuit includes LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), FPGA (Field-Programmable Gate Array), SoC (System-on-a-chip), etc. Further, an analog circuit may be included in a part of the configuration of the integrated circuit, or a configuration in which the first control unit 160 and the integrated circuit are combined may be adopted.

[0020] The projection unit 140 is connected to the first control unit 160. The projection unit 140 includes, for example, a light source, a light modulation device, and a projection optical system. The light source lights up according to the control of the first control unit 160 and emits light toward the light modulation device. The specific configuration of the light source is not limited. For example, the light source includes lamps such as a halogen lamp, a xenon lamp, and an ultra-high pressure mercury lamp, or solid light sources such as an LED and a laser light source. The light modulation device modulates the light emitted by the light source. The specific configuration of the light modulation device is not limited. For example, the light modulation device can be constituted by a transmissive liquid crystal panel, a reflective liquid crystal panel, or a digital micromirror device (Digital Micromirror Device). In the present embodiment, the case where the light modulation device includes a transmissive liquid crystal panel will be described. The projection optical system included in the projection unit 140 includes lenses, mirrors, prisms, etc. that form an image of the image light modulated by the light modulation device on the projection target 30.

[0021] The projection unit 140 may include a light source drive circuit that supplies power to the light source and a drive circuit that causes the light modulation device to execute drawing. The projection unit 140 may include a zoom mechanism, a focus adjustment mechanism that adjusts the focus, etc.

[0022] The imaging unit 150 is a camera equipped with an imaging element (not shown) such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 150 repeatedly performs imaging based on the control of the first control unit 160, and sequentially outputs the captured image, which is the imaging result, to the first control unit 160.

[0023] The first control unit 160 is a computer device including a first storage unit 170 and a first processor 180.

[0024] The first storage unit 170 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). The RAM is used for temporary storage of various data, etc., and the ROM stores a control program 175 for controlling the operation of the projector 100 and various setting information, etc.

[0025] The first processor 180 is an arithmetic processing device composed of a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The first processor 180 executes the control program 175 to control each part of the projector 100. The first processor 180 may be composed of a single processor, or may be composed of a plurality of processors. Also, the first processor 180 may be composed of a SoC integrated with a part or all of the first storage unit 170 and other circuits. Further, the first processor 180 may be composed of a combination of a CPU that executes a program and a DSP that executes predetermined arithmetic processing. Furthermore, all the functions of the first processor 180 may be implemented in hardware, or may be configured using a programmable device.

[0026] [3. Configuration of the Information Processing Device] FIG. 3 is a block diagram showing the configuration of the information processing device 200. The configuration of the information processing device 200 will be described with reference to FIG. 3. The information processing apparatus 200 includes a second communication I / F 210, an input I / F 221, a mouse 223, a keyboard 225, a display unit 230, and a second control unit 250.

[0027] The second communication I / F 210 is connected to the projector 100 via the cable 10 and performs data communication with the projector 100 mutually. The second communication I / F 210 is a wired interface having connection terminals such as a USB connector and an Ethernet connector, and an interface circuit, for example. The second communication I / F 210 may be a wireless communication interface. Ethernet is a registered trademark.

[0028] The input I / F 221 is connected to the mouse 223 and the keyboard 225 which are input devices, and receives operations by these input devices. The input I / F 221 outputs an operation signal corresponding to the received operation to the second control unit 250. The second control unit 250 determines that an operation has been received by inputting the operation signal. The mouse 223 corresponds to the first operation device, and the keyboard 225 corresponds to the second operation device. The mouse 223 receives, as an operation of the first aspect, a click of the left button or a drag & drop operation. The keyboard 225 receives, as an operation of the second aspect, an operation of the cursor keys.

[0029] The display unit 230 includes a display panel 235 and displays an image on the display panel 235 according to the control of the second control unit 250. For example, a liquid crystal panel or an organic EL (Electro Luminescence) panel is used for the display panel 235. The display panel 235 corresponds to the display device.

[0030] The second control unit 250 includes a second storage unit 260 and a second processor 270.

[0031] The second storage unit 260 includes a volatile memory such as a RAM and a non-volatile memory such as a ROM. Further, the second storage unit 260 includes an auxiliary storage device such as a SSD (Solid State Drive) or an HDD (Hard Disk Drive).

[0032] The second storage unit 260 stores control programs 265 such as an OS (Operating System) 265A executed by the second processor 270 and an application program 265B. Hereinafter, the application program 265B will be referred to as APP265B for short.

[0033] The second processor 270 is an arithmetic processing device composed of a CPU or an MPU. The second processor 270 executes the OS 265A and the app 265B to control each part of the information processing device 200. The second processor 270 may be composed of a single processor or may be composed of a plurality of processors. Further, the second processor 270 may be composed of a SoC integrated with a part or all of the second storage unit 260 and other circuits. Further, the second processor 270 may be composed of a combination of a CPU that executes a program and a DSP that executes predetermined arithmetic processing. Furthermore, all the functions of the second processor 270 may be implemented in hardware, or it may be configured using a programmable device.

[0034] [4. Operations of Information Processing Device and Projector] Next, the operations of the information processing device 200 and the projector 100 will be described. When the information processing apparatus 200 receives an operation to select APP265B via the input I / F 221, the second control unit 250 executes the selected APP265B. APP265B is an APP for setting a range in which an actual display image is to be displayed within the projectable range 40, which is the range where the projector 100 can project image light. The range in which the actual display image is to be displayed is referred to as the projection range 60. The projectable range 40 indicates the range where an image can be projected when a display image is displayed using the entire area of the transmissive liquid crystal panel. The projectable range 40 and the projection range 60 will be described with reference to FIG. 5.

[0035] FIG. 4 is a diagram showing an example of the APP screen 300 displayed on the display panel 235 of the information processing apparatus 200. The second control unit 250 executes APP265B and causes the APP screen 300 shown in FIG. 4 to be displayed on the display panel 235. FIG. 4 is a diagram showing an example of the APP screen 300.

[0036] The APP screen 300 includes a first area 310 and a second area 340. Also, a projection button 335 for instructing the projection of the adjustment image 50 is displayed on the APP screen 300.

[0037] A setting screen 315 is displayed in the first area 310. A setting image 316 including a second figure 301 is displayed on the setting screen 315. The setting image 316 corresponds to the second image. The second figure 301 is a figure used for setting the projection range 60. The number and position of the second figures 301 displayed on the setting screen 315 can be changed by the user operating the second area 340. FIG. 4 shows an example in which four second figures 301 are arranged at the four corners of the setting screen 315 as the second figures 301. In a view of the drawing, the second figure 301 located at the upper left of the setting screen 315 is denoted as the second figure 301A, and the second figure 301 located at the upper right of the setting screen 315 is denoted as the second figure 301B. Also, in a view of the drawing, the second figure 301 located at the lower left of the setting screen 315 is denoted as the second figure 301C, and the second figure 301 located at the lower right of the setting screen 315 is denoted as the second figure 301D.

[0038] FIG. 4 shows an example in which a cross-shaped figure is displayed as the second figure 301, but the shape of the second figure 301 is arbitrary. Also, FIG. 4 shows a case where the shapes of the second figures 301A, 301B, 301C, and 301D are all the same, but the second figures 301A, 301B, 301C, and 301D may differ in at least one of shape or color.

[0039] In the second area 340, a switching display 350, a figure number setting display 360, a lens shift 370, a coordinate display 380, and a selected figure display 390 are displayed.

[0040] The switching display 350 is a display that accepts an operation to switch whether or not to display the second figure 301 on the setting screen 315. The switching display 350 displays an on button 331 and an off button 333. When the on button 331 is selected, the second figure 301 is displayed on the setting screen 315, and when the off button 333 is selected, the second figure 301 displayed on the setting screen 315 is erased.

[0041] The figure number setting display 360 is a display that accepts an operation to set the number of second figures 301 to be displayed on the setting screen 315. The figure number setting display 360 displays a display column 361 that displays the number of second figures 301 to be displayed in the vertical direction of the setting screen 315 in a drawing view, and an operator 362 that accepts an operation to change the number of second figures 301 to be displayed in the vertical direction of the setting screen 315. Also, the figure number setting display 360 displays a display column 363 that displays the number of second figures 301 to be displayed in the horizontal direction of the setting screen 315 in a drawing view, and an operator 364 that accepts an operation to change the number of second figures 301 to be displayed in the horizontal direction of the setting screen 315.

[0042] On the lens shift 370, arrow keys 371, 372, 373, and 374 corresponding to each of the up, down, left, and right directions are displayed. By operating the arrow keys 371, 372, 373, and 374 with the mouse 223, the user can send an instruction to the projector 100 to shift the position of the projection lens included in the projection optical system provided in the projection unit 140 of the projector 100. Thereby, the position of the image projected by the projection unit 140 onto the projection target 30 can be changed.

[0043] In the coordinate display 380, when the user moves the first figure �1 corresponding to the second figure 301 selected by the user on the adjustment image 50, the amount of movement on the adjustment image 50 is displayed. The coordinate display 380 includes a vertical movement amount and a horizontal movement amount. The vertical movement amount is the amount of movement in the vertical direction of the adjustment image 50, and the horizontal movement amount is the amount of movement in the horizontal direction of the adjustment image 50. The coordinate display 380 corresponds to an example of a movement display. Also, the coordinate display 380 corresponds to an example of a direction display. Also, the coordinate display 380 corresponds to an example of a movement amount display indicating the movement amount of the first figure.

[0044] In the selected figure display 390, the serial number of the second figure 301 selected by the user's operation among the second figures 301 displayed on the setting screen 315 is displayed. As shown in FIG. 4, serial numbers are respectively displayed on the second figures 301A, 301B, 301C, and 301D displayed at the four corners of the setting screen 315. In the selected figure display 390, the serial number of the second figure selected by the user among these second figures 301 is displayed.

[0045] FIG. 5 is a diagram showing the adjustment image 50 projected onto the projection target 30. The adjusted image 50 includes a first graphic 51. The adjusted image 50 corresponds to an example of the first image. FIG. 5 shows an example in which the first graphic 51 is displayed at each of the four corners of the adjusted image 50. In a drawing view, the first graphic 51 located at the upper left of the adjusted image 50 is denoted as the first graphic 51A, and the first graphic 51 located at the upper right of the adjusted image 50 is denoted as the first graphic 51B. Also, in a drawing view, the first graphic 51 located at the lower left of the adjusted image 50 is denoted as the first graphic 51C, and the first graphic 51 located at the lower right of the adjusted image 50 is denoted as the first graphic 51D.

[0046] The first graphic 51 corresponds to a second graphic displayed on the setting screen 315. The second graphic 301A displayed on the setting screen 315 shown in FIG. 4 corresponds to the first graphic 51A, and the second graphic 301B displayed on the setting screen 315 corresponds to the first graphic 51B. Also, the second graphic 301C displayed on the setting screen 315 corresponds to the first graphic 51C, and the second graphic 301D displayed on the setting screen 315 corresponds to the first graphic 51D.

[0047] The display position of the first graphic 51A in the adjusted image 50 corresponds to the position of the second graphic 301A on the setting screen 315. The display position of the first graphic 51B in the adjusted image 50 corresponds to the position of the second graphic 301B on the setting screen 315. The display position of the first graphic 51C in the adjusted image 50 corresponds to the position of the second graphic 301C on the setting screen 315. The display position of the first graphic 51D in the adjusted image 50 corresponds to the position of the second graphic 301D on the setting screen 315.

[0048] The first graphics 51A, 51B, 51C, and 51D are graphics that define a projection range 60, which is a range within the projectable range 40 where the actual display image is to be displayed. For example, as shown in FIG. 5, the area partitioned by the line segments connecting the centers of the first graphics 51A, 51B, 51C, and 51D arranged at the four corners of the adjusted image 50 is set as the projection range 60.

[0049] When the user presses the projection button 335 displayed on the APP screen 300, the second control unit 250 generates adjustment image data that serves as the basis for the adjustment image 50, and transmits the generated adjustment image data to the projector 100. The projector 100 generates image light based on the received adjustment image data, and projects the generated image light onto the projection target 30. As a result, the adjustment image 50 is displayed on the projection target 30.

[0050] While visually recognizing the adjustment image 50 projected onto the projection target 30 and the display panel 235, the user operates the input device to perform an operation on the second graphic 301, that is, change the position of the second graphic 301 on the setting screen 315. In other words, the information processing apparatus 200 receives an operation on the second graphic 301 from the user. By changing the position of the second graphic 301 on the setting screen 315, the position of the first graphic 51 in the adjustment image 50 is also changed, and the projection range 60 is changed.

[0051] At this time, when the projection target 30 onto which the projector 100 projects an image is not a screen but, for example, a wall surface or a curved surface of a building, a part of the adjustment image 50 may be displayed outside the projection target 30. If the first graphic 51 is included in a part of the adjustment image 50 outside the projection target 30, the visibility of the first graphic 51 decreases. For this reason, even when the user operates the information processing apparatus 200 to change the display position of the first graphic 51, it may be difficult for the user to grasp whether the display position of the first graphic 51 has been changed.

[0052] Next, an operation of changing the display position of the first graphic 51 in the adjustment image 50 by operating the setting screen 315 will be described. FIG. 6 is a diagram showing a state in which the second graphic 301A is selected. First, the user operates the mouse 223 to overlap the mouse cursor 5 on the second graphic 301 corresponding to the first graphic 51 whose display position is to be changed. When the user left-clicks the mouse 223, the second graphic 301 overlapping the mouse cursor 5 is selected.

[0053] When the second control unit 250 receives an operation of selecting any one of the second figures 301 by a left click operation of the mouse 223, as shown in FIG. 6, a rectangular figure 303 surrounding the selected second figure 301 is displayed. FIG. 6 shows a state in which the second figure 301A is selected and the periphery of the second figure 301A is surrounded by the rectangular figure 303. Further, in the selection figure display 390, "1", which is the serial number of the selected second figure 301A, is displayed.

[0054] FIG. 7 is a diagram showing the third figure 305. Next, the movement operation using the mouse 223 will be described. The user moves the mouse cursor 5 to a position where the second figure 301 is to be displayed by a drag & drop operation of the mouse 223. Specifically, the user presses the left button of the mouse 223 with the mouse cursor 5 overlapping the selected second figure 301, and while keeping the left button of the mouse 223 pressed, moves the mouse cursor 5 to a position where the second figure 301 is to be displayed, and releases the pressing of the left button of the mouse 223 at this position where it is to be moved. The drag & drop operation corresponds to an operation of specifying the position of the first figure 51 after the movement.

[0055] The second control unit 250 displays the third figure 305 at the position where the pressing of the left button of the mouse 223 is released. The third figure 305 is a figure congruent or similar to the second figure 301.

[0056] When the second control unit 250 displays the third figure 305 on the setting screen 315, the display position of the first figure 51A corresponding to the selected second figure 301A is gradually moved. The second control unit 250 calculates the moving direction and the moving amount when moving the second figure 301A from the current position to the position of the third figure 305 on the setting screen 315. The second control unit 250 moves the position of the first figure 51A in the adjustment image 50 projected by the projector 100 based on the calculated moving direction and moving amount. The resolution of the setting screen 315 is different from the panel resolution of the transmissive liquid crystal panel included in the light modulation device. Therefore, based on the resolution of the setting screen 315 and the panel resolution of the transmissive liquid crystal panel, the second control unit 250 converts the moving direction and the moving amount calculated on the setting screen 315 into the moving direction and the moving amount on the transmissive liquid crystal panel.

[0057] Next, when the second control unit 250 acquires the moving direction and the moving amount on the transmissive liquid crystal panel, the second control unit 250 sequentially generates adjustment image data in which the position of the first graphic 51A included in the adjustment image 50 is moved in the acquired moving direction by a preset moving amount each time. The second control unit 250 outputs the sequentially generated adjustment image data to the projector 100, and causes the projector 100 to project an adjustment image 50 based on the adjustment image data.

[0058] Note that, while the first graphic 51A is moving, the second control unit 250 may not change the display position of the second graphic 301A on the setting screen 315. That is, the position of the second graphic 301A on the setting screen 315 when the position of the first graphic 51A is the first position before the change may be the same as the position of the second graphic 301 when the position of the first graphic 51A is a second position different from the first position. When the projection target 30 is a building such as a curved surface or a building and the adjustment image 50 is projected, even if the first graphic 51A and the second graphic 301A are moved, the movement of the first graphic 51A in the adjustment image 50 and the movement of the second graphic 301A on the setting screen 315 may not match, which may confuse the user. For this reason, even if the display position of the first graphic 51A is moved in the adjustment image 50, the second graphic 301A is not moved on the setting screen 315.

[0059] In addition, when the movement of the first graphic 51A in the adjustment image 50 is completed, the second control unit 250 deletes the third graphic 305 from the setting screen 315.

[0060] FIG. 8 is a flowchart showing the operation of the information processing apparatus 200. The operation of the information processing apparatus 200 will be described with reference to the flowchart shown in FIG. 8. When the second control unit 250 receives an operation to select APP265B via the input I / F 221, it executes APP265B (step S1) and causes the APP screen 300 to be displayed on the display panel 235 (step S2).

[0061] When the second control unit 250 causes the APP screen 300 to be displayed, it determines whether a projection button 335 has been pressed and a projection instruction for the adjustment image 50 has been received (step S3). When the second control unit 250 has not received a projection instruction for the adjustment image 50 (step S3 / NO), it waits until a projection instruction is received.

[0062] When the second control unit 250 receives a projection instruction for the adjustment image 50 (step S3 / YES), it generates adjustment image data and transmits the generated adjustment image data to the projector 100 (step S4).

[0063] Next, the second control unit 250 determines whether an operation to select any one of the second figures 301 has been received (step S5). When the second control unit 250 has not received an operation to select any one of the second figures 301 (step S5 / NO), it waits until an operation to select any one of the second figures 301 is received.

[0064] When the second control unit 250 receives an operation to select any one of the second figures 301 (step S5 / YES), it causes a rectangular figure 303 surrounding the selected second figure 301 to be displayed on the setting screen 315 (step S6).

[0065] Next, the second control unit 250 determines whether a movement operation to specify a destination for the selected second figure 301 has been received (step S7). When the second control unit 250 has not received a movement operation (step S7 / NO), it waits until a movement operation is received.

[0066] When the second control unit 250 receives a movement operation (step S7 / YES), it causes the third figure 305 to be displayed at the position of the setting screen 315 specified by the movement operation (step S8). Next, the second control unit 250 calculates the movement direction and the movement amount when moving from the position of the selected second figure 301 to the position of the third figure 305 (step S9).

[0067] Next, the second control unit 250 converts the calculated movement direction and movement amount into the movement direction and movement amount on the transmissive liquid crystal panel (step S10). Next, the second control unit 250 causes the calculated movement amount to be displayed on the coordinate display 380 of the second region 340 (step S11).

[0068] Next, the second control unit 250 sequentially generates adjustment image data for moving the display position of the first figure 51 corresponding to the selected second figure 301 in the movement direction by the set amount each time. The second control unit 250 transmits the generated adjustment image data to the projector 100 (step S13). That is, the second control unit 250 transmits a projection instruction for the first figure 51 after the movement to the projector 100. During this time, the second control unit 250 does not change the display position of the selected second figure 301 on the setting screen 315.

[0069] Next, the second control unit 250 determines whether or not the first figure 51 has moved to the position specified by the movement operation (step S14). When the first figure 51 has not moved to the position specified by the movement operation (step S14 / NO), the second control unit 250 returns to step S12 to generate adjustment image data.

[0070] Also, when the first figure 51 moves to the position specified by the movement operation (step S14 / YES), the second control unit 250 deletes the third figure 305 from the setting screen 315 (step S15).

[0071] Next, the second control unit 250 determines whether an operation of selecting any one of the second figures 301 has been received (step S16). When the second control unit 250 has received an operation of selecting any one of the second figures 301 (step S16 / YES), it returns to step S6 and causes a rectangular figure 303 to be displayed around the selected second figure 301.

[0072] Also, when none of the second figures 301 has been selected (step S16 / NO), the second control unit 250 determines whether an operation to terminate APP265B has been received (step S17). When the second control unit 250 has not received an operation to terminate APP265B (step S17 / NO), it returns to the determination in step S15. When the second control unit 250 has received an operation to terminate APP265B (step S17 / YES), it ends this processing flow.

[0073] FIG. 9 is a diagram showing a first display example of the setting screen 315. In this first display example, while moving the first figure 51 on the adjustment image 50, the moving direction of the first figure 51 is displayed by combining a plurality of arrow images 307 and the second figure 301. The plurality of arrow images 307 includes arrow images 307A, 307B, 307C, and 307D corresponding to the up, down, left, and right directions. The arrow images 307A, 307B, 307C, and 307D correspond to a plurality of arrow images that are in one-to-one correspondence with the plurality of moving directions of the first figure 51. In the example shown in FIG. 9, in a plan view, since the moving direction from the second figure 301 to the third figure 305 is diagonally downward to the right, an example is shown in which the right arrow image 307D and the downward arrow image 317B are displayed. The plurality of arrow images 307 corresponds to an example of movement display. Also, the plurality of arrow images 307 corresponds to an example of direction display.

[0074] FIG. 10 is a diagram showing a second display example of the setting screen 315. In Display Example 2, when any one of the second graphics 301 is selected and a rectangular graphic 303 surrounding the selected second graphic 301 is displayed, a range image 309 indicating the range where the third graphic 305 can be set is displayed. By displaying the range image 309, the user can recognize the range where the third graphic 305 can be set, and the user can efficiently set the third graphic 305.

[0075] FIG. 11 is a diagram showing Display Example 3 of the setting screen 315. In Display Example 3, when any one of the second graphics 301 is selected and a rectangular graphic 303 surrounding the selected second graphic 301 is displayed, an enlarged image 320 of a predetermined range including the selected second graphic 301 is generated and overlaid and displayed on the setting screen 315.

[0076] An enlarged second graphic 321A, an enlarged third graphic 325, an enlarged rectangular graphic 323, and an enlarged range image 329 are displayed in the enlarged image 320. The display sizes of the enlarged second graphic 321A, the enlarged third graphic 325, the enlarged rectangular graphic 323, and the enlarged range image 329 displayed in the enlarged image 320 are larger than the display sizes of the second graphic 301A, the third graphic 305, the rectangular graphic 303, and the range image 309 displayed on the setting screen 315. Therefore, the operability when the user operates the setting screen 315 can be improved.

[0077] FIGS. 12 and 13 are diagrams showing Display Example 4 of the setting screen 315. FIG. 12 shows the setting screen 315 displayed when operations of a plurality of cursor keys included in the keyboard 225, which is the second operation device, are received. FIG. 13 shows the setting screen 315 displayed when a drag operation of the mouse 223, which is the first operation device, is received.

[0078] When the cursor keys of the keyboard 225 are operated, an arrow image 307 is displayed on the setting screen 315 as a direction display indicating the moving direction of the first graphic 51. FIG. 12 shows an example in which an arrow image 307A indicating that the moving direction is the right direction in the drawing view is displayed. Also, the movement amount of the first graphic 51 is displayed in the coordinate display 380 of the second area 340. The arrow image 307A is an example of a movement display. Also, the arrow image 307A is an example of a direction display.

[0079] Also, when a drag operation of the mouse 223 is received, a third graphic 305 is displayed on the setting screen 315. In this way, by changing the display of the setting screen 315 according to the operation device that has received the user's operation, the user can be made to recognize the operation device that the user is operating.

[0080] FIG. 14 is a diagram showing a fifth display example of the setting screen 315. In the fifth display example, while moving the display position of the first graphic 51 in the adjustment image 50, a fourth graphic 330 is displayed instead of the second graphic 301 corresponding to the first graphic 51. The fourth graphic 330 is a graphic in which at least one of the shape or color is different from that of the second graphic 301. FIG. 4 shows an example in which an elliptical fourth graphic 330 is displayed instead of the cross-shaped second graphic 301. However, the shape of the fourth graphic 330 is not limited to an ellipse, and it only needs to be different in shape from the second graphic 301. Also, even if the shape of the fourth graphic 330 is the same as the shape of the second graphic 301, it only needs to be different in color from the color of the second graphic 301. By displaying the fourth graphic 330 instead of the third graphic 305 in this way, the user can be made to recognize that the position of the first graphic 51 in the adjustment image 50 has been changed.

[0081] [5. Other configurations] The above-described embodiments are preferred embodiments of the present invention. However, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention. For example, in the above-described embodiment, the configuration in which the display unit 230 included in the information processing apparatus 200 is used as the display device has been described, but the present invention is not limited to this. For example, as the display device, a display such as a liquid crystal display, an organic EL display, or a plasma display externally connected to the information processing apparatus 200 may be used.

[0082] Further, in the above-described embodiment, at least the coordinate display 380 is displayed as an example of the moving display, but the present invention is not limited to this. For example, instead of displaying the coordinate display 380, only the arrow image 307A as the direction display may be displayed.

[0083] Also, in the above-described embodiment, the moving display is displayed in response to a user operation on the second figure 301 displayed on the display unit 230, but the present invention is not limited to this. For example, the moving display may be displayed in response to a user operation on the remote controller 105. Specifically, the remote controller 105 receives an operation to move the first figure 51, the projector 100 receives an operation signal corresponding to the operation from the remote controller 105, the projector 100 moves the first figure 51A in the adjustment image 50 based on the operation signal, and the projector 100 transmits at least one of information indicating the moving amount of the first figure 51A or information indicating the moving direction to the information processing apparatus 200. The information processing apparatus 200 may display at least one of the coordinate display 380 based on the information indicating the moving amount of the first figure 51A or the arrow image 307A based on the information indicating the moving direction on the setting image 316.

[0084] Also, in the above-described embodiment, the information processing apparatus 200 generates adjustment image data and transmits the generated adjustment image data to the projector 100. The projector 100 generates image light based on the adjustment image data received from the information processing apparatus 200 and projects it onto the projection target 30. As another example, the projector 100 may generate adjustment image data based on an instruction from the information processing apparatus 200.

[0085] In addition, each functional unit of the projector 100 shown in FIG. 2 shows a functional configuration, and the specific implementation form is not particularly limited. That is, it is not necessarily required that hardware corresponding to each functional unit be implemented individually, and of course, it is also possible to adopt a configuration in which one processor executes a program to realize the functions of a plurality of functional units. Further, in the above embodiment, part of the functions realized by software may be realized by hardware, and part of the functions realized by hardware may be realized by software. In addition, the specific detailed configuration of other parts of the projector can be arbitrarily changed without departing from the gist of the present invention.

[0086] In addition, each functional unit of the information processing apparatus 200 shown in FIG. 3 shows a functional configuration, and the specific implementation form is not particularly limited. That is, it is not necessarily required that hardware corresponding to each functional unit be implemented individually, and of course, it is also possible to adopt a configuration in which one processor executes a program to realize the functions of a plurality of functional units. Further, in the above embodiment, part of the functions realized by software may be realized by hardware, and part of the functions realized by hardware may be realized by software. In addition, the specific detailed configuration of other parts of the projector can be arbitrarily changed without departing from the gist of the present invention.

[0087] In addition, the processing units of the flowchart shown in FIG. 8 are divided according to the main processing contents in order to facilitate the understanding of the processing of the information processing apparatus 200. The present invention is not limited by the way of dividing the processing units shown in the flowchart of FIG. 8 and the names thereof. Further, the processing of the second control unit 250 can be further divided into more processing units according to the processing contents, or can be divided such that one processing unit includes more processes. In addition, the processing order of the above flowchart is not limited to the illustrated example.

[0088] Also, when the program is realized using the computers (processors) included in the information processing apparatus 200 and the projector 100, it is also possible to configure the program to be executed by these computers in the form of a computer-readable and non-transitory recording medium. Alternatively, it is also possible to configure it in the form of a transmission medium that transmits the program to be executed by these computers. As the recording medium, a magnetic, optical recording medium, or semiconductor memory device can be used. Specifically, examples include portable or fixed recording media such as flexible disks, HDDs (Hard Disk Drives), CD-ROMs, DVDs (Digital Versatile Discs), Blu-ray Discs, magneto-optical disks, flash memories, card-type recording media, etc. Also, the above recording medium may be a non-volatile storage device such as a RAM, ROM, HDD, etc., which is an internal storage device included in the server device. Blu-ray is a registered trademark.

[0089] [Summary of the Present Disclosure] Hereinafter, a summary of the present disclosure will be appended.

[0090] (Appended Note 1) A display method in a projection system including a projection device and an information processing device, comprising: projecting a first image including a first graphic from the projection device; displaying, by the information processing device, a second image including a second graphic corresponding to the first graphic on a display device different from the projection device; when the information processing device receives a movement operation for moving the position of the first graphic, displaying, by the information processing device, a movement display indicating that the first graphic is moving in the first image, in association with the second graphic, on the second image; and projecting the moved first graphic from the projection device.

[0091] According to this, when a movement operation for moving the position of the first figure is received, a movement display indicating that the first figure is moving in the first image is displayed on the second image by the information processing apparatus in association with the second figure. Therefore, it is possible for the user to grasp that the first figure in the first image projected by the projection apparatus is moving through the second image displayed by the display apparatus.

[0092] (Appendix 2) The position of the second figure when the position of the first figure is the first position is the same as the position of the second figure when the position of the first figure is a second position different from the first position, according to the display method described in Appendix 1.

[0093] According to this, even if the position of the first figure in the first image is changed from the first position to the second position, the position of the second figure in the second image is not changed. If the movement of the first figure in the first image is linked to the movement of the second figure in the second image, depending on the projection target on which the projection apparatus projects the first image, there may be a deviation between the movement of the first figure in the first image and the movement of the second figure in the second image, which may give the user a sense of discomfort. Therefore, by not changing the position of the second figure in the second image even when the position of the first figure in the first image is changed, the user is not given a sense of discomfort.

[0094] (Appendix 3) The movement display includes at least one of a direction display indicating the movement direction of the first figure and a movement amount display indicating the movement amount of the first figure, according to the display method described in Appendix 1 or Appendix 2.

[0095] Thereby, the fact that the first figure is moving is displayed by a direction display indicating the movement direction of the first figure and a movement amount display indicating the movement amount of the first figure. Therefore, even when it is difficult for the user to visually recognize the first figure in the first image projected by the projection apparatus, it is possible for the user to grasp that the first figure is moving due to the movement operation.

[0096] (Appendix 4) The movement operation includes an operation of specifying the position of the first figure after movement in the second image. Displaying the second image by the information processing apparatus includes displaying a third figure at the position of the second image specified by the specifying operation by the information processing apparatus. Moving the position of the first figure in the first image by the projection apparatus includes moving the first figure by the projection apparatus to the position of the first image corresponding to the position of the third figure in the second image. Displaying the second image by the information processing apparatus further includes erasing the third figure from the second image by the information processing apparatus after the movement of the first figure is completed. The display method according to Appendix 2 or Appendix 3.

[0097] Thereby, by performing an operation of specifying the position of the first figure after movement in the second image, the third figure is displayed at the specified position of the second image, and the first figure can be moved to the position of the first image corresponding to the position of the third figure in the second image. Therefore, the position of the first figure in the first image can be changed by an intuitive operation. Further, after the movement of the first figure is completed, since the third figure is erased from the second image, figures unnecessary for the operation can be erased.

[0098] (Appendix 5) The third figure is congruent or similar to the second figure. The display method according to Appendix 4.

[0099] Thereby, the design of the third figure becomes the same as the design of the second figure, and the second figure that does not move in the second image seems to move virtually, improving the operability.

[0100] (Appendix 6) The movement display is a plurality of arrow images that correspond one-to-one with a plurality of movement directions of the first figure. Displaying the second image by the information processing apparatus further includes displaying the movement direction of the third figure by combining at least one of the plurality of arrow images and the third figure by the information processing apparatus. The display method according to Appendix 4 or Appendix 5.

[0101] As a result, the moving direction of the third figure is displayed in combination with at least one of the plurality of arrow images and the third figure. Therefore, the user can grasp the moving direction of the third figure.

[0102] (Appendix 7) The method of displaying the second image by the information processing apparatus includes displaying a range image indicating a movable range of the third figure by the information processing apparatus, and is the display method according to any one of Appendices 4 to 6.

[0103] As a result, a range image indicating a movable range of the third figure is displayed as the second image. Therefore, the user can grasp the movable range of the third image.

[0104] (Appendix 8) The information processing apparatus further includes displaying a third image including the second figure, the third figure, and the range image on top of the second image, and the display sizes of the second figure, the third figure, and the range image in the third image are larger than the display sizes of the second figure, the third figure, and the range image in the second image. The display method according to Appendix 7.

[0105] As a result, a third image including the second figure, the third figure, and the range image is displayed on top of the second image. Since the display sizes of the second figure, the third figure, and the range image in the third image are larger than the display sizes of the second figure, the third figure, and the range image in the second image, the operability of the user can be improved.

[0106] (Appendix 9) Displaying the second image includes, when receiving an operation by a first operation device, displaying the third figure by the information processing apparatus, and when receiving an operation by a second operation device different from the first operation device, the information processing apparatus displays a direction display indicating the moving direction of the first figure. The display method according to any one of Appendices 4 to 8.

[0107] As a result, the display of the second image differs depending on whether it is operated by the first operation device or the second operation device. Therefore, it is possible to recognize the operation device that the user is operating.

[0108] (Appendix 10) The display method according to Appendix 9, wherein the first operation device receives an operation of the first mode from the user, and the second operation device receives an operation of a second mode different from the operation of the first mode from the user.

[0109] As a result, the first operation device receives an operation of the first mode from the user, and the second operation device receives an operation of a second mode different from the operation of the first mode from the user. Therefore, by changing the operation device, it is possible to operate the second figure displayed in the second image in different operation modes.

[0110] (Appendix 11) The display method according to Appendix 9 or 10, wherein the first operation device is a mouse, the second operation device is a plurality of cursor keys included in a keyboard, the third figure moves by a drag operation on the mouse, and the direction display corresponds to the direction indicated by the cursor keys.

[0111] As a result, the third figure can be moved by dragging the mouse, and the direction display can be displayed in the direction corresponding to the operation received by the cursor keys of the keyboard.

[0112] (Appendix 12) The display method according to any one of Appendices 3 to 11, wherein the second image includes a first area in which the movement operation is received and the direction display and the second figure are displayed, and a second area in which the movement amount display is displayed.

[0113] As a result, a second image is displayed, which includes a first area that accepts a movement operation and displays a direction display and a second figure, and a second area that displays a movement amount display. The user can recognize the moving direction and the movement amount of the second figure.

[0114] (Appendix 13) Displaying the second image includes, while moving the position of the first figure in the first image, the information processing apparatus displaying, instead of the second figure, a fourth figure that is different from the second figure in at least one of shape or color, according to any one of Appendices 1 to 8.

[0115] As a result, while the position of the first figure is being moved, a fourth figure that is different from the second figure in at least one of shape or color is displayed instead of the second figure. Therefore, the user can grasp that the position of the first figure is being moved.

[0116] (Appendix 14) An information processing apparatus including one or more processors for controlling a display device, wherein the one or more processors display a second image including a second figure corresponding to a first figure included in a first image projected from a projection device on the display device, and when receiving a movement operation for moving the position of the first figure, associate a movement display indicating that the first figure is moving in the first image with the second figure corresponding to the first figure selected by the movement operation and display it on the second image, and transmit a projection instruction for the moved first figure to the projection device.

[0117] According to this, when receiving a movement operation for moving the position of the first figure, a movement display indicating that the first figure is moving in the first image is displayed on the second image by the information processing apparatus in association with the second figure. Therefore, the user can grasp through the second image displayed by the display device that the first figure of the first image projected by the projection device is moving.

[0118] (Appendix 15) A program that causes a processor mounted on an information processing device that controls a display device to display, on the display device, a second image including a second figure corresponding to a first figure included in a first image projected from a projection device; and when a movement operation for moving the position of the first figure is received, to display, in association with the second figure, a movement display indicating that the first figure is moving in the first image on the second image; and to transmit a projection instruction for the moved first figure to the projection device.

[0119] According to this, when a movement operation for moving the position of the first figure is received, a movement display indicating that the first figure is moving in the first image is displayed on the second image by the information processing device in association with the second figure. Therefore, it is possible for the user to grasp, through the second image displayed on the display device, that the first figure of the first image projected by the projection device is moving.

Explanation of Reference Numerals

[0120] 1…Projection system, 5…Mouse cursor, 10…Cable, 30…Projection target, 40…Projectable range, 50…Adjustment image, 51, 51A, 51B, 51C, 51D…First figure, 60…Projection range, 100…Projector, 105…Remote control, 110…Remote control light receiving unit, 120…First communication I / F, 130…Image processing unit, 135…Frame memory, 140…Projection unit, 150…Imaging unit, 160…First control unit, 170…First memory unit, 175…Control program, 180…First processor, 200…Information processing device, 210…Second communication I / F, 221…Input I / F, 223…Mouse, 225…Keyboard, 250…Second control unit, 260…Second memory unit, 265…Control program, 265A…OS, 265B…APP, 270…Second processor, 300…APP screen, 301, 301A, 301B, 301C, 301D…Second figure, 303…Rectangle figure, 305…Third figure, 307, 307A, 307B, 307C, 307D…Arrow image, 309…Range image, 310…First area, 315…Setting screen, 317B…Downward arrow image, 320…Enlarged image, 321…Enlarged second figure, 323…Enlarged rectangle figure, 325…Enlarged third figure, 329…Enlarged range image, 330…Fourth figure, 331…On button, 333…Off button, 335…Projection button, 340…Second area, 350…Switching display, 362, 364…Operator, 370…Cross key display, 371, 372, 373, 374…Arrow keys, 380…Coordinate display, 390…Selected figure display.

Claims

1. A display method in a projection system including a projection device and an information processing device, comprising: projecting a first image including a first figure from the projection device; displaying, by the information processing device, a second image including a second figure corresponding to the first figure on a display device different from the projection device; when the information processing device receives a movement operation for moving the position of the first figure, displaying, by the information processing device, on the second image, in association with the second figure, a movement display indicating that the first figure is moving in the first image; projecting the moved first figure from the projection device; and a display method including the above.

2. The display method according to claim 1, wherein the position of the second figure when the position of the first figure is a first position is the same as the position of the second figure when the position of the first figure is a second position different from the first position.

3. The display method according to claim 1, wherein the movement display includes at least one of a direction display indicating the movement direction of the first figure and a movement amount display indicating the movement amount of the first figure.

4. The movement operation includes an operation of designating the position of the first figure after movement in the second image, displaying the second image by the information processing device includes displaying, by the information processing device, a third figure at the position of the second image designated by the designation operation, moving the position of the first figure in the first image by the projection device includes moving, by the projection device, the first figure to the position of the first image corresponding to the position of the third figure in the second image, displaying the second image by the information processing device includes after the movement of the first figure is completed, further including erasing, by the information processing device, the third figure from the second image, according to the display method of claim 2.

5. The third figure is congruent or similar to the second figure, according to the display method of claim 4.

6. The movement display is a plurality of arrow images that correspond one-to-one with a plurality of movement directions of the first figure, displaying the second image by the information processing device includes further including, by the information processing device, displaying the movement direction of the third figure by combining at least one of the plurality of arrow images and the third figure, according to the display method of claim 4.

7. Displaying the second image by the information processing apparatus includes: The display method according to claim 4, wherein the information processing apparatus displays a range image indicating a movable range of the third figure. **Claim 8** The method further includes: the information processing apparatus superimposes and displays a third image including the second figure, the third figure, and the range image on the second image; The display sizes of the second figure, the third figure, and the range image in the third image are larger than the display sizes of the second figure, the third figure, and the range image in the second image. The display method according to claim 7. **Claim 9** Displaying the second image includes: When receiving an operation by a first operation device, the information processing apparatus displays the third figure; and When receiving an operation by a second operation device different from the first operation device, the information processing apparatus displays a direction display indicating a moving direction of the first figure. The display method according to claim 4. **Claim 10** The first operation device receives an operation of a first mode from a user, The second operation device receives an operation of a second mode different from the operation of the first mode from the user. The display method according to claim 9. **Claim 11** The first operation device is a mouse, The second operation device is a plurality of cursor keys included in a keyboard, The third figure moves by a drag operation on the mouse, The direction display corresponds to the direction indicated by the cursor keys. The display method according to claim 9. **Claim 12** The second image includes: a first area where the movement operation is received and the direction display and the second figure are displayed; and a second area where the movement amount display is displayed. The display method according to claim 3. **Claim 13** Displaying the second image includes: While moving the position of the first figure in the first image, the information processing apparatus displays a fourth figure having at least one of a shape or a color different from that of the second figure instead of the second figure. The display method according to any one of claims 1 to 8. **Claim 14** An information processing apparatus including one or more processors for controlling a display device, wherein the one or more processors display a second image including a second figure corresponding to a first figure included in a first image projected from a projection device on the display device; When a movement operation for moving the position of the first graphic is received, associate a movement display indicating that the first graphic is moving in the first image with the second graphic corresponding to the first graphic selected by the movement operation, and display the movement display on the second image; send a projection instruction for the moved first graphic to the projection device; An information processing apparatus that executes the above.

15. A processor mounted on an information processing apparatus that controls a display device causes a second image including a second graphic corresponding to a first graphic included in a first image projected from a projection device to be displayed on the display device; when a movement operation for moving the position of the first graphic is received, associate a movement display indicating that the first graphic is moving in the first image with the second graphic and display the movement display on the second image; send a projection instruction for the moved first graphic to the projection device; A program that causes the above to be executed.

Images