Terminal device, projection system, and program

Abstract

To provide a terminal device configured to efficiently execute a series of processing from projection of a measurement image to correction of a projection image, a projection system, and a program.SOLUTION: A portable terminal 20 which is a terminal device includes an imaging unit 26 and a control unit 21 which is at least one processor. The control unit 21 issues an instruction to an imaging unit 26 to capture a measurement image 41 projected by a projection device 10, acquires a correction image which is obtained by the imaging unit 26 imaging the measurement image, and generates instruction data for correcting a projection image to be projected by the projection device 10 on the basis of the correction image.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a projection control device, a projection system, and a program.

Background Art

[0002] Conventionally, as in Patent Document 1, a measurement image of a checkered pattern is projected onto a projection surface with irregularities such as a curtain, and correction information of an image is generated based on a correction image obtained by imaging the projected measurement image, so that a rectangular projection can be performed even on a projection surface with irregularities. There was a projection device that could do this.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, conventionally, the procedure was to project a correction image from a PC (personal computer) onto a projection device, capture an image with a digital camera, send the captured image to the PC, and correct the projection image on the PC. Therefore, since the instructions for each process from the projection of the measurement image to the correction of the projection image are performed by different devices, the series of processes takes time.

[0005] An object of the present invention is to provide a terminal device, a projection system, and a program capable of efficiently executing a series of processes from the projection of a measurement image to the correction of a projection image.

Means for Solving the Problems

[0006] The terminal device of the present invention includes an imaging unit and A first communication unit that communicates with a projection device; at least one processor, and the at least one processor Based on the position information of the first measurement point of the terminal device imaged by the imaging unit and the first captured image that is an image captured at the first measurement point, obtain first correction information that is correction information at the first measurement point of the image projected by the projection device. Based on the position information of the second measurement point of the terminal device different from the first measurement point imaged by the imaging unit and the second captured image that is an image captured of the measurement image projected by the projection device at the second measurement point, obtain second correction information that is correction information at the second measurement point of the image projected by the projection device. Obtain a prediction point that is a position different from the first measurement point and the second measurement point and is a position where the image projected by the projection device is visually recognized. Using weighted averaging for the first correction information and the second correction information, obtain third correction information that is correction information at the prediction point of the image projected by the projection device, and output the third correction information to the projection device via the first communication unit; a measurement image projected by a projection device, above It is characterized by the following.

[0007] The projection system of the present invention Second is the above-described terminal device and a communication unit that communicates with the terminal device and correct the projected image based on the third correction information obtained via the second communication unit; Communication unit and the corrected Image correction processing unit A projection device comprising; and a projection unit that projects a projection image, Computer and is characterized by having the following.

[0008] The program of the present invention is a program executed by a computer including an imaging unit and a first communication unit that communicates with a projection device. The computer is configured to, based on the position information of a first measurement point of the above Computer and a first captured image that is an image of a measurement image projected by the projection device at the first measurement point, acquire first correction information that is correction information at the first measurement point of the image projected by the projection device. Based on the position information of a second measurement point of the above

Figure 1

Advantages of the Invention

[0009] According to the present invention, it is possible to provide a terminal device, a projection system, and a program that can efficiently execute a series of processes from projection of a measurement image to correction of a projection image.

Brief Description of the Drawings

[0010]

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments for carrying out the present invention will be described. FIG. 1 shows a state in which a projection system 1 including a projection device 10 equipped with a projection control device 30 and a mobile terminal 20 in a room projects a projection image with a part of a curtain 2 as a projection surface 3. According to the projection system 1 according to the present embodiment, when projecting onto a projection surface 3 (curtain 2) that curves irregularly like a wave, a distorted projection image (also referred to as a projection video including still images and moving images) P1 as shown by the two-dot chain line in FIG. 1 is obtained. This can be corrected to a rectangular projection image P2 as shown by the solid line. And even if the position where the user U views the projection image P2 moves, the projection image is immediately corrected, and the rectangular projection image P2 can be viewed.

[0012] FIG. 2 is a control block diagram including a projection control device 30 provided in the projection device 10. The projection control device 30 includes a CPU (also referred to as a processor or a computer) including a control unit 31, an image correction processing unit 34, and a measurement processing unit 35, a front-end unit including an image input unit 33, and a formatter unit including a projection image processing unit 37. Image signals of various standards input from the image input unit 33 are converted via a system bus SB to be unified into an image signal in a predetermined format suitable for display by an image conversion unit of the control unit 31, and then processed by the image correction processing unit 34 and the measurement processing unit 35 described later are executed and output to the projection image processing unit 37.

[0013] The projection image processing unit 37 drives a display element 50 corresponding to the output image processed by the image correction processing unit 34 and the measurement processing unit 35, and controls the light source device 60 so that light in a predetermined wavelength band required at the time of image generation is emitted from the light source device 60. The projection optical system 15 can emit the image light generated by the display element 50 as projection light. The projection optical system 15 includes a fixed lens and a movable lens, and can perform focus adjustment by a motor 16. A projection unit 11 configured to be able to project image light includes the projection image processing unit 37, the display element 50, the light source device 60, and the projection optical system 15. Note that the display element 50 can be a DMD (Digital Micromirror Device) including a plurality of micromirrors.

[0014] The operation unit 32 is provided with key switches and indicators capable of performing various operations and settings of the projection device 10 (projection control device 30). The operation signals of the operation unit 32 are transmitted to the control unit 31.

[0015] The control unit 31 is connected to the audio processing unit 38 via the system bus SB. The audio processing unit 38 includes a sound source circuit such as a PCM sound source, and converts audio data into analog during the projection mode and the playback mode, and drives the speaker 38a to amplify and play the sound.

[0016] Further, the projection control device 30 has a storage unit 40. The storage unit 40 is connected to the control unit 31, the image correction processing unit 34, and the projection image processing unit 37 via the system bus SB. The storage unit 40 is composed of, for example, an SSD (Solid State Drive) or an SRAM (Static Random Access Memory). In the storage unit 40, there are a measurement image 41 projected during correction, a control program 42 (program) that enables the control unit 31, the image correction processing unit 34, and the measurement processing unit 35 to function. Although details will be described later, there are an image (correction image) captured by the user U at an arbitrary position of the projected measurement image 41, measurement information 43 such as position information, and correction information 44 which is image correction information corresponding to the position where the user U captured the measurement image 41.

[0017] The communication unit 36 is, for example, a network interface controller (NIC), and is formed to be able to transmit and receive communication data to and from the communication unit 22 of the mobile terminal 20 via a wireless communication path. The communication unit 36 is connected to the control unit 31 and the storage unit 40 via the system bus SB.

[0018] FIG. 3 is a control block diagram of the mobile terminal 20 (terminal device). The mobile terminal 20 is here assumed to be a smartphone, but it can also be a tablet PC (Personal Computer), a notebook PC, etc. The mobile terminal 20 includes a control unit 21, a communication unit 22, a storage unit 23, a display unit 24, and an imaging unit 26, and each unit is connected by a bus 29. The control unit 21 is a processor that controls each unit. Also, the storage unit 23 is composed of a ROM (Read Only Memory) and a RAM (Random Access Memory). The processor of the control unit 21 expands and executes the application program stored in the ROM of the storage unit 23 in the RAM.

[0019] The communication unit 22 of the mobile terminal 20 is formed of, for example, an antenna and a transceiver circuit, and is formed to be able to transmit and receive communication data with the communication unit 36 of the projection control device 30 via a wireless communication path. The display unit 24 and the operation unit 25 can be configured by laminating a transparent touch panel, which is the operation unit 25, on the surface of a display panel composed of, for example, a liquid crystal display or an organic EL display, which is the display unit 24, to form a touch panel display having both a display function and an input function. The touch panel is configured by, for example, a capacitance method that senses the position touched by a finger or the like. The user U can operate the operation screen displayed on the display unit 24 via the operation unit 25 to operate the mobile terminal 20 and the projection device 10. Also, the mobile terminal 20 has an imaging unit 26 having a camera function, can image the image projected on the projection surface 3, and can store this imaged image in the storage unit 23.

[0020] The projection method for correcting and projecting the image so that it appears rectangular, as if it were projected onto a smooth screen, even when the user U views the image projected onto the projection surface 3 from any position (prediction point, see FIG. 1), is outlined as follows: first, the projection unit 11 of the projection device 10 projects a measurement image 41 (see FIG. 2) onto the projection surface 3; the measurement image 41 projected from a measurement point (viewpoint 1, also referred to as the first point) on the left side of the projection surface 3 shown in FIG. 1 is captured to calculate correction information 44 (see FIG. 2); similarly, the measurement image 41 is captured from a measurement point (viewpoint 2, also referred to as the second point) on the right side of the projection surface 3 to calculate correction information 44; and a weighted average of these correction information 44 is used to calculate correction information 44 for the prediction point (i.e., the position where the user U views the projection image, also referred to as the third point), thereby correcting the projection image. Note that the second point is different from the first point, and the third point is different from the first and second points.

[0021] Here, each piece of correction information 44 calculated corresponding to viewpoint 1 and viewpoint 2 (referred to as correction information 44-1 (first correction data) and 44-2 (second correction data)) is correction information 44 for making corrections so that the projection surface 3 appears rectangular when viewed from each viewpoint. The calculation of correction information 44 at each viewpoint can use a known correction method that is performed based on measurement information 43 including a captured image (hereinafter also simply referred to as a "correction image") obtained by capturing the projected measurement image 41.

[0022] An example of calculation of correction information 44 at each viewpoint will be described below with reference to the flowchart in Fig. 4. In the process of calculating correction information 44, first, in step S103, the projection unit 11 of the projection control device 30 projects a measurement image 41 stored in the storage unit 40 of the projection control device 30 onto the projection surface 3. Here, the measurement image 41 may be an image of a checkered grid pattern, a horizontal stripe pattern, a vertical stripe pattern, or the like, and multiple measurement images 41 with different grid or stripe widths may be used.

[0023] Next, in step S105, the projected measurement image 41 is captured, and a captured image (correction image) is generated. In the projection system 1, the imaging unit 26 of the mobile terminal 20 captures an image, and the captured image (correction image) is stored in the storage unit 23 of the mobile terminal 20. The correction image stored in the storage unit 23 of the mobile terminal 20 is transmitted from the storage unit 23 of the mobile terminal 20 to the projection control device 30 via the communication unit 36 of the projection control device 30 and stored in the storage unit 40 as measurement information 43.

[0024] Next, in step S107, the image correction processing unit 34 calculates the pixel position (Pi) of the display element 50 corresponding to the pixel position (Si) of the captured image (correction image). As shown in FIG. 5, this is a process indicated by an arrow S107a for obtaining the coordinates Pi of the display element 50 as a DMD (Digital Micromirror Device) projecting the measurement image 41 with respect to the coordinates Si of the pixel positions of the captured image (correction image) 26a that has captured the projected measurement image 41.

[0025] Next, in step S109, the image correction processing unit 34 calculates the pixel position (Si) of the captured image (correction image) corresponding to the pixel position of the output video (or output image, that is, the measurement image 41). This is the process indicated by an arrow S109a shown in FIG. 5. The calculation of the pixel position (Si) of the captured image (correction image) corresponding to the pixel position of the output video uses the correspondence relationship obtained in step S107. Through steps S107 and S109, the positional correspondence relationship between the pixels of the captured image (correction image) and the output video is obtained, and correction information 44 is calculated. In other words, the correction information 44 is calculated using the coordinates Pi of the display element 50 and the coordinates Si of the captured image.

[0026] Next, in step S111, a cutout rectangle is determined from the captured image (image for correction). That is, as shown in FIG. 6, among the ranges of the projected image (measurement image 41 in FIG. 6) that is deformed and projected, a rectangle S111a with an aspect ratio corresponding to the display element 50 is a range that can be projected as a rectangle by the projection device 10 (projection unit 11). The image correction processing unit 34 determines this projectable rectangle range from the measurement image 41 of the captured image (image for correction) 26a.

[0027] Next, in step S113, the image correction processing unit 34 deforms the input video that fits within the cutout rectangle frame based on the correction information 44 on the shape of the projected image. That is, the projection image for the user U to view is deformed using the calculation results (correction information 44) of steps S107 and S109 so as to fit within the range of the rectangle S111a determined in step S111. The calculation of the correction information 44 is based on the fact that the correction image 50-41 of the display element 50 in FIG. 6 is deformed and projected like the correction image 3-41 on the projection surface 3, so in order to project it as a rectangle like S111a on the projection surface 3, it is only necessary to project a deformed image S113a on the display element 50.

[0028] Therefore, for the pixel coordinates of the projected image (i.e., the coordinates of the display element 50), the coordinates Pj of the deformed display element 50 (of the projected image) relative to it are calculated, and further, the coordinates Pk of the display element 50 corresponding to the coordinates Pj of the deformed display element 50 are calculated.

[0029] In this way, the image correction processing unit 34 can calculate the correction information 44 for it to look like a nice rectangle when viewing the projection surface 3 from each viewpoint. This correction information 44 can be calculated for each measurement point (viewpoints 1, 2). At viewpoint 1, correction information 44-1 is calculated, and at viewpoint 2, correction information 44-2 is calculated.

[0030] Then, the measurement processing unit 35 calculates correction information 44-F (third correction data) of the prediction point using a weighted average of each correction information 44 (correction information 44-1 (first correction data), 44-2 (second correction data)) at a plurality of different viewpoints and the position of the prediction point (any position from which the user U views the projection surface 3, third position data) relative to the positions (first position data, second position data) of each measurement point (viewpoint 1, viewpoint 2). As shown in FIG. 7, for example, the predicted deformation (calculation of correction information 44-F at prediction point 1, also called predicted correction) at a position where prediction point 1 divides the distance between viewpoint 1 and viewpoint 2 at a ratio of l1:l2 is calculated using equation (1). P=(l1P2+l2P1) / (l1+l2) ···(1)

[0031] Furthermore, to perform prediction transformation for prediction point 2, which is externally divided by l1:l2, Equation (2) is used, and to perform prediction transformation for prediction point 3, Equation (3) is used. Prediction point 2;l1>l2P=(-l2P1+l1P2) / (l1-l2) ···(2) Prediction point 3;l1 <l2P=(l2P1-l1P2) / (-l1+l2) ···(3)

[0032] Here, P, P1, and P2 in equations (1) to (3) are any of the coordinates Si, Pi, Pj, and Pk used when calculating the correction information 44-1 and 44-2. In other words, the weighted average is applied to the coordinates (Pi, Pj, Pk) of the display element 50 or the coordinate (Si) of the captured image (correction image) 26a. In summary, the weighted average can be performed in the following four cases: (A) Coordinates (Si) of the captured image 26a relative to the coordinates of the display element 50 (B) Coordinates (Pi) of the display element 50 relative to coordinates (Si) of the captured image 26a (C) Coordinates (Pj) of the display element 50 after deformation relative to the coordinates of the display element 50 (D) Coordinates (Pk) of the display element 50 relative to the coordinates (Pj) of the display element 50 after deformation

[0033] The captured image 26a captured from each viewpoint and position information of each viewpoint (each measurement point) are stored in the storage unit 23 as measurement information 43. The predicted point and the position information of each viewpoint can be acquired and determined from GPS information output from a GPS receiving unit included in the mobile terminal 20, angle information at the time of capturing the captured image 26a detected by an acceleration sensor included in the mobile terminal 20, or settings of the user U (user settings).

[0034] Thus, the projection method for calculating correction information 44-F (third correction data) at the predicted point (third location) and correcting the projection image includes the steps of: acquiring a first captured image (correction image) captured at a first location (viewpoint 1) and a second captured image (correction image) captured at a second location (viewpoint 2) via the communication unit 36; acquiring first correction data (correction information 44-1) based on the first captured image and acquiring second correction data (correction information 44-2) based on the second captured image; acquiring first position data of the first location, second position data of the second location, and third position data of a third location (prediction point); determining third correction data (correction information 44-F) corresponding to the third location using the first correction data, second correction data, and first to third position data; and correcting the projection image to be projected by the projection device 10 based on the determined third correction data.

[0035] The third correction data is calculated using a weighted average of the first correction data and the second correction data from the third position data for the first position data and the second position data. The steps of acquiring the first correction data based on the first captured image and acquiring the second correction data based on the second captured image include the steps of: determining coordinates of the display element for coordinates of pixel positions in the first captured image and the second captured image; and calculating the first correction data and the second correction data using the coordinates of the first captured image and the second captured image corresponding to the pixel positions of the output image using this correspondence. The step of determining the third correction data corresponding to the third point using the first correction data, second correction data, first position data, second position data, and third position data applies a weighted average to the coordinates of any of (A) to (D) above.

[0036] Specifically, the correction information 44-F (prediction correction, third correction data) of the prediction point is calculated according to the flowchart shown in FIG. 8. When the prediction correction process is started, at step S203, at viewpoint 1, the mobile terminal 20 captures the measurement image 41 projected onto the projection surface 3, and the measurement information 43-1 (see FIG. 1) including the captured image (correction image) is stored in the storage unit 40 of the projection control device 30. Next, at step 205, the image correction processing unit 34 calculates correction information 44-1 (first correction data) from the measurement information 43-1 (including the first captured image and the first position data) of viewpoint 1 and stores it in the storage unit 40. Then, at step S206, based on the correction information (first correction data) of viewpoint 1, the projection unit 11 projects the corrected video corresponding to viewpoint 1.

[0037] At steps S207 and 209, similar to steps S203 and 205, the measurement at viewpoint 2 is performed (step S203), and the correction information 44-2 (second correction data) (see FIG. 1) of viewpoint 2 is calculated from the measurement information 43-2 (including the second captured image and the second position data) of viewpoint 2 and stored in the storage unit 40 (step S205). Then, at step S210, based on the correction information (second correction data) of viewpoint 2, the projection unit 11 projects the corrected video corresponding to viewpoint 2. Then, at step S211, the position information (third position data) of the prediction point is detected, transmitted to the projection control device 30, and stored in the storage unit 40. Next, at step S213, the measurement processing unit 35 calculates the correction information 44-F (third correction data) (prediction correction) of the prediction point using weighted average for the correction information 44-1 of viewpoint 1 and the correction information 44-2 of viewpoint 2. The measurement processing unit 35 stores the calculated correction information 44-F of the prediction point in the storage unit 40 at step S215.

[0038] Using the correction information 44-F of the predicted points calculated in this way, the image correction processing unit 34 deforms the input image, and based on the projected image after the correction deformation, the light source device 60 and the display element 50 of the projection unit 11 are controlled by the projection image processing unit 37 of the projection device 10, and the projection light of the projection image is projected from the projection optical system 15. Then, even if the user U moves to an arbitrary position even with an image projected on the projection surface 3 having an irregular surface, the user U can immediately see a clean rectangular projection image.

[0039] The operations of projecting and imaging the correction image at each measurement point, setting the predicted point position, and calculating the correction information can be performed by the mobile terminal 20. FIG. 9 shows an example of a user interface screen 300 (UI screen 300) displayed on the display unit 24 (operation unit 25) of the mobile terminal 20. The "Connect" button 301 and the "Disconnect" button 302 at the upper left of the UI screen 300 are buttons for performing connection and disconnection via the communication units 22 and 36 with the projection device 10 (projection control device 30).

[0040] A camera screen 320 is displayed on the left side of the UI screen 300. The camera screen 320 shows an image captured by the imaging unit 26 of the mobile terminal 20. A measurement correction operation unit 310 is provided at the upper right of the UI screen 300. The measurement correction operation unit 310 includes a measurement unit 311 having a "Viewpoint 1 Measurement" button 311-1 and a "Viewpoint 2 Measurement" button 311-2 (the above are the first operation units), a correction unit 312 having a "Viewpoint 1 Correction" button 312-1 and a "Viewpoint 2 Correction" button 312-2 (the above are the second operation units), a "Viewpoint +" button 315, and a "Correction +" button 316. The control unit 21 instructs the imaging unit 26 to capture the measurement image projected by the projection device 10 according to the operation of the first operation unit, and acquires the correction image captured by the imaging unit 26. In addition, the control unit 21 generates instruction data for correcting the projection video projected on the projection device 10 based on the correction image according to the operation of the second operation unit.

[0041] 1, the control unit 21 transmits projection instruction information for causing the projection control device 30 to project a measurement image 41 via the communication units 22 and 36. The measurement image 41 is then emitted from the projection device 10 and projected onto the projection surface 3. The imaging unit 26 then captures the projected measurement image 41 and stores it in the storage unit 23. Similarly, when the user U presses the "measure viewpoint 2" button 311-2 at the position of the viewpoint 2, the measurement image 41 is projected and captured, and stored in the storage unit 23.

[0042] Furthermore, for example, when user U presses the "Viewpoint 1 Correction" button 312-1, instruction data is generated for correcting the projection image to be projected by projection device 10 based on the correction image and the position information of viewpoint 1, which are stored in storage unit 23. Then, mobile terminal 20 transmits this instruction data to projection device 10, and projection device 10 projects the projection image corrected at viewpoint 1.

[0043] A "Viewpoint +" button 315 is displayed to the right of the measurement unit 311. The "Viewpoint +" button 315 is a viewpoint increase unit that can further increase two viewpoints (viewpoints 1 and 2) (i.e., measurement unit 311). The "Correction +" button 316, located to the right of the correction unit 312, is a correction increase unit for forming the correction unit 312 corresponding to the viewpoint increased by the "Viewpoint +" button 315.

[0044] The "No correction" button 303 displayed in the approximate center on the right side of the UI screen 300 is used as a preview operation unit for previewing the capture of the projected measurement image 41 by the imaging unit 26 of the mobile terminal 20. Before capturing an image by the measurement unit 311, a preview can be performed to check whether the projected image fits within the angle of view. In this case, too, the capture status can be confirmed on the camera screen 320.

[0045] On the right side of the "No Correction" button 303, a "Settings" button 340 is displayed. The "Settings" button 340 can be used to set the measurement image 41 projected by the projection device 10. When the "Settings" button 340 is pressed, the display unit 24 switches to the UI screen 349 in FIG. 10. The "Dot Pattern Brightness" 341 at the upper stage of the UI screen 349 can adjust the brightness of the measurement image 41. The "Dot Pattern Pitch Width" 342 can set the pitch width of the stripe pattern of the measurement image 41 composed of a grid pattern (checkerboard pattern), horizontal stripes, or vertical stripes. The "Number of Dot Patterns" 343 can set the number (type) of the measurement images 41 to be projected. These settings can be made when an error occurs in the calculation of the measurement information 43 by the measurement unit 311.

[0046] Below the "No Correction" button 303 and the "Settings" button 340, a predicted point determination unit 330 is displayed. The predicted point determination unit 330 can determine the predicted point position for calculating the correction information of the predicted point by using the weighted average from the multiple correction information and the position of the predicted point relative to the position of the measurement point according to the operations of the combination of the first operation unit and the second operation unit provided in plurality corresponding to different measurement points. On the predicted point determination unit 330, a "Predicted Point Correction" button 331 and a predicted point position determination slider bar 332 are displayed. On the predicted point position determination slider bar 332, a point 332a indicating the position of viewpoint 1, a point 332b indicating the position of viewpoint 2, a line 332c connecting viewpoints 1 and 2, and a point 332d of the predicted point that can be slid and moved on the line 332c are displayed.

[0047] When it is desired to determine a prediction point (position) according to the settings of user U, it can be set by the prediction point positioning slider 332. The setting of the prediction point can be performed by moving the point 332d indicating the prediction point along the line 332c. Therefore, it is possible to determine whether the prediction point is at a position that internally divides or externally divides the measurement points (viewpoints 1, 2) (which side of the viewpoints 1, 2 to externally divide). The measurement processing unit 35 of the projection control device 30 determines whether to apply weighted averaging using any of the aforementioned formulas (1) to (3) based on the prediction point position set by the prediction point positioning slider 332. This operation is not necessary when the prediction point is based on GPS information output from the GPS receiver provided in the mobile terminal 20 or the angle information at the time of imaging of the captured image detected by the acceleration sensor provided in the mobile terminal 20.

[0048] After determining the prediction point by sliding the point 332d indicating the prediction point, by pressing the "prediction point correction" button 331, the position information (prediction point position) of the prediction point set by the prediction point positioning slider 332 is transmitted to the projection control device 30. The projection control device 30 calculates correction information 44-F at the prediction point using weighted averaging by the measurement processing unit 35 based on the received position information of the prediction point. The projection control device 30 corrects the projection image based on the correction information 44-F of the prediction point. When calculating the prediction point from GPS or the captured image, the "prediction point correction" button 331 may be blinked or the like as a display indicating that the prediction point positioning slider 332 cannot be operated, and by pressing the "prediction point correction" button 331, the measurement processing unit 35 may calculate the correction information 44-F at the prediction point using weighted averaging.

[0049] Although the embodiment of the present invention has been described above, the present invention is not limited to the present embodiment and can be implemented with various modifications. For example, the image correction processing unit 34 and the measurement processing unit 35 may be provided in the mobile terminal 20, and the mobile terminal 20 may calculate correction information 44 for each viewpoint (viewpoints 1 and 2) and prediction point. In this case, the measurement information 43 and the correction information 44 may be stored in the storage unit 23 of the mobile terminal 20. Furthermore, the projection device 10 may be a projector, and the projection control device 30 may be a PC (personal computer).

[0050] In addition, although the measurement points (viewpoints) are two in this embodiment, it is also possible to use three or more measurement points and calculate the correction information 44-F of the prediction point using a weighted average of the correction information 44 at these multiple measurement points.

[0051] According to the above embodiment, mobile terminal 20, which is a terminal device, includes imaging unit 26 and control unit 21, which is at least one processor, and control unit 21 instructs imaging unit 26 to capture measurement image 41 projected by projection device 10, acquires a correction image, which is an image obtained by imaging unit 26 capturing the measurement image, and generates instruction data for correcting the projection image to be projected by projection device 10 based on the correction image. This makes it possible to efficiently execute a series of processes from projecting the measurement image to correcting the projection image simply by operating mobile terminal 20.

[0052] Further, it includes a display unit 24, on which a first operation unit (a "Viewpoint 1 Measurement" button 311-1 and a "Viewpoint 2 Measurement" button 311-2) and a second operation unit (a "Viewpoint 1 Correction" button 312-1 and a "Viewpoint 2 Correction" button 312-2) are arranged side by side and displayed. The control unit 21 instructs the imaging unit 26 to capture the measurement image 41 projected by the projection device 10 according to the operation of the first operation unit, obtains the correction image captured by the imaging unit 26, and further, the control unit 21 generates instruction data for correcting the projection video projected onto the projection device 10 based on the correction image according to the operation of the second operation unit. Thereby, an image corrected for each viewpoint can be projected by the projection device 10 by operating the mobile terminal 20.

[0053] Also, it includes a communication unit 22 for communicating with the projection control device 30. Before the imaging unit 26 captures the measurement image 41, the control unit 21 transmits projection instruction information for causing the projection control device 30, which controls the projection of the image by the projection device 10, to project the measurement image 41. After the imaging unit 26 captures the measurement image 41, the control unit 21 transmits the correction image to the projection control device 30. When the second operation unit is operated, the control unit 21 transmits the instruction data to the projection control device 30 to cause the projection control device 30 to perform correction based on the correction information. Thereby, the projection of the corrected image for each viewpoint can be surely performed.

[0054] Also, on the display unit 24, a plurality of combinations of the first operation unit and the second operation unit are provided corresponding to different measurement points, and a prediction point position determination unit 330 for determining a prediction point position for calculating a correction information 44-F of the prediction point using a weighted average from the positions of the plurality of correction information 44-1, 44-2 and the prediction points corresponding to the positions of the measurement points (viewpoints 1, 2) according to the operations of each combination of the first operation unit and the second operation unit is displayed. Thereby, the user U can set the prediction point position. Therefore, by setting the prediction point position in advance and storing it in the storage unit 40 (or the storage unit 23), it is not necessary to set or detect the prediction point position each time projection is performed.

[0055] In addition, a prediction point positioning slider 332 formed to be slidable is displayed on the prediction point determination unit 330. As a result, an easy-to-understand slide operation is displayed, so that the user U can easily set the position of the prediction point.

[0056] In addition, a trial shooting operation unit including a "without correction" button 303 for trial shooting the imaging of the measurement image 41 projected by the imaging unit 26 is displayed on the display unit 24. As a result, appropriate imaging of the measurement image 41 can be performed.

[0057] Further, the projection system 1 includes a mobile terminal 20 and a communication unit 36 that communicates with the mobile terminal 20, and a projection control device 30 including an image correction processing unit 34 that acquires instruction data from the mobile terminal 20 and calculates correction information for the projection video, and a projection unit 10 that projects the projection video. Thereby, the projection system 1 that can project a projection video corrected for the uneven surface can be operated from the mobile terminal 20.

[0058] In addition, the projection control device 30 has a measurement processing unit 35 that calculates correction information for the prediction point using a weighted average from a plurality of correction information corresponding to different measurement points calculated according to the operations of the first operation unit and the second operation unit and the position of the prediction point with respect to the position of the measurement point. As a result, even when the position where the user U views the projection image is moved, a projection system 1 including a terminal device that can easily obtain information necessary for calculating the correction information 44 of the projection image can be provided.

[0059] In addition, the application program of the mobile terminal 20 displays a measurement correction operation unit 310 that operates the imaging of the projected measurement image 41 by the imaging unit 26 and the calculation of the correction information 44 of the projection image based on the captured measurement image 41. As a result, by operating only the mobile terminal 20, a series of processes from the projection of the measurement image to the correction of the projection image can be efficiently executed, and even when the position where the user U views the projection image is moved, a mobile terminal 20 having a UI screen 300 with simple operations necessary for calculating the correction information 44 of the projection image can be provided.

[0060] Note that the embodiments described above are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

[0061] The invention described in the first claim of the present application is appended below. [1] An imaging unit, At least one processor, Comprising, wherein the at least one processor Instructs the imaging unit to capture an image of a measurement image projected by a projection device, Obtains a correction image that is an image captured by the imaging unit of the measurement image, Generates instruction data for correcting a projection video to be projected onto the projection device based on the correction image. A terminal device characterized by the above. [2] Comprising a display unit, on which a first operation unit and a second operation unit are displayed side by side, The processor instructs the imaging unit to capture an image of the measurement image projected by the projection device according to an operation of the first operation unit, and obtains the correction image captured by the imaging unit. The processor generates the instruction data for correcting the projection video to be projected onto the projection device based on the correction image according to an operation of the second operation unit. The terminal device according to [1] above is characterized by this. [3] Comprising a communication unit for communicating with a projection control device, Before the processor causes the imaging unit to capture the measurement image, the processor transmits projection instruction information for causing the projection control device to control the projection of an image by the projection device to project the measurement image. After the processor causes the imaging unit to capture the measurement image, the processor transmits the correction image to the projection control device. When the second operation unit is operated, the processor transmits the instruction data to the projection control device to cause the projection control device to perform correction based on the correction information. The terminal device according to [2], characterized in that. [4] A plurality of combinations of the first operation unit and the second operation unit are provided on the display unit corresponding to different measurement points, and a plurality of the correction information corresponding to the operations of each combination of the first operation unit and the second operation unit and the position of the prediction point with respect to the position of each measurement point. The terminal device according to [3], characterized in that a prediction point position determination unit for determining the position of the prediction point for calculating the correction information of the prediction point using a weighted average is displayed. [5] The terminal device according to [4], characterized in that a prediction point position determination slide bar formed to be slidable is displayed on the prediction point position determination unit. [6] The terminal device according to any one of [2] to [5], characterized in that a trial shooting operation unit for trial shooting the imaging of the projected correction image by the imaging unit is displayed on the display unit. [7] The terminal device according to any one of [3] to [6], A projection control device including a communication unit that communicates with the terminal device, an image correction processing unit that acquires the instruction data from the terminal device and calculates correction information for a projection image, and a projection unit that projects a projection image. A projection system, characterized by having. [8] The projection control device according to [7], characterized by having a measurement processing unit that calculates the correction information of the prediction point using a weighted average from a plurality of the correction information corresponding to different measurement points calculated according to the operations of the first operation unit and the second operation unit and the position of the prediction point with respect to the position of each measurement point. [9] A program executed by a computer, the computer A program that causes a measurement correction operation unit for operating imaging by an imaging unit of a projected measurement image and calculating correction information of a projection image based on the captured correction image, which is the measurement image, to be displayed.

Explanation of Signs

[0062] 1 Projection system 2 Curtain 3 Projection surface 10 Projection device 11 Projection unit 15 Projection optical system 16 Motor 20 Mobile terminal 21 Control unit 22 Communication unit 23 Memory unit 24 Display unit 25 Operation unit 26 Imaging unit 26a Captured image 29 Bus 30 Projection control device 31 Control unit 32 Operation unit 33 Image input unit 34 Image correction processing unit 35 Measurement processing unit 36 Communication unit 37 Projection image processing unit 38 Audio processing unit 38a Speaker 40 Memory unit 41 Measurement image 42 Control program 43 Measurement information 44 Correction information 50 Display element 60 Light source device 300 User interface screen (UI screen) 310 Measurement correction operation unit 311 Measurement unit 312 Correction unit 320 Camera screen 330 Prediction point determination unit 332 Prediction point determination slide bar 349 UI screen U User

Claims

1. An imaging unit, a first communication unit that communicates with a projection device, and at least one processor, in a terminal device comprising the same, the at least one processor acquires first correction information, which is correction information at a first measurement point of an image projected by the projection device, based on position information of a first measurement point of the terminal device imaged by the imaging unit and a first captured image, which is an image capturing a measurement image projected by the projection device at the first measurement point, acquires second correction information, which is correction information at a second measurement point of an image projected by the projection device, based on position information of a second measurement point of the terminal device different from the first measurement point imaged by the imaging unit and a second captured image, which is an image capturing a measurement image projected by the projection device at the second measurement point, acquires a prediction point, which is a position different from the first measurement point and the second measurement point and is a position for visually recognizing an image projected by the projection device, acquires third correction information, which is correction information at the prediction point of an image projected by the projection device, by using a weighted average for the first correction information and the second correction information, and outputs the third correction information to the projection device via the first communication unit. A terminal device characterized by the above.

2. When the prediction point is at a position that internally divides the first measurement point and the second measurement point at a predetermined ratio, the processor acquires the third correction information by using a first algorithm. The terminal device according to claim 1, characterized by the above.

3. When the prediction point is at a position that externally divides the first measurement point and the second measurement point at a predetermined ratio, the processor acquires the third correction information by using a second algorithm different from the first algorithm. The terminal device according to claim 2, characterized by the above.

4. Comprising a display unit, the display unit displays a first point indicating the position of the first measurement point, a second point indicating the position of the second measurement point, a line connecting the first point and the second point, and a third point indicating the position of the prediction point movable on the line, and the processor acquires the position of the prediction point based on the position of the third point on the line. The terminal device according to any one of claims 1 to 3, characterized by the above.

5. The terminal device according to any one of claims 1 to 4, and A projection device comprising: a second communication unit that communicates with the terminal device; an image correction processing unit that corrects a projection image based on the third correction information acquired via the second communication unit; and a projection unit that projects the corrected projection image. A projection system characterized by having the above. **Claim 6** A program executed by a computer comprising an imaging unit and a first communication unit that communicates with a projection device, the computer being caused to acquire first correction information, which is correction information at a first measurement point of an image projected by the projection device, based on position information of a first measurement point of the computer at which imaging is performed by the imaging unit and a first captured image, which is an image of a measurement image projected by the projection device at the first measurement point; caused to acquire second correction information, which is correction information at a second measurement point of an image projected by the projection device, based on position information of a second measurement point of the computer, which is different from the first measurement point at which imaging is performed by the imaging unit, and a second captured image, which is an image of a measurement image projected by the projection device at the second measurement point; caused to acquire a prediction point, which is a position different from the first measurement point and the second measurement point and at which an image projected by the projection device is visually recognized; caused to acquire third correction information, which is correction information at the prediction point of an image projected by the projection device, by using a weighted average for the first correction information and the second correction information; A program that causes the third correction information to be output to the projection device via the first communication unit.

Images