Image adjustment system

Abstract

To provide an image adjustment system capable of improving convenience.SOLUTION: A disclosed image adjustment system includes: a video projection unit that projects images onto a projection target; an imaging unit that acquires a first image of a first area including part of the projected image, and a second image of the second area, which is a second area that includes another part of the projected image, and which includes overlapping areas where the first area and the second area overlap; and a control unit that controls the projection position of the projected image. The control unit is configured to generate a composite image of the first and second images based on the overlapping area and detect information about the projection object in the composite image, determine the position of multiple adjustment images based on the detected information on the projection target, and superimpose multiple adjustment images on the composite image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a video calibration system. [Background technology]

[0002] Cameras are sometimes used to correct projector images. When the projection range of a projector image is large, multiple cameras are used, each capturing an image of a different range of the projector image, and corrections such as adjusting the position of the projector image are made in each captured image.

[0003] For example, the image projection system described in Patent Document 1 unifies the coordinate systems of images captured by multiple image capturing devices, and performs geometric correction on the images projected by the multiple image projection devices based on the image projection range on the unified coordinate system. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2006 / 030501 Summary of the Invention [Problem to be solved by the invention]

[0005] The image projection system of Patent Document 1 still has room for improvement in terms of improving convenience.

[0006] The present disclosure provides a video adjustment system with improved convenience. [Means for solving the problem]

[0007] An image adjustment system according to one aspect of the present disclosure includes an image projection device that projects a projection image onto a projection target, an imaging device that acquires a first image that captures a first area that includes a portion of the projection image and a second image that captures a second area that includes another portion of the projection image, the second area including an overlap area where the first area and the second area overlap, and a control unit that controls the projection position of the projection image, wherein the control unit generates a composite image of the first image and the second image based on the overlap area, detects information about the projection target in the composite image, determines positions of multiple adjustment images based on the detected information about the projection target, and superimposes the multiple adjustment images on the composite image. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to provide a video adjustment system with improved convenience. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating a video adjustment system according to a first embodiment; [Figure 2] Block diagram showing the image adjustment system of FIG. 1. [Figure 3A] FIG. 1 is a diagram showing an example of a first image captured by an imaging device; [Figure 3B] FIG. 10 is a diagram showing an example of a second image captured by the imaging device. [Figure 3C] FIG. 10 is a diagram showing an example of a third image captured by the imaging device. [Figure 3D] FIG. 10 is a diagram showing an example of a fourth image captured by the imaging device. [Figure 4] A diagram showing a test pattern for feature point detection [Figure 5] FIG. 10 is a diagram showing an example of a composite image obtained by combining the first and fourth images and displayed on the display unit of the control unit. [Figure 6] Flowchart showing the operation of the image adjustment system [Figure 7] FIG. 1 is a diagram illustrating a modification of the first embodiment. [Figure 8] FIG. 10 is a schematic diagram illustrating a video calibration system according to a second embodiment. [Figure 9]10 is a flowchart showing the operation of the video calibration system according to the second embodiment. [Figure 10A] FIG. 1 is a diagram illustrating an example of a method for determining the cursor position. [Figure 10B] FIG. 1 is a diagram illustrating an example of a method for determining the cursor position. [Figure 10C] FIG. 1 is a diagram illustrating an example of a method for determining the cursor position. DETAILED DESCRIPTION OF THE INVENTION

[0010] (Background to the invention) The image from the projector is sometimes captured by a camera, and the projector image is adjusted based on the captured image. For example, when projecting an image onto a wall or screen installed outdoors, the captured image may be displayed on the screen of a PC or other device indoors, rather than on-site, and the projection range of the image may be adjusted.

[0011] When the projection range of an image is large, the image cannot fit within the angle of view of a single camera, so methods are being considered that use multiple cameras, each capturing a different part of the image and adjusting the image using each image.

[0012] In this case, since each image displays only a portion of the image, it is difficult to grasp the entire image, and it is difficult to intuitively adjust the projector while looking at the image. In particular, when the projector is installed outdoors and the PC or other device is installed indoors, it is difficult to visually check the actual projected image, making it difficult to intuitively operate the projector using only the captured image.

[0013] Another problem is that the difference between the resolution of the projector and the resolution of the camera makes it difficult to accurately calculate pixel alignment between the camera and the projector.

[0014] Therefore, the inventor(s) have studied an image adjustment system that allows intuitive operation, and have arrived at the following invention.

[0015] A video adjustment system according to a first aspect of the present disclosure includes a video projection device that projects a projection image to be projected onto a projection target, an imaging device that acquires a first image that captures a first area that includes at least a portion of the projection image, and a second image that captures a second area that includes at least a portion of the projection image and includes an area that overlaps with the first area, and a control unit that controls the projection position of the projection image, wherein the control unit converts a first coordinate system of the first image and a second coordinate system of the second image into a composite coordinate system common to the first image and the second image, generates correction information in the composite coordinate system that includes position information that indicates the projection range of the video projection device, and generates the projection image based on the correction information.

[0016] With this configuration, it is possible to provide a video adjustment system with improved convenience.

[0017] In the image adjustment system according to the second aspect of the present disclosure, the control unit may correct the projected image by converting coordinates indicating position information in the composite coordinate system into a first coordinate system and a second coordinate system, and further converting from the first coordinate system and the second coordinate system into a projector coordinate system in the image projection device.

[0018] With this configuration, it is possible to provide an image adjustment system that is more convenient and has improved accuracy in aligning the projection range.

[0019] In the image adjustment system according to the third aspect of the present disclosure, the control unit may generate a composite image of the first image and the second image based on the overlapping portion between the first area and the second area in the first image and the second image, and the composite coordinate system may indicate coordinates in the composite image.

[0020] With this configuration, correction can be performed while grasping the entire image on the composite image.

[0021] In the image adjustment system according to the fourth aspect of the present disclosure, the control unit may superimpose an adjustment image indicating the projection range on the composite image.

[0022] With this configuration, correction can be performed while grasping the entire image on the composite image.

[0023] In the image adjustment system according to the fifth aspect of the present disclosure, the image projection device may project an adjustment image indicating the projection range onto the projection target.

[0024] Since the projection range is adjusted using the image projected from the video projection device, it is possible to prevent misalignment caused by the difference in resolution between the video projection device and the imaging device.

[0025] In the image adjustment system according to the sixth aspect of the present disclosure, the control unit may determine the projection range based on the first image and the second image.

[0026] According to this configuration, the projection range can be adjusted without any operation by the user, thereby improving convenience.

[0027] An image adjustment method according to a seventh aspect of the present disclosure includes the steps of obtaining a first image capturing a first area of ​​a projection image projected onto a projection target and a second image capturing a second area including at least a portion of the projection image and including an area overlapping with the first area; converting a first coordinate system of the first image and a second coordinate system of the second image into a composite coordinate system common to the first image and the second image; generating correction information including position information indicating the projection range of the projection image in the composite coordinate system; and generating the projection image based on the correction information.

[0028] With this configuration, it is possible to provide an image adjustment method that is more convenient.

[0029] In the image adjustment method according to the eighth aspect of the present disclosure, the step of generating correction information including position information indicating the projection range of the projected image in the composite coordinate system may include converting coordinates indicating the position information in the composite coordinate system into a first coordinate system and a second coordinate system, and further converting from the first coordinate system and the second coordinate system into the coordinate system of the image projection device that projects the projected image.

[0030] With this configuration, it is possible to provide an image adjustment method that improves the accuracy of aligning the projection range while improving convenience.

[0031] In the image adjustment method according to the ninth aspect of the present disclosure, the step of converting the first coordinate system and the second coordinate system into a composite coordinate system common to the first image and the second image may include generating a composite image of the first image and the second image based on overlapping portions between the first region and the second region in the first image and the second image.

[0032] With this configuration, correction can be performed while grasping the entire image on the composite image.

[0033] A tenth aspect of the present disclosure provides an image adjustment device that generates a projection image projected onto a projection target by an image projection device, and includes: an image acquisition unit that acquires a first image that captures a first area of ​​the projection image; and a second image that captures a second area that includes at least a portion of the projection image and an area that overlaps with the first area; a coordinate conversion unit that converts a first coordinate system of the first image and a second coordinate system of the second image into a composite coordinate system common to the first image and the second image; a correction information generation unit that generates correction information in the composite coordinate system that includes position information that indicates the projection range of the image projection device; and an image generation unit that generates the projection image based on the correction information.

[0034] With this configuration, it is possible to provide a video adjustment device with improved convenience.

[0035] In the image adjustment device according to the eleventh aspect of the present disclosure, the correction information generation unit may generate the correction information by converting coordinates indicating position information in the composite coordinate system into a first coordinate system and a second coordinate system, and further converting from the first coordinate system and the second coordinate system into a projector coordinate system in the image projection device.

[0036] With this configuration, it is possible to provide an image adjustment device that is more convenient and has improved accuracy in aligning the projection range.

[0037] In the image adjustment device according to the twelfth aspect of the present disclosure, the image generation unit may generate a composite image of the first image and the second image based on the overlapping portion between the first region and the second region in the first image and the second image.

[0038] With this configuration, correction can be performed while grasping the entire image on the composite image.

[0039] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of well-known matters or redundant explanation of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0040] The inventors have provided the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims.

[0041] (Embodiment 1) [Overall configuration] Fig. 1 is a schematic diagram showing a video calibration system 1 according to a first embodiment. Fig. 2 is a block diagram showing the video calibration system 1 of Fig. 1. The video calibration system 1 will be described with reference to Figs. 1 and 2.

[0042] The image calibration system 1 includes image projection devices 11 to 15, imaging devices 21 to 24, and a control unit 31.

[0043] The image projection devices 11 to 15 are devices that project images generated based on input image signals through projection lenses. The image projection devices 11 to 15 are capable of transmitting and receiving data or information such as image signals to and from a control unit 31, which will be described later. The image projection devices 11 to 15 generate images based on the image signals input from the control unit 31, and output projection light (for example, visible light) for projection onto a projection target such as a screen or a wall.

[0044] In this embodiment, as shown in FIG. 1, one image Im is projected by five image projectors 11-15. Specifically, the image projectors 11-15 are arranged in a row horizontally, and each image projector 11-15 projects a different part of the image Im. The image projector 11 projects an image Im1 that is a part of the image Im. The image projector 12 projects an image Im2 that is a part of the image Im. The image projector 13 projects an image Im3 that is a part of the image Im. The image projector 14 projects an image Im4 that is a part of the image Im. The image projector 15 projects an image Im5 that is a part of the image Im. The images Im1-Im5 are simultaneously projected onto the projection target by the image projectors 11-15, thereby displaying one image Im on the projection target.

[0045] In this embodiment, the image projection devices 11 to 15 are arranged so that adjacent images, for example, image Im1 and image Im2, have overlapping portions, but adjacent images do not necessarily have to have overlapping portions.

[0046] The imaging devices 21 to 24 capture an area including at least a portion of the projected image Im. The imaging device 21 captures a first area R1 including images Im1 and Im2. The imaging device 22 captures a second area R2 including images Im2 and Im3. The imaging device 23 captures a third area R3 including images Im3 and Im4. The imaging device 24 captures a fourth area R4 including images Im4 and Im5. The first area R1 includes at least a portion of the image Im. The second area R2 includes at least a portion of the image Im and includes an area R5 (overlapping area) that overlaps with the first area R1. Similarly, the third area R3 includes at least a portion of the image Im and includes an area R6 that overlaps with the second area R2. Furthermore, the fourth area R4 includes at least a portion of the image Im and includes an area R7 that overlaps with the third area R3.

[0047] The control unit 31 controls the image projection devices 11-15 and the imaging devices 21-24 to control the projection position of the image Im. In this embodiment, the control unit 31 includes an image acquisition unit 32, a coordinate conversion unit 33, a correction information generation unit 34, and an image generation unit 35.

[0048] The control unit 31 includes a general-purpose processor such as a CPU or MPU that executes a program to achieve predetermined functions. The control unit 31 also includes a storage unit (not shown). The control unit 31 executes the functions of the image acquisition unit 32, coordinate conversion unit 33, correction information generation unit 34, and video generation unit 35, for example, by calling and executing a control program stored in the storage unit. The control unit 31 is not limited to a unit that executes predetermined functions through the cooperation of hardware and software, but may also be a hardware circuit specifically designed to execute the predetermined functions. In other words, the control unit 31 can be implemented using various processors, such as a CPU, MPU, GPU, FPGA, DSP, or ASIC.

[0049] 2, a display unit 36 ​​such as a liquid crystal display or an input unit 37 such as a keyboard and mouse may be connected to the control unit 31. Alternatively, the control unit 31 may include the display unit 36 ​​or the input unit 37. In this case, images acquired by the imaging devices 21 to 24 can be displayed on the display unit 36 ​​so that the user can check them.

[0050] The control unit 31 can be installed in an electronic device such as a PC. The electronic device equipped with the control unit 31 can be connected to the image projection devices 11-15 and the image capture devices 21-24 via, for example, a wireless or wired network. Alternatively, some of the functions of the control unit 31 may be installed in the image projection devices 11-15.

[0051] The control unit 31 corresponds to the "image adjustment device" of the present disclosure.

[0052] The image acquisition unit 32 controls the imaging devices 21 to 24 to acquire a first image 41 to a fourth image 44 of the video Im. FIGS. 3A to 3D are diagrams showing examples of the first image 41 to the fourth image 44 captured by the imaging devices. For example, the image acquisition unit 32 acquires a first image 41 captured by the imaging device 21 of a first region R1 (see FIG. 1) including the video Im1 and the video Im2. Similarly, the image acquisition unit 32 acquires a second image 42 (FIG. 3B) captured by the imaging device 22 of a second region R2, a third image 43 (FIG. 3C) captured by the imaging device 23 of a third region R3, and a fourth image 44 (FIG. 3D) captured by the imaging device 24 of a fourth region R4.

[0053] The coordinate conversion unit 33 converts the first coordinate system of the first image 41, the second coordinate system of the second image 42, the third coordinate system of the third image 43, and the fourth coordinate system of the fourth image 44 into a composite coordinate system that is common to all of the first coordinate system to the fourth coordinate system.

[0054] FIG. 4 is a diagram showing a feature point detection test pattern 51. The test pattern 51 shown in FIG. 4 is projected onto a projection target from each of the video projectors 11 to 15. The imaging devices 21 to 24 capture first to fourth images 41 to 44, each including the test pattern 51. The coordinate conversion unit 33 can generate a composite coordinate system based on the first to fourth images 41 to 44, which are captured images of the test pattern 51. Note that the test pattern is not limited to the one shown in FIG. 4, and any test pattern can be used as long as it can generate a composite image system.

[0055] Specifically, the coordinate conversion unit 33 detects multiple feature points contained in the test pattern 51 from each of the images 41 to 44 captured by the adjacent imaging devices 21 to 24, and calculates the coordinates of the feature points in each of the coordinate systems (first to fourth coordinate systems). For example, the overlapping region R5 in each of the first image 41 and the second image 42 includes the same feature points projected from the same video projector 12. A coordinate conversion equation is calculated to convert the coordinates of the same feature points in each of the first and second coordinate systems into coordinates in a coordinate system common to both the first and second coordinate systems. The coordinate conversion equation can be calculated, for example, by a method of calculating a planar projection transformation matrix using four or more pairs of correspondences between the coordinates in the first and second coordinate systems obtained from the same feature points projected from the same video projector.

[0056] Similarly, for the other images 42 to 44, identical feature points included in the overlapping regions R6 to R7 of the images are detected, and coordinate transformation formulas for transforming from the second to fourth coordinate systems to a common coordinate system are calculated. Using the coordinate transformation formulas from each of the first to fourth coordinate systems to the composite coordinate system, coordinates can be transformed from each coordinate system to the composite coordinate system.

[0057] 5 is a diagram showing an example in which a composite image 45 obtained by combining first to fourth images 41 to 44 is displayed on the display unit 36 ​​of the control unit 31. The control unit 31 may generate the composite image 45 of the first to fourth images 41 to 44 using the coordinate conversion formula described above, and display the composite image 45 on the display unit 36 ​​as shown in FIG. 5. The user can specify the projection range of the video projectors 11 to 15 on the composite image 45 displayed on the display unit 36.

[0058] For example, as shown in FIG. 5, the control unit 31 may display adjustment images C01 to C12 on a composite image 45 that indicates the projection range of each of the video projectors 11 to 15. The adjustment images C01 to C12 are, for example, images that indicate the four corners of the projection range of each of the video projectors 11 to 15, and as shown in FIG. 5, cross cursors can be used as the adjustment images C01 to C12. Hereinafter, the adjustment images C01 to C12 may also be referred to as cursors C01 to C12. In this embodiment, the control unit 31 determines the number of cursors C01 to C12 according to the number of images Im1 to Im5 (the number of video projectors 11 to 15). Specifically, since the number of images Im1 to Im5 is five, the number of cursors C01 to C12 is determined to be 12 (= 5 × 2 + 2). The control unit 31 generates 12 cursors C01 to C12 and superimposes the cursors C01 to C12 on the composite image 45. The number of cursors may be more or less than the number determined by the above method, and may be freely changeable by the user.

[0059] 5, in this embodiment, cursors C01, C02, C07, and C08 indicate the four corners of image Im1 of image projection device 11. The user can freely specify the projection range of image Im1 by moving cursors C01, C02, C07, and C08 using an input unit (not shown) of control unit 31. Similarly, the user can freely specify the projection range of images Im2 to Im5.

[0060] The correction information generation unit 34 generates correction information including position information indicating the projection range of the video projection device in the composite coordinate system. The correction information is information including coordinates obtained by converting the position information indicating the projection range in the composite coordinate system into the projector coordinate systems of each of the video projection devices 11 to 15. The projection range of the video projection device is specified by the user using coordinates in the composite coordinate system in the composite image 45. The projection range is indicated, for example, by coordinates in the composite coordinate system that indicate the positions of cursors C01 to C12 shown in FIG. 5. The correction information generation unit 34 converts the coordinates in the composite coordinate system that indicate the projection range into coordinates in each of the first to fourth coordinate systems using the coordinate conversion formula calculated by the coordinate conversion unit 33. Furthermore, the correction information generation unit 34 obtains a coordinate conversion table that converts each of the first to fourth coordinate systems into each of the projector coordinate systems of the video projection devices 11 to 15, based on the correspondence between each of the first to fourth coordinate systems and each of the projector coordinate systems of the video projection devices 11 to 15.

[0061] Based on the coordinate conversion table, the correction information generation unit 34 converts the coordinates indicating the projection range from each of the first to fourth coordinate systems into the projector coordinate system of each of the video projectors 11 to 15. The correction information generated by the correction information generation unit 34 includes the coordinates of the projection range in the projector coordinate system of each of the video projectors 11 to 15.

[0062] The image generation unit 35 generates image signals of the images Im1 to Im5 projected from the image projectors 11 to 15 based on the correction information generated by the correction information generation unit 34. Specifically, the image generation unit 35 generates image signals in which the projection range of the image has been corrected based on the coordinates indicating the projection range included in the correction information.

[0063] [Operation] The operation of the image calibration system 1 configured as above will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the operation of the image calibration system 1.

[0064] The image acquisition unit 32 acquires the first image 41 to the fourth image 44 captured by the respective imaging devices 21 to 24 (step S11). The image acquisition unit 32 acquires two types of images: images of the feature point detection test pattern 51 captured by the respective imaging devices 21 to 24, and the first image 41 to the fourth image 44 captured by the imaging devices 21 to 24 as adjustment images. The images 41 to 44 captured as adjustment images are used when the user specifies the projection range in a later step S13. For example, an entirely white image can be used as the adjustment image, as shown in FIGS. 3A to 3D. Alternatively, the adjustment image may be a frame that surrounds the projection range of the image projectors 11 to 15.

[0065] Next, the coordinate conversion unit 33 converts the first to fourth coordinate systems into a common composite coordinate system (step S12). The coordinate conversion unit 33 can convert the coordinate systems by calculating a coordinate conversion formula based on images of the feature point detection test pattern 51 captured by each of the imaging devices 21 to 24. Furthermore, the coordinate conversion unit 33 generates a composite image 45 of the first to fourth images 41 to 44 based on the coordinate conversion formula. Note that the coordinate conversion formula may be calculated in advance. In this case, it is preferable that the coordinate conversion formula be stored in a storage unit of the control unit 31.

[0066] Next, correction information is generated by the correction information generation unit 34 (step S13). The control unit 31 displays the composite image 45 on the display unit 36, and causes cursors C01 to C12 for specifying the projection range to be displayed on the composite image 45 (step S14). The user moves the cursors C01 to C12 to specify the projection range of each of the video projectors 11 to 15 (step S15). The correction information generation unit 34 calculates the coordinates of each of the cursors C01 to C12 in the projector coordinate system based on the coordinates in the composite coordinate system of the cursors C01 to C12 that indicate the projection range specified by the user, and generates correction information (step S16).

[0067] Finally, the video generation unit 35 generates a video signal based on the correction information (step S17).

[0068] [effect] According to the above-described embodiment, it is possible to provide a video adjustment system, a video adjustment method, and a video adjustment device that are more convenient.

[0069] Since the projection range can be specified using a composite image 45 that is a composite of the first image 41 to the fourth image 44 captured by each of the imaging devices 21 to 24, the user can specify the projection range of each of the image projectors 11 to 15 while grasping the entire image Im.

[0070] In the above embodiment, an example has been described in which the image calibration system 1 includes five image projection devices 11 to 15, but the image calibration system 1 is not limited to this. It is sufficient that the image calibration system 1 includes one or more image projection devices.

[0071] In the above-described embodiment, the image projectors 11 to 15 are arranged in a single horizontal row, but the arrangement is not limited to this. For example, the image projectors 11 to 15 can be arranged in any position according to the size of the image to be projected, such as in a single vertical row or in two or more rows.

[0072] Furthermore, in the above-described embodiment, an example has been described in which the image calibration system 1 includes four imaging devices 21 to 24, but the image calibration system 1 is not limited to this. It is sufficient that the image calibration system 1 includes two or more imaging devices.

[0073] In the above-described embodiment, the image capturing devices 21 to 24 are arranged in a single horizontal row, but the present invention is not limited to this. For example, the image capturing devices 21 to 24 can be arranged in any position according to the size of the image to be projected, such as in a single vertical row or in two or more rows.

[0074] Furthermore, in the above-described embodiment, an example has been described in which the first region R1 and the second region R2 include an overlapping region R5, but this is not limiting. For example, it is sufficient if the regions captured by adjacent imaging devices include at least a portion of the same image among the images Im1 to Im5. In other words, it is sufficient if each of the adjacent imaging devices can capture at least a portion of the image output from the same image projection device.

[0075] In the above-described embodiment, an example has been described in which a composite coordinate system is generated using images of test pattern 51 captured by each of imaging devices 21-24, but this is not limiting. For example, the same image may be projected by the same image projection device onto the areas that are captured by multiple imaging devices, such as areas R5-R7. Alternatively, no image may be projected onto the areas that are captured by multiple imaging devices. In this case, a composite coordinate system can be generated without capturing test pattern 51.

[0076] Fig. 7 is a diagram illustrating a modification of the first embodiment. For example, cursors C01 to C12 may be displayed on composite image 45 shown in Fig. 5, and the user may perform coarse adjustment of the projection range while grasping the entire image Im, and then perform fine adjustment of the projection range for each of first image 41 to fourth image 44. After the user performs coarse adjustment of the projection range for composite image 45, as shown in Fig. 7, the user can switch the display on display unit 36 ​​to first image 41, and perform more detailed adjustment of the positions of cursors C01 to C03 and C07 to C09.

[0077] In this case, the coordinates of the positions of cursors C01 to C12 changed on composite image 45 are converted by coordinate conversion unit 33 into coordinates in the coordinate systems of first image 41 to fourth image 44, and are displayed on each image 44. The coordinates of the positions of cursors C01 to C12 changed on each image 41 to 44 are converted by correction information generation unit 34 into the projector coordinate systems of video projection devices 11 to 15, respectively.

[0078] With this configuration, the composite image 45 can be used to grasp the entire image Im, and the projection range can be adjusted more precisely for each of the images 41 to 44. This allows the projection range to be adjusted with high precision.

[0079] The control unit 31 may also determine the projection range based on the first image 41 to the fourth image 44. For example, the control unit 31 estimates the size and position of a projection target, such as a screen, from each of the first image 41 to the fourth image 44. The control unit 31 determines the size and position of the projection range based on the estimated size and position of the projection target. The control unit 31 may also display cursors C01 to C12 on the composite image 45 in accordance with the estimated projection range. The control unit 31 may also determine an appropriate number and display positions of cursors based on the estimated size and position of the projection target or the number of video projectors and imaging devices. The control unit 31 may also estimate the projection range based on the composite image 45.

[0080] An example of a method for determining the position of a cursor to be displayed on composite image 45 will now be described with reference to FIGS. 10A to 10C. In the embodiment described above, image projection devices 11 to 15 and image capture devices 21 to 24 are arranged in a row as shown in FIG. 1. However, in this example, as shown in FIG. 10A, image capture device 20 is arranged at an angle relative to image projection device 10, which projects an image onto screen 60 (projection target). In this example, although not shown in detail, image projection device 10 is made up of four image projection devices, and image capture device 20 is made up of three image capture devices. That is, in this example, four projected images are projected onto screen 60 from the four image projection devices. Note that image projection device 10 may also be a single device equipped with a common light source and simultaneously projecting four projected images onto screen 60.

[0081] As shown in FIG. 10A, when the imaging device 20 is tilted with respect to the screen 60, the composite image generated by the control unit 31 is displayed on the display unit 36 ​​as a trapezoid, as shown on the left side of FIG. 10B. If the positions of cursors C01 to C10 were determined to be evenly spaced horizontally with respect to this composite image on the display unit 36, the cursors C01 to C10 would be positioned as shown on the left side of FIG. 10B. However, if the coordinates of cursors C01 to C10 in the composite coordinate system are converted to coordinates in the projector coordinate system, the cursors C01 to C10 would end up being positioned on the screen 60 as shown on the right side of FIG. 10B when viewed from the front. As a result, the cursors are not evenly spaced on the screen, which may result in distortion of the image projected by the image projection device 10 when the image is corrected.

[0082] Therefore, in this example, as shown in FIG. 10C, cursors C01 to C10 are assigned to the composite image taking into consideration the inclination of the imaging device 20 with respect to the screen 60. Specifically, the control unit 31 recognizes the four corners of the screen 60 in the composite image using image recognition technology to detect the shape of the screen 60. Here, in this example, it is assumed that the projected image is projected onto the screen 60 so as to have substantially the same size as the screen 60. The coordinates of the four corners of this composite coordinate system are projected into a quadrangle (square or rectangle). In the quadrangle obtained by projective transformation (right side of FIG. 10C), cursors C01 to C10 are evenly spaced horizontally to determine their tentative positions. The coordinates (temporary positions) of the evenly spaced cursors C01 to C10 are returned to the composite coordinate system by inverse projective transformation to determine the positions of cursors C01 to C10 in the composite image.

[0083] By performing correction using the cursor at the position determined in this way, it is possible to make the image projected by the image projection device 10 less prone to distortion.

[0084] Instead of the projection range specified by the user as described in the above embodiment, the correction information generating unit 34 may generate correction information using the projection range determined by the control unit 31. Also, the user may further adjust the projection range determined by the control unit 31.

[0085] Furthermore, in one example, the control unit 31 may cause the display unit 36 ​​to display an image based on the correction information as a composite image 45. Specifically, after step S17 in FIG. 6, the image projectors 11 to 15 receive, from the control unit 31, image signals generated based on the correction information. The image projectors 11 to 15 project, based on the received image signals, corrected images Im (images Im1 to Im5) onto the projection target. The imaging devices 21 to 24 capture the corrected images Im projected onto the projection target as first images 41 to fourth images 44. The control unit 31 acquires the first images 41 to fourth images 44 from the imaging devices 21 to 24, and generates a composite image 45 based on the first images 41 to fourth images 44. Furthermore, the control unit 31 causes the display unit 36 ​​to display the generated composite image 45. This allows the user to check the entire image of the corrected image Im projected onto the projection target on the display unit 36 ​​without looking at the projection target (screen).

[0086] (Embodiment 2) A second embodiment will be described with reference to Figures 8 and 9. In the second embodiment, the same or equivalent configurations as those in the first embodiment will be denoted by the same reference numerals. In the second embodiment, descriptions that overlap with those in the first embodiment will be omitted.

[0087] Fig. 8 is a schematic diagram showing an image calibration system 1 according to a second embodiment. As shown in Fig. 8, the second embodiment differs from the first embodiment in that the image projectors 11 to 15 project an adjustment image indicating the projection range onto the projection target. The components of the image calibration system 1 are the same as those in the first embodiment.

[0088] Fig. 9 is a flowchart showing the operation of the video calibration system 1 according to the second embodiment. The operation of the video calibration system 1 according to the second embodiment will be described with reference to Fig. 9. Note that steps S21, S22, and S27 are the same as steps S11, S12, and S17 in the first embodiment described with reference to Fig. 4, and therefore description thereof will be omitted.

[0089] The coordinate conversion unit 33 converts the first to fourth coordinate systems into a common composite coordinate system (step S22), and the correction information generation unit 34 generates correction information (step S23). In this embodiment, adjustment images (cursors) C01 to C12 are projected from the video projectors 11 to 15, respectively, and therefore the cursors C01 to C12 are displayed in advance in the composite image generated in step S22. Therefore, the user adjusts the positions of the cursors C01 to C12 while viewing the composite image displayed on the display unit 36 ​​to specify the projection range (step S24).

[0090] Next, the imaging devices 21 to 24 capture images Im1 to Im5 including the adjusted cursors C01 to C12 to obtain the first to fourth images 41 to 44 again (step S25). The control unit 31 generates a composite image of the captured images 41 to 44 and displays it on the display unit 36 ​​(step S26).

[0091] By repeating steps S24 to S26, the positions of the cursors C01 to C12 projected by the video projectors 11 to 15 can be made to coincide with the positions of the cursors on the composite image.

[0092] In this embodiment, since the cursors C01 to C12 projected by the video projectors 11 to 15 are moved, the correction information can be generated without converting coordinates from the composite coordinate system to the projector coordinate system in step S23 for generating the correction information. Therefore, even if the resolution of the video projectors 11 to 15 and the resolution of the imaging devices 21 to 24 differ and the accuracy of pixel alignment decreases, for example, the projection range can be adjusted with high accuracy.

[0093] Steps S25 and S26 may be performed at a predetermined timing, or may be performed when the user adjusts the cursor in step S24.

[0094] (Embodiment 3) In this embodiment, a combination of the first and second embodiments described above will be described.

[0095] In the first embodiment, an aspect has been described in which the images Im1 to Im5 projected by the image projectors 11 to 15 are corrected by moving the position of the cursor on the composite image 45 displayed on the display unit 36. In this aspect, the position of the cursor can be adjusted without delay in order to move the position of the cursor on the composite image 45. Therefore, the cursor position can be moved without delay in response to a user's operation to move the cursor on the input unit 37 (for example, a keyboard or mouse). However, there is a possibility that an error will occur between the position specified by the user on the composite image 45 and the position of the image Im actually projected on the projection target. This is because, when converting to the projector coordinate system, the resolution of the image projector and the resolution of the imaging device differ, which may reduce the accuracy of pixel alignment.

[0096] Furthermore, in the second embodiment, an aspect has been described in which images Im1 to Im5 projected by image projection devices 11 to 15 are corrected by moving the position of a cursor projected on a projection target. In this aspect, the cursor is moved in the projector coordinate system, so image Im can be corrected with high precision. On the other hand, every time the cursor position is moved, image capture devices 21 to 24 must capture image Im and perform processing to composite first image 41 to fourth image 44 captured from image Im. This can result in a large delay between the time the user moves the cursor position and the time the composite image reflecting the moved cursor state is displayed.

[0097] Based on the above, by performing coarse adjustment of the cursor position in the manner of embodiment 1 and fine adjustment of the cursor position in the manner of embodiment 2, it becomes possible to adjust the cursor position more accurately overall in a shorter time, thereby achieving both a reduction in adjustment time and an improvement in adjustment accuracy.

[0098] (Embodiment 4-1) In this embodiment, a method for recommending which of the aspects of the first embodiment and the second embodiment a user should use will be described.

[0099] In this embodiment, the control unit 31 recommends to the user which aspect to use depending on the number of imaging devices connected to the control unit 31. Specifically, the control unit 31 determines the number of imaging devices, and if the determined number of imaging devices is three or less, the control unit 31 recommends to the user that the aspect of the second embodiment be used. At this time, the control unit 31 may cause the display unit 36 ​​to display a message recommending that the user adjust the cursor position using a method equivalent to the aspect of the second embodiment. If the control unit 31 determines that the number of imaging devices is four or more, the control unit 31 recommends to the user that the aspect of the first embodiment be used. At this time, the control unit 31 may cause the display unit 36 ​​to display a message recommending that the user adjust the cursor position using a method equivalent to the aspect of the first embodiment.

[0100] The reason why the first embodiment is recommended when the number of imaging devices is four or more is that in the second embodiment, the processing time required to generate a composite image increases as the number of imaging devices increases, and the movement of the cursor position becomes slower in response to user operations.

[0101] Since the time required to generate a composite image also depends on the machine specifications, the threshold value (the number of image capturing devices) may be determined according to the machine specifications.

[0102] (Embodiment 4-2) In this embodiment, another method for recommending which of the aspects of the first embodiment and the second embodiment the user should use will be described.

[0103] In this embodiment, the control unit 31 recommends to the user which aspect to use based on the ratio of the projected image projected by the video projection device to the captured image captured by the imaging device. Specifically, the control unit 31 acquires the captured image captured by the imaging device from the imaging device. The control unit 31 uses image recognition technology to recognize the size of the projected image projected by the video projection device in the captured image. The control unit 31 calculates the ratio of the area of ​​the projected image to the area of ​​the captured image. The control unit 31 determines which aspect to use based on the calculated ratio of the area, the resolution of the imaging device, and the resolution of the video projection device. For example, if the resolution of the imaging device is 4000 x 3000 and the resolution of the video projection device is 1920 x 1200, the ratio of the resolution of the video projection device to the resolution of the imaging device is 19.2% (= 1920 x 1200 / 4000 x 3000). When the cursor on the projected image is drawn with a line one pixel thick, if the ratio of the area of ​​the projected image to the area of ​​the captured image calculated above is less than 19.2%, the first embodiment is recommended to the user.

[0104] The reason why the first embodiment is recommended when the area ratio of the projected image is small is that when the area ratio of the projected image is smaller than the resolution ratio, the size of the cursor in the image captured by the imaging device becomes less than one pixel, making it difficult for the imaging device to capture the cursor in the captured image. [Industrial Applicability]

[0105] The present disclosure is applicable to image display using an image projection device. [Explanation of symbols]

[0106] 1. Image Adjustment System 10,11~15 Video projection equipment 20, 21-24 Imaging device 31 Control Unit 32 Image acquisition unit 33 Coordinate conversion section 34 Correction information generation section 35 Image generation unit 41 First Image 42 Second image 43 Third image 44 4th image 45 composite images 51 Test pattern for feature point detection 60 screens C01~C12 Adjustment image (cursor) Im Video Im1~Im5 video R1 1st area R2 2nd area R3 3rd area R4 4th area

Claims

1. an image projection device that projects an image onto a projection target; an imaging device that acquires a first image that captures a first area that includes a portion of the projected image, and a second image that captures a second area that includes another portion of the projected image, the second area including an overlap area where the first area and the second area overlap; a control unit that controls the projection position of the projected image; Equipped with The control unit generating a composite image of the first image and the second image based on the overlapping region; detecting information about the projection target in the composite image; determining positions of a plurality of adjustment images based on the detected information on the projection target, and superimposing the plurality of adjustment images on the composite image; Image adjustment system.

2. The information about the projection target includes a shape of the projection target. The video calibration system of claim 1 .

3. The information about the projection target includes coordinates indicating the shape of the projection target. The video calibration system of claim 1 .

4. The control unit converting the detected coordinates indicating the shape of the projection target by a third coordinate conversion, and determining temporary positions of the plurality of adjustment images based on the converted coordinates; transforming coordinates indicating the provisional positions of the plurality of adjustment images by a fourth coordinate transformation to determine the positions of the plurality of adjustment images within the composite image; superimposing the plurality of adjustment images on the composite image based on the detected shape of the projection target and the determined positions of the plurality of adjustment images within the composite image; The video calibration system according to claim 3 .

5. the third coordinate transformation is a projective transformation, and the fourth coordinate transformation is an inverse projective transformation. The video calibration system according to claim 4 .

6. The control unit transforming a first coordinate system of the first image and a second coordinate system of the second image into a composite coordinate system common to the first image and the second image; generating correction information including position information indicating a projection range of the video projection device in the composite coordinate system; generating the projection image based on the correction information; The video calibration system according to claim 5 .

7. Coordinates in the composite image are represented by the composite coordinate system. The video calibration system of claim 6 .

8. the video projection device projects the plurality of projection images onto the projection target; The video calibration system according to any one of claims 1 to 7.

9. the control unit determines the projection range based on the first image and the second image.

8. The image calibration system according to claim 6 or 7.

Images