Method for adjusting projection image, information processing apparatus, and projection system

Abstract

To provide a method for adjusting a projection image, an information processing apparatus, and a projection system that can smoothly connect, in a superimposition area, a plurality of partial images sharing the superimposition area.SOLUTION: A method for adjusting an entire image Iw is an adjustment method for adjusting the state of the entire image Iw formed by a plurality of partial images Id projected on a projection surface Sp from a plurality of projectors 2 and arranged to be partially superimposed to each other by using an image for adjustment including a plurality of adjustment points, and includes: acquiring the number of the arranged partial images Id; determining the number of adjustment points so that the adjustment points are arranged in a superimposition area Ao; projecting the image for adjustment in which the determined number of adjustment points are arranged; receiving an operation to move the adjustment points; changing the state of the image for adjustment according to the movement of the adjustment points; and determining a correction parameter for correcting the entire image Iw based on the change in the state of the image for adjustment.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a method for adjusting a projected image, an information processing apparatus, and a projection system.

Background Art

[0002] Patent Document 1 describes a projector that projects an image including a plurality of control points and receives an operation of a user who moves the control points, and deforms the image according to the amount of movement of the control points. According to this projector, a user can correct states such as the shape and distortion of an image by moving the control points to desired positions.

[0003] On the other hand, Patent Document 2 describes a multi-projection system that connects partial images projected from each of a plurality of projectors on a projection surface to display one large image. In such a system, each projector is arranged so that the ends of adjacent partial images overlap each other so that the plurality of partial images are smoothly connected.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] When assuming that the state of the image projected by the multi-projection system described in Patent Document 2 is corrected using the technique described in Patent Document 1, depending on the combination of the number of partial images and the number of control points to be arranged, there may be a case where control points are not arranged on the overlapping region where the partial images overlap. If no control points are arranged in the overlapping region, it becomes difficult to smoothly connect a plurality of partial images sharing this overlapping region within the overlapping region.

Means for Solving the Problem

[0006] A method for adjusting a projected image is a method for adjusting a state of a projected image formed by a plurality of partial images that are projected onto a projection surface from a plurality of projectors and arranged in a first direction so as to partially overlap, using an adjustment image including a plurality of adjustment points. The method includes: obtaining projection information including first arrangement information corresponding to the number of the partial images arranged in the first direction; determining the number of the adjustment points in the first direction so that the adjustment points are arranged in an overlapping region where the partial images overlap, based on the first arrangement information; causing the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected onto the projection surface as the projected image from the plurality of projectors; receiving an operation of moving the adjustment points included in the projected adjustment image; changing the state of the projected adjustment image based on the received operation; and determining correction parameters for adjusting the state of the projected image based on the change in the state of the adjustment image.

[0007] The information processing apparatus is an information processing apparatus including a control unit that adjusts a state of a projection image formed by a plurality of partial images arranged in a first direction so as to be projected onto a projection surface from a plurality of projectors and partially overlap, using an adjustment image including a plurality of adjustment points. The control unit acquires projection information including first arrangement information corresponding to the number of the partial images arranged in the first direction, determines the number of the adjustment points in the first direction so that the adjustment points are arranged in an overlapping region where the partial images overlap, based on the first arrangement information, causes the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected onto the projection surface as the projection image from the plurality of projectors, accepts an operation of moving the adjustment points included in the projected adjustment image, changes the state of the projected adjustment image based on the accepted operation, and controls the determination of correction parameters for adjusting the state of the projection image based on the change in the state of the adjustment image.

[0008] The projection system includes a plurality of projectors and an information processing apparatus including a control unit that adjusts a state of a projection image formed by a plurality of partial images arranged in a first direction so as to be projected onto a projection surface from the plurality of projectors and partially overlap, using an adjustment image including a plurality of adjustment points. The control unit acquires projection information including first arrangement information corresponding to the number of the partial images arranged in the first direction, determines the number of the adjustment points in the first direction so that the adjustment points are arranged in an overlapping region where the partial images overlap, based on the first arrangement information, causes the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected onto the projection surface as the projection image from the plurality of projectors, accepts an operation of moving the adjustment points included in the projected adjustment image, changes the state of the projected adjustment image based on the accepted operation, and controls the determination of correction parameters for adjusting the state of the projection image based on the change in the state of the adjustment image.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 7C

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0010] 1. First Embodiment Hereinafter, the projection system of the first embodiment will be described with reference to the drawings. FIG. 1 is an explanatory drawing showing the projection system 100 of the present embodiment. As shown in FIG. 1, the projection system 100 includes a computer 1 which is an information processing device, and a plurality of projectors 2 that project images onto a projection surface Sp such as a screen or a wall surface. Each of the plurality of projectors 2 and the computer 1 are connected to a network NW (see FIG. 2) via a HUB 3, and the computer 1 controls the operation of each projector 2 via the network NW. The plurality of projectors 2 are installed such that the images projected from each projector 2 are arranged adjacent to each other so that they can cooperate to display one large image. Hereinafter, the image projected individually by each projector 2 is also referred to as a "partial image Id", and the large image formed by connecting these is also referred to as an "entire image Iw". The entire image Iw corresponds to the projected image. Also, the cooperation of the plurality of projectors 2 to display one entire image Iw is also referred to as "multi-projection".

[0011] In the multi-projection of this embodiment, four projectors 2 are arranged, and the four partial images Id projected from these projectors 2 are arranged in a matrix of two rows along a first direction D1 and two rows along a second direction D2 that intersects the first direction D1. And the entire image Iw is formed by these four partial images Id. In this embodiment, the first direction D1 is a direction parallel to the horizontal direction, and the second direction D2 is a direction parallel to the vertical direction. However, the first direction D1 and the second direction D2 are not limited to these directions. Also, the arrangement of the partial images Id is not limited to the above, and it may be one or more rows in the first direction D1 and one or more rows in the second direction D2, as long as at least one of these is multiple rows.

[0012] Each projector 2 displays a partial image Id in a partial range of the projection area Ap where an image can be projected. Typically, each projector 2 displays the partial image Id in a range of the projection area Ap where the entire image Iw can be visually recognized as a rectangle of a desired size. Also, each projector 2 is installed such that a part of each projection area Ap overlaps with a part of an adjacent projection area Ap. That is, each projector 2 is installed such that a part of the partial image Id projected by each overlaps with a part of an adjacent partial image Id. For this reason, there is no gap between the partial images Id, and the entire image Iw can be displayed in a state where each partial image Id is smoothly connected. In this way, in the present embodiment, the entire image Iw is formed by four partial images Id that are projected onto the projection surface Sp from four projectors 2 and are arranged in the first direction D1 and the second direction D2 so as to partially overlap. In this specification, the region Ao where the partial images Id overlap with each other is also referred to as the "overlap region Ao". Also, in this specification, the fact that the partial images Id and the projection areas Ap are "adjacent" means that they are adjacent in the first direction D1 or the second direction D2.

[0013] The configuration of the projection system 100 shown in FIG. 1 is a configuration required when initializing the projector 2. After that, when displaying a desired content image as the entire image Iw, an external image supply device 4 (see FIG. 3) is connected to each projector 2, and image data corresponding to the content image is supplied from the image supply device 4.

[0014] FIG. 2 is a block diagram showing a schematic configuration of the projection system 100 and an internal configuration of the computer 1. As shown in FIG. 2, the computer 1 includes a control unit 10, a storage unit 11, a display unit 12, a communication unit 13, and an operation unit 14.

[0015] The control unit 10 is configured to include one or more processors, a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. The control unit 10 operates according to a program stored in the ROM or a program read from the storage unit 11 into the RAM, thereby comprehensively controlling the operation of the computer 1.

[0016] The storage unit 11 is configured to include a storage device such as a hard disk drive or a solid state drive. The storage unit 11 stores an installed OS (Operating System), application programs, various data, and the like. A projection image adjustment program (not shown) is installed in the storage unit 11 of the present embodiment. This projection image adjustment program is an application program for adjusting the state of the overall image Iw, such as the shape of the overall image Iw displayed by multi-projection.

[0017] The display unit 12 is configured to include a display device such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays an image based on the control of the control unit 10.

[0018] The communication unit 13 is configured to include various circuits for communicating with an external device via the network NW. The communication unit 13 of the present embodiment communicates with a plurality of projectors 2 connected via the network NW based on the control of the control unit 10. The communication mode may be wired communication or wireless communication.

[0019] The operation unit 14 is configured by a keyboard, a pointing device, and the like, accepts various input operations of the user, and outputs information corresponding to the input operation to the control unit 10. As the pointing device, a mouse, a touch pad, or the like can be used.

[0020] In addition, when the configuration including the control unit 10 among the above-described configurations is taken as the main body of the computer 1, the configurations other than the control unit 10 do not necessarily have to be integrally configured with the main body of the computer 1.

[0021] FIG. 3 is a block diagram showing the internal configuration of the projector 2, and FIG. 4 is a block diagram showing the schematic configuration of the projection unit 27 provided in the projector 2. In addition, in the present embodiment, all of the plurality of projectors 2 have a common configuration.

[0022] As shown in FIG. 3, the projector 2 is integrally provided with a control unit 20, a storage unit 21, an operation unit 22, a communication unit 23, an imaging unit 24, an image input unit 25, an image correction unit 26, and a projection unit 27. The projector 2 projects an image onto the projection surface Sp from the projection unit 27 based on the image data input to the image input unit 25.

[0023] The control unit 20 includes one or a plurality of processors and operates according to a control program stored in the storage unit 21 to comprehensively control the operation of the projector 2.

[0024] The storage unit 21 is configured to include memories such as a RAM and a ROM. The RAM is used for temporary storage of various data and the like, and the ROM stores a control program, control data, image data, and the like for controlling the operation of the projector 2.

[0025] The operation unit 22 includes a plurality of operation keys for a user to give various instructions to the projector 2. When the user operates the various operation keys of the operation unit 22, the operation unit 22 outputs an operation signal corresponding to the user's operation content to the control unit 20. In addition, a configuration may be adopted in which a remote controller (not shown) capable of remote operation is used as the operation unit 22. In this case, the remote controller transmits an infrared operation signal corresponding to the user's operation content, and a remote controller signal receiving unit (not shown) receives this and transmits it to the control unit 20.

[0026] The communication unit 23 is configured with various circuits for communicating with external devices via the network NW. The communication unit 23 of the present embodiment is connected to the computer 1 and other projectors 2 via the network NW, and transmits and receives information to and from these devices based on the control of the control unit 20.

[0027] The imaging unit 24 is a camera equipped with an imaging element (not shown) such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The imaging unit 24 images the projection plane Sp based on the control of the control unit 20, and outputs imaging image data, which is the result of the imaging, to the control unit 20. The imaging unit 24 images at least a range including its own projection area Ap. For this reason, as shown in FIG. 1, when the projector 2 is installed such that a part of a plurality of partial images Id overlaps, the imaging unit 24 can also image at least the area included in the overlapping area Ao among adjacent partial images Id.

[0028] The image input unit 25 is connected to an external image supply device 4 such as an image playback device. The image input unit 25 receives the supply of image data corresponding to the content image from the image supply device 4 and outputs it to the image correction unit 26.

[0029] The image correction unit 26 performs correction processing on the image data input from the image input unit 25 based on the control of the control unit 20, and outputs the processed image data to the light valve drive unit 34 (see FIG. 4) of the projection unit 27. For example, the image correction unit 26 performs geometric correction processing on the image data to correct the state of the partial image Id, that is, the contour shape of the partial image Id and the distortion of the image inside it. Further, the image correction unit 26 can acquire the image data generated by the control unit 20 from the control unit 20 instead of the image data input from the image input unit 25, and output this image data to the projection unit 27.

[0030] Note that the image input unit 25 and the image correction unit 26 may be configured by one or more processors or the like, or may be configured by a dedicated processing device such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0031] As shown in FIG. 4, the projection unit 27 includes a light source 31, three liquid crystal light valves 32R, 32G, 32B as a light modulation device, a projection optical system 33, a light valve drive unit 34, and the like. The projection unit 27 modulates the light emitted from the light source 31 with the liquid crystal light valves 32R, 32G, 32B to form image light, and projects this image light from a projection optical system 33 including at least one of a lens and a mirror to display an image on the projection surface Sp.

[0032] The light source 31 includes a discharge-type light source lamp such as an ultra-high pressure mercury lamp or a metal halide lamp, or a solid light source such as a light emitting diode or a semiconductor laser. The light emitted from the light source 31 is converted into light with a substantially uniform luminance distribution by an integrator optical system (not shown), separated into respective color light components of red, green, and blue, which are the three primary colors of light, by a color separation optical system (not shown), and then incident on the liquid crystal light valves 32R, 32G, 32B, respectively.

[0033] The liquid crystal light valves 32R, 32G, 32B are each configured by a transmissive liquid crystal panel or the like in which liquid crystal is enclosed between a pair of transparent substrates. A rectangular pixel region 32i composed of a plurality of pixels arranged in a matrix is formed in each liquid crystal panel, and a drive voltage can be applied to the liquid crystal for each pixel.

[0034] The light valve driving unit 34 forms an image on the pixel regions 32i of the liquid crystal light valves 32R, 32G, and 32B. Specifically, the light valve driving unit 34 applies a driving voltage corresponding to the image data input from the image correction unit 26 to each pixel in the pixel region 32i, and sets each pixel to a light transmittance corresponding to the image data. The light emitted from the light source 31 is modulated for each pixel by passing through the pixel regions 32i of the liquid crystal light valves 32R, 32G, and 32B, and image light corresponding to the image data is formed for each color light. The formed image light of each color is synthesized for each pixel by a color synthesis optical system (not shown) to become image light representing a color image, and is enlarged and projected onto the projection surface Sp by the projection optical system 33. As a result, an image based on the image data input from the image correction unit 26 is displayed on the projection surface Sp.

[0035] Returning to FIG. 3, the control unit 20 has a correction control unit 28 as a functional block realized by a control program. The correction control unit 28 performs various processes for adjusting the state of the partial image Id to be projected, and controls the correction process by the image correction unit 26. For example, the correction control unit 28 generates correction parameters for geometrically correcting the image data in the image correction unit 26 based on the image captured by the imaging unit 24 or the control by the computer 1. In this case, the correction control unit 28 outputs the generated correction parameters to the image correction unit 26, and causes the image correction unit 26 to perform correction processing based on the correction parameters. Further, the correction control unit 28 partially generates the image data of the adjustment image Da (see FIG. 8) used when generating the correction parameters based on the control by the computer 1, outputs this to the image correction unit 26, and causes the projection unit 27 to project the adjustment image Da. Details of the adjustment image Da will be described later.

[0036] Next, a method for adjusting the entire image Iw when performing multi-projection by the projection system 100 will be described. First, the user installs each projector 2 so that the projection area Ap of each projector 2 is in an appropriate state. Specifically, as shown in FIG. 1, the user installs each projector 2 such that the area formed by combining the projection areas Ap of each projector 2 covers the area where the entire image Iw is to be displayed, and a part of adjacent projection areas Ap overlaps each other. Then, when the user instructs the computer 1 to start the projection image adjustment program, the computer 1 starts the projection image adjustment process according to the projection image adjustment program.

[0037] The projection image adjustment process is a process for adjusting the size, shape, distortion, etc. of the entire image Iw to a desired state, and is executed as an initial process, for example, when starting multi-projection by the projection system 100. By performing this projection image adjustment process, for example, when a screen is arranged inside the projection surface Sp, the user can adjust the outer shape of the entire image Iw to match the outer shape of the screen. Also, for example, when the projection surface Sp is a curved surface, the user can perform adjustment to reduce the distortion of the image due to the curved surface.

[0038] FIG. 5 is a flowchart for explaining the projection image adjustment process. As shown in FIG. 5, in step S110, the control unit 10 of the computer 1 acquires projection information related to multi-projection from the user. The projection information includes first array information representing the number of partial image Ids arranged in the first direction D1, second array information representing the number of partial image Ids arranged in the second direction D2, and further includes information associating which projector 2 among the plurality of projectors 2 connected to the network NW projects the partial image Id at which position. Thereby, the control unit 10 can recognize the arrangement of the partial image Ids and the projector 2 corresponding to each partial image Id. Note that these projection information may be automatically acquired by causing each projector 2 to project a predetermined test pattern and imaging it. Further, the first array information and the second array information representing the number of partial image Ids can be replaced with information representing the number of projectors 2 that project these partial image Ids.

[0039] Also, in the present embodiment, the control unit 10 acquires information regarding the arrangement density of the adjustment points Pa (see FIG. 8) from the user as one of the projection information. The adjustment points Pa are points arranged at the boundaries of the adjustment image Da (see FIG. 8) used when correcting the state of the entire image Iw or at a plurality of points inside thereof, and are the points to be adjusted. When the projection surface Sp is a plane, appropriate geometric correction is possible even if the arrangement density of the adjustment points Pa is low, but when the projection surface Sp is a curved surface or a surface having unevenness, fine correction becomes possible by arranging the adjustment points Pa at a high density. The control unit 10 acquires, as projection information, first density information representing the arrangement density in the first direction D1 and second density information representing the arrangement density in the second direction D2, in addition to the first array information and the second array information described above. In the present embodiment, the first density information and the second density information are natural numbers, and are set such that the higher the numerical value, the higher the arrangement density.

[0040] In step S120, the control unit 10 executes a projection area connection process for connecting the projection areas Ap of each projector 2. The projection area connection process is a process of obtaining the positional relationship of each projection area Ap and performing geometric correction for connecting the coordinate systems of adjacent projection areas Ap with respect to the coordinate system of each projection area Ap. By the projection area connection process, a single common coordinate system in which the coordinate systems of each projection area Ap on the projection plane Sp are connected is generated.

[0041] If there is a history of the projection area connection process having been executed in the past in the projection area connection process of step S120, and each projection area Ap has already been connected and the common coordinate system is already known, the execution thereof may be omitted. Further, for example, if each projector 2 is installed at a predetermined arrangement position and arrangement posture with respect to the projection plane Sp so that a common coordinate system predetermined on the projection plane Sp is constructed, step S120 may be omitted. The control unit 10 may display a message on the projection plane Sp to inquire of the user whether or not to execute the projection area connection process.

[0042] With reference to FIGS. 6 and 7A to 7C, the projection area connection process will be described. FIG. 6 is a flowchart for explaining the projection area connection process. In the projection area connection process of the present embodiment, the control unit 10 causes each projector 2 to project a measurement pattern image Dm (see FIG. 7A), and causes the imaging unit 24 to image the projection plane Sp on which the measurement pattern image Dm is projected. Then, the control unit 10 uses the captured imaging image SI to connect adjacent projection areas Ap to each other and generates a single coordinate system common to each projection area Ap on the projection plane Sp.

[0043] In step S210, the control unit 10 sequentially gives instructions to each projector 2 to project the measurement pattern image Dm onto each projection area Ap, and causes each imaging unit 24 to image the measurement pattern image Dm projected onto the projection plane Sp. The imaging image SI generated by the imaging at this time is also referred to as a "first imaging image SIa".

[0044] FIG. 7A is a schematic diagram showing an example of a measurement pattern image Dm. In the measurement pattern image Dm, measurement points Pm serving as indices indicating positions of predetermined coordinates are dispersedly arranged. In the example of FIG. 7A, circular point images representing the measurement points Pm are arranged in a matrix at regular intervals. The measurement pattern image Dm is configured such that such measurement points Pm are also projected onto an overlapping region Ao in the projection region Ap. FIG. 7B is a schematic diagram showing an example of a first captured image SIa. The first captured image SIa in FIG. 7B is a captured image SI captured by the imaging unit 24 of the projector 2 that has projected the measurement pattern image Dm in FIG. 7A. In the first captured image SIa, a state in which the measurement points Pm are arranged over the entire projection region Ap is shown.

[0045] Furthermore, the control unit 10 instructs each projector 2 to also cause the imaging unit 24 to image its own projection region Ap when the measurement pattern image Dm is projected onto a projection region Ap adjacent to its own projection region Ap. The captured image SI generated by the imaging at this time is also referred to as a "second captured image SIb". FIG. 7C is a schematic diagram showing an example of the second captured image SIb. In the second captured image SIb, at least the measurement points Pm displayed in the overlapping region Ao among the measurement pattern images Dm projected onto the adjacent projection regions Ap are shown. The second captured image SIb in FIG. 7C is a captured image SI captured by the imaging unit 24 when the projector 2 that has captured the first captured image SIa in FIG. 7B does not project the measurement pattern image Dm itself and the projector 2 immediately to its left projects the measurement pattern image Dm.

[0046] Note that in step S210, it is desirable for the control unit 10 to set the order in which each projector 2 projects the measurement pattern image Dm so that the measurement pattern images Dm are not projected onto adjacent projection regions Ap at the same time. Also, in step S210, the control unit 10 may control to project the measurement pattern images Dm onto two or more projection regions Ap that are separated in the first direction D1 or the second direction D2 and are not adjacent to each other at the same time. Thereby, it is possible to shorten the processing time of step S210.

[0047] In step S220, the control unit 10 acquires the position of the measurement point Pm in the coordinate system of each projection area Ap from each projector 2. Specifically, the control unit 20 of each projector 2 analyzes the first captured image SIa and extracts the position of each measurement point Pm shown in the first captured image SIa. Then, based on the coordinates of each extracted measurement point Pm on the image data, the coordinate system of the projection area Ap is acquired, and the coordinates of each measurement point Pm in the coordinate system of the projection area Ap, that is, the coordinates representing the display position of the measurement point Pm on the projection plane Sp are calculated. This calculation result is transmitted to the computer 1 via the communication unit 23 of each projector 2.

[0048] In step S230, the control unit 10 acquires the position of each measurement point Pm in the overlapping area Ao in the second captured image SIb from each projector 2. Specifically, the control unit 20 of each projector 2 analyzes the second captured image SIb and extracts the position of each measurement point Pm in the overlapping area Ao shown in the second captured image SIb. This extraction result is transmitted to the computer 1 via the communication unit 23 of each projector 2.

[0049] In step S240, the control unit 10 associates the coordinates of the measurement point Pm in the coordinate system of each projection area Ap obtained from the first captured image SIa with the information on the position of the measurement point Pm shown in the overlapping area Ao obtained from the second captured image SIb. Thereby, the control unit 10 specifies the positional relationship between adjacent projection areas Ap. The control unit 10 transmits the information indicating the positional relationship between adjacent projection areas Ap to the corresponding each projector 2. The "information indicating the positional relationship between adjacent projection areas Ap" is information indicating the relative positional relationship of the display positions of the pixels in the overlapping area Ao between adjacent projection areas Ap. Thus, in steps S210 to S240, using the measurement point Pm in the overlapping area Ao shown in the captured image SI captured by each imaging unit 24 of the plurality of projectors 2 as an index, the positional relationship of each projection area Ap of the plurality of projectors 2 is obtained.

[0050] In step S250, the correction control unit 28 of each projector 2 determines correction parameters for geometric correction to convert the coordinate system of each projection area Ap based on the information indicating the positional relationship between adjacent projection areas Ap transmitted from the control unit 10. Specifically, the correction control unit 28 calculates correction parameters for geometric correction so that the display positions of the pixels of the projection area Ap coincide with the display positions of the pixels of adjacent projection areas Ap within the overlapping area Ao, and outputs them to the image correction unit 26. As a result, the coordinate systems of adjacent projection areas Ap are connected, and a single common coordinate system in which the coordinate systems of each projection area Ap on the projection plane Sp are connected is generated. Thereafter, the control unit 10 of the computer 1 can specify positions to each projector 2 using this common coordinate system.

[0051] Returning to FIG. 5, in step S130, the control unit 10 determines the number of adjustment points Pa for adjusting the state of the entire image Iw based on the projection information acquired in step S110. Specifically, when the number of partial images Id along the first direction D1 is a, the number of partial images Id along the second direction D2 is b, the first density information representing the arrangement density in the first direction D1 is a natural number m, and the second density information representing the arrangement density in the second direction D2 is a natural number n, the control unit 10 determines the number of adjustment points Pa along the first direction D1 to be a×m + 1, and determines the number of adjustment points Pa along the second direction D2 to be b×n + 1. That is, the number of adjustment points Pa along each direction is a value obtained by multiplying the number of partial images Id along that direction by a natural number and adding 1. In the present embodiment, the number of partial images Id along the first direction D1 and the number of partial images Id along the second direction D2 are both 2, and 1 is set as both the first density information and the second density information. In this case, the number of adjustment points Pa is 3 in both the first direction D1 and the second direction D2, and a total of 3×3 = 9. Note that the first density information and the second density information do not have to be the same value and may be different.

[0052] In step S140, the control unit 10 controls each projector 2 to display an adjustment image Da (see FIG. 8) for adjusting the state of the entire image Iw on the projection plane Sp by multi-projection. As shown in FIG. 8, the adjustment image Da is a substantially rectangular image that is displayed as the overall image Iw, and the adjustment points Pa in the number determined in step S130 are images arranged in a matrix in the first direction D1 and the second direction D2. In the present embodiment, the adjustment point Pa has a cross shape. However, the shape of the adjustment point Pa is not limited to a cross shape, and various shapes can be adopted. In each of the first direction D1 and the second direction D2, the adjustment points Pa are arranged at equal intervals, but the interval in the first direction D1 and the interval in the second direction D2 are different due to the aspect ratio of the overall image Iw. In addition, auxiliary lines La that are straight lines are arranged in a grid pattern in the adjustment image Da so as to connect adjacent adjustment points Pa. In the present embodiment, the adjustment image Da is an image in which adjustment points Pa and auxiliary lines La having colors different from the background are arranged on a single-color background, but a significant image may be used as the background. In this case, the adjustment points Pa and the auxiliary lines La may be outlined in different colors so that the visibility of the adjustment points Pa and the auxiliary lines La does not decrease.

[0053] In step S140, first, the control unit 10 tentatively determines the size of the adjustment image Da, and based on that size, determines the coordinates of each adjustment point Pa in the common coordinate system. Then, the control unit 10 outputs the coordinates of the adjustment points Pa that can be included in each projection area Ap to each projector 2, and causes each correction control unit 28 to generate image data of a partial image Id corresponding to a part of the adjustment image Da. Then, when the correction control unit 28 of each projector 2 controls each image correction unit 26 to output the generated image data to the projection unit 27, the adjustment image Da is displayed on the projection surface Sp by multi-projection (see FIG. 9). In the present embodiment, the adjustment points Pa and the auxiliary lines La included in the overlapping area Ao are projected overlapped by a plurality of projectors 2 that share this overlapping area Ao. However, the adjustment points Pa and the auxiliary lines La included in the overlapping area Ao may be projected by only one of the projectors 2. In FIG. 9, the desired projection range Ad of the user, that is, the range where the overall image Iw is to be displayed, is indicated by a dashed line. ​

[0054] As shown in FIG. 9, in this embodiment, in the first direction D1 in which two partial image Ids are arranged side by side, the number of adjustment points Pa is 3, and furthermore, these adjustment points Pa are arranged at equal intervals in the first direction D1. Therefore, if the adjustment points Pa at both ends among these three adjustment points Pa are arranged near both ends in the first direction D1 of the entire image Iw, the central adjustment point Pa is arranged at the intermediate position between two adjacent partial image Ids in the first direction D1, that is, in the overlapping region Ao. The same applies to the second direction D2, and the adjustment point Pa is also arranged in the overlapping region Ao existing at the intermediate position between two adjacent partial image Ids in the second direction D2.

[0055] Note that the mode of displaying the adjustment image Da by multi-projection is not limited to the above mode. For example, the control unit 10 may generate adjustment image data representing the adjustment image Da as described above, and generate partial image data of a part corresponding to the arrangement of the partial image Ids of each projector 2 from the generated adjustment image data, and output the corresponding partial image data to each projector 2. In this case, when each projector 2 projects the partial image Id based on the input partial image data, the adjustment image Da is displayed on the projection surface Sp by multi-projection.

[0056] In step S150, the control unit 10 receives, via the operation unit 14, an operation from the user to move the adjustment point Pa. The user selects, by the operation unit 14, one adjustment point Pa to be the target of the movement operation, and specifies the movement direction and the movement distance of the selected adjustment point Pa. Alternatively, the user may directly specify, on the projection surface Sp by a pointer or the like, the position to which the selected adjustment point Pa is to be moved. Note that the control unit 10 may cause the correction control unit 28 to display the movable range of each adjustment point Pa for the convenience of the user.

[0057] In step S160, the control unit 10 changes the state of the adjustment image Da based on the user's operation. Specifically, the control unit 10 outputs the coordinates of the adjusted point Pa after movement to the projector 2 that includes the selected adjustment point Pa within the projection area Ap, and instructs the update of the partial image Id. When the correction control unit 28 of the projector 2 that has received the instruction moves the adjustment point Pa to the input coordinates and updates the partial image Id, the adjusted image Da after the state change is displayed on the projection surface Sp by multi-projection. For example, when the upper-left adjustment point Pa is moved to the upper-left corner of the desired projection range Ad by the user's operation, the adjustment image Da is deformed as shown in FIG. 10.

[0058] In step S170, the control unit 10 determines whether or not the adjustment of the entire image Iw using the adjustment image Da has been completed based on the user's operation by the operation unit 14. If there is no instruction to end from the user, the process returns to step S150 to accept an operation to move the adjustment point Pa. Then, by repeating steps S150 to S170, the user can move a plurality of adjustment points Pa to desired positions, and thereby can instruct the state of the adjustment image Da. On the other hand, when the user instructs the end of the adjustment, the positions of all the adjustment points Pa are determined, and the control unit 10 moves the process to step S180. For example, when all the adjustment points Pa are moved according to the desired projection range Ad by the user's operation, the adjustment image Da is displayed in an appropriately adjusted state as shown in FIG. 11.

[0059] In step S180, the control unit 10 causes the correction control unit 28 of each projector 2 to determine correction parameters for geometric correction based on the position of the determined adjustment point Pa, that is, the change in the state of the adjustment image Da, and ends the process. When receiving this instruction, the correction control unit 28 of each projector 2 updates the correction parameters determined in step S250 based on the coordinates of the adjustment point Pa and outputs them to the image correction unit 26. Thereafter, the image correction unit 26 of each projector 2 performs geometric correction on the image data input to the image input unit 25 based on this correction parameter, thereby projecting the partial image Id in a state corresponding to the position of the adjustment point Pa. As a result, the entire image Iw is adjusted and displayed in the state specified by the user. In this way, since the state of the entire image Iw is adjusted by each projector 2 projecting the partial image Id based on the determined correction parameter, each correction parameter corresponds to a parameter for adjusting the state of the entire image Iw.

[0060] In the present embodiment, the case where both the first density information and the second density information included in the projection information are set to 1 has been described. However, for example, when both the first density information and the second density information are set to 2, the number of adjustment points Pa is 5 in both the first direction D1 and the second direction D2, and the total number is 5 × 5 = 25 (see FIG. 12). As shown in FIG. 12, also in this case, the adjustment point Pa is arranged in the overlapping region Ao. Further, since the correction parameters for geometric correction can be determined using four adjustment points Pa adjacent to each other in the first direction D1 and the second direction D2, in this case, the correction control unit 28 generates four correction parameters in one partial image Id, and the image correction unit 26 performs correction by applying different correction parameters according to the position within the partial image Id.

[0061] Also, although illustration is omitted, when both the first density information and the second density information are set to 3, the number of adjustment points Pa in the first direction D1 is 7 in both the first direction D1 and the second direction D2, and the total number is 7×7 = 49. Also in this case, the adjustment points Pa are arranged in the overlapping region Ao. Further, in this case, in one partial image Id, nine correction parameters are generated, and different correction parameters are applied according to the position within the partial image Id.

[0062] Thus, when a partial images Id are arranged in the first direction D1, setting the number of adjustment points Pa in the first direction D1 to a×m + 1 (m is a natural number) means that three or more odd numbers of adjustment points Pa are arranged in two adjacent partial images Id in the first direction D1. Further, since these adjustment points Pa are arranged at equal intervals in the first direction D1, the adjustment points Pa are arranged in the overlapping region Ao existing at the intermediate position between two adjacent partial images Id in the first direction D1. The same applies to the second direction D2. When b partial images Id are arranged in the second direction D2, by setting the number of adjustment points Pa in the second direction D2 to b×n + 1 (n is a natural number), the adjustment points Pa are also arranged in the overlapping region Ao existing at the intermediate position between two adjacent partial images Id in the second direction D2. That is, regardless of the natural number values indicated by the first density information and the second density information, the adjustment points Pa are arranged in the overlapping region Ao, and further, the larger the natural number values indicated by the first density information and the second density information, the finer the correction can be performed.

[0063] As described above, according to the projection system 100, the computer 1, and the method for adjusting the entire image Iw of the present embodiment, the following effects can be obtained.

[0064] (1) According to this embodiment, the control unit 10 determines the number of adjustment points Pa in the first direction D1 and the second direction D2 such that the adjustment points Pa are arranged in the overlapping region Ao where the partial images Id overlap, based on the number of partial images Id arranged in the first direction D1 and the second direction D2. As a result, since the adjustment points Pa are arranged within the overlapping region Ao, it becomes possible to smoothly connect a plurality of partial images Id sharing this overlapping region Ao within the overlapping region Ao.

[0065] (2) According to this embodiment, since the projection information acquired from the user includes information regarding the arrangement density of the adjustment points Pa, the adjustment points Pa can be arranged at a desired arrangement density according to the three-dimensional shape of the projection surface Sp and the like.

[0066] (3) According to this embodiment, since the number of adjustment points Pa in the first direction D1 and the second direction D2 is set to a value obtained by multiplying the number of partial images Id in each direction by a natural number and adding 1, the adjustment points Pa can be arranged at the intermediate position between adjacent partial images Id, that is, in the overlapping region Ao.

[0067] 2. Second Embodiment Hereinafter, the projection system of the second embodiment will be described. The projection system 100 of this embodiment has the same configuration as that of the first embodiment, but the operation of the projection image adjustment process is partially different. In this embodiment, when acquiring the projection information from the user in step S110, the control unit 10 acquires the first array information representing the number of partial images Id along the first direction D1 and the second array information representing the number of partial images Id along the second direction D2, similar to the first embodiment, but does not acquire the information regarding the arrangement density of the adjustment points Pa, that is, the first density information and the second density information.

[0068] Instead, in step S130 when determining the number of adjustment points Pa, the control unit 10 causes the display unit 12 to display a menu image Mn (see FIG. 13) for allowing the user to select the number of adjustment points Pa. As shown in FIG. 13, the menu image Mn includes three selectable options for the user. Among these, the first option S1 is labeled "3 horizontally × 3 vertically = 9", the second option S2 is labeled "5 horizontally × 5 vertically = 25", and the third option is labeled "7 horizontally × 7 vertically = 49". That is, each option shows the number of adjustment points Pa in the first direction D1, the number of adjustment points Pa in the second direction D2, and the total number of adjustment points Pa. This menu image Mn is generated by the control unit 10 based on the number of partial images Id along the first direction D1 and the number of partial images Id along the second direction D2 obtained from the user as projection information.

[0069] Specifically, when the number of partial images Id along the first direction D1 obtained from the user is a and the number of partial images Id along the second direction D2 is b, the control unit 10 determines the first option S1 as the option where the number of adjustment points Pa in the first direction D1 is a×1 + 1 and the number of adjustment points Pa in the second direction D2 is b×1 + 1. Also, the control unit 10 determines the second option S2 as the option where the number of adjustment points Pa in the first direction D1 is a×2 + 1 and the number of adjustment points Pa in the second direction D2 is b×2 + 1. Further, the control unit 10 determines the third option S3 as the option where the number of adjustment points Pa in the first direction D1 is a×3 + 1 and the number of adjustment points Pa in the second direction D2 is b×3 + 1. Thus, for all options included in the menu image Mn, the number of adjustment points Pa satisfies that the number in the first direction D1 is a×x + 1 (x is a natural number) and the number in the second direction D2 is b×y + 1 (y is a natural number). Note that in each option, x and y are the same natural number, but they may also be different. Also, the number of options to be determined may be plural and may be other than 3.

[0070] The user can select one option from three options by operating the operation unit 14. After the control unit 10 causes the display unit 12 to display the menu image Mn, it receives the operation of the user who selects an option via the operation unit 14. Then, the control unit 10 determines the number of adjustment points Pa in the first direction D1 and the second direction D2 based on the option selected by the user.

[0071] As described above, according to the projection system 100, the computer 1, and the method for adjusting the entire image Iw of the present embodiment, the same effects as those of the first embodiment can be obtained.

[0072] Note that the above embodiment may be modified as follows.

[0073] In the above embodiment, part of the operations performed by the computer 1 may be performed by at least one projector 2, or part of the operations performed by each projector 2 may be performed by the computer 1 to control each projector 2. Further, if all the operations performed by the computer 1 are performed by one projector 2 and this projector 2 controls the operations of the other projectors 2, it is also possible to configure the projection system 100 without including the computer 1.

[0074] In the above embodiment, the control unit 10 determines the correction parameter in step S180 after all the movements of the adjustment point Pa by the user have ended in step S170. However, the correction parameter may be updated each time the movement of one adjustment point Pa ends, and the image may be displayed using this correction parameter. In this mode, for example, when the adjustment image Da is an image in which the adjustment point Pa and the auxiliary line La are superimposed on a significant background image, each time one adjustment point Pa is moved, the background image is geometrically corrected, so that the correction status of the image distortion can be confirmed in real time.

[0075] In the above embodiment, as the first array information, the number of partial images Id arranged in the first direction D1 is obtained, and as the second array information, the number of partial images Id arranged in the second direction D2 is obtained. However, the number of overlapping regions Ao along the first direction D1 may be obtained as the first array information, and the number of overlapping regions Ao along the second direction D2 may be obtained as the second array information. For example, as in the above embodiment, when four partial images Id are arranged in two columns in both the first direction D1 and the second direction D2, the number of overlapping regions Ao along the first direction D1 and the number of overlapping regions Ao along the second direction D2 are both 1. And in this case, the number of partial images Id in each direction can be calculated by adding 1 to the number of obtained overlapping regions Ao.

[0076] In the above embodiment, transmissive liquid crystal light valves 32R, 32G, and 32B are used as the light modulation device, but it is also possible to use a reflective light modulation device such as a reflective liquid crystal light valve. Further, a digital micromirror device or the like that modulates the light emitted from the light source 31 can also be used by controlling the emission direction of the incident light for each micromirror serving as a pixel. Further, the configuration is not limited to including a plurality of light modulation devices for each color light, and a configuration in which a single light modulation device modulates a plurality of color lights in a time-division manner may also be used.

Description of Reference Numerals

[0077] 1... Computer, 2... Projector, 3... HUB, 4... Image supply device, 10... Control unit, 11... Memory unit, 12... Display unit, 13... Communication unit, 14... Operation unit, 20... Control unit, 21... Memory unit, 22... Operation unit, 23... Communication unit, 24... Imaging unit, 25... Image input unit, 26... Image correction unit, 27... Projection unit, 28... Correction control unit, 31... Light source, 32R, 32G, 32B... Liquid crystal light valve, 32i... Pixel region, 33... Projection optical system, 34... Light valve drive unit, 100... Projection system, NW... Network, Iw... Whole image, Id... Partial image, Ap... Projection area, Ao... Overlap area, Ad... Projection range, D1... First direction, D2... Second direction, Da... Adjustment image, Dm... Measurement pattern image, Mn... Menu image, Pa... Adjustment point, La... Auxiliary line, Pm... Measurement point, SI... Captured image, SIa... First captured image, SIb... Second captured image, Sp... Projection surface.

Claims

1. A method for adjusting a projection image, which is formed by a plurality of partial images arranged in a first direction so as to be projected onto a projection surface from a plurality of projectors and partially overlap, and is adjusted using an adjustment image including a plurality of adjustment points, comprising: obtaining projection information including first arrangement information corresponding to the number of the partial images arranged in the first direction; determining the number of the adjustment points in the first direction such that the adjustment points are arranged in an overlapping region where the partial images overlap based on the first arrangement information; causing the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected onto the projection surface as the projection image from the plurality of projectors; receiving an operation of moving the adjustment points included in the projected adjustment image; changing the state of the projected adjustment image based on the received operation; determining a correction parameter for adjusting the state of the projection image based on the change in the state of the adjustment image; and wherein determining the number of the adjustment points in the first direction includes: determining a plurality of options for the number of the adjustment points based on the first arrangement information; receiving an operation of selecting one option from the plurality of options; and 、 determining the number of the adjustment points in the first direction based on the selected option. 、

2. The method for adjusting a projection image according to Claim 1, wherein the first arrangement information is information representing the number of the partial images arranged in the first direction.

3. The method for adjusting a projection image according to Claim 1, wherein the first arrangement information is information representing the number of overlapping regions along the first direction.

4. The method for adjusting a projection image according to any one of Claims 1 to 3, wherein when the number of the partial images arranged in the first direction determined according to the first arrangement information is a, each of the plurality of options satisfies a×x + 1 (x is a natural number).

5. The method for adjusting a projection image according to any one of Claims 1 to 4, wherein the plurality of partial images are arranged such that a part thereof also overlaps in a second direction intersecting the first direction, and the projection information includes second arrangement information corresponding to the number of the partial images arranged in the second direction. ​ ​ ​ ​ 、 ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ In the adjustment image, a plurality of adjustment points are also arranged in the second direction. Based on the second arrangement information, the number of the adjustment points in the second direction is determined such that the adjustment points are arranged in the overlapping region. Adjustment method for a projection image.

6. The adjustment method for a projection image according to claim 5, wherein the second arrangement information is information representing the number of the partial images arranged in the second direction. Adjustment method for a projection image. 、

7. The adjustment method for a projection image according to claim 5, wherein the second arrangement information is information representing the number of the overlapping regions along the second direction. Adjustment method for a projection image.

8. The adjustment method for a projection image according to any one of claims 5 to 7, wherein determining the number of the adjustment points in the second direction includes: determining a plurality of options for the number of the adjustment points based on the second arrangement information; receiving an operation of selecting one option from the plurality of options; and determining the number of the adjustment points in the second direction based on the selected option. Adjustment method for a projection image including the above.

9. The adjustment method for a projection image according to claim 8, wherein when the number of the partial images arranged in the second direction determined according to the second arrangement information is b, each of the plurality of options satisfies b×y + 1 (y is a natural number). Adjustment method for a projection image.

10. An information processing apparatus including a control unit that adjusts a state of a projection image formed by a plurality of partial images arranged in a first direction so as to partially overlap when projected onto a projection surface from a plurality of projectors, using an adjustment image including a plurality of adjustment points, wherein the control unit: acquires projection information including first arrangement information corresponding to the number of the partial images arranged in the first direction; determines the number of the adjustment points in the first direction based on the first arrangement information such that the adjustment points are arranged in an overlapping region where the partial images overlap; causes the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected onto the projection surface as the projection image from the plurality of projectors; receives an operation of moving the adjustment points included in the projected adjustment image; changes the state of the projected adjustment image based on the received operation; and adjusts the state of the projection image based on the change in the state of the adjustment image. ​ ​ ​ ​ ​ ​ ​ 、 ​ 、 ​ Execute controlling the determination of the warp parameter, Determining the number of the adjustment points in the first direction, Based on the first array information, determining a plurality of options for the number of the adjustment points and Receiving an operation of selecting one option from the plurality of options, Determining the number of the adjustment points in the first direction based on the selected option including An information processing apparatus.

11. A plurality of projectors, An information processing apparatus including a control unit that adjusts the state of a projection image formed by a plurality of partial images arranged in a first direction so as to be partially superimposed when projected from the plurality of projectors onto a projection surface, using an adjustment image including a plurality of adjustment points, A projection system including The control unit Obtaining projection information including first array information corresponding to the number of the partial images arranged in the first direction, Based on the first array information, determining the number of the adjustment points in the first direction so that the adjustment points are arranged in a superimposed area where the partial images are superimposed, Causing the adjustment image in which the determined number of adjustment points are arranged in the first direction to be projected from the plurality of projectors onto the projection surface as the projection image, Receiving an operation of moving the adjustment points included in the projected adjustment image, Changing the state of the projected adjustment image based on the received operation, Based on the change in the state of the adjustment image, controlling the determination of a correction parameter for adjusting the state of the projection image, Determining the number of the adjustment points in the first direction, Based on the first array information, determining a plurality of options for the number of the adjustment points, and 、 Receiving an operation of selecting one option from the plurality of options, 、 Determining the number of the adjustment points in the first direction based on the selected option including A projection system. ​ ​ ​ ​ ​ ​

Images