Projection system, projection system control device, and projection method

Abstract

To provide a technique that moves an image projected on a projection surface as the projection surface moves, and that suppresses a plurality of images projected in order on the moving projection surface from shifting in position on the projection surface.SOLUTION: A projection system comprises: a projector which projects a content image and a pattern image; a camera which photographs a projection surface when the pattern image is projected; a camera deformation value calculation part which calculates a texture deformation value representing change in the position, direction or shape of the projection surface on the basis of a photography result; and a projector deformation value acquisition part which acquires a projector deformation value of image deformation for converting the image projected by the projector into an image corresponding to the state of the projection surface on the basis of a pattern image included in the photography result and the texture deformation value. Predetermined conditions include a condition that a plurality of plain regions having a plurality of kinds of plain regions differing in luminance value and differing in luminance value are included.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a projection system, a projection system control device, and a projection method. This application claims priority to Japanese Patent Application No. 2019-156876, filed on August 29, 2019, the contents of which are incorporated herein by reference. [Background technology]

[0002] Conventional projection mapping technology, which tracks an image on a moving projection surface, involves capturing an image of the projection surface with a camera and then tracking the image (projected content) to the movement of the projection surface based on the captured image. This type of projection mapping technology requires prior alignment of the projector and camera and marking on the projection surface, making it difficult for users to use. To address this issue, a tracking method has been proposed that simulates a camera image obtained when projected content is superimposed on a texture on the projection surface (Non-Patent Document 1). This method eliminates the need for prior alignment or marking if the texture on the projection surface and the projected content on the projection surface can be tracked using camera images. However, this method requires a large amount of computation, making it difficult to track the content image to the movement of the projection surface. Other methods have also been proposed, such as limiting the projection surface to a plain surface and a known shape (Non-Patent Document 2), allowing the projected content to track without simulation. However, the texture on the projection surface cannot be used as a tracking clue.

[0003] Furthermore, both of these proposed methods have the problem that the content image must have characteristics that allow the camera to track its movement. Therefore, when the projection surface is not plain, it has been proposed to hide a pattern image that is difficult for humans to perceive in the projected video and use the pattern image to track the movement of the content image. However, this method requires the camera to capture multiple consecutive images (Patent Document 1, Non-Patent Document 3). Therefore, the limit at which the camera can track the movement of the projection surface depends on the camera's shutter speed, and this method does not work well when the projection surface moves quickly. As such, when the projection surface is not plain, there are cases where the positional deviation on the projection surface between multiple images projected sequentially onto the projection surface becomes large. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 002293 [Non-patent literature]

[0005] [Non-Patent Document 1] S. Audet, M. Okutomi, M. Tanaka, “Augmenting moving planar surfaces robustly with videoprojection and direct image alignment”, Virtual Reality, vol. 17, 2013. [Non-patent document 2] S. Kagami and K. Hashimoto, “Sticky projection mapping: 450-fps tracking projection onto a moving planar surface”, Proc. SIGGRAPH Asia 2015 Emerging Technologies, 2015. [Non-patent document 3] Ramesh Raskar, Greg Welch, Matt Cutts, Adam Lake, Lev Stesin, and Henry Fuchs “The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays”, Computer Graphics Proceedings, Annual Conference Series, 1998. Summary of the Invention [Problem to be solved by the invention]

[0006] In view of the above circumstances, the present invention aims to provide a technology for moving an image projected onto a projection surface in accordance with the movement of the projection surface, which suppresses deviations in the positions of multiple images projected in sequence onto a moving projection surface. [Means for solving the problem]

[0007] One aspect of the present invention is a projection system comprising: a projector that projects a content image and a pattern image that satisfies predetermined conditions onto a projection surface that is not a plain surface; a camera that photographs the projection surface at a photographing timing when the pattern image is projected; a camera transformation value calculation unit that calculates a texture transformation value that represents a change in the position, orientation, or shape of the projection surface based on the camera image captured by the camera; and a projector transformation value acquisition unit that acquires a projector transformation value for image transformation that converts the image projected by the projector into an image that corresponds to the state of the projection surface based on the pattern image captured in the camera image and the texture transformation value, wherein the predetermined conditions include a condition that the image contains a plurality of plain areas with different brightness values, and the camera transformation value calculation unit calculates the texture transformation value based on a surface pattern of the projection surface that appears in the camera image, the region within the plain area, and that is located at a distance from the periphery of the plain area that corresponds to the speed of movement or rotation of the projection surface and the interval between photographing timings by the camera.

[0008] One aspect of the present invention includes a projector that projects a content image and a pattern image that satisfies predetermined conditions onto a projection surface that is not a plain surface; a camera that photographs the projection surface at a photographing timing when the pattern image is projected; a camera transformation value calculation unit that calculates a texture transformation value that represents a change in the position, orientation, or shape of the projection surface based on the camera image that is the photographing result of the camera; a projector transformation value acquisition unit that acquires a projector transformation value of an image transformation that converts an image projected by the projector into an image that corresponds to the state of the projection surface based on the pattern image captured in the camera image and the texture transformation value; and an image transformation unit that performs image transformation on the content image based on the projector transformation value. and a transmitted image acquisition unit that executes a process of image transformation of the pattern image based on the projector transformation value, wherein the predetermined conditions include a condition that a plurality of plain areas with different brightness values ​​are included, and a condition that the area of ​​at least one of the plurality of types of plain areas is larger than a predetermined area, and the camera transformation value calculation unit calculates the texture transformation value based on a surface pattern of the projection surface that is captured in an area that is captured in the camera image, and that is within the plain area and is located at a position away from the periphery of the plain area by a distance that depends on the speed of movement or rotation of the projection surface and the interval between timings of capturing images by the camera.

[0009] One aspect of the present invention includes a projection step of projecting a content image and a pattern image that satisfies predetermined conditions onto a projection surface that is not plain; a photographing step of photographing the projection surface at a photographing timing when the pattern image is projected; a camera transformation value calculation step of calculating a texture transformation value that represents a change in the position, orientation, or shape of the projection surface based on a camera image that is the photographing result of the photographing step; and a projector transformation value acquisition step of acquiring a projector transformation value of an image transformation that converts the image projected in the projection step into an image that corresponds to the state of the projection surface based on the pattern image captured in the camera image and the texture transformation value. a transmission image acquisition step of executing a process of image transformation of the content image based on the projector transformation value and a process of image transformation of the pattern image based on the projector transformation value, wherein the predetermined conditions include a condition that a plurality of plain areas with different brightness values ​​are included, and the camera transformation value calculation step calculates the texture transformation value based on a pattern on the surface of the projection surface that is captured in an area that is captured in the camera image, and that is within the plain area and is located at a position away from the periphery of the plain area by a distance that corresponds to the speed of movement or rotation of the projection surface and the interval between timings of capturing images by the camera. [Effects of the Invention]

[0010] The present invention makes it possible to provide a technology that suppresses the positional deviation on a projection surface of multiple images that are projected in sequence onto a moving projection surface, in a technology that moves an image projected onto a projection surface in accordance with the movement of the projection surface. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the system configuration of a projection system 100 according to a first embodiment. [Figure 2] 5 is a timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in the first embodiment. [Figure 3]FIG. 4 is a schematic diagram showing an example of information indicating a history of deformation values ​​in the first embodiment. [Figure 4] FIG. 1 is a first diagram showing an example of a pattern image in the first embodiment. [Figure 5] FIG. 2 is a second diagram showing an example of a pattern image in the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the functional configuration of a control unit 30 according to the first embodiment. [Figure 7] FIG. 3 is a diagram showing an example of a target location in the first embodiment. [Figure 8] 10A and 10B are diagrams showing examples of experimental results illustrating a pattern image before acquiring a projector matrix in the first embodiment and a pattern image after performing a homography transformation represented by the projector matrix. [Figure 9] 5 is a flowchart showing an example of the flow of processing executed by the projection system 100 in the first embodiment. [Figure 10] FIG. 10 is a diagram showing an example of the system configuration of a projection system 100 according to a second embodiment. [Figure 11] FIG. 10 is a diagram showing an example of the functional configuration of a control unit 30a according to a second embodiment. [Figure 12] FIG. 10 is an explanatory diagram illustrating a first control point and a second control point in the second embodiment. [Figure 13] 10 is a flowchart showing an example of the flow of processing executed by the projection system 100a in the second embodiment. [Figure 14] 10 is a first timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in a modified example. [Figure 15] 10 is a second timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in a modified example. [Figure 16] FIG. 10 is a diagram showing an example of a transmission image acquisition unit 308 in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] (First embodiment) FIG. 1 is a diagram showing an example of the system configuration of a projection system 100 according to the first embodiment. The projection system 100 projects multiple still images onto a projection surface 90 that can be moved, rotated, or deformed, while switching between them at a predetermined frame rate. The frame rate is the number of still images projected onto the projection surface 90 per unit time. The frame rate is, for example, 2400 frames per second (fps). The projection system 100 projects at least two types of still images onto the projection surface 90. One of the still images is a content image, and the other is a pattern image. The surface of the projection surface 90 is not plain. Hereinafter, the pattern on the surface of the projection surface 90 will be referred to as the object texture.

[0013] The pattern image is an image projected onto the projection surface 90 to correct the content image in accordance with changes in the position, orientation, or shape (hereinafter referred to as the "three-dimensional state") of the projection surface 90. Specifically, the pattern image is an image that satisfies the following first pattern image condition. The first pattern image condition includes a condition that the image has multiple types of plain areas with different brightness values, and the area of ​​at least one of the multiple types of plain areas is larger than a predetermined area. The predetermined area is, for example, an area that is determined in advance according to the speed of change in the position, orientation, or shape of the projection surface 90 and the object texture. The predetermined area may be, for example, the area of ​​a surface that satisfies a first backprojection condition. The first backprojection condition is a condition that the shape is a rectangle in which the product of the backprojection vector and the frame time is equal to the length of the diagonal. The backprojection vector is a vector obtained by backprojecting the motion vector of the projection surface 90 onto the pattern image. The predetermined area may also be, for example, the area of ​​a surface that satisfies a second backprojection condition. The second backprojection condition is a condition that the area is equal to the square of the product of the backprojection vector and the frame time. If at least one of the plain areas is elongated in one direction, the predetermined area may be, for example, the area of ​​a surface that satisfies the third backprojection condition. The third backprojection condition is a condition that the area is approximately zero. The predetermined area may be, for example, the area of ​​a surface that satisfies the fourth backprojection condition. The fourth backprojection condition is a condition that multiple plain areas of the pattern image appear in the image captured by camera 2, which will be described later. Note that the predetermined area does not necessarily have to be the area of ​​a surface that satisfies only one of the first backprojection condition, the second backprojection condition, the third backprojection condition, or the fourth backprojection condition. The predetermined area may be, for example, the area of ​​a surface that satisfies multiple of the first backprojection condition, the second backprojection condition, the third backprojection condition, or the fourth backprojection condition. The predetermined area may be, for example, the area of ​​a surface that satisfies the third backprojection condition and the fourth backprojection condition.

[0014] Since the pattern image satisfies the first pattern image condition, the amplitude spectrum of the pattern image contains many high-frequency components. Therefore, changes in the position, orientation, or shape of the pattern image are easier to detect than in an image that does not satisfy the first pattern image condition. A more detailed description of the pattern image will be provided later.

[0015] The content image is an image different from the pattern image that the user of the projection system 100 wants the spectators to see. Therefore, the content image is projected onto the projection surface 90 more times than the pattern image during a unit time. Because the content image is projected onto the projection surface 90 more times than the pattern image, the user can more easily see the content image than the pattern image. For example, the content image is projected 5 / 6 times the frame rate during a unit time, and the pattern image is projected 1 / 6 times the frame rate during a unit time. The multiple content images projected during a unit time do not necessarily all need to be the same. For example, they may be multiple different images that are perceived by the spectators as the content image that the user of the projection system 100 wants the spectators to see due to the persistence of vision effect.

[0016] When the three-dimensional state of the projection surface 90 changes, the projection system 100 projects onto the projection surface 90 a content image whose position, orientation, or shape has been changed in the same way as the projection surface 90 .

[0017] The projection system 100 includes a projector 1, a camera 2, and a control device 3. The projector 1 acquires a plurality of image data items one by one in sequence. The projector 1 projects still images represented by image data input in chronological order onto a projection surface 90. The projector 1 is, for example, a DLP (Digital Light Processing) projector equipped with an LED (Light Emitting Diode) and a DMD (Digital Micromirror Device).

[0018] The camera 2 captures an image of the projection surface 90 at a predetermined capture timing. The camera 2 transmits image data of the captured image (hereinafter referred to as the "camera image") to the control device 3. The capture timing is the timing when the pattern image is projected onto the projection surface 90. Because the camera 2 captures the projection surface 90 with the pattern image projected, the camera image shows an image in which the pattern image is superimposed on the object texture. The capture timing may be, for example, the timing when a predetermined signal (hereinafter referred to as the "capture instruction signal") is sent from the control device 3, or may be a predetermined timing. For simplicity's sake, the projection system 100 will be described below using as an example a case in which the camera 2 captures an image of the projection surface 90 at the timing when the capture instruction signal is received.

[0019] FIG. 2 is a timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in the first embodiment. 2 shows that for every five content images projected, a pattern image is projected once. The timing chart of the camera trigger in FIG. 2 shows the timing at which camera 2 takes a photograph. When the camera trigger is "Hi," camera 2 takes a photograph of projection surface 90. When the camera trigger is "Low," camera 2 does not take a photograph of projection surface 90.

[0020] The control device 3 transmits image data to the projector 1. Based on one camera image, the control device 3 calculates a value (hereinafter referred to as a "transformation value") for transforming the reference content image (hereinafter referred to as "image transformation") so that the content image projected onto the projection surface 90 has a position, orientation, or shape that corresponds to the three-dimensional state of the projection surface 90. The image transformation is, for example, a homography transformation. The image transformation may be, for example, a radial basis function approximation. The transformation value is, for example, a matrix representing the homography transformation. The transformation value may be, for example, a coefficient of a radial basis function. The reference content image is a content image projected onto the projection surface 90 when the projection surface 90 is in a predetermined first three-dimensional state (hereinafter referred to as the "initial state"). The control device 3 transforms the reference content image using the calculated transformation value. The control device 3 transmits image data of the reference content image after image transformation (hereinafter referred to as a "transmitted content image") to the projector 1. The control device 3 also performs image transformation on the reference pattern image using the calculated transformation value. The reference pattern image is a pattern image that is projected onto the projection surface 90 when the projection surface 90 is in an initial state. The control device 3 transmits image data of the reference pattern image after image deformation (hereinafter referred to as the "transmission pattern image") to the projector 1 at a predetermined timing, instead of the transmission content image. The number of times that the control device 3 transmits image data of the transmission pattern image during a unit time is less than the number of times that the control device 3 transmits image data of the transmission content image during a unit time.

[0021] The control device 3 includes a control unit 30 having a processor 91 such as a CPU (Central Processing Unit) and a memory 92 connected by a bus, and executes a program. By executing the program, the control device 3 functions as a device including the control unit 30, communication unit 31, and storage unit 32. More specifically, the processor 91 reads the program stored in the storage unit 32 and stores the read program in the memory 92. By the processor 91 executing the program stored in the memory 92, the control device 3 functions as a device including the control unit 30, communication unit 31, and storage unit 32.

[0022] The communication unit 31 includes a communication interface for connecting the device to the projector 1, the camera 2, and an external device. The communication unit 31 communicates with the projector 1, the camera 2, and the external device wirelessly or via a wired connection. For example, the communication unit 31 transmits image data of a transmission content image and a transmission pattern image to the projector 1. For example, the communication unit 31 receives image data of a camera image transmitted by the camera 2. For example, the communication unit 31 acquires image data of a reference content image output by an external device. The communication unit 31 outputs the acquired various information to the control unit 30.

[0023] The storage unit 32 is configured using a non-transitory computer-readable storage medium such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 32 stores various information related to the control device 3. The storage unit 32 stores, for example, image data of a reference content image. The storage unit 32 stores, for example, image data of a reference pattern image. The storage unit 32 stores, for example, image data of a transmission pattern image. The storage unit 32 stores, for example, image data of a transmission content image. The image data of the reference content image stored in the storage unit 32 may be information previously recorded in the storage unit 32, or may be information acquired via the communication unit 31 and then recorded in the storage unit 32. The storage unit 32 stores, for example, information indicating the history of transformation values. More specifically, the storage unit 32 stores the history of texture transformation values, the history of pattern transformation values, and the history of projector transformation values, which will be described later.

[0024] FIG. 3 is a schematic diagram showing an example of information indicating a history of deformation values ​​in the first embodiment. FIG. 3 shows an example of information indicating the transformation value history stored in the memory unit 32. The transformation value history information associates the following items: "Photography Timing," "Texture Transformation Value," "Pattern Transformation Value," and "Photographer Transformation Value." "Photography Timing" indicates the photographing timing. "Texture Transformation Value" indicates the texture transformation value calculated based on the camera image captured at the corresponding photographing timing. "Pattern Transformation Value" indicates the pattern transformation value calculated based on the camera image captured at the corresponding photographing timing. "Photographer Transformation Value" indicates the photographer transformation value calculated based on the camera image captured at the corresponding photographing timing. For example, in FIG. 3, data D101 indicates that the texture transformation value calculated based on the camera image captured at photographing timing T2 is Vc2, the pattern transformation value is Vp2, and the photographer transformation value is Vt2.

[0025] The storage unit 32 stores a reference texture image in advance. The reference texture image is an image obtained by capturing, with the camera 2, the projection surface 90 in a predetermined second predetermined three-dimensional state (hereinafter referred to as the "reference state"), illuminated with predetermined light that substantially uniformly illuminates the projection surface 90. The predetermined light is, for example, monochromatic light such as white light emitted by the projector 1. The predetermined light may also be, for example, indirect lighting in a room where the projection surface 90 is located. For simplicity, the following description will be given taking, as an example, a case where the predetermined light is white light emitted by the projector 1. As such, the reference texture image is an image of the projection surface 90 in a substantially uniformly illuminated state, and therefore the reference texture image is an image that shows the distribution of reflectance on the projection surface 90. Note that the reference state and the initial state may be the same or different.

[0026] (About the pattern image) The pattern image will now be described in detail. The pattern image may be any pattern image as long as it satisfies the first pattern image condition. When the projection surface 90 is flat, it is desirable that the pattern image satisfy, for example, the following second pattern image condition in addition to the first pattern image condition.

[0027] The second pattern image condition includes a condition that an area within a predetermined distance from the periphery (hereinafter referred to as the "outer area") has more plain areas with different brightness values ​​than areas other than the outer area.

[0028] The pattern image may further satisfy the following third pattern image condition in addition to the first and second pattern image conditions.

[0029] The third pattern image condition is that in an area other than the outer area, there are multiple types of plain areas with different brightness values, one or more of each type per unit area, and the area of ​​each plain area is larger than a specified area corresponding to the pattern of the object texture.

[0030] When the pattern image is a polygon, the pattern image may further satisfy the following fourth, fifth, and sixth pattern image conditions in addition to the first, second, and third pattern image conditions. A pattern image that satisfies such conditions is preferably used when the projection surface 90 is flat.

[0031] The fourth pattern image condition includes the condition that there are one or more plain areas of each type within a unit area in an area other than the outer area, and the total area of ​​the plain areas of each type is approximately the same per unit area regardless of type.

[0032] The fifth pattern image condition includes a condition that one or more plain areas of each type exist per unit area in the area other than the corner areas of the outer area, and the total area of ​​the plain areas of each type is approximately the same per unit area regardless of the type. The corner areas are areas of a predetermined size that include the vertices of a polygon and are within the outer area.

[0033] The sixth pattern image condition includes a condition that the corner area is a plain area of ​​one type.

[0034] When the pattern image is a circle or an ellipse, the pattern image may satisfy, for example, in addition to the first, second, and third pattern image conditions, a fourth pattern image condition and the following seventh pattern image condition: A pattern image satisfying such conditions is preferably used when the projection surface 90 is flat.

[0035] The seventh pattern image condition includes the condition that there are one or more plain areas of each type within a unit area in the region within the outer region, and the total area of ​​the plain areas of each type is approximately the same per unit area regardless of type.

[0036] 4 and 5 are diagrams showing an example of a pattern image that satisfies the pattern image conditions in the first embodiment. More specifically, Fig. 4 is a first diagram showing an example of a pattern image in the first embodiment. FIG. 4 shows pattern image A1. FIG. 4 shows that pattern image A1 is square. FIG. 4 shows that the outer region of pattern image A1 is a region with a thickness E extending from the outer periphery of pattern image A1 toward the inside. FIG. 4 shows that white regions and black regions are alternately arranged along the outer periphery of pattern image A1 in the outer region of pattern image A1. The white regions are regions with a predetermined brightness value, and the black regions are regions with a brightness value lower than that of the white regions. FIG. 4 shows that a square region with a side E including a vertex of the square is a white region. In FIG. 4, the square white region with a side E including a vertex of the square is a corner region of pattern image A1. When pattern image A1 is projected onto projection surface 90, the white regions are regions illuminated with light of a predetermined intensity. The black regions are regions illuminated with light of an intensity weaker than that of the light illuminated in the white regions. The intensity of light illuminated in the black regions may be any intensity as long as it is weaker than that of the light illuminated in the white regions, and may be, for example, zero.

[0037] FIG. 4 shows that the white areas in the outer region of pattern image A1 are rectangles with a length of L and a thickness of E, or squares with a side length of E. The length L is longer than E. FIG. 4 shows that the black areas in the outer region of pattern image A1 are rectangles with a length of L and a thickness of E. FIG. 4 shows that in regions other than the outer region of pattern image A1, white areas with a square shape of a side length of L and black areas with a square shape of a side length of L are arranged alternately.

[0038] FIG. 5 is a second diagram showing an example of a pattern image in the first embodiment. FIG. 5 shows pattern image A2. Pattern image A2 is a pattern image in which the white areas and black areas in pattern image A1 shown in FIG. 4 are reversed. That is, pattern image A1 and pattern image A2 have the same shape. The sum of the luminance value of each pixel in pattern image A1 (one side) and the luminance value of the corresponding pixel in pattern image A2 (the other side) is approximately the same regardless of the pixel. In FIG. 5, the square black area with one side E including the vertex of the square is the corner area of ​​pattern image A2. This concludes the detailed description of the pattern image.

[0039] 6 is a diagram showing an example of the functional configuration of the control unit 30 in the first embodiment. The control unit 30 includes a reference content image acquisition unit 301, a camera transformation value calculation unit 302, a projector transformation value acquisition unit 303, a transmission content image acquisition unit 304, a transmission pattern image acquisition unit 305, and a communication control unit 306.

[0040] The reference content image acquisition unit 301 acquires image data of the reference content image. The reference content image acquisition unit 301 may read image data of the reference content image recorded in the storage unit 32, or may acquire image data of the reference content image input via the communication unit 31.

[0041] Each time a camera image is acquired, the camera transformation value calculation unit 302 calculates a transformation value (hereinafter referred to as a "texture transformation value") that represents a change in the position, orientation, or shape of the projection surface 90 as seen from the camera 2, based on the camera image. More specifically, each time a camera image is acquired, the camera transformation value calculation unit 302 calculates a texture transformation value based on an image of the object texture captured in the camera image (hereinafter referred to as an "object texture image"). The camera transformation value calculation unit 302 may calculate the texture transformation value by any method that can calculate a texture transformation value. For example, the camera transformation value calculation unit 302 may calculate the texture transformation value by a convergence calculation that uses a predetermined transformation value as an initial condition and updates the transformation value in accordance with a predetermined rule. In this case, the convergence condition for the convergence calculation is that the degree of match between the object texture image captured in the camera image and the reference texture image after image transformation is equal to or greater than a predetermined degree at a target location. The target location is the location that would appear in the camera image if the reference texture image after image transformation were projected onto the projection surface 90. The degree of match is, for example, the smallness of the sum of squared differences in brightness values ​​between the object texture image and the reference texture image after image deformation. The camera deformation value calculation unit 302 may adjust the brightness of the object texture during the convergence calculation. Specifically, during the convergence calculation, the brightness of the object texture may be divided by the average brightness value of all pixels in the reference texture image. This reduces the difference in brightness values ​​between the reference texture image and the object texture, thereby reducing the possibility that the convergence calculation will converge at an inappropriate saddle point or local solution, or will diverge without converging. The calculated texture deformation value may be recorded in the storage unit 32.

[0042] For example, when the pattern image has two types of regions, a white region and a black region, the camera deformation value calculation unit 302 may calculate the texture deformation value using each position within a region that satisfies the following target region condition, rather than the entire projection surface 90, as the target region. The target region condition is a region where a white region of the previous pattern image was located at the immediately previous capture timing. The previous pattern image is a pattern image captured in a camera image captured at the immediately previous capture timing. For example, the target region does not have to be the entire region within the white region of the previous pattern image, but may be, for example, a portion of the white region of the previous pattern image. More specifically, the target region may be, for example, a region within the white region of the previous pattern image, located at a distance from the periphery of the white region by a distance corresponding to the speed of movement or rotation of the projection surface 90 and the interval between capture timings. The distance is, for example, v × Δt, where v is the speed of movement of the projection surface 90 and Δt is the interval between capture timings. The target region may be a black region instead of a white region, or may be a portion of the black region. The target area may be a white area and a black area, or a part of the white area and a part of the black area.

[0043] When the camera transformation value calculation unit 302 acquires the texture transformation value, it has not yet determined where the pattern image is located within the just-acquired camera image. Therefore, if the target area is the entire plain area, it is difficult to determine whether the area represented by each pixel within the target area captured in the camera image is a white area or a black area of ​​the pattern image. On the other hand, if the target area is a portion of the plain area, the target area is a white area of ​​the pattern image. Therefore, the texture transformation value can be calculated more accurately than when the target area is the entire plain area.

[0044] Fig. 7 is a diagram showing an example of a target location in the first embodiment. Fig. 7 shows an example of a camera image of the experimental results. In Fig. 7, the target locations are positions within region R1. Distance d is the distance between the periphery of region R1 and the periphery of the white region.

[0045] Image transformation using texture transformation values ​​is a transformation that causes the object texture image of a target location in the camera image to match the image of the target location in the reference texture image to a degree of matching that is equal to or greater than a predetermined degree of matching. Therefore, the texture transformation values ​​indicate the correspondence between pixels in the camera image and the reference texture image.

[0046] The projector transformation value acquisition unit 303 acquires the projector transformation value based on the camera image and the texture transformation value. The projector transformation value is a transformation value that represents the image transformation that transforms the reference content image into the transmission content image. The projector transformation value acquisition unit 303, for example, first executes a pattern image extraction process. The pattern image extraction process is a process that extracts a pattern image from the camera image. The pattern image extraction process is, for example, the following process. The projector transformation value acquisition unit 303 first executes image transformation on the camera image using the texture transformation value. Next, the projector transformation value acquisition unit 303 divides the luminance value of each pixel of the camera image after image transformation by the luminance value of the corresponding pixel in the reference texture image, using the correspondence between pixels indicated by the texture transformation value. The reason why a pattern image is extracted by division is as follows.

[0047] The distribution of reflectance on the projection surface 90 is superimposed on the pattern image captured in the camera image. Therefore, the brightness value of the image captured in the camera image is approximately proportional to the sum of the brightness value of the illuminated pattern image and the intensity of the indoor background light multiplied by the reflectance. Note that the indoor background light is assumed to be approximately uniform regardless of position on the projection surface 90. On the other hand, the reference texture image is approximately proportional to the product of the intensity of the uniform light illuminated when it was captured and the reflectance. Therefore, if the brightness value of each pixel in the camera image is divided by the brightness value of the corresponding pixel in the reference texture image, the reflectance term disappears, and the magnitude relationship between the pixels resulting from the division matches the magnitude relationship between the brightness values ​​of the pixels in the pattern image. This is why the pattern image is extracted by division.

[0048] Next, the projector transformation value acquisition unit 303 acquires a transformation value representing an image transformation that brings the degree of match between the reference pattern image and the extracted pattern image to a predetermined degree or more. For example, the projector transformation value acquisition unit 303 acquires a transformation value representing an image transformation that minimizes the sum of squared differences between the luminance values ​​of the reference pattern image and the luminance values ​​of the extracted pattern image. The acquired transformation value is the pattern transformation value.

[0049] Next, the projector transformation value acquisition unit 303 acquires a projector transformation value based on the acquired pattern transformation value and the projector transformation value at the immediately preceding capture timing. To acquire the projector transformation value, the projector transformation value acquisition unit 303 first acquires the positions of the camera reference point and the projector reference point. The camera reference point is a point where a plurality of reference points (hereinafter referred to as "pattern reference points") predefined on the pattern image are projected onto the camera image based on the pattern transformation value. The projector reference point is a point where the pattern reference point is projected onto the projector image based on the projector transformation value at the immediately preceding capture timing. The projector image is the image projected by the projector 1. The camera reference point and the projector reference point are pre-assigned to a different projector reference point for each camera reference point.

[0050] The projector transformation value acquisition unit 303 acquires a transformation value (hereinafter referred to as "target transformation value") representing an image transformation (hereinafter referred to as "target image transformation") in which the projection destination of each camera reference point coincides with the corresponding projector reference point. The projector transformation value acquisition unit 303, for example, minimizes the sum of squares of the distance between the projection destination of the camera reference point and the corresponding projector reference point through convergence calculation. The image transformation that minimizes the sum of squares is the target image transformation, and the transformation value representing the image transformation that minimizes the sum of squares is the target transformation value. The target image transformation is, for example, a homography transformation. The target image transformation may also be, for example, an affine transformation for each mesh region when the image is divided into triangular meshes.

[0051] By using the image transformation from the camera image to the projector image obtained in this way, the texture image that appears on the camera image can be mapped onto the projector image. The projector transformation value acquired by the projector transformation value acquisition unit 303 represents an image transformation that aligns a predetermined position of the content image with a predetermined position of the texture image mapped onto the projector image. The projector transformation value acquisition unit 303 acquires the projector transformation value using the same procedure as for the target transformation value.

[0052] The transmission content image acquisition unit 304 acquires image data of the transmission content image based on the reference content image and the projector transformation value. Specifically, the transmission content image acquisition unit 304 performs image transformation represented by the acquired projector transformation value on the reference content image. The reference content image after image transformation based on the projector transformation value is the transmission content image.

[0053] The transmission pattern image acquisition unit 305 acquires image data of the transmission pattern image based on the reference pattern image and the projector transformation value. Specifically, the transmission pattern image acquisition unit 305 performs image transformation represented by the projector transformation value on the reference pattern image. The reference pattern image after the image transformation represented by the projector transformation value is performed is the transmission pattern image.

[0054] FIG. 8 is an example of an experimental result showing a pattern image before acquiring the projector transformation value in the first embodiment and a pattern image after image transformation represented by the projector transformation value is performed. Fig. 8(A) shows the pattern image captured in the camera image before the projector transformation value is acquired. Fig. 8(B) shows the pattern image captured in the camera image after the image transformation represented by the projector transformation value is performed. Fig. 8 shows that the pattern image also moves in accordance with the movement of the projection surface 90.

[0055] The communication control unit 306 transmits the transmission content image and the transmission pattern image to the projector 1 via the communication unit 31. The communication control unit 306 transmits a shooting instruction signal to the camera 2 at the timing when the transmission pattern image is transmitted.

[0056] FIG. 9 is a flowchart showing an example of the flow of processing executed by the projection system 100 in the first embodiment. The projector 1 projects still images represented by image data input in chronological order onto the projection surface 90 in the order they were input (step S101). The camera 2 captures the projection surface 90 at the capture timing (step S102). The camera transformation value calculation unit 302 then calculates a texture transformation value based on the camera image captured by the camera 2 (step S103). The projector transformation value acquisition unit 303 then acquires a pattern transformation value based on the camera image and the texture transformation value (step S104). The projector transformation value acquisition unit 303 then acquires a projector transformation value based on the pattern transformation value acquired in step S104 and the projector transformation value at the previous capture timing (step S105). The transmission content image acquisition unit 304 then acquires image data of the transmission content image based on the reference content image and the projector transformation value (step S106). The transmission pattern image acquisition unit 305 then acquires image data of the transmission pattern image based on the reference pattern image and the projector transformation value (step S107). Next, the communication control unit 306 transmits image data of the transmission content image and image data of the transmission pattern image to the projector 1 in a predetermined order via the communication unit 31 (step S108). The projector 1 projects the transmission content image or the transmission pattern image onto the projection surface 90 in the order in which the image data was received (step S109). After step S109, the processes of steps S102 to S109 are repeated until a predetermined termination condition is satisfied. The predetermined termination condition is, for example, a condition in which power is not supplied to the control device 3.

[0057] The projection system 100 configured in this manner controls the content image to be projected based on a pattern image that satisfies the first pattern image condition, and is therefore able to project a content image based on a single pattern image in accordance with the movement, rotation, or deformation of the projection surface 90. Therefore, in a technology in which an image projected on a projection surface is moved in accordance with the movement of the projection surface, the projection system 100 can suppress deviations in the positions on the projection surface of multiple images that are projected in sequence onto the moving projection surface.

[0058] (Second embodiment) If the performance of the projector 1 or camera 2 included in the projection system 100 does not meet a predetermined performance standard, the positional deviation of the image on the projection surface will be greater than when the device performance meets the predetermined performance standard. For example, the positional deviation will be greater if the lens of the projector 1 has distortion of a predetermined magnitude or greater, or if there is a delay of a predetermined time or more between when the camera 2 captures the image and when the projector 1 projects it (hereinafter referred to as "delay time Δt"). Furthermore, the positional deviation will be greater if the projection surface 90 is not flat. Specifically, the positional deviation is the deviation between a virtual image and the image projected on the projection surface 90 (hereinafter referred to as "image deviation"). The virtual image is an image that would be displayed on the projection surface 90 if the projection surface 90 were flat and the performance of the projector 1 or camera 2 met the predetermined performance standard. Therefore, to correct the image deviation, the control unit 30 may further include a correction unit 307.

[0059] 10 is a diagram showing an example of the system configuration of a projection system 100a according to the second embodiment. Hereinafter, components having the same functions as those in the projection system 100 will be assigned the same reference numerals as those in FIG. 1, and descriptions thereof will be omitted.

[0060] The projection system 100a differs from the projection system 100 in that it includes a control device 3a instead of the control device 3. The control device 3a differs from the control device 3 in that it includes a control device 30a instead of the control unit 30. The control device 30a differs from the control device 30 in that it further includes a correction unit 307 in addition to the functional units included in the control unit 30.

[0061] 11 is a diagram showing an example of the functional configuration of the control unit 30a in the second embodiment. Hereinafter, components having the same functions as the control unit 30 are denoted by the same reference numerals as in FIG. 6, and description thereof will be omitted.

[0062] The correction unit 307 changes the projector deformation value to a value that reduces image misalignment. Each time a camera image is captured, the correction unit 307 acquires a value (hereinafter referred to as a "projector deformation correction value") that corrects the projector deformation value acquired by the projector deformation value acquisition unit 303 based on the projector deformation value at the time of capture two images prior to that time and the projector deformation value at the time of capture one image prior to that time. The projector deformation correction value is a value that reduces image misalignment. Specifically, the correction unit 307 acquires, as the projector deformation correction value, a value that moves four predetermined positions (hereinafter referred to as "second control points") on the pattern image projected on the projection surface 90 closer to four predetermined positions (hereinafter referred to as "first control points") on the projection surface 90. The projector deformation value at the time of capture two images prior to that time is the camera deformation value acquired based on the camera image captured at the time of capture two images prior to that time. The projector deformation value at the immediately previous shooting timing is the camera deformation value obtained based on the camera image captured at the immediately previous shooting timing.

[0063] Fig. 12 is an explanatory diagram illustrating a first control point and a second control point in the second embodiment. In Fig. 12, an arrow indicates that a second control point moves to a first control point. Fig. 12 shows that multiple second control points do not move to one first control point.

[0064] 13 is a flowchart showing an example of the flow of processing executed by the projection system 100a in the second embodiment. Hereinafter, processing that is the same as processing executed by the projection system 100 is assigned the same reference numerals as in FIG. 9, and description thereof will be omitted.

[0065] After step S105, the correction unit 307 acquires a projector deformation correction value based on the projector deformation value from the two capture timings before and the projector deformation value from the capture timing immediately before that (step S110). Next, the correction unit 307 corrects the projector deformation value acquired in step S104 using the acquired projector deformation correction value (step S111). The correction is performed on the projector deformation value by a predetermined calculation that depends on the definition of the projector correction value. The predetermined calculation may be, for example, multiplying the projector deformation value by the projector deformation correction value, or adding the projector deformation value to the projector deformation value. After step S111, the process of step S105 is performed using the corrected projector deformation value.

[0066] The projection system 100a of the second embodiment configured as described above acquires a projector deformation correction value and corrects the projector deformation value using the acquired projector deformation correction value. Therefore, the projection system 100a can project an image with less image deviation than the projection system 100, which is caused by the projector 1 or the camera 2 not satisfying a predetermined performance or the projection surface 90 not being flat.

[0067] (Variation) The transmission content image acquisition unit 304 does not necessarily need to extract a pattern image and then acquire a projector matrix based on the extracted pattern image. The transmission content image acquisition unit 304 may calculate the projector deformation value using the edges of the pattern image.

[0068] The correction unit 307 may be, for example, a Smith predictor. Furthermore, when acquiring the projector deformation correction value, the correction unit 307 does not necessarily need to use four points on the projection surface 90. Five or more points or fewer than four points may be used.

[0069] The content image and the pattern image do not necessarily have to be one type of pattern image projected in the order shown in Fig. 2. For example, two types of pattern images each having a white area and a black area, in which the white area and the black area are reversed, may be projected once in succession. The two types of pattern images are, for example, the pattern image shown in Fig. 4 (hereinafter referred to as "first pattern image") and the pattern image shown in Fig. 5 (hereinafter referred to as "second pattern image").

[0070] FIG. 14 is a first timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in the modified example. 14 shows that when the content image is projected five times, the first pattern image is projected once, and then the second pattern image is projected once. FIG. 14 shows that the camera 2 captures the first pattern image but does not capture the second pattern image. By projecting the first pattern image and the second pattern image in this manner, it appears to the user that an average image of the first pattern image and the second pattern image is being projected onto the projection surface 90. This makes it difficult for the user to visually recognize the patterns of the first pattern image and the second pattern image.

[0071] In addition, when two types of pattern images, a first pattern image and a second pattern image, are projected, the first pattern image and the second pattern image may be switched and projected each time a pattern image is projected, as shown in Figure 15 below.

[0072] FIG. 15 is a second timing chart showing an example of the order in which content images and pattern images are projected and an example of predetermined image capturing timing in the modified example. 15 shows that when the content image is projected five times, the first pattern image is projected once, and then when the content image is captured five times, the second pattern image is projected once. By alternately projecting the first pattern image and the second pattern image in this manner, it appears to the user that an average image of the first pattern image and the second pattern image is being projected onto the projection surface 90. This makes it difficult for the user to visually recognize the patterns of the first pattern image and the second pattern image.

[0073] Note that the projection systems 100 and 100a may project, for example, a red image, a green image, and a blue image of one content image onto the projection surface 90 while switching between them in sequence. The red image is an image that represents the red luminance value of each pixel of the content image and is projected onto the projection surface with red light. The green image is an image that represents the green luminance value of each pixel of the content image and is projected onto the projection surface with red light. The blue image is an image that represents the blue luminance value of each pixel of the content image and is projected onto the projection surface with blue light.

[0074] The first pattern image and the second pattern image may be any pattern images as long as they satisfy the following two conditions: one condition is that the shapes are identical, and the other condition is that the sum of the luminance value of each pixel in the first pattern image and the luminance value of the corresponding pixel in the second pattern image is approximately the same regardless of the pixel.

[0075] Even if the position, orientation, or shape of the projection surface 90 does not necessarily change, as long as the positional relationship between the projection surface 90, the projector 1, and the camera 2 changes, the projection systems 100 and 100a can project a content image onto the projection surface 90 in accordance with the change in the positional relationship. The positional relationship between the projection surface 90, the projector 1, and the camera 2 changes, for example, when the position or orientation of the projector 1 or the position or orientation of the camera 2 changes.

[0076] The pattern image on the projection surface 90 is located within the area where the object texture is formed or drawn. The size of the pattern image on the projection surface 90 is larger than the area on the projection surface 90 that is viewed by a viewer. The content image may be a color image or a binary image.

[0077] The light emitted by projector 1 to project an image may be visible light, X-rays, ultraviolet light, infrared light, or terahertz waves. Camera 2 may be capable of receiving not only visible light, but also X-rays, ultraviolet light, infrared light, or terahertz waves.

[0078] All or part of the functions of the control devices 3 and 3a may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. The program may be transmitted via a telecommunications line.

[0079] The control devices 3 and 3a may be implemented using a plurality of information processing devices connected to each other so as to be able to communicate with each other via a network. In this case, the functional units of the control device 3 may be distributed and implemented in the plurality of information processing devices. For example, the reference content image acquisition unit 301, the camera transformation value calculation unit 302, the projector transformation value acquisition unit 303, the transmission content image acquisition unit 304, the transmission pattern image acquisition unit 305, the communication control unit 306, and the correction unit 307 may each be implemented in a different information processing device.

[0080] The transmission content image acquisition unit 304 is an example of a first conversion unit. The transmission pattern image acquisition unit 305 is an example of a second conversion unit. The control device 3 is an example of a projection system control device.

[0081] Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and designs within the scope of the gist of the present invention are also included. For example, the transmission content image acquisition unit 304 and the transmission pattern image acquisition unit 305 do not need to be different functional units. For example, the control unit 30 or 30a may include a transmission image acquisition unit 308 instead of the transmission content image acquisition unit 304 and the transmission pattern image acquisition unit 305. The transmission image acquisition unit 308 executes the processing executed by the transmission content image acquisition unit 304 and the processing executed by the transmission pattern image acquisition unit 305.

[0082] Fig. 16 is a diagram showing an example of a transmission image acquisition unit 308 in a modified example. For ease of explanation, Fig. 16 shows a control unit 30a including the transmission image acquisition unit 308, but the control unit 30 may include the transmission image acquisition unit 308.

[0083] The first pattern image condition does not necessarily have to include both the condition of having multiple types of plain areas with different brightness values ​​and the condition that the area of ​​at least one of the multiple types of plain areas is larger than a predetermined area. The first pattern image condition may be a condition that only includes the condition of having multiple types of plain areas with different brightness values. [Explanation of symbols]

[0084] 100, 100a...Projection system, 1...Projector, 2...Camera, 3, 3a...Control device, 30, 30a...Control unit, 31...Communication unit, 32...Storage unit, 301...Reference content image acquisition unit, 302...Camera deformation value calculation unit, 303...Projector deformation value acquisition unit, 304...Transmission content image acquisition unit, 305...Transmission pattern image acquisition unit, 306...Communication control unit, 307...Correction unit, 308...Transmission image acquisition unit

Claims

1. a projector that projects a content image and a pattern image that satisfies predetermined conditions onto a projection surface that is not plain; a camera that captures an image of the projection surface at a timing when the pattern image is projected; a camera deformation value calculation unit that calculates a texture deformation value that represents a change in the position, orientation, or shape of the projection surface based on a camera image that is a result of photographing by the camera; a projector transformation value acquisition unit that acquires a projector transformation value of an image transformation that converts an image projected by the projector into an image corresponding to the state of the projection surface based on the pattern image captured in the camera image and the texture transformation value; Equipped with The predetermined conditions include a condition that the image has a plurality of types of plain areas with different brightness values, the camera deformation value calculation unit calculates the texture deformation value based on a pattern on the surface of the projection surface that appears in an area that is within the plain area and that is located at a position that is separated from the periphery of the plain area by a distance that corresponds to the speed of movement or rotation of the projection surface and the interval between timings of photographing by the camera; Projection system.

2. The predetermined condition further includes a condition that the outer region, which is a region within a predetermined distance from the outer periphery, has more plain regions with different brightness values ​​than regions other than the outer region.

10. The projection system of claim 1.

3. The predetermined conditions further include a condition that, in an area other than the outer area, there are one or more of the plain areas of each type having different brightness values ​​per unit area, and the area of ​​each of the plain areas is larger than a predetermined area corresponding to the pattern on the surface of the projection screen.

3. The projection system of claim 2.

4. The predetermined conditions further include a condition that the pattern image is a polygon; a condition that one or more of each type of plain area is present in a unit area in a region other than the outer region, and the total area of ​​the plain areas for each type is approximately the same per unit area regardless of type; a condition that a region within the outer region of a predetermined size that includes a vertex of the polygon is defined as a corner region, and one or more of each type of plain area is present in a unit area in a region within the outer region other than the corner regions, and the total area of ​​the plain areas for each type is approximately the same per unit area regardless of type; and a condition that the corner region is a plain region of one type.

4. The projection system of claim 3.

5. The predetermined conditions further include a condition that the pattern image is a circle or an ellipse, a condition that one or more of each type of plain area is present in a unit area in the area other than the outer area, and the total area of ​​the plain areas for each type is approximately the same per unit area regardless of the type, and a condition that one or more of each type of plain area is present in a unit area in the area within the outer area, and the total area of ​​the plain areas for each type is approximately the same per unit area regardless of the type.

4. The projection system of claim 3.

6. the projector deformation value acquisition unit further acquires the projector deformation value based on an image showing a distribution of reflectance on the projection surface.

10. The projection system of claim 1.

7. two types of pattern images of the same shape are projected onto the projection surface, and a sum of the luminance value of each pixel in one pattern image and the luminance value of each corresponding pixel in the other pattern image is substantially the same regardless of the pixel; 10. The projection system of claim 1.

8. a correction unit that defines an image displayed on the projection surface when the projection surface is flat and the projector 1 or the camera 2 satisfies a predetermined performance as a virtual image, defines a deviation between the virtual image and the image projected by the projector as an image deviation, and corrects the projector deformation value acquired by the projector deformation value acquisition unit so as to reduce the image deviation based on a projector deformation value acquired based on a camera image captured at the shooting timing two shots before and a projector deformation value acquired based on a camera image captured at the shooting timing one shot before; The projection system of claim 1 further comprising:

9. a projector that projects a content image and a pattern image that satisfies predetermined conditions onto a projection surface that is not plain; a camera that captures an image of the projection surface at a timing when the pattern image is projected; a camera deformation value calculation unit that calculates a texture deformation value that represents a change in the position, orientation, or shape of the projection surface based on a camera image that is a result of photographing by the camera; a projector transformation value acquisition unit that acquires a projector transformation value of an image transformation that converts an image projected by the projector into an image corresponding to the state of the projection surface, based on the pattern image captured in the camera image and the texture transformation value; a transmission image acquisition unit that executes a process of image-transforming the content image based on the projector transformation value and a process of image-transforming the pattern image based on the projector transformation value; Equipped with The predetermined conditions include a condition that the image has a plurality of types of plain areas with different brightness values, and a condition that the area of ​​at least one of the plurality of types of plain areas is larger than a predetermined area; the camera deformation value calculation unit calculates the texture deformation value based on a pattern on the surface of the projection surface that appears in an area that is within the plain area and that is located at a position that is separated from the periphery of the plain area by a distance that corresponds to the speed of movement or rotation of the projection surface and the interval between timings of photographing by the camera; Projection system control device.

10. a projection step of projecting the content image and the pattern image that satisfies a predetermined condition onto a projection surface that is not plain; an imaging step of imaging the projection surface at an imaging timing when the pattern image is projected; a camera deformation value calculation step of calculating a texture deformation value representing a change in the position, orientation, or shape of the projection surface based on a camera image that is a result of the photographing step; a projector transformation value acquisition step for acquiring a projector transformation value of an image transformation for converting the image projected in the projection step into an image corresponding to the state of the projection surface, based on the pattern image captured in the camera image and the texture transformation value; a transmission image acquisition step of executing a process of image-deforming the content image based on the projector deformation value and a process of image-deforming the pattern image based on the projector deformation value; and The predetermined conditions include a condition that the image has a plurality of types of plain areas with different brightness values, the camera transformation value calculation step calculates the texture transformation value based on a pattern on the surface of the projection surface that appears in an area that is within the plain area and that is located at a distance from the periphery of the plain area that corresponds to the speed of movement or rotation of the projection surface and the interval between timings of photographing by the camera; Projection method.

Images