Flower field illumination system, control device, and illumination method

Abstract

[Problem] To beautifully illuminate a flower field. [Solution] An illumination system 100 comprises: an imaging device 10 disposed at a position where an image of a flower field can be captured; a projection device 20 disposed at a position where image light for projection mapping can be projected against the flower field; and a control device 40 connected to the imaging device 10 and the projection device 20. The control device 40 analyses the image of the flower field captured by the imaging device 10 to specify a flower portion within the captured image, and controls the projection device 20 such that the image light is projected against flowers in the flower field on the basis of the result of the analysis.

Description

Flower garden lighting system, control device and lighting method

[0001] The present invention relates to a system and a control device for lighting a flower garden. The present invention also relates to a method for lighting a flower garden.

[0002] Projection mapping, which projects image light from a projector onto an object to create various visual effects, is now widespread. Various technologies for generating projected images for projection mapping have been proposed.

[0003] For example, Patent Document 1 discloses an image projection system including a projection device, an imaging device, and a computing device. In the system of Patent Document 1, conversion information for converting between the camera coordinates of the imaging device and the projector coordinates of the image projection device is generated based on the positional relationship between a predetermined pattern image projected by the image projection device and the same pattern image captured by the imaging device. In this system, when the imaging device captures an image of a subject including a person, the system detects the position of the person in the camera coordinates, and converts the position information of the person in the camera coordinates into position information in the projector coordinates based on the conversion information generated in advance, thereby projecting image light from the projector onto the person in a superimposed manner.

[0004] Japanese Patent Application Laid-Open No. 2018-97148

[0005] It has been known for some time that projection mapping technology can be used to project image light onto people, as in Patent Document 1, or onto other structures such as buildings. The inventors of the present invention therefore considered applying projection mapping technology to illuminate flower fields. However, little progress has been made in projecting light onto flower fields that contain flowers. In particular, flower fields contain numerous flowers, leaves, stems, and other soil parts, creating complex three-dimensional shapes. Therefore, it is necessary to consider how to project light to make flower fields look beautiful.

[0006] Therefore, a main object of the present invention is to provide a technique that can beautifully illuminate a flower garden.

[0007] A first aspect of the present invention relates to a lighting system for a flower field. More specifically, the lighting system according to the present invention is for performing projection mapping on a flower field. The lighting system according to the present invention includes a camera arranged in a position capable of capturing an image of the flower field, a projection device arranged in a position capable of projecting image light for projection mapping onto the flower field, and a control device connected to the camera and projection device. The control device also includes image analysis means and projection control means. The image analysis means analyzes an image of the flower field captured by the camera and identifies flower portions in the captured image. The projection control means controls the projection device to project image light onto flowers in the flower field based on the analysis results of the image analysis means. Note that a "flower" refers to a corolla, which is a collection of multiple petals, of a single plant planted in the soil of the flower field. In this way, by identifying flowers among the plants in the flower field through image analysis and projecting image light onto the flowers, the flower field can be beautifully illuminated. Note that in the present invention, image light may be projected only onto the flowers in the flower field, or, as described below, image light may be projected onto both the flowers and grass in the flower field. However, it is preferable not to project the image light onto the soil in the flower field.

[0008] In the lighting system according to the present invention, the image analysis means may analyze an image of the flower field captured by the image capture device and identify the flower and grass portions in the captured image. In this case, the projection control means may control the projection device to project video light onto the flowers and grass in the flower field based on the analysis results of the image analysis means. Note that "grass" refers to the parts of a plant planted in the soil of the flower field, excluding the flowers, and includes shoots such as leaves, stems, and trunks. However, "grass" does not include roots buried in the soil. By projecting video light not only onto the flowers but also onto the surrounding grass, depending on the type of plant in the flower field, the flower field can appear more beautiful than if only the flowers were illuminated. For example, for plants with relatively large corollas, such as sunflowers and cosmos, it is sufficient to illuminate only the flowers, while for plants with relatively small corollas, such as nemophila, it is sufficient to illuminate both the flowers and the grass.

[0009] In the lighting system according to the present invention, the projection control means may control the projection device so that the lighting conditions for the flowers in the flower field are different from the lighting conditions for the grass. The lighting conditions here include the amount of light, illuminance, and color. For example, it is preferable to make the amount of light projected onto the grass smaller than the amount of light projected onto the flowers. This makes it possible to make the light stand out while illuminating the grass around the flowers. Conversely, if it is desired to make the grass stand out, it is preferable to make the amount of light projected onto the grass larger than the amount of light projected onto the flowers.

[0010] In the lighting system according to the present invention, the image control means preferably generates a flower mask image in which the flower portion of the photographed image of the flower field is a projection area and the rest is a non-projection area. In this case, the projection control means controls the projection device to project video light onto the flowers in the flower field based on the flower mask image. In this way, by generating the flower mask image and controlling the projection device, the flowers in the flower field can be illuminated with high precision. In particular, since the flower mask image can be converted from the camera coordinate system to the projector coordinate system, generating the flower mask image can improve the processing efficiency of generating the projection image.

[0011] In the lighting system according to the present invention, the image control means may generate a flower mask image in which the flower portion of the photographed image of the flower field is a projection area and the rest is a non-projection area, and a grass mask image in which the grass portion of the photographed image of the flower field is a projection area and the rest is a non-projection area. In this case, the projection control means controls the projection device to project video light onto the flowers in the flower field based on the flower mask image, and to project video light onto the grass in the flower field based on the grass mask image. By generating the flower mask image and the grass mask image in this way, the lighting conditions for the flowers and the grass can be controlled individually.

[0012] In the system according to the present invention, it is preferable that the flower mask image represents the projection area of ​​the image light using one of RGB values, and the grass mask image represents the projection area of ​​the image light using another one of RGB values. For example, if the projection area of ​​the flower mask image is represented by an R value, the projection area of ​​the grass mask image can be represented by a G value or a B value. In this case, the flower mask image and the grass mask image can be combined into a single image represented by RGB values. This reduces the data volume of the mask images and improves the processing efficiency of generating the projection image.

[0013] A second aspect of the present invention is a control device constituting the system according to the first aspect. Specifically, the control device according to the present invention is connected to a photographing device positioned to photograph a flower field and a projection device positioned to project video light for projection mapping onto the flower field, and is configured with one or more computers. The control device according to the present invention includes image analysis means for analyzing an image of the flower field photographed by the photographing device and identifying flower portions in the photographed image, and projection control means for controlling the projection device to project video light onto the flowers in the flower field based on the analysis results of the image analysis means. The image analysis means preferably generates a flower mask image in which the flower portions in the photographed image of the flower field are projected with video light and the rest are non-projected, and a grass mask image in which the grass portions in the photographed image of the flower field are projected with video light and the rest are non-projected. Furthermore, the projection control means preferably projects video light onto the flowers in the flower field based on the flower mask image, and controls the projection device to project video light onto the grass in the flower field based on the grass mask image. Note that a computer program for causing one or more computers to function as the control device according to the second aspect may also be considered as one aspect of the present invention. This computer program may be pre-installed on the computer, may be downloadable via the Internet, or may be recorded on a non-transitory recording medium such as a CD-ROM.

[0014] A third aspect of the present invention relates to a method for illuminating a flower field. In the illumination method according to the present invention, a photographing device acquires a photographed image of the flower field (photographing step). A control device analyzes the photographed image of the flower field to identify flower portions in the photographed image, and generates a control signal for a projection device to project video light for projection mapping onto the flowers in the flower field based on the analysis results of the photographed image (control step). The projection device projects onto the flowers in the flower field based on the control signal (projection step). The control device preferably generates a flower mask image in which the flower portions in the photographed image of the flower field are projected with video light and the rest are non-projected, and a grass mask image in which the grass portions in the photographed image of the flower field are projected with video light and the rest are non-projected. The control device preferably projects video light onto the flowers in the flower field based on the flower mask image, and controls the projection device to project video light onto the grass in the flower field based on the grass mask image.

[0015] According to the present invention, a flower garden can be beautifully illuminated.

[0016] Fig. 1 is a schematic diagram showing an example of the overall configuration of the system of the present invention. Fig. 2 is a block diagram showing an example of functional elements of the system of the present invention. Fig. 3 is a schematic diagram showing an example of a captured image before projection of imaging light. Fig. 4 is a schematic diagram showing an example of a mask image. Fig. 5 is a schematic diagram showing an example of a captured image after projection of imaging light.

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, but also includes appropriate modifications of the embodiments below within the scope obvious to those skilled in the art.

[0018] FIG. 1 shows the overall configuration of a lighting system 100 according to an embodiment of the present invention. As shown in FIG. 1, the lighting system 100 according to this embodiment is used to illuminate a flower field containing multiple plants using projection mapping. Projection mapping is a technology that uses a projector to project images onto three-dimensional objects such as buildings and other objects. This technology distorts and adjusts the image to match the shape and characteristics of the object, creating a visual effect that makes the object itself appear to be changing. In this embodiment, applying this technology to a flower field makes it possible to overlay images on the flowers and the surrounding environment, creating a fantastical space. For example, changing the color of the flowers, adding the effect of scattering light particles, or projecting seasonal images can provide viewers with a new experience. In this flower field, multiple plants are planted in soil. Some of the plants in this flower field have flowers (corollas). In addition to flowers, the plants also have grass (leaves, stems, and / or trunks). The types of flowering plants are not limited, but examples include sunflowers, cosmos, nemophila, etc. Furthermore, the plants planted in the flower field do not have to be of only one type, and many types of plants may be planted in the flower field. Furthermore, in addition to flowering plants, non-flowering plants may also be planted in the flower field.

[0019] As shown in FIG. 1 , the lighting system 100 includes an image capture device 10, a projection device 20, a detection device 30, and a control device 40. The image capture device 10, the projection device 20, and the detection device 30 are each connected to the control device 40 via a wired or wireless connection. The control device 40 may be a cloud-based computer, in which case the image capture device 10, the projection device 20, and the detection device 30 may be connected to the control device 40 via the Internet. Although the example shown in FIG. 1 illustrates one image capture device 10 and one projection device 20, multiple of each of these devices may be provided. One image capture device 10 may be provided for one projection device 20, or multiple image capture devices 10 may be provided for one projection device 20, or multiple projection devices 20 may be provided for one image capture device 10.

[0020] The photographing device 10 is positioned so that it can photograph the entire flower field or at least a portion of it. For example, when using one photographing device 10, the flower field can be positioned within the angle of view of the photographing device 10. When using multiple photographing devices 10, each photographing device 10 can photograph a portion of the flower field, and the flower field can be positioned within the combined angle of view of the multiple photographing devices 10. A known digital camera can be used as the photographing device 10. Image data acquired by the photographing device 10 is transmitted to a control device 40 where predetermined image processing is performed. This image data is also stored in the control device 40. A digital camera includes, for example, a lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor unit or a CMOS image sensor unit, a digital signal processor (DSP) that reads the charge amount from the photoelectric conversion element and generates image data, and an IC memory.

[0021] The projection device 20 is positioned at a location where it can project image light for projection mapping onto the entire flower field or at least a portion of the flower field. For example, when a single projection device 20 is used, the flower field can be positioned within the projection range of the projection device 20. When multiple projection devices 20 are used, each projection device 20 projects light onto a portion of the flower field, and the flower field can be positioned within the combined projection range of the multiple projection devices 20. A known commercial projector can be used as the projection device 20. When a projection image is generated by the control device 40, the projection image is transmitted to the projection device 20. Under the control of the control device 40, the projection device 20 converts the projection image received from the control device 40 into image light and irradiates it toward the flower field. Note that the image light irradiated onto the flower field is not monochromatic with only a specific wavelength, but includes light of various wavelengths. The image light may be light for projecting a still image, but is preferably light for projecting a moving image.

[0022] The detection device 30 detects objects within or around the flower field. The objects to be detected are primarily spectators (people). When the detection device 30 detects a spectator, it can change the illumination conditions (e.g., light intensity and color) of the video light irradiated around the spectator. The detection device 30 may be installed within the flower field, around the flower field, or in multiple locations, such as along spectator walkways between the flower fields. The detection device 30 can employ various known sensors, such as a range sensor, infrared sensor, optical sensor, ultrasonic sensor, capacitance sensor, pressure sensor, sound sensor, or temperature sensor. The detection device 30 can also employ multiple sensors with different functions, such as a combination of a range sensor and an infrared sensor. In particular, the detection device 30 preferably employs a range sensor to detect the position of a spectator within the flower field. The range sensor is a two-dimensional scanning optical ranging sensor that emits inspection light (laser) and scans the inspection light to measure the distance and angle to the contact position of the viewer. Information detected by the range sensor is transmitted to the control device 40. If the position coordinates of the range sensor in real space are known, the control device 40 can detect the position coordinates of the object in real space by measuring the distance and angle to the object that has come into contact with the light emitted from the range sensor. Such detection information by the detection device 30 is sent to the control device 40 in real time and is used, for example, to control the projection device 20.

[0023] The control device 40 is a computer for controlling the projection device 20 based on input information from the image capture device 10 and the detection device 30. The block diagram in Fig. 2 mainly shows the functional elements of this control device 40. In the embodiment shown in Fig. 2, the control device 40 is configured as a single computer for convenience, but the control device 40 may also be configured with its functions distributed across multiple computers. Furthermore, a server-type computer on the cloud can also be used as the control device 40.

[0024] As shown in FIG. 2 , the control device 40 includes a processor 41, a memory 42, and a content storage unit 43. Examples of the processor 41 include a known CPU or GPU. The memory is composed of a volatile memory such as a random access memory (RAM). The processor 41 performs predetermined arithmetic processing and image processing according to programs and data stored in the memory 42, and executes various control processes while writing the results of the processing to the memory's workspace. The content storage unit 43 is a storage device for storing image data to be projected onto the flower field by the projection device 20. The content storage unit 43 is composed of a non-volatile memory such as a read-only memory (ROM) or flash memory, and a hard disk drive (HDD). The processor 41 reads image data from the content storage unit 43, temporarily stores it in the memory 42, and performs coordinate transformation and masking on the image data to generate a projection image to be transmitted to the projection device 20. The content storage unit 43 may also store programs used in the processing by the processor 41.

[0025] 2, the processes executed by the processor 41 are shown as functional blocks. That is, in this embodiment, the processor 41 includes a calibration unit 41a, a mask generation unit 41b (image analysis means), and a mapping unit 41c (projection control means). Each of these functional blocks 41a to 41c is a function realized by the processor 41 as a result of the processor 41 executing processes in accordance with a program.

[0026] The calibration unit 41a performs a calibration process to calculate a coordinate transformation map that associates the camera coordinate system of the image capture device 10 with the projector coordinate system of the projection device 20. That is, since there is a physical deviation between the viewpoint of the image capture device 10 and the viewpoint of the projection device 20 as shown in FIG. 1 , in order to control the projection device 20 based on the image captured by the image capture device 10, it is necessary to obtain in advance a map for mutual transformation between the camera coordinates of the image capture device 10 and the projector coordinate system of the projection device 20. The calibration unit 41a performs this calibration process. In this embodiment, the calibration process is performed by projecting a set of black and white striped patterns called Gray code patterns onto a flower field from the projection device 20 and capturing the Gray code patterns projected onto the flower field with the image capture device 10.

[0027] First, the calibration unit 41a controls the projection device 20 to project a Gray code pattern toward the flower field. The calibration unit 41a also adjusts the distortion and focus of the image projected from the projection device 20 while changing the thickness and orientation of the stripes of the Gray code pattern so that the image capture device 10 can correctly capture the Gray code pattern. At this time, the calibration unit 41a may also adjust the gain, shutter speed, white balance, etc. of the image capture device 10. The adjustment of the image capture device 10 and the projection device 20 may be performed automatically or manually.

[0028] Next, the calibration unit 41a starts the calibration process. The calibration unit 41a sends a Gray code pattern to the projection device 20, which projects the Gray code pattern onto the flower field. The image capture device 10 captures the flower field onto which the Gray code pattern is projected and sends the captured image to the calibration unit 41a. The calibration unit 41a determines a C→P map indicating which coordinates (Cx, Cy) in the camera coordinate system of the image capture device 10 correspond to which coordinates (Px, Py) in the projector coordinate system of the projection device 20 by examining the order in which black and white are displayed for each pixel in the captured image. The calibration unit 41a also calculates backward from the C→P map to determine a P→C map indicating which coordinates (Px, Py) in the projector coordinate system correspond to which coordinates (Cx, Cy) in the camera coordinate system. Note that missing portions of the P→C map are filled in by interpolation. The calibration unit 41a stores the C→P map and the P→C map in memory 42. The C→P map and P→C map enable conversion between the camera coordinate system and the projector coordinate system. In this specification, the combination of the C→P map and the P→C map is also referred to as a coordinate transformation map. That is, this coordinate transformation map associates each pixel of the captured image acquired by the image capture device 10 with each pixel of the projected image output by the projection device 20. Note that, when the projection area of ​​one projection device 20 is captured by multiple image capture devices 10, this coordinate transformation map associates coordinates in the camera coordinate systems of the multiple image capture devices 10 with coordinates in the projector coordinate system of one projection device 20. The processor 41 reads and uses this coordinate transformation map from the memory 42 as needed. However, if the positions of the image capture device 10 and the projection device 20 are identical or the positional deviation between them is negligibly small, the calibration unit 41a may be omitted.

[0029] The mask generation unit 41b has a function of analyzing a photographed image of a flower field acquired by the photographing device 10, identifying a predetermined portion in the photographed image, and generating a mask image for distinguishing this predetermined portion from other portions. The mask image used in the present invention is a binary image for distinguishing between an area onto which image light is projected from the projection device 20 (projection area) and other areas (non-projection area). In this embodiment, the mask generation unit 41b generates a flower mask image by extracting flower portions from the photographed image and a grass portion image by extracting grass portions from the photographed image, based on the photographed image. However, the grass mask image is not essential, and the mask generation unit 41b only needs to generate at least a flower mask image.

[0030] As an example, FIG. 3 shows a captured image of a flower field before projection of the image light. While the captured image is actually a photograph, FIG. 3 represents the captured image as a line drawing. As shown in FIG. 3, the captured image shows a flower field, and it can be seen that the captured image includes flower areas, grass areas, and other areas (such as soil). The mask generation unit 41b analyzes the captured image and extracts all of the flower and grass areas. A known method can be used to extract specific areas from the image. For example, the boundary between the specific areas (flower and grass areas) and other areas can be identified based on the color values ​​(RGB) and brightness values ​​of the pixels that make up the image, and the area within that boundary can be extracted as the specific area. Since the color of the flower area varies depending on the type of plant, it is preferable to set a range of color values ​​to be extracted as the flower area in advance. Furthermore, since the color of the grass area is generally green, the range of color values ​​that humans can recognize as green can be set as the grass area. In addition, the mask generation unit 41b may identify flower and grass parts in the captured image using a trained model created by performing well-known machine learning such as deep learning.

[0031] FIG. 4 shows examples of a flower mask image and a grass mask image created from the captured image shown in FIG. 3. As shown in FIG. 4, in the flower mask image, a flower portion is extracted from the captured image (FIG. 3). Similarly, in the grass mask image, a grass portion is extracted from the captured image (FIG. 3). In the flower mask image, the area extracted as the flower portion corresponds to the area onto which the image light is projected from the projection device 20 (projection area), and the other area corresponds to the area onto which the image light is not projected from the projection device 20 (non-projection area). Similarly, in the grass mask image, the area extracted as the grass portion corresponds to the area onto which the image light is projected from the projection device 20 (projection area), and the other area corresponds to the area onto which the image light is not projected from the projection device 20 (non-projection area).

[0032] Furthermore, the flower mask image and the grass mask image can be combined into a single RGB image (although it is also possible not to combine them). Specifically, in a combined mask image having pixel values ​​of three channels (R, G, B), for example, the projection area of ​​the flower mask image is represented by the R channel, and the projection area of ​​the grass mask image is represented by the G channel. In this case, the projection area of ​​the flower mask image is an area with pixel values ​​of (255, 0, 0), and the projection area of ​​the grass mask image is an area with pixel values ​​of (0, 255, 0). It is of course also possible to represent the projection area of ​​the flower mask image by the G channel or the B channel, or the projection area of ​​the grass mask image by the R channel or the B channel.

[0033] Furthermore, in a composite mask image, if two channels are used for the flower mask image and the grass mask image, one channel remains. This remaining channel can be used as a mask image with any projection area. For example, if light from the projection device 20 is difficult to reach a location at the back of the captured image, it is necessary to increase the amount of light irradiated to this location compared to other locations. In such a case, a range corresponding to the back location of the captured image can be set as a projection area in the arbitrary mask image, and the amount of light irradiated to this back location can be individually increased. Alternatively, for example, a location where light from the projection device 20 is difficult to reach can be freely set as a projection area in the arbitrary mask image. The projection area in the arbitrary mask image can also be set manually. As described above, for example, if the flower mask image is set as the R channel and the grass mask image is set as the G channel, this arbitrary mask image will use the remaining B channel. In this case, the projection area of ​​the arbitrary mask image will be an area with pixel values ​​of (0, 0, 255), for example.

[0034] As described above, by generating separate mask images for the flower and grass portions of the captured image, the illumination conditions of the image light projected onto the flowers and grass in the flower field can be individually adjusted. For example, the projection device 20 can irradiate the projection area (flower portion) of the flower mask image with 100% light intensity, while the projection device 20 can irradiate the projection area (grass portion) of the grass mask image with 30% light intensity. In this way, to make the flowers in the flower field stand out, it is preferable to irradiate the flower portion with a greater amount of light than the grass portion. For example, when the maximum light intensity irradiated from the projection device 20 is 100%, the light intensity of the flower portion can be set to 50% or more and 100% or less, while the light intensity of the grass portion can be set to 1% or more and less than 50%. In addition to the light intensity, the color of the light projected from the projection device 20 onto the flower and grass portions can also be individually adjusted. The same applies to the projection area of ​​any mask image.

[0035] Furthermore, since the mask image is generated based on the captured image in the camera coordinate system, the mask image is also expressed in the camera coordinate system. Therefore, the mask generation unit 41b performs calculations to convert this mask image from an image in the camera coordinate system to an image in the projector coordinate system. When performing this calculation, the mask generation unit 41b refers to the coordinate transformation map determined by the calibration unit 41a. Because the coordinate transformation map is stored in the memory 42, the mask generation unit 41b simply reads the coordinate transformation map from the memory 42. Specifically, the coordinate transformation map includes a C→P map that indicates which coordinates (Cx, Cy) in the camera coordinate system correspond to which coordinates (Px, Py) in the projector coordinate system. The mask generation unit 41b simply converts the mask image from an image in the camera coordinate system to an image in the projector coordinate system by referring to the C→P map. In particular, the coordinate values ​​of the transparent regions in the mask image are converted from the camera coordinate system to the projector coordinate system.

[0036] The mapping unit 41c has a function of generating a projection image and controlling the projection device 20 to output this projection image. Specifically, the mapping unit 41c first reads image data in the content storage unit 43 or image data expanded in the memory 42. This image data may simply specify a single color such as white, or may specify multiple colors that differ for each region, or may represent figures, symbols, patterns, etc. The image data may also include information that individually specifies the amount of image light to be projected onto the flowers and grass in the flower field. The image data is not limited to still images, but may also be video.

[0037] Next, the mapping unit 41c synthesizes the mask image (which has undergone coordinate transformation) generated by the mask generation unit 41b with the image data to obtain a synthesized image. In this embodiment, the mask image includes information regarding the light-projected areas of the flower portion and the grass portion, as described above. By synthesizing this mask image with the image data, data of the image data portions corresponding to the light-projected areas of the flower portion and the grass portion is extracted, and data of the remaining portions corresponding to the non-posted areas is removed.

[0038] Next, the mapping unit 41c maps the composite image onto the projection surface of the flower field. Here, the mapping unit 41c converts the composite image expressed in the projector coordinate system into a projection image expressed in the camera coordinate system. In particular, in the projection image, the coordinates of the projection area of ​​the composite image are converted into the corresponding coordinates in the camera coordinate system. When generating this projection image, the mapping unit 41c refers to the coordinate conversion map determined by the calibration unit 41a. Since the coordinate conversion map is stored in the memory 42, the mask generation unit 41b can simply read the coordinate conversion map from the memory 42. Specifically, the coordinate conversion map includes a P→C map that indicates which coordinates (Px, Py) in the projector coordinate system correspond to which coordinates (Cx, Cy) in the camera coordinate system. The mapping unit 41c refers to the P→C map to convert the coordinates of the projected area included in the composite image into coordinates in the camera coordinate system. In particular, the coordinate values ​​of the transparent area in the composite image are converted from the projector coordinate system to the camera coordinate system. The mapping unit 41c sends the projection image in the camera coordinate system obtained in this way to the projection device 20, and causes the projection device 20 to output this projection image. This allows the projection device 20 to project image light onto a targeted position (flowers or grass in a flower field) in the camera coordinate system.

[0039] Furthermore, this projected image is image data used to control the projection device 20. In this embodiment, this data includes information on the illumination conditions for the projection area (flower portion) of the flower mask image and the illumination conditions for the projection area (grass portion) of the grass mask image. The projection device 20 individually adjusts the conditions for illuminating the flower portion and the grass portion with video light based on the information on the illumination conditions. As described above, individual control of the light intensity is possible, for example, by illuminating the flower portion with 100% light intensity from the projection device 20 and illuminating the grass portion with 30% light intensity from the projection device 20. FIG. 5 also schematically illustrates an example of an image captured by the imaging device 10 of a flower field onto which video light is projected from the projection device 20, for example, at night. In this example, the illumination conditions for the light projected onto the flowers and grass in the flower field are different. For example, the flowers in the flower field are illuminated with video light from the projection device 20 at 100%, making the flowers stand out. Furthermore, the projection device 20 projects image light at 30% of the light intensity onto the grass in the flower field, so the grass is also brightly illuminated to the extent that it highlights the flowers. On the other hand, the projection device 20 does not project image light onto areas of the flower field other than the flowers and grass. This makes the flowers and grass in the flower field stand out.

[0040] Furthermore, when the detection device 30 detects the presence of a person (audience) near the flower field, the mapping unit 41c may control the projection device 20 in real time to execute some kind of interaction. For example, the mapping unit 41c may identify the coordinates of the real space where the person exists based on the detection information from the detection device 30, and temporarily increase the amount of video light projected around those coordinates or change the color of the video light projected around those coordinates. In this case, the mapping unit 41c may control the projection device 20 by changing the projection image sent to the projection device 20 in real time. This allows the audience to feel immersed in the video work using the flower field.

[0041] In the above, in order to express the contents of the present invention, the present specification has described the embodiments of the present invention with reference to the drawings. However, the present invention is not limited to the above embodiments, and includes modifications and improvements that are obvious to those skilled in the art based on the matters described in the present specification.

[0042] REFERENCE SIGNS LIST 10: Imaging device 20: Projection device 30: Detection device 40: Control device 41: Processor 41a: Calibration unit 41b: Mask generation unit 41c: Mapping unit 42: Memory 43: Content storage unit 100: Lighting system

Claims

1. A lighting system comprising: a photographing device arranged at a position where a flower field can be photographed; a projection device arranged at a position where it can project image light for projection mapping onto the flower field; and a control device connected to the photographing device and the projection device, wherein the control device includes: image analysis means for analyzing an image of the flower field photographed by the photographing device and identifying flower and grass portions in the photographed image; and projection control means for controlling the projection device to project the image light onto flowers in the flower field based on an analysis result by the image analysis means, wherein the image analysis means generates a flower mask image in which the flower portions in the photographed image of the flower field are projected with the image light and the rest are non-projected areas, and a grass mask image in which the grass portions in the photographed image of the flower field are projected with the image light and the rest are non-projected areas, and the projection control means projects the image light onto the flowers in the flower field based on the flower mask image, and controls the projection device to project the image light onto the grass in the flower field based on the grass mask image.

2. The lighting system according to claim 1, wherein said projection control means controls said projection device so that lighting conditions for flowers in said flower field are different from lighting conditions for grass.

3. The lighting system of claim 1, wherein the flower mask image represents the projection area of ​​the image light in one of RGB values, and the grass mask image represents the projection area of ​​the image light in another of RGB values, and the flower mask image and the grass mask image are combined into a single image expressed in RGB values.

4. A control device composed of one or more computers connected to a photographing device arranged at a position where a flower field can be photographed, and a projection device arranged at a position where it can project image light for projection mapping onto the flower field, the control device including: image analysis means for analyzing an image of the flower field photographed by the photographing device and identifying flower portions in the photographed image; and projection control means for controlling the projection device to project the image light onto the flowers in the flower field based on the analysis results by the image analysis means, the image analysis means generating a flower mask image in which the flower portions in the photographed image of the flower field are projected areas for the image light and the rest are non-projected areas, and a grass mask image in which the grass portions in the photographed image of the flower field are projected areas for the image light and the rest are non-projected areas, the projection control means projects the image light onto the flowers in the flower field based on the flower mask image, and controls the projection device to project the image light onto the grass in the flower field based on the grass mask image.

5. A lighting method comprising: a step of a photographing device acquiring a photographed image of a flower field; a step of a control device analyzing the photographed image of the flower field to identify flower parts in the photographed image and generating a control signal for a projection device to project video light for projection mapping onto the flowers in the flower field based on the analysis result of the photographed image; and a step of the projection device projecting onto the flowers in the flower field based on the control signal, wherein the control device generates a flower mask image in which the flower parts in the photographed image of the flower field are projected areas for the video light and the rest are non-projected areas, and a grass mask image in which the grass parts in the photographed image of the flower field are projected areas for the video light and the rest are non-projected areas, and projects the video light onto the flowers in the flower field based on the flower mask image, and controls the projection device to project the video light onto the grass in the flower field based on the grass mask image.

Images