Apparatus, method, and program

Abstract

The pattern of the object is used to give the perception that the object is moving. In order to create the perception that an object is moving, an image is superimposed on the object, the image including a luminance motion component corresponding to the motion imparted to the object.

Description

[Technical Field]

[0001] The present invention relates to a technique for creating an optical illusion. [Background technology]

[0002] For example, Non-Patent Document 1 discloses a technology that gives the object an illusionary effect of movement by measuring the three-dimensional shape of the object (a single achromatic color) with a camera and projecting an image that expresses movement in accordance with the three-dimensional shape of the object. In Non-Patent Document 1, the object is a flat, single achromatic plane and a single achromatic model car placed on the plane, and this object is used as a canvas. A color scheme that simulates the color and reflective properties of the car body, the road around the car, the daylight conditions in the space in which the car is running, etc. is projected onto the object as an image, creating the illusion that the model car object is moving on a road, giving the object an illusion of movement. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Raskar, R.; Ziegler, R.; Willwacher, T., "Cartoon Dioramas in Motion", International Symposium on Non-Photorealistic Animation and Rendering (NPAR), June 2002. Summary of the Invention [Problem to be solved by the invention]

[0004] Non-Patent Document 1 treats an object as a simple canvas and gives the object the effect of movement without utilizing the object's inherent pattern. The objective of this invention is to make the object appear to be moving by utilizing the object's inherent pattern. [Means for solving the problem]

[0005] To create the perception that an object is moving, an image is superimposed on the object, the image containing a luminance motion component corresponding to the motion of the object. [Effects of the Invention]

[0006] This makes it possible to make the object appear to be moving by utilizing the pattern that the object has. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the configuration of a video display device according to a first embodiment. [Figure 2] 4 is a flowchart showing the operation of the video display device of the first embodiment. [Figure 3] FIG. 1 is a block diagram showing the configuration of a video projection device according to a first embodiment. [Figure 4] 5 is a flowchart showing the operation of the video projection device according to the first embodiment. [Figure 5] 4A to 4C are diagrams for explaining the relationship between the projection angle of view and the target object when the projection unit of the video projection device according to the example of the first embodiment is realized by a projector. [Figure 6] FIG. 10 is a block diagram showing the configuration of a video display device according to a second embodiment. [Figure 7] 10 is a flowchart showing the operation of the video display device according to the second embodiment. [Figure 8] FIG. 10 is a block diagram showing the configuration of a video display device according to a third embodiment. [Figure 9] 10 is a flowchart showing the operation of the image generating unit of the image display device of the third embodiment. [Figure 10] FIG. 10 is a diagram illustrating a stationary component and a moving component. [Figure 11] FIG. 10 is a block diagram showing the configuration of a video display device according to a fourth embodiment. [Figure 12] 10 is a flowchart showing the operation of the video display device according to the fourth embodiment. [Figure 13]13A and 13B are conceptual diagrams illustrating an outline of the process. [Figure 14] 14A and 14B are conceptual diagrams illustrating an outline of the process. [Figure 15] FIG. 15 is a block diagram illustrating a functional configuration of the embodiment. [Figure 16] FIG. 16 is a flow diagram illustrating the process of the embodiment. [Figure 17] FIG. 17 is a flow diagram illustrating the process of the embodiment. [Figure 18] 18A and 18B are diagrams illustrating a method for adding a luminance motion component to an image. [Figure 19] FIG. 19 is a block diagram illustrating a functional configuration of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] An embodiment of the present invention will be described. Unlike conventional techniques, the embodiments described below utilize the pattern of an object while creating an illusionary transformation of the appearance of that pattern. Rather than treating the object as a mere canvas, each embodiment actively utilizes the object's inherent pattern to create the illusion of movement. Therefore, it is preferable that the object be a chromatic or grayscale object, rather than a single achromatic color. Unlike conventional techniques, each embodiment does not require consideration of the object's three-dimensionality. For example, it is possible to create an illusionary transformation of the appearance of a photographed image of a still object. While conventional techniques have focused on the movement and transformation of an object, each embodiment differs from conventional techniques in that it uses both the object's inherent pattern and the image information of a video displayed overlaid on the object to give the object a texture or impression (such as a liquid feel or facial expression) that was not originally present in the object. Furthermore, unlike techniques that simply transform a photographed image on a screen, each embodiment can convey a change in the object's texture and impression by overlaying the video. This provides the viewer with a perceptual experience different from conventional techniques. Each embodiment will be described in detail below. Components having the same functions are assigned the same numbers, and duplicated explanations will be omitted.

[0009] [First embodiment] Hereinafter, with reference to FIGS. 1 and 2, a video display device according to a first embodiment, which is a basic configuration example of the present invention, will be described. FIG. 1 is a block diagram showing the configuration of the video display device 1 according to this embodiment. FIG. 2 is a flowchart showing the operation of the video display device 1 according to this embodiment. As shown in FIG. 1, the video display device 1 includes a display unit 11. The display unit 11 displays a transparent image by superimposing it on an object. Here, the "object" may be an object having a three-dimensional shape (e.g., a vase, a ball, a model) or a predetermined plane (e.g., paper, a board, a wall, a screen). When the object is a plane, it is desirable that the plane include a pattern. Examples of patterns included on the plane include, for example, a photograph or image printed on paper, and a photograph or image projected onto a predetermined plane. When the object is a screen such as a display, an example of a pattern is an image displayed on the screen such as a display.

[0010] A typical method for "superimposing" a transparent image on an object is to project the image onto the object using a projector or the like. In this case, the image projected by the projector or the like naturally has transparency. Alternatively, for example, by placing a transparent LCD screen in front of the object and projecting a transparent image onto the LCD screen, the transparent image can be "superimposed" on the object as seen by an observer facing the object through the LCD screen. Here, the "image" refers to, for example, an image in which images of a distortion distribution having low spatial frequency components are switched over time. In this embodiment, the image is prepared in advance and input from outside the image display device 1. The display unit 11 displays the image so that edges contained in the image overlap with the contour of the object or edges contained in the object. For example, if the object is a vase, the display unit 11 displays the image so that edges contained in the image overlap with the contour of the vase or edges of the vase, such as a pattern painted on the vase. For example, if the object is a plane and an image is projected onto the plane as a pattern, the display unit 11 displays the image so that the edges included in the image overlap with the edges included in the object, such as the pattern of the image projected onto the object.

[0011] Therefore, the display unit 11 displays a transparent image in which images of a distortion distribution having low spatial frequency components are switched over time, superimposed on the object so that the edges contained in the image overlap the contour of the object or the edges contained in the object (S11).

[0012] [Example of the first embodiment] An image projection device 1a according to an embodiment of the present invention will be described below with reference to FIGS. 3 and 4. FIG. 3 is a block diagram illustrating the configuration of the image projection device 1a according to the embodiment of the present invention. FIG. 4 is a flowchart illustrating the operation of the image projection device 1a according to the embodiment of the present invention. As shown in FIG. 3, the image projection device 1a includes a projection unit 11a. In this embodiment, a stationary object is handled. As described above, the projection unit 11a projects an image in which a distortion distribution image having low spatial frequency components is switched over time onto the stationary object so that edges contained in the image overlap with the contour or edges of the object (S11a). The projection unit 11a can be realized, for example, by a projector. For example, when the projection unit 11a is a projector, the horizontal viewing angle θ and vertical viewing angle φ of the object 9 as viewed from the center of the projector's projection lens must match the horizontal and vertical viewing angles of the projected image, as shown in FIG. 5.

[0013] It should be noted that selecting the projection unit 11a as an example of the display unit 11 is also possible in the second, third, and fourth embodiments described below, and each block diagram representing each device in the embodiments described below indicates that the projection unit 11a is included as an example of the display unit 11.

[0014] With the technology of Non-Patent Document 1, it is not easy to create a special illusion that makes a medium (e.g., air, water, or water vapor) surrounding an object appear to be irregularly fluctuating. Applying the technology of Non-Patent Document 1 to create such a special illusion requires a huge amount of calculation to simulate the fluctuation of the medium, the refraction of light rays, and the like. The image projection device 1a of this embodiment can create a special illusion on an object with a small amount of calculation. Furthermore, the image projection device 1a of this embodiment can also create an illusion on flat objects, three-dimensional objects, chromatic objects, and achromatic objects with varying shades of gray.

[0015] [Second embodiment] An image display device 2 according to a second embodiment, which includes an image generation function, will be described below with reference to Figs. 6 and 7. Fig. 6 is a block diagram showing the configuration of the image display device 2 according to this embodiment. Fig. 7 is a flowchart showing the operation of the image display device 2 according to this embodiment. As shown in Fig. 6, the image display device 2 includes the same display unit 11, shooting unit 21, and image generation unit 22 as in the first embodiment.

[0016] The photographing unit 21 photographs an object and acquires an original image (S21). The image generating unit 22 generates a plurality of different narrowband images from the original image, and generates an image in which the generated plurality of different narrowband images are arranged so as to be smoothly connected in time (S22). The display unit 11 displays the generated image superimposed on the object, as described above (S11).

[0017] A narrowband image is an image in which the spatial frequency band of the entire image is narrower than that of the original image while retaining edge information contained in the original image, and is also a transparent image.

[0018] As an example of realizing the image generation unit 22, for example, as described in Reference 1, an image may be convolved with an azimuth filter with a phase difference of 180 degrees, and the convolved images may be smoothly connected in time to generate an image that gives the illusion of movement. (Reference 1: Freeman, WT, Adelson, EH, & Heeger, DJ (1991). Proceedings of the 18th annual conference on computer graphics and interactive techniques, 27-30.)

[0019] [Third embodiment] The following describes a third embodiment of the image display device 3, which incorporates an image generation function, with reference to FIGS. 8 and 9. FIG. 8 is a block diagram illustrating the configuration of the image display device 3 of this embodiment. FIG. 9 is a flowchart illustrating the operation of the image generation unit 32 of the image display device 3 of this embodiment. As shown in FIG. 8, the image display device 3 includes the same display unit 11 and image capture unit 21 as in the second embodiment, and an image generation unit 32 that differs from the second embodiment. The image generation unit 32 includes a deformation image generation unit 321, a Fourier transform unit 322, and a separation unit 323. The deformation image generation unit 321 applies dynamic deformation to an original image of the object, thereby creating an image in which the still object image is deformed (S321). The deformation is expressed by pixel movement. This pixel movement is based on a pre-calculated algorithm. For example, if a liquid impression is to be added to an object, the algorithms in Reference 2 and Japanese Patent Application No. 2013-132609, which was unpublished at the time of filing this application, can be used as reference. (Reference 2: Kawabe, T., Maruya, K., & Nishida, S. (2013). Seeing transparent liquids from dynamic image distortion. Journal of Vision, 13(9): 208.)

[0020] An implementation example of the deformed image generation unit 321 based on the algorithm of Japanese Patent Application No. 2013-132609 is disclosed below. First, the deformed image generation unit 321 generates multiple modulated images by modulating the original image based on the distortion distribution. To strongly perceive the impression of a flowing transparent liquid, it is desirable to modulate the original image with a distortion such that the spatial frequency of the distortion distribution (distortion map) is 3 cpd (cycles per degree) or less. In other words, adding a coarse distortion to the original image that reduces the difference in distortion amount between adjacent pixels can more strongly perceive the impression of a transparent liquid. The original image may be modulated using a distortion distribution with the same low spatial frequency components (e.g., 3 cpd or less) or with different low spatial frequency components (e.g., one with 2 cpd or less and the other with 3 cpd or less). Furthermore, the modulated image is provided with two or more dimensional distortion directions. The type of distortion may be any two-dimensional geometric distortion, such as rotational distortion, translational distortion, or random distortion.

[0021] Next, the transformed image generation unit 321 generates an image based on a plurality of modulated images generated from the original image. For example, the transformed image generation unit 321 may generate, as an image, a sequence of modulated images in which the plurality of modulated images generated from the original image are sequenced so that they are switched over and presented over time. The modulated image sequence is an image in which the presentation time (frame rate) of each image is set within a range in which the viewer can view the modulated image sequence as a moving image rather than a series of still images, i.e., "an image constructed by arranging modulated images in a time sequence." Furthermore, for example, the transformed image generation unit 321 may control the presentation of each of the plurality of modulated images generated from the original image by switching them over. The presentation interval of each image may be controlled within a range in which the viewer can view the image as a moving image (image) rather than a series of still images. For example, the presentation time of each image may be set to not exceed 0.05 (sec), or the frame rate may be set to 20 Hz or higher.

[0022] Next, Fourier transform unit 322 performs a three-dimensional (space-time) Fourier transform on the image created by deformed image generation unit 321 (S322). Separation unit 323 separates DC components (still components) and motion components by time filtering, and outputs only the motion components as an image (S323).

[0023] In other words, the transformed image generation unit 321 generates an image by arranging a group of multiple images obtained by moving pixels of the original image based on a predetermined algorithm so that they are smoothly connected in time (S321). The Fourier transform unit 322 performs a three-dimensional (space-time) Fourier transform on the generated image (S322). The separation unit 323 separates still components and moving components based on the Fourier transform result, and outputs only the moving components as an image (S323).

[0024] FIG. 10 shows an example of the still component (DC component) and the motion component extracted by steps S321 to S323 described above. The motion component is mainly composed of high spatial frequency components. As shown in FIG. 5, by displaying the motion component as an image at the same visual angle as the object and superimposing it on the object, the phase relationship between the two matches that of the original video. Under these conditions, the spatial structure of the object is illusory captured by the motion component. As a result, the object appears to move.

[0025] [Fourth embodiment] Hereinafter, with reference to FIGS. 11 and 12, a fourth embodiment of the image display device 4 will be described, which includes a function for generating an original image to be displayed on an object. FIG. 11 is a block diagram showing the configuration of the image display device 4 of this embodiment. FIG. 12 is a flowchart showing the operation of the image display device 4 of this embodiment. As shown in FIG. 11, the image display device 4 of this embodiment includes the same display unit 11, image capture unit 21, and image generation unit 32 as in the third embodiment, as well as an original image display unit 41 not included in the third embodiment. The image capture unit 21 and image generation unit 32 execute steps S21 and S32, respectively, as in the third embodiment. The original image display unit 41 displays (projects) the original image acquired by capturing the object in step S21 on another still medium (e.g., a display, a screen, etc.) (S41). The display unit 11 then displays (projects) an image by superimposing it on the original image displayed on the still medium. This achieves the same effect as described above. In this case, the image may be generated by convolving an azimuth filter on the image as in the second embodiment, or by applying a deformation to the image as in the third embodiment. When displaying on another static medium, the original image can be printed on the object, projected by another projector, or displayed on electronic paper.

[0026] An external camera is used to capture the subject. It is desirable for the camera to have as high a spatial resolution as possible. Excluding the calculation time, it is possible to instantly add an illusion of movement to a still subject in front of the eyes. A projector can be used as the projection unit 11a of the video projection device 4a, which is an example of this embodiment. This projector can be a commercially available one, but if it is to be used in a bright room, it needs to be a projector with high brightness.

[0027] The ideal viewing distance varies depending on the size of the object. For example, if you add movement to a 16cm square image using this method, a viewing distance of about 1.5m is required. The larger the object, the longer the viewing distance needs to be.

[0028] When projecting an image onto a three-dimensional object without taking into account the depth of the projection surface, if the depth difference between the projection surface and the object is large, a misalignment may occur between the image and the object's structure. It is more suitable to use an object with a relatively small depth difference that can be projected without causing a misalignment. Furthermore, when projecting an image onto a three-dimensional object with a large depth, it is possible to create an illusion of deformation of a surface tilted in the depth direction by grasping the three-dimensional shape of the object and transforming the projected image to match that shape.

[0029] The image display device of this embodiment can, for example, illusorily change the facial expression of a still facial image or change the direction of gaze. Furthermore, by changing the movement pattern, it is possible to create the illusion that an object appears to be fluttering or to be present under flowing liquid. When projecting an image onto a two-dimensional medium such as a printed matter, the dynamic range of the brightness of the object within the medium is generally narrow. However, in this embodiment, an image with a high dynamic range compared to the object is projected onto the object, thereby enhancing the appearance of the object's image quality while adding an illusory movement to the object. Furthermore, the image display device of this embodiment can instantly add an illusory movement effect to an image captured by a camera, excluding processing time.

[0030] The image display device of this embodiment can be applied as an exhibition technique in art galleries and museums, or as a component technology for attractions used in entertainment facilities. For example, it can visually move children's characters printed on static media such as paper.

[0031] In conventional technology, a color difference motion component corresponding to the temporal frequency of a moving image is required to present the moving image. Below, we will explain a method that creates the illusion of an image moving at a temporal frequency equal to or greater than a predetermined magnitude without using a color difference motion component with a temporal frequency equal to or greater than that magnitude.

[0032] Humans detect local motion components from moving images through functions such as spatiotemporal filtering. (Reference 3: Watson, A. B., & Ahumada, A. J., “Model of human visual-motion sensing,” Journal of the Optical Society of America, (1985), A, 2, 322-342.) A similar function can be realized through image information processing, which separates and extracts moving and stationary components from moving images using time-frequency filters.

[0033] It is known that humans perceive static image components defined by color and luminance contrast as being attracted to luminance motion components (Reference 4: Ramachandran, V.S., “Interaction between color and motion in human vision,” Nature (1987), 328, 645-647.; Reference 5: Anstis, S., “Kinetic edges become displaced, segregated, and invisible,” In D.M.-K. Lam (Ed.), Neural mechanisms of visual perception, Proceedings of the Second Retina Research Foundation Conference, Texas (1989): Portfolio Press, 247-260.). Based on this visual characteristic, it is thought that the luminance motion components in moving images can modulate the way images appear.

[0034] The human visual system is insensitive to motion signals defined by color signals, but is sensitive to motion signals defined by luminance signals (Reference 6: Ramachandran, V.S., & Gregory, R.L., “Does color provide an input to human motion perception?” Nature (1978), 275, 55-56.). In addition, the human spatial resolution for objects moving at a certain speed is lower than that for stationary objects (Reference 7: Kelly, D.H. (1979). Motion and vision. II. Stabilized spatio-temporal threshold surface. Journal of the Optical Society of America, 69, 1340-1349.). In other words, removing the color component from the motion component of a moving image does not significantly degrade the quality of the image as perceived by humans.

[0035] In each embodiment, in consideration of the above-described human perception characteristics, a "luminance motion component" with an absolute value of the temporal frequency being equal to or higher than a second value is added to an "image" that includes a spatial frequency component with an absolute value greater than zero and in which the absolute value of the temporal frequency is equal to or lower than a first value. The "luminance motion component" is a component corresponding to the "image", and the "second value" is greater than the "first value". "Including a spatial frequency component with an absolute value greater than zero" means including a non-zero spatial frequency component. For example, if the "first value" is F1 and the "second value" is F2, the relationship 0 ≦ F1 < F2 is satisfied. F1 may be 0, or F1 may be greater than 0. An example of F1 is the absolute value of the temporal frequency that is perceived as if the "image" is stationary (e.g., in the vicinity of zero). However, F1 may be the temporal frequency at which the "image" is perceived to be moving. When F1 = 0, the "image" with an absolute value of the temporal frequency being equal to or lower than the first value is a stationary image. When F1 > 0, the "image" with an absolute value of the temporal frequency being equal to or lower than the first value has a motion with a frequency component having a magnitude equal to or lower than the first value, and moves slower than the "luminance motion component" with an absolute value of the temporal frequency being equal to or higher than the second value. The "image" may have only a single temporal frequency component (e.g., 0 Hz), or may have a plurality of temporal frequency components (i.e., may be a composite of images with a plurality of temporal frequencies). Hereinafter, the "component with an absolute value of the temporal frequency being equal to or lower than the first value" is referred to as a "low temporal frequency component", and the "component with an absolute value of the temporal frequency being equal to or higher than the second value" is referred to as a "high temporal frequency component". It is desirable that the "image" is not uniform and includes a spatial frequency component with an absolute value greater than zero (non-zero spatial frequency component). Both the color difference component and the luminance component of the "image" may include a spatial frequency component with an absolute value greater than zero, or only one of the color difference component or the luminance component may include a spatial frequency component with an absolute value greater than zero. A person who views a video in which the "luminance motion component" is added to such an "image" has an illusion (dynamic illusion) that the "image" is moving at a temporal frequency higher than the temporal frequency of the magnitude of the "first value". For example, when the "luminance motion component" is added to a stationary "image", a person has an illusion that this "image" is moving. Note that "frequency α is higher than frequency β" means that the absolute value |α| of frequency α is greater than the absolute value |β| of frequency β.

[0036] The "image" may be an achromatic (grayscale) image containing only luminance components, or it may be an image containing chromatic (color) components. In particular, adding a luminance motion component to the latter image can create the illusion that the chromatic image is moving at a frequency higher than the low-temporal frequency components, even though it does not contain color difference components higher than the low-temporal frequency components. The "image" may be an image (image information) to be processed, or an image that appears on the surface of an object. Examples of "images" that appear on the surface of an object include images or photographs that are "printed," "drawn," "displayed," or "projected" onto the surface of the object, designs or patterns based on the color of the material that makes up the surface of the object, and patterns based on the shape of the surface of the object (e.g., patterns, borders, shading). The "surface of the object" may be flat, curved, or uneven. The "object" may be an object with a three-dimensional shape (e.g., a vase, a bowl, a model, a building), or an object that can be considered flat for practical purposes (e.g., paper, a board, a wall, a screen, a display, a transmissive display).

[0037] To create a powerful dynamic illusion, it is desirable for the "image" and the "luminance motion component" to correspond to the same "video." For example, the "image" corresponds to a component whose temporal frequency in multiple frames of the "video" is zero (temporal frequency = 0 Hz) or near zero, and the "luminance motion component" corresponds to a luminance component whose absolute temporal frequency in the multiple frames is positive (|temporal frequency| > 0 Hz). A particularly significant effect can be expected when the "video" contains "periodic or repetitive motion components." "Periodic motion components" do not only refer to components that exhibit strictly periodic motion, but also to components that exhibit highly periodic motion. Similarly, "repetitive motion components" do not only refer to components that exhibit strictly repetitive motion, but also to components that exhibit highly repetitive motion. When a "video" contains "periodic or repetitive motion components," the "image" may correspond to a still image of any frame included in the "video." "A corresponds to B" may mean that A is B, that B is derived from (based on) A, or that A is derived from B. "B is derived from A" may mean that A is obtained from B, or that A is obtained from a copy of B, or that A is obtained from an approximation of B. For example, an "image" and "brightness motion component" may be extracted from a "video" or its copy, or a "video" may be generated from a still "image" captured by a camera or scanner, and the "brightness motion component" may be extracted from that "video."

[0038] The function of motion perception is steady when the luminance contrast is above a certain value (Reference 8: Pantle, A., & Sekuler, R. (1969). Contrast response of human visual mechanisms sensitive to orientation and direction of motion. Vision Research, 9, 397-406.). Furthermore, the spatial resolution of motion perception is lower than the spatial resolution of perception of still images (Reference 7). In other words, even if the spatial resolution or contrast of the luminance motion components extracted from a video is manipulated to reduce the quality of the motion information itself, the perceived quality of the moving image is maintained. Therefore, even if the high spatial frequency components or components with reduced contrast of the luminance motion components contained in the "video" are used as the "luminance motion components," a sufficient dynamic illusion can be created. This allows the amount of information to be reduced without significantly reducing the level of the dynamic illusion.

[0039] "Adding a luminance motion component to an image" means, for example, "synthesizing a luminance motion component to an image," "superimposing a luminance motion component on an image," "integrating a luminance motion component into an image," "adding a luminance motion component to an image," "reflecting a luminance motion component in an image," "incorporating a luminance motion component into an image," or "performing an operation including at least addition, multiplication, or exponentiation on pixel values ​​of an image and pixel values ​​of the luminance motion component." Specific methods for "adding a luminance motion component to an image" will be described later.

[0040] [Fifth embodiment] In this embodiment, "low temporal frequency components" and "high temporal frequency components" are extracted from the video M1. The "low temporal frequency components" are referred to as the "image," and the luminance components extracted from the "high temporal frequency components" are referred to as the "luminance motion components." The chrominance components of the "high temporal frequency components" are not used. A calculation is performed to integrate (add) the "luminance motion components" into the "image," and the resulting video M2 is displayed (Figure 13A). As a result, the "image" and the "luminance motion components" are integrated in the visual system of a person viewing the video M2, creating the illusion that the "image" is moving at a higher temporal frequency than the "low temporal frequency." In this way, the perceptual quality of the image is maintained even when the chrominance components of the "high temporal frequency components" are removed from the video M1.

[0041] <Configuration> As illustrated in FIG. 15 , the moving image component extraction device 51 of this embodiment includes a low-temporal frequency component extraction unit 511 (first processing unit), a luminance motion component extraction unit 512 (second processing unit), and an output unit 513. The dynamic illusion presentation device 52 (illusion presentation device) of this embodiment includes an input unit 521, a calculation unit 522, and a display unit 523. The moving image component extraction device 51 and the low-temporal frequency component extraction unit 511 are devices configured by loading a predetermined program into a general-purpose or dedicated computer equipped with, for example, a processor (hardware processor) such as a central processing unit (CPU), memories such as random-access memory (RAM) and read-only memory (ROM), and a display device such as a display. This computer may include one processor and memory, or multiple processors and memories. This program may be installed on the computer or may be pre-recorded in a ROM or the like. Furthermore, some or all of the processing units may be configured using electronic circuits that realize processing functions without using a program, rather than electronic circuits that realize functional configuration by loading a program like a CPU. Furthermore, the electronic circuitry that makes up one device may include multiple CPUs.

[0042] <Processing> The processing of this embodiment will be described with reference to Fig. 16. The moving image M1 of this embodiment is a color moving image including chromatic colors, and is composed of pixel values ​​that represent intensity changes of the R channel, G channel, and B channel (Equation (1)).

number

[0043] The pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) that make up the video M1 (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max) is input to a low-time frequency component extraction unit 511 and a luminance motion component extraction unit 512 of a motion component extraction device 51 (FIG. 15).

[0044] The low-temporal frequency component extraction unit 511 extracts pixel values ​​R(x, y, t), G(x, y, t), and B(x, y, t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) to the stationary component R static (x,y),G static (x,y),B static In this embodiment, the time average values ​​of pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) are calculated as R static (x,y),G static (x,y),B static Let (x, y) (Equation (2)).

number

[0045] The low-time frequency component extraction unit 511 extracts R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) is a two-dimensional matrix {R static (x,y)},{G static (x,y)},{B static (x,y)} static (Equation (3)) is output.

number

[0046] The luminance motion component extraction unit 512 extracts the pixel values ​​R(x, y, t), G(x, y, t), and B(x, y, t) from R original (x,y,t),G original (x,y,t),B original (x, y, t) (step S5121), and these are weighted and added according to the degree to which each color contributes to the luminance, to obtain the luminance component Y original (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained (step S5122) (equation (4)). Y original (x,y,t)=α R R original (x,y,t)+α G G original (x,y,t)+α B B original (x,y,t) (4) However, α R ,α G ,α B is a weighting coefficient (constant) (e.g., α R =0.299,α G =0.587,α B =0.114).

[0047] Furthermore, the luminance motion component extraction unit 512 extracts the luminance component Y original From (x,y,t), the luminance stationary component Y static By subtracting (x,y), the luminance motion component Y motion (x, y, t) is obtained and output (Equation (5)). static (x,y) is the luminance component Y original (x,y,t) (where x min ≦x≦xmax ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained by averaging it over time. Y motion (x,y,t)=Y original (x,y,t)-Y static (x,y) (5) Luminance motion component Y motion (x, y, t) are multiple frames t(t min ≦t≦t max ) is a luminance component whose absolute value of the temporal frequency is positive, and is an example of a "luminance motion component whose absolute value of the temporal frequency is equal to or greater than a second value." This luminance motion component is a component corresponding to the image, and the second value is greater than the first value (step S5123).

[0048] Image M static and the luminance motion component Y motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is input to the output unit 513, and is sent from there to the dynamic illusion exhibiting device 52. static and the luminance motion component Y motion The (x, y, t) is input to the input unit 521 of the dynamic illusion exhibiting device 52 and sent to the calculation unit 522. The calculation unit 522 calculates the image M static to the luminance motion component Y motion For example, the calculation unit 522 calculates and outputs a moving image M2 that takes into account (x, y, t). static Each stationary component R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) to the luminance motion component Y motion (x, y, t) are added to obtain the video M2 (FIG. 18A, Equation (6)).

number

number

[0049] The moving image M2 obtained in this manner is input to the display unit 523 and displayed there (step S523). Although the moving image M2 does not contain any motion components of the color signal, it creates the illusion of movement. In other words, even if the color motion components are removed from the moving image M1, the user can be provided with a visual experience that is comparable to that of the original moving image.

[0050] [Modification 1 of the Fifth Embodiment] The process of separating and extracting each component from the video M1 can also be achieved by converting the video M1 into the time-frequency domain using a Fourier transform or the like and then performing time-frequency filtering. The following will focus on the differences from the matters explained so far. The same reference numbers will be used for matters already explained, and explanations may be omitted.

[0051] <Configuration> 15, the moving image component extraction device 51' of this embodiment has a frequency domain conversion unit 514', a low temporal frequency component extraction unit 511' (first processing unit), a high temporal frequency component extraction unit 515', a luminance motion component extraction unit 512' (second processing unit), time domain conversion units 516' and 517', and an output unit 513. The dynamic illusion presentation device 52 is the same as in the fifth embodiment. The moving image component extraction device 51' is configured, for example, by loading a predetermined program into a computer such as the one described above.

[0052] <Processing> The processing of this modification will be described with reference to FIG. 17. The pixel values ​​R(x, y, t), G(x, y, t), and B(x, y, t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) are input to a frequency domain transform unit 514'. The frequency domain transform unit 514' transforms the pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) into the time-frequency domain to obtain time-frequency domain values ​​FR(x,y,f), FG(x,y,f), and FB(x,y,f). An example using Fourier transform is shown below (Equation (8)).

number

[0053] The low-temporal frequency component extraction unit 511' multiplies FR(x,y,f), FG(x,y,f), and FB(x,y,f) by a low-pass filter LF(f) to extract GR static (x,y,f),GG static (x,y,f),GB static (x, y, f) is obtained and output (Equation (9)). GR static (x,y,f)=FR(x,y,f)LF(f) GG static (x,y,f)=FG(x,y,f)LF(f) (9) GB static (x,y,f)=FB(x,y,f)LF(f) However, the low-pass filter LF(f) is expressed by equation (10).

number

[0054] GR static (x,y,f),GG static (x,y,f),GB static (x, y, f) is input to the time domain transform unit 516'. The time domain transform unit 516' static (x,y,f),GG static (x,y,f),GB static Transform (x,y,f) into the time domain and extract the low time frequency components R static (x,y,t),G static (x,y,t),B static (x, y, t) is obtained. Below we show an example using the inverse Fourier transform (equation (11)).

number

number

[0055] The high-temporal frequency component extractor 515' multiplies FR(x,y,f), FG(x,y,f), and FB(x,y,f) by a high-pass filter HF(f) to extract the GR motion (x,y,f),GG motion (x,y,f),GB motion (x,y,f) (where x min ≦x≦x max ,y min ≦y≦y max ,0≦|f|≦f max ) is obtained and output (Equation (13)). GR motion (x,y,f)=FR(x,y,f)HF(f) GG motion (x,y,f)=FG(x,y,f)HF(f) (13) GB motion (x,y,f)=FB(x,y,f)HF(f) However, the high-pass filter HF(f) is expressed by equation (14).

number

[0056] GR motion (x,y,f),GG motion (x,y,f),GB motion (x, y, f) is input to the time domain transform unit 517'. The time domain transform unit 517' motion (x,y,f),GG motion (x,y,f),GB motion Transform (x,y,f) into the time domain and extract the high temporal frequency components R motion (x,y,t),G motion (x,y,t),B motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained and output. Below we show an example using the inverse Fourier transform (equation (16)).

number

[0057] The luminance motion component extraction unit 512' extracts R motion (x,y,t),G motion (x,y,t),B motion (x, y, t) are weighted and added to obtain the luminance component Y of the video M1. original (x,y,t) (where x min ≦x≦x max ,ymin ≦y≦y max ,t min ≦t≦t max ) is obtained and output (step S512') (equation (17)). Y motion (x,y,t)=α R R motion (x,y,t)+α G G motion (x,y,t)+α B B motion (x,y,t) (17) However, α R ,α G ,α B is a weighting coefficient (constant). motion (x, y, t) is an example of a "luminance motion component whose absolute value of the temporal frequency is equal to or greater than a second value."

[0058] The subsequent processing is the same as in the fifth embodiment. static This gives the user the illusion that video M2 is moving at a higher temporal frequency than

[0059] [Modification 2 of the Fifth Embodiment] In the fifth embodiment and its first modification, an image M consisting of low-time frequency components of a moving image M1 is static The video M2 was generated by adding the luminance motion component extracted from the high-time frequency component of the video M1 (Fig. 13A). However, the image M2, which is determined by the "color difference component" extracted from the low-time frequency component of the video M1, static Alternatively, a moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to the moving image M2 (FIG. 13B). Alternatively, an image M2 may be generated by adding a luminance motion component extracted from the low-time frequency component of the moving image M1 to the moving image M2 (FIG. 13B). static A moving image M2 may be generated by adding luminance motion components extracted from the high-time frequency components of the moving image M1 to the moving image M2 (FIG. 14A). Such a moving image M2 can also produce a dynamic illusion. The moving image M1 may also be a grayscale moving image.

[0060] [Sixth embodiment] If video M1 contains periodic or repetitive motion components (e.g., small movements), a still image of any frame (e.g., frame t=n) included in video M1 may be used as the "image" (FIG. 13A). That is, video M2 may be generated by adding a luminance motion component to a still image of any frame extracted from video M1. Such video M2 can also create the illusion that the still image is moving.

[0061] <Configuration> 15, the moving image component extraction device 61 of this embodiment has an image extraction unit 611 (first processing unit), a luminance moving component extraction unit 612 (second processing unit), and an output unit 513. The dynamic illusion presentation device 52 is the same as in the fifth embodiment. The moving image component extraction device 61 is configured, for example, by loading a predetermined program into the computer described above.

[0062] <Processing> The processing of this embodiment will be described with reference to Fig. 16. The difference from the fifth embodiment is that step S611 is executed instead of step S511, and step S6123 is executed instead of step S5123. Only steps S611 and S6123, which are the differences, will be described below.

[0063] Step S611 The moving image M1 (equation (1)) is input to an image extraction unit 611 and a luminance motion component extraction unit 612. The image extraction unit 611 extracts a still image of the t=nth frame from the moving image M1 and outputs it as an image M static That is, R static (x,y)=R(x,y,n),G static (x,y)=G(x,y,n),B static (x,y)=B(x,y,n)(x min ≦x≦x max ,y min ≦y≦y max ) is a two-dimensional matrix {R static (x,y)},{G static (x,y)},{B static (x,y)} static(Equation (3)) is output. static is also an example of an "image whose absolute value of the temporal frequency is equal to or less than the first value" when the "first value" is set to 0. static It is desirable that the image contains spatial frequency components whose absolute values ​​are greater than zero, and that the image contains chromatic colors (step S611).

[0064] <Step S6123> The luminance motion component extraction unit 612 extracts R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) is the luminance component corresponding to the stationary luminance component Y static (x,y) The luminance stationary component Y static (x, y) is R as in step S5122. static (x,y),G static (x,y),B static The luminance motion component extraction unit 612 extracts the luminance component Y (x, y) of each frame t. original From (x,y,t), the luminance stationary component Y static By subtracting (x,y), the luminance motion component Y motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained and output (step S6123) (equation (5)). motion (x, y, t)) is the luminance component Y original From (x, y, t), the luminance stationary component Y obtained from the still image based on the video corresponding to the object static It includes the luminance motion component obtained by subtracting (x,y).

[0065] In this embodiment, the luminance motion component creates the illusion that a single still image of an arbitrary frame is visually moving, providing the user with a visual experience that is comparable to that of a real moving image.

[0066] [Modification of the sixth embodiment] In this embodiment, a still image M of an arbitrary frame of a moving image M1 is static In addition, we generated video M2 by adding luminance motion components extracted from the high-temporal frequency components of video M1 (Fig. 13A). However, when we extract a still image M of an arbitrary frame of video M1, static A moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to an image of a color component determined from the "color difference component" extracted from the moving image M2 (FIG. 13B). Alternatively, a still image M2 of an arbitrary frame of the moving image M1 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to the image of a color component determined from the "color difference component" extracted from the moving image M2 (FIG. 13B). static A moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to an image consisting of a "luminance component" extracted from the high-time frequency component of the moving image M1 (FIG. 14A). Such a moving image M2 can also produce a dynamic illusion. The moving image M1 may also be a grayscale moving image.

[0067] [Seventh embodiment] As mentioned above, due to the motion perception characteristics of the visual system, the perceived quality of a moving image is maintained even if the spatial resolution or contrast of the "luminance motion component" to be combined with the "image" is reduced. In this embodiment, the "low temporal frequency component" of the moving image M1 is defined as the "image," and the "luminance motion component" is defined as the component obtained by reducing at least one of the high spatial frequency component and contrast of the luminance component extracted from the "high temporal frequency component" of the moving image M1 (the luminance motion component image included in the moving image M1). For example, filtering is performed on the luminance motion component image to reduce at least one of the high spatial frequency component and contrast. After that, a calculation is performed to integrate (add) the "luminance motion component" into the "image," and the resulting moving image M2 is displayed (Figure 14B).

[0068] <Configuration> 15, the moving image component extraction device 71 of this embodiment has a low temporal frequency component extraction unit 511, a luminance motion component extraction unit 512, a filtering unit 719, and an output unit 513. The dynamic illusion presentation device 52 of this embodiment is the same as that of the fifth embodiment. The moving image component extraction device 71 is configured, for example, by loading a predetermined program into the computer described above.

[0069] <Processing> The processing of this embodiment will be described with reference to Fig. 16. The difference between this embodiment and the fifth embodiment is that the luminance motion component Y motion The point is to filter (x, y, t) (luminance motion component image) and add the filtered luminance motion component to the "image" to obtain a moving image M2. motion Only the filtering process for (x, y, t) will be explained.

[0070] The luminance motion component Y obtained in step S5123 motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is input to the filtering unit 719. The filtering unit 719 first motion Transform (x,y,t) into the time-space frequency domain and use FY motion (ξ,η,τ) is obtained, where ξ,η,τ represent the horizontal spatial frequency, vertical spatial frequency, and temporal frequency, respectively. The lower and upper limits of the horizontal spatial frequency are defined as ξ min and ξ max (ξ min <ξ max ) and the lower and upper limits of the vertical spatial frequency are η min and η max (η min <η max ) and the lower and upper bounds of the time frequency are τ min and τ max (τ min <τ max ) ξ,η,τ aremin ≦ξ≦ξ max ,η min ≦η≦η max ,τ min ≦τ≦τ max Below we show an example using the Fourier transform (Equation (18)). In this case, FY motion (ξ,η,τ) is Y motion This is the Fourier spectrum of (x,y,t).

number

[0071] Next, the filtering unit 719 performs the filtering motion (ξ,η,τ) is multiplied by the filter G(ξ,η,τ), and then subjected to an inverse Fourier transform to obtain the luminance motion component gY motion (x, y, t) is obtained (Equation 19).

number

[0072] G(ξ,η,τ) is a filter for reducing high spatial frequency components or contrast. A filter for reducing high spatial frequency components is a low-pass filter, and a filter for reducing contrast (overall contrast) is, for example, a function that linearly converts gray levels or a function that flattens the histogram (spatiotemporal frequency filter). A specific example of a low-pass filter is shown below (Equation (20)).

number

[0073] The subsequent processing is performed using the luminance motion component Y motion (x, y, t) is the luminance motion component gY motion (x, y, t). As mentioned above, even if the high spatial frequency of the luminance motion component video is reduced or the contrast is reduced by filtering, the perceptual quality of the video is not affected within the range allowed by the characteristics of the visual system. Therefore, even if the amount of information of the luminance motion component is reduced by filtering, the user can be provided with a visual experience that is comparable to that of the original video M1.

[0074] [Modification of the Seventh Embodiment] The seventh embodiment is a modification of the fifth embodiment, which uses the luminance motion component Y motion (x, y, t) is the luminance motion component gY motion However, the luminance motion component Y motion (x, y, t) is the luminance motion component gY motion (x, y, t). In this case, steps S7191 and S7192 are executed after step S512' in FIG. 17. Similarly, in the second modification of the fifth embodiment or the sixth embodiment, the luminance motion component Y motion (x, y, t) is the luminance motion component gYmotion It may be replaced with (x,y,t).

[0075] [Eighth embodiment] In the eighth embodiment, the aforementioned "image whose absolute value of temporal frequency is equal to or less than a first value" appears on the surface of the "object," and a "luminance motion component whose absolute value of temporal frequency is equal to or greater than a second value" is superimposed on this "image." This also creates the illusion that the "image" is moving. Below, we will explain an example in which a still "image" is printed on the "object," and a "luminance motion component whose absolute value of temporal frequency is equal to or greater than a second value" is projected onto the "image."

[0076] <Configuration> 19, the moving image component extraction device 81 of this embodiment is obtained by replacing the output unit 513 of any one of the moving image component extraction devices 51, 51', 61, and 71 of the fifth to seventh embodiments or their modified examples with an output unit 813 and a printing unit 814. The dynamic illusion presentation device 82 (illusion presentation device) of this embodiment has an input unit 821 and a projection unit 823 (projector). Furthermore, the object 83 exemplified in this embodiment is a two-dimensional medium such as paper.

[0077] <Processing> A still image M extracted from a video M1 as in any of the above-described embodiments or variations static (Equation (3)) is input to the printing unit 814 (output unit). The printing unit 814 prints the image M static is printed on the surface of the object 83. In any embodiment or modification, the luminance motion component Y motion (x,y,t) or gY motion (x, y, t) is sent to the dynamic illusion exhibiting device 82 and input to the input unit 821. The luminance motion component Y motion (x,y,t) or gY motionThe (x, y, t) is sent to the projection unit 823, which then projects the luminance motion component Y using a known optical production technique (e.g., Reference 9: Takahiro Kawabe, Masataka Sawayama, Kazufumi Maruya, Shinya Nishida, (2014), “A light projection method for illusory deformation of a stationary two-dimensional object using motion information,” 2014 Annual Conference of the Institute of Image Information and Television Engineers, 5-3.). motion (x,y,t) or gY motion (x,y,t) is the image M printed on the object 83 static The video M2 is projected onto the screen and displayed (Equation (21)).

number

[0078] This also creates a dynamic illusion. motion (x,y,t) or gY motion When (x, y, t) are projected by a projector, they cannot have negative values. Therefore, the overall brightness of video M2 increases. Furthermore, the increase in brightness due to projection is partly multiplicative. Therefore, the brightness distribution of video M2 is quite different from the brightness of the original video M1. Nevertheless, the user can visually experience the movement of the original video M1 from video M2. This is thought to be due to adaptive brightness contrast normalization by the visual system, as shown in Reference 9. In this way, this form can provide the user with a visual experience that is comparable to that of the original video M1.

[0079] [Modification of the Eighth Embodiment] In the eighth embodiment, a still image M static is printed on an "object" such as paper. However, instead of printing, it may be projected onto an "object" such as a screen using another projector, or displayed on an "object" such as electronic paper. Also, the "motion component video" may be projected onto a transparent display as shown in Reference 9. Also, the image M static When projecting or displaying an image, static It is not necessary for the image M to be a still image, but it may be a slowly moving image. staticIt is possible to give the user a visual experience of motion at a higher temporal frequency than that of the image M1. static Instead of extracting the luminance motion component Y, we create a video M1 based on still images of the object, such as a building or painting, and then extract the luminance motion component Y motion (x,y,t) or gY motion (x,y,t) or to generate the brightness motion component Y from a still image obtained by photographing the "object" motion (x,y,t) or gY motion (x, y, t) can be generated. The generated luminance motion component Y motion (x,y,t) or gY motion By projecting (x,y,t) onto an image that appears on the surface of an "object," it is possible to create the illusion that the "image" is moving.

[0080] [Other modifications, etc.] The present invention is not limited to the above-described embodiment and its variations. For example, the moving image component extraction device and the dynamic illusion presentation device may be the same device. Alternatively, the processing of each unit included in the moving image component extraction device and the dynamic illusion presentation device may be performed by different devices.

[0081] The various processes described above may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capabilities of the devices executing the processes or as necessary. Needless to say, other modifications are possible within the scope of the present invention.

[0082] When the above-described configuration is realized by a computer, the processing content of the functions that each device should have is described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing this processing content can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such recording media include a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory.

[0083] This program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0084] A computer that executes such a program, for example, first stores the program recorded on a portable recording medium or transferred from a server computer in its own storage device. When executing a process, the computer reads the program stored on its own storage device and executes processing in accordance with the read program. As another form of executing this program, the computer may read the program directly from a portable recording medium and execute processing in accordance with the program. Furthermore, the computer may execute processing in accordance with the received program each time a program is transferred from the server computer to the computer. The above-mentioned processing may be executed by a so-called ASP (Application Service Provider) type service, which realizes processing functions by simply issuing execution instructions and obtaining results without transferring the program from the server computer to the computer.

[0085] A data structure may be distributed that associates "first image data recording a first image obtained by photographing" with "second image data recording a second image having transparency, which is a narrowband image in which the spatial frequency band of the entire image is narrower than the spatial frequency band of the first image while retaining edge information contained in the first image, and which is displayed superimposed on the first image so that edges contained in the narrowband image overlap with edges of the first image when displayed." This distribution may be performed by distributing the data structure via the Internet, etc., or by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the data structure is recorded.

[0086] Furthermore, a data structure including first data representing an "image including a spatial frequency component with an absolute value greater than zero and having a temporal frequency whose absolute value is equal to or less than a first value" output from the motion component extraction device and second data representing a "luminance motion component corresponding to the image and having a temporal frequency whose absolute value is equal to or greater than a second value" may be distributed. This distribution may be achieved by distributing the data structure via the Internet, or by selling, transferring, lending, etc., a portable recording medium such as a DVD or CD-ROM on which the data structure is recorded. A device provided with this data structure inputs the "first data" and "second data" to a calculation unit, which performs a calculation to add the "luminance motion component" to the "image" to obtain a motion image and displays this motion image on a display unit. Alternatively, the first data may be input to an output unit, which displays the "image" on the surface of an "object," and the second data may be input to a projection unit, which projects the "luminance motion component" onto the "image" displayed on the surface of the "object." This allows a motion image that creates a dynamic illusion to be displayed on a display unit or an object. Furthermore, a data structure of a video used in a device that superimposes a video on an "object" may be distributed to create the perception that the "object" is moving. Here, the absolute value of the temporal frequency of the luminance motion component in multiple frames in the video is greater than the absolute value of the temporal frequency in multiple frames in the video corresponding to the "object." Alternatively, this data structure may include a luminance motion component obtained by subtracting a luminance static component obtained from a still image based on the video corresponding to the "object" from the luminance component of each of multiple frames in the video corresponding to the "object." [Industrial Applicability]

[0087] The present invention can be used in a variety of fields, including (1) advertising, where light projection can be used to add the impression of movement to paper media or signs, (2) interior design, where interior patterns on floors, walls, etc. can be transformed to create an illusion of movement, and (3) art, toys, and entertainment, where character illustrations can be given movement or combined with conventional projection mapping technology.

Claims

1. 1. A device for superimposing an image on an object to make the object appear to be moving and / or deforming, comprising: the image includes a luminance motion component corresponding to the movement and / or deformation of the object; An apparatus in which the image is an image that contains only a luminance motion component that creates the illusion that the image of the object itself is moving and / or deforming.

2. 1. A device for superimposing an image onto an image of an object to give an observer the illusion that the image of the object itself is moving and / or deforming, comprising: the image is an image including a luminance motion component that causes the illusion, An apparatus, wherein the image is an image that contains only a luminance motion component that creates the illusion that the image of the object itself is moving and / or deforming.

3. A method for superimposing an image on an image of an object to give an observer the illusion that the image of the object itself is moving and / or deforming, comprising: the image is an image including a luminance motion component that causes the illusion, A method in which the image is an image containing only a luminance motion component intended to create the illusion of movement and / or deformation of the image of an object.

4. A method for superimposing an image on an image of an object to give an observer the illusion that the image of the object itself is moving and / or deforming, comprising: the image is an image including a luminance motion component that causes the illusion, A method in which the image is an image containing only a luminance motion component intended to create the illusion of movement and / or deformation of the image of an object.

5. A program for causing a computer to function as the device according to any one of claims 1 to 4.

Images