Device, method, and program

By superimposing images with luminance motion components onto objects, the method effectively leverages their patterns to create dynamic illusions, addressing the computational challenges of existing technologies and enhancing the perceived movement and texture of objects with reduced computational resources.

JP2024161093AActive Publication Date: 2024-11-15NIPPON TELEGRAPH & TELEPHONE CORP

Patent Information

Application Number
JP2024144008
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2014-11-13
Filing Date
2024-08-26
Publication Date
2024-11-15
Estimated Expiration
2035-04-21

AI Technical Summary

Technical Problem

Existing technologies primarily focus on using objects as mere canvases for motion effects without effectively utilizing the object's inherent patterns to create illusions of movement, requiring significant computational resources for simulating surrounding environments.

Method used

The method involves superimposing images with luminance motion components onto objects, utilizing their patterns to create illusions of movement, which can be applied to both two-dimensional and three-dimensional objects, including chromatic and achromatic ones, by projecting or displaying videos that incorporate low spatial frequency components and pixel movements based on pre-calculated algorithms.

Benefits of technology

This approach allows for the creation of dynamic illusions with reduced computational effort, providing a perceptual experience that enhances the object's texture or appearance, making it seem to move or change in texture, and can be applied to various media types, including paper and screens.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161093000001_ABST
    Figure 2024161093000001_ABST
Patent Text Reader

Abstract

To make an object be perceived as if motion is given to the object by using a pattern of the object.SOLUTION: Video is superimposed on an object in order to make the object be perceived as if motion is given to the object. The video includes a luminance motion component corresponding to the motion given to the object.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for creating an optical illusion. [Background technology]

[0002] For example, Non-Patent Document 1 discloses a technology that gives the object an illusionary effect of movement by measuring the three-dimensional shape of the object (a single achromatic color) with a camera and projecting an image that expresses movement in accordance with the three-dimensional shape of the object. In Non-Patent Document 1, a flat plane of a single achromatic color and a model car of a single achromatic color placed on the plane are taken as objects, and this object is used as a canvas, and a color scheme that simulates the color and reflective properties of the car body, the road around the car, the daylight conditions of the space in which the car is running, etc. is projected onto the object as an image, creating the illusion that the model car, which is the object, is running on a road, giving the object an illusion of movement. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Raskar, R.; Ziegler, R.; Willwacher, T., "Cartoon Dioramas in Motion", International Symposium on Non-Photorealistic Animation and Rendering (NPAR), June 2002. Summary of the Invention [Problem to be solved by the invention]

[0004] Non-Patent Document 1 regards an object as a mere canvas and gives the effect of movement to the object without using the pattern that the object has. The objective of the present invention is to make the object appear as if it is moving by using the pattern that the object has. [Means for solving the problem]

[0005] To make an object appear to be in motion, an image is superimposed on the object, the image including a luminance motion component corresponding to the motion of the object. Effect of the Invention

[0006] This makes it possible to make the object appear to be in motion by utilizing the pattern that the object possesses. [Brief description of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a video display device according to a first embodiment. [Diagram 2] 5 is a flowchart showing the operation of the image display device according to the first embodiment. [Diagram 3] FIG. 1 is a block diagram showing a configuration of a video projection device according to a first embodiment. [Figure 4] 5 is a flowchart showing the operation of the image projection device according to the first embodiment. [Diagram 5] 4A to 4C are diagrams for explaining the relationship between the projection angle of view and an object when the projection unit of the image projection device according to the example of the first embodiment is realized by a projector. [Figure 6] FIG. 11 is a block diagram showing the configuration of a video display device according to a second embodiment. [Figure 7] 10 is a flowchart showing the operation of the image display device according to the second embodiment. [Figure 8] FIG. 11 is a block diagram showing the configuration of an image display device according to a third embodiment. [Figure 9] 11 is a flowchart showing the operation of an image generating unit of an image display device according to a third embodiment. [Figure 10] FIG. 1 is a diagram illustrating stationary components and moving components. [Figure 11] FIG. 13 is a block diagram showing the configuration of a video display device according to a fourth embodiment. [Figure 12] 10 is a flowchart showing the operation of the image display device according to the fourth embodiment. [Figure 13]13A and 13B are conceptual diagrams for illustrating an overview of the process. [Figure 14] 14A and 14B are conceptual diagrams for illustrating an overview of the process. [Figure 15] FIG. 15 is a block diagram illustrating a functional configuration of the embodiment. [Figure 16] FIG. 16 is a flow diagram illustrating the process of the embodiment. [Figure 17] FIG. 17 is a flow diagram illustrating the process of the embodiment. [Figure 18] 18A and 18B are diagrams illustrating a method of adding a luminance motion component to an image. [Figure 19] FIG. 19 is a block diagram illustrating a functional configuration of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] An embodiment of the present invention will be described. In the embodiment described below, unlike the conventional technology, the pattern of an object is utilized while the appearance of the pattern is illusory deformed. In each embodiment, the object is not regarded as a simple canvas, but the pattern of the object is actively utilized to create the illusion of movement. Therefore, it is preferable that the object is not a single achromatic color, but a chromatic color or grayscale. In each embodiment, unlike the conventional technology, the three-dimensionality of the object does not need to be considered. For example, it is also possible to illusory deform the appearance of an image of a still object. In the conventional technology, the focus has been on the movement and deformation of the object, but in each embodiment, by utilizing both the pattern of the object and the image information of the image displayed superimposed on the object, the object is endowed with a texture and impression (liquid feeling or facial expression) that did not originally exist in the object, which is different from the conventional technology. Also, unlike the technology of simply deforming a photographed image on a screen, in each embodiment, the texture impression change of the object can be given to the viewer by superimposing the image. This allows the viewer to have a different perceptual experience than the conventional technology. Each embodiment will be described in detail below. Components having the same functions are given the same numbers and redundant explanations are omitted.

[0009] [First embodiment] Hereinafter, with reference to FIG. 1 and FIG. 2, a video display device according to a first embodiment, which is a basic configuration example of the present invention, will be described. FIG. 1 is a block diagram showing the configuration of the video display device 1 according to this embodiment. FIG. 2 is a flowchart showing the operation of the video display device 1 according to this embodiment. As shown in FIG. 1, the video display device 1 includes a display unit 11. The display unit 11 displays a transparent image by superimposing it on an object. Here, the "object" may be an object having a three-dimensional shape (e.g., a vase, a ball, a model) or a predetermined plane (e.g., paper, a board, a wall, a screen). When the object is a plane, it is desirable that the plane includes a pattern. Examples of patterns included in the plane include, for example, a photograph or image printed on paper, and a photograph or image projected onto a predetermined plane. When the object is a screen such as a display, an example of a pattern includes an image displayed on the screen such as a display.

[0010] A typical method of "superimposing" a transparent image on an object is to project the image on the object using a projector or the like. In this case, the image projected by the projector or the like naturally has transparency. In addition, for example, a transparent liquid crystal screen is placed in front of the object, and a transparent image is projected on the liquid crystal screen, so that the transparent image can be "superimposed" on the object as seen by an observer facing the object through the liquid crystal screen. Here, the "image" is, for example, an image in which an image of a distortion distribution having low spatial frequency components is switched over time. In this embodiment, the image is prepared in advance and input from outside the image display device 1. The display unit 11 displays the image so that an edge included in the image overlaps with the contour of the object or an edge included in the object. For example, when the object is a vase, the display unit 11 displays the image so that an edge included in the image overlaps with an edge included in the vase, such as the contour of the object or a pattern drawn on the vase. For example, when the object is a plane and an image is projected onto the plane as a pattern, the display unit 11 displays the image so that the edges contained in the image overlap with the edges contained in the object, such as the pattern of the image projected onto the object.

[0011] Therefore, the display unit 11 displays a transparent image in which an image of a distortion distribution having low spatial frequency components switches over time, superimposed on the object such that an edge contained in the image overlaps with the contour of the object or an edge contained in the object (S11).

[0012] [Example of the first embodiment] Hereinafter, the image projection device 1a, which is an example of this embodiment, will be described with reference to FIG. 3 and FIG. 4. FIG. 3 is a block diagram showing the configuration of the image projection device 1a of the example of this embodiment. FIG. 4 is a flowchart showing the operation of the image projection device 1a of the example of this embodiment. As shown in FIG. 3, the image projection device 1a includes a projection unit 11a. In this example, a stationary object is treated. As described above, the projection unit 11a projects an image in which an image of a distortion distribution having low spatial frequency components is switched over time onto a stationary object so that an edge included in the image overlaps with the contour of the object or an edge included in the object (S11a). The projection unit 11a can be realized, for example, by a projector. For example, when the projection unit 11a is a projector, the horizontal viewing angle θ and the vertical viewing angle φ of the object 9 as viewed from the center of the projector projection lens must be matched with the horizontal and vertical viewing angles of the image to be projected, as shown in FIG. 5.

[0013] It is also possible to select the projection unit 11a as an example of the display unit 11 in the second, third, fourth, and other embodiments described below, and each block diagram representing each device in the embodiments described below indicates that the projection unit 11a is included as an example of the display unit 11.

[0014] In the technology of Non-Patent Document 1, it is not easy to create a special illusion that makes the medium (e.g., air, water, water vapor) around the object appear to be irregularly fluctuating. In order to create the above-mentioned special illusion by applying the technology of Non-Patent Document 1, a huge amount of calculation is required to simulate the fluctuation of the medium and the refraction of light rays. According to the image projection device 1a of the present embodiment, it is possible to create a special illusion on the object with a small amount of calculation. In addition, according to the image projection device 1a of the present embodiment, it is possible to create an illusion on a flat object, a three-dimensional object, a chromatic object, and an achromatic object having light and shade.

[0015] [Second embodiment] Hereinafter, the image display device 2 of the second embodiment, which includes an image generation function within the device, will be described with reference to Figs. 6 and 7. Fig. 6 is a block diagram showing the configuration of the image display device 2 of this embodiment. Fig. 7 is a flowchart showing the operation of the image display device 2 of this embodiment. As shown in Fig. 6, the image display device 2 includes the same display unit 11, shooting unit 21, and image generation unit 22 as in the first embodiment.

[0016] The photographing unit 21 photographs an object and obtains an original image (S21). The image generating unit 22 generates a plurality of different narrowband images from the original image, and generates an image in which the plurality of different narrowband images are arranged so as to be smoothly connected in time (S22). The display unit 11 displays the generated image by superimposing it on the object, as described above (S11).

[0017] A narrowband image is an image in which the spatial frequency band of the entire image is narrower than that of the original image while retaining edge information contained in the original image, and which has transparency.

[0018] As an example of the implementation of the image generation unit 22, for example, as described in Reference 1, an image may be convolved with a directional filter with a phase difference of 180 degrees, and the convolved images may be smoothly connected in time to generate an image that gives the illusion of movement. (Reference 1: Freeman, WT, Adelson, EH, & Heeger, DJ (1991). Proceedings of the 18th annual conference on computer graphics and interactive techniques, 27-30.)

[0019] [Third embodiment] Hereinafter, the image display device 3 of the third embodiment, which includes an image generating function within the device, will be described with reference to FIG. 8 and FIG. 9. FIG. 8 is a block diagram showing the configuration of the image display device 3 of this embodiment. FIG. 9 is a flowchart showing the operation of the image generating unit 32 of the image display device 3 of this embodiment. As shown in FIG. 8, the image display device 3 includes the same display unit 11 and image capturing unit 21 as in the second embodiment, and an image generating unit 32 different from that in the second embodiment. The image generating unit 32 includes a deformation image generating unit 321, a Fourier transform unit 322, and a separation unit 323. The deformation image generating unit 321 applies dynamic deformation to an original image obtained by capturing an object, thereby creating an image in which a still object image is deformed (S321). The deformation is expressed by pixel movement. This pixel movement is based on an algorithm calculated in advance. For example, when it is desired to add a liquid impression to an object, the algorithms in Reference 2 and Japanese Patent Application No. 2013-132609, which is a patent application that was not published at the time of filing this application, are referred to. (Reference 2: Kawabe, T., Maruya, K., & Nishida, S. (2013). Seeing transparent liquids from dynamic image distortion. Journal of Vision, 13(9): 208.)

[0020] Hereinafter, an example of realizing the deformed image generating unit 321 based on the algorithm of Japanese Patent Application No. 2013-132609 will be disclosed. First, the deformed image generating unit 321 generates a plurality of modulated images by modulating the original image based on the distortion distribution. At this time, in order to strongly perceive the impression that the transparent liquid is flowing, it is desirable to modulate the original image with a distortion in which the spatial frequency of the distortion distribution (distortion map) is 3 cpd (cycles per degree) or less. In other words, if a coarse distortion that reduces the difference in the amount of distortion between adjacent pixels is added to the original image, the impression of the transparent liquid can be more strongly perceived. Note that the original image may be modulated using a distortion distribution having the same low spatial frequency components (for example, 3 cpd or less), or may be modulated using a distortion distribution having different low spatial frequency components (for example, 2 cpd or less on one side and 3 cpd or less on the other side). In addition, the modulated image is given a distortion direction of two or more dimensions. The type of distortion may be any two-dimensional geometric distortion such as rotational distortion, translation distortion, random distortion, etc.

[0021] Next, the transformed image generating unit 321 generates an image based on a plurality of modulated images generated from the original image. For example, the transformed image generating unit 321 may generate an image in the form of a sequence of modulated images in which the plurality of modulated images generated from the original image are switched over in time. The modulated image sequence is an image in which the presentation time (frame rate) of each image is set within a range in which the modulated image sequence can be viewed by a viewer as a moving image rather than a series of still images, that is, the modulated image sequence is an "image formed by arranging modulated images in time." In addition, for example, the transformed image generating unit 321 may perform control to switch and present each of the plurality of modulated images generated from the original image. The presentation interval of each image may be controlled within a range in which the viewer can view the image as a moving image (image) rather than a series of still images. For example, the presentation time of each image may be set to a range not exceeding 0.05 (sec), or the frame rate may be set to 20 Hz or higher.

[0022] Next, Fourier transform unit 322 performs a three-dimensional (time-space) Fourier transform on the image created by deformed image generation unit 321 (S322). Separation unit 323 separates DC components (still components) and motion components by time filtering, and outputs only the motion components as an image (S323).

[0023] In other words, the modified image generating unit 321 generates an image by arranging a plurality of images obtained by moving pixels of an original image based on a predetermined algorithm so as to be smoothly connected in time (S321). The Fourier transform unit 322 performs a three-dimensional (time-space) Fourier transform on the generated image (S322). The separation unit 323 separates still components and moving components based on the Fourier transform result, and outputs only the moving components as an image (S323).

[0024] FIG. 10 shows an example of the still components (DC components) and moving components extracted by the above-mentioned steps S321 to S323. The moving components are mainly composed of high spatial frequency components. As shown in FIG. 5, by displaying the moving components as an image at the same visual angle as the target object and superimposing it on the target object, the phase relationship between the two matches that of the original video. Under these conditions, the spatial structure of the target object is illusory captured by the moving components. As a result, the target object appears to move.

[0025] [Fourth embodiment] Hereinafter, the image display device 4 of the fourth embodiment, which includes a function for generating an original image to be displayed on an object, will be described with reference to FIG. 11 and FIG. 12. FIG. 11 is a block diagram showing the configuration of the image display device 4 of this embodiment. FIG. 12 is a flow chart showing the operation of the image display device 4 of this embodiment. As shown in FIG. 11, the image display device 4 of this embodiment includes the same display unit 11, shooting unit 21, and image generation unit 32 as in the third embodiment, and an original image display unit 41 not included in the third embodiment. The shooting unit 21 and the image generation unit 32 execute steps S21 and S32, respectively, as in the third embodiment. The original image display unit 41 displays (projects) an original image acquired by shooting the object in step S21 on another still medium (for example, a display, a screen, etc.) (S41). The display unit 11 displays (projects) an image by superimposing it on the original image thus displayed on the still medium. This provides the same effect as described above. At this time, the image may be created by convolving an orientation filter on the image as shown in the second embodiment, or may be created by applying a deformation to the image as shown in the third embodiment. When displayed on another stationary medium, the original image can be printed on the object, projected by another projector, or displayed on electronic paper.

[0026] An external camera is used to capture the subject. It is preferable that the camera has as high a spatial resolution as possible. Excluding calculation time, it is possible to instantly add an illusory movement to a still subject in front of the eyes. A projector can be used as the projection unit 11a of the image projection device 4a, which is an example of this embodiment. This projector can be a commercially available one, but if it is used in a bright room, it is necessary to use a projector with high brightness.

[0027] The ideal viewing distance varies depending on the size of the object. For example, if you add movement to a 16cm square image using this method, a viewing distance of about 1.5m is required. The larger the object, the longer the viewing distance needs to be.

[0028] When projecting an image onto a three-dimensional object, if the depth of the projection surface of the object is not taken into consideration, there may be a misalignment between the image and the structure of the object if the depth difference of the projection surface of the object is large. It is more preferable to use an object with a relatively small depth difference of the projection surface that can be projected without causing a misalignment. In addition, when projecting an image onto a three-dimensional object with a large depth, it is possible to grasp the three-dimensional shape of the object and transform the image to be projected according to that shape, thereby creating an illusion of deformation of a surface that is tilted in the depth direction.

[0029] According to the image display device of this embodiment, for example, the expression of a still face image can be changed in an illusory manner, or the direction of the line of sight can be changed. In addition, by changing the pattern of movement, the object can be made to look as if it is fluttering or existing under a flowing liquid. When a two-dimensional medium such as a printed matter is used as the object and an image is projected onto the object, the dynamic range of the brightness of the object in the medium is generally narrow. However, in this embodiment, an image with a high dynamic range compared to the object is projected onto the object, so that the object can be made to look high quality while adding an illusory movement to the object. Furthermore, according to the image display device of this embodiment, it is possible to instantly add an illusory effect of movement to an image captured by a camera, excluding processing time.

[0030] The image display device of this embodiment can be applied as an exhibition technique in art galleries and museums, or as a component technology of attractions used in entertainment facilities. For example, it can visually move children's characters printed on still media such as paper.

[0031] In the conventional technology, in order to present a moving image, a color difference motion component corresponding to the time frequency of the moving image was required. Below, we will explain a form that creates the illusion that an image is moving at a time frequency equal to or greater than a certain magnitude without using a color difference motion component with a time frequency equal to or greater than a certain magnitude.

[0032] Humans detect localized motion components from moving images through functions such as spatiotemporal filtering. (Reference 3: Watson, AB, & Ahumada, A. J., “Model of human visual-motion sensing,” Journal of the Optical Society of America, (1985), A, 2, 322-342.) A similar function can be realized by image information processing, which separates and extracts moving and stationary components from moving images through time-frequency filters, etc.

[0033] It is known that humans perceive static image components defined by color and luminance contrast as being attracted to luminance motion components (Reference 4: Ramachandran, VS, “Interaction between colour and motion in human vision,” Nature (1987), 328, 645-647. and Reference 5: Anstis, S., “Kinetic edges become displaced, segregated, and invisible,” In DM-K. Lam (Ed.), Neural mechanisms of visual perception, Proceedings of the Second Retina Research Foundation Conference, Texas (1989): Portfolio Press, 247-260.). Based on this visual characteristic, it is thought that the way an image appears can be modulated by the luminance motion components in a moving image.

[0034] While the human visual system is insensitive to motion signals defined by color signals, it is sensitive to motion signals defined by luminance signals (Reference 6: Ramachandran, VS, & Gregory, R. L, “Does colour provide an input to human motion perception?” Nature (1978), 275, 55-56.). In addition, the human spatial resolution for objects moving at a certain speed or faster is lower than that for stationary objects (Reference 7: Kelly, DH (1979). Motion and vision. II. Stabilized spatio-temporal threshold surface. Journal of the Optical Society of America, 69, 1340-1349.). In other words, even if the color components are removed from the motion components of a moving image, the quality of the image perceived by humans does not decrease significantly.

[0035] In each embodiment, in consideration of the above human perception characteristics, a "luminance motion component" with an absolute value of the temporal frequency greater than or equal to a second value is added to an "image with an absolute value of the temporal frequency less than or equal to a first value" that includes a spatial frequency component with an absolute value greater than zero. The "luminance motion component" is a component corresponding to the "image", and the "second value" is greater than the "first value". "Including a spatial frequency component with an absolute value greater than zero" means including a non-zero spatial frequency component. For example, if the "first value" is F1 and the "second value" is F2, the relationship 0 ≤ F1 < F2 is satisfied. F1 may be 0 or F1 > 0. An example of F1 is the absolute value of the temporal frequency that is perceived as if the "image" is stationary (e.g., in the vicinity of zero). However, F1 may be the temporal frequency at which the "image" is perceived to be moving. When F1 = 0, the "image with an absolute value of the temporal frequency less than or equal to the first value" is a stationary image. When F1 > 0, the "image with an absolute value of the temporal frequency less than or equal to the first value" has a motion with a frequency component with a magnitude less than or equal to the first value and moves slower than the "luminance motion component with an absolute value of the temporal frequency greater than or equal to the second value". The "image" may have only a single temporal frequency component (e.g., 0 Hz) or may have a plurality of temporal frequency components (i.e., it may be a composite of images with a plurality of temporal frequencies). Hereinafter, the "component with an absolute value of the temporal frequency less than or equal to the first value" is referred to as the "low temporal frequency component", and the "component with an absolute value of the temporal frequency greater than or equal to the second value" is referred to as the "high temporal frequency component". It is desirable that the "image" is not uniform and includes a spatial frequency component with an absolute value greater than zero (non-zero spatial frequency component). Both the color difference component and the luminance component of the "image" may include a spatial frequency component with an absolute value greater than zero, or only one of the color difference component or the luminance component may include a spatial frequency component with an absolute value greater than zero. A person who views a video in which the "luminance motion component" is added to such an "image" has an illusion (dynamic illusion) that the "image" is moving at a temporal frequency higher than the temporal frequency of the magnitude of the "first value". For example, when the "luminance motion component" is added to a stationary "image", a person has an illusion that this "image" is moving. Note that "frequency α is higher than frequency β" means that the absolute value |α| of frequency α is greater than the absolute value |β| of frequency β.

[0036] The "image" may be an achromatic (grayscale) image containing only luminance components, or an image containing chromatic (color) colors. In particular, when the latter image is added with a luminance motion component, it is possible to create the illusion that the chromatic image is moving at a higher frequency than the low-time frequency components, even though it does not contain a chrominance component higher than the low-time frequency components. The "image" may be an image (image information) to be processed, or an image that appears on the surface of an object. Examples of an "image" that appears on the surface of an object include images or photographs that are "printed," "drawn," "displayed," or "projected" onto the surface of the object, patterns or designs based on the color of the material that constitutes the surface of the object, and patterns based on the shape of the surface of the object (e.g., patterns, borders, shading). The "surface of the object" may be flat, curved, or uneven. The "object" may be an object with a three-dimensional shape (e.g., a vase, a ball, a model, a building), or an object that can be considered flat for the purpose of use (e.g., paper, a board, a wall, a screen, a display, a transparent display).

[0037] In order to create a large dynamic illusion, it is desirable that the "image" and the "luminance motion component" correspond to the same "video". For example, the "image" corresponds to a component whose temporal frequency in the "multiple frames" of the "video" is zero (temporal frequency = 0 Hz) or close to zero, and the "luminance motion component" corresponds to a luminance component whose absolute value of the temporal frequency in the "multiple frames" is positive (|temporal frequency|> 0 Hz). In particular, when the "video" includes "periodic or repetitive motion components", a greater effect can be expected. "Periodic motion components" do not mean only components that perform strict periodic motion, but also components that perform highly periodic motion. Similarly, "repetitive motion components" do not mean only components that perform strict repetitive motion, but also components that perform highly repetitive motion. When the "video" includes "periodic or repetitive motion components", the "image" may correspond to a still image of any frame included in the "video". "A corresponds to B" may mean that A is B, that B originates (is based on) A, or that A originates from B. "B is derived from A" may mean that A is obtained from B, that A is obtained from a copy of B, or that A is obtained from an approximation of B. For example, an "image" and a "luminance motion component" may be extracted from a "video" or a copy of the video, or a "video" may be generated from a still "image" captured by a camera or scanner, and a "luminance motion component" may be extracted from the "video."

[0038] The function of motion vision for luminance contrasts above a certain value is steady (Reference 8: Pantle, A., & Sekuler, R. (1969). Contrast response of human visual mechanisms sensitive to orientation and direction of motion. Vision Research, 9, 397-406.). In addition, the spatial resolution of motion vision is lower than the spatial resolution of perception of still images (Reference 7). In other words, even if the spatial resolution or contrast of the luminance motion components extracted from a moving image is manipulated to reduce the quality of the motion information itself, the quality of the perceived moving image is maintained. Therefore, even if the high spatial frequency components or components with reduced contrast of the luminance motion components contained in the "moving image" are used as the "luminance motion components," a sufficient dynamic illusion can be induced. This allows the amount of information to be reduced without significantly reducing the level of the dynamic illusion.

[0039] "Adding a luminance motion component to an image" means, for example, "combining a luminance motion component with an image," "superimposing a luminance motion component on an image," "integrating a luminance motion component with an image," "adding a luminance motion component to an image," "reflecting a luminance motion component in an image," "incorporating a luminance motion component into an image," or "performing an operation including at least addition, multiplication, or exponentiation on the pixel values ​​of an image and the pixel values ​​of the luminance motion component." Specific methods for "adding a luminance motion component to an image" will be described later.

[0040] [Fifth embodiment] In this embodiment, a "low-temporal frequency component" and a "high-temporal frequency component" are extracted from the video M1. The "low-temporal frequency component" is the "image," and the luminance component extracted from the "high-temporal frequency component" is the "luminance motion component." The chrominance component of the "high-temporal frequency component" is not used. A calculation is performed to integrate (add) the "luminance motion component" to the "image," and the resulting video M2 is displayed (FIG. 13A). As a result, the "image" and the "luminance motion component" are integrated in the visual system of the person who sees the video M2, creating the illusion that the "image" moves at a higher temporal frequency than the "low-temporal frequency." In this way, the quality of the perceptual image is maintained even if the chrominance component of the "high-temporal frequency component" is removed from the video M1.

[0041] <Configuration> As illustrated in FIG. 15, the moving image component extraction device 51 of this embodiment has a low-temporal frequency component extraction unit 511 (first processing unit), a luminance motion component extraction unit 512 (second processing unit), and an output unit 513. The dynamic illusion presentation device 52 (illusion presentation device) of this embodiment has an input unit 521, a calculation unit 522, and a display unit 523. The moving image component extraction device 51 and the low-temporal frequency component extraction unit 511 are devices configured by loading a predetermined program into a general-purpose or dedicated computer equipped with a processor (hardware processor) such as a central processing unit (CPU), a memory such as a random-access memory (RAM) and a read-only memory (ROM), a display device such as a display, etc. This computer may have one processor and memory, or may have multiple processors and memories. This program may be installed in the computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units may be configured using electronic circuits that realize processing functions without using programs, rather than electronic circuits that realize functional configurations by reading programs like a CPU. Furthermore, the electronic circuitry constituting one device may include multiple CPUs.

[0042] <Processing> The processing of this embodiment will be described with reference to Fig. 16. The moving image M1 of this embodiment is a color moving image including chromatic colors, and is composed of pixel values ​​that represent intensity changes of the R channel, G channel, and B channel (Equation (1)).

number

[0043] The pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) constituting the video M1 (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max) is input to a low-temporal frequency component extraction section 511 and a luminance motion component extraction section 512 of a motion component extraction device 51 (FIG. 15).

[0044] The low-temporal frequency component extraction unit 511 extracts pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) to the stationary component R static (x,y),G static (x,y),B static In this embodiment, the time average values ​​of pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) are obtained and output as R static (x,y),G static (x,y),B static Let (x, y) be (Equation (2)).

number

[0045] The low-time frequency component extraction unit 511 extracts R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) is a two-dimensional matrix {R static (x,y)},{G static (x,y)},{B static (x,y)} static Output (equation (3)).

number

[0046] The luminance motion component extraction unit 512 converts the pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) into R original (x,y,t),G original (x,y,t),B original (x, y, t) (step S5121), and these are weighted and added according to the degree to which each color contributes to the luminance, to obtain the luminance component Y original (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained (step S5122) (equation (4)). Y original (x,y,t)=α R R original (x,y,t)+α G G original (x,y,t)+α B B original (x,y,t) (4) However, α R ,α G ,α B is a weighting coefficient (constant) (e.g., α R =0.299,α G =0.587,α B =0.114).

[0047] Furthermore, the luminance motion component extraction unit 512 extracts the luminance component Y original From (x,y,t), the luminance stationary component Y static By subtracting (x,y), the luminance motion component Y motion (x, y, t) is obtained and output (equation (5)). static (x,y) is the luminance component Y original (x,y,t) (where x min ≦x≦xmax ,y min ≦y≦y max ,t min ≦t≦t max ) over time. Y motion (x,y,t)=Y original (x,y,t)-Y static (x,y) (5) Luminance motion component Y motion (x, y, t) are multiple frames t(t min ≦t≦t max ) is a luminance component whose absolute value of the temporal frequency is positive, and is an example of a "luminance motion component whose absolute value of the temporal frequency is equal to or greater than the second value." This luminance motion component is a component corresponding to the image, and the second value is greater than the first value (step S5123).

[0048] Image M static and the luminance motion component Y motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is input to the output unit 513, and is sent from there to the dynamic illusion presentation device 52. static and the luminance motion component Y motion The (x, y, t) is input to the input unit 521 of the dynamic illusion presentation device 52 and sent to the calculation unit 522. The calculation unit 522 calculates the image M static to the luminance motion component Y motion For example, the calculation unit 522 calculates and outputs a moving image M2 that takes into account (x, y, t). static Each stationary component R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) with the luminance motion component Y motion (x, y, t) are added together to obtain video M2 (Figure 18A, equation (6)).

number

number

[0049] The moving image M2 obtained in this manner is input to the display unit 523 and displayed there (step S523). Although the moving image M2 does not contain any motion components of color signals, it creates the illusion of motion. In other words, even if the color motion components are removed from the moving image M1, it is possible to provide the user with a visual experience that is comparable to that of the original moving image.

[0050] [Modification 1 of the fifth embodiment] The process of separating and extracting each component from the video M1 can also be realized by converting the video M1 into the time-frequency domain by a Fourier transform or the like and performing time-frequency filtering. The following description will focus on the differences from the matters described so far. The same reference numbers will be used for matters already described and the description may be omitted.

[0051] <Configuration> 15, the moving image component extraction device 51' of this embodiment has a frequency domain conversion unit 514', a low temporal frequency component extraction unit 511' (first processing unit), a high temporal frequency component extraction unit 515', a luminance motion component extraction unit 512' (second processing unit), time domain conversion units 516', 517', and an output unit 513. The dynamic illusion presentation device 52 is the same as in the fifth embodiment. The moving image component extraction device 51' is configured, for example, by loading a predetermined program into a computer such as that described above.

[0052] <Processing> The process of this modification will be described with reference to FIG. 17. The pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) are input to a frequency domain transform unit 514'. The frequency domain transform unit 514' transforms the pixel values ​​R(x,y,t), G(x,y,t), and B(x,y,t) into the time-frequency domain to obtain time-frequency domain values ​​FR(x,y,f), FG(x,y,f), and FB(x,y,f). An example using the Fourier transform is shown below (Equation (8)).

number

[0053] The low-temporal frequency component extraction unit 511′ multiplies FR(x,y,f), FG(x,y,f), and FB(x,y,f) by a low-pass filter LF(f) to obtain GR static (x,y,f),GG static (x,y,f),GB static Obtain (x, y, f) and output it (equation (9)). GR static (x,y,f)=FR(x,y,f)LF(f) GG static (x,y,f)=FG(x,y,f)LF(f) (9) GB static (x,y,f)=FB(x,y,f)LF(f) Here, the low-pass filter LF(f) is expressed by equation (10).

number

[0054] GR static (x,y,f),GG static (x,y,f),GB static (x, y, f) is input to the time domain transform unit 516'. The time domain transform unit 516' transforms the GR static (x,y,f),GG static (x,y,f),GB static Transform (x,y,f) into the time domain and extract the low time frequency components R static (x,y,t),G static (x,y,t),B static (x, y, t) is obtained. Below is an example using the inverse Fourier transform (equation (11)).

number

number

[0055] The high-temporal frequency component extraction unit 515′ multiplies FR(x,y,f), FG(x,y,f), and FB(x,y,f) by a high-pass filter HF(f) to obtain GR motion (x,y,f),GG motion (x,y,f),GB motion (x,y,f) (where x min ≦x≦x max ,y min ≦y≦y max ,0≦|f|≦f max ) is obtained and output (Equation (13)). GR motion (x,y,f)=FR(x,y,f)HF(f) GG motion (x,y,f)=FG(x,y,f)HF(f) (13) GB motion (x,y,f)=FB(x,y,f)HF(f) Here, the high-pass filter HF(f) is expressed by equation (14).

number

[0056] GR motion (x,y,f),GG motion (x,y,f),GB motion (x, y, f) is input to the time domain transform unit 517'. The time domain transform unit 517' transforms the GR motion (x,y,f),GG motion (x,y,f),GB motion Transform (x,y,f) into the time domain and extract the high-frequency components R motion (x,y,t),G motion (x,y,t),B motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained and output. Below, we show an example using the inverse Fourier transform (equation (16)).

number

[0057] The luminance motion component extraction unit 512′ extracts R motion (x,y,t),G motion (x,y,t),B motion (x,y,t) are weighted and added to obtain the luminance component Y original (x,y,t) (where xmin ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained and output (step S512') (equation (17)). Y motion (x,y,t)=α R R motion (x,y,t)+α G G motion (x,y,t)+α B B motion (x,y,t) (17) However, α R ,α G ,α B is a weighting coefficient (constant). motion (x, y, t) is an example of a "luminance motion component whose absolute value of the temporal frequency is equal to or greater than a second value."

[0058] The subsequent processing is the same as in the fifth embodiment. static This gives the user the illusion that video M2 is moving at a higher temporal frequency than

[0059] [Modification 2 of the fifth embodiment] In the fifth embodiment and its first modification, an image M consisting of low-time frequency components of a moving image M1 is static The video M2 was generated by adding luminance motion components extracted from the high-frequency components of the video M1 to the video M1 (FIG. 13A). However, the image M2, which is determined by the “color difference components” extracted from the low-frequency components of the video M1, static Alternatively, a moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to the moving image M2 (FIG. 13B). Alternatively, an image M2 may be generated by adding a luminance motion component extracted from the low-time frequency component of the moving image M1 to the moving image M2 (FIG. 13B). static A moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to the moving image M2 (FIG. 14A). Such a moving image M2 can also produce a dynamic illusion. The moving image M1 may also be a grayscale moving image.

[0060] [Sixth embodiment] When the video M1 includes a periodic or repetitive motion component (e.g., small movement), a still image of any frame (e.g., t=nth frame) included in the video M1 may be used as the "image" (FIG. 13A). That is, a video M2 may be generated by adding a luminance motion component to a still image of any frame extracted from the video M1. With such a video M2, the illusion that the still image appears to be moving can be created.

[0061] <Configuration> 15, the moving image component extraction device 61 of this embodiment has an image extraction unit 611 (first processing unit), a luminance moving component extraction unit 612 (second processing unit), and an output unit 513. The dynamic illusion presentation device 52 is the same as that of the fifth embodiment. The moving image component extraction device 61 is configured, for example, by loading a predetermined program into the computer described above.

[0062] <Processing> The process of this embodiment will be described with reference to Fig. 16. The difference from the fifth embodiment is that step S611 is executed instead of step S511, and step S6123 is executed instead of step S5123. Only steps S611 and S6123, which are the differences, will be described below.

[0063] <Step S611> The moving image M1 (equation (1)) is input to an image extraction unit 611 and a luminance motion component extraction unit 612. The image extraction unit 611 extracts a still image of the t=nth frame from the moving image M1 and designates it as an image M static That is, R static (x,y)=R(x,y,n),G static (x,y)=G(x,y,n),B static (x,y)=B(x,y,n)(x min ≦x≦x max ,y min ≦y≦y max ) is a two-dimensional matrix {R static (x,y)},{G static (x,y)},{B static (x,y)}static (Equation (3)) is output. static Image M is also an example of an image whose absolute value of the temporal frequency is equal to or less than the first value when the first value is set to 0. static It is desirable for the image to include spatial frequency components whose absolute values ​​are greater than zero, and also to include chromatic colors (step S611).

[0064] <Step S6123> The luminance motion component extraction unit 612 extracts R static (x,y),G static (x,y),B static (x,y) (where x min ≦x≦x max ,y min ≦y≦y max ) is the luminance component corresponding to the luminance stationary component Y static (x,y). Luminance stationary component Y static (x, y) is R as in step S5122. static (x,y),G static (x,y),B static The luminance motion component extraction unit 612 extracts the luminance component Y original From (x,y,t), the luminance stationary component Y static By subtracting (x,y), the luminance motion component Y motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is obtained and output (step S6123) (equation (5)). motion (x, y, t) is the luminance component Y original From (x, y, t), the luminance still component Y obtained from a still image based on the video corresponding to the object is static Includes the luminance motion component obtained by subtracting (x,y).

[0065] In this embodiment, the luminance motion component creates the illusion that a single still image in any frame is visually moving, providing the user with a visual experience that compares favorably with an original moving image.

[0066] [Modification of the sixth embodiment] In this embodiment, a still image M of an arbitrary frame of a moving image M1 static In addition, we generated a video M2 by adding luminance motion components extracted from the high-frequency components of the video M1 (FIG. 13A). However, when a still image M static A moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency component of the moving image M1 to an image of a color component determined from a "color difference component" extracted from the moving image M2 (FIG. 13B). static A moving image M2 may be generated by adding a luminance motion component extracted from the high-time frequency components of the moving image M1 to an image composed of a "luminance component" extracted from the high-time frequency components of the moving image M1 (FIG. 14A). Such a moving image M2 can also produce a dynamic illusion. The moving image M1 may also be a grayscale moving image.

[0067] [Seventh embodiment] As described above, due to the motion visual characteristics of the visual system, even if the spatial resolution or contrast of the "luminance motion component" to be synthesized into the "image" is reduced, the perceived quality of the moving image is maintained. In this embodiment, the "low-temporal frequency component" of the moving image M1 is the "image", and the component in which at least one of the high spatial frequency component or contrast of the luminance component (luminance motion component image included in the moving image M1) extracted from the "high-temporal frequency component" of the moving image M1 is reduced is the "luminance motion component". For example, filtering is performed on the luminance motion component image to reduce at least one of the high spatial frequency component or contrast. After that, a calculation is performed to integrate (add) the "luminance motion component" into the "image", and the moving image M2 obtained by this is displayed (Fig. 14B).

[0068] <Configuration> 15, the moving image component extraction device 71 of this embodiment has a low-temporal frequency component extraction unit 511, a luminance motion component extraction unit 512, a filtering unit 719, and an output unit 513. The dynamic illusion presentation device 52 of this embodiment is the same as that of the fifth embodiment. The moving image component extraction device 71 is configured, for example, by loading a predetermined program into the computer described above.

[0069] <Processing> The process of this embodiment will be described with reference to FIG. 16. The difference between this embodiment and the fifth embodiment is that the luminance motion component Y motion The step is to filter (x, y, t) (luminance motion component image) and add the filtered luminance motion component to the “image” to obtain a moving image M2. motion Only the filtering process for (x, y, t) will be explained.

[0070] The luminance motion component Y obtained in step S5123 motion (x,y,t) (where x min ≦x≦x max ,y min ≦y≦y max ,t min ≦t≦t max ) is input to the filtering unit 719. The filtering unit 719 first filters Y motion Transform (x,y,t) into the time-space frequency domain and motion (ξ,η,τ) is obtained, where ξ,η,τ represent the horizontal spatial frequency, vertical spatial frequency, and time frequency, respectively. The lower and upper limits of the horizontal spatial frequency are min and ξ max (ξ min <ξ max ) and the lower and upper limits of the vertical spatial frequency are η min and η max (η min <η max ) and the lower and upper limits of the time frequency are τ min and τ max (τ min <τ max ) ξ,η,τ aremin ≦ξ≦ξ max ,η min ≦η≦η max , τ min ≦τ≦τ max Below is an example using the Fourier transform (equation (18)). motion (ξ,η,τ) is Y motion This is the Fourier spectrum of (x,y,t).

number

[0071] Next, the filtering unit 719 motion (ξ,η,τ) is multiplied by the filter G(ξ,η,τ) and then subjected to an inverse Fourier transform to obtain the luminance motion component gY motion Obtain (x,y,t) (Equation 19).

number

[0072] G(ξ,η,τ) is a filter for reducing high spatial frequency components or contrast. A filter for reducing high spatial frequency components is a low-pass filter, and a filter for reducing contrast (overall contrast) is, for example, a function for linearly converting gray levels or a function for flattening a histogram (spatiotemporal frequency filter). A specific example of a low-pass filter is shown below (Equation (20)).

number

[0073] The subsequent processing is the same as that of the fifth embodiment, except that the luminance motion component Y motion (x,y,t) is the luminance motion component gY motion (x, y, t). As mentioned above, even if the high spatial frequency of the luminance motion component video is reduced or the contrast is reduced by filtering, the perceptual quality of the video is not affected within the range allowed by the characteristics of the visual system. Therefore, even if the amount of information of the luminance motion component is reduced by filtering, the user can be provided with a visual experience that is comparable to that of the original video M1.

[0074] [Modification of the Seventh Embodiment] The seventh embodiment is a modification of the fifth embodiment, except for the luminance motion component Y motion (x,y,t) is the luminance motion component gY motion However, the luminance motion component Y motion (x,y,t) is the luminance motion component gY motion (x, y, t). In this case, steps S7191 and S7192 are executed after step S512′ in FIG. 17. Similarly, in the second modification of the fifth embodiment or the sixth embodiment, the luminance motion component Y motion(x,y,t) is the luminance motion component gY motion It may also be replaced with (x,y,t).

[0075] [Eighth embodiment] In the eighth embodiment, the aforementioned "image with a temporal frequency absolute value of a first value or less" appears on the surface of the "object," and a "luminance motion component with a temporal frequency absolute value of a second value or more" is superimposed on this "image." This also creates the illusion that the "image" is moving. Below, an example is described in which a still "image" is printed on the "object," and a "luminance motion component with a temporal frequency absolute value of a second value or more" is projected onto the "image."

[0076] <Configuration> 19, the moving image component extraction device 81 of this embodiment is obtained by replacing the output unit 513 of any of the moving image component extraction devices 51, 51', 61, 71 of the fifth to seventh embodiments or their modified examples with an output unit 813 and a printing unit 814. The dynamic illusion presentation device 82 (illusion presentation device) of this embodiment has an input unit 821 and a projection unit 823 (projector). Also, the object 83 exemplified in this embodiment is a two-dimensional medium such as paper.

[0077] <Processing> A still image M extracted from a video M1 as in any of the above-described embodiments or variations static (Equation (3)) is input to the printing unit 814 (output unit). The printing unit 814 prints the image M static is printed on the surface of the object 83. The luminance motion component Y motion (x,y,t) or gY motion (x, y, t) is sent to the dynamic illusion presentation device 82 and input to the input unit 821. The luminance motion component Y motion (x,y,t) or gY motion(x, y, t) is sent to the projection unit 823, which projects the luminance motion component Y motion (x,y,t) or gY motion (x,y,t) is the image M printed on the object 83 static Then, the video M2 is projected onto the screen to display the video (equation (21)).

number

[0078] This also creates a dynamic illusion. motion (x,y,t) or gY motion When (x,y,t) are projected by a projector, they cannot have negative values. Therefore, the overall luminance of the video M2 increases. Furthermore, the increase in luminance due to the projection is partly multiplicative. Therefore, the luminance distribution of the video M2 is quite different from that of the original video M1. Nevertheless, the user can visually experience the movement of the original video M1 from the video M2. This is thought to be related to adaptive luminance contrast normalization by the visual system, as shown in Reference 9. In this way, this embodiment can provide the user with a visual experience that is comparable to that of the original video M1.

[0079] [Modification of the eighth embodiment] In the eighth embodiment, a still image M static was printed on a "target" such as paper. However, instead of printing, it may be projected onto a "target" such as a screen using another projector, or displayed on an "target" such as electronic paper. In addition, the "motion component video" may be projected onto a transparent display as shown in Reference 9. In addition, the image M static When projecting or displaying an image, static It is not necessary for image M to be a still image, but it may be an image that is moving slowly. In this case, image M staticIt is possible to give the user a visual experience that seems to move at a higher temporal frequency than that of the moving image M1. static Instead of extracting the luminance motion component Y motion (x,y,t) or gY motion (x,y,t), or obtain the luminance motion component Y from a still image obtained by photographing the "object". motion (x,y,t) or gY motion (x, y, t) may be generated. The luminance motion component Y motion (x,y,t) or gY motion By projecting (x,y,t) onto an image appearing on the surface of an "object," it is possible to create the illusion that the "image" is moving.

[0080] [Other modifications, etc.] The present invention is not limited to the above-described embodiment and its modified examples. For example, the moving image component extraction device and the dynamic illusion presentation device may be the same device. Alternatively, the processing of each unit included in the moving image component extraction device and the dynamic illusion presentation device may be performed by different devices.

[0081] The above-mentioned various processes may be executed not only in chronological order as described, but also in parallel or individually depending on the processing capacity of the device executing the processes or on the necessity. Needless to say, other appropriate modifications are possible without departing from the spirit of the present invention.

[0082] When the above-mentioned configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above-mentioned processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such recording media include a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, etc.

[0083] This program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0084] A computer that executes such a program, for example, first stores in its own storage device the program recorded on a portable recording medium or the program transferred from a server computer. When executing a process, the computer reads the program stored in its own recording device and executes a process according to the read program. As another execution form of this program, the computer may read the program directly from the portable recording medium and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to this computer. The above-mentioned process may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result, without transferring a program from the server computer to this computer.

[0085] A data structure may be distributed that associates "first image data that records a first image obtained by shooting" with "second image data that records a second image having transparency, which is a narrowband image in which the spatial frequency band of the entire image is narrower than the spatial frequency band of the first image while retaining edge information contained in the first image, and which is displayed superimposed on the first image such that edges contained in the narrowband image overlap edges of the first image when displayed." This distribution may be performed by distributing the data structure via the Internet, etc., or by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the data structure is recorded.

[0086] Also, a data structure including first data representing "an image including a spatial frequency component with an absolute value greater than zero and an absolute value of a temporal frequency equal to or less than a first value" output from the moving image component extraction device and second data representing "a luminance motion component with an absolute value of a temporal frequency equal to or greater than a second value corresponding to the image" may be distributed. This distribution may be performed by distributing the data structure via the Internet, etc., or by selling, transferring, lending, etc. a portable recording medium such as a DVD or CD-ROM on which the data structure is recorded. A device provided with this data structure inputs the "first data" and the "second data" to a calculation unit, and the calculation unit performs a calculation to add the "luminance motion component" to the "image" to obtain a moving image, and displays this moving image from a display unit. Alternatively, the first data is input to an output unit, the output unit displays the "image" on the surface of the "object", the second data is input to a projection unit, and the projection unit projects the "luminance motion component" onto the "image" displayed on the surface of the "object". This allows a moving image that creates a dynamic illusion to be displayed on the display unit or the object. Also, in order to make the "object" appear to be in motion, a data structure of an image used in a device that superimposes an image on the "object" may be distributed. Here, the absolute value of the time frequency of the luminance motion component in a plurality of frames in the image is greater than the absolute value of the time frequency in a plurality of frames in the image corresponding to the "object". Alternatively, this data structure may include a luminance motion component obtained by subtracting a luminance still component obtained from a still image based on the image corresponding to the "object" from the luminance component of each of a plurality of frames in the image corresponding to the "object". [Industrial Applicability]

[0087] The present invention can be used in a variety of fields, including (1) advertising, such as adding the impression of movement to paper media or signs by projecting light, (2) interior design, such as creating an illusion of transformation in interior patterns on floors, walls, etc., and (3) art, toys, and entertainment, such as adding movement to character illustrations and combining them with conventional projection mapping technology.

Claims

1. 1. An apparatus for superimposing an image on an object so as to cause a user to perceive the object as moving and / or deforming, comprising: the image includes a luminance motion component corresponding to a motion and / or deformation of the object; An apparatus in which the image of the object contains image information that is subject to an illusion caused by the luminance motion component, i.e., image information that is non-uniform and contains spatial frequency components whose absolute values ​​are greater than zero.

2. 1. A device for superimposing an image on an object image to give an observer the illusion that the image of the object itself is moving and / or deforming, comprising: The image includes a luminance motion component that causes the illusion, An apparatus in which the image of the object contains image information that is subject to an illusion caused by the luminance motion component, i.e., image information that is non-uniform and contains spatial frequency components whose absolute values ​​are greater than zero.

3. A method for superimposing an image on an object image in order to give an observer an illusion that the object image itself is moving and / or deforming, comprising the steps of: The image includes a luminance motion component that causes the illusion, A method in which the image of the object contains image information that is subject to the illusion caused by the luminance motion component, i.e., image information that is non-uniform and contains spatial frequency components whose absolute value is greater than zero.

4. A method for superimposing an image on an object image in order to give an observer an illusion that the object image itself is moving and / or deforming, comprising the steps of: The image includes a luminance motion component that causes the illusion, A method in which the image of the object contains image information that is subject to the illusion caused by the luminance motion component, i.e., image information that is non-uniform and contains spatial frequency components whose absolute value is greater than zero.

5. A program for causing a computer to function as the device according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Method for simulating motion of three-dimensional physical object stationary in changeless scene

    JP2003099799A

  • Image processing device, method and program

    JP2014059691A

  • Simulating motion of static objects in scenes

    US20030043152A1

Cited By

  • Semiconductor device and method of manufacturing the same

    CN121712072A