Content generation method, content projection method, program, and content generation system

Abstract

To provide a content generation method capable of more easily generating image content, a content projection method, a program, and a content generation system.SOLUTION: The content generation method generates an image content D1 to be projected to a projection position P1 in a real space A1 via a projector 30. In a series of first acquisition processing, spatial information about virtual space A2 corresponding to the real space A1 is acquired. In a series of second acquisition processing, display location information regarding a display position P3 corresponding to the projection position P1 in the virtual space A2 is acquired. In a series of third acquisition processing, a piece of reference location information regarding the reference position P2 in the virtual space A2 is acquired. In a series of generation processing, an image containing the object Ob1 as seen from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2 is generated as an image content D1 based on the spatial information and the reference location information and display location information.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure generally relates to a content generation method, a content projection method, a program, and a content generation system. More particularly, the present disclosure relates to a content generation method, a content projection method, a program, and a content generation system that generate image content to be projected by a projector at a projection position in real space. [Background technology]

[0002] Patent Document 1 discloses an image display system that projects an image into a display area of ​​projection mapping using a projector means. In the image display system described in Patent Document 1, the projector means projects one image into an image display area, and projects the image into the image display area in accordance with the movement of the image display area.

[0003] That is, in Patent Document 1, the image display area can be moved, rotated, and turned in any direction, and the projector (projector means) can also be moved and displaced in any direction. If the image display area moves, the image display system detects the movement of the image display area and moves the projector so as to follow the movement of the image display area, thereby changing the projection position where the image is projected in real space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-81497 Summary of the Invention [Problem to be solved by the invention]

[0005] For example, since the projected image becomes larger as the projection distance increases, in order to keep the size of objects in the projected image constant, it is necessary to determine the size of objects in the image content in accordance with the relative positional relationship between the projector and the projection position (image display area). In particular, when the projection position changes as described above, the relative positional relationship between the projector and the projection position also changes, so it takes time and effort to generate image content that matches the positional relationship between the two.

[0006] The present disclosure has been made in view of the above-mentioned circumstances, and aims to provide a content generation method, a content projection method, a program, and a content generation system that enable image content to be generated more easily. [Means for solving the problem]

[0007] A content generation method according to one aspect of the present disclosure is a method for generating image content to be projected by a projector at a projection position in real space, and includes a first acquisition process, a second acquisition process, a third acquisition process, and a generation process. The first acquisition process acquires spatial information regarding a virtual space corresponding to the real space. The second acquisition process acquires display position information regarding a display position in the virtual space corresponding to the projection position. The third acquisition process acquires reference position information regarding a reference position in the virtual space. The generation process generates, as the image content, an image including the object as viewed from the reference position when the object is displayed at the display position in the virtual space, based on the spatial information, the reference position information, and the display position information.

[0008] A content projection method according to an aspect of the present disclosure includes a projection process of projecting the image content generated by the content generation method at the projection position in the real space using the projector.

[0009] A program according to one aspect of the present disclosure is a program for causing one or more processors to execute the content generation method.

[0010] A content generation system according to one aspect of the present disclosure is a system for generating image content to be projected by a projector at a projection position in real space, and includes a first acquisition unit, a second acquisition unit, a third acquisition unit, and a generation unit. The first acquisition unit acquires spatial information regarding a virtual space corresponding to the real space. The second acquisition unit acquires display position information regarding a display position in the virtual space corresponding to the projection position. The third acquisition unit acquires reference position information regarding a reference position in the virtual space. Based on the spatial information, the reference position information, and the display position information, the generation unit generates, as the image content, an image including the object as viewed from the reference position when the object is displayed at the display position in the virtual space. [Effects of the Invention]

[0011] The present disclosure has the advantage of making it easier to generate image content. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic configuration diagram showing an image projection system according to the first embodiment. [Figure 2] FIG. 2 is a schematic block diagram of the image projection system. [Figure 3] Fig. 3A is an explanatory diagram of a virtual space formed by the content generation method according to embodiment 1. Fig. 3B is an explanatory diagram of a state in which an object is displayed in the virtual space by the content generation method according to embodiment 1. Fig. 3C is an explanatory diagram of a state in which a virtual camera is placed in the virtual space by the content generation method according to embodiment 1. [Figure 4] FIG. 4 is an explanatory diagram showing how image content is generated by the content generation method and projected by a projector. [Figure 5] FIG. 5 is a flowchart showing an example of the content generating method. [Figure 6] FIG. 6 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 7]FIG. 7 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 8] FIG. 8 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 9] FIG. 9 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 10] FIG. 10 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 11] FIG. 11 is an explanatory diagram showing an example of an input support tool screen provided by the content generation method. [Figure 12] FIG. 12 is an explanatory diagram of a virtual space formed by the content generation method according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Embodiment 1) (1) Overview An outline of a content generation method, a content projection method, a program, and a content generation system 10 according to this embodiment will be described below with reference to FIG.

[0014] The content generation method according to this embodiment is a method for generating image content D1 to be projected at a projection position P1 in a real space A1 by a projector 30. The projector 30 projects a projection image Im1 at the projection position P1 on the projection surface A11 using the image content D1 by irradiating light onto a projection surface A11, such as a wall, floor, or ceiling, that exists in the real space A1.

[0015] In the present disclosure, "image content" refers to the content of information (data) that forms the image projected by the projector 30 as the projection image Im1. In other words, by inputting the image content D1 to the projector 30, the projector 30 can project the projection image Im1 using the image content D1. In other words, the content of the projection image Im1 is determined by the image content D1. The projection image Im1 may be processed by, for example, trimming, adjusting brightness, or adjusting contrast, relative to the image content D1.

[0016] Here, when the projector 30 projects the projection image Im1, the size and shape of the projection image Im1 change depending on the relative positional relationship between the projector 30 and the projection position P1. Therefore, for example, to maintain a constant size and shape of the object Ob1 in the projection image Im1, it is necessary to determine the size and shape of the object Ob1 in the image content D1 in accordance with the relative positional relationship between the projector 30 and the projection position P1. As an example, if the throw ratio is kept constant and the projection distance from the projector 30 to the projection position P1 increases, the projection image Im1 becomes larger. In this case, to maintain a constant size of the object Ob1 in the projection image Im1, it is necessary to reduce the size of the projection image Im1 itself in order to reduce the size of the object Ob1 in the image content D1.

[0017] As described above, when generating image content D1 to be projected by projector 30, the relative positional relationship between projector 30 and projection position P1 must be taken into consideration, which makes the process more time-consuming than generating image content to be displayed on a non-projection display. In particular, for example, if projection position P1 changes from time to time in real space A1, the relative positional relationship between projector 30 and projection position P1 changes, so generating image content D1 that takes the positional relationship between the two into consideration is extremely time-consuming. The content generation method according to this embodiment makes it easier to generate image content D1 that corresponds to the relative positional relationship between projector 30 and projection position P1 by the following aspects.

[0018] That is, the content generation method according to this embodiment is a method for generating image content D1 to be projected by projector 30 at projection position P1 in real space A1, and includes a first acquisition process, a second acquisition process, a third acquisition process, and a generation process. In the first acquisition process, spatial information is acquired. The spatial information is information about a virtual space A2 (see FIG. 3A) that corresponds to real space A1. In the second acquisition process, display position information is acquired. The display position information is related to a display position P3 (see FIG. 3B) that corresponds to projection position P1 in virtual space A2. In the third acquisition process, reference position information is acquired. The reference position information is information about a reference position P2 (see FIG. 3C) in virtual space A2. In the generation process, an image including object Ob1 viewed from reference position P2 when object Ob1 is displayed at display position P3 in virtual space A2 is generated as image content D1 based on the spatial information, reference position information, and display position information.

[0019] According to this aspect, in a virtual space A2 corresponding to the real space A1, an image including the object Ob1 as viewed from a reference position P2 when the object Ob1 is displayed at a display position P3 corresponding to the projection position P1 in the real space A1 is generated as the image content D1. That is, in the virtual space A2, the object Ob1 can be virtually displayed in any size and shape at any display position P3. For example, when the image content D1 is projected at the projection position P1 by a projector 30 installed at a position corresponding to the reference position P2 in the real space A1, the projected object Ob1 substantially coincides with the object Ob1 virtually displayed at the display position P3. Therefore, by displaying the object Ob1 to be projected at the projection position P1 in the real space A1 at the display position P3 in the virtual space A2, the image content D1 can be generated without considering the relative positional relationship between the projector 30 and the projection position P1. As a result, it becomes easier to generate the image content D1 according to the relative positional relationship between the projector 30 and the projection position P1, which is advantageous in that it becomes possible to generate the image content D1 more easily.

[0020] Moreover, the content generation method according to this embodiment is executed, for example, by a content generation system 10 as shown in FIG. 2. In other words, the content generation system 10 is one aspect for realizing the content generation method described above. The content generation system 10 according to this embodiment is a system for generating image content D1 to be projected by a projector 30 at a projection position P1 in a real space A1, and includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, and a generation unit 14. The first acquisition unit 11 acquires spatial information. The spatial information is information about a virtual space A2 corresponding to the real space A1. The second acquisition unit 12 acquires display position information. The display position information is related to a display position P3 in the virtual space A2 corresponding to the projection position P1. The third acquisition unit 13 acquires reference position information. The reference position information is information about a reference position P2 in the virtual space A2. The generation unit 14 generates, as image content D1, an image including the object Ob1 as seen from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2, based on the space information, the reference position information, and the display position information.

[0021] Here, the content generation system 10 according to this embodiment is mainly composed of a computer system having one or more processors and one or more memories. In other words, the content generation method according to this embodiment is used on a computer system (content generation system 10). In other words, the content generation method can also be embodied as a program. The program according to this embodiment is a program for causing one or more processors to execute the content generation method according to this embodiment.

[0022] The content projection method according to this embodiment is a method for projecting a projection image Im1 using image content D1 generated by the content generation method described above. That is, the content projection method includes a projection process. In the projection process, the image content D1 generated by the content generation method according to this embodiment is projected by a projector 30 onto a projection position P1 in a real space A1.

[0023] (2) Premise In the present disclosure, "image" includes a moving image (moving image) and a still image (still image). Furthermore, "moving image" includes an image composed of a plurality of still images obtained by stop-motion photography or the like. Furthermore, "image" includes a monochrome image and a color image. In this embodiment, as an example, the "image" is an image that changes over time (i.e., a moving image) and is a full-color image. In other words, in this embodiment, as an example, the projection image Im1 projected by the projector 30 is a full-color moving image, and the image content D1, which is the content of the information forming the projection image Im1, is also full-color moving image data.

[0024] In this disclosure, "real space" refers to an actual space, i.e., a space that actually exists, and includes the interior space (indoor space) of an existing facility (building) and outdoor space. Facilities here include, for example, non-residential facilities such as commercial facilities, theme parks, office buildings, schools, welfare facilities, hospitals, and factories, as well as facilities such as apartment buildings and detached houses. Non-residential facilities also include theaters, movie theaters, public halls, amusement parks, complexes, restaurants, department stores, hotels, inns, kindergartens, libraries, museums, art galleries, underground shopping malls, stations, and airports. Furthermore, in this disclosure, "facility" includes not only buildings (structures) but also outdoor facilities such as baseball stadiums, parking lots, sports fields, and parks. For example, in the interior space of a facility consisting of a structure, the surface of the facility's structure, such as a wall, floor, or ceiling (such as a wall, floor, or ceiling), can serve as the projection surface A11 for projecting the projection image Im1. Furthermore, the surfaces of equipment (including fixtures, furniture, equipment, and devices) installed inside the facility, such as doors, partitions, shelves, desks, chairs, home appliances, or screens, can serve as the projection surface A11 for projecting the projection image Im1. Outdoors, the surfaces of objects such as the exterior walls, roofs, or pillars of a building-like facility, or the ground, rocks, or trees, can serve as the projection surface A11 for projecting the projection image Im1. In this embodiment, as an example, the real space A1 is an indoor space (indoor space) of a room in a non-residential facility such as a commercial facility, and the wall, floor, or ceiling surface of this room serves as the projection surface A11.

[0025] In the present disclosure, the term "virtual space" refers to a non-existent virtual space; it does not exist in the real space (real space A1) and does not have any physical entity. Therefore, the virtual space A2 is, for example, a virtual space composed of data processable by one or more processors and reproduced by a computer system. Here, the virtual space A2 is a space equivalent to the real space A1 onto which the projection image Im1 is projected. In other words, the virtual space A2 is a space that mimics the real space A1, in which, for example, structures such as walls, floors, and ceilings are laid out in the same manner as in the real space A1. Therefore, the virtual space A2 includes a virtual projection surface A21 (see FIG. 3A ) that corresponds to the projection surface A11 onto which the projection image Im1 is projected in the real space A1. However, the virtual space A2 only needs to mimic the real space A1 in terms of at least the layout of the surface of the structure, including the projection surface A11, but it does not have to mimic the real space A1 in terms of the interior and backside of the structure, the surface condition of the structure, the layout of equipment such as lighting fixtures, etc. In this embodiment, as an example, the virtual space A2 is a three-dimensional space represented by an XYZ Cartesian coordinate system having three axes, an X axis, a Y axis, and a Z axis, which are orthogonal to each other.

[0026] Furthermore, the term "position" in the present disclosure may have a certain size and shape, or may be a "point" with no size. Therefore, for example, the projection position P1 may be defined as an area with a certain size and shape in the virtual space A2, or as a "point" on coordinates with no size. In this embodiment, as an example, the projection position P1 is assumed to be an area with a certain size (area) on the projection surface A11. Similarly, the display position P3 corresponding to the projection position P1 is assumed to be an area with a certain size (area) in the virtual space A2. On the other hand, the reference position P2 is assumed to be a "point" with no size. Furthermore, the installation position P4, which will be described later, represents the position where the projector 30 is installed in the real space A1, and is assumed to be a "point" with no size. In other words, the installation position P4 is the position of a point (for example, the center point of the mirror unit 32, which will be described later) determined relative to the projector 30. Furthermore, the reference position P2 and the display position P3 in the virtual space A2 are merely defined as existing virtually within the virtual space A2, and do not actually exist in the real space (real space A1) and are therefore not substantial.

[0027] In the present disclosure, an "object" is made up of data that can be processed by one or more processors, and does not actually exist in real space (actual space), nor does it have any physical substance. Object Ob1 is a virtual model that resembles some kind of object (including a living thing), symbol, number, letter, etc., and may be either a two-dimensional model or a three-dimensional model. In this embodiment, as an example, it is assumed that object Ob1 is a two-dimensional model that resembles a "butterfly," as shown in FIG. 1.

[0028] (3) Image projection support system The configuration of an image projection system 100 including a content generation system 10 according to this embodiment will be described below with reference to FIGS.

[0029] (3.1) Overall structure The content generation system 10 according to this embodiment, together with a projector 30 that actually projects the generated image content D1, constitutes an image projection system 100. That is, the image projection system 100 according to this embodiment includes the content generation system 10 and the projector 30. The image projection system 100 can handle all of the processes required to project a projection image Im1 at a projection position P1 in a real space A1, from generating the image content D1 to projecting the generated image content D1.

[0030] As described above, the content generation system 10 is mainly composed of a computer system having one or more processors and one or more memories. In this embodiment, as an example, the content generation system 10 is realized by a single information terminal 1 made up of a personal computer. That is, dedicated application software (program) is installed in the information terminal 1, and by starting this application software, the information terminal 1 functions as the content generation system 10, and the content generation method is realized.

[0031] The content generation system 10 is also connected to a display device 40 and an input device 50. The content generation system 10 displays a screen on the display device 40 to present information to the user, and receives user operations via the input device 50.

[0032] In other words, the content generation method according to this embodiment is used on a graphical user interface (GUI) of a computer system. That is, the content generation method is a method for displaying a screen such as an input support tool screen T1 (see FIG. 1 ) on an information terminal 1 (content generation system 10) whose main component is a computer system. In the present disclosure, a "screen" such as the input support tool screen T1 refers to an image (including text, graphs, icons, etc.) displayed on the display device 40. Therefore, according to the content generation method according to this embodiment, a user can generate image content D1 on a graphical user interface.

[0033] Furthermore, in the image projection system 100 according to this embodiment, the content generation system 10 is configured to be able to communicate with the projector 30. In the present disclosure, "able to communicate" means being able to exchange information directly or indirectly via a network, a repeater, or the like, using an appropriate communication method such as wired or wireless communication. In other words, the content generation system 10 and the projector 30 are able to exchange information with each other. In this embodiment, at least the content generation system 10 is able to transmit image content D1, control information D2, and the like to the projector 30. This allows the image projection system 100 to transmit image content D1 generated by the content generation system 10 to the projector 30, and to have the projector 30 project the image content D1.

[0034] In the present embodiment, for example, the content generation system 10 and the projector 30 are wired together via a video cable 101, a control cable 102, and a communication cable 103. The video cable 101 conforms to a communication standard such as HDMI (registered trademark), for example, and is used to transmit image content D1 from the content generation system 10 to the projector 30. The control cable 102 conforms to a communication standard such as LAN (Local Area Network), for example, and is used to transmit control information D2 from the content generation system 10 to the projector 30. The communication cable 103 conforms to a communication standard such as LAN, for example, and is used to exchange various types of information between the content generation system 10 and the projector 30. The control cable 102 and the communication cable 103 may be a single LAN cable, or may be used separately to control the mirror unit 32 (described later) and the main body of the projector 30, respectively.

[0035] The projector 30 projects the image content D1 generated by the content generation system 10 onto a projection position P1 in the real space A1. Here, the projector 30 is installed at an installation position P4 in the real space A1. In the present embodiment, the installation position P4 is a single point determined relative to the projector 30 as described above, and is, for example, the position of the center point of the mirror unit 32 described below. That is, the projector 30 is installed in the real space A1 so that the center point of the mirror unit 32 is located at the installation position P4. The projector 30 projects a projection image Im1 onto the projection position P1 on the projection surface A11 using the image content D1 by irradiating light onto a projection surface A11 such as a wall, floor, or ceiling surface present in the real space A1.

[0036] In this embodiment, the projector 30 projects in real time the image content D1 transmitted (distributed) from the content generation system 10. In other words, the content generation system 10 functions as a video playback device that plays back the image content D1 and outputs (transmits) a video signal. That is, in the image projection system 100, the content generation system 10 generates and plays back the image content D1, and the projector 30 projects the image content D1.

[0037] The display device 40 is realized by, for example, a liquid crystal display or an organic EL (Electro Luminescence) display, etc. The display device 40 receives a video signal from the content creation system 10 and displays a "screen" such as the input support tool screen T1, thereby presenting information to the user of the content creation method.

[0038] The input device 50 is realized by, for example, a pointing device such as a keyboard or a mouse, a mechanical switch, a gesture sensor, or a voice input device. The input device 50 accepts a user operation (including a voice operation) of the content generation method and outputs an operation signal corresponding to the user operation to the content generation system 10.

[0039] At least one of the display device 40 and the input device 50 may be included as a component of the content generation system 10. In other words, the content generation system 10 may include at least one of the display device 40 and the input device 50. Furthermore, for example, the display device 40 and the input device 50 may be realized by a touch panel display, in which case the display device 40 and the input device 50 are integrated.

[0040] (3.2) Projector Next, the configuration of the projector 30 will be described in more detail.

[0041] In this embodiment, the projector 30 is a floor-standing type that is placed on the floor (including the ground) for use. Therefore, the position (installation position P4) and orientation of the projector 30 in the horizontal plane can be set arbitrarily depending on the position where the projector 30 is placed and the orientation of the projector 30 on the floor. Furthermore, the projector 30 has, for example, an adjuster function on the legs that support the main body, and by adjusting the height of the main body from the floor, the vertical height can also be adjusted. This makes it possible to arbitrarily set the installation position P4, which is the position of the center point of the surface (reflection surface) of the mirror section 32 of the projector 30.

[0042] As an example, the projector 30 is installed at a position that is a suitable distance from any wall surface in a real space A1 that is an indoor space of a facility, as shown in Fig. 1. The projector 30 projects a projection image Im1 at a projection position P1 on the projection surface A11 using the image content D1 by irradiating light onto a projection surface A11 such as a wall surface, floor surface, or ceiling surface that exists around the projector 30.

[0043] Here, the projector 30 is a movable projection system in which the projection position P1 is not fixed but is variable within the real space A1. In other words, the projection position P1 is movable within the real space A1. Here, the projector 30 moves the projection position P1 by changing the direction of light irradiation. As an example, when the projector 30 is installed in front of the projection surface A11 and changes the direction of light irradiation from the left end to the right end of the projection surface A11, the projection position P1 moves to the right on the projection surface A11.

[0044] In this embodiment, the projector 30 is a moving-mirror projection system that moves a mirror unit 32 located on the optical path of the emitted light to change the direction of light irradiation and thereby move a projection position P1. That is, as shown in FIG. 2 , the projector 30 includes a projection unit 31, a mirror unit 32, and a drive unit 33. The projection unit 31 emits light for projecting a projection image Im1 using image content D1. The mirror unit 32 reflects the light emitted from the projection unit 31. The drive unit 33 moves the projection position P1 by driving the mirror unit 32 to change the orientation of the mirror unit 32. In short, in this embodiment, the drive unit 33 oscillates the mirror unit 32, thereby changing the reflection direction of the light emitted from the projection unit 31 at the mirror unit 32 and changing the irradiation direction of the light from the projector 30 (light reflected by the mirror unit 32).

[0045] More specifically, as shown in FIG. 1 , the projection unit 31 has, as an example, a cylindrical optical unit 311 that protrudes upward from the top surface of the main body of the projector 30. The optical unit 311 includes a lens system including a plurality of lens elements. The projection unit 31 emits light upward from an opening on the top surface of the optical unit 311 along the optical axis of the optical unit 311. The light emitted from the projection unit 31 is light for projecting a projection image Im1, and when this light is irradiated onto a projection surface A11, an image is formed on the projection surface A11, and the projection image Im1 is projected. When image content D1 is input to the projection unit 31, the projection unit 31 emits an image corresponding to the image content D1 as light. In this embodiment, the optical axis of the light emitted from the projection unit 31 is along the vertical direction and intersects with the center of the surface (reflecting surface) of the mirror unit 32.

[0046] As an example, the mirror unit 32 is a plane mirror whose surface (reflecting surface) is polygonal (hexagonal) and flat, and is held so as to be able to swing by the drive unit 33. As described above, light from the projection unit 31 is incident from below onto the center point of the surface of the mirror unit 32, so the mirror unit 32 is basically held in a position with its surface facing diagonally downward. As a result, light emitted upward from the projection unit 31 is reflected laterally by the surface of the mirror unit 32 and irradiated toward a projection surface A11 such as a wall, floor, or ceiling surface around the projector 30.

[0047] The drive unit 33 holds the mirror unit 32 so that it can oscillate, and drives the mirror unit 32 to change the orientation of the mirror unit 32. The drive unit 33 is realized, for example, by an actuator including a motor or the like. When the drive unit 33 drives the mirror unit 32, the orientation of the mirror unit 32 changes and the projection position P1 moves within the real space A1. The drive unit 33 is controlled by control information D2 from the content generation system 10. In other words, the control information D2 transmitted from the content generation system 10 to the projector 30 is information used to control the drive unit 33, and is information for moving the projection position P1 within the real space A1.

[0048] In this embodiment, the driver 33 is capable of two types of operation: a "pan operation" that rotates the mirror unit 32 around a vertical axis passing through the center point of the surface of the mirror unit 32, and a "tilt operation" that rotates the mirror unit 32 around a horizontal axis passing through the center point of the surface of the mirror unit 32. The pan operation changes the direction of the normal to the center point of the surface of the mirror unit 32 along the horizontal direction, thereby changing the azimuth angle of the mirror unit 32. The tilt operation changes the direction of the normal to the center point of the surface of the mirror unit 32 along the vertical direction, thereby changing the elevation / depression angle of the mirror unit 32. In this embodiment, the driver 33 combines the pan operation and the tilt operation to freely change the orientation of the mirror unit 32 around the center point of the surface of the mirror unit 32. Therefore, the projection position P1 at which light reflected by the mirror unit 32 is irradiated can be moved in two dimensions, up, down, left, and right, on the projection surface A11, which is, for example, a wall surface.

[0049] (3.3) Content Generation System Next, the configuration of the content generation system 10 will be described in more detail.

[0050] As described above, the content generation system 10 includes the first acquisition unit 11, the third acquisition unit 13, the second acquisition unit 12, and the generation unit 14. As shown in FIG. 2 , the content generation system 10 according to this embodiment includes, in addition to the first acquisition unit 11, the second acquisition unit 12, the third acquisition unit 13, and the generation unit 14, a virtual space creation unit 15 and an object creation unit 16. The content generation system 10 according to this embodiment also includes a control information creation unit 17, a provision unit 18, an input unit 19, an output unit 20, a communication unit 21, and a data storage unit 22. That is, the content generation system 10 includes the first acquisition unit 11, the second acquisition unit 12, the third acquisition unit 13, the generation unit 14, the virtual space creation unit 15, the object creation unit 16, the control information creation unit 17, the provision unit 18, the input unit 19, the output unit 20, the communication unit 21, and the data storage unit 22.

[0051] In this embodiment, as described above, the content generation system 10 is mainly configured as a computer system having one or more processors and one or more memories. Therefore, the functions of the content generation system 10 (such as the first acquisition unit 11, second acquisition unit 12, third acquisition unit 13, and generation unit 14) other than the communication unit 21 and the data storage unit 22 are realized by the one or more processors executing programs.

[0052] As described above, the first acquisition unit 11 executes the first acquisition process to acquire spatial information. The spatial information is information about the virtual space A2 corresponding to the real space A1. In this embodiment, the first acquisition unit 11 acquires the spatial information from the input unit 19.

[0053] As described above, the second acquisition unit 12 executes the second acquisition process to acquire display position information. The display position information relates to a display position P3 that corresponds to the projection position P1 in the virtual space A2. In this embodiment, the second acquisition unit 12 acquires the display position information from the input unit 19.

[0054] As described above, the third acquisition unit 13 executes the third acquisition process to acquire reference position information. The reference position information is information about the reference position P2 in the virtual space A2. In this embodiment, the third acquisition unit 13 acquires the reference position information from the input unit 19.

[0055] As described above, the generation unit 14 executes a generation process for generating image content D1. In the generation process, an image including object Ob1 viewed from reference position P2 when object Ob1 is displayed at display position P3 in virtual space A2 is generated as image content D1 based on spatial information, reference position information, and display position information. That is, the generation unit 14 generates image content D1 based on spatial information acquired by the first acquisition unit 11, reference position information acquired by the third acquisition unit 13, and display position information acquired by the second acquisition unit 12. In this embodiment, the generation unit 14 generates image content D1 in a state in which object Ob1 acquired by the object acquisition unit 16 is virtually displayed at display position P3 in the virtual space A2 formed by the virtual space formation unit 15. The virtual display of object Ob1 at display position P3 is executed by the generation unit 14.

[0056] In this embodiment, as an example, the generation unit 14 virtually installs a virtual camera V30 (see FIG. 3C) at a reference position P2 in the virtual space A2. The "virtual camera" referred to in this disclosure is a virtual device that simulates the functions of a camera, and is merely defined as virtually existing in the virtual space A2; it does not actually exist in the real space (real space A1) and does not have a physical entity. In this embodiment, the virtual camera V30 simulates the functions of a video camera that can capture full-color video. Here, the virtual camera V30 is installed in the virtual space A2 at a position corresponding to the projector 30 installed in the real space A1.

[0057] In this embodiment, the reference position P2 represents the position where the virtual camera V30 is installed in the virtual space A2, and is a "point" with no size, similar to the installation position P4. In other words, the reference position P2 is a position in the virtual space A2 that corresponds to the installation position P4 in the real space A1. Here, the reference position P2 is a single point determined relative to the virtual camera V30, and as an example, is the position of the center point of a virtual mirror section V32, which will be described later. In other words, the virtual camera V30 is virtually installed in the virtual space A2 so that the center point of the virtual mirror section V32 is located at the reference position P2.

[0058] Like a typical camera, the virtual camera V30 has the function of capturing an image of a subject present within the field of view of the virtual camera V30 and outputting the image captured within the field of view as an image (image signal). Therefore, when an object Ob1 is virtually displayed at display position P3 in the virtual space A2 and the virtual camera V30 captures an image of the display position P3, an image including the object Ob1 as seen from the reference position P2 is output from the virtual camera V30. In other words, the image output at this time is an image including the object Ob1 as seen from the reference position P2 when the object Ob1 is displayed at display position P3 in the virtual space A2. Therefore, the generation unit 14 defines such a virtual camera V30 and generates the image content D1 by using the image captured by the virtual camera V30 as the image content D1.

[0059] The virtual space formation unit 15 executes a virtual space formation process to form the virtual space A2. The virtual space formation unit 15 forms the virtual space A2 based on the space information acquired by the first acquisition unit 11.

[0060] The object acquisition unit 16 executes an object acquisition process to acquire object information related to the object Ob1. In this embodiment, the object acquisition unit 16 acquires the object information from the input unit 19. The object Ob1 of the object information acquired by the object acquisition unit 16 is virtually displayed by the generation unit 14 at a display position P3 in the virtual space A2.

[0061] The control information generator 17 executes a control information generation process to generate control information D2. In the control information generation process, control information D2 for the projector 30 is generated to move the projection position P1 in the real space A1 in synchronization with the movement of the display position P3. The control information D2 generated by the control information generator 17 is output from the output unit 20.

[0062] The providing unit 18 executes a providing process to provide an input support tool. The input support tool is a tool for allowing the user to input display position information during the second acquisition process. The input support tool presents at least the display position P3 within the virtual space A2. In this embodiment, the input support tool is embodied in an input support tool screen T1. In other words, the input support tool is a tool used on the graphical user interface of a computer system and does not have a physical form.

[0063] The input unit 19 receives input of various information and signals from outside the content creation system 10. The input unit 19 receives input of at least an operation signal from the input device 50. This allows the content creation system 10 to indirectly receive user operations (including voice operations, etc.) of the content creation method via the input device 50. The input unit 19 may receive input of information and signals from outside via the communication unit 21.

[0064] The output unit 20 outputs at least the image content D1 generated by the generation unit 14. The output unit 20 also outputs the control information D2 generated by the control information generation unit 17. Furthermore, the output unit 20 outputs a video signal for displaying a screen such as the input support tool screen T1 on the display device 40. Here, the output manner of the output unit 20 includes, for example, display, communication (transmission), sound, printing (printout), and recording (writing) to a non-transitory recording medium. In the present embodiment, as an example, the output unit 20 outputs at least the image content D1 and the control information D2 by recording (writing) them to the data storage unit 22 and by communication (transmission) from the communication unit 21 to the projector 30.

[0065] The communication unit 21 has a function of communicating with the projector 30. In this embodiment, the communication unit 21 is connected to the projector 30 by a video cable 101, a control cable 102, and a communication cable 103 in a wired manner.

[0066] The data storage unit 22 stores image content D1, control information D2, etc. The data storage unit 22 also stores spatial information acquired by the first acquisition unit 11, reference position information acquired by the third acquisition unit 13, and display position information acquired by the second acquisition unit 12. Furthermore, the data storage unit 22 stores information necessary for calculations by the first acquisition unit 11, the third acquisition unit 13, the second acquisition unit 12, the generation unit 14, etc. The data storage unit 22 includes a rewritable non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0067] The operation of each part of the content generation system 10 will be explained in detail in the section "(4) Content Generation Method."

[0068] As described above, the content generation system 10 according to this embodiment has a function of generating image content D1 and a function as a video playback device that plays back the generated image content D1 and outputs (transmits) a video signal. However, this embodiment assumes that the generation of image content D1 in the content generation system 10 and the playback of image content D1 (output of a video signal) in the content generation system 10 are not performed simultaneously, but are performed under different circumstances.

[0069] Therefore, the content generation system 10 according to this embodiment has at least two operation modes: a generation mode and a playback mode. The generation mode is an operation mode for generating image content D1, and when operating in the generation mode, the content generation system 10 generates image content D1 and stores (stores) the generated image content D1 in the data storage unit 22. The playback mode is an operation mode for playing back image content D1, and when operating in the playback mode, the content generation system 10 plays back image content D1 stored in the data storage unit 22 and outputs (transmits) a video signal to the projector 30. The operation mode (generation mode, playback mode) may be switched manually by a user's operation on the input device 50, for example, or may be switched automatically.

[0070] (4) Content generation method The operation of the content generation system 10 according to this embodiment in the generation mode, that is, the content generation method according to this embodiment, will be described in detail below with reference to FIGS. 3A to 11. FIG.

[0071] (4.1) Overall First, the overall operation of the content generation system 10 according to this embodiment, that is, the entire content generation method, will be described with reference to Figs. 3A to 5. Figs. 3A to 3C are explanatory diagrams visualizing a virtual space A2 according to the content generation method. Fig. 4 is an explanatory diagram conceptually showing the processing flow in which image content D1 is generated by a virtual camera V30 in the virtual space A2 and the generated image content D1 is projected by a projector 30 in the real space A1. Fig. 5 is a flowchart showing an example of the entire content generation method according to this embodiment.

[0072] When generating image content D1, the content generation system 10 first executes a first acquisition process (S1 in FIG. 5) to acquire spatial information about the virtual space A2, using the first acquisition unit 11, to form a virtual space A2 as shown in FIG. 3A. The spatial information is used to form the virtual space A2 corresponding to the real space A1. For example, the spatial information is information specifying the layout of structures constituting the space, such as walls, floors, and ceilings, in the real space A1. In other words, the spatial information that copies the layout of these structures makes it possible to create a virtual space A2 that mimics the real space A1. In this embodiment, the virtual space A2 is a three-dimensional space represented by an XYZ Cartesian coordinate system, and the spatial information includes coordinate positions in the XYZ Cartesian coordinate system. Specifically, in the first acquisition process, parameters for modeling structures such as walls, floors, and ceilings that form the virtual space A2 are acquired as the spatial information using three-dimensional computer graphics (3DCG) software. Alternatively, in the first acquisition process, three-dimensional measurement results of the real space A1 or CAD data of the real space A1 may be acquired as the spatial information.

[0073] Next, the content generation system 10 uses the acquired spatial information to form a virtual space A2 as shown in FIG. 3A in the virtual space formation unit 15 (S2 in FIG. 5). The virtual space A2 is a non-existent virtual space, consisting of data processable by one or more processors, and is reproduced by a computer system. Therefore, the virtual space A2 is formed by inputting parameters as spatial information into three-dimensional computer graphics (3DCG) software. In the virtual space A2 thus formed, for example, the surfaces of structures such as walls, floors, and ceilings become the virtual projection surface A21, which corresponds to the projection surface A11 of the real space A1. At this stage (the stage in FIG. 3A), a virtual camera V30 has not been installed in the virtual space A2.

[0074] However, in virtual space A2, the same length (dimension) scale as in real space A1 is not required, so for example, in the case of a wall, the aspect ratio needs to be the same in real space A1 and virtual space A2, and the height dimension may be different in real space A1 and virtual space A2.

[0075] Next, the content generation system 10 executes a second acquisition process (S3 in FIG. 5) in which the second acquisition unit 12 acquires display position information related to the display position P3 in order to identify the display position P3 as shown in FIG. 3B. The display position information is used to identify the display position P3 corresponding to the projection position P1 in the virtual space A2, and is therefore, for example, information that identifies a position within a virtual projection plane A21 made up of the surfaces of structures such as walls, floors, and ceilings in the virtual space A2. In particular, since the display position P3 is an area having a certain size (area) within the virtual space A2, the display position information is information that represents a specific area (range) on the virtual projection plane A21. Furthermore, in this embodiment, since the virtual space A2 is a three-dimensional space represented by an XYZ Cartesian coordinate system, the display position information includes a coordinate position in the XYZ Cartesian coordinate system.

[0076] Next, in order to display the object Ob1 as shown in FIG. 3B, the content generation system 10 causes the object acquisition unit 16 to execute an object acquisition process to acquire object information about the object Ob1 (S4 in FIG. 5). As an example in this embodiment, the object Ob1 is a two-dimensional model that resembles a butterfly, and therefore the object acquisition unit 16 acquires data for reproducing such object Ob1 as object information. Specifically, in the object acquisition process, data for displaying the object Ob1 on the virtual projection plane A21 on three-dimensional computer graphics software is acquired as object information.

[0077] Next, the content generation system 10 uses the acquired display position information and object information to execute an object placement process in which the generation unit 14 places the object Ob1 shown in FIG. 3B at a display position P3 in the virtual space A2 (S5 in FIG. 5). That is, the generation unit 14 places the object Ob1 specified by the object information at the display position P3 specified by the display position information. The object Ob1 is virtually displayed by placing the object Ob1 at the display position P3 set on the virtual projection plane A21 in the virtual space A2. However, the object Ob1 only needs to be placed near the virtual projection plane A21, which is the display position P3, and does not necessarily need to be placed exactly on the virtual projection plane A21. Alternatively, the object Ob1 generated as a planar image may be placed on the virtual projection plane A21 as a texture, or may be placed on the virtual projection plane A21 by projection.

[0078] Next, the content generation system 10 executes a third acquisition process (S6 in FIG. 5) in which the third acquisition unit 13 acquires reference position information regarding the reference position P2 in order to set up the virtual camera V30 as shown in FIG. 3C. The reference position information is used to identify the reference position P2 at which the virtual camera V30 is to be set up in the virtual space A2, and is therefore, for example, information that identifies the installation position P4 at which the projector 30 is to be set up in the real space A1. In other words, using the reference position information obtained by copying the installation position P4 in the real space A1, it is possible to set up the virtual camera V30 at the same position in the virtual space A2 as the projector 30 in the real space A1. In this embodiment, the virtual space A2 is a three-dimensional space represented by an XYZ Cartesian coordinate system, and therefore the reference position information includes a coordinate position in the XYZ Cartesian coordinate system.

[0079] Next, the content generation system 10 uses the acquired reference position information to place a virtual camera V30, as shown in FIG. 3C, at a reference position P2 in the virtual space A2 using the generation unit 14 (S7 in FIG. 5). The virtual camera V30 is a virtual device defined to exist virtually in the virtual space A2, is made up of data that can be processed by one or more processors, and is a virtual device reproduced by a computer system. Therefore, the virtual camera V30 is placed by inputting parameters as reference position information into three-dimensional computer graphics software.

[0080] Here, as shown in FIG. 3C, the virtual camera V30 is a virtual device that mimics the projector 30 in terms of its external shape and specifications. That is, in accordance with the movable projector 30, the virtual camera V30 is a movable camera system whose field of view is not fixed but is variable within the virtual space A2. That is, the field of view of the virtual camera V30 is movable within the virtual space A2. Here, the virtual camera V30 moves its field of view by changing the incident direction of light. As an example, when the virtual camera V30 is installed in front of the virtual projection surface A21 and changes the incident direction of light from the left end to the right end of the virtual projection surface A21, the field of view moves to the right on the virtual projection surface A21.

[0081] In this embodiment, the virtual camera V30 is a moving-mirror camera system in accordance with the moving-mirror projector 30. That is, the virtual camera V30 moves the field of view by changing the incident direction of light by moving a virtual mirror unit V32 located on the optical path of the incident light. That is, as shown in FIG. 3C , the virtual camera V30 has a virtual capturing unit V31 and a virtual mirror unit V32. The virtual capturing unit V31 captures an object present in the field of view and outputs the image reflected in the field of view as an image (image signal). The virtual mirror unit V32 reflects light arriving from the surroundings toward the virtual capturing unit V31. The virtual camera V30 moves the field of view of the virtual camera V30 by driving the virtual mirror unit V32 to change the orientation of the virtual mirror unit V32. In short, in this embodiment, the virtual camera V30 oscillates the virtual mirror section V32, thereby changing the direction of light entering the virtual shooting section V31 at the virtual mirror section V32 and changing the direction of light entering the virtual camera V30.

[0082] Furthermore, the virtual camera V30 also imitates the projector 30 in terms of the incident direction of light from the virtual imaging unit V31, the shape of the virtual mirror unit V32, and the operation of the virtual mirror unit V32. For example, the optical axis of light incident on the virtual imaging unit V31 is vertical and intersects with the center or rotation center of the surface (reflecting surface) of the virtual mirror unit V32. Furthermore, the orientation of the virtual mirror unit V32 can be freely changed around the center point of the surface of the virtual mirror unit V32 by combining panning and tilting, for example. In FIG. 3C, the movement (rotation) direction of the virtual mirror unit V32 during panning is indicated by arrow M1, and the movement (rotation) direction of the virtual mirror unit V32 during tilting is indicated by arrow M2. Furthermore, the viewing angle (field angle) of the virtual camera V30 is the same as the field angle of the projector 30.

[0083] At this time, if the reference position P2 is located near the optical section 311 of the projector 30, it becomes easier to reproduce the real space A1 more accurately. Furthermore, if the reference position P2 is located near the pupil (projection pupil) position of the optical section 311 of the projector 30, it becomes easier to reproduce the real space A1 even more accurately.

[0084] As described above, the virtual camera V30 is defined to imitate the external shape and specifications of the projector 30, and therefore, for setting the virtual camera V30, setting parameters for specifying at least the external shape and specifications of the projector 30 are required. Therefore, the content generation system 10 according to this embodiment executes a parameter acquisition process to acquire the setting parameters (S8 in FIG. 5). The setting parameters are used for setting the virtual camera V30, and are, for example, information specifying the scale of the virtual space A2, the specifications of the optical unit 311 of the projector 30, the type of projector 30, and vignetting that occurs in the projector 30. Furthermore, parameters such as the resolution of the virtual camera V30 are also included in the setting parameters. The setting parameters acquired in the parameter acquisition process are reflected in the external shape and specifications of the virtual camera V30.

[0085] As one example, the scale of the virtual space A2 is arbitrarily set by the user based on the ratio of the length (dimension) of the virtual space A2 to the real space A1. The scale of the virtual space A2 is used to adjust the size of the projector 30 or calculate the projection distance. As another example, the specifications of the optical unit 311 are specified by displaying a list of projection lenses on the display device 40 and having the user select a projection lens for the projector 30 from the list. As another example, a screen for accepting input of projection lens parameters (zoom magnification, focal length, lens system brightness, etc.) may be displayed on the display device 40, allowing the user to arbitrarily set the projection lens parameters. As another example, a projection lens recommended based on space information may be specified as the specifications of the optical unit 311, such as recommending a telephoto lens for a large space and a wide-angle lens for a small space. Similarly, the type of projector 30 is specified by displaying a list of projector 30 types on the display device 40 and having the user select a type of projector 30 from the list. As another example, the user may specify a type of projector 30 that is recommended depending on the brightness of the projection image Im1 desired by the user as the type of projector 30. As yet another example, the content generation system 10 may communicate with the projector 30 to acquire the specifications of the projector 30, including the specifications of the optical unit 311, from the projector 30. The resolution and other parameters of the virtual camera V30 are set arbitrarily by the user. The resolution of the virtual camera V30 may also be automatically set to match the resolution of the specified (selected) projector 30.

[0086] After acquiring the setting parameters, the content generation system 10 executes a tracking setting process to set a tracking object (S9 in FIG. 5). The "tracking object" in the present disclosure refers to the object Ob1 that moves within the real space A1, tracking the projection position P1 when the projection position P1 moves within the real space A1. In other words, even if the projection position P1 moves, the object Ob1 set as the tracking object moves, tracking the projection position P1, and is continuously projected onto the projection position P1. The object Ob1 set as the tracking object is a target to be tracked by the virtual camera V30. In this embodiment, since there is only one object Ob1, only one object Ob1 is set as the tracking object. Specifically, for the object Ob1 to be designated as the tracking object, the user sets an object name (for example, "Butterfly") and the position (coordinate position) of the representative point of the object Ob1.

[0087] By the processes up to this point (S1 to S9 in FIG. 5), the content generation system 10 is now ready to generate the image content D1. Therefore, the content generation system 10 starts the generation process of generating the image content D1 in the generation unit 14 (S10 in FIG. 10).

[0088] In the generation process, basically, the generation unit 14 virtually displays an object Ob1 at a display position P3 in the virtual space A2, and then uses a virtual camera V30 in the virtual space A2 to capture an image of the display position P3. That is, in the virtual space A2, an image of the object Ob1 displayed at the display position P3 captured by the virtual camera V30 at a reference position P2 is generated by the virtual camera V30. As a result, the content generation system 10 can generate, as image content D1, an image including the object Ob1 viewed from the reference position P2 as shown in FIG. 4, using the virtual camera V30 defined by the generation unit 14.

[0089] However, the virtual camera V30 virtually captures an image of a subject present within the field of view of the virtual camera V30 and outputs the image captured within the field of view as an image (image signal). Therefore, strictly speaking, the image output from the virtual camera V30 is not an actually captured image, but a rendered image generated (rendered) by arithmetic processing. In any case, the image output from the virtual camera V30 at this time is an image including the object Ob1 as seen from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2. Therefore, the generation unit 14 generates the image content D1 by using the image virtually captured by the virtual camera V30 as the image content D1.

[0090] In this embodiment, since the image content D1 is data of a (full-color) moving image, the generation unit 14 continuously generates the image content D1 by continuously capturing images of the object Ob1 with the virtual camera V30 during the generation process. That is, the virtual camera V30 in the virtual space A2 captures images of the object Ob1 displayed at the display position P3 in the same way as capturing a moving image, and the generation unit 14 generates the image content D1 made up of moving image data. Therefore, if the object Ob1 has movement, that is, if the object Ob1 is an animation in which it transforms over time, the movement of the object Ob1 will be reflected in the image content D1. As an example, if the "butterfly" object Ob1 has a "movement" such as flapping its wings as illustrated in FIG. 4, the image content D1 will be data of a moving image of the "butterfly" flapping its wings.

[0091] The image content D1 generated in this manner is transmitted to the projector 30, as shown in FIG. 4, so that the projected image Im1 can be projected into the real space A1 by the projector 30. As shown in FIG. 4, the projected image Im1 is displayed in the real space A1 at a projection position P1 that corresponds to the display position P3 in the virtual space A2. Furthermore, when the projector 30 projects the image content D1 at the projection position P1, the projected object Ob1 substantially coincides with the object Ob1 virtually displayed at the display position P3. Therefore, in the real space A1, the object Ob1 virtually displayed in the virtual space A2 is projected as the projection image Im1, as if it were a copy of the virtual space A2. Details of the process of actually projecting the image content D1 by the projector 30 will be described in the section "(5) Content Projection Method."

[0092] In this embodiment, the image content D1 is an image of the object Ob1 in the virtual space A2, excluding the background of the object Ob1. That is, since the image content D1 is an image virtually captured by the virtual camera V30, strictly speaking, the virtual projection surface A21, which is the background of the object Ob1, may be reflected in the image content D1. Therefore, the generation unit 14 extracts only the object Ob1 from the image captured by the virtual camera V30 to generate the image content D1. That is, in the image content D1 shown in FIG. 4, the periphery of the object Ob1 is transparent (a state in which no image data exists).

[0093] As a result, the image content D1 generated by the generation unit 14 becomes image data of only the object Ob1, excluding the background. Specifically, when the object Ob1 is displayed at the display position P3 in the virtual space A2, the area inside the display position P3 other than the object Ob1, i.e., the area corresponding to the background, is displayed in black, with a pixel value (brightness value) of 0. Since the projector 30 does not emit light to the black image, the result is that an image of only the object Ob1, excluding the background, is generated as the image content D1.

[0094] In this embodiment, as described above, the projector 30 that projects the generated image content D1 is a movable projection system in which the projection position P1 in the real space A1 is variable, and the projection position P1 is movable within the real space A1. Therefore, the display position P3 corresponding to the projection position P1 is also movable within the virtual space A2, similar to the projection position P1. In other words, when the projection position P1 is moved by the projector 30 that projects the image content D1, the display position P3 is also moved when generating the image content D1. In this case, the object Ob1 (tracked object) displayed at the display position P3 also moves within the virtual space A2.

[0095] Therefore, the content generation system 10 according to this embodiment executes a second acquisition process in which the second acquisition unit 12 acquires display position information related to the display position P3 as needed, even during the execution of the generation process. Furthermore, the content generation system 10 executes an object placement process in which the generation unit 14 places the object Ob1 at the display position P3 as needed, even during the execution of the generation process. That is, in the flowchart of FIG. 5, even after the start of the generation process (S10), the second acquisition process (corresponding to S3) in which the display position information is acquired and the object placement process (corresponding to S5) in which the object Ob1 is placed at the display position P3 are repeatedly executed.

[0096] The movement of the display position P3 is realized, for example, by receiving input of display position information from the user as needed on an input support tool provided by the providing unit 18. That is, on an input support tool used on a graphical user interface, the user specifies an arbitrary position in the virtual space A2 by operating, for example, a pointing device included in the input device 50, and this position is designated as the display position P3. Therefore, by moving the pointing device, the display position information is sequentially input, and the display position P3 is moved in real time in accordance with the movement of the pointing device. The input support tool will be described in detail in the section "(4.2) Input support tool."

[0097] Then, after starting the generation process, the content generation system 10 determines whether the tracking object has moved using the generation unit 14 (S11 in FIG. 5). That is, in this embodiment, of the object Ob1, when the projection position P1 moves in the real space A1, the object Ob1 that moves in the real space A1 to follow the projection position P1 is defined as the "tracking object." Therefore, if there is a change in the display position P3 corresponding to the projection position P1 in the second acquisition process after starting the generation process, the generation unit 14 determines that the tracking object has moved (S11: Yes). On the other hand, if there is no change in the display position P3 corresponding to the projection position P1 in the second acquisition process after starting the generation process, the generation unit 14 determines that the tracking object has not moved (S11: No).

[0098] If the tracked object does not move (S11: No), that is, if the object Ob1 displayed at the display position P3 remains in one place, the content generation system 10 skips steps S12 and S13 in Fig. 5 and outputs the image content D1 (S14). At this time, the content generation system 10 outputs the image content D1 by recording (writing) it in the data storage unit 22 using the output unit 20. As a result, the image content D1 generated in the generation process is recorded (saved) in the data storage unit 22 as needed.

[0099] On the other hand, if the tracking object moves (S11: Yes), that is, if the object Ob1 displayed at the display position P3 moves, the content generation system 10 causes the virtual camera V30 to track the object Ob1 (tracking object) (S12 in FIG. 5). At this time, the content generation system 10 automatically controls the virtual camera V30 using the generation unit 14 so that the representative point of the object Ob1 set as the tracking object is positioned on the optical axis (center of the field of view) of the virtual camera V30. Specifically, the generation unit 14 oscillates the virtual mirror unit V32, thereby changing the direction of arrival of light incident on the virtual shooting unit V31 using the virtual mirror unit V32 and changing the orientation of the optical axis (center of the field of view) of the virtual camera V30.

[0100] The generation unit 14 then tracks the object Ob1 with the virtual camera V30 and continuously captures the object Ob1 with the virtual camera V30 during the generation process, thereby continuously generating image content D1. Here, the content generation system 10 executes a control information generation process in which the control information generator 17 generates control information D2 (S13 in FIG. 5). That is, in the control information generation process, control information D2 for the projector 30 is generated to move the projection position P1 in the real space A1 in synchronization with the movement of the display position P3. In other words, in the control information generation process, control information D2 is generated in synchronization with the movement of the tracked object (object Ob1) in the virtual space A2.

[0101] In this embodiment, the content generation system 10 generates, as control information D2, information used to control the virtual camera V30 when the virtual camera V30 follows the tracked object. Specifically, in the control information generation process, information used by the generation unit 14 to oscillate the virtual mirror unit V32 is generated as control information D2. In short, the parameters used to control the virtual mirror unit V32 of the virtual camera V30 in the virtual space A2 match the control information D2, which is the parameters used to control the mirror unit 32 of the projector 30 in the real space A1.

[0102] After generating the control information D2, the content generation system 10 outputs the control information D2 together with the image content D1 (S14 in FIG. 5). That is, the content generation system 10 outputs the image content D1 and the control information D2 by using the output unit 20 to record (write) them in the data storage unit 22. As a result, the image content D1 generated in the generation process is recorded (saved) as needed in the data storage unit 22 together with the control information D2 generated in the control information generation process.

[0103] As a result, in the generation process, an image of object Ob1 displayed at display position P3 moving within virtual space A2 as viewed from reference position P2 is generated as image content D1 consisting of a moving image. That is, when the tracking object moves within virtual space A2, content generation system 10 continuously captures the object Ob1 with virtual camera V30 while tracking object Ob1 (tracking object) with virtual camera V30. Therefore, an image (moving image) of object Ob1 displayed at display position P3 moving within virtual space A2 as if it were captured as a moving image from reference position P2 is generated as image content D1.

[0104] Furthermore, in the generation process, the orientation of the object Ob1 in the image content D1 is changed according to the position of the display position P3 in the virtual space A2. That is, in the generation process, a rotation process is executed to rotate the object Ob1 in the image content D1 in accordance with the movement of the display position P3. This rotation process is particularly useful when the projector 30 is a moving-mirror projection system. In other words, in the projector 30, if the orientation of the object Ob1 in the image content D1 is constant, for example, when the mirror unit 32 rotates 90 degrees by panning, the object Ob1 in the projection image Im1 projected onto the projection surface A11 rotates 90 degrees. Therefore, in order to maintain a constant orientation (posture) of the object Ob1 in the projection image Im1 in the real space A1, it is necessary to rotate the object Ob1 in the image content D1 according to the orientation of the mirror unit 32, i.e., the projection position P1.

[0105] Therefore, in this embodiment, the generation process changes the orientation of the object Ob1 in the image content D1 according to the position of the display position P3, thereby generating image content D1 in which the object Ob1 rotates according to the projection position P1. Specifically, since the virtual camera V30 is a moving-mirror camera system, the orientation of the object Ob1 in the image content D1 automatically changes according to the position of the display position P3 by capturing an image reflected by the virtual mirror unit V32 with the virtual capturing unit V31. In other words, if the orientation of the object Ob1 at the display position P3 is constant in the virtual camera V30, for example, when the virtual mirror unit V32 rotates 90 degrees by panning, the object Ob1 in the image captured by the virtual capturing unit V31 rotates 90 degrees. At this time, the object Ob1 rotates around the rotation center of the virtual mirror unit V32, i.e., the reference position P2.

[0106] Furthermore, in this embodiment, the control information D2 includes not only information for controlling the drive unit 33 that drives the mirror unit 32, but also information for controlling other components of the projector 30 than the mirror unit 32 and the drive unit 33. Specifically, for example, control parameters such as focus, light intensity of the light source, zoom, or lens shift in the projector 30 may be included in the control information D2. These parameters are also generated in synchronization with the movement of the display position P3 in the control information generation process. That is, parameters such as focus and light intensity of the light source are generated as the control information D2 in synchronization with the movement of the tracked object (object Ob1) in the virtual space A2.

[0107] As an example, for focus, a value corresponding to the distance from the reference position P2 where the virtual camera V30 is placed in the virtual space A2 to the display position P3 is calculated to match the projection distance in the real space A1. For the light intensity of the light source, a value corresponding to the width (area) of the display position P3 in the virtual space A2 is calculated to match the width (area) of the projection position P1 in the real space A1. For example, the light intensity of the light source is set to be greater (brighter) as the projection position P1 (display position P3) becomes wider, so as to suppress fluctuations in the light intensity per unit area in the projection image Im1. However, the light intensity of the light source may be adjusted by the pixel value (luminance value) in the image content D1 instead of the control information D2.

[0108] In the case of zooming, a value corresponding to the distance from the reference position P2 where the virtual camera V30 is placed in the virtual space A2 to the display position P3 is calculated to match the projection distance in the real space A1. For example, the shorter the distance from the reference position P2 to the display position P3, the wider the zoom is set to be, so as to suppress fluctuations in the area of ​​the projection image Im1. Here, it is preferable that the zoom is also reflected in the angle of view of the virtual camera V30. In the case of lens shifting, a value corresponding to the relative positional relationship between the reference position P2 where the virtual camera V30 is placed in the virtual space A2 and the display position P3 is calculated to suppress vignetting in the image captured by the virtual camera V30. It is preferable that the lens shifting is also reflected in the position of the virtual camera V30 (reference position P2).

[0109] Then, as shown in FIG. 5, after outputting the image content D1 (and the control information D2) (S14), the content generation system 10 determines whether or not the operation in the generation mode has ended (S15). The generation mode is ended, for example, by the user performing a predetermined end operation on the input support tool provided by the providing unit 18. If the generation mode has not ended (S15: No), the content generation system 10 returns to step S11 and continues generating the image content D1. On the other hand, if the generation mode has ended (S15: Yes), the content generation system 10 ends the series of operations related to the content generation method.

[0110] As described above, the content generation method according to this embodiment generates image content D1 in a generation process based on the spatial information acquired in the first acquisition process, the reference position information acquired in the third acquisition process, and the display position information acquired in the second acquisition process. At this time, the generation process uses a virtual camera V30 to generate, as image content D1, an image including object Ob1 viewed from reference position P2 when object Ob1 is displayed at display position P3 in virtual space A2.

[0111] Furthermore, in this embodiment, the projection position P1 is movable within the real space A1. Then, in the second acquisition process, display position information is acquired regarding the display position P3, which moves within the virtual space A2 in accordance with the movement of the projection position P1. That is, the display position P3 corresponding to the projection position P1 is movable within the virtual space A2, and the second acquisition unit 12 executes the second acquisition process to acquire the display position information regarding the display position P3 as needed, even during the execution of the generation process. As a result, in the second acquisition process, the display position information regarding the display position P3 moving within the virtual space A2 can be acquired. Here, the virtual camera V30 is a movable camera system whose field of view within the virtual space A2 is variable, and thus it is possible to follow the display position P3 moving within the virtual space A2 and capture an image of the object Ob1 within the display position P3.

[0112] In particular, in this embodiment, the virtual camera V30 has a virtual shooting unit V31 and a virtual mirror unit V32, and the field of view of the virtual camera V30 is moved by driving the virtual mirror unit V32 to change the orientation of the virtual mirror unit V32. In other words, since the virtual camera V30 is a moving-mirror type camera system, by driving the virtual mirror unit V32, it is possible to follow the display position P3 moving within the virtual space A2 and capture an image of the object Ob1 within the display position P3.

[0113] The flowchart shown in FIG. 5 is merely an example, and the order of the processes may be changed as appropriate, and processes may be added or deleted as appropriate.

[0114] (4.2) Input support tools Next, the input support tool will be described with reference to FIGS.

[0115] The content generation method according to this embodiment includes a providing process for providing an input support tool by the providing unit 18. In this embodiment, the providing process is executed to provide the input support tool at least when the operation mode of the content generation system 10 is the generation mode.

[0116] In this embodiment, the input support tool is embodied in an input support tool screen T1 as shown in Fig. 6, and is a tool used on a graphical user interface of a computer system, but does not have any physical form. Figs. 6 to 11 show specific examples of screens displayed on the display device 40, and the dashed-dotted lines, reference symbols, and leader lines indicating areas are shown merely for the purpose of explanation, and these dashed-dotted lines, reference symbols, and leader lines are not actually displayed on the display device 40. In addition, as an example in this embodiment, the input support tool screen T1 is a landscape-long screen in which the horizontal dimension is larger than the vertical dimension.

[0117] The input support tool has two modes: an adjustment mode and a creation support mode. Fig. 6 shows the input support tool screen T1 when the input support tool is in the adjustment mode, and Figs. 7 to 11 show the input support tool screen T1 when the input support tool is in the creation support mode.

[0118] First, as described above, the input support tool is a tool for allowing the user to input display position information during the second acquisition process, and presents at least the display position P3 within the virtual space A2. Therefore, in this embodiment, whether the input support tool is in the adjustment mode or the creation support mode, the input support tool screen T1 displays the display position P3 within the virtual space A2. In other words, by looking at the input support tool screen T1, the user can grasp where in the virtual space A2 the display position P3 is located.

[0119] 6, the input support tool presents the display position P3 by displaying an image of the virtual space A2 in which the object Ob1 is displayed at the display position P3. That is, on the input support tool screen T1, the virtual space A2 is visualized, and the object Ob1 is displayed at the display position P3 in the virtual space A2, making it possible to identify the display position P3 within the virtual space A2. Furthermore, the virtual space A2 visualized on the input support tool screen T1 is reproduced in a state in which the virtual camera V30 is also visualized.

[0120] Furthermore, in this embodiment, the input support tool displays, as the reference screen T11, a rendering image of the object Ob1 displayed at the display position P3 as viewed from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2. That is, on the input support tool screen T1, a rendering image corresponding to the image content D1 generated by the generation unit 14 is displayed as the reference screen T11. In this embodiment, by using a virtual camera V30 in the generation process, an image including the object Ob1 as viewed from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2 is generated as the image content D1. Therefore, the reference screen T11 is a rendering image virtually captured by the virtual camera V30 and generated (rendered) by calculation.

[0121] In the present embodiment, as an example, the reference screen T11 is displayed in a picture-in-picture (PIP) format, superimposed on a portion of the input support tool screen T1. Therefore, the input support tool screen T1 allows the user to simultaneously view the visualized virtual space A2 and a rendering image (reference screen T11) corresponding to the image content D1 to be generated. Displaying such a reference screen T11 makes it easier for the user to visualize the image content D1 to be generated.

[0122] Furthermore, in the present embodiment, as an example, in the virtual space A2 visualized on the input support tool screen T1, a grid G1 is displayed on a virtual projection surface A21 corresponding to the projection surface A11 (wall surface) of the real space A1.

[0123] In the examples of FIGS. 6 to 11, the background of the object Ob1, that is, the area inside the display position P3 and displayed in black other than the object Ob1, is shaded (dot hatched).

[0124] On the other hand, in the image projected from the projector 30, all areas other than the object Ob1 are displayed in black, so that all areas other than the object Ob1 are also displayed in black on the reference screen T11. For example, on the reference screen T11 of FIG. 6, the shaded (dot-hatched) area other than the object Ob1 and the white area outside of it are all displayed in black. Such an image can be generated, for example, by rendering only the object Ob1 and excluding all areas other than the object Ob1 from the rendering target. The image content D1 to be recorded (written) in the data storage unit 22 can be generated in this manner.

[0125] However, while the user is performing various operations on the input support tool screen T1, it is preferable that the grid G1, the virtual camera V30 simulating the projector 30, the periphery of the virtual mirror section V32, and the like are also displayed on the reference screen T11 as rendering targets. This makes it easier for the user to check the projection position and vignetting on the input support tool screen T1. For example, if the grid G1 is displayed on the reference screen T11, the user can easily check the position and rotation state of the object Ob1 relative to the virtual space A2 on the input support tool screen T1. Therefore, on the reference screen T11 in the examples of FIGS. 7 to 11, the grid G1, the virtual camera V30, the periphery of the virtual mirror section V32, and the like are also included in the rendering targets.

[0126] Furthermore, the grid G1, the virtual camera V30, the periphery of the virtual mirror portion V32, and the like, which may be rendered as described above, may be projected onto the real space A1 together with the object Ob1 by the projector 30. That is, for example, when the grid G1 is projected onto the real space A1 by the projector 30, the projection position of the image content D1 in the real space A1 can be easily confirmed using this grid G1 as a guide. As an example, projecting the grid G1 makes it easy to confirm to what extent the corners of a room in the real space A1 coincide with the corners of a room in the image content D1 projected by the projector 30.

[0127] Furthermore, when the periphery of the virtual camera V30 or the virtual mirror unit V32 is projected onto the real space A1 by the projector 30, the user can visually confirm that vignetting actually occurs at the periphery of the projector 30 or the mirror unit 32 present in the real space A1. Furthermore, when the virtual camera V30 is projected onto the real space A1 by the projector 30, the virtual camera V30 is mapped onto the projector 30 present in the real space A1, making it easier to confirm the projection position within the real space A1 from the mapping deviation. Similarly, when the periphery of the virtual mirror unit V32 is projected onto the real space A1 by the projector 30, the periphery of the virtual mirror unit V32 is mapped onto the periphery of the mirror unit 32 present in the real space A1, making it easier to confirm the projection position within the real space A1 from the mapping deviation.

[0128] In addition, the input support tool screen T1 may include a means (such as a check box or button) for accepting a user operation to switch whether or not to render the grid G1, the virtual camera V30, and the periphery of the virtual mirror portion V32.

[0129] The following describes the detailed configuration of the input support tool screen T1 when the input support tool is in the adjustment mode and the creation support mode. In the following description, operation of the button B11 or the like means operation of the button B11 or the like displayed on the input support tool screen T1, and specifically means that the user selects the button B11 or the like on the input support tool screen T1 using the input device 50.

[0130] First, when the input support tool is in adjustment mode, a title field C1, multiple input fields C11 to C18, and multiple buttons B11 to B13 are displayed on the left edge of the input support tool screen T1, as shown in Fig. 6. The title field C1 displays the character string "Adjustment," which indicates the mode of the input support tool.

[0131] The multiple input fields C11 to C13 are used to adjust the overall position of the virtual camera V30. That is, the user inputs coordinate values ​​of the X-axis, Y-axis, and Z-axis in the virtual space A2 expressed in an XYZ Cartesian coordinate system into the input fields C11, C12, and C13, respectively. That is, the information input into the multiple input fields C11 to C13 corresponds to reference position information for specifying the reference position P2 where the virtual camera V30 is to be installed. When the button B11 is operated with values ​​entered into the multiple input fields C11 to C13, the content generation system 10 places the virtual camera V30 at the reference position P2 specified by the input values.

[0132] The multiple input fields C14 to C16 are used to adjust the overall orientation of the virtual camera V30. That is, the user inputs the rotation amounts around the X-axis, Y-axis, and Z-axis in the virtual space A2 represented by the XYZ Cartesian coordinate system into the input fields C14, C15, and C16, respectively. When the button B12 is operated with values ​​input into the multiple input fields C14 to C16, the content creation system 10 rotates the virtual camera V30 by the rotation amounts specified by the input values.

[0133] Here, when adjusting the overall position and orientation of the virtual camera V30, the input support tool preferably executes the following process. For example, the input support tool displays a display on the input support tool screen T1 for checking the field of view of the virtual camera V30 with the virtual camera V30 installed in the virtual space A2. At this time, the input support tool screen T1 displays, for example, an area surrounded by the trajectory of the optical axis (center of the field of view) of the virtual camera V30 when the virtual mirror part V32 is driven along the four sides of the movable area as the field of view of the virtual camera V30. Therefore, the user can specify an appropriate overall position and orientation of the virtual camera V30 while checking the field of view of the virtual camera V30 on the input support tool screen T1.

[0134] The multiple input fields C17 and C18 are used to adjust the orientation (angle) of the virtual mirror part V32 of the virtual camera V30 relative to the virtual capturing part V31. That is, the user inputs the rotation amount of the virtual mirror part V32 in the panning direction and tilting direction in the input fields C17 and C18, respectively. When the button B13 is operated with values ​​input into the multiple input fields C17 and C18, the content generation system 10 rotates the virtual mirror part V32 relative to the virtual capturing part V31 by the rotation amount specified by the input values.

[0135] Therefore, by combining the information input into the multiple input fields C11 to C18, the optical axis (center of the field of view) of the virtual camera V30 is identified, and the information input into the multiple input fields C11 to C18 corresponds to display position information for identifying the display position P3. In other words, when the input support tool is in the adjustment mode, the display position P3 at which the object Ob1 is displayed is determined by the information input into the multiple input fields C11 to C18. In the example of Fig. 6, the display position P3 is set across the two virtual projection surfaces A21 at a corner formed by the two virtual projection surfaces A21 that are orthogonal to each other, and the object Ob1 is displayed at this display position P3.

[0136] Two check boxes B1 and B2 are also displayed on the reference screen T11. Each check box B1 and B2 is checked or unchecked by the user. The check box B1, labeled "Mesh," is used to toggle the display of a grid G1 on the reference screen T11. When the check box B1 is checked, the grid G1 is displayed on the reference screen T11; when the check box B1 is unchecked, the grid G1 is not displayed on the reference screen T11. The check box B2, labeled "Circle," is used to toggle the display of a rotation area R2 (see FIG. 11) on the reference screen T11. When the check box B2 is checked, the rotation area R2 is displayed on the reference screen T11; when the check box B2 is unchecked, the rotation area R2 is not displayed on the reference screen T11. The rotation area R2 indicates the range within which the size of the object Ob1 does not change even when the object Ob1 is rotated.

[0137] Next, if the input support tool is in the creation support mode, a title field C2, a plurality of check boxes B21 to B24, and a slide bar B25 are displayed on the left edge of the input support tool screen T1, as shown in Fig. 7. The character string "Butterfly" indicating the object name is displayed in the title field C2.

[0138] The multiple check boxes B21-B24 are used to set the movement speed of the display position P3 (object Ob1) when the keyboard serving as the input device 50 is operated. That is, one of the multiple check boxes B21-B24, each labeled with a character string such as "0.1 m / sec," "1 m / sec," "5 m / sec," or "10 m / sec," is selected, and the movement speed of the display position P3 is determined by the selected check box B21-B24. For example, when the check box B21 with "0.1 m / sec" is selected, the movement speed of the display position P3 becomes the minimum, and when the check box B24 with "10 m / sec" is selected, the movement speed of the display position P3 becomes the maximum. Each time the user operates the check boxes B21-B24, a check mark is added or removed, and the selected check box B21-B24 is checked.

[0139] The slide bar B25 is used to adjust the size of the object Ob1. That is, the size of the object Ob1 displayed at the display position P3 can be adjusted by operating the slide bar B25. Here, as an example, the size of the object Ob1 increases as the slide bar B25 moves to the right.

[0140] Also, the check boxes B1 and B2 in the reference screen T11 are the same as those on the input support tool screen T1 (see FIG. 6) when the input support tool is in the adjustment mode.

[0141] Furthermore, when the input support tool is in the creation support mode, the user operates the input device 50 to specify an arbitrary position within the virtual space A2, and this position is designated as the display position P3. For example, when the cursor is moved on the input support tool screen T1 by operating a pointing device or a keyboard included in the input device 50, the display position P3 is set at the position on the virtual projection surface A21 where the cursor overlaps. Therefore, for example, by moving a pointing device such as a mouse, display position information is sequentially input, and the display position P3 is moved in real time in accordance with the movement of the pointing device. For example, when an operation such as dragging an object Ob1 is performed with a pointing device such as a mouse, the object Ob1 and the display position P3 move in real time in the virtual space A2 displayed on the input support tool screen T1 in conjunction with this operation.

[0142] Here, the content generation system 10 automatically controls the virtual camera V30 by the generation unit 14 so that the representative point of the object Ob1 set as the tracking object is positioned on the optical axis (center of the field of view) of the virtual camera V30. Therefore, when the object Ob1 moves within the virtual space A2 as described above, the virtual mirror unit V32 swivels in conjunction with this, and the virtual mirror unit V32 also swivels in the virtual camera V30 displayed on the input support tool screen T1.

[0143] For example, the input support tool screen T1 shown in Fig. 8 represents the input support tool screen T1 when the object Ob1 is moved leftward in the virtual space A2 from the state shown in Fig. 7. Therefore, when transitioning from the state shown in Fig. 7 to the state shown in Fig. 8, the virtual mirror unit V32 of the virtual camera V30 performs a panning operation. Therefore, when transitioning from the state shown in Fig. 7 to the state shown in Fig. 8, the object Ob1 in the reference screen T11 corresponding to the image content D1 rotates counterclockwise.

[0144] 9 represents the input support tool screen T1 when the object Ob1 is moved upward and leftward on the virtual projection plane A21 of the virtual space A2 from the state in FIG. 8. Therefore, when transitioning from the state in FIG. 8 to the state in FIG. 9, the virtual mirror unit V32 of the virtual camera V30 simultaneously performs panning and tilting. Therefore, when transitioning from the state in FIG. 8 to the state in FIG. 9, the object Ob1 in the reference screen T11 corresponding to the image content D1 rotates counterclockwise.

[0145] 9, a target range R1 corresponding to the outer periphery of the virtual mirror portion V32 is displayed on the reference screen T11. That is, depending on the rotation angle of the virtual mirror portion V32, for example, the field of view of the virtual camera V30 may extend beyond the virtual mirror portion V32, causing vignetting, and the object Ob1 may not fit within the field of view of the virtual camera V30. If the object Ob1 does not fit within the field of view of the virtual camera V30, the object Ob1 photographed by the virtual camera V30 may be missing or otherwise damaged. The same vignetting may also occur in the moving-mirror projector 30 in the area where vignetting occurs in the virtual camera V30. Therefore, the reference screen T11 displays the area where such vignetting may occur, i.e., the outside of the virtual mirror portion V32, as the target range R1, thereby presenting the user with a range in which the object Ob1 can be displayed without vignetting.

[0146] In short, in this embodiment, a guideline range R1 is defined in the virtual space A2 as a guideline for the range in which the object Ob1 can be displayed. This guideline range R1 is not limited to being displayed on the reference screen T11, and may be displayed, for example, on the virtual space A2 visualized on the input support tool screen T1. Furthermore, vignetting can occur not only outside the virtual mirror unit V32, but also, for example, when the main body of the virtual shooting unit V31 is reflected in the virtual space. That is, in the projector 30, vignetting can also occur when the projected light hits the main body of the projection unit 31. By defining the range of vignetting, etc., which can occur due to such various factors, as the guideline range R1, the user can use this as a guideline for the range in which the object Ob1 can be displayed without being cut off in the virtual space A2.

[0147] 10 represents the input support tool screen T1 when the object Ob1 is moved downward on the virtual projection plane A21 of the virtual space A2 from the state in FIG. 9. Therefore, when transitioning from the state in FIG. 9 to the state in FIG. 10, the virtual mirror unit V32 of the virtual camera V30 performs a tilt operation. In the example in FIG. 10, vignetting occurs in the right half of the reference screen T11 due to the reflection of the main body of the virtual photographing unit V31. In other words, the area where the main body of the virtual photographing unit V31 is reflected corresponds to the outside of the target range R1 (see FIG. 9).

[0148] 11 represents the input support tool screen T1 when the object Ob1 is moved significantly to the right in the virtual space A2 from the state in Fig. 10. Therefore, when transitioning from the state in Fig. 10 to the state in Fig. 11, the virtual mirror part V32 of the virtual camera V30 performs a panning operation. In the example in Fig. 11, the check box B2 on the reference screen T11 is checked, and a rotation region R2 is displayed on the reference screen T11.

[0149] By using the input support tool as described above, the user can relatively easily specify the display position P3 and generate image content D1 for projecting the object Ob1 at the projection position P1 corresponding to this display position P3. Furthermore, the user can relatively easily generate image content D1 when the display position P3 corresponding to the projection position P1 is moved.

[0150] Furthermore, similar to the specification of the display position P3 as described above, the input support tool may allow the user to specify parameters for controlling the shutter, etc., of the projector 30. It is preferable that the parameters specified by the user in this way are also included in the control information D2 if they are parameters related to the control of the projector 30.

[0151] (5) Content projection method Next, the operation of the content generation system 10 according to this embodiment in the playback mode, that is, the content projection method according to this embodiment, will be described.

[0152] In the content projection method according to this embodiment, image content D1 generated by the content generation method is projected onto projection position P1 in real space A1 by projector 30. Therefore, as preparation, it is necessary to install projector 30 at installation position P4 in real space A1. Here, installation position P4 is a position corresponding to reference position P2 in virtual space A2 where virtual camera V30 is installed, so the user installs projector 30 at installation position P4 corresponding to reference position P2 used in the content generation method.

[0153] Furthermore, the content projection method according to this embodiment further includes a correction process for correcting at least one of the installation state of the projector 30 and the control information D2 of the projector 30 according to parameters used in the generation process. The "installation state" here may include both the position and orientation of the projector 30. In this embodiment, in the generation process of the content generation method, adjustment of the overall position of the virtual camera V30 and adjustment of the overall orientation of the virtual camera V30 are performed on the input support tool screen T1 when the input support tool is in the adjustment mode. Therefore, parameters regarding the position and orientation of the virtual camera V30 used when the input support tool is in the adjustment mode are used to correct the installation state of the projector 30. This makes it possible to correct (fine-tune) the position and orientation of the projector 30 in the real space A1 to match the position and orientation of the virtual camera V30 in the virtual space A2. Conversely, it is also possible to correct (fine-tune) the position and orientation of the virtual camera V30 in the virtual space A2 to match the installation state (position and orientation) of the projector 30 in the real space A1.

[0154] The projector 30 installed at the installation position P4 is connected to the content generation system 10 by wire via the video cable 101, the control cable 102, and the communication cable 103, as described above.

[0155] In the content projection method according to this embodiment, the content generation system 10 reproduces image content D1 and outputs (transmits) a video signal to the projector 30, which then projects the image content D1. At this time, since the operation mode of the content generation system 10 is the reproduction mode, the content generation system 10 reproduces image content D1 stored in the data storage unit 22 and outputs (transmits) a video signal to the projector 30. The image content D1 reproduced here is the image content D1 generated by the content generation method according to this embodiment. That is, the content projection method includes a projection process in which the projector 30 projects the image content D1 generated by the content generation method onto a projection position P1 in the real space A1. The image content D1 is an image of the object Ob1 in the virtual space A2, excluding the background of the object Ob1, and therefore the projector 30 can project only the object Ob1.

[0156] Furthermore, in the projection process, control information D2 is also transmitted from the content generation system 10 to the projector 30 along with the image content D1. Here, the image content D1 and the control information D2 are input to the projector 30 in a synchronized state. Therefore, in the projector 30, the drive unit 33 can drive the mirror unit 32 in accordance with the control information D2 while projecting the image content D1. This allows the projector 30 to move the projection position P1 in the real space A1, thereby reproducing the movement of the object Ob1 when the image content D1 was generated in the virtual space A2.

[0157] The projection image Im1 projected by the image projection method described above is displayed in the real space A1 at a projection position P1 that corresponds to the display position P3 in the virtual space A2. Moreover, when the projector 30 projects the image content D1 at the projection position P1, the projected object Ob1 substantially coincides with the object Ob1 virtually displayed at the display position P3. Therefore, in the real space A1, the object Ob1 virtually displayed in the virtual space A2 is projected as the projection image Im1, as if it were a copy of the virtual space A2. Furthermore, for the image content D1 generated while moving the display position P3 in the virtual space A2, the projection position P1 can be moved in the real space A1 in the same way.

[0158] (6) Variations Embodiment 1 is merely one of various embodiments of the present disclosure. Various modifications to Embodiment 1 are possible depending on the design, etc., as long as the object of the present disclosure can be achieved. The figures described in this disclosure are schematic diagrams, and the ratios of the sizes and thicknesses of the components in the figures do not necessarily reflect the actual dimensional ratios. Furthermore, functions similar to those of the content generation system 10 according to Embodiment 1 may be embodied in a content generation method, a computer program, a non-transitory recording medium on which a computer program is recorded, or the like. Functions similar to those of the image projection system 100 according to Embodiment 1 may be embodied in an image projection method, a computer program, a non-transitory recording medium on which a computer program is recorded, or the like.

[0159] Below, we will list some modified examples of embodiment 1. The modified examples explained below can be applied in appropriate combinations.

[0160] The content generation system 10 of the present disclosure includes a computer system. The computer system is primarily composed of a processor and memory as hardware. The functions of the content generation system 10 of the present disclosure are realized by the processor executing a program stored in the memory of the computer system. The program may be pre-stored in the memory of the computer system, provided via a telecommunications line, or provided in a non-transitory recording medium readable by the computer system, such as a memory card, optical disk, or hard disk drive. The processor of the computer system is composed of one or more electronic circuits, including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integrations (VLSIs), or ultra-large-scale integrations (ULSIs). Furthermore, field-programmable gate arrays (FPGAs), which are programmable after the LSI is manufactured, or logic devices capable of reconfiguring the connections within the LSI or the circuit partitions within the LSI, can also be used as processors. The electronic circuits may be integrated into one chip or distributed across multiple chips. The chips may be integrated into one device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

[0161] Furthermore, it is not essential for the content generation system 10 that at least some of the functions of the content generation system 10 are concentrated in one housing, and the components of the content generation system 10 may be distributed across multiple housings. For example, at least some of the functions of the content generation system 10 may be provided in different housings. Furthermore, at least some of the functions of the content generation system 10 may be realized by the cloud (cloud computing), the edge (edge ​​computing), or a combination thereof.

[0162] Conversely, in the first embodiment, at least some of the functions distributed among multiple devices may be integrated into one housing. For example, the functions distributed among the content generation system 10 and the projector 30 may be integrated into one housing.

[0163] In embodiment 1, the content generation system 10 does not include the projector 30, the display device 40, and the input device 50 as components, but at least one of the projector 30, the display device 40, and the input device 50 may be included as components of the content generation system 10.

[0164] Furthermore, when moving the display position P3, it is not essential to accept input of display position information from the user on the input support tool at any time, as in the first embodiment.

[0165] As an example, the content generation method may acquire a scenario created by a user and move the object Ob1 in accordance with the scenario. The "scenario" here is realized by a command or the like that specifies the movement of the object Ob1. When creating a scenario, if the field of view of the virtual camera V30 extends beyond the virtual mirror portion V32 and vignetting may occur, it is preferable to notify the user.

[0166] As another example, the content generation method may acquire the detection results of a sensor device such as a human sensor or a camera (image sensor), and move the object Ob1 in accordance with the detection results. This makes it possible for the content generation method to move the position of the object Ob1 in real time in accordance with, for example, the position of a person that is the detection target of the sensor device.

[0167] Furthermore, the execution procedure (order) of the multiple processes in the content generation method is not limited to the order described in embodiment 1 and can be changed as appropriate. For example, the first acquisition process, the second acquisition process, and the third acquisition process are not limited to the order of the first acquisition process, the second acquisition process, and the third acquisition process, and may be executed in the order of the first acquisition process, the third acquisition process, and the second acquisition process.

[0168] Furthermore, the communication method between the content generation system 10 and the projector 30 is not limited to the method exemplified in the first embodiment, and may be other wired communication, wireless communication, or a hybrid of wired communication and wireless communication. Furthermore, the content generation system 10 is not necessarily required to be able to communicate with the projector 30. For example, the content generation system 10 may record the generated image content D1 on a recording medium such as a memory card, and the projector 30 may play back the image content D1 recorded on the recording medium. This makes it possible to project the image content D1 generated by the content generation system 10 using the projector 30, even if the content generation system 10 and the projector 30 do not communicate with each other.

[0169] Furthermore, the projector 30 is not limited to a floor-standing type, but may be attached to, for example, a ceiling or a wall, etc. Furthermore, it is not essential for the content generation method that the height of the projector 30 be adjustable, and the height of the projector 30 from the floor surface may be fixed.

[0170] Furthermore, the projector 30 is not limited to a moving mirror type projection system, and may be configured such that the projection unit 31 moves (including rotates), or the entire projector 30 moves (including rotates). In either case, the projection position of the light from the projector 30 changes, and the projection position P1 can be moved.

[0171] Furthermore, the projector 30 is not limited to a movable projection system in which the projection position P1 in the real space A1 is variable, but may be a fixed projection system in which the projection position P1 is fixed. Even in this case, the content generation method, content projection method, program, and content generation system 10 according to the present embodiment are useful. If the projector 30 is a fixed projection system, the virtual camera V30 may also be a fixed camera system.

[0172] Furthermore, the content generation system 10 does not necessarily have to include the virtual space creation unit 15, the object acquisition unit 16, the control information generation unit 17, the provision unit 18, the input unit 19, the output unit 20, the communication unit 21, and the data storage unit 22. In other words, at least one of the virtual space creation unit 15, the object acquisition unit 16, the control information generation unit 17, the provision unit 18, the input unit 19, the output unit 20, the communication unit 21, and the data storage unit 22 may be omitted as appropriate.

[0173] Furthermore, in the content generation method, it is not essential to record (write) the generated image content D1 in the data storage unit 22. For example, in the content generation method, the generated image content D1 may be transmitted to the projector 30 in real time.

[0174] Furthermore, in the content generation method, it is not essential to use the virtual camera V30 in generating the image content D1. In other words, in the generation process, it is sufficient to generate, as the image content D1, an image including the object Ob1 as seen from the reference position P2 when the object Ob1 is displayed at the display position P3 in the virtual space A2 based on the space information, the reference position information, and the display position information. In such a generation process, the virtual camera V30 is not essential.

[0175] Furthermore, the input support tool is only required to present the display position P3, and does not necessarily have to display an image of the virtual space A2 in which the object Ob1 is displayed at the display position P3. As one example, the input support tool may present (display) the display position P3 in the virtual space A2 by means of highlighting the display position P3 in the virtual space A2 in a planar view. As another example, the input support tool may present (display) the display position P3 using the coordinate position of the virtual space A2 expressed in an XYZ Cartesian coordinate system.

[0176] Furthermore, displaying the reference screen T11 is not an essential component of the input support tool, and the input support tool does not necessarily need to have the function of displaying a rendering image of the object Ob1 as in the reference screen T11.

[0177] Furthermore, the input support tool screen T1 is not limited to a horizontally long screen, and may be, for example, a vertically long screen in which the horizontal dimension is smaller than the vertical dimension. The input support tool screen T1 does not necessarily have to display the reference screen T11 in a picture-in-picture format, and may, for example, switch to display the reference screen T11.

[0178] Furthermore, when adjusting the overall position and orientation of the virtual camera V30, it is not essential that the input support tool display a display on the input support tool screen T1 to confirm the field of view of the virtual camera V30. As an example, the content generation method may propose a recommended overall position and orientation of the virtual camera V30 based on the layout of structures such as walls, floors, and ceilings in the virtual space A2. That is, the content generation method may calculate a recommended position and orientation of the virtual camera V30 so that the surface of the structure in the virtual space A2 (virtual projection plane A21) is included in the field of view of the virtual camera V30, and present it to the user. As another example, the user may specify a desired area in which he or she wishes to display (project) the object Ob1, and the content generation method may propose a recommended overall position and orientation of the virtual camera V30 according to the desired area. That is, the content generation method may calculate a recommended position and orientation of the virtual camera V30 so that the desired range is included in the field of view of the virtual camera V30, and present it to the user.

[0179] Furthermore, in the content projection method, the user may be able to specify the object Ob1 to be projected. In this case, the projector 30 may project not only the object Ob1 specified as the tracking object, but also other objects Ob1. Furthermore, in the content projection method, multiple objects Ob1 may be projected simultaneously. When the object Ob1 specified as the tracking object and other objects Ob1 are projected simultaneously, according to the content projection method, the object Ob1 that moves within the real space A1 and the object Ob1 that does not move are projected simultaneously.

[0180] Furthermore, a plurality of virtual cameras V30 may be installed in the virtual space A2. In this case, image content D1 can be generated by each of the plurality of virtual cameras V30. Here, the object Ob1 photographed (generated) by the plurality of virtual cameras V30 may be a common object Ob1, or may be different objects Ob1. In either case, in the content projection method, the image content D1 is projected by a plurality of projectors 30 corresponding to the plurality of virtual cameras V30.

[0181] (Embodiment 2) The content generation method according to this embodiment differs from the content generation method according to the first embodiment in that the virtual camera V30 is not a moving mirror type but moves (including rotates) as a whole, as shown in Fig. 12. Hereinafter, the same components as those in the first embodiment will be assigned the same reference numerals and descriptions thereof will be omitted as appropriate.

[0182] That is, in this embodiment, the virtual camera V30 does not have a virtual mirror unit V32 (see FIG. 3C), and the entire virtual camera V30 including the virtual capturing unit V31 moves (including rotates). In FIG. 12, the movement (rotation) direction of the virtual camera V30 during panning is indicated by arrow M1, and the movement (rotation) direction of the virtual camera V30 during tilting is indicated by arrow M2. The intersection of the rotation axis of the panning operation and the rotation axis of the tilting operation is located, for example, at the lens center of the virtual capturing unit V31.

[0183] In this embodiment, too, it is assumed that the projector 30 that projects the generated image content D1 is a moving-mirror projection system. Therefore, in this embodiment, it is preferable that the generation process change the orientation of the object Ob1 in the image content D1 according to the position of the display position P3 in the virtual space A2. That is, in the generation process, the orientation of the object Ob1 in the image content D1 is changed according to the position of the display position P3, thereby generating the image content D1 in which the object Ob1 rotates according to the projection position P1. As a means for rotating the object Ob1 in the image content D1, the virtual camera V30 may be rotated around the optical axis, or a rotation correction may be applied to the object Ob1 photographed by the virtual camera V30.

[0184] The same functions as the content generation method according to the second embodiment may be realized by the content generation system 10, a computer program, or a non-transitory recording medium on which a computer program is recorded.

[0185] As a modification of the second embodiment, the projector 30 is not limited to a moving mirror type projection system, and may be configured, for example, so that the projection unit 31 moves (including rotates), or so that the entire projector 30 moves (including rotates). In the case of a configuration in which the projection unit 31 moves, or a configuration in which the entire projector 30 moves, the process of changing the orientation of the object Ob1 in the image content D1 according to the position of the display position P3 can be omitted.

[0186] As another modification of the second embodiment, the virtual camera V30 may have a virtual mirror part V32. In this case, for example, the virtual mirror part V32 is transparent, and the virtual shooting part V31 is located behind the virtual mirror part V32.

[0187] The various configurations (including modified examples) described in the second embodiment can be adopted in appropriate combination with the various configurations (including modified examples) described in the first embodiment.

[0188] (summary) As described above, the content generation method according to the first aspect is a method for generating image content (D1) to be projected by a projector (30) at a projection position (P1) in a real space (A1), and includes a first acquisition process, a second acquisition process, a third acquisition process, and a generation process. In the first acquisition process, spatial information regarding a virtual space (A2) corresponding to the real space (A1) is acquired. In the second acquisition process, display position information regarding a display position (P3) in the virtual space (A2) corresponding to the projection position (P1) is acquired. In the third acquisition process, reference position information regarding a reference position (P2) in the virtual space (A2) is acquired. In the generation process, an image including an object (Ob1) viewed from a reference position (P2) when the object (Ob1) is displayed at the display position (P3) in the virtual space (A2) is generated as image content (D1) based on the spatial information, the reference position information, and the display position information.

[0189] According to this aspect, an image including the object (Ob1) as viewed from the reference position (P2) when the object (Ob1) is displayed at the display position (P3) in the virtual space (A2) is generated as the image content (D1). That is, in the virtual space (A2), the object (Ob1) can be virtually displayed in any size and shape at any display position (P3). For example, if the image content (D1) is projected at the projection position (P1) by a projector (30) located at a position corresponding to the reference position (P2) in the real space (A1), the projected object (Ob1) will substantially coincide with the object (Ob1) displayed at the display position (P3). Therefore, by displaying the object (Ob1) at the display position (P3) in the virtual space (A2), the image content (D1) can be generated without considering the relative positional relationship between the projector (30) and the projection position (P1). As a result, it becomes easy to generate the image content (D1) according to the relative positional relationship between the projector (30) and the projection position (P1), which has the advantage of making it easier to generate the image content (D1).

[0190] In the content generation method according to the second aspect, in the first aspect, the projection position (P1) is movable within the real space (A1). In the second acquisition process, display position information regarding a display position (P3) that moves within the virtual space (A2) in accordance with the movement of the projection position (P1) is acquired.

[0191] According to this embodiment, it is possible to more easily generate image content (D1) suitable for projection onto a projection position (P1) that moves within the real space (A1).

[0192] In the content generation method according to the third aspect, in the second aspect, the projector (30) has a projection unit (31), a mirror unit (32), and a drive unit (33). The projection unit (31) emits light for projecting a projection image (Im1) using image content (D1). The mirror unit (32) reflects the light emitted from the projection unit (31). The drive unit (33) moves a projection position (P1) by driving the mirror unit (32) to change the orientation of the mirror unit (32).

[0193] According to this aspect, a moving mirror type projection system can realize a projection position (P1) that moves within the real space (A1).

[0194] The content generation method according to the fourth aspect is the second or third aspect, in which the generation process changes the orientation of an object (Ob1) in image content (D1) depending on the position of a display position (P3) in a virtual space (A2).

[0195] According to this aspect, it is also possible to deal with changes in the orientation of the object (Ob1) in the image content (D1).

[0196] The content generation method according to the fifth aspect is any of the second to fourth aspects, in which the generation process generates an image of an object (Ob1) displayed at a display position (P3) moving within a virtual space (A2) as seen from a reference position (P2) as image content (D1) consisting of a moving image.

[0197] According to this embodiment, it is possible to more easily generate image content (D1) suitable for projection onto a projection position (P1) that moves within the real space (A1).

[0198] A content generation method according to a sixth aspect is the method according to any one of the second to fifth aspects, further comprising a control information generation process. The control information generation process generates control information (D2) for a projector (30) for moving a projection position (P1) in a real space (A1) in synchronization with movement of a display position (P3).

[0199] According to this embodiment, it is easy to control the projector (30) to move the projection position (P1) in the real space (A1).

[0200] In the content generation method according to the seventh aspect, in any one of the first to sixth aspects, the image content (D1) is an image of the object (Ob1) in the virtual space (A2) with the background of the object (Ob1) removed.

[0201] According to this embodiment, the process of removing the background of the object (Ob1) in the virtual space (A2) becomes unnecessary.

[0202] A content generation method according to an eighth aspect is the content generation method according to any one of the first to seventh aspects, further comprising a providing process. The providing process provides an input support tool for allowing a user to input display position information in the second acquisition process. The input support tool presents at least a display position (P3) within the virtual space (A2).

[0203] According to this aspect, it becomes easier for the user to input the display position information.

[0204] In the content generation method according to the ninth aspect, in the eighth aspect, the input support tool presents the display position (P3) by displaying an image of a virtual space (A2) in which an object (Ob1) is displayed at the display position (P3).

[0205] According to this aspect, it becomes easier for the user to input the display position information.

[0206] In the content generation method according to the tenth aspect, in the eighth or ninth aspect, the input support tool displays, as a reference screen (T11), a rendering image of the object (Ob1) displayed at the display position (P3) as viewed from the reference position (P2) when the object (Ob1) is displayed at the display position (P3) in the virtual space (A2).

[0207] According to this aspect, the user can check the rendering image corresponding to the image content (D1) when inputting the display position information.

[0208] A content generation method according to an eleventh aspect is any one of the first to tenth aspects, wherein a guide range (R1) that serves as a guide for the range in which the object (Ob1) can be displayed is defined in the virtual space (A2).

[0209] According to this aspect, it becomes easier to specify the display position (P3) within the range in which the object (Ob1) can be displayed.

[0210] A content projection method according to a twelfth aspect includes a projection process in which an image content (D1) generated by the content generation method according to any one of the first to eleventh aspects is projected by a projector (30) onto a projection position (P1) in a real space (A1).

[0211] This embodiment has the advantage that the image content (D1) can be generated more easily.

[0212] A content projection method according to a thirteenth aspect is the twelfth aspect, further including a correction process. In the correction process, at least one of the installation state of the projector (30) and the control information (D2) of the projector (30) is corrected according to parameters used in the generation process.

[0213] According to this embodiment, the installation of the projector (30) is simplified.

[0214] A program according to a fourteenth aspect is a program for causing one or more processors to execute the content generation method according to any one of the first to eleventh aspects.

[0215] This embodiment has the advantage that the image content (D1) can be generated more easily.

[0216] A content generation system (10) according to a fifteenth aspect is a system for generating image content (D1) to be projected by a projector (30) at a projection position (P1) in a real space (A1), and includes a first acquisition unit (11), a second acquisition unit (12), a third acquisition unit (13), and a generation unit (14). The first acquisition unit (11) acquires spatial information related to a virtual space (A2) corresponding to the real space (A1). The second acquisition unit (12) acquires display position information related to a display position (P3) corresponding to the projection position (P1) in the virtual space (A2). The third acquisition unit (13) acquires reference position information related to a reference position (P2) in the virtual space (A2). The generation unit (14) generates, based on the spatial information, the reference position information, and the display position information, an image including the object (Ob1) as seen from the reference position (P2) when the object (Ob1) is displayed at the display position (P3) in the virtual space (A2), as image content (D1).

[0217] This embodiment has the advantage that the image content (D1) can be generated more easily.

[0218] Not limited to the above aspects, various aspects (including variations) of the content generation method according to the first and second embodiments can be embodied in a content generation system (10), a program, and a non-transitory recording medium on which the program is recorded.

[0219] The configurations according to the second to eleventh aspects are not essential for the content generation method and can be omitted as appropriate.

[0220] The configuration according to the thirteenth aspect is not an essential configuration for the content projection method, and can be omitted as appropriate. [Explanation of symbols]

[0221] 10 Content Generation System 11 First acquisition part 12 Second acquisition part 13 Third acquisition part 14 Generation part 30 Projector 31 Projection section 32 Mirror section 33 Drive unit A1 Real Space A2 Virtual Space D1 Image Content D2 Control Information Im1 Projected Image Ob1 object P1 projection position P2 reference position P3 display position R1 Reference Range T11 Reference Screen

Claims

1. 1. A method for generating image content to be projected by a projector at a movable projection position in a real space, comprising: A first acquisition process for acquiring space information regarding a virtual space corresponding to the real space; a second acquisition process for acquiring display position information regarding a display position corresponding to the projection position in the virtual space; a third acquisition process for acquiring reference position information regarding a reference position in the virtual space; a generation process for generating, as the image content, an image including the object as viewed from the reference position when the object is displayed at the display position in the virtual space, based on the spatial information, the reference position information, and the display position information; and generating control information for the projector to move the projection position in the real space in response to the movement of the display position. Content generation methods.

2. The projector, a projection unit that emits light for projecting a projection image using the image content; a mirror unit that reflects light emitted from the projection unit; a drive unit that drives the mirror unit so as to change the orientation of the mirror unit, thereby moving the projection position, The content generation method according to claim 1 .

3. In the generation process, a direction of the object in the image content is changed according to a position of the display position in the virtual space.

3. The content generating method according to claim 1 or 2.

4. In the generation process, an image of the object displayed at the display position moving in the virtual space as viewed from the reference position is generated as the image content consisting of a moving image. The content generating method according to any one of claims 2 to 3.

5. The image content is an image of the object excluding a background of the object in the virtual space. The content generating method according to any one of claims 1 to 4.

6. The method further includes a providing process for providing an input assistance tool for allowing a user to input the display position information during the second acquisition process, the input support tool presents at least the display position in the virtual space; The content generating method according to any one of claims 1 to 5.

7. The input assistance tool presents the display position by displaying an image of the virtual space in which the object is displayed at the display position. The content generating method according to claim 6.

8. The input support tool displays, as a reference screen, a rendering image of the object displayed at the display position as viewed from the reference position when the object is displayed at the display position in the virtual space. The content generating method according to claim 6 or 7.

9. A guideline range is defined in the virtual space as a guideline for a range in which the object can be displayed. The content generating method according to any one of claims 1 to 8.

10. A projection process for projecting the image content generated by the content generation method according to any one of claims 1 to 9 onto the projection position in the real space by the projector. Content projection method.

11. The method further includes a correction process for correcting at least one of an installation state of the projector and control information of the projector according to parameters used in the generation process. The content projection method of claim 10.

12. A program for executing the content generating method according to any one of claims 1 to 9 on one or more processors.

13. A system for generating image content to be projected by a projector at a movable projection position in real space, comprising: A first acquisition unit that acquires space information regarding a virtual space corresponding to the real space; a second acquisition unit that acquires display position information regarding a display position corresponding to the projection position in the virtual space; a third acquisition unit that acquires reference position information regarding a reference position in the virtual space; a generation unit that generates, as the image content, an image including the object viewed from the reference position when the object is displayed at the display position in the virtual space, based on the spatial information, the reference position information, and the display position information; and a control information generation process for generating control information for the projector for moving the projection position in the real space in response to the movement of the display position. Content generation system.

14. A method for generating image content to be projected by a projector at a movable projection position in real space, comprising: A first acquisition process for acquiring space information regarding a virtual space corresponding to the real space; a second acquisition process for acquiring display position information regarding a display position corresponding to the projection position in the virtual space; a third acquisition process for acquiring reference position information regarding a reference position in the virtual space; a generation process for generating, as the image content, an image including the object viewed from the reference position when the object is displayed at the display position in the virtual space, based on the spatial information, the reference position information, and the display position information; In the generation process, a direction of the object in the image content is changed in accordance with a position of the display position in the virtual space that changes due to the movement of the projection position. Content generation methods.

15. A method for generating image content to be projected by a projector at a projection position in real space, comprising: A first acquisition process for acquiring space information regarding a virtual space corresponding to the real space; a second acquisition process for acquiring display position information regarding a display position corresponding to the projection position in the virtual space; a third acquisition process for acquiring reference position information regarding a reference position in the virtual space; a generation process for generating, as the image content, an image including the object viewed from the reference position when the object is displayed at the display position in the virtual space, based on the spatial information, the reference position information, and the display position information; The image content is an image of the object excluding the background of the object in the virtual space. Content generation methods.

Images