Image output system and computer program

Abstract

We provide an image output system that allows users to experience a sense of being tiny. [Solution] The image output system includes a first shooting means for photographing a main subject, a second shooting means for photographing a user operating the first shooting means, and an image synthesis means that creates a composite image by combining the image of the main subject acquired by the first shooting means and the image of the user acquired by the second shooting means, and adjusts at least one of the size of the main subject and the size of the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user.

Description

Technical Field

[0001] The present invention relates to an image output system, a computer program, and the like.

Background Art

[0002] Patent Document 1 describes a photographing apparatus that outputs (prints) an image in which an image of a visitor (user) is superimposed on a background image, with the background image being an image of an animal photographed at a zoo, aquarium, or the like.

[0003] In the photographing apparatus described in Patent Document 1, for example, a background image is obtained by photographing a moving image of a dolphin swimming in an aquarium to display a live view image and then photographing a still image of this dolphin at a desired timing. Furthermore, a composite image can be obtained by synthesizing an image of the user of the photographing apparatus and the above background image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with the photographing apparatus described in Patent Document 1, it has been difficult to generate a composite image that gives a strong impression such as a sense of cuteness, for example.

[0006] Therefore, one object of the present invention is to provide an image output system that can give a sense of cuteness.

Means for Solving the Problems

[0007] To achieve the above object, an image output system according to the present invention includes a first photographing means for photographing a main subject, A second shooting means for photographing a user operating the first shooting means, Image synthesis means that creates a composite image by combining the image of the main subject acquired by the first shooting means and the image of the user acquired by the second shooting means, and adjusts at least one of the size of the main subject and the size of the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user. It is characterized by having the following features. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an image output system that can give the impression of being a miniature person. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example configuration of the image output system IOS according to Embodiment 1. [Figure 2] This flowchart shows an example of the processing flow for a shooting experience using the image output system according to Embodiment 1. [Figure 3] (A) and (B) are diagrams illustrating examples of when a user issues an experience ticket. [Figure 4] (A) and (B) are diagrams illustrating how users experience taking photos. [Figure 5] This figure shows an example of the display screen DS of the display DIA in step S12. [Figure 6] (A) and (B) are diagrams illustrating an example of a user printing a composite image after a photo shoot. [Figure 7] This flowchart shows specific examples of image synthesis in steps S9 to S15. [Figure 8] (A) and (B) are diagrams illustrating examples of the concept of creating composite images. [Figure 9] This is a diagram illustrating a method for estimating the size of the main subject. [Figure 10] It is a diagram showing a modified example of a method for creating a composite image. [Figure 11] It is a diagram showing another modified example of creating a composite image. [Figure 12] It is a diagram showing an example of the concept of creating a composite image in yet another modified example.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiments. In each figure, the same members or elements are given the same reference numerals, and redundant descriptions are omitted or simplified. [Embodiment 1]

[0011] FIG. 1 is a diagram showing a configuration example of an image output system IOS according to this embodiment. Note that some of the functional blocks shown in FIG. 1 are realized by causing a CPU or the like as a computer included in the image output system (not shown) to execute a computer program stored in a memory as a storage medium (not shown).

[0012] However, some or all of them may be realized by hardware. As the hardware, a dedicated circuit (ASIC) or a processor (reconfigurable processor, DSP) or the like can be used. Also, each functional block shown in FIG. 1 does not have to be built in the same housing, and may be constituted by separate devices connected to each other via signal paths.

[0013] The image output system IOS includes a camera unit (camera device) CU including a holder (holding device) HA and an imaging device OIA, and a pipe (tubular member) TM into which the camera unit CU is inserted. Note that the imaging device OIA functions as a first imaging means for photographing a main subject.

[0014] The image output system IOS further includes a driving device (moving device) DRA provided within the pipe TM. The pipe TM, the camera unit CU, and the driving device DRA are provided in the imaging area.

[0015] The camera unit CU can be moved inside the pipe TM, at least part of which is transparent, by the driving device DRA, for example, using pneumatic pressure. Note that the pipe TM does not have to be entirely transparent. If the position and direction that can be photographed by the camera unit are restricted in advance, it only needs to be transparent at the restricted location.

[0016] Here, the imaging area is, for example, an animal breeding area or an aquarium, etc., but is not limited thereto. Also, for example, at least part of the driving device DRA may be provided outside the imaging area.

[0017] The image output system IOS further includes a server (server device, server means) SA, a battery (power supply device) BA, and a sensor (detection device, detection means) DEA. These may be provided outside the imaging area.

[0018] The image output system IOS further includes a display (display device, display means) DIA, an operation device (operation means) OA, an imaging device IIA, a speaker (acoustic device, acoustic means) AA, etc. These are provided, for example, inside an experience booth. Note that the imaging device IIA functions as a second imaging means for photographing a user who operates the first imaging means.

[0019] The image output system IOS further includes a printer (printing device, printing means) PA installed, for example, near an experience booth, and also includes a control device CA for controlling each of the components described above.

[0020] The holder HA is configured to allow the imaging device OIA to move within the pipe TM and to hold the imaging device IA in a detachable manner, and functions as a holding means. The imaging device OIA comprises an imaging optical system and an image sensor for imaging the outside of the pipe TM.

[0021] Furthermore, the imaging device OIA may have a camera body, and the imaging optical system may consist of a lens that is detachable from the camera body. Alternatively, the camera body and lens may be integrated.

[0022] The drive unit DRA can move the camera unit CU within the pipe TM using, for example, pneumatic pressure. Furthermore, the drive unit DRA is configured to rotate the camera unit CU within the pipe TM using the direction of movement as the axis of rotation.

[0023] In other words, the camera unit CU moves inside the pipe TM along its shape (a predetermined path) by the drive unit DRA. Here, the drive unit DRA functions as a moving means for moving the imaging means, and the moving means can move the imaging means along the predetermined path.

[0024] Server SA can receive images captured by imaging devices OIA and IIA, and can also transmit those images to display DIA and printer PA. Furthermore, Server SA can receive signals from control devices OA, such as controllers, located near display DIA, and transmit the received signals to imaging device OIA and drive device DRA. Server SA also stores the audio data that forms the basis of the sound output by sound devices AA, such as speakers.

[0025] This configuration allows users of the IOS image output system, for example, at an experience booth in a zoo or at home, to operate the OIA imaging device and DRA drive device via the OA control device. Furthermore, users can photograph animals or other subjects under their desired shooting conditions and timing.

[0026] The imaging device OIA and imaging device IIA may be connected to the server SA by wire or by wireless connection. When the imaging device OIA and imaging device IIA are connected to the server SA wirelessly, the imaging device OIA and imaging device IIA may be equipped with a communication control component (communication chip) for communicating with the server SA. Alternatively, the communication control component may be provided inside the holder HA, and this communication control component and the imaging device OIA may be connected by wire inside the holder HA.

[0027] Battery BA is a power supply device for supplying power to the drive unit DRA. In this embodiment, the imaging device OIA is driven by an internal power supply, but the imaging device OIA may also be configured to receive power from battery BA.

[0028] Sensor DEA is connected to the imaging device OIA and the drive unit DRA, and is configured to detect the position, shooting direction, and field of view of the imaging device OIA within its pipe TM. Here, sensor DEA functions as a position detection means to detect the position of the imaging device which changes with the movement means, a direction detection means to detect the shooting direction of the imaging device, and a field of view detection means to detect the field of view of the imaging device.

[0029] In this embodiment, the DEA sensor combines the functions of position detection, direction detection, and field of view detection. However, it is sufficient for the image output system IOS to have these position detection, direction detection, and field of view detection functions.

[0030] Therefore, for example, the imaging device OIA may have a field of view detection function, and the sensor DEA may have a position detection function and a direction detection function based on the drive information of the drive device DRA. Furthermore, the sensor DEA may include a GPS sensor mounted on the camera unit.

[0031] Furthermore, the display DIA shows images captured by the imaging device OIA. The operating device OA is used to operate the imaging device OIA, the drive unit DRA, and the imaging device IIA. The operating device OA has shutter buttons for taking images with the imaging device OIA and imaging device IIA, and operating buttons for moving or rotating the imaging device OIA within the pipe TM. The operating device OA also has zoom buttons for the imaging device OIA and imaging device IIA.

[0032] The imaging device IIA is, for example, used to photograph users inside an experience booth, and is installed, for example, on the wall of the experience booth facing the room. At least one of the shooting direction and field of view of the imaging device IIA may be fixed or variable.

[0033] Audio equipment such as speakers (AA) are used to transmit information to users in the experience booth via voice, for example, by making a welcome announcement when a user enters the experience booth, or an announcement that the photo experience has ended after a set amount of time has elapsed.

[0034] A PA printer is installed near the experience booth (of course, the location of the PA printer is not limited to the experience booth). The PA printer has the function of issuing experience tickets and outputting (printing) images.

[0035] Furthermore, the control device CA incorporates a CPU and other components as a computer, and functions as a control means that controls the operation of each part of the imaging system IS based on computer programs stored in the memory as a storage medium. In addition, the control device CA also functions as an image synthesis means that creates a composite image by combining the image of the main subject acquired by the first imaging means and the image of the user acquired by the second imaging means.

[0036] Furthermore, the control unit CA may be integrated with the server SA, or they may be separate units. In addition, the control unit CA and server SA may be installed in a physically relatively close range to the other components of the imaging system IS (for example, within a facility such as a zoo), or in a relatively distant range (domestic or international).

[0037] Figure 2 is a flowchart showing an example of the processing flow of a shooting experience using the image output system according to Embodiment 1. The flow of the shooting experience and image output will be explained using Figure 2 and other figures. Note that the CPU and other components of the computer within the image output system execute computer programs stored in memory, which sequentially perform each step of the flowchart in Figure 2.

[0038] Step S1 is a step in which the control device CA checks whether or not an experience ticket has been issued. If it is determined to be No, the process returns to step S1; if it is determined to be Yes, the process proceeds to step S2. The experience ticket can be obtained from the printing unit PA1 of the printer PA shown in Figure 3(A).

[0039] Figures 3(A) and 3(B) illustrate an example of when a user issues an experience ticket. Figure 3(A) shows the case where there are three users, U1, U2, and U3. When any of the users apply via the touch panel PA2 of the printer PA, an experience ticket TT is issued from the printing unit PA1.

[0040] As shown in Figure 3(B), the experience ticket TT has the user ID information UID and the QR code QRC printed on it. Note that QR code is a registered trademark. In addition to this information, the experience ticket TT may also have date information, experience time information, etc. printed on it.

[0041] For example, if, after the experience start time indicated on the experience ticket TT, the user ID information UID printed on the experience ticket TT is entered by the user into the operating device OA, or if the QR code QRC on the experience ticket is read, that information is transmitted to the control device CA.

[0042] Step S2 is a step in which the control device CA checks whether it has recognized the user ID information written on the experience ticket TT. In other words, in step S2, the control device CA checks whether the experience start time and user ID information UID are in the correct relationship. If the result in step S2 is No, the process returns to step S2; if the result is Yes, the process proceeds to step S3.

[0043] In step S3, the control unit CA activates the battery BA, which in turn activates the camera unit CU, enabling the shooting experience to begin.

[0044] Step S4 is a step in which the control device CA checks whether the user pressed the shutter button on the operating device OA during the shooting experience. If the result in step S4 is No, the process returns to step S4; if the result is Yes, the process proceeds to step S5.

[0045] Figures 4(A) and 4(B) illustrate how users experience taking photos, showing examples of experience booths and the process of taking photos.

[0046] The experience booth shown in Figure 4(A) is a private or semi-private room, and is equipped with a display DIA, an operating device OA, an imaging device IIA (not shown), and an acoustic device AA (not shown). In the state shown in Figure 4(A), user U1 is operating the camera unit CU using the operating device OA, while users U2 and U3 are observing.

[0047] In the example of the experience booth shown in Figure 4(B), it is installed in an open environment such as a corridor, and, like in Figure 4(A), it is equipped with a display DIA and an operating device OA. In Figures 4(A) and (B), the imaging device IIA may be a webcam installed integrally with the display DIA or as a separate unit. Also, the sound device AA may be an in-house sound system such as in a zoo or aquarium (of course, it may also be a small speaker installed near the user).

[0048] Step S5 is the step in which the imaging device OIA takes an image, and it functions as the first imaging step for capturing the main subject. Taking Figure 4(B) as an example, user U1 observes the main subject MS (in this case, a shark) by looking directly at the display DIA or the aquarium in front of them, and moves or rotates the camera unit CU within the pipe TM as needed.

[0049] Once the desired composition is achieved, user U1 presses the shutter button (or equivalent button) on the imaging device OIA located on the control device OA, thereby capturing a photograph of the main subject MS and acquiring a still image of the main subject. The acquired still image of the main subject is then transmitted to, for example, the server SA for storage.

[0050] Step S6 is a step in which the control device CA checks whether a predetermined amount of time has elapsed since the start of the shooting experience. For example, if one shooting experience is set to last 10 minutes, the control device CA checks whether 10 minutes have elapsed or whether it is a time slightly before the actual end time of the experience, such as 9 minutes.

[0051] If the result in step S6 is No, the process returns to step S4, and it is determined again whether the shutter button was pressed or not. If the result in step S6 is Yes, the process proceeds to step S7. In this way, within a predetermined time, the camera unit CU can be moved or rotated within the pipe TM to take multiple images.

[0052] Step S7 is a step in which, if it is confirmed in step S6 that a predetermined time has elapsed, the user is notified that the shooting experience has ended. The notification may be made by voice using the sound device AA, or by displaying an image or text on the display DIA.

[0053] Step S8 is a step in which the control device CA confirms with the user whether they want to select image output and whether that selection has been made. More specifically, for example, after the shooting experience is completed, the display DIA displays an option to output (download or print) the images taken during the shooting experience.

[0054] Furthermore, users will be able to select their preferred option from these choices using the operating device (OA). Image output may be offered free of charge, or it may incur an additional fee separate from the shooting experience fee.

[0055] If the user does not select image output, the process proceeds to step S15, where the user is notified via the sound device AA or display DIA that the experience has ended, prompts them to move out of the experience booth, and then the flow shown in Figure 2 ends. If the answer in step S8 is "Yes," the process proceeds to step S9.

[0056] Step S9 is a step in which, if it is confirmed that image output was selected in step S8, preparations are made to photograph the user inside the experience booth with the imaging device IIA. The timing of the photograph may be determined by a timer, or the user may choose to take the photograph at their desired timing by pressing the shutter button (or equivalent button) on the imaging device IIA provided on the control device OA.

[0057] In the latter method, in step S10, the control device CA checks whether the shutter button (or equivalent button) of the imaging device IIA, located on the operating device OA, has been pressed. If the result is No, the process returns to step S10; if the result is Yes, the process proceeds to step S11, where the user takes a still image.

[0058] Furthermore, still images of the user themselves are also sent to and saved on the server SA, for example. Here, step S11 functions as a second shooting step for capturing images of the user performing the first shooting step.

[0059] Step S12 is the step in which the user selects an image they wish to output from among multiple images captured by the imaging device OIA, and the selection operation is accepted. The processing in step S12 is performed by a user interface as shown in Figure 5. Figure 5 shows an example of the display screen DS of the display DIA in step S12.

[0060] In step S12, a display screen DS, as shown in Figure 5, is displayed on the display DIA. The display screen DS shown in Figure 5 displays multiple images (external images) OI1, OI2, and OI3 taken by the user using the imaging device OIA, and an image of the user themselves (internal image) II taken by the imaging device IIA. These images are acquired, for example, from the server SA.

[0061] Furthermore, a selection frame SF is displayed for the user to select the image they wish to output from among multiple images OI1, OI2, and OI3. The selection frame SF can be operated by the control device OA, and the user selects the image they wish to output by moving the selection frame SF to the desired image based on the image status of the main subjects MS1, MS2, and MS3 in each image.

[0062] In Figure 5, the main subjects MS1, MS2, and MS3 are all cheetahs. The reference information TI notifies the user of text prompting them to select the image they wish to output (e.g., "Choose the image you want to take home!!" in Figure 5).

[0063] In step S12, a composite image is generated, and the result of combining the main subject image selected by the user and the user image II is displayed in the composite image display frame CIF. Here, step S12 functions as an image compositing step that creates a composite image by combining the main subject image captured in the first shooting step and the user image captured in the second shooting step. The method for creating the composite image will be described later.

[0064] Step S13 is an image editing step where the user can adjust the color and brightness of the composite image, or add text or images to the composite image.

[0065] Step S14 is the step in which the control device CA outputs the composite image to the server SA. At this time, the control device CA uploads the composite image CI to the server SA.

[0066] Figures 6(A) and 6(B) illustrate an example of a user printing a composite image after a shooting experience. After the experience completion notification in step S15, the user operates the touch panel PA2 of the printer PA to print the composite image CI from the printing unit PA1, as shown in Figures 6(A) and 6(B). Alternatively, the composite image CI may be printed automatically after the experience completion notification in step S15.

[0067] Furthermore, by scanning the QR code QRC printed on the experience ticket TT with a smartphone or similar device, users may be redirected to a designated website. This would allow users to later view or download images OI1, OI2, OI3, etc., shown in Figure 5, using their smartphone or similar device, as shown in Figure 6(B). Additionally, users may be able to later view or download the combined image.

[0068] Figure 7 is a flowchart illustrating a specific example of composite image creation in steps S9 to S15, showing each step in creating the composite image. The CPU and other components within the image output system execute computer programs stored in memory, sequentially performing the actions shown in the flowchart of Figure 7. Note that the order of steps S9 to S15 in the flowchart of Figure 7 differs slightly from that of steps S9 to S15 in Figure 2.

[0069] Step S16 is the step in which the control device CA acquires the image captured by the imaging device OIA from the server SA. The image acquired by the control device CA here is, for example, an image of the main subject, such as an animal, captured by the imaging device OIA. Of course, it may also be a landscape image or something similar that does not contain any animals.

[0070] Step S17 is the step in which the control device CA determines whether there are multiple main subject images acquired by the control device CA from the server SA in step S16. If the user takes multiple main subject images using the imaging device OIA during the predetermined experience time, the control device CA acquires all of the multiple main subject images from the server SA.

[0071] If the result in step S17 is No, meaning there is only one main subject image, proceed to step S20. If the result in step S17 is Yes, proceed to step S18.

[0072] Step S18 is the step in which the control device CA displays candidate main subject images for synthesis on the display DIA. In step S18, multiple candidate main subject images for synthesis are displayed on the display DIA. These candidates correspond to, for example, the multiple images OI1, OI2, and OI3 shown in Figure 5.

[0073] Step S19 is a step in which the control device CA confirms whether a main subject image for synthesis has been selected by the user, for example, by operating the selection frame SF mentioned above. In step S20, the control device CA determines the main subject image for synthesis according to the selection result in step S19. If there is only one main subject image, the control device CA determines that image to be the main subject image for synthesis in step S20.

[0074] Then, in step S21, the control device CA displays the determined main subject image for synthesis on the display DIA.

[0075] Step S22 is the step in which the imaging device IIA takes an image of the user. This corresponds to steps S9 to S11 mentioned above. Note that in step S22, the imaging device IIA may take multiple images of the user, or it may take an image only once.

[0076] Step S23 is the step in which the control device CA determines whether there are multiple user images captured by the imaging device IIA in step S22. If the control device CA determines "No" in step S23, the process proceeds to step S26. If it determines that there are multiple user images, the process proceeds to step S24.

[0077] In step S24, the control unit CA causes the multiple user images to be displayed on the display DIA.

[0078] Step S25 is a step in which the control device CA confirms whether a user image for synthesis has been selected from multiple user images. The method for selecting a user image for synthesis from multiple user images can be the same as the method used for the selection frame SF described above. If the result in step S25 is No, the process returns to step S25; if the result is Yes, the process proceeds to step S26.

[0079] In step S26, the control device CA determines the user image to be synthesized according to the selection result in step S25. If there is only one user image, the control device CA determines that image to be used as the user image to be synthesized in step S26.

[0080] Then, in step S27, the control device CA displays the determined user image for synthesis on the display DIA.

[0081] Step S28 functions as an image synthesis step in which the control device CA adjusts the size of the main subject image and the user image for synthesis to create a composite image. The adjustment of these sizes will be explained using Figures 8(A) and (B).

[0082] Figures 8(A) and 8(B) are diagrams to illustrate an example of the concept of creating a composite image, and consider the case shown in Figure 8(A) as a hypothetical example. As shown in Figure 8(A), consider the case in real space where the height (distance from the ground to the top of the head) of the main subject MSR, a quadrupedal animal, in its normal posture is the same as the height of the user UR.

[0083] If we were to simply combine the main subject image OI and user image II, both taken from images of the two individuals standing at the same distance from the imaging device, the resulting composite image CIA would simply show the main subject MSR and user UR standing side by side at the same height, as shown in Figure 8(A).

[0084] However, this does not result in a memorable image, so in this embodiment, a composite image is created by intentionally altering the relative sizes of the two in real space. More specifically, as shown in Figure 8(B), the height of the main subject MSR is kept the same as the main subject image OI, and the height of the user UR is deliberately made smaller than the height in the user image II, and a composite image CIB is created by combining it with the main subject image OI.

[0085] This allows for the creation of images where the scale of the main subject, such as animals, and the visitors (users) differs from the scale in real space, making visitors feel as if they have become tiny (give the feeling of being tiny).

[0086] Although Figure 8(B) illustrates the case where the user image II is reduced in size, this embodiment is not limited to this, and the main subject image OI may also be enlarged. In other words, a composite image can be created in which the user is relatively reduced in size relative to the main subject, or in which the main subject is relatively enlarged relative to the user.

[0087] Thus, in step S28, the image synthesis means adjusts at least one of the sizes of the main subject and the user so that the size ratio of the main subject to the user in the synthesized image is greater than the actual size ratio of the main subject to the user.

[0088] Furthermore, if the actual size of the main subject is smaller than the actual size of the user, the composite image will be created so that the main subject is displayed larger than the user.

[0089] One method for estimating the size of the main subject is image recognition. For example, consider a case where this image output system IOS is installed in a zoo. In this case, the server SA would have a database containing data on the species (lion, cheetah, etc.) and size (average total length, average values ​​of maximum head length and width, etc.) of each animal exhibited in the zoo.

[0090] The control device CA then recognizes the type of main subject in the image captured by the imaging device OIA, and retrieves the size of the main subject from the database held by the server SA. Thus, in this embodiment, the image synthesis means can obtain information regarding the size of candidate main subjects from the server device, etc. Furthermore, the information regarding the size of candidate main subjects includes information regarding the size for each type of main subject.

[0091] In this case, the size data can be for each individual animal rather than for each species. For example, if the zoo has lion A and lion B on display, server SA can store data on the size of lion A and data on the size of lion B.

[0092] If the control device CA recognizes that lion A is visible in the image captured by the imaging device OIA, it may retrieve data regarding the size of lion A from the database held by the server SA. Thus, information regarding the size of candidate primary subjects includes information regarding the size of each individual primary subject.

[0093] Alternatively, a second method for estimating the size of the main subject is to calculate its size from the distance from the imaging device to the main subject, etc.

[0094] Figure 9 is a diagram illustrating a method for estimating the size of the main subject. As shown in Figure 9, let F be the focal length of the lens inside the imaging device OIA, D1 be the size of the image on the light-receiving surface of the image sensor in the figure 9, L be the distance from the lens of the imaging device OIA to the main subject MS, and D2 be the size of the main subject MS in the figure 9. In this case, the following equation (1) holds. D2 / L = D1 / F...(1)

[0095] The size D2 of the main subject can be calculated based on equation (1) above. In other words, the image synthesis means can also obtain information about the actual size of the main subject based on equation (1).

[0096] Furthermore, information regarding the focal length F can be obtained by the control device CA from the imaging device OIA. Alternatively, the distance L may be obtained by the imaging device OIA using a distance sensor, and then transmitted to the control device CA. Additionally, the image size D1 on the light-receiving surface of the image sensor can be obtained from the image.

[0097] Thus, the image synthesis means can estimate the size of the main subject based on the focal length of the lens provided by the first shooting means, the size of the subject image on the light-receiving surface of the image sensor provided by the first shooting means, and the distance from the first shooting means to the main subject.

[0098] The same method can be used to estimate the size of the users. As shown in Figures 4(A) and (B), some of the users are seated in chairs, so the distance from the imaging device IIA to the users seated in those chairs can be considered to be approximately constant.

[0099] Therefore, the size of the user can be expressed by replacing each term in equation (1) with values ​​relating to the user and the imaging device IIA, and the distance from the imaging device IIA to the user can be set to a specified value. Furthermore, since it is assumed that the imaging device IIA will not be replaced frequently, the terms relating to the imaging device IIA in equation (1) can be considered to be generally constant.

[0100] Therefore, the user's size may be set to a predetermined value. In this embodiment, the image synthesis means can acquire information regarding the user's size based on equation (1), etc.

[0101] In step S29, the control device CA displays the composite image created in step S28 on the display DIA. Step S30 corresponds to step S13 and is a step in which the user checks the composite image displayed on the display DIA and edits the composite image using the operating device OA as needed. In step S30, the user adjusts the color and brightness of the composite image, or adds text or images to the composite image.

[0102] Step S31 corresponds to step S14, and is the step in which the control device CA outputs a composite image based on the editing results in step S30 (or the image displayed in step S29 if no editing was done).

[0103] Furthermore, in this context, "image output" means that the control device CA sends the composite image to the server SA. This allows the composite image to be viewed and downloaded on a smartphone or other device via the QR code QRC printed on the experience ticket TT. Additionally, "image output" may also include the printing of the composite image by the printer PA's printing unit PA1, or preparation for such printing.

[0104] Step S32 corresponds to step S15, and the control device CA displays text information, images, etc., on the display DIA indicating that the shooting experience has ended. This completes the series of shooting experiences.

[0105] Furthermore, the shooting system IS of this embodiment may display the image being captured by the imaging device OIA, as well as the movement path, position detection results, direction detection results, and field of view detection results of the camera unit CU (imaging device OIA), on the display DIA. This makes it easier to visually grasp the current shooting position and shooting direction of the camera unit.

[0106] In this embodiment, as shown in Figure 5, the user can select a main subject image for synthesis from multiple main subject images taken by the user. However, there may be cases where the user has taken only one main subject image, or where the user is unable to take a main subject image with the desired composition within the predetermined shooting time.

[0107] In such cases, in addition to the main subject image taken by the user, sample main subject images pre-stored on the server SA may also be displayed on the display DIA as candidates for the main subject image to be combined. Furthermore, the user may be allowed to select the main subject image to be combined from these sample main subject images.

[0108] Furthermore, as shown in Figure 8(B), in this embodiment, when combining the main subject image OI and the user image II, the user image is made smaller during the combination process, but the combined image CIB was created without any particular changes to the positions of the main subject and the user. However, when creating the combined image, it may be possible to adjust not only the relative sizes of the main subject and the user but also their relative positions as appropriate.

[0109] Figure 10 shows a modified example of the composite image creation method. For example, as shown in Figure 10, at least a portion of the opening MP, such as the mouth of the main subject MS2, may be made to overlap with the user U2 in the composite image CI.

[0110] Specifically, when combining the main subject image OI and the user image II, the user is relatively reduced in size, as shown from U1 to U2. Furthermore, when combining the main subject image OI and the user image II, the position, angle, and orientation of the main subject MS2 are adjusted so that the aperture MP of the main subject MS1 in the combined image CI is closer to the user U2.

[0111] Furthermore, in the composite image CI, it is not necessary for at least a portion of the aperture MP of the main subject MS2 to overlap with the user U2. The composite image CI should be such that the aperture MP is close to the user U2 and the user appears to be about to be devoured by the main subject. In other words, the image synthesis means should create (adjust) the composite image so that the aperture, such as the mouth of the main subject, is close to the user.

[0112] By automatically performing such adjustments and synthesis when the control device CA creates the composite image, users can take home more impressive images. Alternatively, such adjustments and synthesis may be performed automatically by the control device CA using machine learning or similar methods.

[0113] Alternatively, in step S30, when the user edits the composite image, the control device CA may display a message on the display DIA as shown in Figure 10, allowing the user to perform the adjustments and composite themselves. Furthermore, at that time, image information such as text information or indicators may be displayed to facilitate the user's adjustments and composites.

[0114] Furthermore, instead of, or in addition to, the adjustments and synthesis shown in Figure 10, the control device CA or the user may perform the adjustments and synthesis shown in Figure 11.

[0115] Figure 11 shows another variation of image synthesis, illustrating an example of adjustment and synthesis focusing on the eyes of the main subject. In the image CI shown in Figure 11, the pupil PP2 of the eye EP of the main subject MS3 is adjusted to face the direction of the user U2.

[0116] In other words, when combining the main subject image OI and the user image II, the user is relatively reduced in size from U1 to U2, as described above. Furthermore, the position, angle, and orientation of the main subject MS3 are adjusted so that the eye area EP of the main subject in the combined image CI is closer to the user U2.

[0117] Furthermore, the position of the pupil PP2 can be adjusted to bring it closer to the user U2. Such adjustments can produce a composite image CI that gives the impression the user is being stared at by the main subject.

[0118] Thus, in the example shown in Figure 11, the image synthesis means creates a composite image so that the eyes of the main subject are closer to the user. Furthermore, the composite image is created while changing the position of the pupils of the main subject in the image of the main subject so that the pupils of the main subject are closer to the user.

[0119] Furthermore, instead of, or in addition to, the adjustments and synthesis shown in Figures 10 and 11, the adjustments and synthesis shown in Figure 12 may be performed by the control device CA or the user.

[0120] Figure 12 shows an example of composite image creation in yet another modified example, illustrating an example of adjustment and synthesis focusing on the legs of the main subject. In the composite image CI shown in Figure 12, adjustments have been made so that the user U3 is positioned below the legs LP of the main subject MS4.

[0121] In other words, when combining the main subject image OI and the user image II, the user is relatively reduced in size as shown from U1 to U3, as described above, and the position, angle, and orientation of the main subject MS4 are adjusted so that the legs LP of the main subject are closer to the user U3. That is, the image combining means creates a combined image so that the legs of the main subject are closer to the user.

[0122] Furthermore, the position of the main subject is adjusted so that the user U3 is positioned below the leg LP. In other words, a composite image is created so that the user is positioned below the leg of the main subject. By making such adjustments, it is possible to output a composite image CI that gives the impression that the user is about to be stepped on by the main subject.

[0123] Furthermore, when acquiring user images for adjustment and synthesis as shown in Figures 10 to 12, that is, when the imaging device IIA photographs the user, the control device CA may display text information, image information, etc., on the display DIA prompting the user to assume a predetermined pose or facial expression.

[0124] In other words, for example, if a user wishes to generate a composite image as shown in Figure 10, when the imaging device IIA takes a picture of the user, the control device CA displays text information or image information on the display DIA prompting the user to make a pose such as trying to escape, a surprised pose, a frightened expression, or a surprised expression.

[0125] The control means in this embodiment prompts the user to strike a predetermined pose or make a predetermined facial expression when the second shooting means is photographing the user, allowing the user to take home a more memorable composite image, or to download a composite image from a website.

[0126] In this embodiment, the user image is acquired in step S22 after determining the main subject image for synthesis, as shown in Figure 7. However, the user image may be acquired while the main subject is being photographed.

[0127] In other words, during the shooting experience, the user may use the control device OA to photograph the main subject, such as an animal, with the imaging device OIA at the same time that the imaging device IIA automatically photographs the user, thereby simultaneously acquiring the main subject image and the user image as a pair.

[0128] Furthermore, instead of separately displaying the candidate main subject image for synthesis in step S18 and the candidate user image for synthesis in step S24, the above pair may be displayed. That is, the pair of main subject image and user image acquired simultaneously during the shooting experience may be displayed simultaneously on the display DIA as the candidate main subject image for synthesis and the candidate user image for synthesis.

[0129] Alternatively, if predetermined conditions are met during the shooting experience, the imaging device IIA may automatically photograph the user and acquire a user image. That is, the imaging device IIA may continuously monitor the user's facial expression in video shooting mode, and if the control device CA detects that the user's facial expression has changed by a predetermined amount or more, such as when the user is surprised or laughs, the imaging device IIA may acquire a user image at that moment. In this way, the second shooting means may be configured to photograph the user when it is detected that the user's facial expression has changed by a predetermined amount or more.

[0130] Furthermore, while automatically acquiring user images based on changes in facial expression, the control device CA may also control the imaging device OIA to automatically acquire images of the main subject. That is, the first shooting means may take a picture of the main subject when it is detected that the user's facial expression changes to a predetermined amount or more.

[0131] Alternatively, if the control device CA detects that the user has made a sound above a predetermined volume, the imaging device IIA may acquire an image of the user or the main subject at that time. In other words, the second imaging means may photograph the user when it is detected that the user's voice has reached a predetermined volume or higher.

[0132] Furthermore, the first shooting means may take a picture of the main subject when it is detected that the user's voice has reached a predetermined volume or higher. A microphone (volume detection device, volume detection means) may also be provided in the experience booth to detect the user's voice.

[0133] By automatically acquiring user images and main subject images based on facial expression changes and sound detection, users can take home impressive images even if they are unable to acquire a main subject image with the desired composition within the allotted time. Alternatively, they can download a composite image from the website.

[0134] Furthermore, as shown in Figure 5, when displaying candidate main subject images for synthesis and user images on the display DIA, only main subject images that meet predetermined conditions may be displayed from among the multiple main subject images acquired from the imaging device OIA. That is, only some of the main subject images taken by the user within a predetermined time may be displayed on the display DIA as candidates for main subject images for synthesis.

[0135] For example, from among multiple main subject images acquired from the imaging device OIA, only those main subject images in which the main subject occupies a predetermined proportion (e.g., half) or more of the area may be displayed on the display DIA as candidates for the main subject image to be combined.

[0136] Alternatively, from among the multiple main subject images acquired from the imaging device OIA, only those main subject images in which the main subject is not visible and where the area for placing the user image is greater than a predetermined proportion may be displayed on the display DIA as candidates for the main subject image to be combined.

[0137] Furthermore, the area for placing the user image described above may be a predetermined shape, such as a rectangle or an ellipse. This allows for efficient selection of the main subject image for synthesis, even when the user has taken multiple main subject images.

[0138] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to the above embodiments, and various modifications and combinations of the above embodiments are possible in accordance with the spirit of the present invention, and these are not excluded from the scope of the present invention.

[0139] Furthermore, the present invention includes, for example, a system that realizes the functions of the above embodiment using at least one processor such as a CPU, memory, and circuitry (e.g., an ASIC). Alternatively, multiple processors may be used for distributed processing.

[0140] Furthermore, in order to realize some or all of the control in the above embodiment, a computer program that realizes the functions of the above embodiment may be supplied to an image output system, etc., via a network or various storage media. The computer (or CPU, MPU, etc.) in the image output system, etc., may then read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention. Furthermore, the present invention includes the following combinations.

[0141] (Configuration 1) An image output system characterized by comprising: a first shooting means for photographing a main subject; a second shooting means for photographing a user operating the first shooting means; and an image synthesis means for creating a composite image by combining an image of the main subject acquired by the first shooting means and an image of the user acquired by the second shooting means, and adjusting at least one of the size of the main subject and the size of the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user.

[0142] (Configuration 2) The image output system according to Configuration 1, characterized in that the image synthesis means is capable of acquiring information regarding the actual size of the main subject.

[0143] (Configuration 3) The image output system according to Configuration 2, characterized in that the image synthesis means can estimate the size of the main subject based on the focal length of the lens provided by the first shooting means, the size of the subject image on the light-receiving surface of the image sensor provided by the first shooting means, and the distance from the first shooting means to the main subject.

[0144] (Configuration 4) The image output system according to Configuration 2 or 3, characterized in that the image synthesis means can obtain information regarding the size of the candidate main subject from the server device.

[0145] (Configuration 5) The image output system according to Configuration 4, characterized in that the information regarding the size of the candidate main subject includes information regarding the size of each type of main subject.

[0146] (Configuration 6) The image output system according to Configuration 4 or 5, characterized in that the information regarding the size of the candidate main subject includes information regarding the size of each individual main subject.

[0147] (Configuration 7) The image synthesis means is capable of acquiring information regarding the user's size. An image output system according to any one of configurations 1 to 6, characterized by the above.

[0148] (Configuration 8) The image output system according to any one of Configurations 1 to 7, characterized in that, when the actual size of the main subject is smaller than the actual size of the user, the image synthesis means creates the synthesized image such that the main subject is displayed larger than the user in the synthesized image.

[0149] (Configuration 9) The image output system according to any one of Configurations 1 to 8, characterized in that the image synthesis means creates the synthesized image such that the opening of the main subject is closer to the user.

[0150] (Configuration 10) The image output system according to any one of Configurations 1 to 9, characterized in that the image synthesis means creates the synthesized image so that the eye portion of the main subject is closer to the user.

[0151] (Configuration 11) The image synthesis means creates the synthesized image while changing the position of the pupil of the main subject in the image of the main subject so that the pupil of the main subject is closer to the user. The image output system according to configuration 10, characterized by the above.

[0152] (Configuration 12) The image output system according to any one of Configurations 1 to 11, characterized in that the image synthesis means creates the synthesized image such that the legs of the main subject are closer to the user.

[0153] (Configuration 13) The image output system according to Configuration 12, characterized in that the image synthesis means creates the synthesized image such that the user is positioned below the legs of the main subject.

[0154] (Configuration 14) The image output system according to any one of Configurations 1 to 13, characterized in that the second shooting means captures a photograph of the user when it is detected that the user's facial expression change exceeds a predetermined amount.

[0155] (Configuration 15) The image output system according to any one of Configurations 1 to 14, characterized in that the first shooting means captures the main subject when it is detected that the user's facial expression change is greater than or equal to a predetermined amount.

[0156] (Configuration 16) The image output system according to any one of Configurations 1 to 15, characterized in that the second shooting means captures a photograph of the user when it is detected that the voice emitted by the user has reached a predetermined volume or higher.

[0157] (Configuration 17) The first shooting means captures the main subject when it is detected that the voice emitted by the user has reached a predetermined volume or higher. An image output system according to any one of configurations 1 to 16, characterized by the above.

[0158] (Configuration 18) An image output system according to any one of Configurations 1 to 17, characterized in that when the second shooting means photographs the user, it has a control means that prompts the user to take a predetermined pose or a predetermined facial expression.

[0159] (Computer program) A computer program for causing a computer to perform the following steps: a first shooting step for photographing a main subject; a second shooting step for photographing a user performing the first shooting step; and an image synthesis step for creating a composite image by combining the image of the main subject taken in the first shooting step with the image of the user taken in the second shooting step, and adjusting at least one of the sizes of the main subject and the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user. [Explanation of Symbols]

[0160] OIA: Imaging device (first imaging means) IIA: Imaging device (second imaging means) CA: Control device (image synthesis means) iOS: Image output system

Claims

1. A first photographic means for photographing the main subject, A second photographic means for photographing a user operating the first photographic means, Image synthesis means that creates a composite image by combining the image of the main subject acquired by the first shooting means and the image of the user acquired by the second shooting means, and adjusts at least one of the size of the main subject and the size of the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user. An image output system characterized by having the following features.

2. The image synthesis means is capable of obtaining information regarding the actual size of the main subject. The image output system according to claim 1, characterized in that it is as described above.

3. The image synthesis means can estimate the size of the main subject based on the focal length of the lens provided by the first shooting means, the size of the subject image on the light-receiving surface of the image sensor provided by the first shooting means, and the distance from the first shooting means to the main subject. The image output system according to claim 2, characterized in that it is as described above.

4. The image synthesis means can obtain information regarding the size of the candidate main subject from the server device. The image output system according to claim 2, characterized in that it is as described above.

5. The information regarding the size of the candidate main subject includes information regarding the size of each type of main subject. The image output system according to feature 4.

6. The information regarding the size of the candidate main subject includes information regarding the size of each individual main subject. The image output system according to feature 4.

7. The image synthesis means is capable of acquiring information regarding the user's size. The image output system according to claim 1, characterized in that it is as described above.

8. The image synthesis means, when the actual size of the main subject is smaller than the actual size of the user, creates the synthesized image such that the main subject is displayed larger than the user in the synthesized image. The image output system according to claim 1, characterized in that it is as described above.

9. The image synthesis means creates the synthesized image such that the opening of the main subject is close to the user. The image output system according to claim 1, characterized in that it is as described above.

10. The image synthesis means creates the synthesized image such that the eyes of the main subject are closer to the user. The image output system according to claim 1, characterized in that it is as described above.

11. The image synthesis means creates the synthesized image while changing the position of the pupil of the main subject in the image of the main subject so that the pupil of the main subject is closer to the user. The image output system according to claim 10, characterized in that it is the same as described above.

12. The image synthesis means creates the synthesized image such that the legs of the main subject are closer to the user. The image output system according to claim 1, characterized in that it is as described above.

13. The image synthesis means creates the synthesized image such that the user is positioned below the legs of the main subject. The image output system according to claim 12, characterized in that it is as described above.

14. The second photographing means photographs the user when it is detected that the user's facial expression changes by a predetermined amount or more. The image output system according to claim 1, characterized in that it is as described above.

15. The first photographing means photographs the main subject when it is detected that the user's facial expression change exceeds a predetermined amount. The image output system according to claim 1, characterized in that it is as described above.

16. The second imaging means detects that the voice emitted by the user exceeds a predetermined volume and takes a photograph of the user. The image output system according to claim 1, characterized in that it is as described above.

17. The first shooting means takes a picture of the main subject when it is detected that the voice emitted by the user has reached a predetermined volume or higher. The image output system according to claim 1, characterized in that it is as described above.

18. The second photographing means has a control means that prompts the user to take a predetermined pose or make a predetermined facial expression when photographing the user. The image output system according to claim 1, characterized in that it is as described above.

19. The first shooting step for photographing the main subject, A second shooting step for photographing the user performing the first shooting step, Image synthesis step: Creates a composite image by combining the image of the main subject captured in the first shooting step and the image of the user captured in the second shooting step, and adjusts at least one of the sizes of the main subject and the user so that the size ratio of the main subject to the user in the composite image is greater than the actual size ratio of the main subject to the user. A computer program that causes a computer to execute something.

Images