Imaging output device, method for operating imaging output device, and operation program for imaging output device

Abstract

This imaging output device includes a processor, wherein: when a user instructs a first imaging unit located in a first space to capture an image, the processor instructs a second imaging unit located in a second space different from the first space to capture an image, and causes the second imaging unit to output a second image corresponding to the first image captured by the first imaging unit.

Description

Imaging output device, method of operating an imaging output device, and operating program for an imaging output device

[0001] The technology of the present disclosure relates to an imaging output device, a method of operating an imaging output device, and an operating program for an imaging output device.

[0002] Japanese Unexamined Patent Application Publication No. 2009-176025 describes a virtual space communication system including a server device that provides a virtual space in which a character that operates as a user's avatar is arranged, and a user terminal that is connected to the server device via a network and that a user uses to operate the character. The virtual space communication system described in Japanese Unexamined Patent Application Publication No. 2009-176025 has a virtual camera in the virtual space for a character to take a picture in the virtual space, and the user terminal is connected to a printing device that prints an image. The user terminal acquires, from the server device, a captured image in the virtual space obtained by taking a picture with the virtual camera based on the operation of the character by the user and causes the printing device to print it.

[0003] "Instant Camera (Polaroid Camera) Compatible with Physbone for 'VR Chat: Can Take 3D Photos! Can Draw Handwritten Characters! Can Fly! With the Black Panther Beastman Avatar 'Azul 2' with Griffon Wings' [Accessory for Avatar]", searched on February 26, 2023, Internet <URL: https: / / booth.pm / ja / items / <3961856> (the "Polaroid" described in the title of the above Internet page is a registered trademark) describes a mode of taking an instant photo of a virtual space with an instant camera operated by an avatar in the virtual space.

[0004] One embodiment of the technology according to the present disclosure provides an imaging output device, a method of operating an imaging output device, and an operating program for an imaging output device that can meet the demands of a user.

[0005] The photography output device of the present disclosure includes a processor, and when a user instructs a first photography device located in a first space to photograph an image, the processor instructs a second photography device located in a second space different from the first space to photograph an image, and causes the second photography device to output a second image corresponding to the first image photographed by the first photography device.

[0006] Preferably, the processor generates a second image that replicates the appearance of the subject in the first image.

[0007] The first space is a real space and the second space is a virtual space, and the first photographing device is provided with a sensor that detects the relative position and relative attitude of the first photographing device with respect to a reference position and reference attitude, and it is preferable that the processor generates a second image that reproduces the appearance of the subject in the first image based on the detection result of the sensor when a photographing instruction is given.

[0008] The first space is a real space and the second space is a virtual space, and when the second image shows a first avatar active in the virtual space of a first user who gave a shooting instruction to the first shooting device in the real space and a second avatar of a second user different from the first user, and when only the first user is shown in the first image, the processor preferably composites the second user into the first image.

[0009] Preferably, the first space is a real space and the second space is a virtual space, the first image capturing device has a function of printing an image, and the processor causes a second image to be printed from the first image capturing device in addition to the second image capturing device.

[0010] The operating method of the photography output device disclosed herein includes, when a user instructs a first photography device located in a first space to photograph an image, instructing a second photography device located in a second space different from the first space to photograph an image, and outputting a second image corresponding to the first image photographed by the first photography device from the second photography device.

[0011] The operating program of the photography output device disclosed herein causes a computer to execute processing including, when a user instructs a first photography device located in a first space to photograph an image, instructing a second photography device located in a second space different from the first space to photograph an image, and outputting a second image corresponding to the first image photographed by the first photography device from the second photography device.

[0012] 1 is a diagram showing an instant camera, a user terminal, and a virtual space management server that exist in real space. FIG. 2 is a block diagram showing the internal configuration of an instant camera. FIG. 3 is a diagram showing a virtual space and an avatar. FIG. 4 is a block diagram showing computers that constitute a user terminal and a virtual space management server. FIG. 4 is a block diagram showing a processing unit of a CPU of a user terminal. FIG. 5 is a block diagram showing a processing unit of a CPU of a virtual space management server. FIG. 5 is a diagram showing object information. FIG. 6 is a diagram showing user information. FIG. 6 is a diagram showing the flow of shooting output processing in real space and virtual space. FIG. 7 is a diagram showing the flow of shooting output processing in real space and virtual space. FIG. 7 is a flowchart showing the processing procedure of a virtual space management server. FIG. 7 is a diagram showing the flow of shooting output processing in real space and virtual space of a second embodiment. FIG. 8 is a diagram showing the flow of shooting output processing in virtual space of a second embodiment. FIG. 8 is a diagram showing the flow of shooting output processing in real space and virtual space of a second embodiment.

[0013] [First embodiment] As an example, as shown in FIG. 1 , in a real space RS, a user U owns an instant camera 10 and a user terminal 11. The instant camera 10 is used to take an instant photograph 95 (see FIG. 10 ). The instant camera 10 is connected to the user terminal 11 via short-range wireless communication such as Bluetooth (registered trademark) so that they can communicate with each other. The real space RS is an example of a "first space" according to the technology of the present disclosure. The instant camera 10 is also an example of a "first photographing device" according to the technology of the present disclosure. The instant photograph 95 is also an example of a "first image" according to the technology of the present disclosure.

[0014] Specifically, the user terminal 11 is a smartphone, a tablet terminal, a notebook personal computer, a desktop personal computer, or the like. The user terminal 11 may also be a wearable device such as a head-mounted display having a display function in a portion that covers the eyes. The user terminal 11 is connected to the virtual space management server 13 via a network 12 so as to be able to communicate with each other. The network 12 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network.

[0015] The virtual space management server 13 is, for example, a server computer, a workstation, or the like, and is an example of an "imaging output device" according to the technology of the present disclosure. A plurality of user terminals 11 of a plurality of users U are connected to the virtual space management server 13 via a network 12.

[0016] As an example, as shown in FIG. 2 , the instant camera 10 includes an imaging optical system 20 and an imaging element 21. The imaging optical system 20 includes multiple types of lenses for focusing subject light on the imaging element 21. Specifically, the imaging optical system 20 includes an objective lens 22 and a focus lens 23. The objective lens 22 and the focus lens 23 are arranged in this order from the object side (subject side) toward the image side (image element 21 side). Although simplified in FIG. 1 , the objective lens 22 and the focus lens 23 are actually a lens group formed by combining multiple lenses. The imaging optical system 20 also includes an aperture 24. The aperture 24 is arranged closest to the image side of the imaging optical system 20. The imaging optical system 20 may also include a zoom lens.

[0017] The focus lens 23 is provided with a focus lens drive mechanism 25, and the diaphragm 24 is provided with an aperture opening adjustment mechanism 26. The focus lens drive mechanism 25 includes, for example, a focus cam ring that holds the focus lens 23 and has a cam groove formed on its outer periphery, a focus motor that moves the focus cam ring along the optical axis OA by rotating the focus cam ring around the optical axis OA, and a driver for the focus motor.

[0018] The diaphragm 24 is, for example, an iris diaphragm, and is composed of a combination of multiple diaphragm blades. The diaphragm 24 simultaneously moves the diaphragm blades using a cam mechanism to open and close a central opening formed by the inner edges of the diaphragm blades, i.e., to change the opening degree of the opening (hereinafter referred to as the opening degree of the diaphragm 24), thereby adjusting the amount of light passing through. The diaphragm opening adjustment mechanism 26 includes a diaphragm motor that opens and closes the diaphragm blades, a driver for the diaphragm motor, etc.

[0019] The focus motor and the aperture motor are, for example, stepping motors. In this case, the position of the focus lens 23 on the optical axis OA and the opening of the aperture 24 can be derived from the drive amounts of the focus motor and the aperture motor. Note that a position sensor may be provided to detect the position of the focus lens 23 instead of the drive amount of the focus motor.

[0020] Electrical components such as the focus motor and diaphragm motor or driver of the focus lens drive mechanism 25 and the aperture opening adjustment mechanism 26 are connected to the control unit 27. The electrical components of the focus lens drive mechanism 25 and the aperture opening adjustment mechanism 26 are driven under the control of the control unit 27. More specifically, the control unit 27 issues drive signals corresponding to the distance to the subject, the brightness of the subject, etc., to drive the electrical components of the focus lens drive mechanism 25 and the aperture opening adjustment mechanism 26. For example, the control unit 27 issues a drive signal to a driver of the focus motor of the focus lens drive mechanism 25 to move the focus lens 23 along the optical axis OA, thereby performing automatic focus adjustment. The control unit 27 also issues a drive signal to a driver of the diaphragm motor of the diaphragm opening adjustment mechanism 26 to change the opening of the diaphragm 24, thereby performing automatic exposure adjustment.

[0021] The focus motor and the diaphragm motor output the drive amounts to the control unit 27. The control unit 27 derives the position of the focus lens 23 on the optical axis OA and the opening of the diaphragm 24 from the drive amounts.

[0022] The imaging element 21 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and has an imaging surface 29 that captures subject light. The imaging surface 29 is formed by a plurality of pixels arranged two-dimensionally. The pixels accumulate signal charges corresponding to the subject light and output an image signal (voltage signal) corresponding to the signal charges. The imaging element 21 is arranged so that the center of the imaging surface 29 coincides with the optical axis OA and so that the imaging surface 29 is perpendicular to the optical axis OA. Note that the terms "coincidence" and "orthogonality" used here refer to perfect coincidence and orthogonality, as well as coincidence and orthogonality that include errors generally accepted in the technical field to which the technology of the present disclosure pertains.

[0023] An imaging element driver 30 is connected to the imaging element 21. The imaging element driver 30 is connected to the control unit 27. Under the control of the control unit 27, the imaging element driver 30 controls the timing of capturing subject light by the imaging element 21 by, for example, supplying a vertical scanning signal and a horizontal scanning signal to the imaging element 21. The imaging element driver 30 also sets a gain to be applied to an image signal output from a pixel, which gain corresponds to ISO (International Organization for Standardization) sensitivity.

[0024] A shutter 31 is provided between the imaging optical system 20 and the image sensor 21. The shutter 31 is, for example, a focal plane shutter having a front curtain and a rear curtain. A shutter drive mechanism 32 is connected to the shutter 31. The shutter drive mechanism 32 includes an electromagnet that holds the front curtain and the rear curtain, releases the hold, and causes the front curtain and rear curtain to move, as well as a driver for the electromagnet. The shutter drive mechanism 32 is driven under the control of the control unit 27 to open and close the shutter 31.

[0025] The control unit 27 is connected to various units, such as an image input controller 33, an image memory 34, and an image processing unit 35, via a bus line 36. Other units connected to the bus line 36 include a VRAM (Video Random Access Memory) 37, a display control unit 38, a communication unit 39, a printer driver 40, a position / orientation sensor 41, and an instruction receiving unit 42. Although not shown, the bus line 36 also connects to a strobe drive control unit that controls the drive of a strobe device, and a media controller that controls the writing of image data to a memory card, etc.

[0026] Image data obtained by capturing an image of the subject light is input to the image input controller 33 from the imaging element 21. The image input controller 33 outputs the image data to the image memory 34. The image memory 34 is, for example, a synchronous dynamic random access memory (SDRAM), and temporarily stores the image data.

[0027] The image processing unit 35 reads unprocessed image data from the image memory 34. The image processing unit 35 performs various image processing on the image data. The various image processing includes, for example, offset correction processing, sensitivity correction processing, pixel interpolation processing, white balance correction processing, gamma correction processing, demosaic processing, luminance signal and color difference signal generation processing, edge enhancement processing, color correction processing, etc. The image processing unit 35 writes the image data after the various image processing back to the image memory 34.

[0028] The VRAM 37 receives image data from the image memory 34, which has undergone various image processing and is to be displayed as a live view image (also called a through image). The VRAM 37 has an area for storing image data for two consecutive frames. The image data stored in the VRAM 37 is sequentially overwritten with new image data. The VRAM 37 sequentially outputs the newer image data of the two consecutive frames of image data to the display control unit 38.

[0029] The display control unit 38 functions as a so-called video encoder, converting image data from the VRAM 37 into video data and outputting it to the rear monitor 43. This allows the user U to view a live view image through the rear monitor 43. The display frame rate of the live view image is, for example, 30 fps (frames per second).

[0030] The communication unit 39 is responsible for wireless communication with external devices such as the user terminal 11 by short-range wireless communication such as the aforementioned Bluetooth (registered trademark). The communication unit 39 also is responsible for wired communication with external devices via a connection terminal such as a USB (Universal Serial Bus) terminal.

[0031] When an instruction to take an instant photograph 95 is given by operating the release button on the operation unit 28, the image data after various image processing in the image memory 34 is output to the printer driver 40. A printer 44 is connected to the printer driver 40. Under the control of the printer driver 40, the printer 44 prints the image data as the instant photograph 95 on the built-in instant film 45. The instant film 45 with the printed instant photograph 95 is output to the outside from a film outlet provided on one side of the instant camera 10. The printer driver 40 and the printer 44 are an example of the "function to print an image" according to the technology of the present disclosure. The instant film 45 may be either a silver halide film or a thermal film.

[0032] The position and orientation sensor 41 is composed of an acceleration sensor and a gyro sensor, and detects the relative position and orientation of the instant camera 10 with respect to a reference position and reference orientation at a preset sampling interval (for example, several milliseconds).

[0033] The position and orientation sensor 41 sequentially outputs detection results (hereinafter referred to as sensor detection results) 90 (see FIG. 9 ) of the relative position and orientation of the instant camera 10 with respect to the reference position and reference orientation to the communication unit 39. The communication unit 39 wirelessly transmits the sensor detection results 90 to the user terminal 11.

[0034] The instruction receiving unit 42 receives various operation instructions input by the user U via the operation unit 28 and a touch panel 46 provided integrally with the rear monitor 43. The instruction receiving unit 42 outputs the received various operation instructions to the control unit 27 via the bus line 36.

[0035] The release button of the operation unit 28 is, for example, a two-stage push button that can be pressed halfway or fully pressed. Pressing the release button halfway issues an instruction to prepare for shooting the instant photograph 95, and pressing it fully issues an instruction to shoot the instant photograph 95. When the release button is fully pressed to issue an instruction to shoot the instant photograph 95, the communication unit 39 wirelessly transmits to the user terminal 11 a shooting instruction signal 96 (see FIG. 10 ) indicating that an instruction to shoot the instant photograph 95 has been issued.

[0036] The operation unit 28 also includes a menu button for displaying various setting menus on the rear monitor 43, a cross key used for setting values ​​and switching between options, and a confirmation button operated when confirming settings, etc. The touch panel 46 is superimposed on the display surface of the rear monitor 43. The touch panel 46 recognizes various operation instructions from the user by detecting contact with a dedicated indicator such as the user's finger or a stylus pen.

[0037] As an example, as shown in Figure 3, the virtual space management server 13 manages a virtual space VS provided to users U through user terminals 11. The virtual space VS is a three-dimensional CG (computer graphics) space available on a computer, and is a place where multiple users U can communicate with each other and engage in various social activities such as studying, working, shopping, and playing, separate from the real space RS. Multiple users U act as their own avatars 50 in the virtual space VS. Figure 2 illustrates an example in which avatars 501 and 502 are standing side by side.

[0038] 4, the computers that make up the user terminal 11 and the virtual space management server 13 basically have the same configuration, and include a storage 55, a memory 56, a CPU (Central Processing Unit) 57, a communication unit 58, a display 59, and an input device 60. These are interconnected via a bus line 61.

[0039] The storage 55 is a hard disk drive built into the computer that constitutes the user terminal 11 and the virtual space management server 13, or connected via a cable or network. Alternatively, the storage 55 is a disk array consisting of multiple hard disk drives. The storage 55 stores control programs such as an operating system, various application programs (hereinafter abbreviated as APs (Application Programs)), and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0040] The memory 56 is a work memory for the CPU 57 to execute processing. The CPU 57 loads programs stored in the storage 55 into the memory 56 and executes processing in accordance with the programs. In this way, the CPU 57 comprehensively controls each part of the computer. The CPU 57 is an example of a "processor" according to the technology of the present disclosure. The memory 56 may be built into the CPU 57.

[0041] The communication unit 58 is a network interface that controls the transmission of various information via the network 12, etc. The display 59 displays various screens. The various screens are equipped with operation functions using a GUI (Graphical User Interface). The computer that constitutes the user terminal 11 and the virtual space management server 13 accepts input of operation instructions from an input device 60 via the various screens. The input device 60 is a keyboard, mouse, touch panel, microphone for voice input, etc.

[0042] In the following explanation, the parts of the computer that make up the user terminal 11 (storage 55, CPU 57, display 59, and input device 60) are distinguished by adding the suffix "A" to their symbols, and the parts of the computer that make up the virtual space management server 13 (storage 55, CPU 57, and input device 60) are distinguished by adding the suffix "B" to their symbols.

[0043] As an example, as shown in Figure 5, a virtual space AP 70 is stored in the storage 55A of the user terminal 11. The virtual space AP 70 is installed in the user terminal 11 by the user U. The virtual space AP 70 is an AP that allows the user U to enjoy the virtual space VS on the user terminal 11. When the virtual space AP 70 is launched, the CPU 57A of the user terminal 11 functions as a browser control unit 72 in cooperation with the memory 56 and the like. The browser control unit 72 controls the operation of a web browser dedicated to the virtual space AP 70.

[0044] The browser control unit 72 receives various types of information from the instant camera 10, such as the sensor detection results 90 and the shooting instruction signal 96. The browser control unit 72 then transfers the received information to the virtual space management server 13. The browser control unit 72 also receives various types of information from the virtual space management server 13, such as a print instruction signal 99 (see FIG. 10 ). The browser control unit 72 then transfers the received information to the instant camera 10. In other words, the user terminal 11 functions as a device that relays the exchange of various types of information between the instant camera 10 and the virtual space management server 13.

[0045] The browser control unit 72 reproduces various screens based on various screen data from the virtual space management server 13 and displays the reproduced various screens on the display 59A. The screen data is screen data for web distribution created using a markup language such as XML (Extensible Markup Language). Note that other data description languages ​​such as JSON (Javascript (registered trademark) Object Notation) may be used instead of XML.

[0046] The browser control unit 72 also accepts various operation instructions input by the user U from the input device 60A via various screens. The browser control unit 72 transmits the accepted operation instructions to the virtual space management server 13 together with a terminal ID (Identification Data) for uniquely identifying each user terminal 11.

[0047] 6, an operating program 75 is stored in storage 55B of virtual space management server 13. Operating program 75 is an AP that causes a computer constituting virtual space management server 13 to function as an "imaging output device" according to the technology of the present disclosure. In other words, operating program 75 is an example of an "operating program for an imaging output device" according to the technology of the present disclosure.

[0048] Storage 55B also stores display data 76, object information 77, user information 78, etc. Display data 76 is various 3D CG data necessary to display virtual space VS. Object information 77 is information about various objects included in virtual space VS. Objects are virtual objects with which avatar 50 can interact, such as by holding and manipulating, riding and manipulating, or wearing something. Specifically, objects are various people, buildings, and objects placed within virtual space VS. People include avatar 50 shown in FIG. 2 and characters created by the creator of virtual space VS. A virtual instant camera 92 (see FIG. 9) and a virtual instant film 98 (see FIG. 10), which will be described later, are also types of objects. User information 78 is information about each user U.

[0049] When the operating program 75 is started, the CPU 57B of the virtual space management server 13 functions as a reception unit 80, a processing unit 81, and a distribution control unit 82 in cooperation with the memory 56 and the like.

[0050] The reception unit 80 receives various information from the instant camera 10 transferred by the user terminal 11, such as the sensor detection results 90 and the shooting instruction signal 96. The reception unit 80 also receives various operational instructions and a terminal ID from the user terminal 11. The reception unit 80 outputs the various information from the instant camera 10 and the various operational instructions from the user terminal 11 to the processing unit 81, and outputs the terminal ID to the distribution control unit 82. The reception unit 80 also receives various operational instructions from the creator of the virtual space VS via the input device 60B.

[0051] The processing unit 81 is responsible for overall processing related to management of the virtual space VS in response to various information and various operation instructions from the reception unit 80. The processing performed by the processing unit 81 includes, for example, authentication processing for the user U, display update processing for the virtual space VS, object position update processing, update processing for the display data 76, update processing for the object information 77, update processing for the user information 78, and photograph output processing.

[0052] The authentication process for user U is a process of comparing the user ID and password for uniquely identifying each user U, which are included in the authentication instruction as an operation instruction, with the user ID and password (see FIG. 8 ) stored in user information 78. The display update process for virtual space VS is a process of, for example, reading display data 76 corresponding to an instruction to move avatar 50 as an operation instruction from storage 55B, generating screen data for the virtual space display screen based on the read display data 76, and outputting the generated screen data to distribution control unit 82.

[0053] The object position update process is a process of updating the object position information in response to an instruction to move the object as an operation instruction. The display data 76 update process is a process of storing display data 76 newly created by the creator of the virtual space VS in storage 55B in response to an instruction to update the display data 76 as an operation instruction. The object information 77 update process is a process of updating various items constituting the object information 77 in response to an instruction to update the object information 77 as an operation instruction, and a process of adding information about an object newly created by the creator of the virtual space VS. The user information 78 update process is a process of updating various items constituting the user information 78 in response to an instruction to update the user information 78 as an operation instruction, and a process of adding information about a new user U. The image capture output process will be described later with reference to FIGS. 9 and 10 .

[0054] The delivery control unit 82 controls the delivery of various information to the user terminal 11. For example, the delivery control unit 82 controls the delivery of screen data for a virtual space display screen. At this time, the delivery control unit 82 identifies the user terminal 11 to which the virtual space display screen is to be delivered, based on the terminal ID from the reception unit 80. The delivery control unit 82 also controls the delivery of a print instruction signal 99 to the user terminal 11.

[0055] As shown in FIG. 7 as an example, the object information 77 is information in which the object type, object name, ownership information, etc. are registered for each object ID for uniquely identifying each object. The types include various items such as an avatar 50, a virtual instant camera 92 (simply referred to as "instant camera" in FIG. 7), and a virtual instant film 98 (simply referred to as "instant film" in FIG. 7), as well as automobiles, clothes, etc. For the avatar 50, the name given to the avatar 50 by the user U is registered. For items such as the virtual instant camera 92 and the automobile, the name of the item is registered. For the virtual instant film 98, no name is registered.

[0056] The ownership information is information indicating which user U owns the object. Specifically, the ownership information is the user ID of the user U who owns the object. For example, the avatar 501 is owned by the user U with the user ID "U0000001."

[0057] Objects can be bought, sold, or transferred. When an object is bought, sold, or transferred, the processing unit 81 updates the ownership information of the object, in this case, the user ID. For example, currency valid within the virtual space VS is used for buying and selling objects. Note that "buying and selling" includes cases where a user U purchases an object prepared by the creator of the virtual space VS, and cases where one user U sells an object to another user U.

[0058] Although not shown in the figure, the virtual instant film 98 also stores the date and time of the photo taken based on the time in the virtual space VS, the location where the photo was taken in the virtual space VS, etc. Furthermore, the virtual instant film 98 includes not only those taken by the user U but also those prepared by the creator of the virtual space VS. In the initial state, the virtual instant film 98 prepared by the creator of the virtual space VS does not have a user ID registered as ownership information.

[0059] The avatar 50 represented by the object ID "OBA000001" is the avatar 501 shown in Fig. 3. The avatar 50 represented by the object ID "OBA000002" is the avatar 502 shown in Fig. 3.

[0060] 8 , the user information 78 includes a user ID, a password, a camera ID of each instant camera 10, a terminal ID of each user terminal 11, an age, and a gender. The user information 78 also includes a registered image 85. The registered image 85 is an image registered by the user U himself / herself when installing the virtual space AP 70, and is an image in which the user U is photographed.

[0061] Next, the photographing and outputting process will be described with reference to FIGS.

[0062] As an example, various events occur in the virtual space VS as shown in Fig. 9. In Fig. 9, an event is illustrated as a level-up of an avatar 501. In the following, a user U with a user ID "U0000001" who owns the avatar 501 will be referred to as user U1.

[0063] When an event occurs in the virtual space VS, in the real space RS, the user U1 attempts to capture a selfie instant photograph 95 with the instant camera 10 in order to preserve his or her own feelings, such as the joy he or she felt at the time, as a memory. To begin with, the user U1 turns on the power to the instant camera 10 and sets the reference position and reference attitude of the instant camera 10.

[0064] The reference position is, for example, the center of the user U1's chest. The reference attitude is, for example, a state in which the front of the instant camera 10, from which the objective lens 22 is exposed, is facing the user U1, and the up-down direction of the instant camera 10 is aligned vertically. The following method, for example, can be used to set the reference position and attitude. First, after turning on the power to the instant camera 10, a guidance voice is played to place the instant camera 10 in the reference position and attitude, or a guidance screen is displayed on the rear monitor 43. In response to this, the user U1 moves the instant camera 10 to the reference position and adopts the reference attitude. Then, by selecting the setting button on the operation unit 28, setting of the reference position and attitude is completed.

[0065] After completing the setting of the reference position and reference orientation, user U1 holds the instant camera 10 in front of his or her face to take a selfie. The position / orientation sensor 41 detects the relative position and orientation of the instant camera 10 with respect to the reference position and orientation, and outputs a sensor detection result 90. The communication unit 39 transmits the sensor detection result 90 to the user terminal 11. The sensor detection result 90 includes the camera ID of the instant camera 10 and the terminal ID of the user terminal 11. The browser control unit 72 of the user terminal 11 accepts the sensor detection result 90 and transfers the sensor detection result 90 to the virtual space management server 13. The reception unit 80 of the virtual space management server 13 accepts the sensor detection result 90. The reception unit 80 outputs the sensor detection result 90 to the processing unit 81.

[0066] The processing unit 81 determines the relative position and relative orientation of the instant camera 10 currently being held by the user U in the real space RS based on the sensor detection result 90. The processing unit 81 makes a virtual instant camera 92 appear in the virtual space VS at a position and orientation corresponding to the determined relative position and relative orientation, and has the avatar 501 hold the virtual instant camera 92. At this time, the processing unit 81 generates a virtual space display screen including an animation with a visual effect that makes it appear as if the virtual instant camera 92 is falling from the sky along with a meteor. The virtual instant camera 92 is an example of a "second photographing device" according to the technology of the present disclosure.

[0067] The user U changes the position and orientation of the instant camera 10 to determine the composition. The processing unit 81 determines the current relative position and orientation of the instant camera 10 from the sensor detection results 90 successively transmitted from the user terminal 11. The processing unit 81 then changes the position and orientation of the virtual instant camera 92 in the virtual space VS, as well as the display of the avatar 501 holding the virtual instant camera 92.

[0068] 10, the user U operates the release button of the instant camera 10 to issue an instruction to take an instant photograph 95. The printer 44 prints the instant photograph 95 on the instant film 45 under the control of the printer driver 40.

[0069] The communication unit 39 transmits the sensor detection result 90 and the shooting instruction signal 96 to the user terminal 11 when a shooting instruction is issued. Like the sensor detection result 90, the shooting instruction signal 96 includes the camera ID of the instant camera 10 and the terminal ID of the user terminal 11. The browser control unit 72 of the user terminal 11 accepts the sensor detection result 90 and the shooting instruction signal 96 and transfers the sensor detection result 90 and the shooting instruction signal 96 to the virtual space management server 13. The reception unit 80 of the virtual space management server 13 accepts the sensor detection result 90 and the shooting instruction signal 96. The reception unit 80 outputs the sensor detection result 90 and the shooting instruction signal 96 to the processing unit 81.

[0070] The processing unit 81 receives the sensor detection result 90 and the shooting instruction signal 96 and instructs the virtual instant camera 92 to take a virtual instant photograph 97. Based on the sensor detection result 90, the processing unit 81 determines the relative position and relative orientation of the instant camera 10 when the shooting instruction is issued. Based on this determination result, the processing unit 81 generates a virtual instant photograph 97 corresponding to the instant photograph 95 taken by the instant camera 10. More specifically, the processing unit 81 generates a virtual instant photograph 97 that reproduces the appearance of the subject of the instant photograph 95, in this case, user U1. FIG. 10 illustrates an example of a virtual instant photograph 97 that reproduces the size and orientation of the face of the avatar 501 captured by the instant camera 10 held slightly diagonally above and to the right of the user U1's face. The virtual instant photograph 97 is an example of a "second image" according to the technology disclosed herein.

[0071] The processing unit 81 prints the virtual instant photograph 97 on a virtual instant film 98. At this time, the processing unit 81 generates a virtual space display screen including an animation that creates a visual effect as if the virtual instant photograph 97 were printed out from the virtual instant camera 92. As described above, the virtual instant film 98 is treated as an object in the virtual space VS, and the virtual instant photograph 97 can be viewed, bought, sold, or transferred.

[0072] The distribution control unit 82 transmits a print instruction signal 99 for the virtual instant photograph 97 to the user terminal 11. The browser control unit 72 of the user terminal 11 accepts the print instruction signal 99 and transfers the print instruction signal 99 to the instant camera 10. The communication unit 39 of the instant camera 10 receives the print instruction signal 99 and outputs the print instruction signal 99 to the printer driver 40. The printer 44 prints the virtual instant photograph 97 on the instant film 45 under the control of the printer driver 40.

[0073] Next, the operation of the above configuration will be described with reference to the flowchart shown in Fig. 11 as an example. As shown in Fig. 5, the CPU 57A of the user terminal 11 functions as a browser control unit 72 when the virtual space AP 70 is activated. Also, as shown in Fig. 6, the CPU 57B of the virtual space management server 13 functions as a reception unit 80, a processing unit 81, and a distribution control unit 82 when the operating program 75 is activated.

[0074] 9 and 10 , when an event such as a level-up of the avatar 50 occurs in the virtual space VS, the user U takes a selfie instant photograph 95 with the instant camera 10. At this time, the sensor detection result 90 of the position / orientation sensor 41 of the instant camera 10 is received by the reception unit 80 (step ST100). The sensor detection result 90 is output from the reception unit 80 to the processing unit 81.

[0075] The processing unit 81 determines the relative position and relative orientation of the instant camera 10 based on the sensor detection result 90. Then, a virtual instant camera 92 appears at a position and orientation according to the determined relative position and relative orientation, and the avatar 501 holds the virtual instant camera 92. If the user U changes the position and orientation of the instant camera 10, the position and orientation of the virtual instant camera 92 are also changed (step ST110).

[0076] When the user U operates the release button of the instant camera 10 to issue an instruction to take an instant photograph 95, the sensor detection result 90 and the photographing instruction signal 96 are received by the reception unit 80 (YES in step ST120). The sensor detection result 90 and the photographing instruction signal 96 are output from the reception unit 80 to the processing unit 81. In the instant camera 10, the printer 44 prints the instant photograph 95 on the instant film 45.

[0077] The processing unit 81 also instructs the virtual instant camera 92 to take a virtual instant photograph 97. The processing unit 81 determines the relative position and relative attitude of the instant camera 10 when the instruction to take a photograph was issued, based on the sensor detection result 90. Then, based on this determination result, a virtual instant photograph 97 is generated that reproduces how the user U appears in the instant photograph 95 (step ST130).

[0078] The processing unit 81 prints the virtual instant photograph 97 on the virtual instant film 98 (step ST140). The distribution control unit 82 also transmits a print instruction signal 99 for the virtual instant photograph 97 to the user terminal 11 (step ST150).

[0079] The print instruction signal 99 is received by the communication unit 39 of the instant camera 10. The print instruction signal 99 is output from the communication unit 39 to the printer driver 40. Then, the printer 44 prints the virtual instant photograph 97 on the instant film 45.

[0080] As described above, the CPU 57B of the virtual space management server 13 functions as the processing unit 81. When the user U instructs the instant camera 10 present in the real space RS to take an instant photograph 95, the processing unit 81 instructs the virtual instant camera 92 present in the virtual space VS to take a virtual instant photograph 97. The processing unit 81 causes the virtual instant camera 92 to output a virtual instant photograph 97 corresponding to the instant photograph 95 taken by the instant camera 10. This allows for coordination between photography by the instant camera 10 in the real space RS and photography by the virtual instant camera 92 in the virtual space VS. This makes it possible to meet the user U's desire to enjoy photography without distinction between the real space RS and the virtual space VS. It is also possible to meet the user U's desire to keep the excitement of an event, such as a level-up, experienced in the virtual space VS as a memory in the real space RS.

[0081] The processing unit 81 generates a virtual instant photograph 97 that reproduces how the user U appears in the instant photograph 95. This makes it possible to obtain a virtual instant photograph 97 that is more appealing to the user U. Furthermore, compared to a case where the instant photograph 95 and the virtual instant photograph 97 show the user U in completely different ways, the virtual instant photograph 97 can more easily evoke in the user U the excitement of the event that the user U experienced in the virtual space VS.

[0082] The instant camera 10 is provided with a position / orientation sensor 41 that detects the relative position and orientation of the instant camera 10 with respect to a reference position and reference orientation. The processing unit 81 generates a virtual instant photograph 97 that reproduces how the user U appears in the instant photograph 95, based on a sensor detection result 90 of the position / orientation sensor 41 when a shooting instruction is issued. Therefore, it is possible to generate a virtual instant photograph 97 that more accurately reproduces how the user U appears in the instant photograph 95, compared to when the virtual instant photograph 97 is generated without referring to the sensor detection result 90. This can further improve the quality of the virtual instant photograph 97.

[0083] The instant camera 10 has a function for printing images (printer driver 40 and printer 44). The processing unit 81 prints a virtual instant photograph 97 from the instant camera 10 in addition to the virtual instant camera 92. This allows the user U to view the actual virtual instant photograph 97 in the real space RS. Furthermore, by viewing the virtual instant photograph 97 paired with the instant photograph 95, the user U can more easily recall the excitement of the event he or she experienced in the virtual space VS. Conversely, the virtual instant camera 92 may print the instant photograph 95, and a virtual instant film 98 with the instant photograph 95 printed on it may appear in the virtual space VS.

[0084] 12 to 14, the image capture output process of the second embodiment will be described. As with the first embodiment, the second embodiment also uses an example in which a user U1 takes a selfie in real space RS with an instant camera 10. The user U1 is an example of a "first user" according to the technology of the present disclosure.

[0085] As shown in FIG. 12 as an example, in the second embodiment, the instant camera 10 includes an instant photograph 95 in the shooting instruction signal 96. The processing unit 81 receives this shooting instruction signal 96 and performs the following processing. First, using well-known face recognition technology, the processing unit 81 compares the face in the instant photograph 95 with the face in the registered image 85 of the user information 78 to determine whether only the user U1 appears in the instant photograph 95. Similarly to the first embodiment, the processing unit 81 generates a virtual instant photograph 97 in response to the shooting instruction signal 96, but determines whether only the avatar 501 of the user U1 appears in the virtual instant photograph 97. If the avatar 501 of the user U1 and the avatar 50 of a user U other than the user U1 appear in the virtual instant photograph 97 and only the user U1 appears in the instant photograph 95, the processing unit 81 further performs the following processing. 12 illustrates an example in which avatar 502 owned by user U with user ID "U0000002" appears in virtual instant photograph 97 together with avatar 501. Hereinafter, user U with user ID "U0000002" who owns avatar 502 will be referred to as user U2 (see FIG. 13). User U2 is an example of a "second user" according to the technology of the present disclosure. Avatar 501 is an example of a "first avatar" according to the technology of the present disclosure, and avatar 502 is an example of a "second avatar" according to the technology of the present disclosure.

[0086] 13 , the processing unit 81 refers to the object information 77 to identify the user U (user U2) who owns the avatar 502 that appears together with the avatar 501 in the virtual instant photograph 97. Next, the processing unit 81 selects a registered image 85 of the identified user U2 from the user information 78. The processing unit 81 composites the user U2 who appears in the selected registered image 85 onto the instant photograph 95 transmitted together with the shooting instruction signal 96, thereby generating a composite instant photograph 110. In this case, the processing unit 81 generates a natural-looking composite instant photograph 110 by using a machine learning model specialized for combining images of people, for example.

[0087] 14 , the distribution control unit 82 transmits a print instruction signal 99 for the virtual instant photograph 97 and the composite instant photograph 110 to the user terminal 11. The browser control unit 72 of the user terminal 11 receives the print instruction signal 99 and transfers it to the instant camera 10. The communication unit 39 of the instant camera 10 receives the print instruction signal 99 and outputs it to the printer driver 40. The printer 44 prints the virtual instant photograph 97 and the composite instant photograph 110 on the instant film 45 under the control of the printer driver 40.

[0088] In this way, in the second embodiment, when the avatar 501 of user U1, who has issued a shooting instruction to the instant camera 10 in the real space RS, and the avatar 502 of a different user U2, who is different from user U1, appear in the virtual instant photograph 97, and only user U1 appears in the instant photograph 95, the processing unit 81 composites user U2 into the instant photograph 95 to create a composite instant photograph 110. Therefore, user U2, who is in the same space as avatar 501 in the virtual space VS and whose avatar 502 appears in the virtual instant photograph 97 but is not present in the real space RS, can be captured in a real instant photograph as if he or she were present at the actual location.

[0089] Appearing together in the virtual instant photograph 97 means that the avatar 502 has an intimate relationship with the avatar 501, at least in the virtual space VS. Therefore, generating a composite instant photograph 110 in the real space RS that inherits the intimate relationship in the virtual space VS can further stimulate the interest of the user U. This makes it possible to meet the desire of a user U to share the excitement of an event experienced in the virtual space VS with a close friend in the real space RS who is not present at the event.

[0090] The number of second users to be combined into the composite instant photograph 110 is not limited to one as illustrated. Two or more users may be combined. Furthermore, the users U to be combined may be limited to users U who have registered as friends. This prevents a user U with an avatar 50 that happens to appear in the virtual instant photograph 97 and has no relationship with the user U from being combined into the instant photograph 95.

[0091] The virtual instant photograph 97 may be taken from a slightly distant perspective so that it is clear that the avatar 50 is holding the virtual instant camera 92 .

[0092] The event is not limited to the level-up example. It may also be when a friend joins the team or when an item that is key to winning the game is obtained. The event may also be a seasonal gathering such as a Christmas party or a New Year's party held in each virtual space VS, or a gathering unrelated to the season such as a birthday party for a user U, an anniversary party for the virtual space VS, or a virtual cultural lecture.

[0093] A virtual instant photograph 97 that reproduces how the user U appears in the instant photograph 95 may be generated by performing image analysis on how the face of the user U appears in the instant photograph 95, without referring to the sensor detection result 90 by the position and orientation sensor 41. A machine learning model that outputs a virtual instant photograph 97 that reproduces how the user U appears in the instant photograph 95 in response to input of the instant photograph 95 may be used.

[0094] In the above description, the first space is the real space RS, the second space is the virtual space VS, the first photographing device is the instant camera 10, the second photographing device is the virtual instant camera 92, the first image is the instant photograph 95, and the second image is the virtual instant photograph 97. However, this is not limited to this. Conversely, the first space may be the virtual space VS, the second space may be the real space RS, the first photographing device may be the virtual instant camera 92, the second photographing device may be the instant camera 10, the first image may be the virtual instant photograph 97, and the second image may be the instant photograph 95. In this case, in order to generate an instant photograph 95 that reproduces the appearance of the avatar 50 in the virtual instant photograph 97, the instant camera 10 may play a guidance voice or display a guidance screen on the rear monitor 43 to guide the avatar 50 to a position and posture that will result in the avatar 50 appearing in the same appearance as the avatar 50 in the virtual instant photograph 97.

[0095] For example, the virtual space VS may be a reproduction of a scene from an earlier time than the real space RS, such as the Edo period. In this case, when a current scene is captured in the real space RS with the instant camera 10, a scene from that time is captured in the virtual space VS with the same angle of view with the virtual instant camera 92. In this way, the method of capturing images with the instant camera 10 is not limited to selfies by the user U.

[0096] Although an example has been given in which various information such as the sensor detection result 90, the shooting instruction signal 96, and the print instruction signal 99 is exchanged between the instant camera 10 and the virtual space management server 13 via the user terminal 11, the present invention is not limited to this. Various information may also be exchanged directly between the instant camera 10 and the virtual space management server 13 without going through the user terminal 11.

[0097] The hardware configuration of the computer that constitutes the virtual space management server 13 can be modified in various ways. For example, the virtual space management server 13 can be configured with multiple computers separated as hardware in order to improve processing power and reliability. For example, the functions of the reception unit 80 and distribution control unit 82 and the function of the processing unit 81 can be distributed and performed by two computers. In this case, the virtual space management server 13 is configured with two computers. Some or all of the functions of the virtual space management server 13 may be performed by the user terminal 11.

[0098] In this way, the hardware configuration of the computer of the virtual space management server 13 can be changed as appropriate depending on the required performance such as processing power, safety, reliability, etc. Furthermore, it is not limited to hardware, but APs such as the virtual space AP 70 and the operating program 75 can also be duplicated or stored in multiple storage devices in order to ensure safety and reliability.

[0099] In each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the browser control unit 72, the reception unit 80, the processing unit 81, and the distribution control unit 82. The various processors include CPUs 57A and 57B, which are general-purpose processors that execute software (virtual space AP 70 and operating program 75) and function as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as programmable logic devices (PLDs) that are processors whose circuit configuration can be changed after manufacture, such as field programmable gate arrays (FPGAs), and / or application specific integrated circuits (ASICs).

[0100] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0101] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0102] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0103] From the above description, the technology described in the following supplementary paragraphs can be understood.

[0104] [Supplementary Item 1] An imaging output device comprising a processor, wherein when a user issues an instruction to a first imaging device located in a first space to capture an image, the processor issues an instruction to a second imaging device located in a second space different from the first space to capture an image, and causes the second imaging device to output a second image corresponding to the first image captured by the first imaging device. [Supplementary Item 2] The imaging output device described in Supplementary Item 1, wherein the processor generates the second image that reproduces how a subject of the first image is captured. [Supplementary Item 3] The imaging output device described in Supplementary Item 2, wherein the first space is a real space and the second space is a virtual space, the first imaging device is provided with a sensor that detects a relative position and relative orientation of the first imaging device with respect to a reference position and a reference orientation, and the processor generates the second image that reproduces how the subject of the first image is captured based on a detection result of the sensor when the imaging instruction is issued. [Supplementary Item 4] The imaging output device according to any one of Supplementary Items 1 to 3, wherein the first space is a real space and the second space is a virtual space, and the processor, when the second image shows a first avatar of a first user who issued the imaging instruction to the first imaging device in the real space and who is active in the virtual space, and a second avatar of a second user different from the first user, and when only the first user appears in the first image, superimposes the second user into the first image. [Supplementary Item 5] The imaging output device according to any one of Supplementary Items 1 to 4, wherein the first space is a real space and the second space is a virtual space, the first imaging device has a function of printing images, and the processor causes the second image to be printed from the first imaging device in addition to the second imaging device.

[0105] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs, but also to storage media that non-temporarily store programs, and computer program products that include programs.

[0106] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0107] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0108] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An image capture output device comprising a processor, which, when a user instructs a first image capture device located in a first space to capture an image, instructs a second image capture device located in a second space different from the first space to capture an image, and causes the second image capture device to output a second image corresponding to the first image captured by the first image capture device.

2. The photographing output device according to claim 1, wherein the processor generates the second image that reproduces the appearance of the subject in the first image.

3. The photographing output device of claim 2, wherein the first space is a real space and the second space is a virtual space, the first photographing device is provided with a sensor that detects a relative position and relative attitude of the first photographing device with respect to a reference position and a reference attitude, and the processor generates the second image that reproduces the appearance of the subject of the first image based on the detection result of the sensor when the photographing instruction is given.

4. The photographing output device of claim 1, wherein the first space is a real space and the second space is a virtual space, and the processor superimposes the second user into the first image when the second image shows a first avatar active in the virtual space of a first user who issued the photographing instruction to the first photographing device in the real space and a second avatar of a second user different from the first user, and when only the first user appears in the first image.

5. The photographing output device of claim 1, wherein the first space is a real space and the second space is a virtual space, the first photographing device has a function of printing an image, and the processor causes the second image to be printed from the first photographing device in addition to the second photographing device.

6. A method for operating an image capture output device, comprising: when a user instructs a first photographing device located in a first space to capture an image, issuing an instruction to a second photographing device located in a second space different from the first space to capture an image; and outputting from the second photographing device a second image corresponding to the first image captured by the first photographing device.

7. An operating program for an image capture output device that causes a computer to execute a process including: when a user instructs a first image capture device located in a first space to capture an image, instructing a second image capture device located in a second space different from the first space, and outputting from the second image capture device a second image corresponding to the first image captured by the first image capture device.

Images