Composite image generation apparatus, composite image generation system, composite image generation method, and program

Abstract

One aspect of the present invention provides a composite image generation apparatus, a composite image generation system, a composite image generation method, and a program. An image exposure apparatus according to one aspect of the present invention is a composite image generation apparatus comprising a processor and a display device. The processor: causes a plurality of first images to be displayed in time series in a first region of the display device; receives a first operation on the plurality of first images; causes a second image selected by the first operation to be displayed in a second region of the display device different from the first region by increasing the visibility relative to that of the first images; receives a second operation on the second image; causes a third image selected by the second operation to be displayed in a third region of the display device different from the first region and the second region; and generates a composite image by using the third image.

Description

Synthetic Image Generation Device, Synthetic Image Generation System, Synthetic Image Generation Method, and Program

[0001] The present invention relates to a synthetic image generation device, a synthetic image generation system, a synthetic image generation method, and a program that generate a synthetic image according to a user's instruction.

[0002] Regarding the generation of a synthetic image, for example, Patent Document 1 describes the playback of video materials and the creation of a slide show using still images.

[0003] Japanese Patent Application Laid-Open No. 2017-072953

[0004] One embodiment according to the technology of the present disclosure provides a synthetic image generation device, a synthetic image generation system, a synthetic image generation method, and a program.

[0005] The synthetic image generation device according to the first aspect of the present invention is a synthetic image generation device including a processor and a display device. The processor displays a plurality of first images in a time series in a first area of the display device, receives a first operation on the plurality of first images, and displays the second image selected by the first operation in a second area that is an area different from the first area of the display device with higher visibility than the first image. The processor receives a second operation on the second image, displays the third image selected by the second operation in a third area that is an area different from the first area and the second area of the display device, and generates a synthetic image using the third image.

[0006] In the synthetic image generation device according to the second aspect of the present invention, in the first aspect, the processor receives, as the first operation, an operation of scrolling the first image in the first area, and sets the image displayed at a determined position in the first area by the scrolling as the second image.

[0007] In the synthetic image generation device according to the third aspect, in the first or second aspect, the processor enhances the visibility of the second image by one or more of the following: making the size of the second image larger than that of the first image; attaching one or more of characters, symbols, numbers, and figures to the second image; changing one or more of the color, brightness, and display pattern of the image from those of the first image.

[0008] In the second embodiment, the composite image generation apparatus according to the fourth embodiment highlights the first image displayed at a predetermined position by one or more of the following: adding one or more characters, symbols, numbers, or figures to the first image displayed at a predetermined position; or changing one or more of the color, brightness, or display pattern of the first image displayed at a predetermined position from that of other first images among a plurality of first images.

[0009] In the composite image generation apparatus according to the fifth embodiment, in any one of the first to fourth embodiments, the processor adds numbers to the images to be displayed in the first area in the order in which the second operation was performed and displays them.

[0010] In the composite image generation apparatus according to the sixth embodiment, in any one of the first to fifth embodiments, the processor adds and displays information indicating that the first image has first audio data if the first image has first audio data.

[0011] In the seventh embodiment of the composite image generation apparatus, in any one of the first to sixth embodiments, the processor adds at least a portion of the first audio data to the composite image if the first image has first audio data.

[0012] In the seventh embodiment, the composite image generation apparatus according to the eighth embodiment adds at least a portion of the first audio data to the composite image if the third image has first audio data.

[0013] The composite image generation apparatus according to the ninth embodiment further comprises a recording medium for recording audio data in any one of the first to eighth embodiments, and the processor receives a third operation to select second audio data which is audio data recorded on the recording medium, and adds at least a portion of the second audio data selected by the third operation to the composite image.

[0014] In the composite image generation apparatus according to the tenth embodiment, in any one of the first to fifth embodiments, the processor sets the relationship between the volume of the first audio data contained in the first image and the volume of the second audio data recorded on the recording medium.

[0015] In the composite image generation apparatus according to the eleventh embodiment, in any one of the first to tenth embodiments, the processor causes the printing device to print a fourth image representing a representative image of the composite image in response to a fourth operation that instructs printing.

[0016] In the twelfth embodiment, the composite image generation apparatus is an image based on at least the first image, in the eleventh embodiment.

[0017] In the composite image generation apparatus according to the 13th embodiment, in the 11th or 12th embodiment, the processor causes the composite image to be recorded in the recording device when the fourth image has been printed by the printing device.

[0018] In the thirteenth embodiment, the composite image generation apparatus has a processor record the composite image using a device that operates independently of the composite image generation apparatus as a recording device.

[0019] In the composite image generation apparatus according to the 15th embodiment, in any one of the 11th to 14th embodiments, the processor superimposes information regarding the composite image onto the fourth image and causes the printing device to print it.

[0020] In the sixteenth embodiment of the composite image generation apparatus, in any one of the first to fifteenth embodiments, the processor performs a first evaluation, which is an evaluation of the image displayed in the first region, and outputs first information relating to the result of the first evaluation.

[0021] In the 16th embodiment, the composite image generation apparatus according to the 17th embodiment is information that recommends including images in the composite image that satisfy the first criteria as a result of the first evaluation.

[0022] In the composite image generation apparatus according to the 18th embodiment, in any one of the first to 17 embodiments, the processor performs a second evaluation, which is an evaluation of the image on which the second operation has been performed, and outputs second information relating to the result of the second evaluation.

[0023] In the composite image generation apparatus according to the 19th embodiment, in the 18th embodiment, the second information is information recommending that the result of the second evaluation satisfy the second criterion for printing.

[0024] In the composite image generation apparatus according to the 20th embodiment, in any one of the first to 19 embodiments, the processor displays the images in the third area in the order in which the second operations were performed.

[0025] In the composite image generation apparatus according to the 21st embodiment, in any one of the first to 20th embodiments, the processor displays the images in the third region in the order of the time-series information contained in the images.

[0026] In the composite image generation apparatus according to the 22nd embodiment, in any one of the first to 21st embodiments, the processor generates a composite image by adding a background to the image displayed in the third region.

[0027] In the 22nd embodiment, the composite image generation apparatus according to the 23rd embodiment generates a composite image using an image selected from the first image as a background.

[0028] In the composite image generation apparatus according to the 24th embodiment, in any one of the first to 23 embodiments, the processor generates a composite image in which images displayed in the third region are switched and displayed.

[0029] A composite image generation system according to the 25th embodiment is a composite image generation system comprising a composite image generation device according to any one of the first to 24 embodiments and an imaging device, wherein the processor causes the image captured by the imaging device to be displayed as a first image in a first area.

[0030] The composite image generation system according to the 26th embodiment includes an operating member that accepts at least a first operation and a second operation, according to the 25th embodiment.

[0031] The composite image generation method according to the 27th embodiment is a composite image generation method performed by a composite image generation apparatus comprising a processor and a display device, wherein the processor displays a plurality of first images in a time series in a first area of ​​the display device, accepts a first operation on the plurality of first images, displays the second image selected by the first operation in a second area which is a different area from the first area of ​​the display device, with improved visibility than the first image, accepts a second operation on the second image, displays the third image selected by the second operation in a third area which is a different area from the first and second areas of the display device, and generates a composite image using the third image. The composite image generation method according to the 27th embodiment may have the same configuration as the second to 24th embodiments.

[0032] The program according to the 28th aspect is a program for operating a composite image generation system comprising a processor, an imaging device, and a display device, wherein the program causes the processor to display a plurality of first images in a time series in a first area of ​​the display device, to accept a first operation on the plurality of first images, to display a second image selected by the first operation in a second area which is different from the first area of ​​the display device, with improved visibility than the first image, to accept a second operation on the second image, to display a third image selected by the second operation in a third area which is different from the first and second areas of the display device, and to generate a composite image using the third image. The program according to the 28th aspect may have the same configuration as the second to 24th aspects. Furthermore, a non-temporary and tangible recording medium that records computer-readable code of the programs of each of the above aspects can also be cited as one aspect of the present invention.

[0033] Figure 1 is a diagram showing the configuration of the system according to the first embodiment. Figure 2 is a front perspective view of the camera-equipped printer. Figure 3 is a rear perspective view of the camera-equipped printer. Figure 4 is a diagram showing the main components of the camera-equipped printer. Figure 5 is a diagram showing the main functions of the camera-equipped printer. Figure 6 is an external view of instant film. Figure 7 is an external view of the information and communication terminal. Figure 8 is a diagram showing the main components of the information and communication terminal. Figure 9 is a diagram showing the main functions of the information and communication terminal. Figure 10 is a diagram showing the processing in the system according to the first embodiment. Figure 11 is a diagram showing the process of selecting the generation of a composite image. Figure 12 is a diagram showing an example of the configuration of the display area. Figure 13 is a diagram showing the process of selecting the first image for the composite image. Figure 14 is a diagram showing the process of selecting the second image. Figure 15 is a diagram showing the process of selecting the ninth and tenth images. Figure 16 is a diagram showing the process of adding effects to the composite image. Figure 17 is a diagram showing an example of an animation effect. Figure 18 is a diagram showing another example of an animation effect. Figure 19 is a diagram showing yet another example of an animation effect. Figure 20 is a diagram showing the configuration of the system according to the second embodiment. Figure 21 is a front perspective view of the printer. Figure 22 is a rear perspective view of the printer. Figure 23 is a diagram showing the processing in the system according to the second embodiment.

[0034] Preferred embodiments of the composite image generation apparatus, composite image generation system, composite image generation method, and program according to the present invention will be described below with reference to the attached drawings. In the following drawings, some components may be omitted from the display in order to make the explanation easier to understand. Also, the drawings do not necessarily show the shape and dimensions of each component accurately.

[0035] [First Embodiment] [System Configuration] Figure 1 is a diagram showing the configuration of system 10 according to the first embodiment. As shown in Figure 1, system 10 comprises an image processing system 1, a server device 500, and an information communication terminal 800.

[0036] The image processing system 1 comprises an information and communication terminal 100 and a camera-equipped printer 200 (composite image generation device, composite image generation system), and can be connected to a server device 500 via a network NW. The user can use the camera-equipped printer 200 to acquire images (image capture), perform operations for generating composite images, communicate with the information and communication terminal 100 (including communication of data necessary for generating composite images (source data)), and print images. In Figure 1, print 99 is a print of a representative image with a two-dimensional code 99A (an example of "information related to the composite image") attached. The composite image and representative image will be described in detail later. In the following description, the representative image may be referred to as the "cover image".

[0037] The information and communication terminal 100 consists of terminal devices such as smartphones, mobile phones, tablet terminals, and personal computers, and performs tasks such as generating composite images based on data received from the camera-equipped printer 200, and communicating with the server device 500 via the network NW (including uploading composite images). Communication between the information and communication terminal 100 and the camera-equipped printer 200 may be wired or wireless. Wireless communication can be performed by short-range wireless communication such as Bluetooth®, Wi-Fi®, or IrDA (Infrared Data Association), but is not limited to these examples (the same applies hereinafter). The connection between the image processing system 1 and the network NW may also be wired or wireless.

[0038] In the first embodiment, the communication terminal 100 communicates with the server device 500, but depending on the information processing performance of the camera-equipped printer 200 (processor processing speed, memory capacity, etc.), the camera-equipped printer 200 may also communicate with the server device 500.

[0039] The server device 500 comprises a server body 510 and a database 520, and communicates (wired or wireless) with the image processing system 1 and the information communication terminal 800. The server device 500 is a device that operates independently of the image processing system 1 and can record data (such as composite images) transmitted and received between the image processing system 1 and the information communication terminal 800 in the database 520 (recording device).

[0040] The information and communication terminal 800, like the information and communication terminal 100, is composed of terminals such as smartphones, mobile phones, tablet devices, and personal computers. It can read the two-dimensional code 99A (information related to the composite image) of the print 99 printed by the camera-equipped printer 200, connect to the server device 500 via the network NW (either wired or wireless), and view or download the composite image. The user of the image processing system 1 and the user of the information and communication terminal 800 may be the same or different.

[0041] The system 10 may include one or more image processing systems 1 and information and communication terminals 800.

[0042] [Configuration of the camera-equipped printer] [External configuration] Figure 2 is a front perspective view of the camera-equipped printer 200, and Figure 3 is a rear perspective view. As shown in these figures, the camera-equipped printer 200 includes a main unit 201. A camera 210 (so-called "outside camera") with a lens 212 is provided on the front (+Z side) of the main unit 201, and the user can take an image of a subject on the +Z side by operating the shutter button 207, and the film after imaging is ejected from the output port 205. The shutter button 207 may be a button that can be pressed halfway through its stroke (so-called "half-press") and pressed all the way through its stroke (so-called "full-press"). In this case, "half-press" determines the exposure and other shooting conditions, and "full-press" acquires the image. The flash device 203 can be fired when taking an image.

[0043] Also, a camera 202 (so-called "in-camera") is provided on the back surface (-Z side) of the main body 201, and the user can use this camera 202 to image a subject existing on the -Z side. This imaging may be so-called "selfie" (the user photographs himself / herself). The camera 202 includes a lens 204.

[0044] The user can add voice to an image by using the microphone 244 during imaging, and the voice attached to the image can be output from the speaker 246. The monitor 292 (display device) is composed of a device such as a liquid crystal display device, and is used as a display screen of the imaged image, an editing screen of the composite image, a printing instruction screen, and an output screen of a message from the system to the user.

[0045] The lock switch 208 is a switch for locking or unlocking the film door 293. When the lock is released, the film door 293 is opened, and it becomes possible to store a film pack in the main body 201. The card slot 245 is a slot for storing a recording medium such as a memory card. A detachable recording medium such as a memory card may be inserted into the card slot 245 and used as the data memory 238.

[0046] On the back surface (-Z side surface) of the main body 201, a dial 226A (operation member), a Menu / OK button 226B (operation member), a print button 226C, a "return" button 226D, a play button 226E, and a mode switch button 226F are provided. Also, on the -X side surface of the main body 201, a button 226G for switching the cameras (camera 210, camera 202) used for shooting and a power button 226H are provided. The dial 226A is rotatable and can be pushed in the up, down, left, and right directions, and is used for an image scroll operation (first operation) and an image selection operation (second operation) for a composite image, which will be described later. The Menu / OK button 226B to the power button 226H are composed of push buttons (buttons that are pushed down). The dial 226A to the power button 226H constitute the operation unit 226 (see FIG. 4).

[0047] In the first embodiment, the case where the printer 200 with a camera includes hardware buttons and switches has been described. However, the monitor 292 may be configured by a touch panel display device, and the user may be able to perform various operations (which may include the first operation and the second operation) on the touch panel.

[0048] [Internal Configuration] Figure 4 is a diagram showing the main configuration of the printer 200 with a camera. The printer 200 with a camera includes a system controller 222 (processor) that controls each part and performs image processing. Also, each part described below is mainly connected via a bus 260. The printer 200 with a camera also includes a power supply device (not shown), and power is supplied from this power supply device to each part.

[0049] In addition, as a configuration for acquiring an image, the printer 200 with a camera includes, in addition to the above-described camera 202 (in-camera) and camera 210 (out-camera), a signal processing unit 216, a lens driving unit 218, and an imaging element driving unit 220. The camera 202 includes a lens 204 and an imaging element 206, and the camera 210 includes a lens 212 and an imaging element 214. The lens 204 and the lens 212 are composed of one or more lenses and may include a zoom lens and a focus lens. The lens driving unit 218 is used for the forward and backward movement and driving of those zoom lenses and focus lenses. As the imaging elements 206 and 214, various photoelectric conversion elements such as CMOS (Complementary Metal-Oxide Semiconductor) and CCD (Charge-Coupled Device) can be used, and the control of reading the signals from the imaging elements 206 and 214 is performed by the imaging element driving unit 220. Image generation based on the read signals is performed by the signal processing unit 216. Data of the imaging time can be associated with the generated image.

[0050] Cameras 202 and 210 may have independent image sensors, as shown in Figure 4, or they may share a common image sensor. In addition to or instead of capturing images with cameras 202 and 210, the camera-equipped printer 200 may acquire images via the card slot 245 or by communication with external devices via the information and communication terminal 100, and print or use the acquired images in composite images.

[0051] [Functions of the Camera-Equipped Printer] Figure 5 shows the main functions of the camera-equipped printer 200. As shown in Figure 5, the camera-equipped printer 200 has an imaging control unit 200A, a display control unit 200B, an image processing unit 200C, a communication control unit 200D, a recording control unit 200E, and a print control unit 200F.

[0052] In the first embodiment, each of the above functions can be executed by any computer, including the system controller 222. This computer may also execute these processes using a hardware processor, a software program, or a combination thereof. In that case, the hardware processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as each unit or means in this embodiment. Furthermore, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. The computer may be a general-purpose computer, a computer for a specific application, a workstation, or any other system capable of executing each process. The units shown in Figure 4 (for example, the signal processing unit 216, the lens drive unit 218, the image sensor drive unit 220, etc.) may perform some of the functions shown in Figure 5.

[0053] The "computer" such as the system controller 222 can be configured using one or more hardware components, and the type of hardware is not limited. For example, the computer may be composed of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing specific processing such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the computer has various parts (Units) or means (Means) that execute the various processing in this embodiment. The type of hardware may also be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processing steps of a processor, these multiple hardware components may reside in physically separate devices or in the same device. In any embodiment, the order of processing steps performed by the processor is not particularly limited and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0054] Furthermore, in this embodiment, the computer described above may be realized by hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are composed of a program, which may include a program for operating the camera-equipped printer 200 (synthetic image generation device, synthetic image generation system) (a program for executing the operating method according to the present invention, which is one aspect of the program according to the present invention). The program may also be, for example, a group of program modules, and each of its functions may be realized by a processor configured to execute its respective function. The program may also be program code or multiple code segments stored in one or more non-temporary and tangible computer-readable media (e.g., storage media or other storage). The "non-temporary and tangible computer-readable media" may be ROM 236 or data memory 238, or EEPROM or flash memory, etc. (the same applies hereinafter). Individual identification information of the camera-equipped printer 200 can also be stored in these "non-temporary and tangible computer-readable media".

[0055] A program may be divided and stored on multiple non-temporary, tangible computer-readable media located on devices that are physically separated from each other. Program code, or code segments, may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code, or code segments, may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0056] In this embodiment, the "non-temporary and tangible computer-readable medium" does not include intangible recording media such as carrier signals or propagated signals themselves. The computer, such as the system controller 222, can use the RAM 240 as a temporary storage area or work area when processing using a program.

[0057] [Communication Configuration] The camera-equipped printer 200 is equipped with a wireless transceiver 224 and can communicate wirelessly with the information communication terminal 100 (see Figure 1). Wired communication may also be performed by connecting a cable to the input / output interface 242. The GPS receiver 252 receives signals from satellites of a positioning system such as GPS (Global Positioning System), and the system controller 222 can calculate position information indicating the location of the camera-equipped printer 200 based on these signals. The GPS receiver 252 may also receive signals from satellites of positioning systems other than GPS, such as GLONASS (Global Navigation Satellite System) or the Japanese version of GPS.

[0058] [Switching shooting modes using the mode switch button] The mode switch button 226F is used to switch between four modes: (1) normal shooting (still image shooting) mode, (2) shooting mode with sound, (3) simple photolog mode, and (4) layered shooting mode.

[0059] [Shooting with Sound] In the "Shooting with Sound Mode" described in (2) above, still image data with sound is generated. "Still image data with sound" refers to still image data associated with sound data. When shooting in Shooting with Sound Mode, the sound immediately after shooting, or the sound before and after shooting, is captured and recorded associated with the still image data. The user can choose whether to record the sound immediately after shooting or the sound before and after shooting. The recording time is predetermined, for example it is 10 seconds, but other times are also acceptable. If recording the sound immediately after shooting, the sound for 10 seconds after fully pressing the shutter button 207 will be recorded. If recording the sound before and after shooting, the sound for 5 seconds before and 5 seconds after fully pressing the shutter button 207 will be recorded. Pressing the Menu / OK button 226B during recording will forcibly terminate the recording.

[0060] Audio recording is possible even after shooting. If the user selects the "Record Later" option (not shown) from the generation menu during image generation, audio will be captured and recorded, associated with the image data being played back. Audio capture is, for example, limited to 10 seconds from the start of recording. If audio data is already associated with an image, it can be overwritten.

[0061] In the "Simple Photolog Mode" described in (3) above, you can easily create a photo movie with sound (a moving image with animation effects) by taking normal photos or photos with sound a certain number of times (for example, 10 times). The "Layered Shooting Mode" described in (4) above is a mode that creates an image by layering separately taken images as the foreground and background.

[0062] [Printer Configuration] The camera-equipped printer 200 includes a printer 230 (output device, printer). The printer 230 includes a film transport mechanism 232, a print head 233, and a print head drive unit 234, and can print the following images on instant film, which will be described later. Details of (3) will be described later. (1) Printing of normal images (still images) (2) Printing of still images with sound (information such as barcodes or QR codes for accessing the sound data is superimposed on the image and printed) (3) Printing of a representative image of a simple photolog (information such as barcodes or QR codes for accessing the composite image is superimposed on the representative image of the simple photolog and printed) (4) Printing of layered photos

[0063] The film transport mechanism 232 includes a pair of transport rollers and a pair of unfolding rollers (not shown), and the print head 233 includes a line-type exposure head (not shown). The printer 230 also includes a film storage section (not shown), and when the film door 293 is opened by the aforementioned lock switch 208, an instant film pack can be stored in the film storage section.

[0064] [Instant Film] Figure 6 is an external view of the instant film 310 (part (a) is a front view, and part (b) is a rear view). The instant film 310 is a self-developing instant film, and the camera-equipped printer 200 uses an instant film pack in which multiple sheets of instant film 310 are housed in a case not shown. In the present invention, "self-developing instant film" is a film that has a developing solution pod containing a developing solution, and when the film is passed between a pair of rollers, the developing solution is squeezed out from the pod and spread in the exposure area, and the developing process is carried out.

[0065] In Figure 6, the direction indicated by arrow F is the direction in which the instant film 310 is used. That is, the instant film 310 is transported in the direction indicated by arrow F and used. Therefore, when loaded into the camera-equipped printer 200, the direction indicated by arrow F is the direction in which the instant film 310 is ejected.

[0066] The instant film 310 is a self-developing instant film having a rectangular card shape. The instant film 310 is configured with an exposure surface 310a on the back side and an observation surface 310b on the front side. The exposure surface 310a is the surface that records an image through exposure, and the observation surface 310b is the surface for observing the recorded image.

[0067] As shown in part (a) of Figure 6, the observation surface 310b of the instant film 310 is provided with an observation section 318. The observation section 318 is the part where the image is displayed (image forming area). By developing the exposure section 312 (see part (b) of Figure 6), an image is displayed on the observation section 318. The observation section 318 is positioned in correspondence with the exposure section 312. A frame 318a is provided around the observation section 318, and the image is displayed within the frame 318a (image forming area). As will be described in detail later, the observation section 318 (image forming area) can display barcodes or two-dimensional codes (such as the URL (Uniform Resource Locator) where the composite image is stored and information corresponding to that URL; information related to the composite image) superimposed on the image.

[0068] The aspect ratio of the observation section 318 is the same as that of the exposure section 312, 1:1, and it has a square shape. On the observation surface 310b, the area where the pod section 314 (see part (b) in Figure 6; developer pod) is located on the back surface, that is, the area on one side of the instant film 310 (the front side in the transport direction) that is wider in the transport direction, separate from the image forming area on the observation surface 310b, is the margin area 315, and the user can write a message in the margin area 315.

[0069] As shown in part (b) of Figure 6, the exposure surface 310a of the instant film 310 is provided with an exposure section 312, a pod section 314, and a trap section 316.

[0070] The exposure unit 312 is the part that prints (records) an image by exposure. As described above, a two-dimensional code may be superimposed on the image and printed. The pod unit 314 and the trap unit 316 are arranged front and back (front and back) on either side of the exposure unit 312. The instant film 310 has a square-shaped exposure unit 312 with an aspect ratio of, for example, 1:1.

[0071] The pod section 314 is positioned in front of the exposure section 312 in the direction of arrow F. The pod section 314 contains a developing solution pod 314a that encloses the developing solution. In the first embodiment, the area of ​​the pod section 314 on the observation surface 310b side is the blank area 315.

[0072] The trap section 316 is positioned behind the exposure section 312 in the direction of arrow F. The trap section 316 contains an absorbent material 316a for the developing solution.

[0073] Normally, the instant film 310 is printed with the trap section 316 at the top of the image and the pod section 314 at the bottom of the image, and is viewed with the trap section 316 at the top and the pod section 314 at the bottom (the opposite orientation to when it was printed). However, the image can be rotated and printed according to the user's instructions. In the initial state (when there are no user instructions), the orientation of the image (whether the trap section 316 or the pod section 314 is on top, or whether the margin area is on the top or bottom of the image) can be set by operating the rear control panel 294.

[0074] After exposure, the instant film 310 is developed by spreading the developing solution from the pod section 314 onto the exposure section 312. The developing solution from the pod section 314 is squeezed out of the pod section 314 by passing the instant film 310 between the roller pair and spread onto the exposure section 312. Any excess developing solution during the spreading process is captured by the trap section 316.

[0075] [Configuration of the Information and Communication Terminal] Figure 7 is an external view of the information and communication terminal 100, where (a) is a front perspective view and (b) is a rear perspective view. In the first embodiment, the case in which the information and communication terminal 100 is a so-called smartphone will be described, but in the present invention, the information and communication terminal 100 can be composed of terminals other than smartphones, such as mobile phones, tablet terminals, and personal computers.

[0076] The information and communication terminal 100 shown in Figure 7 has a flat casing 102, and one side of the casing 102 is equipped with a display input unit 120 which integrates a display panel 121 (display device) as a display unit and an operation panel 122 (operation unit) as an input unit. The casing 102 also includes a speaker 131, a microphone 132, an operation unit 140 (operation unit), a camera unit 141 (so-called front camera), a camera unit 142 (so-called rear camera), and a strobe 43. The configuration of the casing 102 is not limited to this, and for example, a configuration in which the display unit and the input unit are independent may be adopted, or a configuration with a folding structure or a sliding mechanism may be adopted.

[0077] Figure 8 shows the main configuration of the information and communication terminal 100 shown in Figure 7. As shown in Figure 8, the main components of the information and communication terminal 100 include a wireless communication unit 111, a display input unit 120, a call unit 130, an operation unit 140, camera units 141 and 142, a strobe 143, a storage unit 150, an external input / output unit 160, a GPS receiver unit 170 (GPS: Global Positioning System), a motion sensor unit 180, a power supply unit 190, and a main control unit 101. In addition, the main function of the information and communication terminal 100 is a wireless communication function that performs mobile wireless communication via a base station device and a mobile communication network.

[0078] The wireless communication unit 11 performs wireless communication with base station equipment connected to the mobile communication network in accordance with instructions from the main control unit 101. Using such wireless communication, it sends and receives various file data such as voice data and image data, as well as email data, and receives web data and streaming data.

[0079] The display input unit 120 is a so-called touch panel that, under the control of the main control unit 101, displays images (still images and / or moving images) and text information to visually convey information to the user, and also detects user operations on the displayed information, and comprises a display panel 121 and an operation panel 122.

[0080] In the display panel 121, an LCD (Liquid Crystal Display), OLED (Organic Electro-Luminescence Display), etc., is used as the display device. The operation panel 122 is a device that detects one or more coordinates operated by a conductor such as a user's finger or pen, and is visibly mounted on the display surface of the display panel 121. When such a device is operated by a conductor such as a user's finger or pen, the operation panel 122 outputs a detection signal generated by the operation to the main control unit 101. The main control unit 101 then detects the operation position (coordinates) on the display panel 121 based on the received detection signal.

[0081] As shown in Figures 7 and 8, the display panel 121 and the operation panel 122 of the information communication terminal 100, which is illustrated as one embodiment of the imaging device of the present invention, together constitute a display input unit 120, with the operation panel 122 completely covering the display panel 121. When such an arrangement is adopted, the operation panel 122 may also have a function to detect user operations in areas outside the display panel 121. In other words, the operation panel 122 may have a detection area for the overlapping portion that overlaps the display panel 121 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap the display panel 121 (hereinafter referred to as the non-display area).

[0082] The communication unit 130 is equipped with a speaker 131 and a microphone 132, and can convert the user's voice input through the microphone 132 into audio data that can be processed by the main control unit 101 and output it to the main control unit 101, and can decode audio data received by the wireless communication unit 111 or the external input / output unit 160 and output it from the speaker 131. Also, as shown in Figure 7, for example, the speaker 131 can be mounted on the same side as the display input unit 120, and the microphone 132 can be mounted on the side of the housing 102.

[0083] The operation unit 140 is a hardware key, such as a key switch, that receives instructions from the user. For example, as shown in Figure 7, the operation unit 140 is mounted on the side of the housing 102 of the information and communication terminal 100 and is a push-button type switch that turns on when pressed with a finger and turns off when the finger is released due to a restoring force such as a spring.

[0084] The storage unit 150 stores control programs and control data from the main control unit 101, application software, address data associated with the names and phone numbers of communication partners, sent and received email data, web data downloaded through web browsing, downloaded content data, and also temporarily stores streaming data. The storage unit 150 is composed of an internal storage unit 151 built into the smartphone and an external storage unit 152 with a removable external memory slot. The internal storage unit 151 and the external storage unit 152 that constitute the storage unit 150 can be implemented using known storage media.

[0085] The external input / output unit 160 serves as an interface with all external devices connected to the information and communication terminal 100. The information and communication terminal 100 is directly or indirectly connected to other external devices via communication through the external input / output unit 160. Examples of means of communication include Universal Serial Bus (USB), IEEE 1394, and networks (e.g., the Internet, wireless LAN). Other examples of means of communication include Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra-Wide Band) (registered trademark), and ZigBee (registered trademark).

[0086] External devices that can be connected to the information and communication terminal 100 include, for example, wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards and SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards connected via card sockets. External audio and video equipment connected via audio and video I / O (Input / Output) terminals, wirelessly connected external audio and video equipment, wired / wireless connected smartphones, wired / wireless connected PDAs, wired / wireless connected personal computers, and earphones can also be connected. The external input / output unit 160 can transmit data received from such external devices to the various internal components of the information and communication terminal 100, and can also transmit data from inside the information and communication terminal 100 to external devices.

[0087] The motion sensor unit 180 includes, for example, a three-axis acceleration sensor and a tilt sensor, and detects the physical movement of the information and communication terminal 100 according to the instructions of the main control unit 101. By detecting the physical movement of the information and communication terminal 100, the direction of movement, acceleration, and orientation of the information and communication terminal 100 are detected. These detection results are output to the main control unit 101. The power supply unit 190 supplies power stored in a battery (not shown) to each part of the information and communication terminal 100 according to the instructions of the main control unit 101.

[0088] The main control unit 101 is equipped with a microprocessor and operates according to the control programs and control data stored in the memory unit 150, and comprehensively controls each part of the information communication terminal 100, including the camera unit 141. In addition, the main control unit 101 has a mobile communication control function that controls each part of the communication system for voice communication and data communication via the wireless communication unit 111, and an application processing function.

[0089] Furthermore, the main control unit 101 is equipped with image processing functions, such as displaying video on the display input unit 120 based on image data (still image data and moving image data) such as received data and downloaded streaming data. The image processing function refers to the function in which the main control unit 101 decodes the image data, applies image processing to the decoded result, and displays the image on the display input unit 120.

[0090] Camera units 141 and 42 are digital cameras (imaging devices) that perform electronic imaging using image sensors such as CMOS and CCD. Furthermore, under the control of the main control unit 101, camera units 141 and 142 can convert image data (video and still images) obtained through imaging into compressed image data such as MPEG or JPEG, record it in the storage unit 150, or output it via the external input / output unit 160 or the wireless communication unit 111. Camera unit 141 can also, under the control of the main control unit 101, split and combine videos, acquire high-quality still images (RAW images, etc.), rearrange and process frames, and extract still images from videos. In the information communication terminal 100 shown in Figures 7 and 8, shooting can be performed using either camera unit 141 or 142, or by using both camera units 141 and 142 simultaneously. When using camera unit 142, the strobe 143 can be used.

[0091] Furthermore, the camera units 141 and 142 can be used for various functions of the information and communication terminal 100. For example, the information and communication terminal 100 can display images acquired by the camera units 141 and 142 on the display panel 121. The information and communication terminal 100 can also use images from the camera units 141 and 142 as one of the operation inputs for the operation panel 122. In addition, the information and communication terminal 100 can also use images from the camera units 141 and 142 to detect the position when the GPS receiver 170 detects the position. Moreover, the information and communication terminal 100 can use images from the camera units 141 and 142 to determine the optical axis direction of the camera unit 141, or to determine the current operating environment, either without using the 3-axis accelerometer or in combination with the 3-axis accelerometer, by referring to the images from the camera units 141 and 142. Of course, the information and communication terminal 100 can also use images from the camera units 141 and 142 within application software. In addition, the information and communication terminal 100 can also record still or video image data in the storage unit 150 by adding location information acquired by the GPS receiver 170, voice information acquired by the microphone 132 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 180, etc. The information and communication terminal 100 can also output this still or video image data through the external input / output unit 160 or the wireless communication unit 111. The information and communication terminal 100 can also transmit images to the camera-equipped printer 200.

[0092] [Functional Configuration of Information and Communication Terminal] Figure 9 shows the main functions of the information and communication terminal 100. As shown in Figure 9, the information and communication terminal 100 includes an imaging control unit 100A, a display control unit 100B, an image processing unit 100C, a communication control unit 100D, and a recording control unit 100E. These functions are mainly realized by the main control unit 101 (processor), but some functions may be handled by the parts shown in Figures 7 and 8 (wireless communication unit 111, camera units 141, 142, external input / output unit 160, etc.).

[0093] The main control unit 101 and the components shown in Figure 9 can be implemented using hardware (various processors, electrical circuits, etc.), software (programs), firmware, microcode, or a combination thereof, similar to what was described above for the camera-equipped printer 200. The software can be stored in ROM or EEPROM (non-temporary and tangible recording media) not shown, and RAM (not shown) can be used as a temporary storage area or working area. The ROM or RAM may be provided in the main control unit 101 or in the internal storage unit 151.

[0094] [Processing in the System] Figure 10 is a diagram showing the processing in the system according to the first embodiment, and Figures 11 to 16 are diagrams showing examples of operation screens. The generation of composite images in the system 10 (information communication terminal 100, camera-equipped printer 200, server device 500) with the above configuration will be explained with reference to these diagrams as appropriate.

[0095] [Generation of Data for Composite Images] The system controller 222 (processor) of the camera-equipped printer 200 generates data for composite images (material data) using images, audio data associated with images (ambient sound; first audio data), background audio data such as BGM (Background Music) (background sound; second audio data), background images, etc. (Step S11 in Figure 10). The generation of data for composite images will be described below. In the first embodiment, the camera-equipped printer 200 generates material data for the composite image, and the information communication terminal 100 generates the composite image as a video container using the material data. However, this embodiment is also included in "generation of composite images" in the present invention.

[0096] [Menu Display and Selection] Figure 11 shows the selection of "Generate Composite Image" on the camera-equipped printer 200. The system controller 222 (display control unit 200B, etc.) can display any image 90A on the screen 90 (monitor 292). Image 90A may be a single image (for example, the latest image) or multiple images may be displayed in chronological order. Icon 90B indicates that image 90A contains audio data.

[0097] Menu 90C is displayed at the bottom of screen 90, and the user can select the desired menu by rotating or pressing dial 226A up or down. When the user selects "Sound Log Print" from this menu, a composite image can be generated and a representative image printed. The following explanation assumes that "Sound Log Print" has been selected. Alternatively, the menu may be displayed and / or selected by operating the mode switching button 226F.

[0098] [Image Acquisition] When generating composite images, you may use images taken in the normal shooting (still image shooting) mode, sound-enabled shooting mode, or layered shooting mode described above, or you may use images taken in the simple photolog mode. Alternatively, you may acquire images taken with other devices via recording media or a network.

[0099] [Configuration of Display Areas in the Generation of Composite Images] Figure 12 is a diagram showing an example of the configuration of the display areas in the monitor 292 for the generation of composite images (the specific image content is omitted in this figure). As shown in part (a) of Figure 12, the monitor 292 can display by dividing it into multiple display areas (in the example shown in this part, the first area 901, the second area 902, and the third area 903). The first area 901 is an area where multiple images 901A (first images) are displayed in chronological order, and the second area 902 (second area) is an area where an image 902A (second image) selected from among the multiple images 901A in the first area is displayed with improved visibility than image 901A. When image 902A is selected by a predetermined operation (second operation), the selected image (image 903A; third image) is displayed in the third area 903, which is different from the first area 901 and the second area 902. A composite image is generated using this third image. Symbol 903B indicates the image that has been selected from among the images used to create the composite image.

[0100] Part (b) of Figure 12 shows the display of the fourth area 904. The fourth area 904 is an area where executable operations are displayed along with button images. In the example shown in this part, area 904A indicates that the selection is confirmed by pressing the Menu / OK button 226B, and area 904B indicates that pressing the Play button 226E moves to background selection for the composite image. Other operations may also be shown in the fourth area 904. The fourth area 904 may be displayed at all times, or only during the period when operations are possible. The fourth area 904 may be displayed in a way that obstructs the display in other areas, or it may be displayed semi-transparently without completely obstructing it. It is preferable to distinguish and display "executable operations (selectable operations)" and "operations that are not executable (selectable) at the time of display, but will soon become executable" in the fourth area 904.

[0101] [Image Selection and Display] The screen 91 shown in Figure 13 illustrates the process of selecting the first image for the composite image. The composite image is generated using multiple images, and in the first embodiment, a case using up to 10 images will be described, but the number of images can be any number. In Figure 13, areas etc. having the same reference numerals as in Figure 12 mean the same areas etc. as in Figure 12.

[0102] [Display of images in chronological order] The system controller 222 (display control unit 200B, etc.; processor) displays multiple images 911, 912, 913 (multiple first images) in chronological order in the first area 901 (first area) of the monitor 292, as shown in screen 91 in part (a) of Figure 13. The system controller 222 can, for example, display images stored in a designated folder, with older images at the top of the screen and newer images at the bottom, and can also display images in chronological order according to the creation time or editing time of the image files.

[0103] [Identification and display of images containing audio data] The system controller 222 also identifies and displays images (in this case, images 911 and 912) by adding microphone-shaped icons 911B and 912B to the images if the images displayed in the first area 901 contain audio data (first audio data). Identification may also be done using other icons such as speakers or musical notes, or by means other than icons (adding letters, numbers, symbols, shapes, coloring, etc.). Here, "images containing audio data" may be images taken in "sound-enabled shooting mode" or "simple photolog mode" that contain audio data, or images to which audio has been associated retrospectively.

[0104] [Adding audio data to a composite image] The system controller 222 can add at least a portion of the audio data (first audio data) contained in the image to the composite image. Specifically, based on user operation, the system controller 222 can set which range of the audio data contained in the image is to be added to the composite image, and the volume of the audio data. As will be described later, based on user operation, it can also set the relationship between the volume of the first audio data and the volume of ambient sound (second audio data).

[0105] [First Evaluation of Images and Output of Information Based on the First Evaluation] The system controller 222 (image processing unit 200C, etc.) performs a first evaluation of the image displayed in the first area 901 when selecting an image as described above, and can output first information regarding the results of the first evaluation. The output of the first information can be performed at any time, for example, when it becomes possible to select an image to be used for composite images. The first evaluation can include, for example, evaluating whether the blur, shake, and brightness of the image are within a specified range, what kind of subject (e.g., a person) is in the image, and whether the person is smiling, whether the entire face of the person is visible, whether there are other images with similar subjects and compositions, and whether there are other images taken at a similar time. However, the evaluation is not limited to these items.

[0106] The system controller 222 may perform the first evaluation using AI (Artificial Intelligence) for image processing constructed using machine learning algorithms such as deep learning, or it may perform the first evaluation using image processing means other than AI. The criteria for the first evaluation may be, for example, "the numerical value of the evaluation result for a specific evaluation item is above a threshold" or "the rank of the evaluation result among multiple images for a specific evaluation item is above a threshold," but other criteria may be used. A comprehensive evaluation may be performed by weighting and adding the evaluation results for multiple evaluation items. The system controller 222 may set the criteria for the first evaluation in response to user operation, or it may set the criteria independently of user operation.

[0107] The system controller 222 can output information as first information recommending that images satisfying the first criteria be included in the composite image based on the results of the first evaluation. The output of the first information can be done by identifying the first image with characters, numbers, shapes, symbols, coloring, etc. (for example, by adding the word "like", a star, a heart, etc.), or by outputting audio from the speaker 246.

[0108] This initial information allows users to easily generate composite images using highly-rated images.

[0109] [Selection of the second image and improvement of visibility] On screen 91, the user can rotate or press down the dial 226A (operating member), and the system controller 222 accepts this operation (first operation) and scrolls the image displayed in the first area 901 in the vertical direction of the figure. The system controller 222 then selects the image (image 911 in the example of Figure 13) displayed at a predetermined position in the first area 901 (for example, the center, but not limited to this) by this scrolling and displays it as the second image 911A (second image) in the second area 902 in the upper right of the screen with improved visibility compared to images 911 to 913. If the visibility is different, the content of the image itself does not need to be changed. In the example shown in Figure 13, the system controller 222 displays the second image 911A larger than the first image 911 to improve visibility. In addition to this, or instead, the visibility of the second image may be improved by adding one or more characters, symbols, numbers, or shapes (such as outlines), or by changing one or more of the colors, brightness, or display patterns (such as image blinking) from the first image. When displaying the first image 911 as the second image 911A, the system controller 222 may also add animation effects (such as the image transforming, enlarging, or moving, or the addition of sound).

[0110] The system controller 222 also highlights the first image displayed at the "determined position" mentioned above, i.e., the image selected as the second image (image 911 in the example of Figure 13). In the example of Figure 13, a border line 911C of a different color from the background is added to image 911, but the manner of highlighting is not limited to this. The first image may be highlighted by adding one or more of the following to the first image displayed at the determined position: a character, symbol, number, or graphic; or by changing one or more of the color, brightness, or display pattern of the first image displayed at the determined position from that of other first images among a plurality of first images.

[0111] [Confirming Image Selection and Display in Third Area, etc.] In the state shown in part (a) of Figure 13, as described above for Figure 6, the menus that the user can select are displayed (brightly) in the fourth area 904. In this state, it is indicated that "the user can press the Menu / OK button 226B, thereby selecting the second image 911A," and that "the 'Background Selection' menu is displayed dimly, and in this state, the background cannot be selected (because no images have been selected)." The brightness and darkness of the menus are one way of distinguishing between selectable and non-selectable menus, and other ways of distinguishing them (changing the color, changing the font weight, etc.) may also be used (the difference will be clear if you compare the example in part (a) and part (b) of Figure 13).

[0112] When the user presses the Menu / OK button 226B (an example of the "second operation"), the system controller 222 displays the second image 911A (image 911) selected by this operation as the third image 911E (third image) in the third region 903 (third region), which is a different region from the first region 901 and the second region 902 (see part (b) of Figure 13). This completes the "selection of the first image".

[0113] Once the image selection is complete, the system controller 222 moves the symbol 903B indicating the selected image to the position of the second image, and adds the number 911D (here, "1") to the image (image 911) displayed in the first area 901 in the order in which the images were selected (in the order in which the second operation was performed). The user can easily understand which image has been selected by viewing the list of images in the third area 903 and the first area 901. Once one or more images have been selected, the system controller 222 brightens the "Background Selection" which was dimmed in the fourth area 904 in the state shown in part (a) of Figure 13, allowing the user to select a background. The user can select a background at any time after one or more images have been selected.

[0114] [Second Evaluation] The system controller 222 may perform a second evaluation (evaluation of the image on which the second operation has been performed) and output second information regarding the results of the second evaluation. The "image on which the second operation has been performed" is the image on which the Menu / OK button 226B has been pressed (second operation) while it is displayed in the second area 902, and is the image used to generate the composite image. The "second information" is, for example, information recommending that the user print the image on which the result of the second evaluation satisfies the second criterion. Images with a high second evaluation are considered suitable not only for use in composite images but also for printing on their own, and the camera-equipped printer 200 according to the first embodiment can suggest such printing to the user. The user can easily print images with a high evaluation based on such second information. The system controller 222 can perform the second evaluation using the same AI or image processing means as the first evaluation, and can adopt the same output method as the first evaluation.

[0115] [Image Selection: Second Image] In the camera-equipped printer 200, images to be used to generate a composite image can be selected in the same way for the second and subsequent images. Figure 14 shows the process of selecting the second image after selecting the first image (screen 92). In part (a) of the figure, multiple first images are scrolled by rotating or pressing down the dial 226A (first operation) as described above, and image 912 (first image) is positioned in the center (determined position) of the first area 901. Pressing down the Menu / OK button 226B selects this image 912, and it is displayed in the second area 902 as the second image, image 912A (second image). The system controller 222 enhances visibility by enlarging image 912A (image 912), and also highlights the selected image 912 by adding a border line 912C. The modes of visibility and highlighting can be performed in various ways, similar to those described above for the first image.

[0116] In this state, when the user presses down the Menu / OK button 226B (second operation), the system controller 222 selects image 912A (image 912) and displays it as the third image 912E in the third area 903. The system controller 222 also adds the number 912D (here, "2") to image 912 in the first area 901 according to the order in which the second operation was performed, and moves the symbol 903B in the third area 903 to the third position.

[0117] Furthermore, if a second or subsequent image is selected, the system controller 222 may display those images in the third area 903 in the order they were selected, or in the order of the time-series information contained in the images.

[0118] [Image Selection: 9th and 10th Images] Figure 15 shows the process of selecting the 9th and 10th images. Specifically, part (a) of Figure 15 shows the screen 93 for selecting the 9th image, and part (b) of the same figure shows the screen 94 for selecting the 10th image. The user can select the 9th and 10th images in the same way as the 1st and 2nd images.

[0119] [Adding effects to composite images] [Adding background sound and adjusting volume balance] The camera-equipped printer 200 can add BGM (background sound: second audio data) to the composite image. This BGM may be audio data recorded at the time of product shipment, audio data acquired via a network NW, or audio data acquired via a recording medium such as a memory card. The BGM can be one recorded in the data memory 238 (recording medium) or ROM (recording medium), but it may also be temporarily recorded in the RAM 240 (recording medium) during processing. The system controller 222 accepts an operation (third operation) in which the user selects BGM using the dial 226A or the Menu / OK button 226B, and can add at least a part of the BGM selected by that operation to the composite image. The system controller 222 accepts an operation to select which BGM to add and an operation to set which range of the selected BGM to add, and adds the BGM according to the accepted content. In this case, it is preferable to accept the user's operation via the screen of the monitor 292.

[0120] Figure 16 shows how effects are added to a composite image. When adding background music (BGM), as shown in screen 95 in part (a) of Figure 16, the user can set the relationship between the BGM volume and the volume of ambient sound (sound contained in the image: first audio data, displayed as "recorded sound" in Figure 16). Screen 95 displays a representative image 95A (representative image) of the composite image and an area 95B for adjusting the volume balance. The user can adjust the volume balance by moving the circle 95C in area 95B left or right using the dial 226A or the Menu / OK button 226B (pressed left or right). If the monitor 292 is configured as a touch panel, the user may perform such operations using the touch panel. The system controller 222 can set the volume balance according to such operations. In the example of screen 95, ambient sound is labeled "recorded sound".

[0121] The system controller 222 can output background music and ambient sounds from the speaker 246 at a set volume balance and prompt the user for confirmation. The user can also confirm their selection using the Menu / OK button 226B, and the system controller 222 adds audio data to the composite image in response to this operation.

[0122] [Setting the Background] Part (b) of Figure 16 is an example of a screen showing how to set the background of a composite image. In the screen 96 shown in this part, multiple images (in this case, solid colors) that can be set as the background are displayed in the lower area 96A. Different patterns in area 96A correspond to different images on the actual screen. These images do not have to be solid colors; they may be gradients or combinations of multiple colors, and may also have letters, numbers, shapes, symbols, patterns, etc. Also in area 96A, the currently selected image 96B is identified and displayed by a border line 96C. Furthermore, in addition to the solid color images, a captured image 96D is displayed in area 96A. This captured image 96D may also be the image displayed in the first area 901 (first image). The user can move the border line 96C of area 96A up, down, left, and right using the dial 226A or the Menu / OK button 226B to select the desired image, and the system controller 222 sets the image corresponding to this operation as the background of the composite image. It is preferable that the system controller 222 switches the background of the screen 96 in accordance with the movement of the frame line 96C (switching the selection of the background image).

[0123] [Selection of Animation Effects for Composite Images] Part (c) of Figure 16 is an example of a screen showing how to set the animation effects when composite images are switched and displayed. In the screen 97 shown in this part, icons 97B, 97C, and 97E indicating selectable patterns are displayed in the area 97A at the bottom of the screen, and the user can select the desired image using the dial 226A or the Menu / OK button 226B. Icon 97B indicates a pattern in which multiple images slide horizontally to switch, icon 97C indicates a pattern in which multiple images slide vertically to switch, and icon 97E indicates a pattern in which multiple images overlap and switch while slightly shifting. These patterns are examples of the switching display of images (images selected to be used for composite images) displayed in the third area 903 described above. Specific embodiments of these patterns will be explained in the section "Specific Examples of Animation Effects".

[0124] These patterns are just examples of animation effects, and users may be allowed to select other patterns. For example, when images slide or stack, users may be allowed to select patterns that change or transform the image size, or add color or patterns. Alternatively, users may be allowed to select patterns that simply switch between images without any animation effects. The selected pattern (in the example of section (c) in Figure 16, the pattern shown by icon 97C) is identified by a border line 97D.

[0125] In addition, in the screens shown in parts (b) and (c) of Figure 16, the user can confirm their selection using the Menu / OK button 226B, similar to part (a) of the same figure.

[0126] The system controller 222 (display control unit 200B, image processing unit 200C, etc.) can display a composite image according to the selected pattern. The system controller 222 can add the aforementioned background image and sound data (BGM and / or ambient sound, i.e., first sound data and / or second sound data) to this display.

[0127] [Specific Examples of Animation Effects (Part 1)] Specific examples of animation effects for composite images will be explained with reference to Figures 17 to 19. Figure 17 shows the animation effect corresponding to the icon 97B described above. In Specific Example (Part 1), as shown in part (a) of Figure 17, the first image 921A among the multiple images used in the composite image is displayed in the center of the screen 921, and a part of the next image 921B (near the left edge) is displayed to its right. In this state, the system controller 222 stops the movement of the images for a predetermined time, and then slides images 921A and 921B to the left side of the screen 921. Part (b) of Figure 17 shows the state after the slide, with a part of the first image 921A (near the right edge) displayed at the left edge of the screen 921, the second image 921B in the center, and a part of the third image 921C at the right edge. This sliding is repeated until the last image of the composite image (for example, the 10th image) is displayed in the center. The system controller 222 may also slide the image to the right or diagonally across the screen 921.

[0128] [Specific Example of Animation Effect (Part 2)] Figure 18 shows an animation effect corresponding to the icon 97C described above. In specific example (part 2), as shown in part (a) of Figure 18, the first image 922A of the multiple images used in the composite image is displayed in the center of the screen 922, and a part of the next image 922B (near the top edge) is displayed below it. In this state, the system controller 222 stops the movement of the images for a predetermined time, and then slides images 922A and 922B to the upper part of the screen 922. Part (b) of Figure 18 shows the state after the slide, with a part of the first image 922A (near the bottom edge) at the top edge of the screen 922, the second image 922B in the center, and a part of the third image 922C (near the top edge) at the bottom edge. The system controller 222 repeats this sliding until the last image of the composite image (for example, the 10th image) is displayed in the center. The system controller 222 may also slide the image downwards or diagonally across the screen 922.

[0129] [Specific Example of Animation Effect (Part 3)] Figure 19 shows an animation effect corresponding to the icon 97E described above. In specific example (part 3), as shown in part (a) of Figure 19, the first image 923A of the multiple images used in the composite image is displayed in the center of the screen 923. In this state, the system controller 222 stops the movement of the image for a predetermined time, and then, as shown in part (b) of Figure 19, the second image 923B is superimposed on the first image 923A with a slight offset. The offset is not limited to this example and may be performed in different directions or positions, but it is preferable to allow the user to understand how the multiple images gradually overlap. The system controller 222 continues this display until the last image of the composite image (for example, the 10th image) is displayed at the top (towards the front of the screen).

[0130] [Setting Titles, etc.] The system controller 222 may set and display titles, etc. on the composite image in response to user operations (for example, operations via the monitor 292 and / or the operation unit 226). Titles, etc. can be made up of combinations of letters, numbers, shapes, symbols, and colors, and can be displayed at any timing such as the beginning, middle, or end of the composite image. For example, the system controller 222 may, in response to user operations, display "I went to a cat cafe on May 1, 2025!" at the beginning of the composite image and "All the cats were so cute! I want to go again!" at the end. The system controller 222 may superimpose titles, etc. on the image or display them separately from the image.

[0131] [Settings Complete and Preview of Composite Image] In the camera-equipped printer 200 according to the first embodiment, the data necessary for generating a composite image (source data) can be prepared by the operations and processes described above. In the first embodiment, the "composite image" is, for example, a moving image including an audio codec and a video codec in a container format, and may include the animation effects described above. Specifically, the "data necessary for generating a composite image" may include data for images to be switched and displayed, audio data attached to the image (ambient sound; first audio data) and / or BGM data (second audio data), background image data, and various setting conditions. It may also include data such as the title described above (for example, in text format). As shown in the camera-equipped printer 200 portion of Figure 10, this source data may be in format such as jpeg or wav, but this is just one example of a data format, and other data formats may be used. The system controller 222 can also convert the format of the image data and audio data using known methods as needed.

[0132] Once the data necessary for generating the composite image is available, the system controller 222 can display a preview of the composite image, either in response to user input or automatically, regardless of user input (step S12 in Figure 10). For example, the system controller 222 can display the images frame by frame in sync with the audio data (only switching between images without animation effects such as slides), and by pre-combining ambient sounds and background music for playback, it can play the composite image like a video without using a video container.

[0133] [Printing of Representative Image, etc.] Once the data necessary for generating the composite image (the material data described above) is available, the system controller 222 (print control unit 200F, etc.) can cause the printer 230 (printing device) to print a representative image (fourth image) of the composite image in response to a user operation (fourth operation) that instructs printing (step S13 in Figure 10). In the first embodiment, the printer 230 prints a representative image (cover image) on the instant film 310 shown in Figure 6. The "representative image" is, for example, one of the images used in the composite image (the first image, the last image, etc.), which may be determined according to the user's operation or may be determined automatically by the system controller 222. The representative image is at least an image based on the first image, and the first image may have undergone image processing, or it may be an image with the title, etc. described above superimposed on it.

[0134] The system controller 222 can print a representative image with a barcode or two-dimensional code superimposed on it. This barcode or two-dimensional code may include the URL to which the composite image is uploaded (an example of "information about the composite image") or information corresponding to that URL.

[0135] The user can distribute a printed copy of the representative image (print 99 in Figure 1) with a barcode or other superimposed information to others. Those who receive this print copy can then read the barcode or barcode using an information and communication terminal (smartphone, tablet, etc.) to display the composite image and / or download the composite image (video). The system controller 222 may print any number of representative images according to the user's operation, or it may set a limit on the number of prints depending on conditions such as user registration, whether or not charges are made, and the amount of charges.

[0136] [Data transmission to information and communication terminal] The system controller 222 (communication control unit 200D, etc.) transmits the data necessary for generating the composite image (source data) to the information and communication terminal 100 (step S14 in Figure 10). At this time, the system controller 222 may convert the file format as needed (e.g., converting from wav format to mp3 format).

[0137] [Generating and Uploading Composite Images at the Information and Communication Terminal] The information and communication terminal 100 (main control unit 101, image processing unit 100C, etc.), which receives material data from the camera-equipped printer 200, generates a composite image as a video container using the received material data (step S15 in Figure 10). In Figure 10, this composite image is labeled as "photolog". This composite image may be in MP4 format, for example, but may be in other formats as well.

[0138] [Uploading and Recording of Composite Images] The information communication terminal 100 uploads the generated composite image (composite image as a video container) to the server device 500 (step S16 in Figure 10). The server device 500 records the received composite image in the database 520 (recording device; a device that operates independently of the camera-equipped printer 200). The server device 500 may set restrictions on the data storage period and capacity depending on conditions such as user registration, whether or not billing is performed, and the amount of billing. The recording destination is the area corresponding to the URL mentioned above (labeled "URL information" in Figure 10). The information communication terminal 100 may also upload the image when it receives information indicating that "the camera-equipped printer 200 has printed the representative image (fourth image) mentioned above to the printer 230 (printing device)." In this case, the camera-equipped printer 200 (system controller 222; processor) shall notify the information communication terminal that the representative image has been printed. The camera-equipped printer 200 and the information and communication terminal 100 may transmit the composite image along with user identification information such as an ID to the server device 500, and record the composite image in association with that identification information.

[0139] In the first embodiment, the case is described in which the camera-equipped printer 200 transmits the data (material data) necessary for generating a composite image to the information and communication terminal 100, and the information and communication terminal 100 generates the composite image as a video container. However, if the camera-equipped printer 200 has the processing power (processor speed, RAM capacity 240, etc.) necessary for generating a composite image, the camera-equipped printer 200 may generate the composite image as a video container and upload it to the server device 500 (see also the second embodiment described later). In the first embodiment, only the person who received the print 99 may be able to view or download the composite image, or the image processing system 1 may upload the composite image to the user's blog or various video posting sites so that other users (users who did not receive the print 99) can view the composite image. In this case, it is preferable that the image processing system 1 protects privacy and prevents the unintended publication of composite images by setting publication conditions such as "to what extent (service name, site name, publication period, attributes of users who can view, etc.)" based on the user's instructions, "whether to allow viewing only or downloading of the composite image", and "the degree of publication (for example, whether to add audio data or animation effects to the published composite image)" (the same applies to the second embodiment described later). The image processing system 1 may also set printing of a representative image as a condition for the publication of such composite images.

[0140] [Downloading and Viewing Composite Images] When a user receives a printout of the representative image described above, they can scan the barcode or QR code with the information and communication terminal 800 to view and download the composite image as a video container (step S17 in Figure 10). Alternatively, a user of the image processing system 1 may perform the scan with the information and communication terminal 100. The user of the information and communication terminal 800 may be the same as or different from the user of the image processing system 1. Through such scanning and downloading, the person(s) who received the printout (e.g., printout 99 in Figure 1) can enjoy the composite image. The server device 500 may set restrictions on the number of views and downloads, and the duration, depending on conditions such as user registration, whether or not charges are made, and the amount of charges.

[0141] [Second Embodiment] Figure 20 is a diagram showing the configuration of system 10A according to the second embodiment. As shown in Figure 20, system 10A comprises an image processing system 2, a server device 500, and an information and communication terminal 800. The image processing system 2 comprises a composite image generation system 400 and a printer 700 (printing device). The configurations of the server device 500 and the information and communication terminal 800 are the same as in the first embodiment, so a detailed explanation is omitted.

[0142] The composite image generation system 400 (composite image generation device, composite image generation system) is composed of terminal devices such as smartphones, mobile phones, tablet terminals, and personal computers, similar to the information and communication terminal 100 in the first embodiment, and can adopt the same configuration as the information and communication terminal 100 (see Figures 7 to 8). The composite image generation system 400 has a control unit (processor; not shown) similar to the main control unit 101 of the information and communication terminal 100 and a storage unit similar to the storage unit 150. Furthermore, in the composite image generation system 400, if the display panel (display device) is a touch panel like the display panel 121 of the information and communication terminal 100, the user can perform operations that are performed by buttons on the camera-equipped printer 200 (such as scrolling and selecting images; including the first to fourth operations) via the touch panel.

[0143] The composite image generation system 400 may employ the same functions as the information and communication terminal 100 according to the first embodiment (see Figure 9), but it differs from the information and communication terminal 100 in that it performs both the generation of data for composite images (source data) and the generation of composite images as video containers.

[0144] [Printer Configuration] Figure 21 is a front perspective view of the printer 700, and Figure 22 is a rear perspective view. This printer 700 is configured as a portable mobile printer. The printer 700 is also configured as an instant printer that prints images onto instant film 310, for example, as shown in Figure 6. The instant film 310 is loaded into the printer 700 in the form of a film pack (not shown) containing multiple sheets in a case.

[0145] The printer body 724 has a rounded, flat rectangular box shape and is designed to be held in one hand and carried. The printer body 724 is also designed to be placed vertically (upright on a flat surface) and horizontally (lying down on a flat surface). Figures 21 and 22 show the printer 700 in a vertical position. Note that x, y, and z in Figures 21 and 22 represent a three-dimensional Cartesian coordinate system, where the x direction is the width direction, the y direction is the height direction, and the z direction is the depth direction of the printer body 724.

[0146] On the front of the printer 700, a push-button power button 726 is located almost in the center. The printer 700 is turned on and off by pressing and holding this power button 726 (for a certain period of time or longer). The power button 726 also functions as a light-emitting part, emitting light by illuminating a light source located inside it.

[0147] The printer 700 is equipped with an output port 728 at its top (the top when placed vertically), and the printed instant film 310 is ejected from this output port 728.

[0148] The printer 700 has a film pack lid 730 on its rear side for opening and closing the film pack loading compartment. It also has a lock release lever 732 for releasing the lock on the film pack lid 730. When the lock is released with the lock release lever 732 and the film pack lid 730 is opened, the film pack loading compartment is opened. After loading an instant film pack into this film pack loading compartment, when the film pack lid 730 is closed, the film pack lid 730 is locked by a locking mechanism (not shown), and the film pack is sealed in a light-shielding state.

[0149] One side of the printer 700 is provided with a USB cable connector cover 734 that opens and closes a USB (Universal Serial Bus) cable connector (not shown). The printer 700's built-in battery is charged via the USB cable connector, which is exposed when this USB cable connector cover 734 is opened.

[0150] The other configurations of the printer 700 are the same as those of the camera-equipped printer 200 in the first embodiment, but the printer 700 does not need to have a camera or display. In the second embodiment, the printer 700 does not generate data for composite images (source data), but prints a representative image in response to instructions from the composite image generation system 400.

[0151] [Processing in the System] Figure 23 shows the processing in system 10A according to the second embodiment. In system 10A, the composite image generation system 400 generates data (source data) to be used for the composite image in the same manner as in steps S11 and S12 of the first embodiment, and generates a composite image as a video container based on this data (step S21).

[0152] When a composite image as a video container is generated, the composite image generation system 400 sends a representative image of the composite image (cover image; fourth image) to the printer 700 (step S22), superimposes information such as a barcode or two-dimensional code (a URL indicating the upload destination of the composite image and information corresponding to that URL; information related to the composite image) onto it, and has the printer 700 print it (step S23). Note that the information such as the URL can be determined by the printer 700 and the result can be sent to the composite image generation system 400 (step S24). The composite image generation system 400 uploads the composite image as a video container to the URL received from the printer 700 and has it recorded in the server device 500 (database 520) (step S25). As described above for the first embodiment, the upload may be performed on the condition that the representative image has been printed by the printer 700.

[0153] Furthermore, similar to the first embodiment described above, when a user receives a printout of a representative image (in Figure 20, a printout 99 with a two-dimensional code 99A superimposed), they can scan the two-dimensional code with the composite image generation system 400 to view and download the composite image as a video container (step S26). The user of the information communication terminal 800 may be the same as or different from the user of the composite image generation system 400. Through such scanning and downloading, the person(s) who received the printout can enjoy the composite image. The server device 500 may set restrictions on the number of views and downloads, and the duration, depending on conditions such as user registration, whether or not charges are made, and the amount of charges.

[0154] Although embodiments and variations of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are possible.

[0155] 1 Image Processing System 2 Image Processing System 10 System 10A System 11 Wireless Communication Unit 42 Camera Unit 43 Strobe 90 Screen 90A Image 90B Icon 90C Menu 91 Screen 92 Screen 93 Screen 94 Screen 95 Screen 95A Representative Image 95B Area 95C Circle Mark 96 Screen 96A Area 96B Image 96C Border 96D Captured Image 97 Screen 97A Area 97B Icon 97C Icon 97D Border 97E Icon 99 Print 99A QR Code 100 Information Communication Terminal 100A Imaging Control Unit 100B Display Control Unit 100C Image Processing Unit 100D Communication Control Unit 100E Recording Control Unit 101 Main Control Unit 102 Housing 111 Wireless communication unit 120 Display input unit 121 Display panel 122 Operation panel 130 Communication unit 131 Speaker 132 Microphone 140 Operation unit 141 Camera unit 142 Camera unit 143 Strobe 150 Memory unit 151 Internal memory unit 152 External memory unit 160 External input / output unit 170 GPS receiver unit 180 Motion sensor unit 190 Power supply unit 200 Printer 200A Imaging control unit 200B Display control unit 200C Image processing unit 200D Communication control unit 200E Recording control unit 200F Print control unit 201 Main unit 202 Camera 203 Flash device 204 Lens 205 Output port 206 Image sensor 207 Shutter button 208 Lock switch 210 Camera 212 Lens 214 Image sensor 216 Signal processing unit 218 Lens drive unit 220 Image sensor drive unit 222 System controller 224 Wireless transceiver 226 Control unit 226A Dial 226B Menu / OK button 226C Print button 226D Button 226E Playback button 226F Mode switching button 226G Button 226H Power button 230 Printer 232 Film transport mechanism 233 Print head 234 Print head drive unit238 Data memory 242 Input / output interface 244 Microphone 245 Card slot 246 Speaker 252 GPS receiver 260 Bus 292 Monitor 293 Film door 294 Rear control panel 310 Instant film 310a Exposure surface 310b Observation surface 312 Exposure section 314 Pod section 314a Developer solution pod 315 Margin area 316 Trap section 316a Absorbent material 318 Observation section 318a Frame 400 Composite image generation system 500 Server device 510 Server main unit 520 Database 700 Printer 724 Printer main unit 726 Power button 728 Output port 730 Film pack lid 732 Lock release lever 734 USB cable connection cover 800 Information and communication terminal 901 First area 901A Image 902 Second area 902A Image 903 Third area 903A Image 903B Symbol 904 Fourth area 904A Area 904B Area 911 Image 911A Second image 911B Icon 911C Border 911E Third image 912 Image 912A Image 912B Icon 912C Border 912E Third image 913 Image 921 Screen 921A Image 921B Image 921C Image 922 Screen 922A Image 922B Image 922C Image 923 Screen 923A Image 923B Image

Claims

1. A composite image generation apparatus comprising a processor and a display device, wherein the processor displays a plurality of first images in a time series in a first area of ​​the display device, receives a first operation on the plurality of first images, displays a second image selected by the first operation in a second area of ​​the display device which is different from the first area, with improved visibility than the first image, receives a second operation on the second image, displays a third image selected by the second operation in a third area of ​​the display device which is different from the first and second areas, and generates a composite image using the third image.

2. The composite image generation apparatus according to claim 1, wherein the processor accepts an operation to scroll the first image in the first region as the first operation, and the image displayed at a predetermined position in the first region by the scrolling is the second image.

3. The composite image generating apparatus according to claim 1 or 2, wherein the processor enhances the visibility of the second image by making the size of the second image larger than that of the first image, adding one or more characters, symbols, numbers, and figures to the second image, and changing one or more of the color, brightness, and display pattern of the image from that of the first image.

4. The composite image generating apparatus according to claim 2, wherein the processor applies highlighting to the first image displayed at the predetermined position by one or more of the following: adding one or more characters, symbols, numbers, or figures to the first image displayed at the predetermined position; or changing one or more of the color, brightness, or display pattern of the first image displayed at the predetermined position from that of the other first images among the plurality of first images.

5. The composite image generation apparatus according to claim 1 or 2, wherein the processor adds numbers to the images to be displayed in the first area in the order in which the second operation was performed and displays them.

6. The composite image generation apparatus according to claim 1 or 2, wherein the processor adds and displays information indicating that the first image has first audio data if the first image has first audio data.

7. The composite image generating apparatus according to claim 1 or 2, wherein the processor adds at least a portion of the first audio data to the composite image if the first image has first audio data.

8. The composite image generating apparatus according to claim 7, wherein the processor adds at least a portion of the first audio data to the composite image if the third image has the first audio data.

9. The composite image generation apparatus according to claim 1 or 2, further comprising a recording medium for recording audio data, wherein the processor receives a third operation for selecting second audio data which is audio data recorded on the recording medium, and adds at least a portion of the second audio data selected by the third operation to the composite image.

10. The composite image generation apparatus according to claim 1 or 2, wherein the processor sets the relationship between the volume of the first audio data contained in the first image and the volume of the second audio data recorded on the recording medium.

11. The composite image generation apparatus according to claim 1 or 2, wherein the processor causes a printing device to print a fourth image representing the composite image in response to a fourth operation instructing printing.

12. The composite image generation apparatus according to claim 11, wherein the fourth image is at least an image based on the first image.

13. The composite image generation apparatus according to claim 11, wherein the processor causes the fourth image to be printed by the printing device, and the composite image is recorded by the recording device.

14. The composite image generation apparatus according to claim 13, wherein the processor causes a device that operates independently of the composite image generation apparatus to record the composite image as the recording device.

15. The composite image generation apparatus according to claim 11, wherein the processor superimposes information relating to the composite image onto the fourth image and causes the printing apparatus to print it.

16. The composite image generation apparatus according to claim 1 or 2, wherein the processor performs a first evaluation, which is an evaluation of the image displayed in the first area, and outputs first information relating to the result of the first evaluation.

17. The composite image generating apparatus according to claim 16, wherein the first information is information recommending that the composite image include an image in which the result of the first evaluation satisfies the first criterion.

18. The composite image generation apparatus according to claim 1 or 2, wherein the processor performs a second evaluation, which is an evaluation of the image on which the second operation has been performed, and outputs second information relating to the result of the second evaluation.

19. The composite image generating apparatus according to claim 18, wherein the second information is information recommending that the result of the second evaluation satisfy the second criterion for printing.

20. The composite image generation apparatus according to claim 1 or 2, wherein the processor displays the images in the third region in the order in which the second operations were performed.

21. The composite image generation apparatus according to claim 1 or 2, wherein the processor displays images in the third region in the order of the time-series information contained in the images.

22. The composite image generation apparatus according to claim 1 or 2, wherein the processor generates the composite image by adding a background to the image displayed in the third area.

23. The composite image generation apparatus according to claim 22, wherein the processor generates the composite image using an image selected from the first image as the background.

24. The composite image generation apparatus according to claim 1 or 2, wherein the processor generates an image as the composite image, which is a switching display of the image shown in the third region.

25. A composite image generation system comprising a composite image generation device according to claim 1 or 2 and an imaging device, wherein the processor causes the image captured by the imaging device to be displayed as the first image in the first region.

26. The composite image generation system according to claim 25, further comprising an operating member that receives at least the first operation and the second operation.

27. A composite image generation method performed by a composite image generation apparatus comprising a processor and a display device, wherein the processor displays a plurality of first images in a time series in a first area of ​​the display device, receives a first operation on the plurality of first images, displays a second image selected by the first operation in a second area of ​​the display device which is different from the first area, with improved visibility than the first image, receives a second operation on the second image, displays a third image selected by the second operation in a third area of ​​the display device which is different from the first and second areas, and generates a composite image using the third image.

28. A program for operating a composite image generation system comprising a processor, an imaging device, and a display device, the program comprising: causing the processor to display a plurality of first images in a time series in a first area of ​​the display device; receiving a first operation on the plurality of first images; displaying a second image selected by the first operation in a second area of ​​the display device that is different from the first area, with improved visibility than the first image; receiving a second operation on the second image; displaying a third image selected by the second operation in a third area of ​​the display device that is different from the first and second areas; and generating a composite image using the third image.

29. A non-temporary and computer-readable recording medium on which the program described in claim 28 is recorded.

Images