Information equipment and camera image sharing system

Abstract

An information device equipped with a camera and a display connects to a camera image sharing service provider via a communication unit, uploads images captured by the camera and their feature quantities, and downloads images captured by other information devices and their feature quantities. Furthermore, the device connects to an automatic composition service provider and downloads multiple composition patterns for synthesizing multiple captured images. The composition unit of the information device calculates the feature quantities of the camera's image, selects one of the downloaded composition patterns based on the calculated feature quantities and the feature quantities of the downloaded images captured by the other information devices, and synthesizes the images captured by the camera and the other information devices. The synthesized images captured by the camera and the other information devices are displayed on the display.

Description

Technical Field

[0001] The present invention relates to an information device equipped with a camera and a display, and a camera image sharing system for sharing images captured by a plurality of cameras. Background Art

[0002] Among information devices composed of a CPU (Central Processing Unit) and memory, those integrating wireless networking, camera, and display functions are widely used. Specifically, these devices allow users to take photos anytime, anywhere using smartphones and tablet PCs (Personal Computers), and share these images online on SNS (Social Network Services). Furthermore, some information devices include internal cameras for selfies in addition to external cameras.

[0003] As a method for synthesizing images taken with multiple cameras, Patent Document 1 discloses the following method: multiple camera images are processed in layers, the user cuts out a portion of the image of the first layer, displays the image of the second layer in the cut area, and synthesizes and displays the multiple images.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2014-057258 Summary of the Invention

[0007] In Patent Document 1, the user needs to spend time and effort to cut out a portion of the first layer image and display the second layer image in the cutout area to obtain a composite image. This operation becomes troublesome for the user, especially when there are many camera images.

[0008] An object of the present invention is to provide an information device and a camera image sharing system that do not require special user operations for combining images captured by multiple cameras and that facilitate selection of images to be combined.

[0009] In the present invention, an information device having a camera and a display is connected to a camera image sharing service provider via a communication unit, uploading images captured by the camera and their feature quantities, and downloading images captured by other information devices and their feature quantities. Furthermore, the device is connected to an automatic composition service provider to download multiple composition patterns for synthesizing multiple captured images. A composition unit in the information device calculates the feature quantities of the camera's image, selects one of the downloaded composition patterns based on the calculated feature quantities and the feature quantities of the downloaded images captured by the other information devices, and synthesizes the images captured by the camera and the other information devices. The synthesized images captured by the camera and the other information devices are displayed on the display.

[0010] According to the present invention, it is possible to provide an information device and a camera image sharing system that do not require special user operations for combining images captured by a plurality of cameras and that facilitate selection of images to be combined. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a diagram showing the appearance of the information device according to the first embodiment.

[0012] Figure 2 It is a block diagram showing the structure of an information device.

[0013] Figure 3 This is a diagram showing the configuration of a camera image sharing system composed of information equipment and a service providing apparatus.

[0014] Figure 4A This is a diagram showing the operational flow of image capture processing in an information device.

[0015] Figure 4B This is a diagram showing the operational flow of playback processing in an information device.

[0016] Figure 4C This is a diagram showing the operational flow of SNS processing by an information device.

[0017] Figure 5 This is a diagram showing the operational flow of automatic composition processing in an information device.

[0018] Figure 6 : is a diagram showing an example of feature quantities of a camera image used in the automatic composition process.

[0019] Figure 7A It is a diagram showing an example of a display screen of an information device (at the time of shooting).

[0020] Figure 7B It is a diagram showing an example of a display screen of an information device (automatic composition processing).

[0021] Figure 7CThis is a diagram showing an example of a display screen of an information device (during playback).

[0022] Figure 7D It is a diagram showing an example of a display screen of an information device (composition mode change).

[0023] Figure 7E It is a diagram showing an example of a display screen of an information device (composition mode change).

[0024] Figure 7F It is a diagram showing an example of a display screen of an information device (a different composite screen).

[0025] Figure 8 This is a diagram showing an example of a user management database according to the second embodiment.

[0026] Figure 9 This is a diagram showing the operation flow of the automatic composition service providing device.

[0027] Figure 10 : is a diagram showing examples of evaluation index values of composition patterns.

[0028] Figure 11 This is a diagram showing the flow of the imaging operation of the information device according to the third embodiment.

[0029] Figure 12A 1 and 2 are diagrams showing examples of display screens of information equipment (when photographing in a landscape orientation).

[0030] Figure 12B 1 and 2 are diagrams showing examples of display screens of information equipment (when photographing in a vertical orientation).

[0031] Figure 13A This is a diagram showing an example of a display screen of an information device (a method of selecting an image).

[0032] Figure 13B This is a diagram showing an example of a display screen of an information device (a method of selecting an image).

[0033] Figure 13C This is a diagram showing an example of a display screen of an information device (a method of selecting an image).

[0034] Figure 14A This is a diagram showing an example of a display screen of an information device (during playback).

[0035] Figure 14B This is a diagram showing an example of a display screen of an information device (during playback).

[0036] (Explanation of Symbols)

[0037] 1: Information device (camera); 2: OUT camera; 3: IN camera; 4: Display; 9: Internet; 10: Automatic composition service provider; 11: Composition processing unit; 12: Composition table; 13: Camera image sharing service provider; 14: User management DB; 15: Image storage device; 16: SNS service provider; 101a: Mobile communication IF; 101b: Wireless LAN IF; 102: CPU; 103: RAM; 104: FROM; 104a: Camera sharing application; 104b: Image communication process; 104c: Automatic composition process; 104d: SNS application; 105b: SDC; 108: Position sensor; 109: Image processor; 110: Graphics processor. DETAILED DESCRIPTION

[0038] The present invention relates to an image sharing system including an information device having a camera function and a camera image sharing service providing device, and each device will be described separately.

[0039] Example 1

[0040] In the first embodiment, an information device including a camera and a display is described.

[0041] Figure 1 The figures show the appearance of the information device 1 involved in Example 1, (a) is a front surface diagram, and (b) is a back surface diagram. The information device 1 is an information device that integrates a camera function and a display function, for example, equivalent to a smartphone, a tablet PC, etc. In its structure, 1a is a return button, 1b is a home button, 1c is a multitasking button, 2 is an external (OUT) camera, 2a is an autofocus (AF) sensor, 2b / 2c is an RGB sensor / LED flash, 3 is an internal (IN) camera, 4 is a touch sensor integrated display, 4a is a light sensor, and 5 is an earpiece. The external (OUT) camera 2 on the back surface side can shoot external subjects, and the internal (IN) camera 3 on the front surface side (display 4 side) can take selfies. Hereinafter, the information device 1 is also simply referred to as a "camera."

[0042] Figure 2 OUT camera 2 includes an image sensor 201 and a camera processor 202. When shooting with OUT camera 2, AF sensor 2a supports the auto focus function, RGB sensor 2b supports the auto white balance function, and LED flash 2c supports the flash function in dark shooting environments. RGB sensor 2b and LED flash 2c can operate exclusively, such as Figure 1As shown in (b), the RGB sensor 2b and the LED flash 2c share a window on the back surface of the OUT camera 2. In addition, the IN camera 3 includes an image sensor 301 and a camera processor 302. Figure 1 As shown in (a), it is arranged on the front surface and is used mainly for the self-photographing of the user of the information device 1.

[0043] The light sensor 4a detects the ambient brightness and controls the backlight of the display 4, setting the brightness of the image to be displayed to be easy for the user to see. The position sensor 108 obtains the position information of the information device 1 through the GPS (Global Positioning System).

[0044] The image processor 109 processes image data downloaded from the network and image data read from the SDC (SD card) 105b, and then sends the processed image data to the graphics processor 110. The graphics processor 110 converts the image data and the HTML (Hyper Text Markup Language) data processed by the CPU 102 into graphic data for display on the display 4.

[0045] The mobile communication interface (IF) 101a supports communication methods such as 3G and LTE (Long Term Evolution), and together with the wireless LAN interface (IF) 101b provides a wireless network function suitable for downloading large amounts of data such as image data. The wireless network may also be other methods.

[0046] CPU 102, RAM (Random Access Memory) 103, and FROM (Flash ROM) 104 perform the core computing and program execution functions of information device 1. FROM 104 stores programs for image and audio processing, HTML browsers, and other such functions. These programs are loaded into RAM 103 as needed and executed by CPU 102. RAM 103 also temporarily stores image and HTML data.

[0047] In particular, FROM 104 stores programs for the camera sharing application 104a and SNS application 104d, which are features of this embodiment. These programs are also loaded into RAM 103 and executed by CPU 102. Camera sharing application 104a includes an image communication process 104b for transmitting and receiving image data to and from the Internet 9 via mobile communication IF 101a and wireless LAN IF 101b, and an automatic composition process 104c for defining a structure for synthesizing and displaying multiple camera images on display 4.

[0048] The SDIF 105a can store various data in the SDC (SD card) 105b. For example, by storing image data captured by the OUT camera 2 and IN camera 3 in the SDC 105b in addition to the FROM 104, large-capacity storage can be achieved. The USBIF 106 is an interface for connecting USB devices or receiving external power supply.

[0049] The microphone 107a and the earpiece speaker 5 are used for voice calls, and the speaker 107b is used for listening to music data.

[0050] Figure 3 This diagram shows the structure of a camera image sharing system consisting of multiple information devices 1 and a service provider. Assume that multiple users 6 (A through F) possess information devices (cameras) 1 in a specific area 7 (e.g., an event venue). Each user A through F is presumed to be registered with the camera image sharing service provider 13, but the registration method is not limited. Each camera 1 uploads the captured image and its feature values to the camera image sharing service provider 13 via the Internet 9 each time a user 6 presses the shutter button to capture an image. The following description will focus on user A's capturing action and explain the use of the camera image sharing service provider 13.

[0051] Each camera 1 is connected to the Internet 9 via a mobile communication base station or a wireless LAN access point 8. Furthermore, the Internet 9 is connected to an automatic composition service provider 10, a camera image sharing service provider 13, and an SNS service provider 16 (hereinafter, each "service provider" is referred to as a "service").

[0052] The camera image sharing service 13 has a user management database (DB) 14 and an image storage device 15, which stores camera-captured image data and feature quantities of the captured image data. The automatic composition service 10 has a composition processing unit 11 and a composition table 12 that describes a composition pattern generated by the composition processing unit 11. Here, a composition pattern refers to content that describes the display rules such as the arrangement and size of each image when synthesizing multiple camera images. The automatic composition service 10 is set as a service that is linked to the camera image sharing service 13. In this example, the system supports the following content: providing the camera image sharing service 13 and the automatic composition service 10 to users registered in the user management DB 14 of the camera image sharing service 13, sharing camera images captured by multiple users, and appropriately synthesizing these multiple images (automatic composition processing).

[0053] Next, divide them into Figures 4A to 4C, explaining the operation flow of shooting / playback / SNS of the information device (camera) 1.

[0054] Figure 4A 1 is a diagram showing the operational flow (S100) of the photographing process of the information device 1. In this process, an image photographed by the own camera and an image synthesized therewith by another camera are selected and saved by the automatic composition process.

[0055] The user starts the OUT camera 2 or IN camera 3 of the information device (camera) 1 (S101). In this embodiment, the OUT camera 2 is used for the purpose of sharing the camera images of multiple cameras 1, but the IN camera 3 can also be used. When the camera is started, various camera settings (auto focus, automatic white balance, automatic exposure, automatic flash, default settings for the number of camera pixels, etc.) are automatically set (S102). Of course, some settings can also be set manually. When the camera settings are completed, the camera starts shooting and displays the captured image on the display 4 (S103). However, at this point in time, the user has not pressed the shutter button.

[0056] Next, the user presses the camera image sharing service button ( Figure 7A The camera sharing application 104a is started and the camera image sharing service 13 is logged in (S104). At this time, the automatic composition service 10 is also started, a part of the composition table 12 is downloaded, and the composition table 12a stored in the camera 1 is updated (S105).

[0057] Next, image data from other cameras is downloaded from the camera image sharing service 13 (S106). The image data downloaded from other cameras is limited to cameras suitable for sharing. Specifically, the location information acquired by the location sensor 108 is used to limit the download to cameras located near the current camera. For example, cameras located in the vicinity of the current camera are selected in eight directions: front, back, left, right, and diagonal. Furthermore, the capture time is limited to the most recent image data captured within a predetermined time from the current time. This image data from other cameras is downloaded along with the image's feature values.

[0058] In addition, regarding the image captured by this camera, feature quantities such as whether there are people in the image, the number of people in the image, the size of people or groups of objects, and the positions within the image are calculated (S107).

[0059] Next, the automatic composition process 104c is used to perform automatic composition processing (S108) using the feature values of the image of this camera and the images of other cameras. In this process, the composition pattern of display rules such as which camera's image is displayed at which position and in what size on the display 4, and which image is displayed in front when multiple images overlap is automatically determined. The graphics processor 110 creates a display screen that is a composite of the images of this camera and other cameras according to the determined composition pattern, and displays it on the display 4 (S109). Figure 5 、 Figure 6 , the automatic composition processing (S108) will be described in detail.

[0060] The user observes the composite screen of the plurality of camera images displayed on the display 4 and selects another camera image to be composited with the camera image (S110). After selecting another camera image, when the user presses the shutter button ( Figure 7A 、 7B After the camera is captured (see symbol 413), the camera performs the capture process, and the image captured by this camera is saved to a storage medium such as SDC105b (S111). At this time, the image selected by the other camera is already downloaded and is not re-saved. Instead, it is associated with the image data of this camera. Furthermore, when the shutter is pressed, the image data captured by this camera, its features, and information about the composition mode selected in S110 are uploaded to the camera image sharing service 13 (S112). This uploaded data can also be used by users of other cameras when using the camera image sharing service 13.

[0061] The user selects whether to continue shooting (S113). If so, the process returns to S106 and the process repeats itself, starting with downloading images from other cameras. In this way, images from other cameras to be downloaded are updated as needed based on the movement of camera 1 and the passage of time. If the downloaded data is valid, there is no need to download it again.

[0062] Figure 4B 1 is a diagram showing the operation flow (S120) of the reproduction process of the information device 1. In this process, the image taken by the camera is synthesized with the image of other cameras to be reproduced and displayed. When the user presses the reproduction button ( Figure 7B When the camera 104 is displayed (S121), the image captured and stored by the camera itself and the images associated with the other cameras are read from the storage medium (S122). These read images are automatically composed using the automatic composition process 104c (S123), and the resulting composite image is reproduced and displayed on the display 4 (S124). In addition, information on the composition mode selected during composition is uploaded (S125).

[0063] In addition, when the composition mode is changed according to the user's instructions during playback, for example, when images other than the camera image are displayed preferentially at the front of the screen (S126), the process returns to the automatic composition processing of S123, and in S125, the information of the changed composition mode is uploaded again.

[0064] Figure 4C 1 is a diagram showing the operation flow (S130) of the SNS process of the information device 1. Here, the captured image and annotation are uploaded using the SNS (Social Network Service) service 16.

[0065] The user presses the SNS button ( Figure 7C 401) to start the SNS application 104d (S131), log in to the SNS service 16 (S132). Then, upload the camera image, annotations, etc. (S133). In addition, in the reproduction process ( Figure 4B ) is transferred to SNS processing, the reproduced screen can be easily uploaded to SNS. That is, the screen after the automatic composition processing (S123) and the reproduction display of the synthesized screen (S124) can be uploaded as it is (described later). Figure 7C 、 Figure 7D ). When continuing SNS (S134), return to S133 and repeatedly upload images, comments, etc.

[0066] Figure 5 is a diagram showing the operation flow of the automatic composition process (S140) of the information device 1, and Figure 4A However, here, the basic description of the automatic composition processing is given without being limited to the processing in S108.

[0067] The automatic composition process begins upon logging into the camera image sharing service 13 (S140). This camera continuously captures images, refreshing them at a rate of, for example, several to several dozen frames per second. A single frame is captured from the continuously refreshed images (S141), and feature values are calculated for the captured image (S142). Furthermore, for other cameras, the feature values downloaded from the camera image sharing service 13 in S106 are used (S143).

[0068] Figure 6This figure shows examples of feature quantities for camera images used in automatic composition processing. In the case of the self-camera image, these include the number of people photographed, the size and position of the group, the size of faces, whether the user is photographing themselves, the focal length, and the position and size of the focused object. On the other hand, in addition to these, the position of each camera (direction and distance from the self-camera) and the time of capture are added as feature quantities.

[0069] Next, based on the feature values of the camera and other cameras, a composition pattern is selected from the composition table 12a stored in the camera (S144). This method uses a search index vector based on the feature values to automatically select a composition pattern from the composition table 12a that has an index vector that is close in weighted distance to the index vector. Furthermore, when the camera image sharing service 13 is started, the composition table 12a is updated in S105. The graphics processor 110 uses the selected composition pattern to create a display screen that combines images from multiple cameras, including the camera (S145). The created composite screen is displayed on the display 4.

[0070] Figures 7A to 7F 1 is an example of a display screen of an information device (camera) 1. Here, a display screen of the camera 1 held by user A is shown.

[0071] Figure 7A Show Figure 4A The display screen in the captured image display state (S103) is displayed on the display 4 of the camera 1. A captured image 41 of user A's camera, a user logo 411 indicating that user A is user A, a camera image sharing service button 412, a shutter button 413, and a setting button 414 are displayed. User A starts logging into the camera image sharing service 13 by pressing the camera image sharing service button 412.

[0072] Figure 7B Shown by Figure 4A The display state of the composite screen synthesized by the automatic composition processing (S108) is shown (S109). In addition to the captured image 41 of the camera of user A, the captured image 42 of the camera of user B, the captured image 43 of the camera of user C, the captured image 44 of the camera of user D, the captured image 45 of the camera of user E, and the captured image 46 of the camera of user F are displayed. The captured images of these cameras are images downloaded from the camera image sharing service 13 ( Figure 4AIn S106 (see step 106), image 41 captured by camera A is the current image, while the other captured images are images captured at a slightly earlier time and uploaded to the camera image sharing service 13. User icons 421, 431, 441, 451, and 461 are displayed in each captured image. Furthermore, selection buttons 422, 432, 442, 452, and 462 are displayed in each captured image for selecting whether to save the image in association with user A's captured image 41 (step 110). This combined display allows user A's captured image 41 and images 42-46 of other users B-F, taken during the same activity, to be viewed together, easily conveying a sense of presence during the activity.

[0073] Then, in Figure 7B In the example, the captured image 42 of user B and the captured image 43 of user C are selected (associated) for synthesis. When user A presses the shutter button 413, the captured image 41 of A and the feature value are saved to the storage medium, and the captured image 42 of B and the captured image 43 of C, as well as their respective feature values, are associated and saved (S111).

[0074] Figure 7C Shown by Figure 4B The display state of the composite screen synthesized by the automatic composition process (S123) during playback (S124) is shown. The automatic composition process (S123) is performed for user A's captured image 41 and its associated images 42, 43, and 43 of user C. User A's captured image 41 is displayed in front of the other images. Furthermore, pressing the SNS button 401 activates the SNS application 104d, enabling the upload of the image.

[0075] Figure 7D Shown in Figure 7C Here, user A touches the captured image 42 of user B to give priority to B's captured image 42, and the display screen is executed again after the automatic composition process. Generally, the user (user A) uses his / her OUT camera to capture the surroundings of himself / herself, rather than himself / herself. On the other hand, there is a possibility that the user (user A) may be captured using another person's camera. Figure 7D In the example, in the case where user A is photographed in B's photographed image 42, user A touches B's photographed image 42, and the photographed image 42 of B in which user A is photographed is enlarged and displayed in front.

[0076] Figure 7E Shown in Figure 7C Here, the captured image 41 of user A is touched and enlarged and displayed on the entire screen.

[0077] Figures 7C to 7E The displayed screen can be uploaded to the SNS service 16. In particular, Figure 7C 、 Figure 7D By uploading the image captured by A together with the images captured by B and C to the SNS, the scene of the event venue can be conveyed to other SNS users as a realistic image.

[0078] Figure 7F Shown with Figure 7B Examples of different composite images. In this example, when adjacent captured images overlap at their boundaries, the overlapping images are cropped obliquely to create the composite image. This prevents overlapping, unseen areas from being concentrated on a specific image, effectively displaying the boundaries between each image evenly.

[0079] As described above, according to Example 1, it is possible to provide an information device and an image sharing system that facilitate the selection of a composite camera image without requiring special user operations to obtain a display screen that synthesizes multiple camera images. Furthermore, during an event, for example, the status of multiple cameras can be checked at a glance, and the screen that synthesizes multiple camera images can be uploaded to a social networking service (SNS) to share the immersive image among users.

[0080] Example 2

[0081] In the second embodiment, the operations of the camera image sharing service providing device (camera image sharing service) 13 and the automatic composition service providing device (automatic composition service) 10 in the camera image sharing system will be described.

[0082] Figure 8 This figure shows an example of a user management database (DB) 14 of a camera image sharing service providing device 13. Users registered in the user management DB 14 form a user group named "Activity T" for the purpose of pre-sharing or exchanging camera images. In the user management DB 14, N users are registered, starting with "Sakura", and data such as the user's nickname, mobile phone number, IP address, password, and email address are recorded. The information device (camera) used by these users does not need to be a specific device; any information device with camera and display functions can be used. For example, it is possible to log in to the camera image sharing service 13 by entering an email address and password to receive the service.

[0083] Figure 9This diagram illustrates the operational flow of the composition processing unit 11 in the automatic composition service providing device 10. The composition processing (S200) herein involves collecting feature quantities of the user's camera images and determining index vectors based on the feature quantities of the grouped (associated) camera images. A composition pattern suitable for synthesizing and displaying the multiple camera images is then determined and registered in the composition table 12 in association with the index vectors.

[0084] When image data and feature values are uploaded and received from the camera 1 (S201), the updating of the composition table 12 is started (S202). First, the uploaded feature values are registered in the camera feature value database 11a (S203). Then, the position information of the uploaded camera is retrieved, for example, Figure 3 The camera of user A and the cameras around it (users B, C, D, E, and F) are detected and their feature quantities are read from the camera feature quantity database 11a (S204).

[0085] Next, the read feature values are used to create composition pattern candidates (S205). The so-called composition pattern is the display rule (disposition and size, etc.) when synthesizing each camera image. Figure 10 The composition pattern is generated based on the evaluation index of the composition pattern shown in FIG206. If the generated candidate meets the evaluation index, it is added to the composition table 12 (S207), and the process ends (S208). If the generated candidate does not meet the evaluation index, the process returns to S205, the composition pattern candidate is corrected, and the candidate is generated again, and the evaluation is repeated.

[0086] Figure 10 This figure shows an example of evaluation indicators for composition patterns. Examples of these conditions include minimizing blank areas on display 4 where no camera images are displayed, and ensuring that the central subject (e.g., a person) in each camera image is easily recognizable (to achieve this, the camera images are zoomed in / out and cropped). Furthermore, the system requires maintaining the actual relative position of the cameras using camera position information while displaying images corresponding to the distance. Specifically, camera images closer to camera A are displayed larger, while camera images farther from camera A are displayed smaller. By satisfying these requirements, a composite image can be created that recreates the immersive feeling of the event venue.

[0087] Furthermore, when creating composition pattern candidates (S205), past composition pattern selection results are fed back. Composition table 12 contains a large number of composition patterns with past selection results, accumulating results from cameras that have widely used the automatic composition service 10, rather than limiting the selection to a single set. By performing AI (artificial intelligence) learning on these numerous results, composition table 12 can be created that includes more suitable composition patterns.

[0088] According to the second embodiment, suitable composition patterns can be provided from the automatic composition service providing device 10 on the Internet 9, thereby reducing user operations for image synthesis in the information device 1. Furthermore, the information device 1 can be made smaller and lighter.

[0089] Example 3

[0090] In the third embodiment, the case where the OUT camera 2 and the IN camera 3 are operated simultaneously in the information device 1 to perform automatic composition processing of these images is described. The structure of the information device 1 (camera) is similar to that of the first embodiment ( Figure 2 ) is the same as the structure shown in FIG. 1 , and the automatic composition processing of the images of the OUT camera 2 and the IN camera 3 is performed through the automatic composition process 104c stored in FROM104.

[0091] Figure 11 1 is a diagram showing the flow of the photographing operation of the information device 1 in the third embodiment. Figure 4A ) of the action flow (S100) are assigned the same number to Figure 4A The explanation focuses on the different steps in the action flow.

[0092] When the camera is activated (S101), settings for OUT camera 2 and IN camera 3 are performed (S102a). In camera feature calculation (S107a), feature values for the camera images of OUT camera 2 and IN camera 3 are calculated. In the automatic composition process (S108a), the two camera images of OUT camera 2 and IN camera 3 are used as targets. Furthermore, if the information device 1 includes cameras other than OUT camera 2 and IN camera 3, all camera images can be used as targets. As a result of the automatic composition process, a screen is displayed that combines the camera images of OUT camera 2 and IN camera 3 (S109). The user selects an image to be captured from these images (S110a), captures the image, and saves it (presses the shutter button) (S111).

[0093] In this embodiment, the camera image sharing service 13 can also be used, with its path indicated by a dotted line. Log in at a desired time (S104), download the composition table 12 and other users' camera images (S106), and perform automatic composition processing (S108). This is captured and saved, and then the camera image data, feature values, and selected composition mode are uploaded (S112).

[0094] In addition, regarding the regeneration treatment and SNS treatment, the Figure 4B 、 Figure 4C ) has the same action flow.

[0095] 12 to 14 show examples of display screens of the information device (camera) 1 .

[0096] Figure 12A and Figure 12B Shows the display status of the composite image during shooting, and Figure 11 Corresponding to step S109 in Figure 12A In the application, the composition mode is applied (S108a) in which the camera is photographed in a horizontal screen and the image 48 of the OUT camera 2 and the image 47 of the IN camera 3 are displayed in a horizontal arrangement. In the selection of the photographed image (S110a), the selection button 482 is checked to select the image 48 of the OUT camera 2.

[0097] exist Figure 12B In this mode, the camera capture is performed in a vertical orientation, with image 48 of OUT camera 2 and image 47 of IN camera 3 displayed vertically side by side. When selecting the captured image, select buttons 482 and 472 are checked to select both image 48 of OUT camera 2 and image 47 of IN camera 3. Pressing shutter button 413 in this state (S111) captures and stores both images of OUT camera 2 and IN camera 3 simultaneously.

[0098] Figures 13A to 13C 4. This is a diagram showing a method for selecting a camera image to be synthesized. In this example, the camera image is selected not by the selection button but by the shutter button 413, which is moved (dragged) to the position of the camera image to be selected.

[0099] exist Figure 13A In FIG, the shutter button 413 is moved to the image 47 of the IN camera 3, and the outer frame of the image 47 is highlighted with a bold line to select it. Figure 13B In FIG. 4 , the shutter button 413 is moved to the image 48 of the OUT camera 2, and the image 48 is highlighted and selected. Figure 13CIn the example, the shutter button 413 is moved to the boundary line between the image 48 and the image 47, and the image 48 and the image 47 are highlighted, thereby selecting both the images of the OUT camera 2 and the IN camera 3.

[0100] In this example, a simple operation method is used to select an image to be captured and saved using shutter button 413. Furthermore, if the shutter button 413 can be moved (dragged) and the shutter button can be pressed when the finger is released from the shutter button 413, the user can perform operations such as selecting a camera image and pressing the shutter button in a single operation, further improving usability.

[0101] Figure 14A and Figure 14B Shows the state where the camera image captured and stored is reproduced and displayed. Figure 4B In the operation flow of the playback, after the automatic composition process (S123), the corresponding synthetic image is displayed (S124). Figure 14A 、 Figure 14B In the example, all objects near the center of the camera image are displayed in close-up. Figure 14A In the example, the composition mode of treating the image 48 of the OUT camera 2 and the image 47 of the IN camera 3 equally is applied. Figure 14B In FIG. 4 , a composition mode is applied in which the image 48 of the OUT camera 2 is mainly displayed.

[0102] As described above, according to Example 3, it is possible to provide an information device and a camera image sharing system that facilitates selection of a camera for shooting without requiring special user operations to obtain a display screen that combines images from the OUT and IN cameras. Furthermore, it is possible to provide a GUI (graphical user interface) that seamlessly combines camera image selection and shutter operation.

[0103] While various embodiments have been described above, the present invention is not limited to these. For example, a portion of the structure of one embodiment can be replaced with another embodiment. Furthermore, the structure of one embodiment can be combined with the structure of another embodiment. All of these fall within the scope of the present invention. Furthermore, the numerical values, messages, etc. that appear in the text and figures are merely examples, and using different examples will not impair the effects of the present invention.

[0104] Furthermore, the functions of the invention may be partially or entirely implemented in hardware, for example, by designing them using integrated circuits. Alternatively, they may be implemented in software by interpreting and executing an operating program that implements each function, such as a microprocessor unit or CPU. Furthermore, the scope of software implementation is not limited, and both hardware and software may be used.

Claims

1. An information device having a first camera and a display with a touch sensor, characterized in that: have: a communication unit that downloads, via a network, an image captured by a second camera of another information device different from the information device; and a composition unit for synthesizing the image captured by the first camera and the downloaded image captured by the second camera to obtain a composite picture in which the image captured by the first camera and the downloaded image captured by the second camera are arranged according to a first composition pattern; When the downloaded captured image of the second camera in the composite picture displayed on the display is an image associated with the captured image of the first camera, and the touch sensor receives a selection of a predetermined icon displayed simultaneously with the displayed composite picture, information related to the association between the captured image of the second camera and the captured image of the first camera is saved to a storage medium together with the captured image of the first camera, and an image obtained by arranging the captured image of the first camera and the downloaded captured image of the second camera in a second composition mode different from the first composition mode is output via the communication unit.

2. The information device according to claim 1, wherein The communication unit downloads the captured image of the second camera together with the feature amount of the captured image, The composition unit calculates a feature value of the image captured by the first camera, and determines a composition pattern for combining the images captured by the first camera and the second camera based on the calculated feature value and the feature value of the downloaded image captured by the second camera.

3. The information device according to claim 2, characterized in that The communication unit downloads a set of a plurality of composition patterns for synthesizing captured images, The composition unit selects one composition pattern from the downloaded set of composition patterns according to the feature amount.

4. The information device according to claim 1, wherein When the communication unit downloads the captured images of the second camera, the composition unit limits the captured images to be downloaded using the position information and the captured time information of the second camera.

5. The information device according to claim 1, wherein: When the first camera takes a picture, the picture taken by the first camera and the feature quantity and position information of the picture taken are uploaded to the network through the communication unit.

6. The information device according to claim 1, wherein: The communication unit downloads the captured image of the second camera together with the feature amount of the captured image, The composition unit calculates a feature quantity of the image captured by the first camera, and determines a composition pattern for combining the image captured by the first camera and the image captured by the second camera based on the calculated feature quantity and the feature quantity of the associated image captured by the second camera. According to the determined composition pattern, a screen in which the image captured by the first camera and the associated image captured by the second camera are combined is displayed on the display.

7. The information device according to claim 2, characterized in that The information of the composition mode determined by the composition unit is uploaded to the network through the communication unit.

8. The information device according to claim 6, characterized in that When a user touches any captured image on the composite screen displayed on the display, the composition unit changes the determination of the composition mode.

9. The information device according to claim 1, wherein: When a plurality of captured images displayed on the display have overlapping boundaries, the composition unit changes the display boundaries of the captured images so that the boundaries of the captured images are displayed evenly.

10. A camera image sharing system, comprising: connecting a plurality of information devices having cameras and displays with touch sensors to a camera image sharing service provider and an automatic composition service provider via a network; wherein: The camera image sharing service providing device comprises: A user management database for managing the plurality of information devices; and an image storage device for storing the captured images uploaded from the plurality of information devices and the feature quantities of the captured images; The automatic composition service providing device comprises: a composition processing unit that generates a composition pattern for synthesizing a plurality of captured images using feature quantities of the captured images uploaded from the plurality of information devices; and A composition table registers the generated composition pattern and the feature amount in correspondence with each other, The information equipment has: a communication unit that uploads, via the network, a captured image of the camera and a feature quantity of the captured image to the camera image sharing service providing apparatus, downloads captured images of other information devices and the feature quantity of the captured image from the camera image sharing service providing apparatus, and downloads a plurality of composition patterns for synthesizing a plurality of captured images from the automatic composition service providing apparatus; and a composition unit that calculates a feature quantity of the image captured by the camera, selects one composition pattern from the plurality of downloaded composition patterns based on the calculated feature quantity and the feature quantity of the image captured by the downloaded other information device, and synthesizes the image captured by the camera and the image captured by the other information device to obtain a composite picture in which the image captured by the camera and the image captured by the downloaded other information device are arranged according to the selected composition pattern; displaying a composite image of the image captured by the camera and the image captured by the other information device, which is composited by the composition unit, on the display; When the downloaded captured image of the other information device in the composite screen displayed on the display is an image associated with the captured image of the camera, and the touch sensor receives a selection of a predetermined icon displayed simultaneously with the displayed composite screen, information related to the association between the captured image of the other information device and the captured image of the camera is saved to a storage medium together with the captured image of the camera, and an image obtained by arranging the captured image of the camera and the downloaded captured image of the other information device in a composition mode other than the selected composition mode is output via the communication unit.

11. The camera image sharing system according to claim 10, wherein: The composition processing unit of the automatic composition service providing device generates a composition pattern using the positional relationship, size, and image area occupied in the composite screen of the plurality of composited captured images as evaluation indicators, and registers the composition pattern in the composition table.

12. The camera image sharing system according to claim 11, wherein: The composition processing unit of the automatic composition service providing apparatus learns past selection results of composition patterns selected in the information device and generates a composition pattern.

Images