Program, method, information processing device, and system

Abstract

To add new interest to looking back at past images. [Solution] A program to be executed by a computer having a processor and a memory causes the processor to execute the following steps: registering a specified image, accepting a request for a composite image, acquiring the context of the image by analyzing the registered image in response to the request, acquiring parameter values ​​associated with the registered image from parameter values ​​updated in response to user activity, determining an object from the acquired context and parameter values, and synthesizing the determined object with the registered image and presenting it to the user.

Description

[Technical Field]

[0001] The present disclosure relates to a program, a method, an information processing device, and a system. [Background technology]

[0002] 2. Description of the Related Art In order to preserve memories of a particular event, it is common to use a photographic device to capture images of the scene of the event, such as photographs. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-98102 Summary of the Invention [Problem to be solved by the invention]

[0004] It's fine to take pictures of scenes from events as memories, but if you leave them as they are, you may forget that you even took the pictures. Also, although people expect to look back at the images they have taken someday, the reality is that they rarely look back at them voluntarily unless they have some kind of motivation.

[0005] There is also a technology that uses captured images for growing living organisms (see Patent Document 1), but this technology does not take into consideration looking back at the captured images because the images are taken in order to output images of the growing organisms at the time of shooting.

[0006] An object of the present disclosure is to add new interest to looking back and reviewing past images. [Means for solving the problem]

[0007] A program to be executed by a computer having a processor and a memory, the program causes the processor to execute the following steps: registering a specified image, accepting a request for a composite image, acquiring a context for the image by analyzing the registered image in response to the request, acquiring parameter values ​​associated with the registered image from parameter values ​​updated in response to user activity, determining an object from the acquired context and the acquired parameter values, and synthesizing the determined object with the registered image and presenting it to the user. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to add new interest to looking back at past images. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the overall configuration of a system. [Figure 2] FIG. 2 is a block diagram illustrating a configuration of a terminal device shown in FIG. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of a server. [Figure 4] FIG. 2 is a diagram showing the data structure of an image information DB. [Figure 5] FIG. 2 is a diagram illustrating the data structure of a parameter DB. [Figure 6] FIG. 2 is a diagram illustrating a data structure of an object information DB. [Figure 7] 10 is a diagram illustrating the operations of a terminal device and a server when a composite image is presented to a user. FIG. [Figure 8] FIG. 10 is a diagram showing an image determined as a first image. [Figure 9] FIG. 10 is a diagram illustrating a composite image displayed on a display. [Figure 10] 10A and 10B are diagrams illustrating other examples of composite images displayed on the display. [Figure 11]10A and 10B are diagrams illustrating other examples of images determined as the first images. [Figure 12] 10A and 10B are diagrams illustrating other examples of composite images displayed on the display. [Figure 13] 10A and 10B are diagrams illustrating other examples of composite images displayed on the display. [Figure 14] 10A and 10B are diagrams illustrating other examples of images determined as the first images. [Figure 15] 10A and 10B are diagrams illustrating other examples of composite images displayed on the display. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of the components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0011] <Summary> The system according to this embodiment is a system that adds new interest to looking back at past images by superimposing an image according to predetermined parameters on an image taken in the past.

[0012] <1 Overall system configuration> 1 is a block diagram showing an example of the overall configuration of a system 1. The system 1 shown in Fig. 1 includes a plurality of terminal devices 10 and a server 20. The terminal devices 10 and the server 20 are connected for communication via a network 80.

[0013] 1 shows an example in which the system 1 includes three terminal devices 10, but the number of terminal devices 10 included in the system 1 is not limited to three. The number of terminal devices 10 may be less than three, or may be three or more.

[0014] In this embodiment, a collection of multiple devices may be considered as one server. The allocation of multiple functions required to realize the server 20 according to this embodiment to one or more pieces of hardware can be determined appropriately in consideration of the processing capacity of each piece of hardware and / or the specifications required for the server 20.

[0015] The terminal device 10 is, for example, a device operated by a user that receives a composite image from the server 20. The terminal device 10 may be, for example, a mobile terminal such as a smartphone or a tablet, or may be a stationary PC (Personal Computer) or a laptop PC.

[0016] The terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage 16, and a processor 19.

[0017] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with an external device.

[0018] The input device 13 is a device for receiving input operations from a user (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.).

[0019] The output device 14 is a device (such as a display or speaker) for presenting information to the user.

[0020] The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0021] The storage 16 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0022] The processor 19 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, a register, a peripheral circuit, and the like.

[0023] The server 20 is, for example, a device that provides a composite image to a user. The composite image according to this embodiment is an image in which a predetermined object is composited in a predetermined manner with an image previously taken by the user. The image previously taken by the user may be a still image or a video. The object includes, for example, a person, a character, an object, text, etc. The manner includes size, color, angle, video, still image, etc.

[0024] The server 20 includes a communication IF 22 , an input / output IF 23 , a memory 25 , a storage 26 , and a processor 29 .

[0025] The communication IF 22 is an interface for inputting and outputting signals so that the server 20 can communicate with external devices.

[0026] The input / output IF 23 functions as an interface with an input device for receiving input operations from the user and an output device for presenting information to the user.

[0027] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0028] The storage 26 is for storing data, and is, for example, a flash memory or a hard disk drive (HDD).

[0029] The processor 29 is hardware for executing an instruction set written in a program, and is composed of an arithmetic unit, registers, peripheral circuits, and the like.

[0030] <1.1 Terminal device configuration> Fig. 2 is a block diagram showing an example configuration of the terminal device 10 shown in Fig. 1. As shown in Fig. 2, the terminal device 10 includes a communication unit 120, an input device 13, an output device 14, an audio processing unit 17, a microphone 171, a speaker 172, a camera 161, a position information sensor 150, a storage unit 180, and a control unit 190. The blocks included in the terminal device 10 are electrically connected by, for example, a bus or the like.

[0031] The communication unit 120 performs processing such as modulation and demodulation for the terminal device 10 to communicate with other devices. The communication unit 120 performs transmission processing on the signal generated by the control unit 190 and transmits it to the outside (for example, the server 20). The communication unit 120 performs reception processing on the signal received from the outside and outputs it to the control unit 190.

[0032] The input device 13 is a device for inputting instructions or information by a user operating the terminal device 10. The input device 13 is realized, for example, by a touch-sensitive device 131 or the like, which inputs instructions by touching an operation surface. When the terminal device 10 is a PC or the like, the input device 13 may be realized by a reader, keyboard, mouse, or the like. The input device 13 converts instructions input by the user into electrical signals and outputs the electrical signals to the control unit 190. The input device 13 may include, for example, a receiving port that receives electrical signals input from an external input device.

[0033] The output device 14 is a device for presenting information to a user operating the terminal device 10. The output device 14 is realized, for example, by a display 141 or the like. The display 141 displays data according to the control of the control unit 190. The display 141 is realized, for example, by an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display or the like.

[0034] The audio processing unit 17 performs, for example, digital-to-analog conversion processing of an audio signal. The audio processing unit 17 converts a signal provided from the microphone 171 into a digital signal and provides the converted signal to the control unit 190. The audio processing unit 17 also provides the audio signal to the speaker 172. The audio processing unit 17 is realized, for example, by a processor for audio processing. The microphone 171 receives audio input and provides an audio signal corresponding to the audio input to the audio processing unit 17. The speaker 172 converts the audio signal provided from the audio processing unit 17 into audio and outputs the audio to the outside of the terminal device 10.

[0035] The camera 161 is a device that receives light with a light receiving element and outputs the light as an image capturing signal.

[0036] The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. The satellite positioning system receives signals from at least three or four satellites, and detects the current position of the terminal device 10 equipped with the GPS module based on the received signals.

[0037] The storage unit 180 is realized by, for example, the memory 15, the storage 16, etc., and stores data and programs used by the terminal device 10. The storage unit 180 stores, for example, user information 181 and image information 182.

[0038] The user information 181 includes, for example, information about the user who uses the terminal device 10. The information about the user includes, for example, information for identifying the user, the user's name, age, address, date of birth, telephone number, email address, and predetermined account information.

[0039] The user information 181 includes parameters about the user (for example, information based on the user's activity, information about photography, etc.). The parameters about the user are acquired, for example, by sensors provided in the terminal device 10 (such as the position information sensor 150, the camera 161, an acceleration sensor (not shown, but an inertial sensor may also be used), a gyro sensor (not shown), etc.). The parameters about the user may also be acquired by a clock function provided in the terminal device 10. The parameters about the user may also be acquired, for example, by linking with a server (not shown) that provides a predetermined service used by the user. The parameters about the user include, for example, the following: Information based on user activity (information about the user's movements, such as the number of steps, distance traveled, and route traveled, as well as information about the user's photography, such as the fact that a photo was taken at a specific location and the total number of photos taken) Information based on the activities of other users associated with you (such as family and friends) - Information about the passage of time (time elapsed since shooting, number of days elapsed)

[0040] Information based on the activities of other users associated with a user may include, for example, when a user is managing specific images on the cloud in collaboration with other users, information about the user's movements such as the number of steps taken, distance traveled, and route traveled, as well as information about photos taken by other users such as the fact that photos were taken at specific locations and the total number of photos taken.

[0041] The image information 182 is information about an image based on a shooting signal, and includes, for example, pixel information that represents the image, and meta information that is additional information about the image.

[0042] The meta information includes, for example, information identifying the image, information identifying the photographer, the date of shooting, the shooting location, the shooting direction, brightness, hue, shooting mode, and the subject of the image. The shooting location indicates the location where the image was taken and is based on, for example, information acquired by the position information sensor 150. The shooting direction indicates the direction in which the image was taken and is based on, for example, information acquired by a geomagnetic sensor and a gyro sensor provided in the terminal device 10. The brightness indicates the brightness of the image and is based on, for example, information set when the image was taken. The hue indicates the color tone of the image and is obtained, for example, by analyzing pixel information. The shooting mode indicates the mode when the image was taken and is based on, for example, information set when the image was taken. The subject of the image indicates an object included in the image and is obtained, for example, by analyzing pixel information.

[0043] Note that the function of analyzing an image (pixel information) may not be provided depending on the terminal device 10. In this case, for example, the hue or the photographed subject may not be included in the image information 182.

[0044] The control unit 190 is realized by the processor 19 reading a program stored in the storage unit 180 and executing instructions included in the program. The control unit 190 controls the operation of the terminal device 10. By operating in accordance with the program, the control unit 190 fulfills the functions of an operation reception unit 191, a transmission / reception unit 192, an image analysis unit 193, a display control unit 194, and a storage control unit 195.

[0045] Operation reception unit 191 performs processing for receiving instructions or information input from input device 13. Specifically, for example, operation reception unit 191 receives information based on instructions input from touch-sensitive device 131 or the like. Instructions input via touch-sensitive device 131 or the like are, for example, as follows: - Instruction to register image to server 20 An instruction to request a composite image from the server 20

[0046] Furthermore, the operation reception unit 191 receives voice instructions input from the microphone 171. Specifically, for example, the operation reception unit 191 receives a voice signal that is input from the microphone 171 and converted into a digital signal by the voice processing unit 17. For example, the operation reception unit 191 analyzes the received voice signal and extracts a predetermined noun, thereby acquiring an instruction from the user.

[0047] The transmitting / receiving unit 192 performs processing for the terminal device 10 to transmit and receive data to and from an external device such as the server 20 in accordance with a communication protocol. Specifically, for example, the transmitting / receiving unit 192 transmits an instruction to register an image and image information to be registered to the server 20. The transmitting / receiving unit 192 also transmits an instruction to request a composite image to the server 20.

[0048] Furthermore, the transmitting / receiving unit 192 transmits at least a portion of the user information 181 to the server 20. For example, the transmitting / receiving unit 192 transmits the information to the server 20 at the following times: -Predetermined period When motion is detected by a sensor In response to user input (for example, input of image registration instructions or input of a request for a composite image)

[0049] The image analysis unit 193 analyzes the image signal captured by the camera 161. Specifically, the image analysis unit 193 analyzes pixel information and acquires information contained in the image. For example, the image analysis unit 193 acquires hue as information contained in the image. The image analysis unit 193 also acquires photographed people, places, objects, events, etc. as information contained in the image. Note that the image analysis unit 193 may not exist depending on the terminal device 10.

[0050] The display control unit 194 controls the output device 14 to present to the user an image provided by the server 20. Specifically, for example, the display control unit 194 causes the display 141 to display an image based on information transmitted from the server 20.

[0051] The storage control unit 195 controls the storage of data in the storage unit 180. For example, the storage control unit 195 stores parameters about the user in the user information 181. Specifically, for example, the storage control unit 195 stores the travel distance, travel route, and the like acquired by the position information sensor 150 in the user information 181. The storage control unit 195 also stores the number of steps acquired by the acceleration sensor in the user information 181. The storage control unit 195 also stores the total number of images taken by the camera 161, information about images taken at a predetermined location, and the like acquired by the camera 161 in the user information 181. The storage control unit 195 also stores information provided from a predetermined server in the user information 181. The storage control unit 195 also stores the time elapsed from a predetermined reference point in the user information 181.

[0052] Furthermore, the storage control unit 195 stores information about the image captured by the camera 161 in the image information 182. Specifically, for example, the storage control unit 195 stores information about the image based on the image capturing signal acquired by the camera 161 in the image information 182.

[0053] <1.2 Functional configuration of the server> 3 is a diagram showing an example of the functional configuration of the server 20. As shown in FIG. 3, the server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0054] The communication unit 201 performs processing for the server 20 to communicate with external devices.

[0055] The storage unit 202 has databases in which data for generating a composite image is accumulated, such as an image information database (DB) 281, a parameter database (DB) 282, and an object information database (DB) 283.

[0056] The image information DB 281 is a database for storing information about images registered by users, as will be described in detail later.

[0057] The parameter DB 282 is a database for holding information managed about users who receive the provision of services. In the parameter DB 282, parameter values ​​indicating user activities and the like are managed for items corresponding to registered images. Hereinafter, the items managed in the parameter DB 282 will be referred to as parameter items. Details will be described later.

[0058] The object information DB 283 is a database for holding information about objects to be combined with images, as will be described in detail later.

[0059] The control unit 203 is realized by the processor 29 reading a program stored in the storage unit 202 and executing instructions included in the program. By operating in accordance with the program, the control unit 203 performs functions shown as a reception control module 2041, a transmission control module 2042, a registration module 2043, a management module 2044, an analysis module 2045, an acquisition module 2046, a decision module 2047, and a presentation module 2048.

[0060] The reception control module 2041 controls the process by which the server 20 receives signals from external devices in accordance with a communication protocol.

[0061] The transmission control module 2042 controls the process in which the server 20 transmits signals to external devices in accordance with a communication protocol.

[0062] The registration module 2043 controls the registration process of an image designated by a user. For example, when the registration module 2043 receives from the user an instruction to register an image and image information about the image, the registration module 2043 stores the received image information in the image information DB 281.

[0063] The management module 2044 manages parameters related to the user. For example, when the management module 2044 receives at least a portion of the user information 181 from the terminal device 10, it updates the parameter DB 282 based on the received information. Specifically, for example, when an image is registered, the management module 2044 sets parameter items related to the registered image. For example, the management module 2044 sets, as parameter items, items related to the passage of time (e.g., number of days, number of months, number of years, etc.), items related to the number of times a location has been experienced (e.g., number of visits, number of photos taken, etc.), items related to movement (e.g., number of steps, distance traveled, destination, etc.), or total number of photos taken, etc.

[0064] The management module 2044 updates the values ​​of the parameter items based on the information about the user's movements included in the received information. For example, the management module 2044 determines that the user has visited a specific location based on the information about the user's movements. The management module 2044 also manages the number of times the specific location has been visited.

[0065] The management module 2044 also determines that the user has taken a photo at a predetermined location from information about the user's photoshoot included in the received information. The management module 2044 also manages the number of times a photo has been taken at a predetermined location.

[0066] Furthermore, the management module 2044 updates the values ​​of the parameter items based on information based on the activities of other users associated with the user, which information is included in the received information. Note that the management module 2044 may receive information based on the activities of other users associated with the user from a server (not shown) that provides a predetermined service.

[0067] Furthermore, the management module 2044 updates the values ​​of the parameter items based on the information regarding the passage of time contained in the received information.

[0068] The management module 2044 may set parameter items based on instructions from an administrator or a user of the server 20. For example, the management module 2044 may set conditions imposed by an administrator, such as a quota, as parameter items. For example, the management module 2044 may set conditions imposed by a user, such as a quota, as parameter items.

[0069] The analysis module 2045 controls the analysis process of image information. For example, the analysis module 2045 analyzes information about a specific first image among the images registered in the image information DB 281, and acquires a context contained in the first image. In this embodiment, the context represents, for example, the context when the image was captured, in other words, the meaning of when the image was captured, the significance of when the image was captured, the scene in which the image was captured, etc. The analysis module 2045 acquires one or more contexts from multiple contexts registered in advance in the storage unit 202 based on information contained in the image information. For example, the analysis module 2045 acquires the following information as context by performing image analysis on pixel information contained in the image information: You are in a location where there is a building shown in the image (amusement park, baseball stadium, soccer stadium, athletics stadium, live music venue, etc.) - There is a person in the image (the user, a person registered by the user, a person identifiable based on a public database, etc.) - The user must be with the character(s) shown in the image (e.g., a character registered by the user, a character that can be identified based on a public database, etc.) The event shown in the image (birthday, Christmas, Halloween, etc.) is taking place The image contains objects (cakes, Christmas trees, jack-o'-lanterns, etc.) The atmosphere of the image The atmosphere of an image may be determined from the hue of the image or from the facial expressions of people appearing in the image.

[0070] Furthermore, the analysis module 2045 acquires the following information as context based on, for example, meta information included in the image information. -The image must be taken on a specific date -The image must be taken at a designated location -The image must be taken in the specified direction. -The image must have a specific atmosphere (based on color tone) The image must include a specific subject (object, person, character, event, etc.)

[0071] The analysis module 2045 may perform image analysis processing on pixel information contained in the image information and extraction processing of meta information contained in the image information, or may perform either one of the processing.

[0072] The analysis module 2045 determines an image that satisfies certain requirements from among the registered images as the first image. The certain requirements are, for example, as follows: -Images specified by the user Randomly selected images Images related to the current date (images taken on the same day, images taken in the same month, etc.) Images related to the current location (images taken at the same location, images taken at different locations within the same facility, etc.) Images related to an event that is believed to be currently taking place (images of the same event or a series of events) - Images that meet specified shooting conditions (images that include specified objects, images taken in a specified direction, images taken at a specified brightness, images with a specified hue, etc.)

[0073] The acquisition module 2046 controls the acquisition of parameter values ​​related to an image. For example, the acquisition module 2046 refers to the parameter DB 282 to acquire values ​​of predetermined parameter items for, for example, the first image analyzed by the analysis module 2045 among the registered images.

[0074] Specifically, the acquisition module 2046 selects parameter items to acquire for the first image. The acquisition module 2046 selects the set parameter items. Here, the values ​​of the parameter items may change between the time the image is captured or registered and the time the image is synthesized. For example, the acquisition module 2046 selects at least one of the following parameter items: number of days elapsed, number of visits, number of photos taken, number of steps taken, distance traveled, destination, total number of photos taken, etc.

[0075] The acquisition module 2046 may select one of the parameter items based on, for example, the context acquired by the analysis module 2045. For example, if the analysis module 2045 acquires as the context that the user is at a specific baseball stadium, the acquisition module 2046 selects as the parameter item the number of visits to this baseball stadium. Also, for example, if the analysis module 2045 acquires as the context that the image was taken a long time ago (an image taken on a specific date), the acquisition module 2046 selects as the parameter item the number of days that have passed since the image was registered or the number of days that have passed since the image was taken.

[0076] The acquisition module 2046 may also randomly select any of the parameter items regardless of the acquired context.

[0077] The acquisition module 2046 acquires a value for the selected parameter item based on the parameter DB 282. For example, if the number of visits is selected as a parameter item, the acquisition module 2046 acquires the number of visits to a predetermined location stored in the parameter DB 282.

[0078] Furthermore, for example, when the number of days elapsed is selected as a parameter item, the acquisition module 2046 acquires the number of days that have elapsed since the image was registered based on the update date stored in the parameter DB 282 and the registration date stored in the image information DB 281. The acquisition module 2046 may also acquire the number of days that have elapsed since the image was captured based on the update date stored in the parameter DB 282 and the capture date stored in the image information DB 281.

[0079] The determination module 2047 determines the object to be combined with the image and the aspect of the object. For example, the determination module 2047 determines the object based on the context acquired by the analysis module 2045. In this embodiment, the object is classified into, for example, a character, a person, an object, a letter, etc., and the content corresponds to the classification.

[0080] The determination module 2047 determines the level of the determined object based on the values ​​of the parameter items acquired by the acquisition module 2046. In this embodiment, the level of the object represents, for example, the luxury of the object, the degree of change of the object, etc. For example, when the object is classified as a character, the luxury of the object is represented by the following: Number of characters (number increases as level increases) Character's inventory (As the level increases, the inventory increases and becomes larger) Character facial expressions (the higher the level, the more they smile) Character evolution (the character will evolve as its level increases) Characters move (characters that didn't move at low levels will move at higher levels, and character movements will change as the level increases)

[0081] The determination module 2047 determines the display mode of the object based on the configuration in the image. For example, the determination module 2047 determines the position, size, angle, etc. at which the determined object is to be synthesized based on the position, size, shape, etc. of an area without an object in the image.

[0082] The presentation module 2048 presents to the user a composite image in which a second image of a predetermined object is composited with a first image of any of the registered images. Specifically, for example, the presentation module 2048 creates a second image based on the object determined by the determination module 2047 and the display mode of the object. The presentation module 2048 creates a composite image by composite- ing the created second image with the first image. The presentation module 2048 presents the created composite image to the user.

[0083] <2 Data Structure> 4 to 6 are diagrams showing the data structure of the database stored in the server 20. Note that FIGS. 4 to 6 are merely examples, and do not exclude data not shown.

[0084] Fig. 4 is a diagram showing the data structure of the image information DB 281. As shown in Fig. 4, each record of the image information DB 281 includes an item "image ID," an item "registrant," an item "registration date," an item "photographer," an item "photography date," an item "location," an item "photography direction (azimuth, elevation angle)," an item "luminance," an item "hue," an item "photography mode," an item "photographed subject," etc. Although not shown in Fig. 4, pixel information is associated with each record.

[0085] The item "Image ID" stores information for identifying an image. The image ID is automatically set when registering image information, for example.

[0086] The item "Registrant" stores the name of the user who registered the image. The item "Registrant" may also store a user ID that identifies the user.

[0087] The item "registration date" stores the date on which the user registered the image.

[0088] The item "Photographer" stores the name of the user who took the image. The item "Photographer" may also store a user ID that identifies the user.

[0089] The item "photographed date" stores the date on which the image was photographed.

[0090] The item "location" stores the location where the user captured the image. The item "location" is represented by, for example, longitude and latitude.

[0091] The item "photography direction" stores the direction in which the user captured the image. The item "photography direction" is expressed by, for example, an azimuth angle and an elevation angle.

[0092] The item "brightness" stores the brightness that was set when the user captured the image.

[0093] The item "hue" stores the hue contained in the image.

[0094] The item "photography mode" stores the photography mode that was set when the user captured the image.

[0095] The item "photographed object" stores an object that is determined to be included in the image through analysis after the image is captured.

[0096] Fig. 5 is a diagram showing the data structure of the parameter DB 282. As shown in Fig. 5, each record of the parameter DB 282 includes an item "user ID", an item "parameter item", an item "value", and the like.

[0097] The item "user ID" stores information for identifying a user.

[0098] The item "parameter item" stores the items of the parameters being managed. A parameter item is added, for example, when an image is registered.

[0099] The item "value" stores the value for the parameter item.

[0100] Fig. 6 is a diagram showing the data structure of the object information DB 283. As shown in Fig. 6, each record of the object information DB 283 includes an item "object ID", an item "classification", an item "related context", an item "content", and the like.

[0101] The item "object ID" stores information that identifies an object.

[0102] The item "classification" stores the classification to which the object belongs.

[0103] The item "associated context" stores the context associated with the object.

[0104] The "content" item stores the content of an object. The "content" item includes multiple sub-items. In this embodiment, the "content" item is set for each level. For each object, the sub-item "level 1," sub-item "level 2," sub-item "level 3," and sub-item "level 4" store content according to the level. The content of each level may be set for each parameter item selected by the acquisition module 2046, for example.

[0105] <3 operations> The operations of the terminal device 10 and the server 20 when an image related to an object is synthesized with an image registered by a user and provided will be described.

[0106] FIG. 7 is a diagram illustrating an example of the operations of the terminal device 10 and the server 20 when a composite image is presented to a user.

[0107] First, the user operates the terminal device 10 to start a service that provides a composite image. The terminal device 10 displays, for example, an image registration screen on a service provision screen. On the registration screen, the user selects at least one image from the images stored in the terminal device 10. To confirm the image selection, the user presses, for example, a confirmation button displayed on the terminal device 10. Upon receiving the image selection from the user, the terminal device 10 transmits an instruction to register the image and information about the selected image to the server 20 (step S11).

[0108] When the server 20 receives the image registration instruction and information about the selected image, the registration module 2043 stores the received image information in the image information DB 281 (step S21). The server 20 creates parameter items related to the selected image based on the received image information using the management module 2044. The management module 2044 adds the created parameter items to the parameter DB 282.

[0109] The terminal device 10 transmits at least a part of the user information 181 to the server 20 at a predetermined timing (step S12). The terminal device 10 transmits the information to the server 20 regardless of, for example, an instruction to register an image.

[0110] When the server 20 receives the information transmitted from the terminal device 10, the management module 2044 updates the parameter DB 282 based on the received information (step S22). Specifically, for example, the management module 2044 updates the values ​​in each parameter item of the parameter DB 282.

[0111] After transmitting the instruction to register the image, the terminal device 10 transmits an instruction to the server 20 at a predetermined timing to request the server 20 to present the composite image (step S13). The predetermined timing is, for example, as follows. When the user inputs a request for a composite image When a specific date and time is reached When a user's activity meets a specified condition (such as a specified number of visits, a specified number of photos taken, a specified number of steps taken, a specified distance traveled, or reaching a specified location) When the user performs a process related to viewing an image stored in the image information 182 (when opening an album in which images related to a registered image are stored, when viewing a registered image, etc.) When the activity of other users associated with the user meets certain conditions

[0112] The terminal device 10 may accept from the user a designation of an image that will be the basis for the composite image. The user selects an image that will be the basis for the composite image from among the registered images and presses a confirm button. Upon receiving the image selection from the user, the terminal device 10 transmits to the server 20 an instruction to select the image together with an instruction to request the composite image.

[0113] When the server 20 receives an instruction requesting a composite image, the analysis module 2045 determines a first image from among the registered images. If the user has specified an image that will be the basis for the composite image, the analysis module 2045 determines the image specified by the user as the first image. If the user has not specified an image that will be the basis for the composite image, the analysis module 2045 determines the first image based on, for example, information stored in the image information DB 281 and the user's current situation. The analysis module 2045 analyzes information about the determined first image and acquires the context included in the first image (step S23).

[0114] Fig. 8 is a diagram showing an example of an image determined as the first image. The image shown in Fig. 8 was taken during a birthday celebration. The image shown in Fig. 8 includes the person being celebrated, the person celebrating, a birthday cake, etc. Furthermore, the image shown in Fig. 8 has meta information added thereto, including, for example, items to be listed in the image information DB 281.

[0115] The analysis module 2045 acquires a context from the image shown in Fig. 8. For example, the analysis module 2045 acquires a context of "an image about AAAA's birthday party held on 20YY / MM / DD (longitude: Lat1, latitude: Lon1)" based on pixel information included in the image information and information stored in the image information DB 281.

[0116] The server 20 uses the acquisition module 2046 to acquire values ​​for predetermined parameter items for the first image by referring to the parameter DB 282 (step S24). Specifically, the acquisition module 2046 selects parameter items to acquire for the first image. The acquisition module 2046 selects at least one of preset parameter items, such as the number of days elapsed, the number of visits, the number of photos taken, the number of steps taken, the distance traveled, the destination, and the total number of photos taken. The acquisition module 2046 acquires the value for the selected parameter item based on the parameter DB 282.

[0117] In the example shown in FIG. 8 , the acquisition module 2046 selects a parameter item based on the context of, for example, “images about AAAA’s birthday party held on 20YY / MM / DD at (longitude: Lat1, latitude: Lon1)” acquired by the analysis module 2045. The acquisition module 2046 selects, as the parameter item, for example, one of “number of photos taken at (longitude: Lat1, latitude: Lon1),” “number of photos taken at the birthday party,” “number of days elapsed since the photo was taken,” “number of days elapsed since the registration date,” or “total number of photos taken.” In this description, it is assumed that the acquisition module 2046 selects, for example, “number of photos taken at the birthday party” as the parameter item. The acquisition module 2046 references the parameter DB 282 and acquires “number of photos taken at the birthday party.”

[0118] The server 20 determines, by the determination module 2047, the object to be composited into the first image, the level of the object, and the manner of the object (step S25). Specifically, the determination module 2047 determines the object based on the context acquired by the analysis module 2045. The determination module 2047 determines the level of the determined object based on the value of the parameter item acquired by the acquisition module 2046. The determination module 2047 determines the display manner of the object based on the configuration within the image.

[0119] In the example shown in FIG. 8, the determination module 2047 determines an object based on the context of, for example, "an image about AAAA's birthday party held on 20YY / MM / DD (longitude: Lat1, latitude: Lon1)" acquired by the analysis module 2045. The determination module 2047 selects an object whose "related context" item is, for example, a birthday. The determination module 2047 determines the level of the object according to, for example, the value of "number of times the birthday party was photographed" determined by the acquisition module 2046. The determination module 2047 pre-sets a plurality of reference values ​​according to the level. The determination module 2047 compares the number of times the photograph was taken with the reference value to determine the level of the object. The determination module 2047 determines the dashed-line area 1411 in the first image shown in FIG. 8 as the area into which the object is to be composited. The determination module 2047 determines the size, angle, etc. of the object so that it can be composited into the dashed-line area 1411.

[0120] The server 20 creates a composite image using the presentation module 2048 (step S26). Specifically, for example, the presentation module 2048 creates a second image based on the object determined by the determination module 2047 and the display mode of the object. The presentation module 2048 creates a composite image by combining the created second image with the first image. The server 20 stores the created composite image in the storage unit 202. For example, the server 20 stores the composite image in the storage unit 202 as an image separate from the first image.

[0121] The server 20 presents the created composite image to the user (step S27). Specifically, the server 20 transmits information about the created composite image to the terminal device 10.

[0122] The terminal device 10 displays the received composite image on the display 141 (step S14).

[0123] Fig. 9 is a diagram showing an example of a composite image displayed on display 141. In the composite image shown in Fig. 9, a second image 1412 including characters and characters celebrating a birthday is composited.

[0124] The content of the composite image created based on the first image is not limited to that shown in Fig. 9. If the value of the parameter item increases by the time the composite image is displayed, a composite image with a higher level of objects is displayed.

[0125] For example, suppose that after the user checks the composite image shown in FIG. 9, the value of a predetermined parameter item increases. Specifically, suppose that after the user checks the composite image shown in FIG. 9, there are multiple opportunities to photograph birthday parties. The birthday parties to be photographed may be limited to AAAA birthday parties, or birthday parties of other people may be allowed. As a result, the value of "number of times birthday parties have been photographed" selected as a parameter item increases. The user requests the composite image again after a predetermined period has elapsed since requesting the composite image shown in FIG. 9. The determination module 2047 determines the level of the object according to the increased value of "number of times birthday parties have been photographed." The presentation module 2048 creates a composite image in which the object is more luxurious.

[0126] Fig. 10 is a diagram showing another example of a composite image displayed on display 141. The composite image shown in Fig. 10 is an image created in response to a request for a composite image input after a predetermined period of time has elapsed since the composite image shown in Fig. 9 was presented. The composite image shown in Fig. 10 has more luxurious content than the composite image shown in Fig. 9.

[0127] Fig. 11 is a diagram showing another example of an image determined as the first image. The image shown in Fig. 11 was taken in a park. The image shown in Fig. 11 includes children playing in the park, playground equipment, benches, etc. Furthermore, the image shown in Fig. 11 has meta information added thereto, including items to be listed in the image information DB 281, for example.

[0128] The analysis module 2045 acquires a context from the image shown in Fig. 11. For example, the analysis module 2045 acquires a context of "an image of AAAA playing in a park at longitude: Lat2, latitude: Lon2 on 20YY / MM / DD" based on pixel information included in the image information and information stored in the image information DB 281.

[0129] The acquisition module 2046 selects a parameter item based on the context of "an image of AAAA playing in a park at (longitude: Lat2, latitude: Lon2) on 20YY / MM / DD" acquired by the analysis module 2045. The acquisition module 2046 selects, as a parameter item, one of, for example, "number of times photographed in the park at (longitude: Lat2, latitude: Lon2)," "number of visits to the park at (longitude: Lat2, latitude: Lon2)," "number of steps taken in the park," "distance traveled in the park," "number of days elapsed since the photographing date," "number of days elapsed since the registration date," or "total number of times photographed." In this description, it is assumed that the acquisition module 2046 selects, for example, "number of visits to the park at (longitude: Lat2, latitude: Lon2)" as a parameter item. The acquisition module 2046 references the parameter DB 282 and acquires "number of visits to the park at (longitude: Lat2, latitude: Lon2)."

[0130] The determination module 2047 determines an object based on the context of, for example, "an image of AAAA playing in a park at (longitude: Lat2, latitude: Lon2) on 20YY / MM / DD" acquired by the analysis module 2045. The determination module 2047 selects, for example, an object whose "related context" item is a park. The determination module 2047 determines the level of the object according to, for example, the value of "number of visits to a park at (longitude: Lat2, latitude: Lon2)" determined by the acquisition module 2046. Here, it is assumed that the number of visits is insufficient to reach the level at which the object is synthesized. The determination module 2047 determines an object that encourages a repeat visit.

[0131] The presentation module 2048 creates a second image based on the object and the display mode of the object determined by the determination module 2047. The presentation module 2048 combines the created second image with the first image to create a composite image.

[0132] FIG. 12 is a diagram showing another example of a composite image displayed on the display 141. In the composite image shown in FIG. 12, text encouraging a repeat visit is synthesized. In FIG. 12, the parameter item is "number of visits," so the text "If you come to the park one more time, it will be a fun image" is displayed as the text encouraging a repeat visit. If the acquisition module 2046 selects another parameter item, a different statement is displayed. For example, if the parameter item is "number of times photos are taken at the park," the determination module 2047 determines "If you take one more photo at the park, it will be a fun image" as the text object encouraging a repeat photo shoot at the park.

[0133] Furthermore, for example, when "number of steps in the park" is selected as the parameter item, the determination module 2047 determines "if you walk a little more, the image will be more fun" as a character object encouraging further walking. Furthermore, for example, when "distance traveled in the park" is selected as the parameter item, the determination module 2047 determines "if you explore the park a little more, the image will be more fun" as a character object encouraging further movement.

[0134] Furthermore, for example, when "number of days elapsed since the date of shooting" or "number of days elapsed since the date of registration" is selected as the parameter item, the determination module 2047 determines "It will be a fun image tomorrow" as the text object that prompts the user to take another photo. Furthermore, for example, when "total number of times photos have been taken" is selected as the parameter item, the determination module 2047 determines "It will be a fun image if you take one more photo" as the text object that prompts the user to take another photo.

[0135] The number of times the prompt to perform the next action is displayed is not limited to one. It may be set to display a preset number of times. Each time the corresponding action is performed, the number decreases, allowing the user to enjoy the countdown.

[0136] Assume that after the user checks the composite image shown in FIG. 12, the user performs the prompted action. Specifically, assume that after the user checks the composite image shown in FIG. 12, the user has had one or more opportunities to visit the park. As a result, the value of "number of visits to park (longitude: Lat2, latitude: Lon2)" selected as a parameter item increases. After the user performs the requested activity, the user requests a composite image again. The determination module 2047 determines the level of the object according to the increased value of "number of visits to park (longitude: Lat2, latitude: Lon2)". The presentation module 2048 creates a composite image in which the object is more luxurious.

[0137] Fig. 13 is a diagram showing another example of a composite image displayed on the display 141. The composite image shown in Fig. 13 is an image created in response to a request instruction issued after the user has performed the requested activity after the composite image shown in Fig. 12 has been presented. The composite image shown in Fig. 13 is based on the first image shown in Fig. 11. In the composite image shown in Fig. 13, a rainbow, sun, and light objects are combined with an image of a park.

[0138] Fig. 14 is a diagram showing another example of an image determined as the first image. The image shown in Fig. 14 was taken at an amusement park. The image shown in Fig. 14 includes children playing at the amusement park, amusement park buildings, amusement park attractions, etc. Furthermore, the image shown in Fig. 14 has meta information added thereto, including, for example, items to be listed in the image information DB 281.

[0139] The analysis module 2045 acquires a context from the image shown in Fig. 14. For example, the analysis module 2045 acquires a context of "an image of AAAA playing on attraction A1 at an amusement park at (longitude: Lat3, latitude: Lon3) on 19YY / MM / DD" based on pixel information included in the image information and information stored in the image information DB 281.

[0140] The acquisition module 2046 selects a parameter item based on the context of "an image of AAAA playing on attraction A1 at an amusement park at (longitude: Lat3, latitude: Lon3) on 19YY / MM / DD" acquired by the analysis module 2045. The acquisition module 2046 selects, as a parameter item, one of, for example, "number of times photographed at the amusement park at (longitude: Lat3, latitude: Lon3)," "number of visits to the amusement park at (longitude: Lat3, latitude: Lon3)," "number of steps taken at the amusement park," "distance traveled at the amusement park," "number of days elapsed since the photograph was taken," "number of days elapsed since the registration date," or "total number of times photographs were taken." In this description, it is assumed that the acquisition module 2046 selects, for example, "number of days elapsed since the photograph was taken" as a parameter item. The acquisition module 2046 references the parameter DB 282 and acquires "number of days elapsed since the photograph was taken."

[0141] The determination module 2047 determines an object based on the context of, for example, "an image of AAAA playing on attraction A1 at an amusement park (longitude: Lat3, latitude: Lon3) on 19YY / MM / DD" acquired by the analysis module 2045. For example, the determination module 2047 selects an object whose "related context" item is an amusement park. The determination module 2047 determines the level of the object based on, for example, the value of the "number of days elapsed since the photo was taken" determined by the acquisition module 2046. Depending on the amusement park, attractions previously installed may have been replaced with new attractions. Therefore, if a predetermined number of days or more have passed since the photo was taken, that is, if the value of the parameter item exceeds a predetermined value, an object introducing an existing attraction may be set. The determination module 2047 determines an object introducing an existing attraction for a first image for which the number of days elapsed is greater than or equal to a predetermined number. The determination module 2047 determines the size, angle, etc. of the object so that the determined object can be composited with the first image shown in FIG. 14.

[0142] The presentation module 2048 creates a second image based on the object and the display mode of the object determined by the determination module 2047. The presentation module 2048 combines the created second image with the first image to create a composite image.

[0143] FIG. 15 is a diagram showing another example of a composite image displayed on the display 141. In the composite image shown in FIG. 15, "Now it's new attraction A2!" is displayed near attraction A1 as an object introducing a new attraction A2 that currently exists. By seeing the display introducing the new attraction, the user becomes interested in the new attraction and can reminisce about the attraction that no longer exists. In this way, the server 20 may composite information useful to the user as an object.

[0144] As described above, in this embodiment, the server 20 registers an image specified by a user using the registration module 2043. The server 20 receives a request for a composite image from a user using the analysis module 2045. The server 20 acquires the context of the image by analyzing the registered image in response to the request using the analysis module 2045. The server 20 acquires parameter values ​​associated with the registered image, among parameter values ​​updated according to the user's activity, using the acquisition module 2046. The server 20 determines an object based on the acquired context and the acquired parameter values ​​using the determination module 2047. The server 20 composites the determined object with the registered image and presents it to the user using the presentation module 2048. This enables the server 20 to composite an image not included in the image captured in the past in a manner consistent with predetermined parameters from the time of registration or capture to the time of the request. This allows the user to enjoy unexpected images based on past images.

[0145] Therefore, the server 20 according to this embodiment can add new interest to looking back at past images.

[0146] The server 20 also receives a request for a composite image using the original image into which the object has been composited, via the analysis module 2045. The server 20 also acquires, via the acquisition module 2046, parameter values ​​that have been updated in accordance with the user's activity since the previous composite. As a result, when the server 20 receives a new request for a composite image for the same image after creating it, the server 20 can reflect, in the composite image, parameter values ​​that have been updated between the previous request and the current request. This allows the server 20 to present to the user a composite image that is different from the previously presented composite image, for example, a composite image in which a more luxurious object has been composited than in the previous composite image. This enhances the user's enjoyment of previously captured images.

[0147] In the above embodiment, the acquisition module 2046 acquires parameter values ​​from parameter items such as a value related to the passage of time, the number of times a location has been experienced, a value related to movement, or the total number of times photography has been taken, etc. This allows the acquisition module 2046 to acquire parameter values ​​from any parameter item that has been set.

[0148] Furthermore, in the above embodiment, the acquisition module 2046 acquires the parameter values ​​from the values ​​of the parameter items set by the user or administrator. This allows the user to set the parameter items for making the composite image appear by himself / herself. In other words, the user can regard the presentation of the composite image as a reward for achieving the goal (quota) that he / she has set himself / herself. This increases the interest of the presented composite image.

[0149] In addition, the administrator can freely set conditions associated with images. For example, the administrator can include a predetermined time in the parameter items for a location. Specifically, the administrator can set, for example, "the number of visits to a predetermined location between XX and XX." By allowing the administrator to set parameter items in this way, the variety of conditions for making a composite image appear increases. This allows users to view the presentation of a composite image as a reward for achieving a goal (quota) they have set for themselves. This increases the interest of the composite image being presented. In addition, the administrator can provide users with an opportunity to look back on past images.

[0150] In the above embodiment, the analysis module 2045 obtains the context by analyzing the location, person, or event contained in the image, which enables the server 20 to automatically extract the context from the content contained in the pixel information.

[0151] In the above embodiment, the analysis module 2045 obtains the context by determining the location, direction, or date and time when the image was taken from the meta information related to the image, which enables the server 20 to automatically extract the context from the meta information.

[0152] In the above embodiment, the determination module 2047 determines an object from a character, person, object, or letter that corresponds to the acquired context, which enables the server 20 to automatically determine an object that corresponds to the extracted context.

[0153] Furthermore, in the above embodiment, the determination module 2047 determines the content of the object based on the magnitude of the parameter value that is updated according to the user's activities, etc. This allows the server 20 to change the content of the object to be combined with the image according to the magnitude of the parameter value, allowing the user to enjoy seeing how their own activities, etc. have affected the image.

[0154] In the above embodiment, the acquisition module 2046 analyzes images that satisfy predetermined requirements among the registered images, which allows the server 20 to select an image that will be the basis for a composite image when multiple images are registered.

[0155] <4 Variations> The server 20 may add information relating the created composite image to the original first image, for example, by writing the information relating to the first image into the meta information of the composite image.

[0156] When the terminal device 10 receives the composite image from the server 20, it displays the received composite image on the display 141. The terminal device 10 references the meta information attached to the composite image and detects images associated with the composite image from the image information 182. Specifically, the terminal device 10 detects, for example, images captured under the same conditions as the first image associated with the composite image. Examples of images captured under the same conditions include the following: - Images whose shooting time is within a specified range (regardless of whether they are still images or videos. The same applies below.) - Images taken within a specified range - Images taken within a specified range of shooting directions - Images associated with the subject

[0157] The terminal device 10 displays the detected image on the display 141 so that it can be distinguished from other images. Specifically, for example, while the terminal device 10 is displaying a composite image on the display 141, the terminal device 10 suggests to the user that an image related to the composite image be displayed. If the user accepts the suggestion, the terminal device 10 displays on the display 141 an image related to the composite image, for example, a thumbnail image of an image captured under the same conditions as the first image associated with the composite image. The user selects an image from the thumbnail images displayed on the display 141 and visually confirms the image. In this way, by distinguishing an image related to an image into which an object has been composited from other images and presenting it to the user, the server 20 can make it easier for the user to select an image related to an image that was brought to mind by the composite image. This allows the user to reminisce about their own memories when prompted by the composite image.

[0158] The server 20 may also share the generated image signal with other users. For example, when the server 20 transmits the composite signal to the terminal device 10, the server 20 suggests to the user that the transmitted composite signal be shared.

[0159] If the user likes the composite image and agrees to share it with others, the user accepts the proposal from the server 20.

[0160] When a user agrees to share the composite image, the server 20 spreads information about access to the composite image to other users using a predetermined SNS (Social Network Service) service, email service, bulletin board, etc. The server 20 may also limit the period during which the composite image can be accessed. The server 20 provides a predetermined reward to users who agree to sharing. Examples of the predetermined reward include the following: Adding objects to the composite image · Providing advantageous benefits (points, coupons, etc.) Adding values ​​for a given parameter

[0161] In this way, by sharing an image in which an object has been composited with other users, the server 20 can present the fun of creating a composite image to the other users. This allows the server 20 to make users who have not created a composite image before interested in composite images. Furthermore, for users who share a composite image, their own enjoyment can be spread, allowing them to share the enjoyment with other users.

[0162] Furthermore, the server 20 may display the conditions (parameter items and values ​​for the parameter items) for generating the composite image on the site where the composite image is displayed, thereby enabling users who share the composite image to understand the conditions for obtaining the composite image.

[0163] <5 Other embodiments> In the above embodiment, the case has been described in which the acquisition module 2046 selects one parameter item and acquires a value for the selected parameter item. However, the parameter item selected by the acquisition module 2046 is not limited to one. The acquisition module 2046 may select two or more parameter items. The acquisition module 2046 may output multiple values ​​for the selected parameter item to the determination module 2047. The acquisition module 2046 may output any one of the multiple values ​​for the selected parameter item to the determination module 2047. The acquisition module 2046 may calculate an evaluation value from the multiple values ​​for the selected parameter item.

[0164] The determination module 2047 determines the level of the object based on the value output from the acquisition module 2046. When multiple values ​​are output from the acquisition module 2046, the determination module 2047 may determine the level of the object using any one of the multiple values. Alternatively, the determination module 2047 may determine the level of the object using an evaluation value calculated based on the multiple values.

[0165] In the above embodiment, information introducing new attractions has been described as an example of useful information to be presented to the user. However, the useful information is not limited to information introducing new attractions. The determination module 2047 may also include the following as useful information: Information about discounts Information about available rights Information to notify you of your winnings

[0166] If there is a possibility that such information will be synthesized, the server 20 displays the conditions for generating useful information on the synthesized image so that the user can understand them when registering the image.

[0167] In the above embodiment, the image is an image taken by a user. However, the image is not limited to this. The image may be, for example, a screenshot of a predetermined application or an image taken within the predetermined application. When the image is a screenshot of a predetermined application or an image taken within the predetermined application, the information based on the user's activity may include, for example, an operation log by the user. The user's operation log may include, for example, information based on the activity of a character operated by the user.

[0168] In the above embodiment, the parameter items managed in the parameter DB 282 are parameters indicating the user's activities, etc. The parameter DB 282 may also manage parameters indicating the activities, etc. of other users associated with the user. The acquisition module 2046 may select a parameter item for the first image from the parameter items indicating the activities, etc. of other users associated with the user. The determination module 2047 determines the level of the object based on the value of the selected parameter item. In this way, by changing the level of the object based on the value of the parameter item indicating the activities, etc. of other users, the user's own photo is affected by the activities of others. Furthermore, the other users' photos are affected by the user's own activities. This increases the interest when creating or changing a composite image.

[0169] In the above embodiment, the determination module 2047 determines the object based on the context acquired by the analysis module 2045. However, the determination module 2047 does not have to determine the object based on the context. The determination module 2047 may determine the object, for example, in the following manner. - Specified by the user ·random The object has already been determined

[0170] Although several embodiments of the present disclosure have been described above, these embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and modifications are intended to be included in the scope of the inventions and their equivalents as defined in the claims, as well as in the scope and spirit of the inventions.

[0171] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0172] (Appendix 1) A program to be executed by a computer having a processor 29 and a memory 25, the program causing the processor to execute the steps of: registering a specified image (step S21); accepting a request for a composite image (step S13); acquiring the context of the image by analyzing the registered image in response to the request (step S23); acquiring parameter values ​​associated with the registered image from parameter values ​​updated in accordance with user activity (step S24); determining an object from the acquired context and the acquired parameter values ​​(step S25); and synthesizing the determined object with the registered image and presenting it to the user (step S26).

[0173] (Appendix 2) The program according to (Appendix 1), wherein in the step of acquiring the parameter value, the parameter value is acquired from a value relating to the passage of time, a number of times a location has been experienced, a value relating to movement, or a total number of times photographs have been taken.

[0174] (Appendix 3) The program according to (Supplementary Note 1) or (Supplementary Note 2), wherein in the step of acquiring a parameter value, the parameter value is acquired from a value for an item set by a user or an administrator.

[0175] (Appendix 4) The program according to any one of (Appendix 1) to (Appendix 3), wherein in the step of acquiring the context, the context is acquired by understanding a place, person, or event contained in the image through image analysis.

[0176] (Appendix 5) A program described in any one of (Appendix 1) to (Appendix 4), wherein in the step of acquiring context, the context is acquired by determining the location, direction, or date and time the image was taken from meta information related to the image.

[0177] (Appendix 6) A program according to any one of (Appendix 1) to (Appendix 5), wherein in the step of determining an object, the object is determined from a character, person, object, or letter corresponding to the acquired context.

[0178] (Appendix 7) The program according to any one of (Supplementary Note 1) to (Supplementary Note 6), wherein in the step of determining an object, the content of the object is determined based on the magnitude of the acquired parameter value.

[0179] (Appendix 8) The program according to any one of (Supplementary Note 1) to (Supplementary Note 7), wherein in the step of determining an object, the object includes information useful to a user.

[0180] (Appendix 9) The program according to any one of (Supplementary Note 1) to (Supplementary Note 8), wherein in the step of acquiring a context, images among the registered images that satisfy predetermined requirements are targeted for analysis.

[0181] (Appendix 10) A program described in any of (Appendix 1) to (Appendix 9), wherein in the step of receiving a request, a request for a composite image using an original image into which an object has been composited is received, and in the step of obtaining parameter values, parameter values ​​that have been updated in accordance with user activity since the previous synthesis are obtained.

[0182] (Appendix 11) A program according to any one of (Appendix 1) to (Appendix 10), which causes a processor to execute a step of distinguishing an image related to an image into which an object has been synthesized from other images and presenting the image to a user.

[0183] (Appendix 12) The program according to any one of (Supplementary Note 1) to (Supplementary Note 11), which causes a processor to execute a step of sharing the image into which the object has been combined with other users.

[0184] (Appendix 13) A method executed by a computer having a processor and a memory, wherein the processor performs the steps of: registering a specified image; accepting a request for a composite image; analyzing the registered image in response to the request to obtain a context for the image; obtaining parameter values ​​associated with the registered image from parameter values ​​that are updated in response to user activity; determining an object from the obtained context and the obtained parameter values; and synthesizing the determined object with the registered image and presenting it to the user.

[0185] (Appendix 14) An information processing device comprising a control unit and a memory unit, wherein the control unit executes the steps of registering a specified image, accepting a request for a composite image, acquiring the context of the image by analyzing the registered image in response to the request, acquiring parameter values ​​associated with the registered image from among parameter values ​​that are updated in accordance with user activity, determining an object from the acquired context and the acquired parameter values, and synthesizing the determined object with the registered image and presenting it to the user.

[0186] (Appendix 15) A system comprising: means for registering a specified image; means for accepting a request for a composite image; means for acquiring the context of the image by analyzing the registered image in response to the request; means for acquiring parameter values ​​associated with the registered image from among parameter values ​​updated in response to user activity; means for determining an object from the acquired context and the acquired parameter values; and means for synthesizing the determined object with the registered image and presenting it to the user.

[0187] (Appendix 16) A program to be executed by a computer having a processor and a memory, the program causing the processor to execute the steps of: registering a specified image; associating the registered image with parameter values; determining a first object based on the parameter values; presenting to a user a first composite image in which the determined first object is combined with the image; updating the parameter values ​​when a predetermined condition is met; determining a second object based on the updated parameter values; and presenting to a user a second composite image in which the determined second object is combined with the image.

[0188] (Appendix 17) A method executed by a computer having a processor and a memory, wherein the processor executes the steps of registering a specified image, associating the registered image with parameter values, determining a first object based on the parameter values, presenting to a user a first composite image in which the determined first object is combined with the image, updating the parameter values ​​when a predetermined condition is met, determining a second object based on the updated parameter values, and presenting to a user a second composite image in which the determined second object is combined with the image.

[0189] (Appendix 18) An information processing device comprising a control unit and a storage unit, wherein the control unit executes the steps of registering a specified image, associating the registered image with parameter values, determining a first object based on the parameter values, presenting to a user a first composite image in which the determined first object is combined with the image, updating the parameter values ​​when a predetermined condition is met, determining a second object based on the updated parameter values, and presenting to a user a second composite image in which the determined second object is combined with the image.

[0190] (Appendix 19) A system comprising: means for registering a designated image; means for associating the registered image with a parameter value; means for determining a first object based on the parameter value; means for presenting to a user a first composite image in which the determined first object is combined with the image; means for updating the parameter value when a predetermined condition is met; means for determining a second object based on the updated parameter value; and means for presenting to a user a second composite image in which the determined second object is combined with the image. [Explanation of symbols]

[0191] 1. System 10...Terminal device 12...Communication IF 120…Communications Department 13...Input device 131...Touch-sensitive devices 14...Output device 141...Display 15...Memory 150...Location information sensor 16…Storage 161...Camera 17...Audio processing unit 171...Mike 172...Speaker 180...Storage section 181...User information 182...Image information 19...Processor 190...Control unit 191...Operation reception section 192...Transmitter / receiver 193...Image analysis unit 194...Display control unit 195...Memory control unit 20...Server 201…Communications Department 202...Storage section 203...Control unit 2041...Receiver control module 2042...Transmission control module 2043...Registration module 2044...Management module 2045…Analysis module 2046…Acquisition module 2047…Decision Module 2048…presentation module 22...Communication IF 23...Input / output interface 25…Memory 26…Storage 29...Processor 281...Image information DB 282...Parameter DB 283...Object information DB 80…Network

Claims

1. A program to be executed by a computer having a processor and a memory, the program causing the processor to: registering the designated image; receiving a request for a composite image; acquiring information about a character included in a registered image by analyzing the image; updating the parameter values ​​in response to user activity occurring after registration of the image; determining an object from the character information and the updated parameter values; a step of combining the determined object with the registered image and presenting the result to the user; A program that executes the following.

2. 2. The program according to claim 1, wherein in the step of updating the parameter value, the parameter value is a value relating to the time elapsed since photography or image registration, a number of experiences relating to a location, a value relating to movement, or a total number of photography times.

3. 3. The program according to claim 1, wherein in the step of updating the parameter value, the parameter value is a value for an item set by a user or an administrator.

4. 4. The program according to claim 1, wherein in the step of determining the object, the content of the object is determined based on the magnitude of the updated parameter value.

5. 5. The program according to claim 1, wherein in the step of determining the object, the object includes information useful to a user.

6. 6. The program according to claim 1, wherein, in the step of acquiring character information, images that satisfy predetermined requirements among the registered images are targeted for the analysis.

7. In the step of receiving the request, a request for a composite image using the original image on which the object is composited is received; 7. The program according to claim 1, wherein the parameter value updating step updates the parameter value in accordance with user activity after the previous synthesis.

8. 8. The program according to claim 1, further comprising: making the processor execute a step of distinguishing an image related to the image into which the object has been combined from other images and presenting the image to the user.

9. 9. The program according to claim 1, further comprising causing the processor to execute a step of sharing the image in which the object is combined with other users.

10. 1. A computer-implemented method comprising a processor and a memory, the processor: registering the designated image; receiving a request for a composite image; acquiring information about a character included in a registered image by analyzing the image; updating the parameter values ​​in response to user activity occurring after registration of the image; determining an object from the character information and the updated parameter values; a step of combining the determined object with the registered image and presenting the result to the user; How to do it.

11. An information processing device including a control unit and a storage unit, wherein the control unit: registering the designated image; receiving a request for a composite image; acquiring information about a character included in a registered image by analyzing the image; updating the parameter values ​​in response to user activity occurring after registration of the image; determining an object from the character information and the updated parameter values; a step of combining the determined object with the registered image and presenting the result to the user; An information processing device that executes the above.

12. A means for registering a designated image; means for receiving a request for a composite image; means for analyzing a registered image to acquire information about a character included in the image; means for updating the parameter values ​​in response to user activity occurring after registration of the image; means for determining an object from the character information and the updated parameter values; means for combining the determined object with the registered image and presenting the combined image to the user; A system comprising:

Images