Image processing system, image processing method, and program

Abstract

To provide an image processing system, an image processing method, and a program which are capable of providing recommendation information matching a disposable income estimated on the basis of an analysis result of an image group.SOLUTION: An image processing device recognizes an object of an analysis object image of a user (44), converts information of the object to disposable income zone information of the user (50), acquires incidental information containing photographed date and time of the analysis object image and calculates an appearance frequency of the object on the basis of the information of the object or the incidental information (52), and estimates the disposable income of the user by using the disposable income zone information and the appearance frequency of the object (58).SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing system, an image processing method, and a program. [Background technology]

[0002] As a photo management service, a form using cloud computing has been proposed. Non-Patent Document 1 discloses a cloud computing system that manages still images, moving images, and the like uploaded from user terminals such as smartphones.

[0003] The system described in this document automatically classifies uploaded images according to the date and time of capture and the location of capture. The system also analyzes the subject of the image, automatically classifies the image according to the subject analysis content, and provides recommendation information according to the user's preferences, etc. based on the image analysis results.

[0004] Patent Document 1 describes a service providing system that includes a meeting device that meets with a user. The meeting device described in the document includes an imaging device, and analyzes image data captured by the imaging device to detect people, vehicles, and belongings of people within the imaging field of view.

[0005] The system also recognizes the detected people individually, acquires characteristic information such as the attributes and type of the people, and provides recommendation information according to the characteristic information.The system also estimates income information from information on the car used and changes the products it recommends.

[0006] Patent Document 2 describes a device that estimates asset information of a user based on the user's behavior history on a network, such as the amount of image data posted by the user on a SNS, and provides services tailored to the asset information of the user. Note that SNS is an abbreviation for Social Networking Service.

[0007] The device inputs the user's behavior history and generates a model that outputs an index value indicating which user has more assets. As an example of the model, the document gives an example of a model in which an explanatory variable x corresponding to feature information such as a search query transmission history is specified, a weighting coefficient ω is specified for the explanatory variable, and x×ω, which is obtained by multiplying the explanatory variable x by the weighting coefficient ω, is combined for multiple pieces of feature information. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Patent No. 6305483 [Patent Document 2] Patent No. 6494576 [Non-patent literature]

[0009] [Non-Patent Document 1] Google Photos, [online], [Searched June 7, 2019], Internet〈URL: https: / / www.google.com / photos / about / 〉 Summary of the Invention [Problem to be solved by the invention]

[0010] However, the system described in Patent Document 1 detects a subject from an image captured by an imaging device provided in the meeting device, and acquires feature information of the subject. In this case, a vehicle or the like that happens to be captured may be detected as a subject, and appropriate recommendation information may not be provided to the appropriate user.

[0011] Patent Document 2 does not specifically disclose the weighting coefficients that are applied when generating a model representing a user's assets.

[0012] The present invention has been made in consideration of the above circumstances, and aims to provide an image processing system, an image processing method, and a program that enable the provision of recommendation information according to disposable income estimated based on the results of analyzing an image group containing multiple images. [Means for solving the problem]

[0013] In order to achieve the above object, the following aspects of the invention are provided.

[0014] The image processing system according to the first embodiment includes an object recognition unit that analyzes two or more analysis target images included in an image group including a plurality of images related to a user, and recognizes objects included in each of the two or more analysis target images; a disposable income band conversion unit that converts information of the object recognized using the object recognition unit into disposable income band information that represents the range of the user's disposable income; an additional information acquisition unit that acquires additional information of the analysis target images including imaging date information that represents the imaging date of the analysis target images for the two or more analysis target images in which objects are recognized using the object recognition unit; a frequency derivation unit that derives the appearance frequency of the object based on the imaging date information; a coefficient derivation unit that derives a weighting coefficient corresponding to the object based on the appearance frequency information that represents the appearance frequency; a disposable income estimation unit that estimates the disposable income of the user using the disposable income band information and the weighting coefficient; and a recommendation information transmission unit that transmits recommendation information related to the object to the user according to the user's disposable income.

[0015] According to the first aspect, an image group including a plurality of images is analyzed, information on objects recognized from the images is converted into disposable income band information of a user, and a weighting coefficient derived based on information on the frequency of appearance of the objects is applied to the disposable income band information to estimate the disposable income of the user. This makes it possible to provide recommendation information according to the disposable income estimated based on the analysis result of the image group including a plurality of images.

[0016] The objects may include items and / or events that the imaged scene represents.

[0017] The term image may include the concept of image data, which is a signal representing an image.

[0018] The disposable income may be any estimate of an index representing the user's purchasing power, and is not limited to the take-home pay calculated by deducting taxes and the like from the user's income.

[0019] In a second aspect, in the image processing system of the first aspect, the object recognition section may be configured to recognize an article included in the image to be analyzed as the object.

[0020] According to the second aspect, it is possible to estimate the disposable income of a user based on an item included in an image to be analyzed.

[0021] A third aspect may be configured such that, in the image processing system of the second aspect, a possession determination unit is provided which determines whether an object is a possession or a rental item based on the frequency with which the object appears, and the recommendation information sending unit sends information on a purchased item as recommendation information if the object is a possession of the user, and sends information on a rental item as recommendation information if the object is a rental item.

[0022] According to the third aspect, it is possible to provide recommendation information according to whether the object is a user's possession or whether the object is a rental item.

[0023] A fourth aspect may be configured such that the image processing system according to any one of the first to third aspects further includes a type determination unit that determines a type of an object recognized using the object recognition unit.

[0024] According to the fourth aspect, the type of the object may be used to convert information about the object into disposable income band information of the user.

[0025] A fifth aspect may be configured such that the image processing system of the fourth aspect further comprises a type storage section that stores a relationship between an object and an object type, and the type determination section determines the type of the object by referring to the type storage section.

[0026] According to the fifth aspect, a certain level of accuracy can be ensured in determining the type of object.

[0027] In a sixth aspect, in the image processing system of the first aspect, the object recognition section may be configured to recognize, as the object, a captured scene of the image to be analyzed.

[0028] According to the sixth aspect, the disposable income of the user can be estimated based on the imaging scene of the image to be analyzed.

[0029] In a seventh aspect, in the image processing system according to the sixth aspect, the object recognition section may be configured to determine an event corresponding to the captured scene as the type of object.

[0030] According to the seventh aspect, the disposable income of the user can be estimated based on the event corresponding to the imaging scene of the image to be analyzed.

[0031] Examples of events include traveling, visiting a theme park, dining out, and watching a sporting event.

[0032] According to an eighth aspect, in the image processing system of the sixth or seventh aspect, a type determination unit that determines a type of an object recognized by the object recognition unit may be further provided.

[0033] According to the eighth aspect, the type of the object may be used to convert information about the object into disposable income range information of the user.

[0034] As a ninth aspect, in the image processing system of the eighth aspect, a type storage unit that stores a relationship between an object and an object type may be provided, and the type determination unit may determine the type of the object by referring to the type storage unit.

[0035] According to the ninth aspect, a certain level of accuracy can be ensured in determining the type of object.

[0036] A tenth aspect may be configured such that, in the image processing system of the eighth or ninth aspect, an imaging location information acquisition unit is provided that acquires information on the imaging location of the image to be analyzed, and the type determination unit determines an event corresponding to an object based on the positional relationship between a reference location of a user corresponding to the image group and the imaging location of the image to be analyzed.

[0037] According to the tenth aspect, an event can be determined based on the positional relationship between the user's reference location and the capture location of the analysis target image.

[0038] In an eleventh aspect, in the image processing system according to the tenth aspect, the type determination section may determine the type of event corresponding to the object, based on the distance from the reference location to the imaging location of the analysis target image.

[0039] According to the eleventh aspect, an event can be determined based on the reference location of the user and the imaging location of the analysis target image and the distance therebetween.

[0040] A twelfth aspect may be configured such that, in the image processing system of the tenth or eleventh aspect, the imaging location information acquisition unit uses an additional information acquisition unit to acquire additional information including imaging location information indicating the imaging location of the image to be analyzed.

[0041] According to the twelfth aspect, the location where the image was captured can be identified based on the information on the location where the image was captured that is included in the supplementary information.

[0042] A thirteenth aspect may be configured such that in the image processing system according to the tenth or eleventh aspect, the imaging location information acquisition section analyzes the analysis target image to identify the imaging location of the analysis target image.

[0043] According to the thirteenth aspect, the imaging location of the image can be identified based on the analysis result of the image to be analyzed.

[0044] A fourteenth aspect may be configured such that in the image processing system according to any one of the tenth to thirteenth aspects, the auxiliary information acquisition unit acquires auxiliary information including information on the reference location.

[0045] According to the fourteenth aspect, the reference location can be identified based on the information on the reference location included in the supplementary information.

[0046] A fifteenth aspect may be configured such that the image processing system according to any one of the tenth to thirteenth aspects further comprises a reference location identifying section that analyzes an analysis target image and identifies the reference location.

[0047] According to the fifteenth aspect, the reference location can be specified based on the analysis result of the analysis target image.

[0048] A sixteenth aspect may be configured such that the image processing system according to any one of the first to fifteenth aspects further comprises a price range determination section that determines a price range of an object corresponding to the type of the object.

[0049] According to the sixteenth aspect, the price range of the object may be converted into the user's disposable income range.

[0050] A 17th aspect may be configured such that, in the image processing system of the 16th aspect, a price range memory unit is provided that stores the relationship between the type of object and the price range, and the price range determination unit determines the price range of the object by referring to the price range memory unit.

[0051] According to the seventeenth aspect, a certain level of accuracy can be ensured in determining the price range of an object.

[0052] An 18th aspect may be configured such that, in the image processing system of the 17th aspect, a disposable income band memory unit is provided that stores a relationship between a price range and a disposable income band representing a range of a user's disposable income, and the disposable income band conversion unit refers to the disposable income band memory unit and converts price range information representing the price range into disposable income band information representing the user's disposable income band.

[0053] According to the eighteenth aspect, a certain degree of accuracy can be ensured in converting the price range of an object into the disposable income range of a user.

[0054] A 19th aspect may be configured such that, in an image processing system of any one of the 1st to 18th aspects, the system further includes a coefficient memory unit that stores a relationship between an object type and a weighting coefficient, and the coefficient derivation unit derives a weighting coefficient to be applied to the object by referring to the coefficient memory unit.

[0055] According to the nineteenth aspect, a certain degree of accuracy can be ensured in deriving the weighting coefficients.

[0056] A twentieth aspect may be configured such that the image processing system according to any one of the first to nineteenth aspects further comprises a user information acquisition unit that acquires user information that identifies a user.

[0057] According to the twentieth aspect, the user of the image to be analyzed can be identified.

[0058] The image processing method according to the 21st aspect is an image processing method including an object recognition step of analyzing two or more analysis target images included in an image group including a plurality of images related to a user, and recognizing an object included in each of the two or more analysis target images; a disposable income band conversion step of converting information of the object recognized in the object recognition step into disposable income band information representing the range of the user's disposable income; an additional information acquisition step of acquiring additional information of the analysis target images including imaging date information representing the imaging date of the analysis target images for the two or more analysis target images in which the object is recognized in the object recognition step; a frequency derivation step of deriving an appearance frequency of the object based on the imaging date information; a coefficient derivation step of deriving a weighting coefficient corresponding to the object based on the appearance frequency information representing the appearance frequency; a disposable income estimation step of estimating the disposable income of the user using the disposable income band information and the weighting coefficient; and a recommendation information transmission step of transmitting recommendation information related to the object to the user according to the user's disposable income.

[0059] According to the twenty-first aspect, it is possible to obtain the same effects as those of the first aspect.

[0060] In the 21st aspect, it is possible to appropriately combine the same features as those specified in the 2nd to 20th aspects. In this case, the components performing the processes or functions specified in the image processing system can be understood as the components of the image processing method performing the corresponding processes or functions.

[0061] The program of the 22nd aspect is a program that causes a computer to realize an object recognition function that analyzes two or more analysis target images included in an image group including a plurality of images related to a user, and recognizes objects included in each of the two or more analysis target images, a disposable income band conversion function that converts information of an object recognized using the object recognition function into disposable income band information that represents the range of the user's disposable income, an additional information acquisition function that acquires additional information of the analysis target image including imaging date information that represents the imaging date of the analysis target image for two or more analysis target images in which an object is recognized using the object recognition function, a frequency derivation function that derives the appearance frequency of the object based on the imaging date information, a coefficient derivation function that derives a weighting coefficient corresponding to the object based on the appearance frequency information that represents the appearance frequency, a disposable income estimation function that estimates the disposable income of the user using the disposable income band information and the weighting coefficient, and a recommendation information transmission function that transmits recommendation information related to the object to the user according to the user's disposable income.

[0062] According to the twenty-second aspect, it is possible to obtain the same effects as those of the first aspect.

[0063] In the 22nd aspect, the same features as those specified in the 2nd to 20th aspects can be appropriately combined. In this case, components that perform the processes or functions specified in the image processing system can be understood as components of a program that performs the corresponding processes or functions. Effect of the Invention

[0064] According to the present invention, an image group including a plurality of images is analyzed, information on objects recognized from the images is converted into disposable income band information of a user, and a weighting coefficient derived based on information on the frequency of appearance of the objects is applied to the disposable income band information to estimate the disposable income of the user. This makes it possible to provide recommendation information according to the disposable income estimated based on the analysis result of the image group including a plurality of images. [Brief description of the drawings]

[0065] [Figure 1] FIG. 1 is a diagram showing the overall configuration of an image processing system according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram of the server device according to the first embodiment. [Diagram 3] FIG. 3 is an explanatory diagram of image analysis. [Figure 4] FIG. 4 is a flowchart showing the procedure of the recommendation information providing method according to the first embodiment. [Diagram 5] FIG. 5 is a flow chart showing the procedure of the image content analysis process shown in FIG. [Figure 6] FIG. 6 is a schematic diagram showing an example of a price range determination table. [Figure 7] FIG. 7 is a schematic diagram showing an example of a disposable income band table. [Figure 8] FIG. 8 is a flowchart showing the procedure of the auxiliary information analysis step shown in FIG. [Figure 9] FIG. 9 is a schematic diagram showing an example of the coefficient table. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a recommendation information display screen. [Figure 11] FIG. 11 is a functional block diagram of a server device according to the second embodiment. [Figure 12] FIG. 12 is a flowchart showing the procedure of a recommendation information providing method according to the second embodiment. [Figure 13] FIG. 13 is a flowchart showing the procedure of the auxiliary information analysis step shown in FIG. [Figure 14] FIG. 14 is an explanatory diagram showing an example of a rental item recommendation information display screen. [Figure 15] FIG. 15 is a functional block diagram showing an example of the configuration of a type determination unit applied to an image processing system according to the third embodiment. [Figure 16] FIG. 16 is a schematic diagram of a price range determination table applied to the image processing system according to the third embodiment. [Figure 17] FIG. 17 is a schematic diagram of a disposable income band conversion table applied to the image processing system according to the third embodiment. [Figure 18] FIG. 18 is a schematic diagram of a coefficient table applied to the image processing system according to the third embodiment. [Figure 19] FIG. 19 is a functional block diagram of a type determination unit showing another mode of acquiring user's home location information. [Figure 20] FIG. 20 is a flowchart showing the procedure of an object type determination step applied to the recommendation information providing method according to the third embodiment. [Figure 21] FIG. 21 is a flowchart showing the procedure of an object type determination step applied to the recommendation information providing method according to the third embodiment. [Figure 22] FIG. 22 is a correlation diagram showing the correlation between users and related parties according to age group. [Figure 23] FIG. 23 is a correlation diagram showing the correlation between users and related parties according to hobbies. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0066] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification, the same components are given the same reference numerals and duplicated explanations will be omitted as appropriate.

[0067] [Image processing system configuration example] [Overall structure] Fig. 1 is a diagram showing the overall configuration of an image processing system according to an embodiment. The image processing system 10 shown in the figure includes a server device 12. The server device 12 reads out a plurality of images related to a user stored in an image database 14, analyzes the read out images, and provides recommendation information to the user based on the image analysis results. A user terminal device 20 uses a display unit 22 to display the recommendation information.

[0068] It should be noted that the term "image" in this specification may include the concept of image data, which is a signal representing an image.

[0069] The image processing system 10 includes a server device 12, an image database 14, a disposable income database 16, a recommendation information database 18, and an administrator terminal device 19. The server device 12 is connected to the image database 14, the disposable income database 16, the recommendation information database 18, and the administrator terminal device 19 so as to be capable of data communication. The data communication may be performed via a specified network. The data communication may be performed in either a wireless communication form or a wired communication form.

[0070] The image database 14 is configured to be able to search for images related to a user using the identification information of the user as a search key. The identification information of the user may be a user name, a user ID, or the like. Note that ID is an abbreviation of identification.

[0071] An image related to a user may include an image to which a user name or the like is added as incidental information of the imager, an image tagged with a user name, etc. In other words, an image related to a user may include an image captured using an imaging device owned by the user, and an image including the user as a subject.

[0072] The disposable income database 16 includes a price range determination database in which a price range determination table expressing the relationship between the type of object and the price range is stored, and a disposable income range conversion database in which a disposable income range conversion table expressing the conversion relationship between the price range and the disposable income range is stored.

[0073] The disposable income database 16 includes a weighting coefficient database in which a weighting coefficient table is stored that indicates the relationship between the appearance frequency of an object extracted from an image and the weighting coefficient.

[0074] The price range determination database is shown in Fig. 2 with reference numeral 16A, the disposable income range conversion database is shown in Fig. 2 with reference numeral 16B, and the weighting coefficient database is shown in Fig. 2 with reference numeral 16C.

[0075] The price range determination database in which the price range determination table is stored corresponds to an example of a price range storage unit in which the relationship between the type of object and the price range is stored. The disposable income range conversion database in which the disposable income range conversion table showing the conversion relationship between the price range and the disposable income range corresponds to an example of a disposable income range storage unit.

[0076] The weighting coefficient database, which stores a weighting coefficient table indicating the relationship between the appearance frequency of an object and the weighting coefficient, corresponds to an example of a coefficient storage unit that stores the relationship between the type of object and the weighting coefficient.

[0077] The recommendation information database 18 stores a recommendation information table that indicates the relationship between disposable income and recommended products, etc. The server device 12 refers to the recommendation information database 18 using the disposable income information, and transmits recommendation information based on the user's disposable income to the user terminal device 20.

[0078] The manager terminal device 19 controls the updating of tables included in the disposable income database 16, etc. The updating of tables included in the disposable income database 16, etc. may be automatically performed by the server device 12.

[0079] In this specification, the term "disposable income" refers to any estimated index representing a user's purchasing power, and is not limited to the take-home pay calculated by deducting taxes and the like from the user's income.

[0080] The user terminal device 20 may be a smart device used by each user, such as a smartphone or a tablet. The user terminal device 20 includes a communication unit. The user terminal device 20 may perform data communication with the server device 12 via the communication unit. The communication unit is not shown in the figure.

[0081] The user terminal device 20 includes a display unit 22. The user terminal device 20 receives recommendation information transmitted from the server device 12. The user terminal device 20 can use the display unit 22 to display the recommendation information.

[0082] The user terminal device 20 may include an imaging unit. The user terminal device 20 may include a storage unit that temporarily stores images captured using the imaging unit. The user terminal device 20 may upload images to the image database 14 via the server device 12. The imaging unit and the storage unit are not shown in the drawings.

[0083] The server device 12 is connected to the Internet 30 so as to be able to perform data communications with the Internet 30. The server device 12 receives, via the Internet 30, information on products and services that are applied to recommendation information from providers of products and the like.

[0084] [Hardware configuration of image processing system] The server device 12 and the like applied to the image processing system 10 may be a computer. The computer may apply the following hardware and execute a specified program to realize the functions of the image processing system 10. Note that the program is synonymous with software.

[0085] The server device 12 and the like may employ various processors as a signal processing unit that performs signal processing. Examples of processors include a CPU and a GPU (Graphics Processing Unit). A CPU is a general-purpose processor that executes a program and functions as a signal processing unit. A GPU is a processor specialized for image processing. An electric circuit that combines electric circuit elements such as semiconductor elements is applied to the hardware of the processor. Each control unit is equipped with a ROM in which programs and the like are stored and a RAM that is a working area for various calculations.

[0086] Two or more processors may be applied to one signal processing unit. The two or more processors may be the same type of processor or different types of processors. Also, one processor may be applied to multiple signal processing units.

[0087] [Example of configuration of server device according to the first embodiment] Fig. 2 is a functional block diagram of the server device according to the first embodiment. Note that DB in Fig. 2 is an abbreviation for Database. Also, two image databases 14 are shown in Fig. 2, but these are the same component.

[0088] The server device 12 includes an image acquisition unit 40 and an additional information acquisition unit 42. The server device 12 includes an object recognition unit 44, a type determination unit 46, a price range determination unit 48, and a disposable income range conversion unit 50. The server device 12 includes a frequency derivation unit 52 and a coefficient derivation unit 54. The server device 12 includes a disposable income estimation unit 56 and a recommendation information transmission unit 58. The server device 12 includes a user information acquisition unit 59.

[0089] The image acquisition unit 40 acquires an image to be analyzed from the image database 14. The server device 12 may search the image database 14 using the user's identification information as a search key to acquire an image related to the user as an image to be analyzed. The image database 14 transmits the acquired image to the object recognition unit 44.

[0090] The incidental information acquisition unit 42 acquires incidental information of the image acquired from the image database 14. The incidental information includes information on the image capture date. The incidental information may include GPS information including information on the image capture location. Note that GPS is an abbreviation for Global Positioning System. The incidental information acquisition unit 42 transmits the incidental information to the frequency derivation unit 52.

[0091] The object recognition unit 44 performs an object recognition process to recognize an object from an image acquired by the image acquisition unit 40. The object recognition unit 44 can recognize multiple objects from one image. A known method can be applied for the object recognition process. Here, a detailed description of the object recognition process is omitted.

[0092] The object may be an object included in the image, the background of the image, the scene of the image, etc. The object may be a non-person such as food, a vehicle, a person's clothes, and a person's accessories, etc. The background of the image may be a building or a landscape at a travel destination, a theme park, etc.

[0093] When an object is specified in advance and object information is assigned as tag information or the like, the object recognition unit 44 may use the tag information or the like as a recognition result of the object. That is, the object recognition unit 44 may include a tag information acquisition unit that acquires the tag information.

[0094] When an object is not recognized from an image, the object recognition unit 44 may output a recognition result indicating that an object is not present. The server device 12 may not perform subsequent processing of object recognition for an image in which an object is not recognized.

[0095] The type determination unit 46 determines the type of an object for an object recognized by the object recognition unit 44. When determining the type of an object, the type determination unit 46 may refer to a table showing the correspondence between objects and object types. The type determination unit 46 transmits object type information for each object to the price range determination unit 48.

[0096] In this embodiment, a type determination database in which a table showing the correspondence between objects and object types is stored is not shown in the drawings. The type determination database corresponds to an example of a type storage unit.

[0097] The price range determination unit 48 determines the price range of an object based on the type of the object. The price range determination unit 48 refers to the price range determination database 16A when determining the price range of an object. The price range determination unit 48 transmits the price range information for each object to the disposable income range conversion unit 50.

[0098] The disposable income band conversion unit 50 converts the price range information for each object into disposable income band information that indicates the disposable income band of the user. The disposable income band conversion unit 50 refers to the disposable income band conversion database 16B when converting the price range information for each object into disposable income band information. The disposable income band conversion unit 50 transmits the disposable income band information to the disposable income estimation unit 56.

[0099] The frequency derivation unit 52 derives the appearance frequency of objects included in the multiple images using the image capture date information included in the supplementary information for each image. The frequency derivation unit 52 transmits appearance frequency information representing the appearance frequency of each object to the coefficient derivation unit 54.

[0100] The coefficient derivation unit 54 derives a weighting coefficient for each object based on the appearance frequency information for each object. When deriving the weighting coefficient for each object, the coefficient derivation unit 54 refers to the weighting coefficient database 16C. The coefficient derivation unit 54 transmits the weighting coefficient for each object to the disposable income estimation unit 56.

[0101] The disposable income estimation unit 56 estimates the disposable income of the user by using the disposable income band information indicating the disposable income band for each object and the weighting coefficient for each object. The disposable income estimation unit 56 transmits the disposable income estimation information indicating the estimation result of the disposable income of the user to the recommendation information transmission unit 58.

[0102] The recommendation information transmission unit 58 sets recommendation information for the user based on the user's disposable income estimation information. The recommendation information transmission unit 58 refers to the recommendation information database 18 when setting the recommendation information. The recommendation information transmission unit 58 transmits the recommendation information to the user terminal device 20.

[0103] The recommendation information transmitting unit 58 provides recommendation information of products etc. corresponding to the object. For example, when the object is a mandarin orange, the recommendation information transmitting unit 58 can provide the user with recommendation information of fruits, processed fruit products, sweets using fruit as an ingredient, etc.

[0104] The user information acquisition unit 59 acquires user information that identifies a user. The user information acquisition unit 59 can acquire the user information transmitted from the user terminal device 20 shown in Fig. 1. The user information acquisition unit 59 transmits the user information to the object recognition unit 44.

[0105] [Handling of personal information, etc.] Personal information and the like is handled in the image processing system 10 shown in FIGS. 1 and 2 as follows.

[0106] The administrator of the image processing system 10 obtains consent from the user for the analysis of the user's image. The administrator here may include an organization such as a company. An example of consent is when a check mark is placed in a checkbox representing consent on a consent screen displayed on the display unit 22 of the user terminal device 20, and information representing consent is transmitted from the user terminal device 20 to the server device 12. The same applies to the following consents.

[0107] The administrator of the image processing system 10 obtains the user's consent to providing the user with recommendation information based on the analysis results of the user's images.

[0108] The entity that provides the recommendation information to the user may be an administrator of the image processing system 10. The entity that provides the recommendation information may be a provider of the product or the like included in the recommendation information. The provider here may include an organization such as a company.

[0109] When a provider of a product or the like provides recommendation information, the provider must obtain the user's consent to the delivery of necessary information for the provision of recommendation information from the administrator of the image processing system 10. The necessary information is the minimum information necessary for providing the recommendation information, such as an email address.

[0110] When the administrator of the image processing system 10 provides information to affiliated organizations, etc., the provision of user information such as user names and information that can lead to the identification of users is prohibited in principle. When performing image analysis of multiple users, the above measures are applied to all users. The user's consent will be obtained before providing the information after anonymizing it in advance.

[0111] [Explanation of image analysis] Fig. 3 is an explanatory diagram of image analysis. Image analysis is processing of an image performed by the server device 12 shown in Fig. 2. The server device 12 can perform analysis of supplementary information and analysis of image contents as image analysis.

[0112] The server device 12 may apply image capture date information as the additional information to be analyzed. The server device 12 analyzes the image capture date information for a plurality of images included in the image group to derive the appearance frequency of an object common to the plurality of images. The server device 12 may derive the appearance frequency of a plurality of objects.

[0113] The server device 12 may apply GPS information as the additional information of the analysis target. The server device 12 may analyze the GPS information to identify the location where the image was captured. The server device 12 may analyze information about the imaging device used to capture the image as the additional information of the analysis target. The server device 12 may analyze information about a terminal device or the like equipped with the imaging device.

[0114] The server device 12 can analyze the content of the image to identify the subject. Examples of the subject include food such as food, clothing such as clothes, and accessories such as watches. Based on the subject, the server device 12 recognizes the item that is the subject as an object.

[0115] The server device 12 may analyze the image content to identify the captured scene. Examples of the captured scene include visiting a tourist spot, visiting a theme park, eating at a restaurant, watching a sporting event, etc. The server device 12 may recognize an event represented by the captured scene as an object.

[0116] The server device 12 may apply a memo image such as a screenshot as the image. The server device 12 may analyze the screenshot and identify, as objects, items purchased by the user and services used by the user based on the analysis result of the screenshot.

[0117] [Recommendation information providing method according to the first embodiment] [Overall flow of recommendation information provision method] 4 is a flowchart showing the procedure of the method for providing recommended information according to the first embodiment. The method for providing recommended information according to the first embodiment described below acquires a plurality of images related to an arbitrary user, estimates the disposable income of the user based on the analysis results of the plurality of analysis target images, and provides the user with recommended information according to the disposable income of the user.

[0118] In the image acquisition step S10, the image acquisition unit 40 shown in Fig. 2 acquires an image to be analyzed from the image database 14. The image acquisition step S10 may include an image storage step for storing the acquired image to be analyzed. After the image acquisition step S10, the process proceeds to an image content analysis step S12.

[0119] The image acquisition step S10 may include a search step for searching for an image related to a user. The image acquisition step may also include a login step for a user to log in. The search step or the login step may be performed before the image acquisition step S10 is performed.

[0120] In the image content analysis step S12, the object recognition unit 44 performs a process of recognizing an object from the image. That is, in the image content analysis step S12, the object recognition unit 44 analyzes the pixels that make up the image to be analyzed. Also, in the image content analysis step S12, the type determination unit 46 determines the type of the object.

[0121] In the image content analysis step S12, the price range determination unit 48 determines the price range of the object. In the image content analysis step S12, the disposable income range conversion unit 50 converts the price range of the object into a disposable income range.

[0122] The image content analysis step S12 may include a storage step for storing each piece of information. Details of the image content analysis step S12 will be described later. After the image content analysis step S12, the process proceeds to an additional information analysis step S14.

[0123] In the incidental information analysis step S14, the incidental information acquisition unit 42 acquires image capture date information as incidental information for each image. In the incidental information analysis step S14, the frequency derivation unit 52 derives the appearance frequency of each object. In the incidental information analysis step S14, the coefficient derivation unit 54 derives a weighting coefficient for each object.

[0124] The additional information analysis step S14 may include a storage step for storing each piece of information. The additional information analysis step S14 will be described in detail later. After the additional information analysis step S14, the process proceeds to an analysis end determination step S16.

[0125] In the analysis completion determination step S16, the server device 12 determines whether or not analysis has been performed on all images to be analyzed. In the analysis completion determination step S16, if the server device 12 determines that there are images to be analyzed that have not been analyzed, the determination is No.

[0126] If the determination is No, the process returns to the image content analysis step S12, and each step from the image content analysis step S12 to the analysis end determination step S16 is repeatedly performed until the determination is Yes in the analysis end determination step S16.

[0127] On the other hand, if the server device 12 determines that all images to be analyzed have been analyzed in the analysis completion determination step S16, the determination is Yes. If the determination is Yes, the process proceeds to a disposable income estimation step S18.

[0128] In the disposable income estimation step S18, the disposable income estimation unit 56 estimates the user's disposable income based on the image analysis result in the image content analysis step S12 and the incidental information analysis result in the incidental information analysis step S14. The disposable income estimation step S18 may include a storage step for storing the estimated disposable income. After the disposable income estimation step S18, the process proceeds to the recommendation information transmission step S20.

[0129] In the recommendation information transmission step S20, the recommendation information transmission unit 58 sets recommendation information including products and services recommended to the user based on the estimated result of the user's disposable income. In the recommendation information transmission step S20, the recommendation information transmission unit 58 transmits the recommendation information to the user terminal device 20. After the recommendation information transmission step S20, the process proceeds to an end determination step S22.

[0130] In the termination determination step S22, the server device 12 determines whether or not to terminate the recommendation information providing method. In the termination determination step S22, if the server device 12 determines to continue the recommendation information providing method, the determination is No. If the determination is No, the process proceeds to the image acquisition step S10, and the steps from the image acquisition step S10 to the termination determination step S22 are repeatedly performed for the next image group until the determination is Yes in the termination determination step S22.

[0131] On the other hand, in the end determination step S22, if the server device 12 determines that the recommendation information providing method is to be ended, a Yes determination is made. If the Yes determination is made, the server device 12 performs a prescribed end process. An example of an end condition of the recommendation information providing method is the end of providing recommendation information for a preset image group. Note that the recommendation information providing method shown in the embodiment corresponds to an example of an image processing method.

[0132] [Details of the image content analysis process] Fig. 5 is a flow chart showing the procedure of the image content analysis step shown in Fig. 4. In the analysis target image setting step S100, the object recognition unit 44 shown in Fig. 2 specifies a group of images to be analyzed, and extracts an analysis target image from the specified group of images. That is, in the analysis target image setting step S100, the object recognition unit 44 removes images that are not suitable for analysis.

[0133] In addition, in the analysis target image setting step S100, the object recognition unit 44 identifies a group of similar images having similar image content, such as a group of images acquired by applying continuous shooting, extracts one image from the group of similar images, and removes the remaining images. After the analysis target image setting step S100, the process proceeds to the object recognition step S102.

[0134] In the object recognition step S102, the object recognition unit 44 recognizes an object from the image to be analyzed. After the object recognition step S102, the process proceeds to an object type determination step S104.

[0135] In the object type determination step S104, the type determination unit 46 determines the type of the object by referring to the correspondence between the specified object and the object type. For example, if the object is a mandarin orange, the type determination unit 46 determines the type of the object as a fruit. Also, if the object is amber, the type determination unit 46 determines the type of the object as jewelry. After the object type determination step S104, the process proceeds to a price range determination step S106.

[0136] In the price range determination step S106, the type determination unit 46 determines the price range of each object by referring to the price range determination database 16A. Fig. 6 is a schematic diagram showing an example of a price range determination table.

[0137] The price range determination table 100 shown in Fig. 6 defines the price range of objects for each type of object. In Fig. 6, price ranges are indicated using numerical values ​​from 1 to 5. The numerical values ​​from 1 to 5 indicating the price ranges shown in Fig. 6 can be converted to numerical ranges indicating the price ranges. Although Fig. 6 shows one object for each price range, one price range may include multiple objects. After the price range determination step S106, the process proceeds to the undetermined object determination step S108.

[0138] In the undetermined object determination step S108, the server device 12 determines whether or not price range determination has been performed for all objects. If the server device 12 determines in the undetermined object determination step S108 that there is an object whose price range has not been determined, the result is No. If the result is No, the process proceeds to the object type determination step S104, and each step from the object type determination step S104 to the undetermined object determination step S108 is repeatedly performed until the result is Yes in the undetermined object determination step S108.

[0139] On the other hand, if the server device 12 determines that the price range determination has been performed for all objects in the undetermined object determination step S108, the result is Yes. If the result is Yes, the process proceeds to the disposable income range conversion step S110.

[0140] In the disposable income band conversion step S110, the disposable income band conversion unit 50 refers to the disposable income band conversion database 16B and converts the price band determination result of each object into the disposable income band of the user.

[0141] Fig. 7 is a schematic diagram showing an example of a disposable income band table. In the disposable income band conversion table 110 shown in Fig. 7, alphabets A to G are used to indicate disposable income bands corresponding to price bands for each type of object. The alphabets A to G shown in Fig. 7 can be processed as numerical values.

[0142] When different price ranges are determined for multiple objects included in the same image, the disposable income band conversion unit 50 can derive the user's disposable income band based on a specified priority order. After the disposable income band conversion step S110, the process proceeds to an image content analysis completion determination step S112.

[0143] In the image content analysis completion determination step S112, the server device 12 determines whether or not to end the image content analysis. In the image content analysis completion determination step S112, if the server device 12 determines that the processing of all the images to be analyzed has not been completed, the determination is No. If the determination is No, the process proceeds to the object recognition step S102, and each step from the object recognition step S102 to the image content analysis completion determination step S112 is repeatedly performed until the determination is Yes in the image content analysis completion determination step S112.

[0144] On the other hand, if the server device 12 determines that the processing of all the analysis target images has been completed in the image content analysis completion determination step S112, the determination is Yes. If the determination is Yes, the server device 12 ends the image content analysis step S12 shown in FIG.

[0145] [Details of the additional information analysis process] Fig. 8 is a flowchart showing the procedure of the incidental information analysis step shown in Fig. 4. In the imaging date information acquisition step S200, the incidental information acquisition unit 42 shown in Fig. 2 acquires imaging date information as incidental information. After the imaging date information acquisition step S200, the process proceeds to an image specification step S202. Note that the imaging date information acquisition step S200 described in the embodiment corresponds to an example of the incidental information acquisition step.

[0146] In the image specification step S202, the frequency derivation unit 52 specifies an image for which the appearance frequency of an object is to be derived. That is, the frequency derivation unit 52 specifies a plurality of images including the same object for each object. After the image specification step S202, the process proceeds to an appearance frequency derivation step S204.

[0147] In the appearance frequency derivation step S204, the frequency derivation unit 52 derives the appearance frequency of the object. The appearance frequency may be the imaging period. The imaging period is the reciprocal of the imaging period. When two or more imaging periods are derived for three or more images, the frequency derivation unit 52 may set the statistical value of the two or more imaging periods as the imaging period. Examples of the statistical value include the average value, median value, maximum value, and minimum value. After the appearance frequency derivation step S204, the process proceeds to the coefficient derivation step S206.

[0148] In the coefficient derivation step S206, the coefficient derivation unit 54 derives a weighting coefficient for each object with reference to the weighting coefficient database 16C. Fig. 9 is a schematic diagram showing an example of the coefficient table.

[0149] The weighting coefficient table 120 shown in Fig. 9 indicates the relationship between the appearance frequency and the weighting coefficient for each type of object. For example, when the image capture cycle is six months, the weighting coefficient for fruits is 0.1. The weighting coefficient table 120 shown in Fig. 9 prescribes a relatively large weighting coefficient when the appearance frequency is relatively high. After the coefficient derivation step S206 shown in Fig. 8, the process proceeds to a coefficient derivation determination step S208.

[0150] In the coefficient derivation determination step S208, the server device 12 determines whether or not weighting coefficients have been derived for all objects. If the server device 12 determines in the coefficient derivation determination step S208 that there is an object for which a weighting coefficient has not been derived, the determination is No. If the determination is No, the process proceeds to the image specification step S202, and each step from the image specification step S202 to the coefficient derivation determination step S208 is repeatedly performed until the determination is Yes in the coefficient derivation determination step S208.

[0151] On the other hand, if the server device 12 determines in the coefficient derivation determination step S208 that weighting coefficients have been derived for all objects, the determination is Yes. If the determination is Yes, the server device 12 ends the supplementary information analysis step S14 shown in FIG.

[0152] Furthermore, in the present embodiment, a weighting coefficient is set for each object, but a weighting coefficient may be set for each object.

[0153] [Examples of providing recommendation information] 10 is an explanatory diagram showing an example of a recommended information display screen. The recommended information display screen 140 is displayed using the display unit 22 of the user terminal device 20. The recommended information display screen 140 includes user identification information 142, recommended information 144, and an operation button 146. The user can purchase a product included in the recommended information 144 by operating the user terminal device 20.

[0154] When a single analysis target image includes multiple objects, the user's disposable income may be estimated for each object. When multiple different disposable incomes are estimated for a single user, the maximum estimated value of the disposable income may be set as the user's disposable income, and recommendation information may be provided.

[0155] When multiple different disposable incomes are estimated for one user, recommendation information may be provided for each of the multiple estimated disposable incomes. Priorities may be set for object types in advance, and the disposable income of the user may be estimated based on the object with the highest priority.

[0156] [Effects of the First Embodiment] According to the image processing system 10 and the recommendation information providing method according to the first embodiment, it is possible to obtain the following advantageous effects.

[0157] [1] The system recognizes objects from images, identifies the type of object, determines the price range for each type of object, and converts the price range for each type of object into a disposable income range. In addition, for multiple images with a common object, the system derives the frequency of appearance of the object from the shooting date information, and derives a weighting coefficient based on the frequency of appearance of the object. The system multiplies the disposable income range by the weighting coefficient to estimate the user's disposable income. This makes it possible to provide recommendation information according to the disposable income estimated based on the analysis results of multiple images.

[0158] [2] The system recognizes items as objects from an image, which allows the system to estimate a user's disposable income based on the items contained in the image.

[0159] [3] The price range for each object type is determined by referring to a price range determination table that indicates the relationship between the object type and the price range, thereby ensuring a certain level of accuracy in determining the price range.

[0160] [4] The disposable income range is derived from the price range of the object by referring to a disposable income range table that shows the relationship between the price range of the object and the disposable income range, thereby ensuring a certain degree of accuracy in the disposable income range conversion.

[0161] [5] A weighting coefficient table showing the relationship between the appearance frequency of an object and the weighting coefficient is referenced to derive the weighting coefficient from the appearance frequency of the object, thereby ensuring a certain level of accuracy in the weighting coefficient.

[0162] In the present embodiment, a table is referred to in deriving the weighting coefficient, but a deep learning model may be applied to the process of deriving the weighting coefficient. For example, the coefficient derivation unit 54 shown in FIG. 2 may apply a learned model that has learned the conversion relationship between the appearance frequency of an object and the weighting coefficient. The same applies to the type determination unit 46, the price range determination unit 48, and the disposable income range conversion unit 50.

[0163] When the image processing system 10 selects a product, etc. that is relatively more expensive or in a relatively lower price range than the products, etc. included in the recommendation information provided to the user, the image processing system 10 may conduct learning using information about the products, etc. purchased by the user as learning data, and update parameters related to the estimation of disposable income and parameters related to the provision of recommendation information.

[0164] [Image processing system according to the second embodiment] Next, an image processing system according to a second embodiment will be described. The image processing system 200 according to the second embodiment determines whether an object is a user's possession or a rental item, and provides recommendation information according to the determination result.

[0165] [Example of the configuration of the server device] Fig. 11 is a functional block diagram of a server device according to the second embodiment. The server device 212 is obtained by adding a belongings determination unit 60 to the server device 12 shown in Fig. 2. The belongings determination unit 60 uses image capture date information to determine whether an object is a user's belongings or a rental item.

[0166] For example, the belongings determination unit 60 may determine that an object that appears again after a day or more has elapsed since the image capture date is the user's belongings, and may determine that the object is a rental item if the object does not appear again. In other words, the belongings determination unit 60 may determine whether an object is the user's belongings or a rental item according to the image capture period of the object and the frequency with which the object appears.

[0167] The server device 12 may transmit both the recommended information for purchased items and the recommended information for rental items to the user terminal device 20 without determining whether the object is a user's possession or a rental item.

[0168] [Example of a method for providing recommendation information] Fig. 12 is a flowchart showing the procedure of the recommendation information providing method according to the second embodiment. In the flowchart shown in Fig. 12, the additional information analysis step S14 in the flowchart shown in Fig. 4 is changed to an additional information analysis step S14A.

[0169] 12, a belongings determination step S30, a rental item recommendation information acquisition step S32, and a purchase item recommendation information acquisition step S34 are added to the flowchart shown in FIG.

[0170] The image acquisition step S10 and the image content analysis step S12 are similar to the recommendation information providing method according to the first embodiment shown in Fig. 4. In the accessory information analysis step S14A shown in Fig. 12, the belongings determination unit 60 uses information on the shooting date of each image to identify whether each object belongs to the user or is a rental item.

[0171] Fig. 13 is a flowchart showing the procedure of the auxiliary information analysis step shown in Fig. 12. In the flowchart shown in Fig. 13, a belongings identification step S220 is added to the flowchart shown in Fig. 8.

[0172] In the belongings identification step S220, the belongings determination unit 60 identifies for each object whether it is a user's belongings or a rental item. After the belongings identification step S220, the process proceeds to the coefficient derivation determination step S208. When the process of the coefficient derivation determination step S208 is performed and the specified conditions are satisfied, the server device 12 ends the supplementary information analysis step S14A shown in FIG.

[0173] 12, after the additional information analysis step S14A, the process proceeds to the analysis end determination step S16. The analysis end determination step S16 and the disposable income estimation step S18 are similar to the analysis end determination step S16 and the disposable income estimation step S18 shown in FIG.

[0174] After the disposable income estimation step S18, the process proceeds to a possessions determination step S30. In the possessions determination step S30, the possessions determination unit 60 shown in Fig. 11 determines whether the object is a possession of the user or a rental item.

[0175] In the belongings determination step S30, if the belongings determination unit 60 determines that the object is a rental item, the determination is No. If the determination is No, the process proceeds to a rental item recommendation information acquisition step S32.

[0176] In the rental item recommendation information acquisition step S32, the recommendation information transmission unit 58 acquires recommendation information for rental items by referring to the recommendation information database 18. After the rental item recommendation information acquisition step S32, the process proceeds to the recommendation information transmission step S20.

[0177] On the other hand, in the belongings determination step S30, if the belongings determination unit 60 determines that the object is a belonging of the user, a Yes determination is made. If a Yes determination is made, the process proceeds to a purchase recommendation information acquisition step S34.

[0178] In the purchase recommendation information acquisition step S34, the recommendation information transmission unit 58 acquires recommendation information for purchases by referring to the recommendation information database 18. After the purchase recommendation information acquisition step S34, the process proceeds to the recommendation information transmission step S20.

[0179] In the recommendation information transmission step S20, the recommendation information transmission unit 58 transmits recommendation information for purchases to the user terminal device 20 if the object is a user's possession. In the recommendation information transmitting step S20, the recommendation information transmitting unit 58 transmits recommendation information for rental items to the user terminal device 20 if the object is a rental item.

[0180] After the recommendation information transmitting step S20, the process proceeds to a termination determination step S22. The termination determination step S22 shown in FIG.

[0181] [Example of providing recommendation information] 14 is an explanatory diagram showing an example of a recommended information display screen for rental items. A recommended information display screen 240 shown in the drawing includes user identification information 242, recommended information 244 for rental items, and operation buttons 246.

[0182] User identification information 242 shown in the figure is the same as the user identification information 142 shown in Figure 10. Moreover, operation buttons 246 shown in Figure 14 are the same as the operation buttons 146 shown in Figure 10.

[0183] [Effects of the second embodiment] According to the image processing system 200 and the recommendation information providing method according to the second embodiment, the following advantageous effects can be obtained.

[0184] [1] It is determined whether the object is a user's possession or a rental item. If the object is a rental item, information about the rental item is provided as recommended information to the user. In this way, the user can obtain recommended information about the rental item.

[0185] [2] If the object is an item owned by the user, information on items to be purchased is provided as recommended information to the user, so that the user can obtain recommended information on items to be purchased.

[0186] [Image processing system according to the third embodiment] 〔overview〕 Next, a description will be given of an image processing system according to a third embodiment. An image processing system 300 according to the third embodiment identifies an image capture scene from the analysis result of the image content, and recognizes an event based on the image capture scene.

[0187] The image processing system 300 estimates the disposable income of the user based on the event, and provides the user with recommendation information related to the event of the image based on the disposable income of the user. The image processing system 300 according to the third embodiment will be described below, focusing mainly on the differences from the image processing system 10 according to the first embodiment.

[0188] Fig. 15 is a functional block diagram of a server device applied to an image processing system according to the third embodiment. Fig. 15 shows only the configuration of a server device 312 that has different functions from the server device 12 shown in Fig. 2.

[0189] [Object Recognition Unit] 15 recognizes the imaging scene of the analysis target image as an object of the analysis target image. Specifically, the imaging scene of the analysis target image is specified, and the location of the analysis target image is specified.

[0190] Examples of the location of the image to be analyzed include tourist spots, restaurants, and schools. The location to be analyzed may be a place name, an address, longitude and latitude, etc. The place name may be a common name or an old place name. The object recognition unit 344 transmits the recognition result of the object of the image to be analyzed to the type determination unit 346.

[0191] The location of the analysis target image may be the imaging location of the analysis target image. The object recognition unit 344 may acquire imaging location information included in the supplementary information and recognize the imaging location of the analysis target image as an object of the analysis target image. The object recognition unit described in the embodiment corresponds to an example of an imaging location information acquisition unit that acquires imaging location information.

[0192] [Type determination section] The type determination section 346 uses the recognition result of the object in the analysis target image transmitted from the object recognition section 344 to determine the type of event represented by the analysis target image as the type of object.

[0193] Examples of types of events include trips, visits to theme parks, meals at restaurants, school events such as athletic meets, sporting events such as watching sports, and beauty services at beauty salons.

[0194] 15 illustrates an example of the type determination unit 346 in which a separation distance between the user's home and the location of the image to be analyzed is derived, and the type of event is identified based on the separation distance. That is, the type determination unit 346 includes a home location information acquisition unit 372, a separation distance derivation unit 374, and an event type determination unit 376.

[0195] The home location information acquisition unit 372 acquires the home information of the user. Fig. 15 illustrates an example of acquiring the home information of the user transmitted from the user terminal device 20. The home location information acquisition unit 372 transmits the home information of the user to the distance derivation unit 374.

[0196] The distance deriving unit 374 derives the distance from the user's home to the location of the analysis target image, using the user's home information and analysis target image location information indicating the location of the analysis target image.

[0197] The event type determination unit 376 determines the type of event based on the location of the analysis target image and the distance from the user's home to the location of the analysis target image. For example, if the distance from the user's home to the location of the analysis target image is equal to or greater than a specified distance and the same location as the analysis target image is captured continuously for two or more days, the event type determination unit 376 determines the type of event to be travel.

[0198] If the location of the image to be analyzed does not satisfy the above travel conditions and is included in a predefined list of theme parks, the event type determination unit 376 may determine the type of event as a visit to a theme park.

[0199] When a school included in the specified school list is recognized as an object, the event type determination unit 376 may determine the type of the event as a school event. When a stadium or the like included in the specified sports spectating list is recognized as an object, the event type determination unit 376 may determine the type of the event as sports spectating.

[0200] That is, an event type list that specifies the relationship between the imaging scene and the type of event is created and stored in advance for each user. The event type determination unit 376 can refer to the event type list for each user to identify the type of event for the imaging scene. The event type determination unit 376 transmits determination information of the type of event to the price range determination unit 348.

[0201] Fig. 16 is a schematic diagram of a price range determination table applied to an image processing system according to the third embodiment. The price range determination table 370 shown in the figure defines the price range of objects for each type of object. In Fig. 16, price ranges are represented using numerical values ​​from 1 to 5. The numerical values ​​from 1 to 5 representing price ranges shown in Fig. 17 can be converted into numerical ranges representing price ranges.

[0202] 16, when the type of event is travel, the price range determination table 370 applies a positional relationship between the user's home and the location of the image to be analyzed as a rank. If it is within the same prefecture, the first rank is applied. If it is within the same region, the second rank is applied. If it is within an adjacent region, the third rank is applied.

[0203] For non-adjacent regions within the country, the fourth rank is applied. For overseas, the fifth rank is applied. Note that the positional relationship between the user's home and the location of the analysis target image may be determined by the distance from the user's home to the location of the analysis target image.

[0204] When the type of event is a visit to a theme park, a predetermined rank may be applied to each theme park in the price range determination table 370. The letters P to T in the price range determination table 370 shown in Fig. 17 represent the names of the theme parks.

[0205] In price range determination table 370, the first rank is applied when the theme park is P. When the theme park is Q, R, S, or T, the second, third, fourth, and fifth ranks are applied, respectively.

[0206] The price range determination unit 348 shown in Fig. 15 transmits the price range information to the disposable income range conversion unit 350. The disposable income range conversion unit 350 converts the price range information into the disposable income range information of the user. When converting the price range information into the disposable income range information of the user, the disposable income range conversion unit 350 refers to the disposable income range conversion table 380 shown in Fig. 17.

[0207] Fig. 17 is a schematic diagram of a disposable income band conversion table applied to the image processing system according to the third embodiment. In the disposable income band conversion table 380 shown in the figure, alphabets A to G are used to indicate disposable income band information corresponding to price band information. The alphabets A to G shown in Fig. 18 can be processed as numerical values. Note that the alphabets A to G shown in Fig. 17 may have a numerical range different from that of the alphabets A to G shown in Fig. 7.

[0208] The disposable income band conversion unit 350 shown in FIG. 15 transmits the disposable income band information to the disposable income estimation unit 56. The disposable income estimation unit is the same as the disposable income estimation unit 56 shown in FIG. 2. The disposable income estimation unit estimates the disposable income of the user using the disposable income band information and the weighting coefficient. Note that the disposable income estimation unit is not shown in FIG. 15.

[0209] The server device 312 shown in Fig. 15 includes a frequency derivation unit and a coefficient derivation unit. The frequency derivation unit is the same as the frequency derivation unit 52 shown in Fig. 2. The coefficient derivation unit is the same as the coefficient derivation unit 54 shown in Fig. 2. Note that the frequency derivation unit and the coefficient derivation unit are omitted from Fig. 15.

[0210] The frequency derivation unit derives the occurrence frequency of an event using the shooting dates of multiple images of the same shooting scene. The coefficient derivation unit derives a weighting coefficient based on the occurrence frequency of the event. When deriving the weighting coefficient, the coefficient derivation unit refers to a weighting coefficient table 390 shown in FIG.

[0211] 18 is a schematic diagram of a weighting coefficient table applied to the image processing system according to the third embodiment. This figure shows a weighting coefficient table 390 that specifies weighting coefficients when the event type is travel and weighting coefficients when the event type is visiting a theme park.

[0212] 18, a relatively large weighting coefficient is assigned to an event with a relatively high occurrence frequency, and a relatively small weighting coefficient is assigned to an event with a relatively low occurrence frequency.

[0213] Weighting coefficient table 390 also specifies a relatively small weighting coefficient for events that are relatively close to the user's home, while weighting coefficient table 390 specifies a relatively large weighting coefficient for events that are relatively far from the user's home.

[0214] The server device 312 shown in Fig. 15 includes a recommendation information transmission unit. The recommendation information transmission unit is the same as the recommendation information transmission unit 58 shown in Fig. 2. Note that the illustration of the recommendation information transmission unit is omitted in Fig. 15. The recommendation information transmission unit transmits recommendation information according to the disposable income of the user. The recommendation information transmission unit may transmit recommendation information taking into account the type of event.

[0215] [Other aspects of acquiring user's home location information] Fig. 19 is a functional block diagram of a type determination unit showing another mode of acquiring user's home location information. A home location information acquisition unit 372A provided in the type determination unit 346A shown in Fig. 19 analyzes the image content of one or more images from the group of images to be analyzed acquired using the image acquisition unit 40 to identify the user's home. That is, the home location information acquisition unit 372A identifies an image captured at the user's home from the group of images to be analyzed, and identifies the user's home location using GPS information or the like included in the supplementary information of the identified image.

[0216] The home location information acquisition section 372A may extract characteristic regions from each image in the group of images to be analyzed, and identify images in which the user's home has been captured by referring to a home characteristic region list that defines the characteristic regions in the home images.

[0217] According to this aspect, even if the user's home location information cannot be acquired from the user terminal device 20, the user's home location information can be acquired based on the analysis results of the group of images to be analyzed.

[0218] When the image capturing frequency of the feature area having the same feature is high, the home location information acquiring unit 372A may determine that the image is an image of the user's home. In addition, the image capturing frequency, the image capturing date, and the image capturing time may be taken into consideration to determine whether the image is an image of the user's home.

[0219] In addition, the home described in the embodiment corresponds to an example of the reference location. As another example of the reference location, any location designated in advance by the user, such as a workplace or a school, may be applied. In addition, the home location information acquisition unit 372A corresponds to an example of a reference location identification unit that identifies the reference location.

[0220] [Recommendation information providing method according to the third embodiment] Next, a recommended information providing method according to the third embodiment will be described. The procedure of the recommended information providing method according to the third embodiment is different from the procedure of the recommended information providing method according to the first embodiment in the object type determination step S104 shown in FIG.

[0221] Fig. 20 is a flowchart showing the procedure of an object type determination step applied to the recommendation information providing method according to the third embodiment. Fig. 20 illustrates an object type determination step for determining whether an event is a trip or a visit to a theme park.

[0222] In the event location acquisition step S300, the object recognition unit 344 shown in Fig. 15 analyzes the image content of the analysis target image to identify the location of the event. After the event location acquisition step S300, the process proceeds to home location information acquisition S302.

[0223] In the home location information acquisition S302, the home location information acquisition unit 372 acquires the home location information of the user. After the home location information acquisition S302, the process proceeds to a distance information acquisition step S304.

[0224] In the distance information acquisition step S304, the separation distance derivation unit 374 acquires information on the distance from the user's home to the location of the image to be analyzed. After the distance information acquisition step S304, the process proceeds to a distance determination step S306.

[0225] In the distance determination step S306, the event type determination unit 376 determines whether or not the distance from the user's home to the location of the analysis target image is equal to or greater than a specified distance. In the distance determination step S306, if the event type determination unit 376 determines that the distance from the user's home to the location of the analysis target image is less than the specified distance, the determination is No. If the determination is No, the process proceeds to the end determination step S316.

[0226] On the other hand, in the distance determination step S306, if the event type determination unit 376 determines that the distance from the user's home to the location of the image to be analyzed is equal to or greater than the specified distance, the determination is Yes. If the determination is Yes, the process proceeds to a number of days determination step S308.

[0227] In the number of days determination step S308, the event type determination unit 376 determines whether or not there are images of the same event captured over two or more consecutive days. If the event type determination unit 376 determines in the number of days determination step S308 that there are no images of the same event captured over two or more consecutive days, the determination is No. If the determination is No, the process proceeds to the theme park list comparison step S310.

[0228] On the other hand, in the number of days determination step S308, if the event type determination unit 376 determines that there are images of the same event captured over two or more consecutive days, the determination is Yes. If the determination is Yes, the process proceeds to a travel determination step S312. In the travel determination step S312, the event type determination unit 376 determines that the event of the image to be analyzed is travel, and transmits the determination result to the price range determination unit 348. After the travel determination step S312, the process proceeds to an end determination step S316.

[0229] In theme park list checking step S310, event type determination unit 376 determines whether or not the location of the image to be analyzed is included in the defined theme park list. In theme park list checking step S310, if event type determination unit 376 determines that the location of the image to be analyzed is not included in the defined theme park list, the determination is No. If the determination is No, the process proceeds to end determination step S316.

[0230] On the other hand, if the event type determination unit 376 determines in the theme park list comparison step S310 that the location of the image to be analyzed is included in the specified theme park list, the determination is Yes. If the determination is Yes, the process proceeds to theme park determination S314.

[0231] In the theme park determination S314, the event type determination unit 376 determines that the event of the image to be analyzed is a visit to a theme park, and transmits the determination result to the price range determination unit 348. After the theme park determination S314, the process proceeds to an end determination step S316.

[0232] In the termination determination step S316, it is determined whether or not to terminate the object type determination step S104 shown in Fig. 5. In the termination determination step S316, if the type determination unit 346 determines that the specified termination condition is not satisfied, a No determination is made. If a No determination is made, the process proceeds to the event location acquisition step S300. Thereafter, each step from the event location acquisition step S300 to the termination determination step S316 is repeatedly performed until a Yes determination is made in the termination determination step S316.

[0233] On the other hand, if the type determination unit 346 determines that the specified end condition is satisfied in the end determination step S316, a Yes determination is made. If the determination is Yes, the type determination unit 346 ends the object type determination step S104.

[0234] [Effects of the Third Embodiment] According to the image processing system and the recommendation information providing method according to the third embodiment, it is possible to obtain the following advantageous effects.

[0235] [1] Events are identified as objects in the images to be analyzed, which allows for the estimation of a user's disposable income based on the events.

[0236] [2] The type of event is identified based on the distance between the user's home and the location of the event, and the user's disposable income can be estimated based on the type of event.

[0237] [Variations] Next, a modified example of the image processing system and the recommendation information providing method described above will be described. Fig. 21 is a flowchart showing the procedure of the recommendation information providing method according to the modified example. The recommendation information providing method according to the modified example described below estimates the photographer for each analysis target image in the analysis target image group, and estimates the disposable income for each photographer. When estimating the disposable income of the user, the estimated value of the disposable income of other photographers is used.

[0238] The image acquisition step S10 is the same as the image acquisition step S10 shown in Fig. 4. After the image acquisition step S10, the process proceeds to an unnecessary image removal step S400. In the unnecessary image removal step S400, the object recognition unit 44 shown in Fig. 2 performs a process similar to the removal of images not suitable for analysis in the analysis target image setting step S100 shown in Fig. 5. After the unnecessary image removal step S400, the process proceeds to a photographer estimation step S402.

[0239] In the photographer estimation step S402, the object recognition unit 44 estimates the photographer of the image to be analyzed. For example, using information on the imaging device of each image, it is estimated whether the image was captured by the user or by someone other than the user.

[0240] The object recognition unit 44 can estimate that an image captured using an imaging device with a high imaging frequency is an image captured by a user. After the photographer estimation step S402, the process proceeds to an image determination step S404.

[0241] In the image determination step S404, the object recognition unit 44 determines whether the photographer of each analysis target image is the user or a person other than the user. In the image determination step S404, if the object recognition unit 44 determines that the photographer of the analysis target image is a person other than the user, the result is No. In the case of a No determination, the process proceeds to the disposable income estimation step S406 of the other photographer.

[0242] In the disposable income estimation step S406 of the other image capturers, the disposable income estimation unit 56 etc. performs each step from the image content analysis step S12 to the disposable income estimation step S18 shown in Fig. 4 for image capturers other than the user, to estimate the disposable income of the other image capturers. After the disposable income estimation step S406, the process proceeds to the disposable income estimation step S408.

[0243] On the other hand, if the object recognition unit 44 determines in the image determination step S404 that the photographer of the image to be analyzed is the user, the determination is Yes. If the determination is Yes, the process proceeds to a user's disposable income estimation step S408.

[0244] In the user disposable income estimation step S408, the disposable income estimation unit 56 etc. performs each step from the image content analysis step S12 to the disposable income estimation step S18 shown in FIG. 4 for the user to estimate the user's disposable income.

[0245] The disposable income estimation unit 56 etc. estimates the disposable income of the user, taking into consideration the estimated values ​​of the disposable income of other image capturers estimated in the disposable income estimation step S406 of other image capturers. That is, the disposable income estimation unit 56 etc. estimates the disposable income of the user, giving priority to the analysis target image captured by the user himself.

[0246] The disposable income estimation unit 56, etc., can estimate the disposable income of the photographer other than the user based on the image to be analyzed that was taken by someone other than the user, and can estimate the user's disposable income using the estimated value of the disposable income of the other photographer.

[0247] The disposable income estimation unit 56 etc. may take into consideration the correlation between the user and the other photographers when estimating the disposable income of the user using the estimated value of the disposable income of the other photographers. For example, the disposable income of the user may be estimated using the correlation between the ages of the user and the other photographers.

[0248] 22 is a correlation diagram showing the correlation between users and related parties according to age. A correlation diagram 500 shown in the figure shows the strength of the correlation between a user 501 and a first related party 502, etc., using the distance from the user 501. The strength of the correlation takes into account the age of the user 501 and the age of the first related party 502, etc.

[0249] The first boundary line 510 represents a boundary of 10 years of age difference between the user 501 and related parties on the first boundary line 510. The age difference between the user 501 and related parties on the inside of the first boundary line 510 is less than 10 years.

[0250] The second boundary line 512 represents a boundary line where the age difference between the user 501 and the related party is 20 years. The age difference between the related party on the second boundary line 512 and the user 501 is 20 years. The age difference between the related party outside the first boundary line 510 and inside the second boundary line 512 and the user 501 is more than 10 years and less than 20 years.

[0251] The third boundary line 514 represents a boundary line where the age difference between the user 501 and the related party is 30 years. The age difference between the related party and the user 501 and the related party on the third boundary line 514 is 30 years. The age difference between the related party and the user 501 and the related party on the inside of the third boundary line 514 and the related party is more than 20 years but less than 30 years.

[0252] The age difference between a first related party 502 and a user 501 shown in the figure is more than 20 years but less than 30 years. The age difference between a second related party 504 and the user 501 is 30 years. The first related party 502 is closer to the user 501 than the second related party 504, and has a stronger correlation.

[0253] The age difference between the third party 506 and the user 501 is more than 10 years and less than 20 years. The third party 506 is closer to the user 501 than the first party 502 and the second party 504, and has a stronger correlation.

[0254] It may be determined that the difference in disposable income between the user 501 and other photographers of a similar age is relatively small, and the disposable income of the user 501 may be estimated using the estimated values ​​of the disposable income of the other photographers. In addition, it may be determined that the difference in disposable income between the user 501 and the disposable income of related parties who are relatively close to the user 501 shown in the correlation diagram 500 shown in FIG. 22 is relatively small.

[0255] Fig. 23 is a correlation diagram showing the correlation between users and related parties according to hobbies. In the correlation diagram 520 shown in the figure, a fourth related party 530, a fifth related party 532, a sixth related party 534, and a seventh related party 536 are classified based on their hobbies. Fig. 23 shows bicycles, wine, and lunches as examples of hobbies.

[0256] It is possible to estimate the disposable income of user 501 by specifying that the difference between the disposable income of other photographers who have the same hobby as user 501 is relatively small and by using an estimate of the disposable income of the other photographers.

[0257] In the present embodiment, the embodiment has been described in which related persons of a user are classified according to age and hobbies. However, the correlation with a user may be defined according to educational background, hometown, occupation, and the like.

[0258] [Effects of modified examples] According to the modified example described above, the photographer is estimated for multiple images to be analyzed, and disposable income is estimated for each photographer. When estimating the disposable income of the user, the estimated values ​​of the disposable income of other photographers are used. This makes it possible to estimate the disposable income of the user taking into account the estimated values ​​of the disposable income of other photographers. In addition, the disposable income of the user can be estimated taking into account the correlation between the user and other photographers.

[0259] [Application to network systems] The server device 12 and the image database 14 shown in Fig. 1 may be communicatively connected via a network. Also, the components of the server device 12 shown in Fig. 2 may be communicatively connected via a network.

[0260] That is, the components constituting the server device 12 etc. shown in Fig. 2 may be distributed or collectively arranged. Cloud computing may be applied to the image processing system 10 etc. shown in Fig. 1.

[0261] [Application example to image management system] The image processing system 10 shown in Fig. 1 may be configured as a part of an image management system. The image management system automatically analyzes the subject and the captured scene of the image uploaded from the user terminal device 20 shown in Fig. 1, and automatically performs tagging processing.

[0262] A user may use the tags assigned to any image to perform searches of images stored in the image database 14. A user may also perform a manual tagging process on images that the user has uploaded.

[0263] [Examples of application to programs] A program corresponding to the image processing system 10 and the method for providing recommended information disclosed in this specification may be configured. That is, this specification discloses a program for enabling a computer to realize an object recognition function, a disposable income band conversion function, ancillary information acquisition function, a frequency derivation function, a coefficient derivation function, a disposable income estimation function, and a recommendation information transmission function.

[0264] The object recognition function analyzes two or more analysis target images included in an image group that includes a plurality of images related to a user, and recognizes objects included in each of the two or more analysis target images.

[0265] The disposable income band conversion function converts information of an object recognized using the object recognition function into disposable income band information that indicates the range of the user's disposable income. The incidental information acquisition function acquires incidental information of the analysis target image including image capture date information that indicates the image capture date of the analysis target image for two or more analysis target images in which objects are recognized using the object recognition function.

[0266] The frequency derivation function derives the appearance frequency of the object based on the image capture date information, and the coefficient derivation function derives a weighting coefficient corresponding to the object based on the appearance frequency information indicating the appearance frequency.

[0267] The disposable income estimation function estimates the disposable income of the user using the disposable income band information and the weighting coefficient. The recommendation information transmission function transmits recommendation information related to objects to the user according to the disposable income of the user.

[0268] The above-described embodiment of the present invention may be modified, added, or deleted as appropriate within the scope of the gist of the present invention. The present invention is not limited to the above-described embodiment, and many modifications may be made by a person having ordinary knowledge in the relevant field within the technical concept of the present invention. [Explanation of symbols]

[0269] 10 Image Processing System 12 Server equipment 14 Image Database 16 Disposable Income Database 16A Price Range Determination Database 16B Disposable Income Band Conversion Database 16C Weighting Factor Database 18 Recommendation information database 19 Administrator terminal device 20 User terminal device 22 Display section 30 Internet 40 Image acquisition unit 42 Additional Information Acquisition Department 44 Object Recognition Unit 46 Type determination section 48 Price Range Judgment Section 50 Disposable Income Band Conversion Section 52 Frequency Derivation Part 54 Coefficient derivation section 56 Disposable Income Estimation Division 58 Recommendation information transmission unit 59 User information acquisition unit 60 Possession Judgment Department 100 Price Range Judgment Table 110 Disposable Income Band Conversion Table 120 Weighting Factor Table 140 Recommendation information display screen 142 User Identification Information 144 Recommended Information 146 Operation Button 200 Image Processing System 212 Server device 240 Recommendation information display screen 242 User Identification Information 244 Recommended Information 246 Operation Button 300 Image Processing System 312 Server device 344 Object Recognition Unit 346 Type determination section 346A Type determination section 348 Price Range Judgment Department 350 Disposable Income Band Conversion Department 370 Price Range Judgment Table 372 Home Location Information Acquisition Department 372A Home location information acquisition unit 374 Separation distance derivation part 376 Event Type Judgment Section 380 Disposable Income Band Conversion Table 390 Weighting Factor Table 500 Relationship diagram 501 users 502 First Person 504 Second Person 506 Third Parties 510 First Boundary Line 512 Second boundary line 514 Third Boundary Line 520 Relationship diagram 530 Fourth Person 532 Fifth Person 534 The Sixth Person 536 Seventh Person S10 to S408: Each step of the method for providing recommendation information

Claims

1. an object recognition unit that analyzes two or more analysis target images included in an image group including a plurality of images related to a user, and recognizes objects included in each of the two or more analysis target images; a disposable income band conversion unit that converts information about the object recognized by the object recognition unit into disposable income band information that represents a range of disposable income of the user; an additional information acquisition unit that acquires additional information of the analysis target image including image capture date information indicating an image capture date of the analysis target image for the two or more analysis target images in which the object is recognized using the object recognition unit; a frequency deriving unit that derives an appearance frequency of the object based on the information of the object or the auxiliary information; a disposable income estimation unit that estimates the disposable income of the user by using the disposable income band information and the frequency of appearance of the object; An image processing system comprising:

2. The image processing system according to claim 1 , wherein the object recognition unit recognizes an article included in the image to be analyzed as the object.

3. a possession determination unit that determines whether the object is a possession or a rental item based on the appearance frequency of the object; a recommendation information sending unit that sends recommendation information related to the object to the user according to the disposable income of the user; Equipped with The image processing system of claim 2, wherein the recommendation information sending unit sends information about a purchased item as the recommendation information if the object is a user's possession, and sends information about a rental item as the recommendation information if the object is a rental item.

4. The image processing system according to claim 1 , further comprising a type determination unit that determines a type of the object recognized by the object recognition unit.

5. a type storage unit that stores a relationship between the object and a type of the object; The image processing system according to claim 4 , wherein the type determining section determines the type of the object by referring to the type storage section.

6. The image processing system according to claim 1 , wherein the object recognition unit recognizes a captured scene of the analysis target image as the object.

7. The image processing system according to claim 6 , wherein the object recognition unit determines an event corresponding to the captured scene as the type of the object.

8. The image processing system according to claim 6 or 7, further comprising a type determination unit that determines a type of the object recognized by the object recognition unit.

9. a type storage unit that stores a relationship between the object and a type of the object; The image processing system according to claim 8 , wherein the type determination unit determines the type of the object by referring to the type storage unit.

10. An imaging location information acquisition unit that acquires information on an imaging location of the analysis target image, The image processing system according to claim 8 , wherein the type determination unit determines the event corresponding to the object based on a positional relationship between a reference location of a user corresponding to the image group and a location where the analysis target image is captured.

11. The image processing system according to claim 10 , wherein the type determination unit determines the type of the event corresponding to the object based on a distance from the reference location to a location where the analysis target image is captured.

12. The image processing system according to claim 10 , wherein the image capture location information acquisition unit acquires the incidental information including image capture location information indicating a location where the analysis target image was captured, using the incidental information acquisition unit.

13. The image processing system according to claim 10 , wherein the imaging location information acquisition unit analyzes the analysis target image to identify the imaging location of the analysis target image.

14. The image processing system according to claim 10 , wherein the additional information acquisition unit acquires additional information including information on the reference location.

15. The image processing system according to claim 10 , further comprising a reference location identifying section that analyzes the analysis target image and identifies the reference location.

16. The image processing system according to claim 1 , further comprising a price range determination unit that determines a price range of the object corresponding to the type of the object.

17. a price range storage unit that stores a relationship between the type of the object and a price range; The image processing system according to claim 16 , wherein the price range determination unit determines a price range of the object by referring to the price range storage unit.

18. a disposable income band storage unit that stores a relationship between the price band and a disposable income band that represents a range of disposable income of the user; The image processing system according to claim 17 , wherein the disposable income band conversion unit converts price range information representing the price range into disposable income band information representing the disposable income band of the user with reference to the disposable income band storage unit.

19. a coefficient derivation unit that derives a weighting coefficient corresponding to the object based on appearance frequency information representing the appearance frequency; a coefficient storage unit that stores a relationship between the type of the object and the weighting coefficient; The image processing system according to claim 1 , wherein the coefficient derivation unit derives the weighting coefficient to be applied to the object by referring to the coefficient storage unit.

20. The image processing system according to claim 1 , further comprising a user information acquisition unit that acquires user information for identifying the user.

21. The computer an object recognition step of analyzing two or more analysis target images included in an image group including a plurality of images related to a user, and recognizing objects included in each of the two or more analysis target images; a disposable income band conversion step of converting information of the object recognized in the object recognition step into disposable income band information representing a range of disposable income of the user; an additional information acquisition step of acquiring additional information of the analysis target image including imaging date information indicating an imaging date of the analysis target image for the two or more analysis target images in which the object is recognized in the object recognition step; a frequency deriving step of deriving an appearance frequency of the object based on the information of the object or the auxiliary information; a disposable income estimating step of estimating the disposable income of the user by using the disposable income band information and the frequency of appearance of the object; An image processing method that performs

22. On the computer, an object recognition function that analyzes two or more analysis target images included in an image group including a plurality of images related to a user, and recognizes objects included in each of the two or more analysis target images; a disposable income band conversion function that converts information about the object recognized using the object recognition function into disposable income band information that represents a range of disposable income of the user; an additional information acquisition function for acquiring additional information of the analysis target image including image capture date information indicating the image capture date of the analysis target image for two or more analysis target images in which the object is recognized using the object recognition function; A frequency derivation function for deriving an appearance frequency of the object based on the information of the object or the accompanying information; and A program for implementing a disposable income estimation function that estimates the disposable income of the user by using the disposable income band information and the frequency of appearance of the object.

Images