Image Processing System, Image Processing Method, and Recording Medium

Abstract

The present invention provides an image processing system, an image processing method, and a recording medium that can provide recommendation information corresponding to disposable income estimated based on the analysis result of an image group. An object (44) that identifies an analysis target image of a user converts the information of the object into disposable income segment information (50) of the user, obtains incidental information including the shooting date information of the analysis target image, derives the appearance frequency (52) of the object based on the shooting date information, derives a weighting coefficient (54) based on the appearance frequency information of the object, estimates the disposable income of the user (56) using the disposable income segment information and the weighting coefficient, and sends recommendation information (58) corresponding to the disposable income of the user.

Description

Technical Field

[0001] The present invention relates to an image processing system, an image processing method, and a recording medium. Background Art

[0002] As a photo management service, a method using cloud computing has been proposed. Non-Patent Document 1 proposes a cloud computing system for managing static images, moving images, etc. uploaded from user terminals such as smartphones.

[0003] The system described in Non-Patent Document 1 automatically classifies the uploaded images according to the shooting date and shooting location. Further, the system analyzes the subject of the image, automatically classifies the image according to the analysis content of the subject, and provides recommended information corresponding to the user's hobbies, etc. based on the image analysis result.

[0004] Patent Document 1 describes a service providing system including a face-to-face device facing the user. The face-to-face device described in this document includes a photographing device, analyzes image data captured using the photographing device, and detects people, vehicles, and personal belongings of people within the photographing field of view.

[0005] Further, the system individually identifies the detected people, etc., obtains feature information such as the attributes and categories of people, etc., and provides recommended information corresponding to the feature information. Further, the system estimates income information from the information of the car used, and changes the recommended products.

[0006] Patent Document 2 describes a device that estimates user asset information based on the user's action history on the network, such as the number of image data submitted by the user to SNS, and provides a service matching the user's asset information. In addition, SNS is an abbreviation for Social Networking Service.

[0007] The device generates a model that takes the user's action history as input and outputs an index value indicating which user has more assets. As an example of the model, this document exemplifies a model in which explanatory variables x corresponding to feature information such as the transmission history of search queries are defined, weighting coefficients ω are defined for the explanatory variables, and for multiple feature information, x×ω obtained by multiplying the explanatory variable x by the weighting coefficient ω is combined.

[0008] Patent Document 1: Japanese Patent No. 6305483 Gazette

[0009] Patent Document 2: Japanese Patent No. 6494576 Gazette

[0010] Non-Patent Document 1: Google Photos, [online], [searched on June 7, 2019], Internet <URL: https: / / www.google.com / photos / about / >

[0011] However, the system described in Patent Document 1 detects a subject from a single image captured by a camera device facing the apparatus, and obtains feature information of the subject. In this way, it may cause a vehicle or the like that accidentally enters the frame to be detected as a subject, and there is a possibility that appropriate recommendation information cannot be provided to appropriate users.

[0012] Patent Document 2 does not specifically disclose a weighting coefficient applicable when generating a model representing user assets. Summary of the Invention

[0013] The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing system, an image processing method, and a recording medium capable of providing recommendation information corresponding to disposable income estimated based on an analysis result of an image group including a plurality of images.

[0014] To achieve the above object, the following inventive aspects are provided.

[0015] The image processing system according to the first aspect includes: an object recognition unit that analyzes two or more analysis target images included in an image group including a plurality of images associated with a user, and recognizes objects respectively included in the two or more analysis target images; a disposable income segment conversion unit that converts information of the objects recognized by the object recognition unit into disposable income segment information indicating a range of the user's disposable income; an attached information acquisition unit that acquires attached information of the analysis target images, including imaging date information indicating the imaging date of the analysis target images, for the two or more analysis target images in which the objects are recognized by the object recognition unit; a frequency derivation unit that derives the appearance frequency of the objects based on the imaging date information; a coefficient derivation unit that derives a weighting coefficient corresponding to the objects based on appearance frequency information indicating the appearance frequency; a disposable income estimation unit that estimates the user's disposable income using the disposable income segment information and the weighting coefficient; and a recommendation information transmission unit that transmits recommendation information associated with the objects to the user according to the user's disposable income.

[0016] According to the first aspect, an image group including a plurality of images is analyzed, information of the objects recognized from the images is converted into disposable income segment information of the user, and the weighting coefficient derived from the information of the appearance frequency of the objects is applied to the disposable income segment information to estimate the user's disposable income. Thereby, it is possible to provide recommendation information corresponding to the disposable income estimated based on the analysis result of the image group including a plurality of images.

[0017] The object can include either an item or an event represented by a shooting scene.

[0018] The term "image" can include the concept of an image signal representing an image, i.e., image data.

[0019] Disposable income only needs to be an estimated value of an indicator representing the purchasing power of a user, and is not limited to the actual income calculated by subtracting taxes, etc. from the user's income.

[0020] In the second mode of the image processing system of the first mode, it can be configured as follows: the object recognition unit recognizes and analyzes the structure of the item included in the object image as an object.

[0021] According to the second mode, based on the item included in the analysis object image, it is possible to estimate the disposable income of the user.

[0022] In the third mode of the image processing system of the second mode, it can also be configured as follows: it includes a carried item determination unit that determines whether the object is a carried item or a rented item based on the appearance frequency of the object. When the object is a carried item of the user, the recommended information sending unit sends information about the purchased item as recommended information. When the object is a rented item, the recommended information sending unit sends information about the rented item as recommended information.

[0023] According to the third mode, it is possible to provide recommended information corresponding to whether the object is a carried item of the user or the object is a rented item.

[0024] In the fourth mode of the image processing system of any one of the first mode to the third mode, it can also be configured as follows: it includes a type determination unit that determines the type of the object recognized by the object recognition unit.

[0025] According to the fourth mode, using the type of the object, it is possible to convert the information of the object into disposable income segment information of the user.

[0026] In the fifth mode of the image processing system of the fourth mode, it can be configured as follows: it includes a type storage unit that stores the relationship between the object and the type of the object. The type determination unit refers to the type storage unit to determine the type of the object.

[0027] According to the fifth mode, regarding the determination of the type of the object, a certain accuracy can be ensured.

[0028] In the sixth mode of the image processing system of the first mode, it can be configured as follows: the object recognition unit recognizes and analyzes the shooting scene of the object image as an object.

[0029] According to the sixth mode, based on the analysis of the shooting scene of the object image, it is possible to estimate the disposable income of the user.

[0030] In the seventh mode, in the image processing system of the sixth mode, the structure can be set as follows: the object recognition unit determines an event corresponding to the shooting scene as the type of the object.

[0031] According to the seventh mode, based on the event corresponding to the shooting scene of the analysis target image, it is possible to estimate the disposable income of the user.

[0032] As examples of events, traveling, sightseeing in a theme park, dining, and watching a sports game, etc. can be cited.

[0033] In the eighth mode, in the image processing system of the sixth mode or the seventh mode, the structure can be set as follows: there is a type determination unit that determines the type of the object recognized by using the object recognition unit.

[0034] According to the eighth mode, by using the type of the object, it is possible to convert the object information into disposable income segment information of the user.

[0035] In the ninth mode, in the image processing system of the eighth mode, the structure can be set as follows: there is a type storage unit that stores the relationship between the object and the type of the object, and the type determination unit refers to the type storage unit to determine the type of the object.

[0036] According to the ninth mode, regarding the determination of the type of the object, a certain accuracy can be ensured.

[0037] In the tenth mode, in the image processing system of the eighth mode or the ninth mode, the structure can be set as follows: there is a shooting location information acquisition unit that acquires information on the shooting location of the analysis target image, and the type determination unit determines an event corresponding to the object based on the positional relationship between the user's reference location corresponding to the image group and the shooting location of the analysis target image.

[0038] According to the tenth mode, based on the positional relationship between the user's reference location and the shooting location of the analysis target image, it is possible to determine an event.

[0039] In the eleventh mode, in the image processing system of the tenth mode, the structure can be set as follows: the type determination unit determines the type of the event corresponding to the object based on the distance from the reference location to the shooting location of the analysis target image.

[0040] According to the eleventh mode, based on the distance between the user's reference location and the shooting location of the analysis target image, it is possible to determine an event.

[0041] In the twelfth mode, in the image processing system of the tenth mode or the eleventh mode, the structure can be set as follows: the shooting location information acquisition unit uses an attached information acquisition unit to acquire attached information including shooting location information indicating the shooting location of the analysis target image.

[0042] According to the 12th mode, based on the information of the shooting location included in the attached information, the shooting location of the image can be determined.

[0043] In the 13th mode, in the image processing system of the 10th mode or the 11th mode, it can be configured as follows: the shooting location information acquisition unit analyzes the analysis target image to determine the shooting location of the analysis target image.

[0044] According to the 13th mode, based on the analysis result of the analysis target image, the shooting location of the image can be determined.

[0045] In the 14th mode, in the image processing system of any one of the 10th mode to the 13th mode, it can be configured as follows: the attached information acquisition unit acquires the attached information including the information of the reference location.

[0046] According to the 14th mode, based on the information of the reference location included in the attached information, the reference location can be determined.

[0047] In the 15th mode, in the image processing system of any one of the 10th mode to the 13th mode, it can be configured as follows: there is a reference location determination unit that analyzes the analysis target image to determine the reference location.

[0048] According to the 15th mode, based on the analysis result of the analysis target image, the reference location can be determined.

[0049] In the 16th mode, in the image processing system of any one of the 1st mode to the 15th mode, it can be configured as follows: there is a price range determination unit that determines the price range of the object corresponding to the type of the object.

[0050] According to the 16th mode, the price range of the object can be converted into the disposable income range of the user.

[0051] In the 17th mode, in the image processing system of the 16th mode, it can be configured as follows: there is a price range storage unit that stores the relationship between the type of the object and the price range, and the price range determination unit refers to the price range storage unit to determine the price range of the object.

[0052] According to the 17th mode, for the determination of the price range of the object, a certain accuracy can be ensured.

[0053] In the 18th mode, in the image processing system of the 17th mode, the following structure can be adopted, that is, it is provided with a disposable income segment storage unit that stores the relationship between the price segment and the disposable income segment representing the range of the user's disposable income. The disposable income segment conversion unit refers to the disposable income segment storage unit and converts the price segment information representing the price segment into the disposable income segment information representing the user's disposable income segment.

[0054] According to the 18th mode, a certain accuracy can be ensured for the conversion of the price segment of the object into the disposable income segment of the user.

[0055] In the 19th mode, in the image processing system of any one of the 1st to 18th modes, the following structure can be adopted, that is, it is provided with a coefficient storage unit that stores the relationship between the type of the object and the weighting coefficient. The coefficient derivation unit refers to the coefficient storage unit and derives the weighting coefficient applicable to the object.

[0056] According to the 19th mode, a certain accuracy can be ensured for the derivation of the weighting coefficient.

[0057] In the 20th mode, in the image processing system of any one of the 1st to 19th modes, the following structure can be adopted, that is, it is provided with a user information acquisition unit that acquires the user information for determining the user.

[0058] According to the 20th mode, the user of the analysis target image can be determined.

[0059] The image processing method according to the 21st mode includes: an object recognition process of analyzing two or more analysis target images included in an image group including a plurality of images associated with the user and recognizing the objects respectively included in the two or more analysis target images; a disposable income segment conversion process of converting the information of the objects recognized in the object recognition process into the disposable income segment information representing the range of the user's disposable income; an accessory information acquisition process of acquiring the accessory information of the two or more analysis target images of the objects recognized in the object recognition process, including the shooting date information representing the shooting date of the analysis target image; a frequency derivation process of deriving the appearance frequency of the object according to the shooting date information; a coefficient derivation process of deriving the weighting coefficient corresponding to the object according to the appearance frequency information representing the appearance frequency; a disposable income estimation process of estimating the user's disposable income by using the disposable income segment information and the weighting coefficient; and a recommended information sending process of sending the recommended information associated with the object to the user according to the user's disposable income.

[0060] According to the 21st mode, the same effect as that of the 1st mode can be obtained.

[0061] In the 21st mode, it is possible to appropriately combine the same matters as those determined in the 2nd to 20th modes. In this case, the constituent elements that undertake the processing or functions determined in the image processing system can be grasped as the constituent elements of the image processing method that undertakes the corresponding processing or functions.

[0062] The program recorded on the recording medium storing the program according to the 22nd mode causes a computer to implement the following functions: an object recognition function that analyzes two or more analysis target images included in an image group including a plurality of images associated with a user and recognizes the objects respectively included in the two or more analysis target images; a disposable income segment conversion function that converts the information of the objects recognized using the object recognition function into disposable income segment information representing the range of the user's disposable income; an attached information acquisition function that, for two or more analysis target images in which the objects are recognized using the object recognition function, acquires the attached information of the analysis target images including the shooting date information indicating the shooting date of the analysis target images; a frequency derivation function that derives the appearance frequency of the objects based on the shooting date information; a coefficient derivation function that derives a weighting coefficient corresponding to the objects based on the appearance frequency information indicating the appearance frequency; a disposable income estimation function that estimates the user's disposable income using the disposable income segment information and the weighting coefficient; and a recommended information sending function that sends recommended information associated with the objects to the user based on the user's disposable income.

[0063] According to the 22nd mode, the same effect as that of the 1st mode can be obtained.

[0064] In the 22nd mode, it is possible to appropriately combine the same matters as those determined in the 2nd to 20th modes. In this case, the constituent elements that undertake the processing or functions determined in the image processing system can be grasped as the constituent elements of the image processing program that undertakes the corresponding processing or functions.

[0065] Advantages of the Invention

[0066] According to the present invention, an image group including a plurality of images is analyzed, the information of the objects recognized from the images is converted into disposable income segment information of the user, the weighting coefficient derived from the information of the appearance frequency of the objects is applied to the disposable income segment information, and the user's disposable income is estimated. Thereby, it is possible to provide recommended information corresponding to the disposable income estimated based on the analysis result of the image group including a plurality of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is the overall structure diagram of the image processing system according to the embodiment.

[0068] Figure 2 is the functional block diagram of the server device according to the first embodiment.

[0069] Figure 3 It is an explanatory diagram of image analysis.

[0070] Figure 4 It is a flowchart showing the order of the recommended information providing method according to the first embodiment.

[0071] Figure 5 It shows Figure 4 A flowchart showing the order of the image content analysis process shown.

[0072] Figure 6 It is a schematic diagram showing an example of a price range determination table.

[0073] Figure 7 It is a schematic diagram showing an example of a disposable income range table.

[0074] Figure 8 It shows Figure 4 A flowchart showing the order of the additional information analysis process shown.

[0075] Figure 9 It is a schematic diagram showing an example of a coefficient table.

[0076] Figure 10 It is an explanatory diagram showing an example of a recommended information display screen.

[0077] Figure 11 It is a functional block diagram showing the server device according to the second embodiment.

[0078] Figure 12 It is a flowchart showing the order of the recommended information providing method according to the second embodiment.

[0079] Figure 13 It shows Figure 12 A flowchart showing the order of the additional information analysis process shown.

[0080] Figure 14 It is an explanatory diagram showing an example of a recommended information display screen for rental items.

[0081] Figure 15 It is a functional block diagram showing a structural example of the type determination unit applicable to the image processing system according to the third embodiment.

[0082] Figure 16 It is a schematic diagram of the price range determination table applicable to the image processing system according to the third embodiment.

[0083] Figure 17 It is a schematic diagram of the disposable income range conversion table applicable to the image processing system according to the third embodiment.

[0084] Figure 18 It is a schematic diagram of a coefficient table applicable to the image processing system according to the third embodiment.

[0085] Figure 19 It is a functional block diagram of a type determination unit that represents another method of obtaining the user's home location information.

[0086] Figure 20 It is a flowchart showing the order of an object type determination process applicable to the recommendation information providing method according to the third embodiment.

[0087] Figure 21 It is a flowchart showing the order of an object type determination process applicable to the recommendation information providing method according to the third embodiment.

[0088] Figure 22 It is a correlation diagram showing the correlation between a user corresponding to an age group and a related person.

[0089] Figure 23 It is a correlation diagram showing the correlation between a user corresponding to an interest and a related person. Detailed Embodiments

[0090] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In this specification, the same reference numerals are assigned to the same components, and repeated descriptions are appropriately omitted.

[0091] [Example of the Structure of the Image Processing System]

[0092] 〔Overall Structure〕

[0093] Figure 1 It is an overall structure diagram of the image processing system according to the embodiment. The image processing system 10 shown in this figure includes a server device 12. The server device 12 reads out a plurality of images associated with the user stored in the image database 14, analyzes the read images, and provides recommendation information to the user based on the image analysis results. The user terminal device 20 uses the display unit 22 to display the recommendation information.

[0094] In addition, the term "image" in this specification can include the concept of an image signal representing an image, that is, image data.

[0095] The image processing system 10 includes a server device 12, an image database 14, a disposable income database 16, a recommendation information database 18, and a manager terminal device 19. The server device 12 is communicably connected to the image database 14, the disposable income database 16, the recommendation information database 18, and the manager terminal device 19. Data communication can be implemented via a prescribed network. All communication methods of data communication can apply wireless communication methods and wired communication methods.

[0096] The image database 14 is configured to be able to search for images associated with a user by using the identification information of any user as a search keyword. The identification information of the user can be a user name, a user ID, etc. Additionally, ID is an abbreviation of identification.

[0097] Images associated with the user can include images given as additional information such as the user name of the photographer, and images labeled with tags such as the user name. That is, images associated with the user can include images taken using the user's own imaging device and images having the user as a subject.

[0098] The disposable income database 16 includes a price range determination database that stores a price range determination table representing the relationship between the type of object and the price range. The disposable income database 16 includes a disposable income range conversion database that stores a disposable income range conversion table representing the conversion relationship between the price range and the disposable income range.

[0099] The disposable income database 16 includes a weighting coefficient database that stores a weighting coefficient table representing the relationship between the appearance frequency of the object extracted from the image and the weighting coefficient.

[0100] In addition, the price range determination database is illustrated by the symbol 16A in Figure 2 In addition, the disposable income range conversion database is illustrated by the symbol 16B in Figure 2 In addition, the weighting coefficient database is illustrated by the symbol 16C in Figure 2 In

[0101] The price range determination database that stores the price range determination table is an example of a price range storage unit that stores the relationship between the type of object and the price range. The disposable income range conversion database that stores the disposable income range conversion table representing the conversion relationship between the price range and the disposable income range is an example of a disposable income range storage unit.

[0102] The weighting coefficient database that stores the weighting coefficient table representing the relationship between the appearance frequency of the object and the weighting coefficient is an example of a coefficient storage unit that stores the relationship between the type of object and the weighting coefficient.

[0103] The recommended information database 18 stores a recommended information table representing the relationship between the disposable income and the recommended goods, etc. The server device 12 uses the information of the disposable income and refers to the recommended information database 18 to send recommended information based on the user's disposable income to the user terminal device 20.

[0104] The management terminal device 19 performs update control of tables provided in the disposable income database 16 and the like. The update of the tables provided in the disposable income database 16 and the like can also be automatically performed by the server device 12.

[0105] In this specification, the term "disposable income" only needs to represent an estimated value of an index of the user's purchasing power, and is not limited to the actual income calculated by subtracting taxes and the like from the user's income.

[0106] The user terminal device 20 can be applied to smart devices used by each user such as smartphones and tablet computers. The user terminal device 20 includes a communication unit. The user terminal device 20 can perform data communication with the server device 12 via the communication unit. In addition, the illustration of the communication unit is omitted.

[0107] The user terminal device 20 includes a display unit 22. The user terminal device 20 receives recommendation information sent from the server device 12. The user terminal device 20 can use the display unit 22 to display the recommendation information.

[0108] The user terminal device 20 can include a camera unit. The user terminal device 20 can include a storage unit that temporarily stores images obtained by photographing using the camera unit. The user terminal device 20 can upload images to the image database 14 via the server device 12. In addition, the illustrations of the camera unit and the storage unit are omitted.

[0109] The server device 12 is connected to the Internet 30 so as to be able to perform data communication. The server device 12 receives information on products and services applicable to recommendation information from providers via the Internet 30.

[0110] 〔Hardware Structure of Image Processing System〕

[0111] The server device 12 and the like applicable to the image processing system 10 can be applied to a computer. The computer applies the following hardware and executes a prescribed program to be able to realize the functions of the image processing system 10. In addition, the program has the same meaning as software.

[0112] The server device 12 and the like can be applied to various processors as a signal processing unit that performs signal processing. As an example of the processor, a CPU and a GPU (Graphics Processing Unit) can be cited. The CPU is a general-purpose processor that functions as a signal processing unit for executing a program. The GPU is a processor dedicated to image processing. The hardware of the processor can be applied to an electric circuit that combines electric circuit elements such as semiconductor elements. Each control unit includes a ROM that stores programs and the like and a RAM that is a working area for various operations.

[0113] It is also possible to apply two or more processors to a signal processing unit. The two or more processors can be of the same type or different types. Further, it is also possible to apply one processor to a plurality of signal processing units.

[0114] [Structural example of server device according to the first embodiment]

[0115] Figure 2 is a functional block diagram of the server device according to the first embodiment. Further, Figure 2 the DB shown is an abbreviation for Database. Further, in Figure 2 two image databases 14 are shown, but they are the same constituent elements.

[0116] The server device 12 includes an image acquisition unit 40 and an attached information acquisition unit 42. The server device 12 includes an object recognition unit 44, a type determination unit 46, a price range determination unit 48, and a disposable income range conversion unit 50. The server device 12 includes a frequency derivation unit 52 and a coefficient derivation unit 54. The server device 12 includes a disposable income estimation unit 56 and a recommendation information transmission unit 58. The server device 12 includes a user information acquisition unit 59.

[0117] The image acquisition unit 40 acquires an analysis target image from the image database 14. The server device 12 sets the identification information of the user as a search keyword and searches the image database 14, so that an image associated with the user can be acquired as the analysis target image. The image database 14 transmits the acquired image to the object recognition unit 44.

[0118] The attached information acquisition unit 42 acquires the attached information of the image acquired from the image database 14. The attached information includes information on the shooting date. The attached information may also include GPS information including information on the shooting location. Further, GPS is an abbreviation for Global Positioning System. The attached information acquisition unit 42 transmits the attached information to the frequency derivation unit 52.

[0119] The object recognition unit 44 performs object recognition processing for recognizing an object from the image acquired by using the image acquisition unit 40. The object recognition unit 44 can recognize a plurality of objects from one image. The object recognition processing can apply a known method. Here, a detailed description of the object recognition processing is omitted.

[0120] Examples of objects to which the present invention can be applied include articles included in the image, the background of the image, and the scene of the image. Examples of articles include non-human objects such as food, vehicles, clothing of a person, and ornaments of a person. Examples of the background of the image include buildings and scenery in a travel destination and a theme park.

[0121] When an object is determined in advance and object information is given as tag information or the like, the object recognition unit 44 may also set the tag information or the like as the recognition result of the object. That is, the object recognition unit 44 includes a tag information acquisition unit that acquires tag information.

[0122] When an object cannot be recognized from an image, the object recognition unit 44 may also output a recognition result indicating that there is no object. The server device 12 may also not perform the processing after object recognition on the image in which the object cannot be recognized.

[0123] The type determination unit 46 determines the type of the object recognized by the object recognition unit 44. When determining the type of the object, the type determination unit 46 can refer to a table showing the correspondence between the object and the type of the object. The type determination unit 46 sends the type information of the object for each object to the price range determination unit 48.

[0124] In addition, in the present embodiment, the illustration of the type determination database that stores the table showing the correspondence between the object and the type of the object is omitted. The type determination database corresponds to an example of the type storage unit.

[0125] The price range determination unit 48 determines the price range of the object according to the type of the object. When determining the price range of the object, the price range determination unit 48 refers to the price range determination database 16A. The price range determination unit 48 sends the price range information of each object to the disposable income range conversion unit 50.

[0126] The disposable income range conversion unit 50 converts the price range information of each object into disposable income range information indicating the disposable income range of the user. When converting the price range information of each object into disposable income range information, the disposable income range conversion unit 50 refers to the disposable income range conversion database 16B. The disposable income range conversion unit 50 sends the disposable income range information to the disposable income estimation unit 56.

[0127] The frequency derivation unit 52 derives the appearance frequency of the objects included in the plurality of images using the shooting date information included in the attached information of each image. The frequency derivation unit 52 sends the appearance frequency information indicating the appearance frequency of each object to the coefficient derivation unit 54.

[0128] The coefficient derivation unit 54 derives the weighting coefficient of each object according to the appearance frequency information of each object. When deriving the weighting coefficient of each object, the coefficient derivation unit 54 refers to the weighting coefficient database 16C. The coefficient derivation unit 54 sends the weighting coefficient of each object to the disposable income estimation unit 56.

[0129] The disposable income estimation unit 56 estimates the user's disposable income using the disposable income segment information indicating the disposable income segments of each object and the weighting factor of each object. The disposable income estimation unit 56 sends the disposable income estimation information indicating the estimation result of the user's disposable income to the recommendation information sending unit 58.

[0130] The recommendation information sending unit 58 sets the recommendation information for the user based on the user's disposable income estimation information. When setting the recommendation information, the recommendation information sending unit 58 refers to the recommendation information database 18. The recommendation information sending unit 58 sends the recommendation information to the user terminal device 20.

[0131] The recommendation information sending unit 58 provides recommendation information such as products corresponding to the object. For example, when the object is an orange, the recommendation information sending unit 58 can provide the user with recommendation information such as fruits, processed fruits, and pastries made from fruits.

[0132] The user information acquisition unit 59 acquires the user information for identifying the user. The user information acquisition unit 59 can acquire the user information sent from the Figure 1 shown user terminal device 20. The user information acquisition unit 59 sends the user information to the object recognition unit 44.

[0133] 〔Regarding the use of personal information, etc.〕

[0134] Regarding Figure 1 and Figure 2 the use of personal information, etc. in the image processing system 10 shown below.

[0135] The administrator of the image processing system 10 needs to obtain the user's permission for the analysis of the user's image. The administrator mentioned here can include groups such as enterprises. As an example of permission, a method can be cited in which a check mark is added to the check box indicating permission in the permission screen displayed on the display unit 22 of the user terminal device 20, and the information indicating permission is sent from the user terminal device 20 to the server device 12. The same applies to the following permissions.

[0136] The administrator of the image processing system 10 needs to obtain the user's permission to provide the user with recommendation information based on the analysis result of the user's image.

[0137] The entity implementing the provision of recommendation information to the user can be the administrator of the image processing system 10. The entity providing the recommendation information can be the provider of the products, etc. included in the recommendation information. The provider mentioned here can include groups such as enterprises.

[0138] When a provider of goods or the like provides recommendation information, the user's permission must be obtained regarding the submission of the necessary information related to the provision of recommendation information by the administrator of the image processing system 10. The necessary information is set to the minimum necessary information such as an email address for the provision of recommendation information.

[0139] In the provision of information from the administrator of the image processing system 10 to a cooperating organization or the like, it is generally prohibited to provide user information such as user names and specific information of users. When performing image analysis on multiple users, the above measures apply to all users. Regarding the provision of information after prior anonymization, the user's permission must be obtained.

[0140] 〔Explanation of Image Analysis〕

[0141] Figure 3 It is an explanatory diagram of image analysis. Image analysis refers to the processing of the images implemented by the Figure 2 server device 12 shown. As image analysis, the server device 12 can perform the analysis of attached information and the analysis of image content.

[0142] The server device 12 can apply the shooting date information as the attached information of the analysis object. The server device 12 analyzes the shooting date information in the image group and derives the appearance frequency of the common objects in the multiple images. It is also possible to derive the appearance frequency of multiple objects.

[0143] The server device 12 can apply the GPS information as the attached information of the analysis object. The server device 12 analyzes the GPS information and can determine the shooting location of the image. The server device 12 can also analyze the information related to the imaging device used in the shooting of the image as the attached information of the analysis object. The server device 12 can also analyze the information related to the terminal device equipped with the imaging device, etc.

[0144] The server device 12 analyzes the image content and can determine the subject. As examples of the subject, food such as cuisine, clothes such as dresses, and accessories such as watches can be cited. The server device 12 identifies the item, that is, the object, based on the subject.

[0145] The server device 12 analyzes the image content and can determine the shooting scene. As examples of the shooting scene, sightseeing in tourist destinations, sightseeing in theme parks, dining in restaurants, and watching sports games, etc. can be cited. The server device 12 can identify the event represented by the shooting scene as the object.

[0146] The server device 12 can apply a note image such as a screenshot as the image. The server device 12 analyzes the screenshot and can determine the purchased items by the user and the services used by the user, etc. as the objects based on the analysis result of the screenshot.

[0147] [Recommendation Information Providing Method According to the First Embodiment]

[0148] [Overall Process of the Recommendation Information Providing Method]

[0149] Figure 4 It is a flowchart showing the sequence of the recommendation information providing method according to the first embodiment. The recommendation information providing method according to the first embodiment described below acquires a plurality of images associated with an arbitrary user, estimates the disposable income of the user based on the analysis results of the plurality of images to be analyzed, and provides recommendation information corresponding to the disposable income of the user.

[0150] In the image acquisition step S10, Figure 2 The image acquisition unit 40 shown in FIG. acquires the images to be analyzed from the image database 14. The image acquisition step S10 can include an image storage step of storing the acquired images to be analyzed. After the image acquisition step S10, it proceeds to the image content analysis step S12.

[0151] The image acquisition step S10 can include a search step of searching for images associated with the user. Also, the image acquisition step can include a login step for the user to log in. Before performing the image acquisition step S10, the search step or the login step may be performed.

[0152] In the image content analysis step S12, the object recognition unit 44 performs the process of recognizing an object from the image. That is, in the image content analysis step S12, the object recognition unit 44 analyzes the pixels constituting the image to be analyzed. Also, in the image content analysis step S12, the type determination unit 46 determines the type of the object.

[0153] In the image content analysis step S12, the price range determination unit 48 determines the price range of the object. In the image content analysis step S12, the disposable income range conversion unit 50 converts the price range of the object into a disposable income range.

[0154] The image content analysis step S12 can include a storage step of storing each piece of information. In addition, the detailed content of the image content analysis step S12 will be described later. After the image content analysis step S12, it proceeds to the additional information analysis step S14.

[0155] In the additional information analysis step S14, the additional information acquisition unit 42 acquires the shooting date information as the additional information of each image. In the additional information analysis step S14, the frequency derivation unit 52 derives the appearance frequency of each object. In the additional information analysis step S14, the coefficient derivation unit 54 derives the weighting coefficient of each object.

[0156] The accompanying information analysis process S14 can include a storage process for storing each piece of information. Additionally, the detailed content of the accompanying information analysis process S14 will be described later. After the accompanying information analysis process S14, the analysis end determination process S16 is entered.

[0157] In the analysis end determination process S16, the server device 12 determines whether all the images to be analyzed have been analyzed. In the analysis end determination process S16, when it is determined that there is an image to be analyzed that the server device 12 has not analyzed, it becomes a "no" determination.

[0158] When it is a "no" determination, the process returns to the image content analysis process S12, and the processes from the image content analysis process S12 to the analysis end determination process S16 are repeatedly executed until it becomes a "yes" determination in the analysis end determination process S16.

[0159] On the other hand, in the analysis end determination process S16, when the server device 12 determines that all the images of the analysis object have been analyzed, it becomes a "yes" determination. When it is a "yes" determination, the disposable income estimation process S18 is entered.

[0160] In the disposable income estimation process S18, the disposable income estimation unit 56 estimates the user's disposable income based on the image analysis result in the image content analysis process S12 and the accompanying information analysis result in the accompanying information analysis process S14. The disposable income estimation process S18 can include a storage process for storing the estimation result of the disposable income. After the disposable income estimation process S18, the recommended information sending process S20 is entered.

[0161] In the recommended information sending process S20, the recommended information sending unit 58 sets recommended information including products and services recommended to the user based on the estimation result of the user's disposable income. In the recommended information sending process S20, the recommended information sending unit 58 sends the recommended information to the user terminal device 20. After the recommended information sending process S20, the end determination process S22 is entered.

[0162] In the end determination process S22, the server device 12 determines whether to end the recommended information providing method. In the end determination process S22, when the server device 12 determines to continue the recommended information providing method, it becomes a "no" determination. When it is a "no" determination, the image acquisition process S10 is entered, and the processes from the image acquisition process S10 to the end determination process S22 are repeatedly executed for the next image group until it becomes a "yes" determination in the end determination process S22.

[0163] On the other hand, in the end determination step S22, when the server device 12 determines that the recommendation information providing method ends, it becomes a "Yes" determination. When it is a "Yes" determination, the server device 12 performs a prescribed end process. As an example of the end condition of the recommendation information providing method, it can be cited that providing recommendation information ends for a preset image group. In addition, the recommendation information providing method shown in the embodiment corresponds to an example of an image processing method.

[0164] 〔Details of the image content analysis step〕

[0165] Figure 5 represents Figure 4 is a flowchart showing the order of the image content analysis step shown. In the analysis target image setting step S100, Figure 2 the object recognition unit 44 shown determines the analysis target image group and extracts the analysis target image from the determined image group. That is, in the analysis target image setting step S100, the object recognition unit 44 removes images that are not suitable for analysis.

[0166] Moreover, in the analysis target image setting step S100, the object recognition unit 44 determines a similar image group with similar image content such as an image group suitable for continuous shooting acquisition, extracts one image from the images included in the similar image group, and removes the remaining images. After the analysis target image setting step S100, it proceeds to the object recognition step S102.

[0167] In the object recognition step S102, the object recognition unit 44 recognizes an object from the analysis target image. After the object recognition step S102, it proceeds to the object type determination step S104.

[0168] In the object type determination step S104, the type determination unit 46 refers to the correspondence between a prescribed object and the type of the object and determines the type of the object. For example, when the object is an orange, the type determination unit 46 determines the type of the object as a fruit. And when the object is amber, the type determination unit 46 determines the type of the object as a jewelry ornament. After the object type determination step S104, it proceeds to the price range determination step S106.

[0169] In the price range determination step S106, the type determination unit 46 refers to the price range determination database 16A and determines the price range of each object. Figure 6 is a schematic diagram showing an example of a price range determination table.

[0170] Figure 6 The price range determination table 100 shown prescribes the price range of an object for each type of the object. Figure 6 uses numerical values from 1 to 5 to represent the price range. Represents Figure 6 the numerical values from 1 to 5 representing the price range shown can be converted into a numerical range representing the price range. InFigure 6 One object is shown for each price range, but a price range can contain multiple objects. After the price range determination step S106, the undetermined object determination step S108 is entered.

[0171] In the undetermined object determination step S108, the server device 12 determines whether price range determination has been performed on all objects. In the undetermined object determination step S108, when the server device 12 determines that there is an object for which the price range has not been determined, it becomes a "no" determination. When it is a "no" determination, the object type determination step S104 is entered, and the steps from the object type determination step S104 to the undetermined object determination step S108 are repeatedly performed until it becomes a "yes" determination in the undetermined object determination step S108.

[0172] On the other hand, in the undetermined object determination step S108, when the server device 12 determines that price range determination has been performed on all objects, it becomes a "yes" determination. When it is a "yes" determination, the disposable income range conversion step S110 is entered.

[0173] In the disposable income range conversion step S110, the disposable income range conversion unit 50 refers to the disposable income range conversion database 16B and converts the price range determination result of each object into the user's disposable income range.

[0174] Figure 7 It is a schematic diagram showing an example of the disposable income range table. In Figure 7 In the shown disposable income range conversion table 110, letters A to G are used to represent the disposable income ranges corresponding to the price ranges of each type of object. Figure 7 The letters A to G shown can be processed as numerical values.

[0175] When different price ranges are determined for multiple objects included in the same image, the disposable income range conversion unit 50 can derive the user's disposable income range according to a specified priority order. After the disposable income range conversion step S110, the image content analysis end determination step S112 is entered.

[0176] In the image content analysis end determination step S112, the server device 12 determines whether to end the image content analysis. In the image content analysis end determination step S112, when the server device 12 determines that the processing of all analysis target images has not been completed, it becomes a "no" determination. When it is a "no" determination, the object recognition step S102 is entered, and the steps from the object recognition step S102 to the image content analysis end determination step S112 are repeatedly performed until it becomes a "yes" determination in the image content analysis end determination step S112.

[0177] On the other hand, in the image content analysis end determination step S112, when the server device 12 determines that the processing of all the analysis target images has ended, it becomes a "Yes" determination. When it is a "Yes" determination, the server device 12 ends Figure 4 the image content analysis step S12 shown.

[0178] 〔Details of the attached information analysis step〕

[0179] Figure 8 It shows Figure 4 a flowchart showing the order of the attached information analysis step shown. In the shooting date information acquisition step S200, Figure 2 the attached information acquisition unit 42 shown acquires the shooting date information as the attached information. After the shooting date information acquisition step S200, it proceeds to the image determination step S202. In addition, the shooting date information acquisition step S200 described in the embodiment corresponds to an example of the attached information acquisition step.

[0180] In the image determination step S202, the frequency derivation unit 52 determines the images of the objects for which the appearance frequency is to be derived. That is, the frequency derivation unit 52 determines, for each object, a plurality of images including the same object. After the image determination step S202, it proceeds to the appearance frequency derivation step S204.

[0181] In the appearance frequency derivation step S204, the frequency derivation unit 52 derives the appearance frequency of the object. The appearance frequency can apply the shooting cycle. The shooting cycle is the reciprocal of the shooting period. When two or more shooting cycles are derived for three or more images, the frequency derivation unit 52 obtains the statistical value of the two or more shooting cycles as the shooting cycle. Examples of the statistical value include the average value, the median value, the maximum value, and the minimum value. After the appearance frequency derivation step S204, it proceeds to the coefficient derivation step S206.

[0182] In the coefficient derivation step S206, the coefficient derivation unit 54 refers to the weighting coefficient database 16C and derives the weighting coefficient for each object. Figure 9 It is a schematic diagram showing an example of the coefficient table.

[0183] Figure 9 The weighting coefficient table 120 shown represents the relationship between the appearance frequency of each type of object and the weighting coefficient. For example, when the shooting cycle is six months, the weighting coefficient of fruits is 0.1. Figure 9 In the weighting coefficient table 120 shown, when the appearance frequency is relatively high, a relatively large weighting coefficient is specified. After Figure 8 the coefficient derivation step S206 shown, it proceeds to the coefficient derivation determination step S208.

[0184] In the coefficient derivation determination step S208, the server device 12 determines whether weighted coefficients have been derived for all objects. In the coefficient derivation determination step S208, when the server device 12 determines that there is an object for which the weighted coefficient has not been derived, it becomes a "no" determination. When it is a "no" determination, the process proceeds to the image determination step S202, and the steps from the image determination step S202 to the coefficient derivation determination step S208 are repeatedly executed until it becomes a "yes" determination in the coefficient derivation determination step S208.

[0185] On the other hand, in the coefficient derivation determination step S208, when the server device 12 determines that weighted coefficients have been derived for all objects, it becomes a "yes" determination. When it is a "yes" determination, the server device 12 ends Figure 4 the attached information analysis step S14 shown.

[0186] Moreover, in the present embodiment, a method of setting weighted coefficients for each object is exemplified, but weighted coefficients may also be set for each object.

[0187] [Specific example of providing recommended information]

[0188] Figure 10 It is an explanatory diagram showing an example of a recommended information display screen. The recommended information display screen 140 is displayed using the display unit 22 of the user terminal device 20. The recommended information display screen 140 includes user identification information 142, recommended information 144, and operation buttons 146. The user can purchase the products included in the recommended information 144 by operating the user terminal device 20.

[0189] When a plurality of objects are included in one analysis target image, the disposable income of the user can be estimated for each object. When a plurality of different disposable incomes are estimated for one user, the maximum value of the estimated disposable income values can be set as the user's disposable income to provide recommended information.

[0190] When a plurality of different disposable incomes are estimated for one user, recommended information may also be provided for the estimated values of the plurality of disposable incomes respectively. The priority order may be set in advance for the types of objects, and the disposable income of the user may be estimated based on the object with the highest priority order.

[0191] [Function and effect of the first embodiment]

[0192] According to the image processing system 10 and the recommended information providing method according to the first embodiment, the following function and effect can be obtained.

[0193] [1]

[0194] Identify an object from an image, determine the type of the object, determine the price range for each type of the object, and convert the price range for each type of the object into a disposable income range. Further, for a plurality of images common to the object, derive the appearance frequency of the object from the shooting date information, and derive a weighting coefficient based on the appearance frequency of the object. Multiply the disposable income range by the weighting coefficient to estimate the disposable income of the user. Thereby, it is possible to provide recommendation information corresponding to the disposable income estimated from the analysis results of the plurality of images.

[0195] [2]

[0196] Identify an article from an image as an object. Thereby, it is possible to estimate the disposable income of the user based on the article included in the image.

[0197] [3]

[0198] Refer to a price range determination table indicating the relationship between the type of the object and the price range, and determine the price range for each type of the object. Thereby, a certain degree of accuracy can be ensured for the price range determination.

[0199] [4]

[0200] Refer to a disposable income range table indicating the relationship between the price range of the object and the disposable income range, and derive the disposable income range from the price range of the object. Thereby, a certain degree of accuracy can be ensured for the conversion of the disposable income range.

[0201] [5]

[0202] Refer to a weighting coefficient table indicating the relationship between the appearance frequency of the object and the weighting coefficient, and derive the weighting coefficient from the appearance frequency of the object. Thereby, a certain degree of accuracy can be ensured for the weighting coefficient.

[0203] In the present embodiment, in the derivation of the weighting coefficient and the like, a method of referring to a table is exemplified, but the processing of the derivation of the weighting coefficient and the like may also be applicable to a deep learning type. For example, Figure 2 The coefficient derivation unit 54 shown can be applied to a learned type that has learned the conversion relationship between the appearance frequency of the object and the weighting coefficient. The same applies to the type determination unit 46, the price range determination unit 48, and the disposable income range conversion unit 50.

[0204] When a product or the like having a relatively high price or a relatively low price range compared to the product or the like included in the recommendation information already provided to the user is selected, the image processing system 10 may also use the information of the product or the like purchased by the user as learning data to perform learning, and update the parameters related to the estimation of the disposable income and the parameters related to the provision of the recommendation information.

[0205] [Image Processing System According to the Second Embodiment]

[0206] Next, the image processing system according to the second embodiment will be described. The image processing system 200 according to the second embodiment determines whether the object is a user's carried item or a rental item, and provides recommended information corresponding to the determination result.

[0207] 〔Structural example of server device〕

[0208] Figure 11 is a functional block diagram of the server device according to the second embodiment. The server device 212 adds a carried item determination unit 60 to Figure 2 the server device 12 shown. The carried item determination unit 60 uses the shooting date information to determine whether the object is a user's carried item or a rental item.

[0209] For example, the carried item determination unit 60 determines an object that reappears with a shooting date more than one day apart as a user's carried item, and can determine it as a rental item when the object no longer appears. That is, the carried item determination unit 60 can determine whether the object is a user's carried item or a rental item based on the shooting period of the object and the appearance frequency of the object.

[0210] The server device 12 may also send both the recommended information for purchased items and the recommended information for rental items to the user terminal device 20 without determining whether the object is a user's carried item or a rental item.

[0211] 〔Structural example of recommended information providing method〕

[0212] Figure 12 is a flowchart showing the order of the recommended information providing method according to the second embodiment. Figure 12 The flowchart shown is changed to the additional information analysis process S14A with respect to Figure 4 the flowchart shown.

[0213] And, Figure 12 the flowchart shown adds a carried item determination process S30, a recommended information acquisition process S32 for rental items, and a recommended information acquisition process S34 for purchased items to Figure 4 the flowchart shown.

[0214] The image acquisition process S10 and the image content analysis process S12 are the same as Figure 4 the recommended information providing method according to the first embodiment shown. In Figure 12 the additional information analysis process S14A shown, the carried item determination unit 60 uses the information of the shooting date of each image to determine whether it is a user's carried item or a rental item for each object.

[0215] Figure 13 is showing Figure 12Flowchart showing the sequence of the attached information analysis process. Figure 13 The shown flowchart Figure 8 adds a carried item determination process S220 to the shown flowchart.

[0216] In the carried item determination process S220, the carried item determination unit 60 determines whether the item is a user's carried item or a rented item for each object. After the carried item determination process S220, the coefficient derivation determination process S208 is entered. When the process of the coefficient derivation determination process S208 is implemented and the specified conditions are met, the server device 12 ends Figure 12 the attached information analysis process S14A shown.

[0217] Return Figure 12 , after the attached information analysis process S14A, the analysis end determination process S16 is entered. The analysis end determination process S16 and the disposable income estimation process S18 are the same as Figure 4 the analysis end determination process S16 and the disposable income estimation process S18 shown.

[0218] After the disposable income estimation process S18, the carried item determination process S30 is entered. In the carried item determination process S30, Figure 11 the carried item determination unit 60 shown determines whether the object is a user's carried item or a rented item.

[0219] In the carried item determination process S30, when the carried item determination unit 60 determines that the object is a rented item, it becomes a "no" determination. When it is a "no" determination, the recommended information acquisition process S32 for rented items is entered.

[0220] In the recommended information acquisition process S32 for rented items, the recommended information sending unit 58 refers to the recommended information database 18 to obtain the recommended information for rented items. After the recommended information acquisition process S32 for rented items, the recommended information sending process S20 is entered.

[0221] On the other hand, in the carried item determination process S30, when the carried item determination unit 60 determines that the object is a user's carried item, it becomes a "yes" determination. When it is a "yes" determination, the recommended information acquisition process S34 for purchased items is entered.

[0222] In the recommended information acquisition process S34 for purchased items, the recommended information sending unit 58 refers to the recommended information database 18 to obtain the recommended information for purchased items. After the recommended information acquisition process S34 for purchased items, the recommended information sending process S20 is entered.

[0223] In the recommendation information sending step S20, when the object is a user's carried item, the recommendation information sending unit 58 sends recommendation information for purchasing an item to the user terminal device 20. On the other hand, in the recommendation information sending step S20, when the object is a rented item, the recommendation information sending unit 58 sends recommendation information for the rented item to the user terminal device 20.

[0224] After the recommendation information sending step S20, the process proceeds to the end determination step S22. Figure 12 The end determination step S22 shown is the same as Figure 4 the end determination step S22 shown.

[0225] 〔Example of providing recommendation information〕

[0226] Figure 14 It is an explanatory diagram showing an example of a recommendation information display screen for a rented item. The recommendation information display screen 240 shown in this figure includes user identification information 242, recommendation information 244 for the rented item, and operation buttons 246.

[0227] Figure 14 The user identification information 242 shown is the same as Figure 10 the user identification information 142 shown. And, Figure 14 the operation buttons 246 shown are the same as Figure 10 the operation buttons 146 shown.

[0228] [Operation and effect of the second embodiment]

[0229] According to the image processing system 200 and the recommendation information providing method according to the second embodiment, the following operation and effects can be obtained.

[0230] 〔1〕

[0231] Determine whether the object is a user's carried item or a rented item. When the object is a rented item, provide information on the rented item as the recommendation information for the user. Thus, the user can obtain recommendation information related to the rented item.

[0232] 〔2〕

[0233] When the object is a user's carried item, provide information on the purchased item as the recommendation information for the user. Thus, the user can obtain recommendation information related to the purchased item.

[0234] [Image processing system according to the third embodiment]

[0235] 〔Outline〕

[0236] Next, the image processing system according to the third embodiment will be described. The image processing system 300 according to the third embodiment determines the shooting scene based on the analysis result of the image content, and recognizes an event based on the shooting scene.

[0237] The image processing system 300 estimates the disposable income of the user based on the event, and provides the user with recommendation information associated with the event of the image according to the disposable income of the user. Hereinafter, regarding the image processing system 300 according to the third embodiment, the differences from the image processing system 10 according to the first embodiment will be mainly described.

[0238] Figure 15 It is a functional block diagram of a server device applicable to the image processing system according to the third embodiment. In addition, Figure 15 extracts and shows the structure in the server device 312 whose function is different from that of Figure 2 the server device 12 shown.

[0239] 〔Object Recognition Unit〕

[0240] Figure 15 The object recognition unit 344 shown recognizes the shooting scene of the analysis target image as the object of the analysis target image. Specifically, the shooting scene of the analysis target image is determined, and the location of the analysis target image is determined.

[0241] Examples of the location of the analysis target image include tourist destinations, restaurants, and schools. The location of the analysis target can be applied with place names, addresses, latitude and longitude, etc. The place name can also be applied with common names and old place names. The object recognition unit 344 sends the recognition result of the object of the analysis target image to the type determination unit 346.

[0242] The location of the analysis target image can be applied with the shooting location of the analysis target image. The object recognition unit 344 obtains the information of the shooting location included in the attached information, and can recognize the shooting location of the analysis target image as the object of the analysis target image. In addition, the object recognition unit described in the embodiment is an example of a shooting location information acquisition unit that acquires shooting location information.

[0243] 〔Type Determination Unit〕

[0244] The type determination unit 346 uses the recognition result of the object of the analysis target image sent from the object recognition unit 344 to determine the type of the event represented by the analysis target image as the type of the object.

[0245] Examples of the type of event include travel, sightseeing in a theme park, dining in a restaurant, school activities such as sports meets, sports events such as watching a sports game, and beauty services in a beauty salon.

[0246] In Figure 15 , as an example of the type determination unit 346, a method of deriving the distance between the user's home and the location of the analysis target image, that is, the separation distance, and determining the type of event based on the separation distance is illustrated. That is, the type determination unit 346 includes a home location information acquisition unit 372, a separation distance derivation unit 374, and an event type determination unit 376.

[0247] The home location information acquisition unit 372 acquires the home information of the user. In Figure 15 , a method of acquiring the home information of the user transmitted from the user terminal device 20 is illustrated. The home location information acquisition unit 372 transmits the home information of the user to the separation distance derivation unit 374.

[0248] The separation distance derivation unit 374 uses the home information of the user and the analysis target image location information indicating the location of the analysis target image to derive the distance from the user's home to the location of the analysis target image.

[0249] The event type determination unit 376 determines the type of event based on the location of the analysis target image and the distance from the user's home to the location of the analysis target image. For example, when the distance from the user's home to the location of the analysis target image is equal to or greater than a specified distance and the same location as the analysis target image has been continuously photographed for two days or more, the event type determination unit 376 determines the type of event as a trip.

[0250] When the location of the analysis target image does not satisfy the above trip conditions but is included in a pre-specified theme park list, the event type determination unit 376 can determine the type of event as a visit to a theme park.

[0251] When a school included in a specified school list is identified as the object, the event type determination unit 376 can determine the type of event as a school activity. When an arena or the like included in a specified sports game viewing list is identified as the object, the event type determination unit 376 can determine the type of event as watching a sports game.

[0252] That is, an event type list that prescribes the relationship between the shooting scene and the type of event for each user is pre-made and stored. The event type determination unit 376 can determine the type of event for the shooting scene with reference to the event type list for each user. The event type determination unit 376 transmits the determination information of the type of event to the price range determination unit 348.

[0253] Figure 16 is a schematic diagram of a price range determination table applicable to the image processing system according to the third embodiment. The price range determination table 370 shown in this figure prescribes the price range of the object for each type of object. In Figure 16 , numerical values from 1 to 5 are used to represent the price range. RepresentFigure 17 The numerical values from 1 to 5 in the indicated price range can be converted into a numerical range representing the price range.

[0254] When the type of event is travel, Figure 16 The indicated price range determination table 370 applies to the positional relationship between the user's home in the applicable rank and the location of the image to be analyzed. The first rank is applicable within the same prefecture. The second rank is applicable within the same region. The third rank is applicable within adjacent regions.

[0255] For non - adjacent regions within the country, the fourth rank is applicable. For overseas regions, the fifth rank is applicable. Additionally, the positional relationship between the user's home and the location of the image to be analyzed can also be determined by the distance from the user's home to the location of the image to be analyzed.

[0256] When the type of event is sightseeing at a theme park, the price range determination table 370 can also apply the ranks specified in advance for each theme park. Figure 17 The letters P to T in the indicated price range determination table 370 represent the names of theme parks.

[0257] When the theme park is P, the price range determination table 370 applies the first rank. When the theme park is Q, R, S, and T, the second rank, third rank, fourth rank, and fifth rank are applicable respectively.

[0258] Figure 15 The indicated price range determination unit 348 sends price range information to the disposable income range conversion unit 350. The disposable income range conversion unit 350 converts the price range information into the user's disposable income range information. When the disposable income range conversion unit 350 converts the price range information into the user's disposable income range information, it refers to Figure 17 the indicated disposable income range conversion table 380.

[0259] Figure 17 It is a schematic diagram of the disposable income range conversion table applicable to the image processing system according to the third embodiment. In the disposable income range conversion table 380 shown in this figure, the letters A to G are used to represent the disposable income range information corresponding to the price range information. Figure 18 The indicated letters A to G can be processed as numerical values. Additionally, Figure 17 the indicated letters A to G can also be a numerical range different from Figure 7 the indicated letters A to G.

[0260] Figure 15 The indicated disposable income range conversion unit 350 sends the disposable income range information to the disposable income estimation unit 56. The disposable income estimation unit is related to Figure 2It is the same as the disposable income estimation unit 56 shown. The disposable income estimation unit estimates the user's disposable income using disposable income segment information and weighting coefficients. In addition, in Figure 15 the illustration of the disposable income estimation unit is omitted.

[0261] Figure 15 The server device 312 shown includes a frequency derivation unit and a coefficient derivation unit. The frequency derivation unit is the same as the Figure 2 frequency derivation unit 52 shown. And the coefficient derivation unit is the same as the Figure 2 coefficient derivation unit 54 shown. In addition, the illustration of the Figure 15 frequency derivation unit and the coefficient derivation unit is omitted.

[0262] The frequency derivation unit uses the shooting dates of multiple images of the same shooting scene to derive the occurrence frequency of an event. The coefficient derivation unit derives a weighting coefficient based on the occurrence frequency of the event. When deriving the weighting coefficient, the coefficient derivation unit refers to the Figure 18 weighting coefficient table 390 shown.

[0263] Figure 18 is a schematic diagram of a weighting coefficient table applicable to the image processing system according to the third embodiment. In this figure, a weighting coefficient table 390 that specifies the weighting coefficient for the event type of travel and the weighting coefficient for the event type of visiting a theme park is shown.

[0264] Figure 18 The weighting coefficient table 390 shown specifies a relatively large weighting coefficient for events with a relatively high occurrence frequency. A relatively small weighting coefficient is specified for events with a relatively low occurrence frequency.

[0265] And the weighting coefficient table 390 specifies a relatively small weighting coefficient for events at places relatively close to the user's home. On the other hand, the weighting coefficient table 390 specifies a relatively large weighting coefficient for events at places relatively far from the user's home.

[0266] Figure 15 The server device 312 shown includes a recommendation information sending unit. The recommendation information sending unit is the same as the Figure 2 recommendation information sending unit 58 shown. In addition, in Figure 15 the illustration of the recommendation information sending unit is omitted. The recommendation information sending unit sends recommendation information corresponding to the user's disposable income. The recommendation information sending unit can send recommendation information considering the type of event.

[0267] 〔Another method for obtaining the user's home location information〕

[0268] Figure 19 is a functional block diagram of a type determination unit showing another method for obtaining the user's home location information. It is provided inFigure 19 The home location information acquisition unit 372A of the type determination unit 346A shown determines the user's home by analyzing the image content of one or more images in the analysis target image group acquired by the usage image acquisition unit 40. That is, the home location information acquisition unit 372A determines an image in the analysis target image group whose shooting location is the user's home, and determines the user's home location using GPS information and the like included in the attached information of the determined image.

[0269] The home location information acquisition unit 372A extracts the feature regions of each image with respect to the analysis target image group, and can determine an image that has captured the user's home by referring to the home feature region list that defines the feature regions in the home image.

[0270] According to this method, even when the user's home location information cannot be obtained from the user terminal device 20, the user's home location information can be obtained based on the analysis result of the analysis target image group.

[0271] When the shooting frequency of the feature regions with the same features is high, the home location information acquisition unit 372A may also determine that the image has captured the user's home. Also, the number of shootings, the shooting date, and the shooting time may be considered to determine whether the image has captured the user's home.

[0272] In addition, the home described in the embodiment corresponds to an example of the reference location. As another example of the reference location, any location specified in advance by the user, such as a workplace and a school, can be applied. And the home location information acquisition unit 372A corresponds to an example of the reference location determination unit that determines the reference location.

[0273] [Recommendation Information Providing Method According to the Third Embodiment]

[0274] Next, the recommendation information providing method according to the third embodiment will be described. The order of the recommendation information providing method according to the third embodiment is Figure 5 different from the order of the recommendation information providing method according to the first embodiment in the

[0275] Figure 20 flowchart showing the order of the object type determination process applicable to the recommendation information providing method according to the third embodiment. In Figure 20 an object type determination process for determining whether the event is a trip or a visit to a theme park is illustrated.

[0276] In the event location acquisition process S300, Figure 15The object recognition unit 344 shown analyzes the image content of the analysis target image to determine the location of the event. After the event location acquisition step S300, the home location information acquisition S302 is entered.

[0277] In the home location information acquisition S302, the home location information acquisition unit 372 acquires the home location information of the user. After the home location information acquisition S302, the distance information acquisition step S304 is entered.

[0278] In the distance information acquisition step S304, the separation distance derivation unit 374 acquires information on the distance from the user's home to the location of the analysis target image. After the distance information acquisition step S304, the distance determination step S306 is entered.

[0279] In the distance determination step S306, the event type determination unit 376 determines whether the distance from the user's home to the location of the analysis target image is equal to or greater than a specified distance. In the distance determination step S306, when the event type determination unit 376 determines that the distance from the user's home to the location of the analysis target image is less than the specified distance, a "no" determination is made. When a "no" determination is made, the determination step S316 is entered.

[0280] On the other hand, in the distance determination step S306, when the event type determination unit 376 determines that the distance from the user's home to the location of the analysis target image is equal to or greater than the specified distance, a "yes" determination is made. When a "yes" determination is made, the number of days determination step S308 is entered.

[0281] In the number of days determination step S308, the event type determination unit 376 determines whether there are images of the same event taken on two or more consecutive days. In the number of days determination step S308, when the event type determination unit 376 determines that there are no images of the same event taken on two or more consecutive days, a "no" determination is made. When a "no" determination is made, the theme park list matching step S310 is entered.

[0282] On the other hand, in the number of days determination step S308, when the event type determination unit 376 determines that there are images of the same event taken on two or more consecutive days, a "yes" determination is made. When a "yes" determination is made, the travel determination step S312 is entered. In the travel determination step S312, the event type determination unit 376 determines the event of the analysis target image as a travel and sends the determination result to the price range determination unit 348. After the travel determination step S312, the end determination step S316 is entered.

[0283] In the theme park list matching process S310, the event type determination unit 376 determines whether the location of the analysis target image is included in a specified theme park list. In the theme park list matching process S310, when the event type determination unit 376 determines that the location of the analysis target image is not included in the specified theme park list, it becomes a "no" determination. When it is a "no" determination, the process proceeds to the end determination process S316.

[0284] On the other hand, in the theme park list matching process S310, when the event type determination unit 376 determines that the location of the analysis target image is included in the specified theme park list, it becomes a "yes" determination. When it is a "yes" determination, the process proceeds to the theme park determination S314.

[0285] In the theme park determination S314, the event type determination unit 376 determines the event of the analysis target image as a sightseeing event in the theme park and sends the determination result to the price range determination unit 348. After the theme park determination S314, the process proceeds to the end determination process S316.

[0286] In the end determination process S316, it is determined whether to end Figure 5 the object type determination process S104 shown. In the end determination process S316, when the type determination unit 346 determines that the type determination unit 346 does not satisfy the specified end condition, it becomes a "no" determination. When it is a "no" determination, the process proceeds to the event location acquisition process S300. Thereafter, the processes from the event location acquisition process S300 to the end determination process S316 are repeatedly executed until it becomes a "yes" determination in the end determination process S316.

[0287] On the other hand, in the end determination process S316, when the type determination unit 346 determines that the type determination unit 346 satisfies the specified end condition, it becomes a "yes" determination. When it is a "yes" determination, the type determination unit 346 ends the object type determination process S104.

[0288] [Effects of the Third Embodiment]

[0289] According to the image processing system and the recommended information providing method according to the third embodiment, the following effects can be obtained.

[0290] [1]

[0291] An event is determined as the object of the analysis target image. Thus, the disposable income of the user can be estimated based on the event.

[0292] [2]

[0293] Based on the distance between the user's home and the location of the event, the type of the event is determined. The disposable income of the user can be estimated based on the type of the event.

[0294] [Modified Example]

[0295] Next, a modified example related to the above image processing system and recommended information providing method will be described. Figure 21 It is a flowchart showing the order of the recommended information providing method related to the modified example. The recommended information providing method related to the modified example described below estimates the photographer for each analysis target image in the analysis target image group, and estimates the disposable income for each photographer. When estimating the disposable income of the user, the estimated value of the disposable income of other photographers is used.

[0296] The image acquisition step S10 is the same as Figure 4 the image acquisition step S10 shown. After the image acquisition step S10, the image removal step S400 is not entered. In the image removal step S400 not required, Figure 2 the object recognition unit 44 shown performs the same processing as the removal of the image not suitable for analysis in the analysis target image setting step S100 shown in Figure 5 . After the image removal step S400 not required, the photographer estimation step S402 is entered.

[0297] In the photographer estimation step S402, the object recognition unit 44 estimates the photographer of the analysis target image. For example, using the information of the imaging device of each image, it is estimated whether the image is taken by the user or by someone other than the user.

[0298] The object recognition unit 44 can estimate the image taken by the imaging device with a high imaging frequency as the image taken by the user. After the photographer estimation step S402, the image determination step S404 is entered.

[0299] In the image determination step S404, the object recognition unit 44 determines whether the photographer of each analysis target image is the user or someone other than the user. In the image determination step S404, when the object recognition unit 44 determines that the photographer of the analysis target image is someone other than the user, it becomes a "no" determination. When it is a "no" determination, the disposable income estimation step S406 for other photographers is entered.

[0300] In the disposable income estimation step S406 for other photographers, the disposable income estimation unit 56 etc. perform each step from Figure 4 the image content analysis step S12 shown to the disposable income estimation step S18 for the photographer other than the user, and estimate the disposable income of the photographer other than the user. After the disposable income estimation step S406, the disposable income estimation step S408 is entered.

[0301] On the other hand, in the image determination step S404, when the object recognition unit 44 determines that the photographer of the analysis target image is the user, it becomes a "yes" determination. When it is a "yes" determination, it proceeds to the disposable income estimation step S408 of the user.

[0302] In the disposable income estimation step S408 of the user, the disposable income estimation unit 56 etc. perform each step from Figure 4 the image content analysis step S12 shown to the disposable income estimation step S18 on the user, and estimate the disposable income of the user.

[0303] The disposable income estimation unit 56 etc. estimate the disposable income of the user in consideration of the estimated value of the disposable income of other photographers estimated in the disposable income estimation step S406 of other photographers. That is, the disposable income estimation unit 56 etc. prioritize the analysis target image in which the user himself / herself is the photographer and estimate the disposable income of the user.

[0304] The disposable income estimation unit 56 etc. estimate the disposable income of other photographers based on the analysis target image in which someone other than the user is the photographer, and can estimate the disposable income of the user by using the estimated value of the disposable income of other photographers.

[0305] When the disposable income estimation unit 56 etc. estimate the disposable income of the user by using the estimated value of the disposable income of other photographers, they can refer to the correlation between the user and other photographers. For example, by using the correlation relationship of age between the user and other photographers, the disposable income of the user can be estimated.

[0306] Figure 22 is a correlation diagram showing the correlation between the user and relevant persons corresponding to the age group. The correlation diagram 500 shown in this figure represents the intensity of the correlation between the user 501 and the first relevant person 502 etc. by using the distance from the user 501. The intensity of the correlation takes into account the age group of the user 501 and the age group of the first relevant person 502 etc.

[0307] The first boundary line 510 represents the boundary with an age difference of 10 years from the user 501. The relevant persons on the line of the first boundary line 510 have an age difference of 10 years from the user 501. The relevant persons inside the first boundary line 510 have an age difference of less than 10 years from the user 501.

[0308] The second boundary line 512 represents the boundary line with an age difference of 20 years from the user 501. The relevant persons on the line of the second boundary line 512 have an age difference of 20 years from the user 501. The relevant persons outside the first boundary line 510 and inside the second boundary line 512 have an age difference of more than 10 years and less than 20 years from the user 501.

[0309] The third boundary line 514 represents the boundary line with an age difference of 30 years from the user 501. The stakeholders on the third boundary line 514 have an age difference of 30 years from the user 501. The stakeholders outside the second boundary line 512 and inside the third boundary line 514 have an age difference of more than 20 years and less than 30 years from the user 501.

[0310] Figure 22 The age difference between the first stakeholder 502 and the user 501 shown is more than 20 years and less than 30 years. The age difference between the second stakeholder 504 and the user 501 is 30 years. Compared with the second stakeholder 504, the first stakeholder 502 is closer to the user 501 and has a stronger correlation.

[0311] The age difference between the third stakeholder 506 and the user 501 is more than 10 years and less than 20 years. Compared with the first stakeholder 502 and the second stakeholder 504, the third stakeholder 506 is closer to the user 501 and has a stronger correlation.

[0312] Other photographers with similar age ranges are defined as having a relatively small difference in disposable income from the user 501, so that the estimated value of the disposable income of other photographers can be used to estimate the disposable income of the user 501. And, Figure 22 The disposable income of the stakeholders relatively close to the user 501 shown in the correlation diagram 500 can also be defined as having a relatively small difference from the disposable income of the user 501.

[0313] Figure 23 It is a correlation diagram showing the correlation between the user and the stakeholders corresponding to the interests. The correlation diagram 520 shown in this figure is classified into the fourth stakeholder 530, the fifth stakeholder 532, the sixth stakeholder 534, and the seventh stakeholder 536 according to interests. Figure 23 Bicycles, wine, and lunch are exemplified as interests.

[0314] The disposable income of other photographers having the same interests as the user 501 is defined as having a relatively small difference from the disposable income of the user 501, so that the estimated value of the disposable income of other photographers can be used to estimate the disposable income of the user 501.

[0315] In this embodiment, a method of classifying the stakeholders of the user according to age range and interests is exemplified, but the correlation with the user can also be defined according to education level, place of birth, occupation, etc.

[0316] [Function and effect of the modified example]

[0317] According to the modified example described above, a photographer is estimated for a plurality of images of an analysis object, and the disposable income is estimated for each photographer. When estimating the disposable income of a user, the estimated value of the disposable income of other photographers is used. Thereby, it is possible to estimate the disposable income of the user in consideration of the estimated value of the disposable income of other photographers. Also, with reference to the correlation between the user and other photographers, the disposable income of the user can be estimated.

[0318] [Applicable Example to Network System]

[0319] Figure 1 The server device 12 shown and the image database 14 etc. may also be communicably connected via a network. And, Figure 2 The constituent elements such as the server device 12 shown may also be communicably connected via a network.

[0320] That is, the Figure 2 Each part of the server device 12 shown etc. may be applicable to a distributed configuration or a centralized configuration. Figure 1 The image processing system 10 shown etc. may also be applicable to cloud computing.

[0321] [Applicable Example to Image Management System]

[0322] Figure 1 The image processing system 10 shown etc. may also be configured as a part of an image management system. The image management system automatically analyzes the subject, the shooting scene, etc. of the images uploaded from the Figure 1 user terminal device 20 shown, and automatically performs a tagging process.

[0323] The user can perform a search of the images stored in the image database 14 using the tags assigned to arbitrary images. Also, the user can manually perform a tagging process on the images uploaded by the user.

[0324] [Applicable Example to Program]

[0325] It is possible to configure a program corresponding to the image processing system 10 and the recommended information providing method disclosed in this specification. That is, this specification discloses a program that causes a computer to implement an object recognition function, a disposable income segment conversion function, an attached information acquisition function, a frequency derivation function, a coefficient derivation function, a disposable income estimation function, and a recommended information sending function.

[0326] The object recognition function analyzes an image group including two or more analysis object images included in a plurality of images associated with a user, and recognizes the objects respectively included in the two or more analysis object images.

[0327] The disposable income segment conversion function converts the information of the object identified by the object recognition function into disposable income segment information representing the range of the user's disposable income. The additional information acquisition function acquires the additional information of the analysis object image including the shooting date information indicating the shooting date of the analysis object image for two or more analysis object images of the object identified by the object recognition function.

[0328] The frequency derivation function derives the appearance frequency of the object based on the shooting date information. The coefficient derivation function derives the weighting coefficient corresponding to the object based on the appearance frequency information indicating the appearance frequency.

[0329] The disposable income estimation function estimates the user's disposable income using the disposable income segment information and the weighting coefficient. The recommended information sending function sends recommended information associated with the object to the user based on the user's disposable income.

[0330] The embodiments of the present invention described above can be appropriately changed, added, or deleted in constituent elements without departing from the gist of the present invention. The present invention is not limited to the embodiments described above, and various modifications can be made by those skilled in the art within the technical idea of the present invention.

[0331] Symbol Explanation

[0332] 10 - Image processing system, 12 - Server device, 14 - Image database, 16 - Disposable income database, 16A - Price segment determination database, 16B - Disposable income segment conversion database, 16C - Weighting coefficient database, 18 - Recommendation information database, 19 - Manager terminal device, 20 - User terminal device, 22 - Display unit, 30 - Internet, 40 - Image acquisition unit, 42 - Attached information acquisition unit, 44 - Object recognition unit, 46 - Category determination unit, 48 - Price segment determination unit, 50 - Disposable income segment conversion unit, 52 - Frequency derivation unit, 54 - Coefficient derivation unit, 56 - Disposable income estimation unit, 58 - Recommendation information sending unit, 59 - User information acquisition unit, 60 - Carried item determination unit, 100 - Price segment determination table, 110 - Disposable income segment conversion table, 120 - Weighting coefficient table, 140 - Recommendation information display screen, 142 - User identification information, 144 - Recommendation information, 146 - Operation button, 200 - Image processing system, 212 - Server device, 240 - Recommendation information display screen, 242 - User identification information, 244 - Recommendation information, 246 - Operation button, 300 - Image processing system, 312 - Server device, 344 - Object recognition unit, 346 - Category determination unit, 346A - Category determination unit, 348 - Price segment determination unit, 350 - Disposable income segment conversion unit, 370 - Price segment determination table, 372 - Home location information acquisition unit, 372A - Home location information acquisition unit, 374 - Separation distance derivation unit, 376 - Event category determination unit, 380 - Disposable income segment conversion table, 390 - Weighting coefficient table, 500 - Correlation diagram, 501 - User, 502 - First related person, 504 - Second related person, 506 - Third related person, 510 - First boundary line, 512 - Second boundary line, 514 - Third boundary line, 520 - Correlation diagram, 530 - Fourth related person, 532 - Fifth related person, 534 - Sixth related person, 536 - Seventh related person, S10 to S408 - Each process of the recommendation information providing method.

Claims

1. An image processing system, comprising: An object recognition unit that analyzes two or more analysis target images included in a group of images including a plurality of images associated with a user, and recognizes objects respectively included in the two or more analysis target images; A disposable income segment conversion unit that converts the information of the objects recognized by the object recognition unit into disposable income segment information indicating the range of the user's disposable income; An attached information acquisition unit that acquires attached information of the two or more analysis target images in which the objects are recognized by the object recognition unit, including shooting date information indicating the shooting date of the analysis target images; A frequency derivation unit that derives the appearance frequency of the objects based on the shooting date information; A coefficient derivation unit that derives a weighting coefficient corresponding to the objects based on the appearance frequency information indicating the appearance frequency; A disposable income estimation unit that estimates the user's disposable income using the disposable income segment information and the weighting coefficient; and A recommended information sending unit that sends recommended information associated with the objects to the user according to the user's disposable income.

2. The image processing system according to claim 1, wherein The object recognition unit recognizes the articles included in the analysis target images as the objects.

3. The image processing system according to claim 2, wherein The image processing system comprises: A carried item determination unit that determines whether the object is a carried item or a rented item based on the appearance frequency of the object, When the object is a carried item of the user, the recommended information sending unit sends information on purchasing items as the recommended information, and when the object is a rented item, the recommended information sending unit sends information on renting items as the recommended information.

4. The image processing system according to any one of claims 1 to 3, wherein The image processing system comprises: A type determination unit that determines the type of the objects recognized by the object recognition unit.

5. The image processing system according to claim 4, wherein The image processing system comprises: A type storage unit that stores the relationship between the objects and the types of the objects, The type determination unit refers to the type storage unit to determine the type of the objects.

6. The image processing system according to claim 1, wherein The object recognition unit recognizes the shooting scene of the analysis target images as the objects.

7. The image processing system according to claim 6, wherein The object recognition unit determines the event corresponding to the shooting scene as the type of the objects.

8. The image processing system according to claim 6, wherein The image processing system comprises: A type determination unit that determines the type of the objects recognized by the object recognition unit.

9. The image processing system according to claim 8, wherein The image processing system comprises: A type storage unit that stores the relationship between the objects and the types of the objects, The type determination unit refers to the type storage unit to determine the type of the objects.

10. The image processing system according to claim 8, wherein the image processing system includes: a shooting location information acquisition unit that acquires information on the shooting location of the analysis target image, the type determination unit determines an event corresponding to the object based on the positional relationship between the reference location of the user corresponding to the image group and the shooting location of the analysis target image.

11. The image processing system according to claim 10, wherein the type determination unit determines the type of the event corresponding to the object based on the distance from the reference location to the shooting location of the analysis target image.

12. The image processing system according to claim 10, wherein the shooting location information acquisition unit uses the attached information acquisition unit to acquire the attached information including the shooting location information indicating the shooting location of the analysis target image.

13. The image processing system according to claim 10, wherein the shooting location information acquisition unit analyzes the analysis target image and determines the shooting location of the analysis target image.

14. The image processing system according to any one of claims 10 to 13, wherein the attached information acquisition unit acquires attached information including information on the reference location.

15. The image processing system according to any one of claims 10 to 13, wherein the image processing system includes: a reference location determination unit that analyzes the analysis target image and determines the reference location.

16. The image processing system according to any one of claims 1 to 3, 6 to 13, wherein the image processing system includes: a price range determination unit that determines the price range of the object corresponding to the type of the object.

17. The image processing system according to claim 16, wherein the image processing system includes: a price range storage unit that stores the relationship between the type of the object and the price range, the price range determination unit refers to the price range storage unit and determines the price range of the object.

18. The image processing system according to claim 17, wherein the image processing system includes: a disposable income range storage unit that stores the relationship between the price range and the disposable income range indicating the range of the disposable income of the user, the disposable income range conversion unit refers to the disposable income range storage unit and converts the price range information indicating the price range into disposable income range information indicating the disposable income range of the user.

19. The image processing system according to any one of claims 1 to 3, 6 to 13, wherein the image processing system includes: a coefficient storage unit that stores the relationship between the type of the object and the weighting coefficient, the coefficient derivation unit refers to the coefficient storage unit and derives the weighting coefficient applicable to the object.

20. The image processing system according to any one of claims 1 to 3, 6 to 13, wherein the image processing system includes: a user information acquisition unit that acquires user information for identifying the user.

21. An image processing method, which includes: an object recognition step of analyzing two or more analysis target images included in an image group including a plurality of images associated with a user, and recognizing objects respectively included in the two or more analysis target images; a disposable income segment conversion step of converting the information of the object identified in the object identification step into disposable income segment information indicating a range of disposable income of the user; an incidental information acquisition step of acquiring incidental information of the analysis target images including shooting date information indicating shooting dates of the analysis target images, with respect to the two or more analysis target images in which the object is recognized in the object recognition step; A frequency derivation step of deriving the appearance frequency of the object based on the photographing date information; a coefficient derivation step of deriving a weighting coefficient corresponding to the object based on the appearance frequency information indicating the appearance frequency; a disposable income estimating step of estimating the disposable income of the user using the disposable income segment information and the weighting coefficient; and The recommendation information sending step sends recommendation information related to the object to the user based on the disposable income of the user.

22. A recording medium having a program recorded thereon, the program causing a computer to implement the following functions: an object recognition function for analyzing two or more analysis target images included in an image group including a plurality of images associated with a user, and recognizing objects respectively included in the two or more analysis target images; a disposable income segment conversion function that converts the information of the object identified by the object identification function into disposable income segment information indicating a range of disposable income of the user; an incidental information acquisition function for acquiring incidental information of the analysis target images including shooting date information indicating shooting dates of the analysis target images, with respect to two or more analysis target images in which the object is recognized using the object recognition function; A frequency export function, which exports the occurrence frequency of the object according to the shooting date information; A coefficient derivation function for deriving a weighting coefficient corresponding to the object based on the occurrence frequency information indicating the occurrence frequency; a disposable income estimation function, using the disposable income segment information and the weighting coefficient to estimate the disposable income of the user; and The recommendation information sending function sends recommendation information related to the object to the user according to the disposable income of the user.

Images