Recommendation information presentation device, operation method for recommendation information presentation device, and operation program for recommendation information presentation device

Abstract

To provide a recommendation information presentation device capable of presenting recommendation information with a high probability that a user has an interest in without giving burdens to the user, an operation method for the recommendation information presentation device, and an operation program for the recommendation information presentation device.SOLUTION: A CPC of an image management sever comprises an estimation unit, an information acquisition unit, and a distribution control unit. The estimation unit, if a plurality of images serving as the basis for estimating a future event that a user will experience after a set period among images within the set period which the user has acquired is equal to or greater than a preset first threshold, estimates that the user will experience the future event after the set period. The information acquisition unit generates recommendation information corresponding to the estimated future event by selecting recommendation information corresponding to the estimated future event from among a plurality of recommendation information previously registered in a recommendation information DB. The distribution control unit presents the recommendation information to the user by distributing the recommendation information to a user terminal.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The technology of the present disclosure relates to a recommendation information presentation device, a method for operating the recommendation information presentation device, and an operation program for the recommendation information presentation device.

Background Art

[0002] Recommendation information suitable for a user is presented. For example, Patent Document 1 describes a technology for estimating recommendation information that a user will be interested in from schedule information of future events registered by the user and presenting the estimated recommendation information to the user. In Patent Document 1, for example, when a child's birthday is registered as schedule information, information on a toy bargain sale is presented as recommendation information.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] At present, user terminals with camera functions such as smartphones and tablet terminals are explosively popular, and most users can easily take pictures with the user terminals. In the images thus obtained by the user, there may be a subject that serves as a basis for estimating an event that the user will experience in the future. For example, a bridal salon is shown in the image of a user who is about to get married.

[0005] Therefore, the present inventors have conceived a method of saving the labor of the user who registers schedule information by estimating recommendation information that the user will be interested in based on the images obtained by the user, rather than the schedule information described in Patent Document 1. However, if the estimation accuracy of the recommendation information based on the images is poor, incorrect recommendation information will be presented, resulting in the loss of business opportunities.

[0006] One embodiment of the technology according to the present disclosure provides a recommendation information presentation device, an operation method of the recommendation information presentation device, and an operation program of the recommendation information presentation device that can present recommendation information that the user is highly likely to be interested in without causing the user trouble.

Means for Solving the Problems

[0007] The recommendation information presentation device of the present disclosure includes a processor and a memory connected to or built in the processor. When there are a plurality of images that serve as a basis for estimating future events, which are events that the user will experience after the period, in the images obtained by the user within a preset period and are equal to or greater than a preset first threshold, the processor estimates that the user will experience a future event after the period, generates recommendation information corresponding to the estimated future event, and presents the recommendation information to the user.

[0008] Preferably, the processor determines whether an image is an image that serves as a basis for estimating a future event based on at least one of the analysis result of the image and the information attached to the image.

[0009] Preferably, when there are all images related to specific related events, which are at least two related events related to a future event, and are equal to or greater than a preset second threshold, the processor estimates that the user will experience a future event after the period.

[0010] When the adoption frequency of the recommendation information by the user satisfies a preset condition, it is preferable for the processor to stop presenting the recommendation information.

[0011] It is preferable for the processor to preferentially present the recommendation information that is adopted relatively more by other users.

[0012] Preferably, the other users are users whose attributes are similar to or match those of the user presenting the recommendation information.

[0013] Preferably, the other users are users whose order of event experience is similar to or matches that of the user presenting the recommendation information.

[0014] It is preferable for the processor to select the recommendation information corresponding to the estimated future event from among a plurality of pre-registered recommendation information.

[0015] The operation method of the recommendation information presentation device of the present disclosure includes, when there are a plurality of images serving as the basis for estimating a future event, which is an event that the user will experience after the period, in the images obtained by the user within a preset period, and the number of such images is equal to or more than a preset first threshold value, estimating that the user will experience a future event after the period, generating recommendation information corresponding to the estimated future event, and presenting the recommendation information to the user.

[0016] The operation program of the recommendation information presentation device of the present disclosure causes a computer to execute a process including, when there are a plurality of images serving as the basis for estimating a future event, which is an event that the user will experience after the period, in the images obtained by the user within a preset period, and the number of such images is equal to or more than a preset first threshold value, estimating that the user will experience a future event after the period, generating recommendation information corresponding to the estimated future event, and presenting the recommendation information to the user.

Advantages of the Invention

[0017] According to the technology of the present disclosure, it is possible to provide a recommendation information presentation device, a method for operating the recommendation information presentation device, and an operation program for the recommendation information presentation device that can present recommendation information that the user is likely to be interested in without causing the user any trouble.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Mode for Carrying Out the Invention

[0019] [First Embodiment] As an example, as shown in FIG. 1, the image management system 2 includes an image management server 10 and a plurality of user terminals 11. The image management server 10 and the user terminals 11 are connected to be communicable with each other via a network 12. The network 12 is, for example, a WAN (Wide Area Network) such as the Internet or a public communication network.

[0020] The image management server 10 is, for example, a server computer, a workstation, etc., and is an example of the "recommendation information presentation device" according to the technology of the present disclosure. The user terminal 11 is a terminal possessed by each user 13. The user terminal 11 has at least a function of reproducing and displaying an image 22 (see FIG. 2 etc.) and a function of transmitting the image 22 to the image management server 10. The user terminal 11 is, for example, a smartphone, a tablet terminal, a personal computer, etc.

[0021] As an example, as shown in FIG. 2, an image database (hereinafter abbreviated as DB (Data Base)) server 20 and a recommendation information DB server 21 are connected to the image management server 10 via a network (not shown) such as a LAN (Local Area Network). The image management server 10 transmits the image 22 from the user terminal 11 to the image DB server 20. The image DB server 20 has an image DB 23. The image DB server 20 accumulates and manages the image 22 from the image management server 10 in the image DB 23. Also, the image DB server 20 transmits the image 22 accumulated in the image DB 23 to the image management server 10 in response to a request from the image management server 10.

[0022] The recommendation information DB server 21 has a recommendation information DB 24. The recommendation information DB 24 stores recommendation information 25. The recommendation information 25 is information such as products, stores, and facilities recommended to the user 13. The recommendation information 25 is pre-registered by an employee of the product seller or an employee of the store or facility. The recommendation information DB server 21 transmits the recommendation information 25 of the recommendation information DB 24 to the image management server 10 in response to a request from the image management server 10. The image management server 10 distributes the recommendation information 25 to the user terminal 11.

[0023] As an example, as shown in FIG. 3, a plurality of image folders 30 are provided in the image DB 23. The image folder 30 is a folder assigned one by one to each user 13 and is a folder unique to one user 13. Therefore, the number of image folders 30 is provided according to the number of users 13. The user ID (Identification Data) for uniquely identifying the user 13, such as [U0001] and [U0002], is associated with the image folder 30.

[0024] The image folder 30 stores the images 22 owned by the user 13. The images 22 owned by the user 13 include images taken by the user 13 using the camera function of the user terminal 11. Also, the images 22 owned by the user 13 include images taken using a digital camera other than the user terminal 11. Furthermore, the images 22 owned by the user 13 include images received by the user 13 from other users 13 such as friends and family, images downloaded by the user 13 from an Internet site, and images read by the user 13 using a scanner. The images 22 in the image folder 30 are periodically synchronized with the images 22 stored locally in the user terminal 11.

[0025] In the image folder 30, the attribute information 31 of the user 13 and the face image 32 are associated. The attribute information 31 and the face image 32 are registered by the user 13. The attribute information 31 includes the date of birth, gender, residential area, and family composition, etc. of the user 13. The residential area is a combination of prefecture and city / town / village. The face image 32 is an image in which the faces of the user 13 himself / herself, the family and / or relatives of the user 13, and the lover and / or friends of the user 13, etc. are shown. In the face image 32, the relationship with the user 13 such as "parent", "grandchild", "lover", and "friend" is also registered together.

[0026] As shown in FIG. 4 as an example, the recommendation information DB 24 is divided into a plurality of categories 33, and a plurality of pieces of recommendation information 25 are stored in each category 33. The category 33 is provided for each future event which is an event that the user 13 will experience. The future events are so-called life events such as "employment", "marriage", etc. exemplified.

[0027] The recommendation information 25 includes the recommendation information 25 of products and the recommendation information 25 of stores or facilities. In the recommendation information 25 of products, an image of the product, the product name, the desired retail price, the seller, and related events related to the product, etc. are registered. In the recommendation information 25 of stores or facilities, an image of the store or facility, the store or facility name, the address, the main products, and related events related to the store or facility, etc. are registered. The related events are events related to the future events. For example, when the future event is "marriage", the related events are "venue inspection", "dress fitting", and "ring purchase", etc. (see also FIG. 8). In FIG. 4, a wedding information magazine is exemplified as a product, and a jewelry store is exemplified as a store or facility.

[0028] As an example, as shown in FIG. 5, the computers constituting the image management server 10 and the user terminal 11 basically have the same configuration and include a storage 40, a memory 41, a CPU (Central Processing Unit) 42, a communication unit 43, a display 44, and an input device 45. These are interconnected via a bus line 46.

[0029] The storage 40 is a hard disk drive built into the computers constituting the image management server 10 and the user terminal 11, or connected via a cable or network. Alternatively, the storage 40 is a disk array with multiple hard disk drives installed. The storage 40 stores control programs such as an operating system, various application programs (hereinafter abbreviated as AP (Application Program)), and various data associated with these programs. Note that a solid state drive may be used instead of the hard disk drive.

[0030] The memory 41 is a work memory for the CPU 42 to execute processing. The CPU 42 loads the programs stored in the storage 40 into the memory 41 and executes processing according to the programs. Thereby, the CPU 42 comprehensively controls each part of the computer. The CPU 42 is an example of the "processor" according to the technology of the present disclosure. Note that the memory 41 may be built into the CPU 42.

[0031] The communication unit 43 is a network interface that controls the transmission of various information via a network 12 or the like. The display 44 displays various screens. The various screens are provided with an operation function by a GUI (Graphical User Interface). The computers constituting the image management server 10 and the user terminal 11 receive input of operation instructions from the input device 45 through the various screens. The input device 45 is a keyboard, a mouse, a touch panel, or the like.

[0032] In the following description, each part of the computer constituting the image management server 10 is assigned a subscript "A" as a symbol, and each part of the computer constituting the user terminal 11 is assigned a subscript "B" as a symbol for distinction.

[0033] As an example, as shown in FIG. 6, the storage 40A of the image management server 10 stores an operation program 50. The operation program 50 is an AP for causing the computer constituting the image management server 10 to function as a "recommendation information presentation device" according to the technology of the present disclosure. That is, the operation program 50 is an example of an "operation program of a recommendation information presentation device" according to the technology of the present disclosure. In addition to the operation program 50, the storage 40A also stores a content analysis machine learning model (hereinafter abbreviated as a content analysis model) 51, estimated reference information 52, and estimation conditions 53.

[0034] When the operation program 50 is started, the CPU 42A of the image management server 10 functions as a request reception unit 60, an image acquisition unit 61, a read / write (hereinafter abbreviated as RW) control unit 62, an analysis unit 63, an estimation unit 64, an information acquisition unit 65, and a distribution control unit 66 in cooperation with the memory 41 and the like.

[0035] The request reception unit 60 receives various requests from the user terminal 11. For example, the request reception unit 60 receives a recommendation information distribution request 70. The recommendation information distribution request 70 requests the distribution of the recommendation information 25. The recommendation information distribution request 70 is automatically transmitted from the user terminal 11 every preset period (hereinafter referred to as a set period). The set period is, for example, one week, two weeks, one month, or half a year.

[0036] The recommendation information distribution request 70 includes a user ID and a terminal ID. The terminal ID is the ID of the user terminal 11 that transmitted the recommendation information distribution request 70. The request reception unit 60 outputs the user ID in the recommendation information distribution request 70 to the image acquisition unit 61. In addition, the request reception unit 60 outputs the terminal ID in the recommendation information distribution request 70 to the distribution control unit 66.

[0037] When a recommendation information distribution request 70 is input from the reception unit 60 for requests, the image acquisition unit 61 transmits an image acquisition request 71 to the image DB server 20. The image acquisition request 71 is a copy of the user ID of the recommendation information distribution request 70, and is a request for the images 22 obtained by the user 13 with the user ID within the set period. For example, when the set period is two weeks and the date when the image acquisition request 71 is transmitted is February 4th, the image acquisition request 71 is a request for the images 22 obtained by the user 13 from January 22nd, two weeks before February 4th, to February 4th.

[0038] The image DB server 20 reads out the images 22 corresponding to the image acquisition request 71 from the image DB 23, and transmits the read images 22 to the image management server 10. The image acquisition unit 61 acquires the images 22 transmitted from the image DB server 20 in response to the image acquisition request 71. The image acquisition unit 61 outputs the acquired images 22 to the analysis unit 63. Although not shown in the figure, in addition to the images 22, the image acquisition unit 61 also acquires the attribute information 31 and the face images 32. The image acquisition unit 61 outputs the attribute information 31 to the information acquisition unit 65 and outputs the face images 32 to the estimation unit 64.

[0039] The RW control unit 62 controls the storage of various information in the storage 40A and the reading of various information in the storage 40A. For example, the RW control unit 62 reads out the content analysis model 51 from the storage 40A, and outputs the read content analysis model 51 to the analysis unit 63. Also, the RW control unit 62 reads out the estimation reference information 52 and the estimation conditions 53 from the storage 40A, and outputs the read estimation reference information 52 and estimation conditions 53 to the estimation unit 64.

[0040] The analysis unit 63 generates content analysis information 72 from the images 22 using the content analysis model 51. The content analysis information 72 is information obtained by analyzing the content of the images 22 (see also FIG. 7). The analysis unit 63 outputs the content analysis information 72 to the estimation unit 64. The content analysis information 72 is an example of the "analysis result" according to the technology of the present disclosure.

[0041] Based on the estimated reference information 52 and the content analysis information 72, the estimation unit 64 determines whether the image 22 from the image DB server 20 is an image 22 that serves as the basis for estimating future events. Then, the estimation unit 64 determines whether the image 22 determined to be the basis for estimating future events satisfies the estimation condition 53. If it is determined that the image 22 determined to be the basis for estimating future events satisfies the estimation condition 53, the estimation unit 64 estimates that the user 13 will experience the future event after the set period. The estimation unit 64 outputs information (hereinafter referred to as future event information) 73 of the future event estimated that the user 13 will experience after the set period to the information acquisition unit 65.

[0042] The information acquisition unit 65 transmits an information acquisition request 74 for requesting recommendation information 25 corresponding to the future event information 73 to the recommendation information DB server 21. The recommendation information DB server 21 reads out the recommendation information 25 requested in the information acquisition request 74 from the recommendation information DB 24 and transmits the read recommendation information 25 to the image management server 10. The information acquisition unit 65 acquires the recommendation information 25 transmitted from the recommendation information DB server 21. In this way, the information acquisition unit 65 selects the recommendation information 25 corresponding to the future event information 73 from among a plurality of pieces of recommendation information 25 registered in advance in the recommendation information DB 24. The information acquisition unit 65 outputs the acquired recommendation information 25 to the distribution control unit 66. Note that the selection of the recommendation information 25 by the information acquisition unit 65 is an example of "generating recommendation information" and "generating recommendation information" according to the technology of the present disclosure.

[0043] The distribution control unit 66 performs control to distribute the recommendation information 25 from the information acquisition unit 65 to the user terminal 11 that is the transmission source of the recommendation information distribution request 70. At this time, the distribution control unit 66 identifies the user terminal 11 that is the transmission source of the recommendation information distribution request 70 based on the terminal ID from the request reception unit 60. By distributing the recommendation information 25 to the user terminal 11, the distribution control unit 66 presents the recommendation information 25 to the user 13.

[0044] As an example, as shown in FIG. 7, the analysis unit 63 inputs the image 22 into the content analysis model 51 and causes the content analysis model 51 to output content analysis information 72. The content analysis model 51 is, for example, a combination of a convolutional neural network (CNN) that extracts feature amounts of the image 22 and a recurrent neural network (RNN) that extracts feature amounts of words. The content analysis model 51 outputs a plurality of words representing the content of the input image 22 as the content analysis information 72. Note that a short sentence (caption) representing the content of the input image 22 may be output as the content analysis information 72.

[0045] Further, the analysis unit 63 determines whether or not a person in whom the face image 32 is registered, such as the user 13 himself / herself, the family and / or relatives of the user 13, and the lover and / or friend of the user 13, is shown in the image 22. When it is determined that a person in whom the face image 32 is registered is shown in the image 22, a word indicating that fact is included in the content analysis information 72. For example, when the user 13 himself / herself is shown in the image 22, the word "oneself" is included in the content analysis information 72. When the lover of the user 13 is shown in the image 22, the word "lover" is included in the content analysis information 72.

[0046] FIG. 7 shows an example of the image 22 that is a photograph of a pre-wedding face-to-face meeting of both families. An example is shown in which content analysis information 72 having the content of "oneself, parents, lover, couple, formal dress, dinner, restaurant, glass, alcohol, smiling face,..." is output for the image 22.

[0047] As an example, as shown in FIG. 8, the estimated reference information 52 is prepared for each future event. In the estimated reference information 52, keywords for each related event of the future event are registered. The related events include events that the user 13 will generally experience prior to the future event. Also, the related events are registered according to the general order that the user 13 will follow. For this reason, not all users 13 will necessarily experience all related events. Also, not all users 13 will necessarily experience the related events in this order.

[0048] In FIG. 8, the estimated reference information 52 of the future event "marriage" is illustrated. Related events in this case include "face-to-face meeting", "betrothal gift presentation", "inspection of the wedding venue", "trying on wedding dresses", "pre-wedding photo shoot", and "ring purchase", etc. Also, as keywords, for example, for the related event "face-to-face meeting", there are "oneself, parents, lover, siblings, formal dress, dinner, restaurant, Japanese restaurant, alcohol...", and for the related event "inspection of the wedding venue", there are "shrine, church, wedding venue, cuisine, food tasting...", etc.

[0049] As an example, as shown in FIGS. 9 and 10, the estimation unit 64 collates the keywords registered for each related event of the estimated reference information 52 of each future event with the words included in the content analysis information 72. Then, it checks whether the collation result satisfies a preset condition for each future event and for each related event. If there is a related event whose collation result satisfies the condition, the estimation unit 64 determines that the event shown in the image 22 is the said related event. The image 22 thus determined to depict the related event is, that is, the image 22 that serves as the basis for the estimation of the future event. Note that the condition is, for example, that the number of words that match between the keywords registered in the estimated reference information 52 and the words included in the content analysis information 72 is 5 or more. Alternatively, the condition may be, for example, that the number of words that match between the keywords registered in the estimated reference information 52 and the words included in the content analysis information 72 is 70% or more of the number of keywords registered in the estimated reference information 52.

[0050] In FIG. 9, an image 22 is exemplified in which the state of face alignment shown in FIG. 7 is photographed. In this case, among the keywords registered in the related event "face alignment" of the estimated reference information 52 of the future event "marriage" and the words included in the content analysis information 72, "oneself", "parents", "lover", "formal dress", "dining", "restaurant", and "alcohol" etc. match. For this reason, the estimation unit 64 determines that the event captured in the image 22 is "face alignment".

[0051] In FIG. 10, an image 22 is exemplified in which the state of betrothal is photographed. The content analysis information 72 is in the content of "oneself, parents, lover, couple, formal dress, alcove, sitting on one's heels, gift wrapping, folding fan...". In this case, among the keywords registered in the related event "betrothal" of the estimated reference information 52 of the future event "marriage" and the words included in the content analysis information 72, "oneself", "parents", "lover", "formal dress", "alcove", "gift wrapping", and "folding fan" etc. match. For this reason, the estimation unit 64 determines that the event captured in the image 22 is "betrothal".

[0052] As an example, as shown in Table 80 of FIGS. 11 and 12, the estimation unit 64 counts the total number of images 22 determined to depict related events, that is, the images 22 that are the basis for the estimation of future events. The estimation unit 64 determines whether the total number of images 22 that are the basis for the estimation of future events satisfies the estimation condition 53. The estimation condition 53 is that the total number of images 22 that are the basis for the estimation of future events is equal to or more than the first threshold value. In FIGS. 11 and 12, the case where the first threshold value is 5 (estimation condition: total number of images ≥ 5) is exemplified.

[0053] When the total number of images 22 that serve as the basis for the estimation of future events is equal to or greater than the first threshold value, the estimation unit 64 estimates that the user 13 will experience a future event after the set period. Then, the future event information 73 including the estimated future event and the related event that serves as the basis for the estimation is output to the information acquisition unit 65. On the other hand, when the total number of images 22 that serve as the basis for the estimation of future events is less than the first threshold value, the estimation unit 64 estimates that the user 13 will not experience a future event after the set period. In this case, the estimation unit 64 does not output the future event information 73 to the information acquisition unit 65.

[0054] In FIG. 11, an example is shown where the total number of images 22 that serve as the basis for the estimation of future events is 7, which is determined to have captured the related event "face matching". In this case, the estimation unit 64 estimates that the user 13 will get married after the set period. Then, the future event information 73 including the future event "marriage" and the related event "face matching" is output to the information acquisition unit 65.

[0055] In FIG. 12, an example is shown where the total number of images 22 that serve as the basis for the estimation of future events is 8, which is the sum of 4 images determined to have captured the related event "betrothal gift presentation" and 4 images determined to have captured the related event "pre-wedding venue visit". In this case, the estimation unit 64 estimates that the user 13 will get married after the set period. Then, the future event information 73 including the future event "marriage" and the related events "betrothal gift presentation" and "pre-wedding venue visit" is output.

[0056] As shown in FIGS. 13 and 14 as an example, the information acquisition unit 65 generates an information acquisition request 74 based on the attribute information 31 and the future event information 73. More specifically, the information acquisition unit 65 generates an information acquisition request 74 in which areas other than the area corresponding to the residential area of the attribute information 31, the future event of the future event information 73, and the related event of the future event information 73 are registered. For this reason, the information acquisition request 74 is the recommendation information 25 stored in the category 33 of the future event of the future event information 73, in which the area corresponding to the residential area of the attribute information 31 is registered and areas other than the related event of the future event information 73 are registered.

[0057] In FIG. 13, the case where the residential area of the attribute information 31 is "Minato Ward, Tokyo" and the future event information 73 is as shown in FIG. 11 is illustrated. In this case, the information acquisition unit 65 generates an information acquisition request 74 in which "Tokyo Kanto" is registered as the area corresponding to the residential area of the attribute information 31, "marriage" is registered as the future event, and "other than face-to-face" is registered as the related event.

[0058] In FIG. 14, the case where the residential area of the attribute information 31 is "Sakai City, Osaka Prefecture" and the future event information 73 is as shown in FIG. 12 is illustrated. In this case, the information acquisition unit 65 generates an information acquisition request 74 in which "Osaka Prefecture Kansai" is registered as the area corresponding to the residential area of the attribute information 31, "marriage" is registered as the future event, and "other than inspection of the wedding hall for dowry" is registered as the related event.

[0059] As shown in FIG. 15 as an example, an image viewing AP 85 is stored in the storage 40B of the user terminal 11. When the image viewing AP 85 is executed and a web browser dedicated to the image viewing AP 85 is launched, the CPU 42B of the user terminal 11 functions as a browser control unit 90 in cooperation with the memory 41 and the like. The browser control unit 90 controls the operation of the web browser.

[0060] The browser control unit 90 receives various operation instructions input from the input device 45B by the user 13 through various screens. The browser control unit 90 transmits a request according to the operation instruction or the like to the image management server 10. For example, the browser control unit 90 transmits a recommendation information distribution request 70 to the image management server 10 every set period. In addition, the browser control unit 90 generates various screens such as an image list display screen 95 (see FIG. 16 etc.) for displaying the images 22 in a list and displays them on the display 44B.

[0061] FIG. 16 shows an example of the image list display screen 95. On the image list display screen 95, thumbnail images 96 obtained by cutting out the images 22 in a square shape are arranged at equal intervals in the vertical and horizontal directions.

[0062] When the recommendation information 25 is distributed from the image management server 10, a display button 97 for displaying the recommendation information 25 is provided at the lower part of the image list display screen 95. When the display button 97 is selected, as shown in FIG. 17 as an example, the browser control unit 90 displays a list 98 of the recommendation information 25 on the image list display screen 95. The recommendation information 25 in the list 98 is selectable. When the recommendation information 25 is selected, the entire content of the recommendation information 25 is enlarged and displayed.

[0063] A non-display button 99 is provided at the upper part of the list 98. When the non-display button 99 is selected, the browser control unit 90 makes the list 98 non-displayed and returns the image list display screen 95 to the display state shown in FIG. 16.

[0064] In FIG. 17, an example is shown in which the future event estimated to be experienced by the user 13 is "marriage", and the recommendation information 25 of a wedding information magazine and the recommendation information 25 of a jewelry store etc. are displayed in the list 98.

[0065] Next, regarding the operation of the above configuration, as an example, it will be described with reference to the flowchart shown in FIG. 18. When the operation program 50 is started, the CPU 42A of the image management server 10 functions as a request reception unit 60, an image acquisition unit 61, an RW control unit 62, an analysis unit 63, an estimation unit 64, an information acquisition unit 65, and a distribution control unit 66, as shown in FIG. 6.

[0066] Also, when the image viewing AP 85 is started, the CPU 42B of the user terminal 11 functions as a browser control unit 90, as shown in FIG. 15.

[0067] A recommendation information distribution request 70 is issued from the browser control unit 90 at each set period. The recommendation information distribution request 70 is transmitted from the user terminal 11 to the image management server 10.

[0068] As shown in FIG. 18, when the request reception unit 60 receives the recommendation information distribution request 70 from the user terminal 11 (YES in step ST100), an image acquisition request 71 for requesting the image 22 obtained by the user 13 within the set period is transmitted from the image acquisition unit 61 to the image DB server 20 (step ST110). Then, the image 22 transmitted from the image DB server 20 in response to the image acquisition request 71 is acquired by the image acquisition unit 61 (step ST120). The image 22 is output from the image acquisition unit 61 to the analysis unit 63.

[0069] As shown in FIG. 7, in the analysis unit 63, content analysis information 72 is generated from the image 22 using the face image 32 and the content analysis model 51 (step ST130). The content analysis information 72 is output from the analysis unit 63 to the estimation unit 64.

[0070] As shown in FIGS. 9 and 10, in the estimation unit 64, the keywords registered in the estimation reference information 52 are collated with the words included in the content analysis information 72. Then, based on the collation result, it is determined whether the image 22 from the image DB server 20 is an image 22 that serves as the basis for estimating future events (step ST140).

[0071] As shown in FIGS. 11 and 12, in the estimation unit 64, the total number of images 22 that serve as the basis for estimating future events is counted (step ST150). Then, the total number of images 22 that serve as the basis for estimating future events is compared with the first threshold value of the estimation condition 53.

[0072] When the total number of images 22 that serve as the basis for estimating future events is equal to or greater than the first threshold value (YES in step ST160), the estimation unit 64 estimates that the user 13 will experience a future event after the set period, and future event information 73 is generated (step ST170). The future event information 73 is output from the estimation unit 64 to the information acquisition unit 65.

[0073] As shown in FIGS. 13 and 14, an information acquisition request 74 corresponding to the attribute information 31 and the future event information 73 is transmitted from the information acquisition unit 65 to the recommendation information DB server 21 (step ST180). Then, the recommendation information 25 transmitted from the recommendation information DB server 21 in response to the information acquisition request 74 is acquired by the information acquisition unit 65 (step ST190). Thereby, the recommendation information 25 corresponding to the estimated future event is selected. The recommendation information 25 is output from the information acquisition unit 65 to the distribution control unit 66.

[0074] Under the control of the distribution control unit 66, the recommendation information 25 is distributed to the user terminal 11 that is the transmission source of the recommendation information distribution request 70 (step ST200).

[0075] In the user terminal 11, as shown in FIG. 17, the distributed recommendation information 25 is displayed and provided for the user 13 to view. The user 13 makes a plan to go to the store or facility of the recommendation information 25, or considers purchasing the product of the recommendation information 25.

[0076] As described above, the CPU 42A of the image management server 10 includes an estimation unit 64, an information acquisition unit 65, and a distribution control unit 66. When there are a plurality of images 22 serving as the basis for estimating future events that the user 13 will experience after the set period among the images 22 obtained by the user 13 within the set period and the number of such images 22 is equal to or greater than a preset first threshold, the estimation unit 64 estimates that the user 13 will experience a future event after the set period. The information acquisition unit 65 generates recommendation information 25 corresponding to the estimated future event by selecting the recommendation information 25 corresponding to the estimated future event from among a plurality of pieces of recommendation information 25 registered in advance in the recommendation information DB 24. The distribution control unit 66 presents the recommendation information 25 to the user 13 by distributing the recommendation information 25 to the user terminal 11. Therefore, it is possible to present the recommendation information 25 that the user 13 is highly likely to be interested in without having the user 13 go through the trouble of registering schedule information as in the technique described in Patent Document 1.

[0077] For example, consider a case where an older brother is about to get married and a younger sister who has no plans to get married takes pictures of the betrothal ceremony of her older brother. In this case, it is considered that the number of images 22 serving as the basis for estimating future events among the images 22 owned by the younger sister will be relatively small. In such a case, if the setting is such that if there is even one image 22 serving as the basis for estimating a future event, it is estimated that the user 13 will experience a future event after the period, then it will be erroneously estimated that the younger sister will get married, and recommendation information 25 regarding marriage will be presented to the younger sister who has no plans to get married. However, in the technique of the present disclosure, since it is estimated that the user 13 will experience a future event after the period when there are a plurality of images 22 serving as the basis for estimating the future event and the number of such images 22 is equal to or greater than the first threshold, the risk of making such an erroneous estimation can be reduced. As a result, it is possible to suppress the occurrence of inconveniences such as inappropriate recommendation information 25 being presented and missing out on business opportunities.

[0078] Based on the content analysis information 72, the estimation unit 64 determines whether the image 22 is the basis for estimating future events. Thus, it is possible to determine whether the image 22 is the basis for estimating future events without causing trouble to the user 13.

[0079] The information acquisition unit 65 selects the recommendation information 25 corresponding to the estimated future event from among a plurality of pieces of recommendation information 25 registered in advance in the recommendation information DB 24. Thus, the recommendation information 25 can be easily generated.

[0080] The mode shown in FIG. 19 may be applied. As an example, as shown in FIG. 19, in this mode, the shooting location is specified from the shooting position information 110 attached to the image 22, and the name of the store or facility at the specified shooting location is included in the content analysis information 72. The shooting position information 110 is, for example, the longitude, latitude, and altitude acquired by the GPS (Global Positioning System) function mounted on the user terminal 11. The shooting position information 110 is an example of the "information attached to the image" according to the technology of the present disclosure. FIG. 19 shows an example in which the name of the facility "Fuji Church" at the shooting location specified from the shooting position information 110 is included in the content analysis information 72 in the image 22 that captured the state of the pre-shot.

[0081] Also, the mode shown in FIG. 20 may be applied. As an example, as shown in FIG. 20, in this mode, based on the tag information 112 attached to the image 22, it is determined whether the image 22 is the basis for estimating future events. The tag information 112 is a word representing the content of the image 22. The tag information 112 is, for example, input by the user 13 operating the input device 45B of the user terminal 11. The tag information 112 is, like the shooting position information 110, an example of the "information attached to the image" according to the technology of the present disclosure. FIG. 20 illustrates a case where the estimation unit 64 determines that the event shown in the image 22 is a "betrothal" based on the "betrothal of both Fuji and Ashigara families" registered in the tag information 112. Note that the words in the content analysis information 72 may be registered as the tag information 112.

[0082] As shown in FIGS. 19 and 20, in addition to, or instead of, the content analysis information 72 output by the content analysis model 51, it may be determined whether the image 22 is an image that serves as a basis for estimating a future event based on information attached to the image 22 such as the shooting position information 110 and the tag information 112. By doing so, the reliability of the determination as to whether the image 22 is an image that serves as a basis for estimating a future event can be enhanced. Further, when using the tag information 112, the content analysis model 51 and the estimation reference information 52 become unnecessary.

[0083] Note that, as information attached to the image, shooting date and time information may be used. For example, an image 22 captured within a period based on the shooting date and time of the image 22 determined to depict the related event "engagement" by the content analysis information 72 or the tag information 112 is unconditionally determined to be an image 22 that depicts the related event "engagement".

[0084] [Second Embodiment] In the above-described first embodiment, when the total number of images 22 that serve as a basis for estimating a future event is equal to or greater than the first threshold value, it is assumed that the user 13 will experience a future event after the set period, but the present invention is not limited to this. It may be estimated as in the second embodiment shown in FIGS. 21 and 22.

[0085] As an example, as shown in FIGS. 21 and 22, in the second embodiment, estimation conditions 115 for two specific related events, which are two of a plurality of related events, are prepared. That is, the estimation condition 115 is that the number of images 22 related to a first related event, which is one of the specific related events, is equal to or greater than a second threshold value, and the number of images 22 related to a second related event, which is one of the specific related events, is equal to or greater than the second threshold value. In FIGS. 21 and 22, the case where the second threshold value is 5 (estimation condition: the number of images 22 related to the first related event ≧ 5 and the number of images 22 related to the second related event ≧ 5) is illustrated.

[0086] When the number of images 22 related to the first related event and the number of images 22 related to the second related event are both equal to or greater than the second threshold value, the estimation unit 64 estimates that the user 13 will experience a future event after the set period.

[0087] In FIG. 21, an example is illustrated in which there are 6 images 22 determined to depict the related event "Venue Inspection" and 7 images 22 determined to depict the related event "Betrothal Gift Presentation", and it is estimated that the user 13 will get married after the set period. In this case, the related events "Venue Inspection" and "Betrothal Gift Presentation" are an example of the "specific related events" according to the technology of the present disclosure.

[0088] In FIG. 22, an example is illustrated in which there are 6 images 22 determined to depict the related event "Venue Inspection (first time)" and 10 images 22 determined to depict the related event "Venue Inspection (second time)", and it is estimated that the user 13 will get married after the set period. In this case, the related events "Venue Inspection (first time)" and "Venue Inspection (second time)" are an example of the "specific related events" according to the technology of the present disclosure. As is clear from the example of FIG. 22, the first related event and the second related event may be the same.

[0089] As described above, in the second embodiment, when the images 22 related to the two specific related events are both equal to or greater than the preset second threshold value, the estimation unit 64 estimates that the user 13 will experience a future event after the set period. Therefore, the risk of making an incorrect estimation can be further reduced.

[0090] Note that the specific related events are not limited to two, namely the first related event and the second related event. There may be three or more related events. Also, the second threshold value may not be the same for all of the plurality of specific related events. For example, the second threshold value for the images 22 related to the first related event may be 3, and the second threshold value for the images 22 related to the second related event may be 5.

[0091] [Third Embodiment] As an example, as shown in FIG. 23, in the third embodiment, in the image management server 10, as shown in Table 120, the number of times the recommendation information 25 by the user 13 is adopted is aggregated for each distribution date of the recommendation information 25. The number of times of adoption is, for example, the number of times the user 13 selects the recommendation information 25 in order to expand and display the entire content of the recommendation information 25 in the list 98. The number of times of adoption is an example of the "adoption frequency" according to the technology of the present disclosure.

[0092] The distribution control unit 66 determines whether to stop the distribution of the recommendation information 25 based on the distribution stop condition 121. The distribution stop condition 121 is, for example, that the distribution dates when the number of times of adoption is equal to or less than the third threshold value continue three times in a row. When the distribution dates when the number of times of adoption is equal to or less than the third threshold value continue three times in a row, the distribution control unit 66 stops the distribution of the recommendation information 25 on the next distribution date.

[0093] In FIG. 23, the case where the third threshold value is 1 (distribution stop condition: the number of times of adoption ≤ 1 continues three times in a row) is illustrated. Also, in FIG. 23, the number of times of adoption on the distribution dates "2021.01.03", "2021.01.10", and "2021.01.17" is "1", "0", and "0" respectively, and the number of times of adoption is 1 or less three times in a row, and the distribution of the recommendation information 25 on the distribution date "2021.01.24" is stopped.

[0094] As described above, in the third embodiment, when the number of times the recommendation information 25 by the user 13 is adopted satisfies the preset distribution stop condition 121, the presentation of the recommendation information 25 is stopped. Therefore, it is possible to prevent the useless distribution of the recommendation information 25 that the user 13 is considered to have lost interest in after the user 13 has already experienced a future event.

[0095] The number of times of adoption may be the number of times the product of the recommendation information 25 is purchased. Also, the adoption frequency may be the average of the number of times of adoption on each distribution date. In this case, the distribution stop condition is, for example, that the distribution dates when the average of the number of times of adoption is equal to or less than the third threshold value continue three times in a row.

[0096] [Embodiment 4_1] As an example, as shown in FIG. 24, in Embodiment 4_1, recommendation information 25 in which the cumulative adoption count 125 is registered is used. The cumulative adoption count 125 is the cumulative count of the number of times each user 13 has selected the recommendation information 25 in order to expand and display the entire content of the recommendation information 25 in the list 98. In FIG. 24, the recommendation information 25 in which "200 times" is registered as the cumulative adoption count 125 is illustrated.

[0097] As an example, as shown in FIG. 25, the distribution control unit 66 sets the display order in the list 98 of a plurality of pieces of recommendation information 25 from the information acquisition unit 65 in descending order of the cumulative adoption count 125. By setting the display order in descending order of the cumulative adoption count 125 in this way, the distribution control unit 66 preferentially presents the recommendation information 25 that has been adopted relatively more by other users 13. The distribution control unit 66 distributes the recommendation information 25 to the user terminal 11 that is the transmission source of the recommendation information distribution request 70, together with the set display order. The browser control unit 90 of the user terminal 11 displays the recommendation information 25 in the list 98 according to the display order.

[0098] FIG. 25 shows an example of setting the display order of four pieces of recommendation information 25A to 25D, namely, recommendation information 25A with a cumulative adoption count 125 of "200 times", recommendation information 25B with a cumulative adoption count 125 of "300 times", recommendation information 25C with a cumulative adoption count 125 of "50 times", and recommendation information 25D with a cumulative adoption count 125 of "100 times". In this case, the distribution control unit 66 sets the display order in the order of recommendation information 25B, recommendation information 25A, recommendation information 25D, and recommendation information 25C.

[0099] Thus, in the fourth first embodiment, the recommendation information 25 that is relatively frequently adopted by other users 13 is preferentially presented. If the recommendation information 25 is related to a product, the recommendation information 25 that is relatively frequently adopted by other users 13 is the recommendation information 25 of best-selling products. If the recommendation information 25 is related to a store or a facility, the recommendation information 25 that is relatively frequently adopted by other users 13 is the recommendation information 25 of popular stores or popular facilities. Therefore, the distribution control unit 66 can preferentially present the recommendation information 25 that is more beneficial to the user 13.

[0100] [Fourth second embodiment] As an example, as shown in FIG. 26, in the fourth second embodiment, the recommendation information 25 in which the cumulative adoption times 130 for each attribute of the user 13 are registered is used. The attribute of the user 13 is, for example, a combination of the age and gender of the user 13, such as "men in their 20s" and "women in their 40s". The age of the user 13 can be derived from the date of birth in the attribute information 31. In FIG. 26, the recommendation information 25 in which "60 times" is registered as the cumulative adoption times 130 for men in their 20s and "15 times" is registered as the cumulative adoption times 130 for women in their 30s is exemplified. If the date of birth is not registered in the attribute information 31, the age of the user 13 may be estimated from the face image 32.

[0101] As an example, as shown in FIG. 27, the distribution control unit 66 sets the display order in the list 98 of a plurality of pieces of recommendation information 25 from the information acquisition unit 65 in descending order of the cumulative adoption times 130 in the attribute that matches the user 13 who presents the recommendation information 25. By setting the display order in descending order of the cumulative adoption times 130 in the attribute that matches the user 13 who presents the recommendation information 25 in this way, the distribution control unit 66 preferentially presents the recommendation information 25 that is relatively frequently adopted by the user 13 whose attribute matches the user 13 who presents the recommendation information 25. The distribution control unit 66 distributes the recommendation information 25 to the user terminal 11 that is the transmission source of the recommendation information distribution request 70, together with the set display order. The browser control unit 90 of the user terminal 11 displays the recommendation information 25 in the list 98 according to the display order.

[0102] In FIG. 27, an example is shown where the user 13 presenting the recommendation information 25 is a man in his 30s. Also shown is an example of setting the display order of three pieces of recommendation information 25E to 25G, namely, recommendation information 25E where the cumulative adoption count 130 of men in their 30s is "50 times", recommendation information 25F where the cumulative adoption count 130 of men in their 30s is "150 times", and recommendation information 25G where the cumulative adoption count 130 of men in their 30s is "350 times". In this case, the distribution control unit 66 sets the display order in the order of recommendation information 25G, recommendation information 25F, and recommendation information 25E.

[0103] Thus, in the 4_2nd embodiment, the "other user" according to the technology of the present disclosure is the user 13 whose attribute matches that of the user 13 presenting the recommendation information 25. Therefore, the distribution control unit 66 can preferentially present the recommendation information 25 that has been widely adopted by the user 13 whose attribute matches its own.

[0104] Note that the attributes for registering the cumulative adoption count 130 may include the residential area, family composition, etc. Also, the attributes for registering the cumulative adoption count 130 may be the ages of the users 13 at 5-year intervals such as 20 years old, 25 years old, 30 years old, and so on. In this case, the user 13 presenting the recommendation information 25 may be of an age that does not fall within such attributes, for example, 23 years old. In such cases, the cumulative adoption count 130 of the closer age at 5-year intervals is used. For example, if the age of the user 13 presenting the recommendation information 25 is 34 years old, the cumulative adoption count 130 of 35 years old is used out of the cumulative adoption count 130 of 30 years old and the cumulative adoption count 130 of 35 years old. That is, the "other user" according to the technology of the present disclosure may be a user 13 whose attribute is similar to that of the user 13 presenting the recommendation information 25.

[0105] [4_3rd Embodiment] As an example, as shown in FIG. 28, in the fourth-third embodiment, recommendation information 25 in which the cumulative adoption times 135 for each experience order of events of user 13 are registered is used. The experience order of events of user 13 is the order of related events experienced by user 13, such as "face-to-face meeting → betrothal → pre-wedding visit to the wedding venue", for example. The order of related events can be obtained by arranging the related events determined by the estimation unit 64 based on the estimation reference information 52 and the content analysis information 72 in chronological order with reference to the shooting date and time information of the image 22. In FIG. 28, recommendation information 25 is exemplified in which the cumulative adoption times 135 are "100 times" for the experience order of events "face-to-face meeting → betrothal → pre-wedding visit to the wedding venue" and "40 times" for the cumulative adoption times 135 for the experience order of events being only "face-to-face meeting", etc.

[0106] As an example, as shown in FIG. 29, the distribution control unit 66 sets the display order in the list 98 of a plurality of pieces of recommendation information 25 from the information acquisition unit 65 in descending order of the cumulative adoption times 135 in the experience order of events that match the user 13 who presents the recommendation information 25. By thus setting the display order in descending order of the cumulative adoption times 135 in the experience order of events that match the user 13 who presents the recommendation information 25, the distribution control unit 66 preferentially presents the recommendation information 25 that has been adopted relatively more by the user 13 whose experience order of events matches the user 13 who presents the recommendation information 25. The distribution control unit 66 distributes the recommendation information 25 to the user terminal 11 that is the transmission source of the recommendation information distribution request 70, together with the set display order. The browser control unit 90 of the user terminal 11 displays the recommendation information 25 in the list 98 according to the display order.

[0107] In FIG. 29, an example is illustrated where the order of experience of events of user 13 presenting recommendation information 25 is "face-to-face meeting → pre-wedding venue visit". Also shown is an example of setting the display order of three pieces of recommendation information 25H to 25J, namely, recommendation information 25H where the cumulative adoption count 135 of the order of experience of events "face-to-face meeting → pre-wedding venue visit" is "500 times", recommendation information 25I where the cumulative adoption count 135 of the order of experience of events "face-to-face meeting → pre-wedding venue visit" is "50 times", and recommendation information 25J where the cumulative adoption count 135 of the order of experience of events "face-to-face meeting → pre-wedding venue visit" is "100 times". In this case, the distribution control unit 66 sets the display order in the order of recommendation information 25H, recommendation information 25J, and recommendation information 25I.

[0108] Thus, in the 4_3rd embodiment, the "other user" according to the technology of the present disclosure is user 13 whose order of experience of events matches that of user 13 presenting recommendation information 25. Therefore, the distribution control unit 66 can preferentially present the recommendation information 25 that is highly adopted by user 13 whose order of experience of events matches its own.

[0109] Note that the order of experience of events for registering the cumulative adoption count 135 may be limited to several representative types. In this case, the order of experience of events of user 13 presenting recommendation information 25 may not match the order of experience of representative events. In such cases, the cumulative adoption count 135 of the order of experience of representative events similar to the order of experience of events of user 13 presenting recommendation information 25 is used. For example, if the order of experience of events of user 13 presenting recommendation information 25 is "face-to-face meeting → betrothal gift presentation → pre-wedding venue visit → dress fitting" and there is no matching order of experience of events, the cumulative adoption count 135 of the order of experience of representative events "face-to-face meeting → betrothal gift presentation → pre-wedding venue visit → dress fitting → pre-wedding photo shoot" is used. That is, the "other user" according to the technology of the present disclosure may be user 13 whose order of experience of events is similar to that of user 13 presenting recommendation information 25.

[0110] As a method for preferentially presenting recommendation information 25 that is relatively frequently adopted by other users 13, it is not limited to the method of setting the display order in the illustrated list 98 in descending order of the cumulative adoption times 125, 130, or 135. For example, only the recommendation information 25 whose cumulative adoption times 125, 130, or 135 are equal to or greater than a preset threshold value may be distributed to the user terminal 11, or a blinking frame may be displayed for the recommendation information 25 with relatively large cumulative adoption times 125, 130, or 135, so as to adopt a display mode that is more prominent than the recommendation information 25 with relatively small cumulative adoption times 125, 130, or 135. Such methods may also be used.

[0111] Similar to the adoption times in the third embodiment above, the cumulative adoption times 125 may be the number of times the products of the recommendation information 25 are purchased. Alternatively, instead of the cumulative adoption times 125, the monthly average adoption times may be registered.

[0112] The fourth - 2 embodiment and the fourth - 3 embodiment above may be implemented in combination. That is, the recommendation information 25 for which the cumulative adoption times are registered for each attribute of the user 13 and in the order of the user 13's event experience is used. Then, the recommendation information 25 that is relatively frequently adopted by users whose attributes and event experience order are similar or identical to the user 13 presenting the recommendation information 25 is preferentially presented.

[0113] In each of the above embodiments, "marriage" is exemplified as a future event, but it is not limited thereto.

[0114] Figure 30 shows an example of the estimated reference information 52 for the future event "child-rearing". In this case, the related events include "pregnancy", "childbirth", "visiting a shrine", "first taste of food", "half birthday", "Shichigosan", and "starting kindergarten", etc. Also, as keywords, for example, for the related event "pregnancy", there are "swollen belly, ultrasonic echo, fetus, mother and child handbook...", and for the related event "Shichigosan", there are "oneself, wife, parents, son, daughter, shrine, formal dress, kimono, thousand-year candy...". In this case, the image management server 10 presents maternity goods, baby bottles, milk, infant toys, celebratory clothes for shrine visits, rental kimonos for Shichigosan, etc. to the user 13 as the recommended information 25 for products. Also, as the recommended information 25 for stores or facilities, it presents maternity classrooms, baby product stores, nurseries, toy stores, etc. to the user 13.

[0115] Figure 31 shows an example of the estimated reference information 52 for the future event "end of life". In this case, the related events include "kanreki", "retirement at the mandatory retirement age", "koki", "beiju", "hakuju", and "hyakuju", etc. Also, as keywords, for example, for the related event "koki", there are "oneself, wife, son, daughter, grandchild, purple baby clothes, purple headscarf, purple futon, fan...", and for the related event "hyakuju", there are "oneself, wife, son, daughter, grandchild, great-grandchild, pink baby clothes, pink headscarf, pink futon, fan...". In this case, the image management server 10 presents golfing supplies, reading glasses, walking sticks, etc. to the user 13 as the recommended information 25 for products. Also, as the recommended information 25 for stores or facilities, it presents go salons, social dance circles, travel companies organizing senior group tours, facilities arranging funerals, facilities holding seminars on so-called end-of-life planning such as property distribution, etc. to the user 13. Note that, paying attention to the fact that there are many elderly users among the users 13 who digitize images taken on photo film in the past, the digitized image 22 may be determined as the image 22 serving as the basis for the estimation of the future event "end of life".

[0116] FIG. 32 shows an example of the estimated reference information 52 for the future event "employment". In this case, the related events include "internship", "job guidance", and "joint company briefing", etc. Also, as keywords, for example, for the related event "internship", there are "oneself, suit, work clothes, office chair, desk, personal computer, projector screen...", and for the related event "joint company briefing", there are "oneself, student, large number of people, booth, chair, desk, hanging banner...", etc. In this case, the image management server 10 presents employment information magazines, writing utensils, etc. to the user 13 as the recommendation information 25 for products. Also, as the recommendation information 25 for stores or facilities, it presents to the user 13 the facilities where joint company briefings are held, job training schools where mock interviews are being conducted, etc.

[0117] In major life milestones such as future events "marriage", "child-rearing", "employment", etc., people often purchase relatively expensive products such as new houses, private cars, and home appliances. Therefore, as the image 22 serving as the basis for the estimation of the above future events, images 22 of the purchased new house, images 22 of the moving process, images 22 of the purchased private car, and images 22 of the purchased home appliances, etc. may be added. Also, when it is estimated that the user 13 will experience the above future events, recommendation information 25 regarding new houses, moving, private cars, and home appliances, etc. may be presented to the user 13.

[0118] “Family structure changes” such as a child becoming independent and User 13 getting married as a couple, or conversely, User 13 leaving the parental home and living alone, etc., may be considered future events. In this case, paying attention to the fact that many people buy pets out of loneliness, an image 22 of the purchased pet may be added as an image 22 serving as the basis for estimating the future event “family structure changes”. Also, for example, if the people traveling together change, such as the daughter getting married and leaving home, and the opportunities for the user and their wife to travel together increase, an image 22 of the travel situation may be added as an image 22 serving as the basis for estimating the future event “family structure changes”. Note that as recommendation information 25 for the future event “family structure changes”, recommendation information 25 from a pet shop, a travel magazine featuring couple travel, etc. may be presented.

[0119] Recommendation information 25 is generated by selecting recommendation information 25 corresponding to the estimated future event from among a plurality of pieces of recommendation information 25 registered in the recommendation information DB 24, but it is not limited to this. A machine learning model that takes the estimated future event as input data and outputs recommendation information 25 may be used to generate recommendation information 25 corresponding to the estimated future event.

[0120] In the first embodiment etc., content analysis information 72 is generated from the image 22 using the content analysis model 51, and the related event shown in the image 22 is determined from the estimated reference information 52 and the content analysis information 72, but it is not limited to this. A machine learning model that outputs the related event shown in the image 22 when the image 22 is input may be used.

[0121] In the first embodiment etc., the recommendation information distribution request 70 is transmitted from the user terminal 11 to the image management server 10 for each set period, but it is not limited to this. When the image viewing AP 85 is executed and a dedicated web browser is launched for the image viewing AP 85, the recommendation information distribution request 70 may be transmitted from the user terminal 11 to the image management server 10.

[0122] In the first embodiment and the like described above, a list 98 of recommendation information 25 is displayed on the image list display screen 95, but it is not limited to this. The list 98 of recommendation information 25 may be displayed on an independent screen different from the image list display screen 95.

[0123] The image management server 10 may generate various screens such as the image list display screen 95 and distribute them to the user terminal 11 in the form of screen data for web distribution created by a markup language such as XML (Extensible Markup Language). In this case, the browser control unit 90 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 44B. Note that, instead of XML, other data description languages such as JSON (Javascript (registered trademark) Object Notation) may be used.

[0124] The user terminal 11 that transmits the image 22 to the image management server 10 and the user terminal 11 that receives the distribution of the recommendation information 25 from the image management server 10 may be different. For example, when there are a plurality of user terminals 11 having the accounts of the same user 13, the image 22 may be transmitted from one of them to the image management server 10, and the recommendation information 25 may be distributed from the image management server 10 to another one.

[0125] The form of presenting the recommendation information 25 to the user 13 is not limited to the form of distributing it to the user terminal 11. The recommendation information 25 may be printed on a paper medium and mailed to the user 13, or the recommendation information 25 may be attached to an email and sent.

[0126] The hardware configuration of the computer constituting the image management server 10 can be variously modified. For example, for the purpose of improving processing power and reliability, the image management server 10 can also be configured with multiple computers separated as hardware. For example, the functions of the request reception unit 60, the image acquisition unit 61, the information acquisition unit 65, and the distribution control unit 66, and the functions of the RW control unit 62, the analysis unit 63, and the estimation unit 64 are distributed and borne by two computers. In this case, the image management server 10 is configured with two computers. Also, the image management server 10, the image DB server 20, and the recommendation information DB server 21 may be integrated into one server.

[0127] Thus, the hardware configuration of the computer of the image management server 10 can be appropriately changed according to the required performance such as processing power, safety, and reliability. Furthermore, not only the hardware, but also the AP such as the operation program 50 can of course be duplicated or stored distributedly in multiple storages for the purpose of ensuring safety and reliability.

[0128] Part or all of the functions of each processing unit of the image management server 10 may be borne by the user terminal 11.

[0129] In each of the above embodiments, for example, as the hardware structure of a processing unit that executes various processes such as a request reception unit 60, an image acquisition unit 61, an RW control unit 62, an analysis unit 63, an estimation unit 64, an information acquisition unit 65, a distribution control unit 66, and a browser control unit 90, the following various processors can be used. Among the various processors, in addition to CPUs 42A and 42B which are general-purpose processors that execute software (operation program 50 and image viewing AP 85) and function as various processing units, there are also a programmable logic device (PLD) which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array), and / or a dedicated electric circuit or the like which is a processor having a circuit configuration specifically designed to execute specific processes, such as an ASIC (Application Specific Integrated Circuit).

[0130] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, and / or a combination of a CPU and an FPGA). Also, a plurality of processing units may be composed of one processor.

[0131] As an example of configuring a plurality of processing units with one processor, firstly, as represented by computers such as clients and servers, there is a form in which one processor is configured by a combination of one or more CPUs and software, and this processor functions as a plurality of processing units. Secondly, as represented by a system on chip (SoC) or the like, there is a form in which a processor that realizes the functions of an entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. Thus, as a hardware structure, the various processing units are configured using one or more of the above various processors.

[0132] Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit (Circuitry) combined with circuit elements such as semiconductor elements can be used.

[0133] The technology of the present disclosure can also appropriately combine the above-described various embodiments and / or various modifications. Needless to say, various configurations can be adopted without departing from the gist, not limited to the above-described embodiments. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program in addition to the program.

[0134] The description and illustration shown above are detailed descriptions of the part related to the technology of the present disclosure and are only examples of the technology of the present disclosure. For example, the descriptions regarding the above-described configuration, function, operation, and effect are descriptions of an example of the configuration, function, operation, and effect of the part related to the technology of the present disclosure. Therefore, it goes without saying that within the scope not departing from the gist of the technology of the present disclosure, unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustration shown above. In addition, in order to avoid complication and facilitate the understanding of the part related to the technology of the present disclosure, descriptions regarding common technical knowledge that does not particularly require explanation for implementing the technology of the present disclosure are omitted in the description and illustration shown above.

[0135] In this specification, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0136] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually stated to be incorporated by reference.

Explanation of Symbols

[0137] 2 Image Management System 10 Image Management Server 11 User Terminal 12 Network 13 User 20 Image Database Server (Image DB Server) 21 Recommendation Information Database Server (Recommendation Information DB Server) 22 Image 23 Image Database (Image DB) 24 Recommendation Information Database (Recommendation Information DB) 25, 25A~25J Recommendation Information 30 Image Folder 31 Attribute Information 32 Facial Image 33 Category 40, 40A, 40B Storage 41 Memory 42, 42A, 42B CPU 43 Communication Unit 44, 44B Display 45, 45B Input Device 46 Bus Line 50 Operating Program 51 Machine Learning Model for Content Analysis (Content Analysis Model) 52 Estimated Reference Information 53, 115 Estimation Conditions 60 Request Reception Unit 61 Image Acquisition Unit 62 Read / Write Control Unit (RW Control Unit) 63 Analysis Unit 64 Estimation Unit 65 Information Acquisition Unit 66 Distribution Control Unit 70 Recommendation Information Distribution Request 71 Image Acquisition Request 72 Content Analysis Information 73 Future Event Information 74 Information Acquisition Request 80 and 120 Tables 85 Image Browsing Application Program (Image Browsing AP) 90 Browser Control Unit 95 Image List Display Screen 96 Thumbnail Image 97 Display Button 98 List 99 Hide Button 110 Shooting Position Information 112 Tag Information 121 Distribution Stop Condition 125 Cumulative Adoption Count 130 Cumulative Adoption Count for Each User Attribute 135 Cumulative Adoption Count for Each Order of User Event Experience ST100, ST110, ST120, ST130, ST140, ST150, ST160, ST170, ST180, ST190, ST200 Steps

Claims

1. A processor; A memory connected to or embedded in the processor, The processor, When a number of images serving as a basis for estimating a future event, which is an event that the user will experience after the period, among images acquired by the user within a predetermined period is equal to or greater than a first threshold value set in advance, recommendation information corresponding to the future event is generated; presenting the recommendation information to the user; Recommendation information presentation device.

2. The processor, The recommendation information presentation device according to claim 1 , wherein when the number of the plurality of images is equal to or greater than a first threshold value set in advance, recommendation information corresponding to the future event is generated.

3. The processor, A recommendation information presentation device as described in claim 1 or claim 2, which determines whether or not the image is a basis for estimating the future event based on at least one of the analysis results of the image and information attached to the image.

4. The processor, A recommendation information presentation device as described in any one of claims 1 to 3, which estimates that the user will experience the future event after the period if all images related to a specific related event, which is at least two of the related events that are events related to the future event, are equal to or greater than a predetermined second threshold.

5. The processor, The device for presenting recommendation information according to claim 1 , further comprising: a step of: stopping presentation of the recommendation information when a frequency of adoption of the recommendation information by the user satisfies a preset condition.

6. The processor, The device for presenting recommendation information according to claim 1 , wherein the recommendation information that is relatively frequently adopted by other users is presented with priority.

7. The device for presenting recommendation information according to claim 6 , wherein the other users are users whose attributes are similar or identical to those of the user presenting the recommendation information.

8. The device for presenting recommendation information according to claim 6 or 7, wherein the other users are users who have a similar or identical order of experience of the events as the user presenting the recommendation information.

9. The processor, The device for presenting recommendation information according to claim 1 , further comprising: selecting recommendation information corresponding to the estimated future event from a plurality of pieces of recommendation information registered in advance.

10. When a number of images serving as a basis for estimating a future event, which is an event that the user will experience after a predetermined period, among images acquired by the user within the predetermined period is equal to or greater than a first threshold value set in advance, generating recommendation information corresponding to the future event; and presenting the recommendation information to the user; A method for operating a recommendation information presentation device comprising the steps of:

11. When a number of images serving as a basis for estimating a future event, which is an event that the user will experience after a predetermined period, among images acquired by the user within the predetermined period is equal to or greater than a first threshold value set in advance, generating recommendation information corresponding to the future event; and presenting the recommendation information to the user; An operating program for a recommendation information presentation device for causing a computer to execute a process including the steps of:

Images