Image suggestion device, image suggestion method, and computer readable recording medium

Abstract

The present invention provides an image suggestion device, method and program that can suggest images that match the user's room and reflect the user's preferences to a user. Data related to the combination of "room and image" preferred by the user is collected for each user and stored in a data storage unit (222). A generation unit (211) uses the data collected for each user to generate information indicating the association between "room and image" that reflects the user's preferences. If a room image (second image) (230) taken by the user is obtained, an image selection unit (212) selects an image (third image) (240) that reflects the user's preferences from an image group logged in a content DB (218) and a user image DB based on the obtained second image and the information indicating the association between "room and image", and suggests it to the user. As a result, an image that matches the user's room and reflects the user's preferences can be suggested as an image to be decorated in the user's room.

Description

Technical Field

[0001] The present invention relates to an image suggestion device, an image suggestion method and a computer-readable recording medium, and more particularly to a technology for suggesting suitable images for decorating a room to a user. Background Art

[0002] When a user selects an image for image printing to decorate his or her room, the user needs to select an image that matches the atmosphere of the room. However, there is a problem that it is difficult to select an image reflecting the user's preference from a large number of image groups within a realistic time.

[0003] Patent Document 1 describes an information processing terminal that acquires biological information of a user for each content while the user is viewing video or audio of the content on a television receiver, and recommends highly rated content to the user based on the biological information.

[0004] Furthermore, Patent Document 2 describes a product sales device that allows a user to purchase a product while checking the atmosphere and coordination of the product when it is placed in an actual room or whether it fits the size of the room.

[0005] In the product sales device described in Patent Document 2, a product image obtained by photographing the product and a room image obtained by photographing the user's room are synthesized, and the synthesized image is output for customer guidance. In particular, when synthesizing the product image and the room image, the image size is adjusted according to the ratio between the size of the room where the product is set and the size of the product, and the product image is rotated according to the shooting direction of the room, thereby adjusting the orientation of the product image to the shooting direction of the room image, and the product image is synthesized and arranged in the room image.

[0006] As a result, when purchasing a product, the user can purchase it while checking the atmosphere and coordination when the product is placed in an actual room or checking whether it fits the size of the room.

[0007] Previous technical literature

[0008] Patent Literature

[0009] Patent Document 1: Japanese Patent Application Publication No. 2009-134671

[0010] Patent Document 2: Japanese Patent Application Publication No. 2004-318359 Summary of the invention

[0011] Technical issues to be solved by the invention

[0012] The invention described in Patent Document 1 identifies content highly evaluated by the user based on the user's biometric information and recommends the content to the user, but does not recommend an image that matches the user's room.

[0013] On the other hand, the invention described in Patent Document 2 acquires an image of a user's room and synthesizes an image of the product with the image of the room to recommend products suitable for the room (matching the size of the room). It is not an invention that recommends images that match the user's room and reflect the user's preferences.

[0014] The present invention has been made in view of such circumstances, and an object of the present invention is to provide an image suggesting device, an image suggesting method, and an image suggesting program that can suggest an image that matches the user's room and reflects the user's preference to the user.

[0015] Means for solving technical problems

[0016] In order to achieve the above-mentioned purpose, an image suggestion device involved in one embodiment of the present invention comprises: a data storage unit, which collects and stores data related to the combination of the first room and the first image collected for each user, the data being data related to the combination preferred by the user; a generation unit, which uses the data corresponding to the user stored in the data storage unit to generate information indicating the association between the first room and the first image reflecting the user's preference; an image acquisition unit, which acquires an image of the second room photographed by the user, namely, a second image; an image selection unit, which selects an image reflecting the user's preference, namely, a third image from a registered image group or an image group owned by the user, based on the second image acquired by the image acquisition unit and the information indicating the association generated by the generation unit; and a suggestion unit, which suggests to the user the third image selected by the image selection unit as an image corresponding to the image print decorated in the user's second room.

[0017] According to one aspect of the present invention, data related to the combination of "room and image" preferred by the user is collected, and information indicating the association between "room and image" reflecting the user's preference is generated using the collected data. Furthermore, if an image of the room taken by the user (second image) is acquired, an image (third image) reflecting the user's preference is selected from the registered image group or the image group owned by the user based on the acquired second image and the information indicating the association between "room and image", and is recommended to the user. As a result, an image that matches the user's room and reflects the user's preference can be recommended as an image to be decorated in the user's room.

[0018] In another aspect of the present invention, the image suggestion device preferably comprises: an information acquisition unit that acquires biological information of a user; and an image browsing unit that can browse images, wherein when the user browses images using the image browsing unit, and a fourth image that is an image of a first room decorated with a first image appears in the image browsing unit, the data storage unit acquires the fourth image reflecting the user's preference based on the biological information of the user acquired from the biological information acquisition unit, and collects data related to the combination preferred by the user from the acquired fourth image. Thus, information related to the combination of "room and image" preferred by the user can be automatically collected.

[0019] In the image suggestion device involved in another embodiment of the present invention, it is preferred that the living body information acquisition unit is a wearable terminal worn by the user, the image browsing unit is a mobile terminal that acquires images browsed through the network, displays the acquired images on the display unit, and communicates with the wearable terminal to acquire the user's living body information, and the data storage unit acquires a fourth image reflecting the user's preference from the mobile terminal.

[0020] In another embodiment of the present invention, the image suggestion device preferably comprises: an image generating unit that generates a fifth image matching the second room based on the second image and information indicating association generated by the generating unit; and an image selecting unit that selects one or more images similar to the generated fifth image from a logged-in image group or an image group owned by the user as the third image.

[0021] In the image suggestion device involved in another embodiment of the present invention, it is preferred that the image selection unit extracts a feature quantity representing the characteristics of the fifth image matching the second room based on the second image and the information representing the association generated by the generation unit, and selects one or more images having a feature quantity similar to the extracted feature quantity from the logged-in image group or the image group owned by the user as the third image.

[0022] In the image suggestion device involved in another aspect of the present invention, it is preferable as follows that the generation unit is a recognizer that performs machine learning using data corresponding to the user stored in the data storage unit as learning data, the image selection unit inputs the second image acquired by the image acquisition unit as an input image of the recognizer, and selects one or more images similar to the recognition result from a registered image group or an image group owned by the user as the third image based on the recognition result output from the recognizer.

[0023] In an image suggestion device according to another embodiment of the present invention, it is preferred that the image selection unit extracts a feature quantity representing a feature of a recognition result outputted from the recognizer, and selects one or more images having a feature quantity similar to the extracted feature quantity from a registered image group or an image group owned by the user as the third image.

[0024] In the image suggestion device involved in another embodiment of the present invention, it is preferred that the generation unit uses the data of the second user whose attributes are similar to those of the first user among the data stored in the data storage unit as the data corresponding to the user, and generates information indicating the correlation between the first room and the first image reflecting the user's preference.

[0025] In the image suggesting device according to still another aspect of the present invention, it is preferable that the suggesting unit suggests a plurality of third images that are ranked high and reflect a user's preference.

[0026] In the image suggesting device according to still another aspect of the present invention, it is preferable that the suggesting unit synthesizes the selected third image with a wall region of the second image acquired by the image acquiring unit, and presents the synthesized fifth image to the user.

[0027] Another embodiment of the image suggestion method includes: a step of collecting data related to the combination of the first room and the first image collected by the user and storing it in a data storage unit, wherein the data is data related to the combination preferred by the user; a step of generating information indicating the association between the first room and the first image reflecting the user's preference using the data corresponding to the user stored in the data storage unit; a step of acquiring an image of the second room taken by the user, namely, a second image; a step of selecting an image reflecting the user's preference, namely, a third image from a logged-in image group or an image group owned by the user based on the acquired second image and the generated information indicating the association; and a step of suggesting the selected third image to the user as an image corresponding to the image print decorated in the user's second room.

[0028] The image suggestion program involved in another embodiment executes the following functions through a computer: a function of collecting data related to the combination of the first room and the first image collected by the user and storing it in a data storage unit, wherein the data is data related to the combination preferred by the user; a function of generating information indicating the association between the first room and the first image reflecting the user's preference using the data corresponding to the user stored in the data storage unit; a function of acquiring an image of the second room taken by the user, namely, a second image; a function of selecting an image reflecting the user's preference, namely, a third image from a logged-in image group or an image group owned by the user based on the acquired second image and the generated information indicating the association; and a function of suggesting the selected third image to the user as an image corresponding to the image print decorated in the user's second room.

[0029] Effects of the Invention

[0030] According to the present invention, an image that matches the user's room and reflects the user's preference can be suggested to the user as an image to be decorated in the user's room. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 1 is a system configuration diagram showing a system including an image recommendation device according to one embodiment of the present invention.

[0032] Figure 2 1 is a block diagram showing an embodiment of an image suggesting device 200 according to one aspect of the present invention.

[0033] Figure 3 This is an example of “a combination of a room and an image” data stored in the data storage unit 222 .

[0034] Figure 4 This is a diagram showing an example of an image 230 of a user's room.

[0035] Figure 5 Yes means Figure 1 FIG. 1 is a diagram showing the appearance of a user terminal 10 .

[0036] Figure 6 Yes means Figure 5 FIG. 1 is a block diagram of the structure of the smartphone 100 shown in FIG.

[0037] Figure 7 Yes means Figure 5 The block diagram of the structure of the wearable terminal 1 is shown.

[0038] Figure 8 Yes means Figure 1 and Figure 2 FIG. 2 is a block diagram of a first embodiment of an image suggesting device 200 shown in FIG.

[0039] Fig. 9 Yes means Figure 8 A functional block diagram of an embodiment of the generation unit 211 is shown.

[0040] Fig.10 1 is a diagram showing an example of a display screen of the smartphone 100 displaying an image recommended by the image recommending device 200 .

[0041] Fig.11 Yes means Figure 1 and Figure 2 2 is a block diagram of a second embodiment of an image suggesting device 200 shown in FIG.

[0042] Fig.12 FIG. 2 is a diagram showing another example of the display screen of the smartphone 100 displaying an image recommended by the image recommending device 200 .

[0043] Fig.13 This is a flowchart showing an implementation of an image suggestion method according to one aspect of the present invention. DETAILED DESCRIPTION

[0044] Hereinafter, preferred embodiments of an image suggesting device, an image suggesting method, and an image suggesting program according to one aspect of the present invention will be described with reference to the accompanying drawings.

[0045] [Structure of image suggestion device]

[0046] Figure 1 1 is a system configuration diagram showing a system including an image recommendation device according to one embodiment of the present invention.

[0047] Figure 1 The system shown is composed of a user terminal 10 for each user and an image suggestion device 200 . Each user terminal 10 and the image suggestion device 200 can communicate with each other via a network (Internet) 3 .

[0048] Each website 4 connected to the network 3 publishes content such as images on a web page, and a user can browse or use the content through a web browser of the user terminal 10. Each website 4 may be an existing website or a website created for the service provided by the image recommendation device 200 described later.

[0049] The user terminal 10 functions as an image browsing unit for browsing the contents (in this example, images) disclosed by the website 4 through the network 3, and has a function of acquiring the user's own living body information through communication (for example, short-range wireless communication) with the wearable terminals 1 and 2 worn by the user, and in particular functions as a data acquisition unit for acquiring the "room and image combination" data preferred by the user. Here, the "room and image combination" data is a combination of a room (including a living room, a dining room, a bedroom, a study, a porch, etc.) and an image (photograph) decorated in the room (the wall of the room), and is data related to the "room and image combination" preferred by the user, for example, including image data of a room decorated with a photo. Here, "room" refers to a space divided by at least a floor, a ceiling, and one or more walls. It mainly refers to a space surrounded by a floor, a ceiling, and four surfaces, but it can also include a space that is open without some of the four surfaces, such as a garage.

[0050] The user terminal 10 is composed of: a wristband-type wearable terminal 1 or a finger-ring-type wearable terminal 2 having a built-in living information sensor for measuring living information; and a mobile terminal (in this example, a smartphone 100 ) of the owner of the wearable terminals 1 and 2 that communicates with the wearable terminals 1 and 2 .

[0051] Wearable terminals 1 and 2 are equipped with a heart rate sensor, a thermometer, an acceleration sensor, etc., which function as a living body information sensor. These sensors measure heart rate, body temperature, and acceleration, and based on these measurement results, calculate (analyze) movement status, body posture, calories consumed, number of steps, sleep time, psychological state (relaxation, tension, excitement, etc.), etc. The living body information (including analysis results) measured by the wearable terminal 1 can be stored and managed in the internal memory of the wearable terminal 1 or displayed on a monitor.

[0052] Furthermore, the wearable terminals 1 and 2 can cooperate with the smartphone 100 on which applications corresponding to the wearable terminals 1 and 2 are installed, and can exchange various information including living body information with the smartphone 100 .

[0053] The smartphone 100 is installed with application software corresponding to the wearable terminals 1 and 2. By executing the application software, communication is performed between the wearable terminals 1 and 2 to acquire living body information from the wearable terminals 1 and 2, store and manage the acquired living body information, and output a monitor for confirming the living body information.

[0054] In addition, the wearable terminal for acquiring living body information is not limited to the wristband-type wearable terminal 1 and the ring-type wearable terminal 2. For example, various wearable terminals such as glasses-type, clothing-type, pendant-type, and helmet-type can be considered.

[0055] In the case of glasses-type wearable terminals, they are equipped with a camera unit, a display, an acceleration sensor, an electroencephalogram sensor, an electrooculogram sensor, etc., and have the function of measuring electroencephalograms, blinking, gaze direction, gaze movement, body movement and other living body information, and can display other necessary information in addition to the measurement data on the display installed on the glasses.

[0056] Figure 2 1 is a block diagram showing an embodiment of an image suggesting device 200 according to one aspect of the present invention.

[0057] Figure 2 The image suggestion device 200 shown functions as a server for the user terminal 10, i.e., the smartphone 100, and is mainly composed of a communication unit 210, a generation unit 211 that generates information indicating the association between a room and an image ("room and image" association information), an image selection unit 212, an image processing unit 214, a content DB (DB: Database) 218, a user image DB 220, and a data storage unit 222.

[0058] The hardware structure for executing various controls of the image suggestion device 200 is various processors of a computer as shown below. The various processors include a general-purpose processor, i.e., a CPU (Central Processing Unit), which executes software (a program (including an image suggestion program)) to function as various control units, a processor, i.e., a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array) whose circuit structure can be changed after manufacturing, and a processor, i.e., a dedicated circuit, such as an ASIC (Application Specific Integrated Circuit), which has a circuit structure specially designed to execute specific processing.

[0059] One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same or different types (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, multiple control units may be composed of one processor. As an example of multiple control units composed of one processor, first, a method in which one processor is composed of a combination of one or more CPUs and software, such as a client and a server, and the processor functions as multiple control units. Second, a method in which a processor is used to implement the functions of the entire system including multiple control units through one IC (Integrated Circuit) chip, such as a system on chip (SoC), is used. In this way, various control units are composed of one or more of the above-mentioned various processors as a hardware structure.

[0060] The communication unit 210 is a part that transmits and receives necessary information to and from a user terminal with a camera (in this example, the smartphone 100 ) via the network 3 .

[0061] The communication unit 210 receives the "room and image combination" data acquired by the smartphone 100. The received "room and image combination" data is stored in the data storage unit 222 for each user. In addition, the details of the method for acquiring the "room and image combination" data by the smartphone 100 will be described later.

[0062] The data storage unit 222 stores, for each user, data on “a combination of a room and an image” collected for each user by the smartphone 100 .

[0063] Figure 3This is an example of "room and image combination" data stored in the data storage unit 222. The "room and image combination" data a, b, c, ... in this example are images (fourth images) of a room (first room) decorated with a photograph (first image), and are images of the room obtained from the website 4 by the user through the smartphone 100 while browsing the website 4.

[0064] Furthermore, the communication unit 210 functions as an image acquisition unit that acquires (receives) a second image that is an image of the user's room (second room) captured by the smartphone 100 .

[0065] Figure 4 2 is a diagram showing an example of an image 230 of a user's room. In addition, the image 230 may be a still image or an animation obtained by photographing a room with a decorative image printed thereon. In the case where the image 230 is an animation, in the subsequent steps, it may be divided into a plurality of still images that do not overlap with each other (or overlap with each other less) by a known method, and the image processing may be performed as an input of the plurality of still images.

[0066] Furthermore, the communication unit 210 transmits an image (third image) corresponding to the image print to be decorated in the user's room to the smartphone 100 in order to suggest it to the user.

[0067] Back to Figure 2 A large number of images (image groups) for printing images to be decorated in a room (wall of a room) are pre-registered in the content DB 218. The image groups registered here are, for example, image groups taken by professional photographers.

[0068] The user image DB 220 has a user's image group taken by each user and is managed for each user. In addition, the user's image group can be logged in from the smartphone 100 via the communication unit 210 in the user image DB 220. In addition, the image group recorded in the user's memory card (recording medium) can be read out and logged in by a card reader or the like. In addition, the user's image group may also include images taken by someone other than the user and sent to the user, and images stored on the Internet that the user has the right to access at any time.

[0069] The content DB 218 and the user image DB 220 are not limited to those built in the image recommendation device 200 , and may be connected via a LAN (Local Area Network) or the like and may be accessed as appropriate.

[0070] The generation unit 211 is a part that generates the relationship information of “room and image” reflecting the preference of the user, using the data corresponding to the user stored in the data storage unit 222 .

[0071] The image selection unit 212 selects the image 230 ( Figure 4 ) and the association information of "room and image" generated by the generating unit 211, an image (third image) reflecting the user's preference is selected from the image group registered in the content DB 218 or the image group owned by the user registered in the user image DB 220. The third image may be one or more images.

[0072] Note that the details of the generating unit 211 and the image selecting unit 212 will be described later.

[0073] The image processing unit 214 and the communication unit 210 function as a suggestion unit that suggests the third image selected by the image selection unit 212 as an image (third image) corresponding to the image print to be decorated in the user's room. The image processing unit 214 is a part that processes the third image selected by the image selection unit 212, and generates an image (display image) including the third image to be displayed on the display unit of the smartphone 100 based on the selected third image. The communication unit 210 sends the display image generated by the image processing unit 214 to the smartphone 100.

[0074] <Collection of “Room and Image Combination” Data>

[0075] Figure 5 Yes means Figure 1 FIG. 1 is a diagram showing the appearance of a user terminal 10 .

[0076] Figure 5 The user terminal 10 shown is composed of a wearable terminal 1 and a smartphone 100 .

[0077] The smartphone 100 starts the application software corresponding to the wearable terminal 1 and acquires the living body information measured and analyzed by the wearable terminal 1 from the wearable terminal 1 through wireless communication.

[0078] Furthermore, when the user browses the content provided by the website 4, the smartphone 100 detects the "room and image combination" image if a "room and image combination" image appears through a program (a program for acquiring "room and image combination" data) that is always running in the smartphone 100. At this time, if the emotions such as joy, anger, sorrow, and happiness (emotions that have a high correlation with the user's living body information) obtained by analyzing the living body information obtained from the wearable terminal 1 change to a positive direction, the browsing image (+living body information) at this time is acquired as the "room and image combination" data (+living body information) preferred by the user.

[0079] [Smartphone 100]

[0080] Figure 5The illustrated smartphone 100 has a flat housing 102, and a display input unit 120 in which a display panel 121 as a display surface and an operation panel 122 as an input unit are integrated on one surface of the housing 102. The housing 102 also has a speaker 131, a microphone 132, an operation unit 140, and a camera unit 141. The structure of the housing 102 is not limited to this, and for example, a structure in which the display unit and the input unit are independent or a structure having a folding structure or a sliding mechanism may be adopted.

[0081] Figure 6 Yes means Figure 5 FIG. 1 is a block diagram of the structure of the smart phone 100 shown in FIG. Figure 6 As shown, the smartphone 100 includes a wireless communication unit 110, a display input unit 120, a call unit 130, an operation unit 140, a camera unit 141 that functions as an image pickup unit, a storage unit 150, an external input / output unit 160, a GPS (Global Positioning System) receiving unit 170, a motion sensor unit 180, a power supply unit 190, and a main control unit 101 as main components. In addition, the smartphone 100 includes a wireless communication function for performing mobile wireless communication via a base station device and a mobile communication network as a main function.

[0082] The wireless communication unit 110 performs wireless communication with a base station device included in the mobile communication network according to the instruction of the main control unit 101. The wireless communication is used to send and receive various file data such as audio data and image data, e-mail data, and receive web data and streaming data.

[0083] The display input unit 120 is a so-called touch panel including a display panel 121 and an operation panel 122. It displays images (still images and animations) or character information to visually convey information to the user and detects the user's operation on the displayed information under the control of the main control unit 101. When viewing the generated three-dimensional image, the display panel 121 is preferably a three-dimensional display panel.

[0084] The display panel 121 uses an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like as a display device.

[0085] The operation panel 122 is a device that is mounted in a manner that enables visual recognition of an image displayed on the display surface of the display panel 121, and detects one or more coordinates operated by a user's finger or a stylus. If the device is operated by a user's finger or a stylus, a detection signal generated by the operation is output to the main control unit 101. Then, the main control unit 101 detects the operation position (coordinates) on the display panel 121 based on the received detection signal.

[0086] like Figure 5 As shown, the display panel 121 and the operation panel 122 of the smartphone 100 are integrated to form the display input unit 120, and the operation panel 122 is configured to completely cover the display panel 121. When this configuration is adopted, the operation panel 122 can have a function of detecting user operations in areas other than the display panel 121. In other words, the operation panel 122 can have a detection area for the overlapping portion overlapping with the display panel 121 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap with the display panel 121 (hereinafter referred to as the non-display area).

[0087] In addition, the size of the display area can be made completely consistent with the size of the display panel 121, but it is not necessary to make them consistent. In addition, the operation panel 122 can have two sensing areas, an outer edge portion and an inner portion other than the outer edge portion. In addition, the width of the outer edge portion is appropriately designed according to the size of the housing 102, etc. In addition, as the position detection method adopted in the operation panel 122, a matrix switch method, a resistance film method, a surface elastic wave method, an infrared method, an electromagnetic induction method, an electrostatic capacitance method, etc. can be cited, and any method can also be adopted.

[0088] The communication unit 130 includes a speaker 131 or a microphone 132. The communication unit 130 converts the user's audio input through the microphone 132 into audio data that can be processed in the main control unit 101 and outputs it to the main control unit 101, or decodes the audio data received through the wireless communication unit 110 or the external input / output unit 160 and outputs it from the speaker 131. Figure 5 As shown, for example, the speaker 131 and the microphone 132 can be mounted on the same surface as the surface on which the display input unit 120 is provided.

[0089] The operation unit 140 is a hardware key using a key switch, etc., and receives instructions from the user. For example, the operation unit 140 is mounted on the lower part or lower side of the display unit of the housing 102 of the smartphone 100, and is a push-button switch that opens when pressed by a finger or the like, and closes due to the restoring force of a spring or the like when the finger is released.

[0090] The storage unit 150 stores the control program and control data of the main control unit 101, address data for establishing a corresponding association with the name or telephone number of the communication object, data of sent and received emails, web data downloaded through web browsing, downloaded content data, and temporarily stores streaming data, etc. In addition, the storage unit 150 is composed of an internal storage unit 151 built into the smartphone and an external storage unit 152 having a removable external memory slot. In addition, each internal storage unit 151 and external storage unit 152 constituting the storage unit 150 is implemented using a storage medium such as a flash memory type, a hard disk type, a micro multimedia card type, a card type memory (for example, Micro SD (registered trademark) memory, etc.), a RAM (Random Access Memory), and a ROM (Read Only Memory).

[0091] The external input / output unit 160 serves as an interface with all external devices connected to the smartphone 100, and is directly or indirectly connected to other external devices through communications (e.g., a universal serial bus, etc.) or a network (e.g., the Internet, a wireless LAN (Local Area Network), Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communications (Infrared Data Association: IrDA), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.).

[0092] As external devices connected to the smartphone 100, for example, there are wired / wireless headphones, wired / wireless external chargers, wired / wireless data ports, memory cards or SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards connected via card slots, external audio / video devices connected via audio / video I / O (input / output) terminals, wirelessly connected external audio / video devices, wired / wireless connected smartphones, wired / wireless connected personal computers, wired / wireless connected mobile information terminals, headphones, etc. The external input / output unit can transmit the data received and transmitted from such external devices to the internal components of the smartphone 100, or transmit the data inside the smartphone 100 to the external device. In this example, the external input / output unit 160 functions as a living body information acquisition unit that performs short-range wireless communication such as Bluetooth with the wearable terminals 1 and 2 to acquire living body information.

[0093] The GPS receiving unit 170 receives GPS signals transmitted from GPS satellites ST1 to STn according to the instruction of the main control unit 101, performs positioning calculation processing based on the received plurality of GPS signals, and detects position information (GPS information) including the latitude, longitude, and altitude of the smartphone 100. When the GPS receiving unit 170 can obtain position information from the wireless communication unit 110 or the external input / output unit 160 (e.g., wireless LAN), it can also detect the position using the position information.

[0094] The motion sensor unit 180 includes, for example, a three-axis acceleration sensor, and detects the physical motion of the smartphone 100 according to the instruction of the main control unit 101. By detecting the physical motion of the smartphone 100, the moving direction or acceleration of the smartphone 100 can be detected. The detection result is output to the main control unit 101.

[0095] The power supply unit 190 supplies power stored in a battery (not shown) to each unit of the smartphone 100 in accordance with an instruction from the main control unit 101 .

[0096] The main control unit 101 includes a microprocessor, operates according to the control program or control data stored in the storage unit 150, and generally controls each unit of the smartphone 100. In addition, the main control unit 101 includes a mobile communication control function and an application processing function for controlling each unit of the communication system in order to perform audio communication or data communication through the wireless communication unit 110.

[0097] The application processing function is realized by the main control unit 101 operating according to the application software stored in the storage unit 150. As the application processing function, for example, in addition to the infrared communication function of controlling the external input / output unit 160 to perform data communication with the target device, the email function of sending and receiving emails, the web browsing function of browsing web pages, etc., there are also functions of automatically acquiring living body information from the wearable terminal 1 through application software dedicated to the wearable terminal 1 and functions of acquiring "room and image combination" data through a program for acquiring "room and image combination" data.

[0098] In addition, the main control unit 101 has an image processing function for displaying an image or the like on the display input unit 120 based on image data (still image or animation data) such as received data or downloaded streaming data. The image processing function is a function for the main control unit 101 to decode the image data, perform image processing on the decoded result, and display the image on the display input unit 120.

[0099] Furthermore, the main control unit 101 performs display control on the display panel 121 and operation detection control for detecting a user operation performed through the operation unit 140 and the operation panel 122 .

[0100] By executing display control, the main control unit 101 displays software keys such as an icon or a scroll bar for starting application software, or displays a window for creating an e-mail. In addition, the scroll bar is a software key for receiving an instruction to move the displayed portion of an image, such as a large image that cannot fall into the display area of ​​the display panel 121.

[0101] Furthermore, by executing operation detection control, the main control unit 101 detects user operations performed through the operation unit 140, or accepts operations on the above-mentioned icons and input of character strings to the input field of the above-mentioned window through the operation panel 122, or accepts scrolling requests for displayed images made through the scroll bar.

[0102] Moreover, by executing operation detection control, the main control unit 101 has a touch panel control function for determining whether the position of the operation on the operation panel 122 is the overlapping part (display area) overlapping with the display panel 121 or the outer edge part (non-display area) that does not overlap with the display panel 121, and controlling the sensing area of ​​the operation panel 122 or the display position of the software key.

[0103] In addition, the main control unit 101 can also detect gesture operations on the operation panel 122 and execute a preset function according to the detected gesture operations. Gesture operations are not simple touch operations as in the past, but operations such as drawing a trajectory with a finger or the like, or specifying multiple positions at the same time, or combining these operations to draw a trajectory on at least one of multiple positions.

[0104] The camera unit 141 is an imaging device that electronically captures images using an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device).

[0105] Furthermore, the camera unit 141 can, under the control of the main control unit 101, convert the image data of visible light obtained by shooting into compressed image data such as JPEG (Joint Photographic coding Experts Group) and record it in the storage unit 150 or output it through the external input / output unit 160 or the wireless communication unit 110, and can also record a distance image indicating the distance of the subject in the storage unit 150 or output it through the external input / output unit 160 or the wireless communication unit 110. Figure 2 As shown in FIG. 1 , in the smartphone 100, the camera unit 141 is mounted on the same surface as the display input unit 120, but the mounting position of the camera unit 141 is not limited thereto, and the camera unit 141 may be mounted on the back of the display input unit 120, or a plurality of camera units 141 may be mounted. In addition, when a plurality of camera units 141 are mounted, it is possible to switch the camera unit 141 used for shooting to shoot individually or to use a plurality of camera units 141 to shoot simultaneously.

[0106] Furthermore, the camera unit 141 can be used for various functions of the smartphone 100. For example, an image acquired by the camera unit 141 can be displayed on the display panel 121, or an image of the camera unit 141 can be used as one of the operation inputs of the operation panel 122. Furthermore, when the GPS receiving unit 170 detects a position, it can also refer to the image from the camera unit 141 to detect the position. Furthermore, it is also possible to refer to the image from the camera unit 141 to determine the optical axis direction of the camera unit 141 of the smartphone 100 or determine the current usage environment without using a three-axis acceleration sensor or using it together with a three-axis acceleration sensor.

[0107] Of course, images from the camera unit 141 can also be used within the application software.

[0108] [Wearable terminal 1]

[0109] Figure 5 The wearable terminal 1 shown is a wristband-type wearable terminal worn on a user's wrist and can detect the user's living body information.

[0110] Living body information refers to information that can be measured from a living body, typically including vital signs such as heart rate, pulse, blood pressure, body temperature, respiratory rate, sweating, brain wave patterns, and may also include emotions such as joy, anger, sorrow, and happiness obtained by analyzing these measurement results, posture (lying down, standing, sitting, walking, running, etc.), amount of exercise, number of steps, calorie consumption, or sleep time.

[0111] Figure 7 Yes means Figure 5 The block diagram of the structure of the wearable terminal 1 is shown.

[0112] Figure 5 and Figure 7 The wearable terminal 1 shown has: a display unit 20 having a display screen 21 (e.g., LCD, OELD, etc.) which is an example of a screen capable of displaying various information such as measurement results of user's vital information; a touch sensor 22 capable of detecting a touch operation position on the display screen 21 of the display unit 20, i.e., a touch position; a fingerprint sensor 23 which is an example of a detection unit capable of detecting a user's fingerprint; an operation button 24 which is provided on the side of the wearable terminal 1 body and can input various operations to the wearable terminal 1; a controller 25; a storage unit 26; a vital information sensor 27; and a communication unit 28. In this embodiment, the display unit 20 and the touch sensor 22 function as a so-called touch panel display.

[0113] The controller 25 is composed of a CPU (Central Processing Unit) and the like, and controls the various components of the wearable terminal 1 by executing a program stored in the storage unit 26. The storage unit 26 is composed of a ROM and a RAM and the like, and can store various information such as the program executed by the controller 25 and the data used when the controller 25 executes the program.

[0114] The biological information sensor 27 includes a heart rate sensor, a thermometer, an acceleration sensor, etc., and measures biological information (eg, heart rate, body temperature, exercise status, etc.) of the user wearing the wearable terminal 1. The communication unit 28 can communicate with external devices such as the smartphone 100 through wireless communication.

[0115] [First Embodiment of Image Suggestion Device]

[0116] Figure 8 Yes means Figure 1 and Figure 2 FIG. 2 is a block diagram of a first embodiment of an image suggesting device 200 shown in FIG.

[0117] The user takes a picture of his / her room (second room) using the smartphone 100, and transmits the image 230 of the room (second image) taken via the network 3 to the image suggestion device 200 ( Figure 1 ).

[0118] Figure 8 In the process, the image selection unit 212 inputs the image through the communication unit 210 ( Figure 2 ) receives an image 230 of the room.

[0119] The generation unit 211 generates the “room and image” association information reflecting the preference of the user using the “room and image combination” data corresponding to the user who sent the room image 230 among the “room and image combination” data stored in the data storage unit 222, and outputs the generated “room and image” association information to the image selection unit 212. In addition, the generation unit 211 preferably generates the “room and image” association information by a machine learning method using the “room and image combination” data in advance (at a time different from the time of receiving the room image 230).

[0120] Fig. 9 Yes means Figure 8 A functional block diagram of an embodiment of the generation unit 211 is shown.

[0121] Fig. 9 The illustrated generation unit 211 is mainly composed of a convolutional neural network (CNN) 232 , a loss value calculation unit 234 , and a parameter control unit 236 .

[0122] CNN232 has a multi-layer structure and a plurality of weight parameters. CNN232 can change from an unlearned model to a learned model by updating the weight parameters from initial values ​​to optimal values.

[0123] The CNN 232 includes an input layer 232A, an intermediate layer 232B including a plurality of groups of convolutional layers and pooling layers, and an output layer 232C, and each layer has a structure in which a plurality of “nodes” are connected by “edges”.

[0124] The input layer 232A inputs the image (fourth image) 250 (of the first room) which is the data of the “combination of room and image” stored in the data storage unit 222. Figure 3 ).

[0125] The intermediate layer 232B has a plurality of groups with a convolution layer and a pooling layer as one group, and is a part that extracts features from the image 250 input by the input layer 232A. The convolution layer filters the nodes located near the previous layer (performs a convolution operation using a filter) to obtain a "feature map". The pooling layer reduces the feature map output from the convolution layer to set it as a new feature map. The "convolution layer" has the role of extracting features such as edges from the image, and the "pooling layer" has the role of giving durability to prevent the extracted features from being affected by parallel movement, etc. In addition, the intermediate layer 232B is not limited to the case where the convolution layer and the pooling layer are grouped as one, but also includes the case where the convolution layer is continuous and the normalization layer.

[0126] Regarding the coefficients of the filters of each convolutional layer of the CNN 232 before learning and the weights of the connection with the next layer in the offset values, arbitrary initial values ​​are set, but it is preferable to set the parameters of the CNN used for image classification, etc. as the initial values.

[0127] The loss value calculation unit 234 obtains the output (feature map) of the output layer 232C of the CNN 232 and the image (first image) 250A as training data (also referred to as teacher data), and calculates the loss value between the two. The loss value calculation method may be, for example, soft maximum intersection entropy, S-shaped, etc. Here, the image 250A as training data is an image included in the image 250 of the room and an image decorated in the room, corresponding to the image cut out from the image 250.

[0128] The parameter control unit 236 adjusts the weight parameters of CNN 232 by the error back propagation method according to the loss value calculated by the loss value calculation unit 234.

[0129] The parameters of the learned CNN 232 thus learned become the correlation information between “room and image”.

[0130] The image selection unit 212 selects the image 230 ( Figure 4 ) and the association information of “room and image” generated by the generating unit 211, an image (third image) 240 reflecting the user's preference is selected from the image group logged in the content DB 218 or the image group owned by the user logged in the user image DB 220.

[0131] Thus, as an image to be decorated in the user's room, an image 240 that matches the user's room and reflects the user's preference can be selected. In addition, whether to select the image 240 from the image group registered in the content DB 218 or the image group registered in the user image DB 220 can be set to be able to be specified by the user, or the image 240 can be selected from both image groups.

[0132] in addition, Fig. 9 In the generating unit 211 shown, as the "combination of room and image" data stored in the data storing unit 222, the image 250 of the room obtained from the website 4 during browsing of the website 4 through the smart phone 100 is used, and the image 250 and the image (first image) 250A of the decoration in the room contained in the image 250 are used as learning data to obtain the parameters of the CNN 232 corresponding to the association information of the "room and image", but it is also possible to store a pair of feature quantities of the image 250 and the image 250A of the room in the data storing unit 222 as the "combination of room and image" data, and perform machine learning based on the pair of feature quantities to obtain the association information of the "room and image".

[0133] Image processing unit 214 ( Figure 2 ) generates a display image including the image 240 based on the image 240 selected by the image selection unit 212. The image suggestion device 200 transmits the display image to the smartphone 100 of the user via the communication unit 210 and the network 3.

[0134] Fig.10 1 is a diagram showing an example of a display screen of the smartphone 100 displaying an image recommended by the image recommending device 200 .

[0135] exist Fig.10 The display screen of the smartphone 100 shown in the figure displays an image 240-1 including the image 240 selected by the image selection unit 212 based on the display image received from the image suggestion device 200. Thus, the image 240 corresponding to the image print to decorate the user's room is suggested to the user.

[0136] Fig.10 In the example shown, the following situation is shown: through the image selection unit 212, three images (No. 1, No. 2 and No. 3) with high suitability are selected as images that match the user's room and reflect the user's preferences, and thumbnail images of the three images and the main image corresponding to the thumbnail image of No. 1, i.e., image 240, are displayed on the display screen.

[0137] When the user wishes to check the details of an image other than image 240 , for example, by selecting thumbnail image No. 2 or No. 3 , a main image corresponding to the thumbnail image can be displayed instead of image 240 .

[0138] In addition, an icon button "enter order screen" is displayed on the display screen, and the user can switch to a screen for placing an order for photo printing corresponding to the image displayed enlarged on the display screen by operating the icon button.

[0139] Thus, the image suggesting device 200 can suggest, to the user, an image that matches the user's room and reflects the user's preference.

[0140] in addition, Fig.10 In the example shown, it is assumed that the user can select the image desired from the three images corresponding to the top three images with the highest degree of suitability, but the number of images suggested is not limited to this. In addition, only the image with the highest degree of suitability may be suggested. In this case, it is preferable to be able to request the suggestion of other images (for example, the image with the second highest degree of suitability, etc.) through the instruction input from the user, so as to be able to deal with the situation where the suggested image is not satisfactory.

[0141] Furthermore, when multiple images are suggested, the suggested order may be determined by inferring the user's preference order based on the user's living body information.

[0142] [Second Embodiment of Image Suggestion Device]

[0143] Fig.11 Yes means Figure 1 and Figure 2 FIG. 2 is a block diagram of a second embodiment of an image suggestion device 200 shown in FIG. Fig.11 In, with Figure 8 The same reference numerals are used for the common parts of the image suggesting device 200 in the first embodiment, and detailed description thereof will be omitted.

[0144] Fig.11 The second embodiment of the image suggesting device 200 shown is different from the first embodiment mainly in that it includes a generating unit 211 - 2 and an image generating unit 213 .

[0145] Generator 211-2 is equivalent to Fig. 9 The generator 211 shown in the figure corresponds to the learned CNN 232 (recognizer). The learned CNN 232 can output a recognition result (feature map) based on the feature quantity of the image of the room, and the recognition result indicates the feature quantity of the image decorated in the room, and the feature quantity of the image matching the room and reflecting the user's preference.

[0146] Therefore, when the image 230 of the user's room is input as an input image to the generation unit 211 - 2 , a feature map indicating the feature amount of the image that matches the user's room and reflects the user's preference is output.

[0147] The image generating unit 213 is composed of, for example, an inverse convolutional neural network, which converts the feature map of the feature quantity space generated by the generating unit 211-2 into the image space, and generates (restores) an image (the fifth image) that matches the user's room shown in the image 230 and reflects the user's preference.

[0148] The image selection unit 212 selects one or more images similar to the fifth image from the image group registered in the content DB 218 or the image group owned by the user registered in the user image DB 220, based on the fifth image generated by the image generation unit 213. The image 240 selected by the image selection unit 212 is thus an image that matches the user's room and reflects the user's preference.

[0149] In addition, Fig.11 In the second embodiment shown, the image (fifth image) is restored from the feature map output by the generating unit 211-2 by the image generating unit 213, and based on the fifth image, one or more images similar to the fifth image are selected from the image group logged in the content DB 218 or the image group logged in the user image DB 220, but this is not limited to this, and one or more images with similar feature quantities may also be selected from the image group logged in the content DB 218 or the image group logged in the user image DB 220 based on the feature map (feature quantity) output from the generating unit 211-2.

[0150] [Other implementations of the proposed image]

[0151] Fig.12 FIG. 2 is a diagram showing another example of the display screen of the smartphone 100 displaying an image recommended by the image recommending device 200 .

[0152] The image processing unit 214, which functions as a part of the suggestion unit, has the following functions: Figure 4 ) and the image 240 selected by the image selection unit 212, a composite image (fifth image) 240-2 is generated as a display image by synthesizing the image 240 in the wall area of ​​the image 230 of the room.

[0153] exist Fig.12 On the display screen of the smartphone 100 shown in the figure, a composite image 240 - 2 is displayed in which the image 240 selected by the image selection unit 212 is composited with the wall region of the room image based on the display image received from the image suggesting device 200 .

[0154] This allows the user to more accurately check the atmosphere and harmony when the suggested image 240 is arranged on the wall of the user's room.

[0155] [Image Proposal Method]

[0156] Fig.13 is a flowchart showing an implementation of an image suggestion method according to one embodiment of the present invention, and Figure 2 The operations of the respective components of the image suggesting device 200 are shown in relation to each other.

[0157] exist Fig.13In the overall control Figure 2 The control unit (not shown) of each unit of the image suggestion device 200 determines whether an image of the user's room photographed by the user has been acquired via the communication unit 210 (step S10 ).

[0158] When the image of the user's room is not obtained ("No" case), the "combination of room and image" data obtained by the program that is always running in the user's smartphone 100 when browsing on the website is received from the smartphone 100, and collected and stored in the data storage unit 222 (step S12).

[0159] When the image of the user's room has been acquired (in the case of "yes"), the generating unit 211 generates the association information of "room and image" reflecting the user's preference using the "combination of room and image" data corresponding to the user stored in the data storage unit 222 (step S14). Alternatively, the association information of "room and image" may be generated and stored in advance for each user.

[0160] Next, the image selection unit 212 selects the user's room based on the image ( Figure 4 ) and the generated “room and image” association information, an image (third image) reflecting the user's preference is selected from the image group logged in the content DB 218 or the image group logged in the user image DB 220 (step S16).

[0161] The image processing unit 214 and the communication unit 210 display the third image selected in step S16 on the display screen of the smartphone 100, thereby suggesting an image (third image) corresponding to the image print to decorate the user's room (step S18).

[0162] Thus, the user can receive image recommendations that match his or her room and reflect his or her preferences simply by sending the image of his or her room to the image recommendation device 200 .

[0163] [other]

[0164] The image suggestion device 200 of this embodiment can be a storefront terminal device installed in a store that accepts image printing orders. In this case, the storefront terminal device can directly receive images of the user's room, display the images recommended to the user on the monitor of the storefront terminal device, and receive user operations such as printing orders.

[0165] Furthermore, in the case of a user who has no "combination of room and image" data or a user whose "combination of room and image" data is insufficient (first user) in the data storage unit 222, "combination of room and image" data of other users (second user) who have attributes (gender, age, hobbies, etc.) similar to those of the first user can be used to generate "room and image" association information reflecting the first user's preferences.

[0166] Furthermore, the present invention includes an image recommendation program which, when installed in a computer, causes the computer to function as the image recommendation device according to the present invention, and a recording medium recording the image recommendation program.

[0167] Furthermore, the present invention is not limited to the above-described embodiments, and it is a matter of course that various modifications can be made without departing from the spirit of the present invention.

[0168] From the above description, the image suggesting device described in the following Supplementary Note 1 can be understood.

[0169] [Supplementary Item 1]

[0170] An image suggestion device comprising:

[0171] a memory for collecting and storing data related to a combination of a first room and a first image collected for each user, wherein the data is data related to the combination preferred by the user; and

[0172] processor,

[0173] The processor performs the following processing:

[0174] generating information indicating a relationship between the first room and the first image reflecting a preference of the user using data corresponding to the user stored in the memory,

[0175] acquiring an image of the second room photographed by the user, namely a second image,

[0176] selecting a third image that reflects the user's preference from a registered image group or an image group owned by the user based on the acquired second image and the generated information indicating the association,

[0177] The selected third image is suggested to the user as an image corresponding to the image print that is decorated in the second room of the user.

[0178] Explanation of symbols

[0179] 1, 2- wearable terminal, 3- network, 4- website, 10- user terminal, 20- display unit, 21- display screen, 22- touch sensor, 23- fingerprint sensor, 24- operation button, 25- controller, 26- storage unit, 27- living information sensor, 28, 210- communication unit, 100- smart phone, 101- main control unit, 102- housing, 110- wireless communication unit, 120- display input unit, 121- display panel, 122- operation panel, 130- call unit, 131- speaker, 132- microphone, 140- operation unit, 141- camera unit, 150- storage unit, 151- internal Storage unit, 152-external storage unit, 160-external input and output unit, 170-GPS receiving unit, 180-motion sensor unit, 190-power supply unit, 200-image suggestion device, 210-communication unit, 211, 211-2-generation unit, 212-image selection unit, 213-image generation unit, 214-image processing unit, 218-content DB, 220-user image DB, 222-data storage unit, 230, 240, 240-1, 240-2, 250, 250A-image, 232-CNN, 234-loss value calculation unit, 236-parameter control unit, S10~S18-steps.

Claims

1. An image suggestion device, comprising: A living body information acquisition unit, which acquires the living body information of the user; An image viewing unit capable of viewing images; a data storage unit that collects and stores data related to a combination of a first room and a first image collected for each user, the data being data related to the combination preferred by the user; a generating unit that generates information indicating a correlation between the first room and the first image reflecting a preference of the user, using the data corresponding to the user stored in the data storing unit; an image acquisition unit, which acquires a second image which is an image of the second room photographed by the user; an image selection unit that selects a third image that reflects the preference of the user from a registered image group or an image group owned by the user, based on the second image acquired by the image acquisition unit and the information indicating the association generated by the generation unit; and a suggestion unit that suggests the third image selected by the image selection unit to the user as an image corresponding to the image print decorated in the second room of the user, in, When the user is browsing images using the image browsing unit, and a fourth image, which is an image of the first room decorated with the first image, appears in the image browsing unit, the data storage unit acquires the fourth image reflecting the user's preference based on the user's biometric information acquired from the biometric information acquisition unit, and collects data related to the combination preferred by the user from the acquired fourth image. The generating unit includes a convolutional neural network, a loss value calculating unit and a parameter controlling unit. The convolutional neural network includes an input layer, an intermediate layer and an output layer. The fourth image is input to the input layer, The intermediate layer includes a plurality of groups including convolutional layers and pooling layers respectively, and obtains a feature map, outputs the obtained feature map to the output layer, The loss value calculation unit obtains the feature map output from the output layer of the convolutional neural network and the first image, and calculates a loss value between the feature map and the first image. The parameter control unit adjusts the weight parameters of the convolutional neural network based on the loss value calculated by the loss value calculation unit, The generation unit obtains the weight parameter of the convolutional neural network as information representing the correlation.

2. The image suggestion device according to claim 1, wherein: The living body information acquisition unit is a wearable terminal worn by the user, The wearable terminal is any one of a wristband type, a ring type, a clothing type, and a pendant type.

3. The image suggestion device according to claim 1, wherein: The living body information includes any one of heart rate, pulse, blood pressure, body temperature, respiratory rate, and sweat amount.

4. The image suggestion device according to claim 1, wherein: The living body information acquisition unit is a wearable terminal worn by the user, The image browsing unit is a mobile terminal that acquires images browsed through the network, displays the acquired images on a display unit, and communicates with the wearable terminal to acquire the user's living body information. The data storage unit acquires the fourth image reflecting the user's preference from the mobile terminal.

5. The image suggestion device according to any one of claims 1 to 4, wherein: The image suggestion device comprises: an image generating unit that generates a fifth image matching the second room based on the second image and the information indicating the association generated by the generating unit, The image selection unit selects one or more images similar to the generated fifth image from the registered image group or the image group owned by the user as the third image.

6. The image suggestion device according to claim 5, wherein: The image selection unit extracts a feature quantity representing the characteristics of the fifth image matching the second room based on the second image and the information representing the association generated by the generation unit, and selects one or more images having a feature quantity similar to the extracted feature quantity from the logged-in image group or the image group owned by the user as the third image.

7. The image suggestion device according to any one of claims 1 to 4, wherein: The generating unit is a recognizer that performs machine learning using the data corresponding to the user stored in the data storage unit as learning data. The image selection unit inputs the second image acquired by the image acquisition unit as an input image of the identifier, and selects one or more images similar to the recognition result from the logged-in image group or the image group owned by the user as the third image based on the recognition result output from the identifier.

8. The image suggestion device according to claim 7, wherein: The image selection unit extracts a feature quantity representing a feature of a recognition result outputted from the recognizer, and selects one or more images having a feature quantity similar to the extracted feature quantity from the registered image group or the image group owned by the user as the third image.

9. The image suggestion device according to any one of claims 1 to 4, wherein: The generating unit generates information indicating a relationship between the first room and the first image reflecting a preference of the user, using data of a second user having attributes similar to those of the first user among the data stored in the data storing unit as data corresponding to the user.

10. The image suggestion device according to any one of claims 1 to 4, wherein: The suggestion unit suggests the plurality of third images that are ranked high and reflect the preference of the user.

11. The image suggestion device according to any one of claims 1 to 4, wherein: The suggestion unit synthesizes the selected third image with a wall region of the second image acquired by the image acquisition unit, and presents the synthesized fifth image to the user.

12. An image suggestion method, comprising: Steps for obtaining user's living body information; A step of browsing an image by an image browsing unit; A step of collecting data related to a combination of a first room and a first image collected by each user and storing the data in a data storage unit, wherein the data is data related to the combination preferred by the user; A step of generating information indicating a correlation between the first room and the first image reflecting a preference of the user, using data corresponding to the user stored in the data storage unit; A step of acquiring an image of the second room taken by the user, namely a second image; A step of selecting a third image reflecting the user's preference from a registered image group or an image group owned by the user based on the acquired second image and the generated information indicating the association; and a step of suggesting the selected third image to the user as an image corresponding to the image print decorated in the second room of the user, wherein, when the user is browsing images using the image browsing unit, and the image of the first room decorated with the first image, i.e., the fourth image appears in the image browsing unit, the fourth image reflecting the preference of the user is acquired based on the acquired living body information of the user, and data related to the combination preferred by the user is collected from the acquired fourth image; The information representing the correlation is generated by a generation unit including a convolutional neural network, a loss value calculation unit, and a parameter control unit, The convolutional neural network includes an input layer, an intermediate layer and an output layer. The fourth image is input to the input layer, The intermediate layer includes a plurality of groups including convolutional layers and pooling layers respectively, and obtains a feature map, outputs the obtained feature map to the output layer, The loss value calculation unit obtains the feature map output from the output layer of the convolutional neural network and the first image, and calculates a loss value between the feature map and the first image. The parameter control unit adjusts the weight parameters of the convolutional neural network based on the loss value calculated by the loss value calculation unit, The generation unit obtains the weight parameter of the convolutional neural network as information representing the correlation.

13. The image suggestion method according to claim 12, wherein: The living body information is obtained by a wearable terminal worn by the user, The wearable terminal is any one of a wristband type, a ring type, a clothing type, and a pendant type.

14. The image suggestion method according to claim 12, wherein: The living body information includes any one of heart rate, pulse, blood pressure, body temperature, respiratory rate, and sweat amount.

15. A computer-readable recording medium having an image suggestion program recorded thereon, the image suggestion program realizing the following functions through a computer: The function of obtaining the user's living body information; A function of viewing images through the image viewing section; a function of collecting data related to a combination of a first room and a first image collected by each user and storing the data in a data storage unit, wherein the data is data related to the combination preferred by the user; A function of generating information indicating a correlation between the first room and the first image reflecting a preference of the user, using data corresponding to the user stored in the data storage unit; A function of acquiring an image of a second room taken by the user, namely, a second image; A function of selecting a third image that reflects the user's preference from a registered image group or an image group owned by the user based on the acquired second image and the generated information indicating the association; and A function of suggesting the selected third image to the user as an image corresponding to the image printed in the second room of the user, wherein, when the user is browsing images using the image browsing unit, and the image of the first room decorated with the first image, i.e., the fourth image appears in the image browsing unit, the fourth image reflecting the preference of the user is acquired based on the acquired living body information of the user, and data related to the combination preferred by the user is collected from the acquired fourth image; The information representing the correlation is generated by a generation unit including a convolutional neural network, a loss value calculation unit, and a parameter control unit, The convolutional neural network includes an input layer, an intermediate layer and an output layer. The fourth image is input to the input layer, The intermediate layer includes a plurality of groups including convolutional layers and pooling layers respectively, and obtains a feature map, outputs the obtained feature map to the output layer, The loss value calculation unit obtains the feature map output from the output layer of the convolutional neural network and the first image, and calculates a loss value between the feature map and the first image. The parameter control unit adjusts the weight parameters of the convolutional neural network based on the loss value calculated by the loss value calculation unit, The generation unit obtains the weight parameter of the convolutional neural network as information representing the correlation.

16. The computer-readable recording medium according to claim 15, wherein: The living body information is obtained by a wearable terminal worn by the user, The wearable terminal is any one of a wristband type, a ring type, a clothing type, and a pendant type.

17. The computer-readable recording medium according to claim 15, wherein: The living body information includes any one of heart rate, pulse, blood pressure, body temperature, respiratory rate, and sweat amount.

Images