Information processing apparatus, information processing method, and program

Abstract

To support user's house selection in an intuitive and easy-to-understand manner.SOLUTION: A server 1 is configured: acquire an image selected from a plurality of images based on a user operation and text information indicating a request for the image; generate feature information on user's preference on the basis of the acquired image and text information; select, based on the generated feature information, a housing company from a plurality of housing companies; and display housing company information related to the selected housing company on a UI unit 460.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, construction of a house is an important matter for many users, and several technologies have been proposed for constructing a house desired by the user. For example, a technology capable of proposing a house plan, floor plan, etc. based on information input by the user has been disclosed (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-189657 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional techniques including the technique described in the above-mentioned Patent Document 1, in many cases, a house plan or the like is proposed based on the user's answers to questions, etc. However, in general, although a user has an abstract ideal image or preferences for the house to be built, it is difficult for the user to clearly communicate the image or desire for the house.

[0005] An object of the present invention is to support a user in selecting a home in an intuitive and easy-to-understand manner. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system including: a housing contractor database that stores the characteristics of homes constructed by housing contractors in association with the housing contractors for each of a plurality of housing contractors; an acquisition unit that acquires from the electronic device an image selected according to the user's preferences and text information indicating the user's requests for the image; a generation unit that generates a first feature related to the user's preferences based on the text information, generates a second feature that is a feature of the image related to what is contained in the image based on the image, and generates weights for each of the first feature and the second feature based on the text information; and a selection unit that selects a housing contractor from the plurality of housing contractors that corresponds to the user's preferences based on a comparison result between the feature information indicating the user's preferences generated based on the weight, the first feature, and the second feature and the features of the home; an information processing device that constitutes the information processing system; an information processing method that includes each of the processes; and a program that causes a computer to execute each of the processes. Another aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system comprising: a housing developer database that stores the characteristics of homes constructed by a housing developer and the housing developer in association with each of a plurality of housing developers; an image information database that stores housing images of each home constructed by each of the plurality of housing developers; an acquisition unit that acquires an image according to the user's preferences from the electronic device; a feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing developer according to the user's preferences from among the plurality of housing developers based on a comparison result obtained by comparing the features of the image with the features of the home; and an information provision unit that causes the electronic device to display the housing image extracted from the image information database based on the features of the image acquired by the feature acquisition unit when a new image is requested by the user's operation; an information processing device that constitutes the information processing system; an information processing method that includes each of the processes; and a program that causes a computer to execute each of the processes. Moreover, one aspect of the present invention is an information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, the information processing system including a housing contractor database that stores the characteristics of homes constructed by housing contractors in association with the housing contractors for each of a plurality of housing contractors, an acquisition unit that acquires an image according to the user's preferences from the electronic device, a feature acquisition unit that acquires the features of the image based on the image, a selection unit that selects a housing contractor according to the user's preferences from among the plurality of housing contractors based on a comparison result obtained by comparing the features of the image with the features of the home, and an information provision unit that causes the electronic device to display housing contractor information related to the housing contractor selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected, an information processing device that constitutes the information processing system, an information processing method that includes each of those processes, and a program that causes a computer to execute each of those processes. Another aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a plurality of housing developers, the information processing device comprising: an acquisition unit that acquires an image selected according to a user's preference and text information indicating a user's desire for the image; a generation unit that generates a first feature related to the user's preference based on the text information, generates a second feature based on the image, which is a feature of the image related to what is contained in the image, and generates weights for each of the first feature and the second feature based on the text information; and a selection unit that selects a housing developer from the plurality of housing developers that corresponds to the user's preference based on a comparison result between the feature information indicating the user's preference generated based on the weight, the first feature, and the second feature, and the features of the home; an information processing system including the information processing device; an information processing method including each of the processes; and a program for causing a computer to execute each of the processes. Moreover, one aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a plurality of housing developers, and an image information database that stores housing images of each home constructed by each of the plurality of housing developers, and the information processing device comprises an acquisition unit that acquires images according to a user's preferences, a feature acquisition unit that acquires features of the image based on the image, a selection unit that selects a housing developer according to the user's preferences from among the plurality of housing developers based on a comparison result obtained by comparing the features of the image with the features of the home, and an information provision unit that, when a new image re-proposal is requested by the user's operation, causes the housing image extracted from the image information database based on the features of the image acquired by the feature acquisition unit to be displayed on a display unit, an information processing system including the information processing device, an information processing method including each of these processes, and a program that causes a computer to execute each of these processes. Moreover, one aspect of the present invention is an information processing device capable of handling a database that associates the characteristics of homes constructed by a housing developer with the housing developer for each of a number of housing developers, and the information processing device comprises an acquisition unit that acquires an image according to a user's preferences, a feature acquisition unit that acquires features of the image based on the image, a selection unit that selects a housing developer according to the user's preferences from among the multiple housing developers based on a comparison result of comparing the features of the image with the features of the home, and an information provision unit that causes a display unit to display housing developer information related to the housing developer selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected, an information processing system including the information processing device, an information processing method including each of these processes, and a program that causes a computer to execute each of these processes. Effect of the Invention

[0007] According to the present invention, it is possible to support a user in selecting a home in an intuitive and easy-to-understand manner. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a system configuration diagram showing a configuration of an information processing system. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a server that configures the information processing system. [Diagram 3] 2 is a diagram illustrating an example of the functional configuration of a server, a user terminal, and a housing business terminal that constitute an information processing system. [Figure 4] FIG. 13 is a diagram illustrating an example of an image displayed on a user terminal. [Diagram 5] FIG. 1 is a conceptual diagram showing an overview of a tagging method. [Figure 6] FIG. 6 is an image diagram showing an overview of a tagging method, which is an example different from the example in FIG. 5. [Figure 7] FIG. 1 is an image diagram for explaining the concept of a user vector. [Figure 8]FIG. 5 is a diagram showing an example of an image displayed on the user terminal, which is a diagram showing an example different from the example in FIG. 4. [Figure 9] FIG. 9 is a diagram showing an example of an image displayed on a user terminal, which is an example different from the examples of FIGS. 4 and 8. [Figure 10] 1 is an arrow chart showing an example of the flow of various processes executed by a server and a user terminal. [Figure 11] 1 is a block diagram illustrating an example of the configuration of an information processing system. [Figure 12] FIG. 13 is a diagram showing an example of generating tags to be added to a house image. [Figure 13] FIG. 2 is a simplified diagram showing the stored contents of image information. [Figure 14] FIG. 2 is a diagram showing the stored contents of housing business operator information in a simplified form. [Figure 15] FIG. 13 is a diagram showing a display example of a selection screen. [Figure 16] FIG. 13 is a diagram showing a display example of a selected business operator information screen. [Figure 17] FIG. 13 is a diagram showing a display example of a selection screen. [Figure 18] 13 is a flowchart showing an example of a housing business operator selection process. [Figure 19] FIG. 2 is a diagram showing the stored contents of item information in a simplified manner. [Figure 20] FIG. 13 is a diagram showing a display example of an edit screen. [Figure 21] 13 is a flowchart showing an example of a house image editing process. [Figure 22] FIG. 2 is a simplified diagram showing the stored contents of image information. [Diagram 23] FIG. 13 is a diagram showing a display example of a selection screen. [Figure 24] 13 is a flowchart illustrating an example of an image selection process. [Diagram 25] FIG. 13 is a diagram showing a display example of a text input screen. [Figure 26] FIG. 13 is a diagram showing a display example of a selection screen. [Figure 27] 13 is a flowchart showing an example of a housing business operator selection process. [Figure 28]FIG. 13 is a diagram showing a display example of a text input screen. [Figure 29] FIG. 13 is a diagram showing a display example of a selection screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] <Summary> First, prior to a specific description using FIG. 1, an example of a service (hereinafter, referred to as "this service") to which an information processing system according to one embodiment of the present invention is applied will be described.

[0011] This service is, for example, a service that proposes housing businesses that match the user's taste and new housing images based on a housing image selected by the user. Note that housing businesses are various businesses related to the construction and design of houses, such as first-class architectural offices, house manufacturers, and construction companies.

[0012] In this service, an information processing system according to one embodiment of the present invention (hereinafter referred to as "this system") utilizes techniques such as statistical machine learning to analyze home images selected by the user and home images provided by home builders, select home builders that are close to the user's preferred home, and display them to the user. This enables the user to select promising home builders who are likely to be able to build the home of the user's preference, simply by selecting the home image of the user's preference, and to request the creation of home plans, design, etc.

[0013] FIG. 1 is a system configuration diagram showing the configuration of an information processing system according to one embodiment of the present invention.

[0014] The information processing system shown in Fig. 1 includes a server 1, a user terminal 2, and housing provider terminals 3-1 to 3-n (n is any natural number equal to or greater than 1). The server 1, the user terminal 2, and the housing provider terminals 3-1 to 3-n are connected to each other via a predetermined network N such as the Internet. However, the network N is not a required component, and for example, Near Field Communication (NFC), Bluetooth (registered trademark), or a Local Area Network (LAN) may be used. In the following description, when there is no need to particularly distinguish between the housing provider terminals 3-1 to 3-n, they will be collectively referred to as housing provider terminals 3.

[0015] The server 1 is configured with a general-purpose personal computer or the like, and is managed by the provider of this service, etc. The server 1 executes various processes related to the provision of this service.

[0016] The user terminal 2 is composed of a smartphone, a tablet, a general-purpose PC, etc., and is used by a user who wishes to use this service. The user terminal 2 accepts various input operations by the user. The user terminal 2 also acquires various information transmitted from the server 1 and displays it on a display unit (not shown) or the like.

[0017] The housing business operator terminal 3 is configured with a general-purpose personal computer or the like, and is managed by various housing business operators who wish to participate in this service. The housing business operator terminal 3 transmits image information and the like of houses selected by each housing business operator to the server 1.

[0018] <Hardware configuration> FIG. 2 is a diagram showing an example of a hardware configuration of a server constituting an information processing system according to one embodiment of the present invention.

[0019] As shown in FIG. 2, the server 1 includes a control unit 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.

[0020] The control unit 11 is composed of a microcomputer including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a semiconductor memory, and executes various processes according to a program recorded in the ROM 12 or a program loaded from the memory unit 18 to the RAM 13.

[0021] The RAM 13 also stores information necessary for the control unit 11 to execute various processes as appropriate.

[0022] The control unit 11, ROM 12, and RAM 13 are connected to one another via a bus 14. An input / output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

[0023] The output unit 16 outputs image information, audio information, etc. to various liquid crystal displays, speakers, etc. The image information and audio information output by the output unit 16 are output from various liquid crystal displays, speakers, etc. in a state that can be recognized by humans as images and audio.

[0024] The input unit 17 is composed of a keyboard, a mouse, etc., and is used to input various information.

[0025] The storage unit 18 is configured with a hard disk drive (HDD), a solid state drive (SSD), etc., and stores various data. In this embodiment, for example, various information including various programs and various databases is stored.

[0026] The communication unit 19 controls communications with other information processing devices via a network N including the Internet.

[0027] The drive 20 is provided as necessary. Removable media 31, which may be a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately mounted in the drive 20. The program read from the removable media 31 by the drive 20 is installed in the storage unit 18 as necessary. The removable media 31 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

[0028] Here, the hardware configuration of the user terminal 2 and the housing provider terminal 3 can be basically the same as the hardware configuration of the server 1, and therefore a description thereof will be omitted.

[0029] FIG. 3 is a diagram showing an example of the functional configuration of a server, a user terminal, and a housing business operator terminal that constitute an information processing system according to one embodiment of the present invention.

[0030] As shown in FIG. 3, in the control unit 11 of the server 1, an image information acquisition unit 60, a tag information acquisition unit 62, a vector generation unit 64, and a housing business operator selection unit 66 function by executing various programs and the like.

[0031] Furthermore, a housing provider DB 400 is provided in one area of ​​the storage unit 18 of the server 1. The housing provider DB 400 stores vectors (hereinafter referred to as "provider vectors") corresponding to each housing provider that are generated based on housing images provided by each housing provider, as well as profiles provided by each housing provider. The method by which the system generates vectors for each housing provider can be basically the same as the method of generating vectors for users, which will be described later.

[0032] The image information acquiring unit 60 acquires information on an image including a house from a user related to the house as second image information. Specifically, the image information acquiring unit 60 acquires information on a favorite house image selected by the user and transmitted from the user terminal 2 (hereinafter referred to as "favorite image information") via the communication unit 19.

[0033] The tag information acquisition unit 62 acquires information regarding classification based on a predetermined criterion for the second image information. That is, the tag information acquisition unit 62 acquires information regarding the tags assigned to each of the favorite images.

[0034] Here, a tag is a standard for classifying each image based on the characteristics of each image. The act of classifying each image based on the tag standard is called tagging. This tagging is performed automatically for all house images handled by this system based on the results of learning by statistical machine learning performed in advance, and the results are stored in a database (not shown) in advance.

[0035] Therefore, the tag information acquisition unit 62 acquires information about tags attached to favorite images stored in the database (not shown).

[0036] Specifically, in this embodiment, for example, the following five types of criteria are adopted as tags. (1) Location This is a classification standard for the locations included in each house image. For example, information about locations within the house that are actually included in each house image, such as the exterior of the house as a whole, the entrance, stairs, living room, kitchen, bathroom, rooms, etc., is added as a tag. (2) Style This is a classification standard for the style of the places included in each house image. If the exterior of the house is made of wood and close to nature, information such as "natural" or "woody" is added as a tag. As a specific classification of the style, various style standards known in various fields of architecture, such as "urban," "dark tone," "woody," "natural," "Japanese style," "vintage," "old folk house," "modern," and "simple" are adopted. (3) Material This is a classification standard for the materials contained in each house image. For example, information about characteristic materials such as "wood," "wood grain," "tile," "concrete / mortar," "steel / iron," "design wallpaper," "glass," and "stainless steel" are added as tags. (4) Items This is the classification criterion for whether or not each house image contains a wood stove or solar panels. For example, if each house image contains a wood stove or solar panels, it is tagged as "Yes (plus)," and if each house image does not contain a wood stove or solar panels, it is tagged as "No (minus)." (5) Emotions This is a classification standard for the atmosphere of each house image and the emotions felt when viewing it. Emotional standards such as "family gathering / warmth," "openness," "luxurious / elegant / luxurious," "calm," "cool / chic," "flashy," "casual / pop," "stately," "sophisticated / stylish," and "cute" are adopted. For example, information regarding the results of evaluating these emotional standards for each house image on a five-point scale from 0 to 4 may be added as a tag to each house image.

[0037] The vector generating unit 64 generates a predetermined vector based on the result of learning by a statistical method. Specifically, the vector generating unit 64 generates a vector (hereinafter, referred to as a "user vector") indicating the user's desired house preference based on information on tags attached to favorite images acquired by the tag information acquiring unit 62. Details of the generated vector will be described later with reference to FIG. 7 etc.

[0038] The housing provider selection unit 66 compares the user vector generated by the vector generation unit 64 with the business vectors of each housing provider stored in the housing provider DB 400 to determine a housing provider that can be recommended to the user. Note that the housing provider selection unit 66 may select one housing provider corresponding to the business vector that is closest to the user vector, or may select multiple housing providers corresponding to business vectors that are closest in distance in order of proximity.

[0039] Furthermore, the housing business selection unit 66 transmits to the user terminal 2 information relating to the selected housing business (hereinafter referred to as “selected business information”).

[0040] As shown in FIG. 3, in the control unit 200 of the user terminal 2, a display image management unit 220, an image selection receiving unit 222, and a housing business owner information acquisition unit 224 function by executing various programs and the like.

[0041] The display image management unit 220 generates and manages various images to be displayed on a display unit (not shown) of the user terminal 2. Specifically, the display image management unit 220 generates various images for the user to select a favorite image and various images for presenting the user with housing business information (described later), and executes control to display the images on a display unit (not shown) of the user terminal 2.

[0042] The image selection receiving unit 222 receives information on images including houses from a user related to a house. Specifically, the image selection receiving unit 222 receives a selection of a favorite house image (hereinafter referred to as a "favorite image") selected by the user. In addition, the image selection receiving unit 222 transmits favorite image information related to the received selection of favorite image to the server 1. Note that a method in which a user actually selects a favorite image will be described later with reference to FIG. 4 etc.

[0043] The housing business information acquisition unit 224 acquires the selected business information transmitted from the server 1 via the communication unit 210.

[0044] Fig. 4 is a diagram showing an example of an image displayed on a user terminal constituting an information processing system according to an embodiment of the present invention. Specifically, Fig. 4 shows an example of an image displayed when a user selects a favorite image.

[0045] In the example of Fig. 4, a plurality of house images selected by the user as favorite images are displayed. Note that the favorite images displayed in Fig. 4 are selected by the user from house images presented in advance.

[0046] 4, the selection of a favorite image can be easily cancelled by clicking a favorite button (e.g., display areas A1 to D1) displayed together with the house image. In this embodiment, the selection (and deletion) of a favorite image is performed, for example, by such a method.

[0047] Next, the tagging method will be specifically described with reference to Figures 5 and 6. Figures 5 and 6 are conceptual diagrams showing an overview of the tagging method according to one embodiment of the present invention.

[0048] An example of a house image selected as a favorite image is displayed in the example of Figure 5. As described above, in this system, target house images are automatically tagged based on the results of learning by statistical machine learning.

[0049] Specifically, in the example of Figure 5, since the image shows a house in which the living room was captured, "living room" is assigned as the location tag. And since the overall structure is natural and made of wood (S1 in Figure 5), style tags such as "natural" and "woody" are assigned. Similarly, the material tag is "wood grain." Since there is no wood stove or solar panels, the item tag is "none (minus)." Finally, taking into account the impression given by the overall atmosphere, emotion tags such as "family gathering, warmth" or "calming" may be assigned.

[0050] Here, in the present system, tagging of locations in particular can be performed at multiple levels. That is, in addition to tagging of a wide area location, the present system can also assign tags indicating detailed locations. Specifically, in the example of FIG. 5, it is also possible to tag locations included in the house image as "open-to-view kitchen (S2 in FIG. 5)," "dining table (S3 in FIG. 5)," "stairs (S4 in FIG. 5)," and the like.

[0051] Similarly, in the example of Figure 6, tags such as "exterior," "garden / courtyard," "wooden deck," and "triangular roof / gable roof" are assigned as location tags, tags such as "dark tones" as style tags, tags such as "siding" as material tags, the tag "none (negative)" as item tags, and tags such as "cool / chic" and "sophisticated / stylish" as emotion tags.

[0052] FIG. 7 is an image diagram for explaining the concept of a user vector according to one embodiment of the present invention.

[0053] In the example of Fig. 7, the following numerical values ​​(vectors) are displayed based on the tagging results: location (0.2), style (0.8), material (0.5), item (0.2), and emotion (0.5). Note that the vectors are generated, for example, by converting the tags assigned to each favorite image selected by the user into numerical values ​​and then statistically processing the numerical values. Therefore, the generated user vector can be evaluated as a comprehensive vector representation of the user's housing preferences.

[0054] FIG. 8 is a diagram showing an example of an image displayed on a user terminal constituting an information processing system according to one embodiment of the present invention, and is a diagram showing an example different from the example of FIG.

[0055] Specifically, in the example of FIG. 8, new house image candidates are displayed to the user based on the selected favorite image. These new house images are selected by comparing the user vector generated by the present system with a vector generated from any house image (such as a house image provided by a housing developer or any house image acquired in advance by the present system). While checking these house images, if the user finds a house image that suits his / her taste, he / she can easily add it to his / her favorite images by clicking a favorite button (for example, display areas E1 to H1) displayed together with the house image. By adding the newly added favorite image in this way and generating a user vector, the present system can improve the accuracy of responding to the user's tastes.

[0056] FIG. 9 is a diagram showing an example of an image displayed on a user terminal constituting an information processing system according to one embodiment of the present invention, and is a diagram showing an example different from the examples of FIG. 4 and FIG.

[0057] Specifically, in the example of FIG. 9, candidates of the selected housing business are displayed together with information such as the name of the housing business and the area covered based on the selected favorite image. The user can select the housing business to which the user wishes to request the construction of a house while checking the displayed candidates of the housing business. In the example of FIG. 9, the user can check detailed information (e.g., company information, registered profile, etc.) of each housing business by clicking a detail confirmation button (e.g., display areas I1 and J1). In the example of FIG. 9, the user can request detailed information of each housing business by clicking a document request button (e.g., display areas I2, J2, M1) and then clicking an arbitrary decision button (e.g., display area K1). For example, when this document request is made, the information is sent to each housing business, and each housing business sends the specified document to the user.

[0058] FIG. 10 is an arrow chart showing an example of the flow of various processes executed by the server and user terminal constituting the information processing system according to one embodiment of the present invention.

[0059] In step S1, the image selection receiving unit 222 of the user terminal 2 receives a selection of a favorite image selected by the user.

[0060] In step S2, the image selection receiving unit 222 of the user terminal 2 transmits to the server 1 favorite image information related to the received favorite image selection.

[0061] In step S21, the image information acquisition unit 60 of the server 1 acquires, via the communication unit 19, favorite image information selected by the user and transmitted from the user terminal 2.

[0062] In step S22, the tag information acquisition unit 62 of the server 1 acquires information about the tags assigned to the respective favorite images.

[0063] In step S23, the vector generating unit 64 of the server 1 generates a user vector indicating the user's desired house preference, based on information related to the tag attached to the favorite image acquired by the tag information acquiring unit 62.

[0064] In step S24, the housing business selection unit 66 of the server 1 compares the user vector generated by the vector generation unit 64 with the business vectors of each housing business stored in the housing business DB 400, and determines a housing business that can be recommended to the user.

[0065] In step S25, the housing business operator selection unit 66 of the server 1 transmits to the user terminal 2 selected business operator information relating to the selected housing business operator.

[0066] In step S3, the housing business information acquisition unit 224 of the user terminal 2 acquires the selected business information transmitted from the server 1 via the communication unit 210.

[0067] In step S4, the display image management unit 220 of the user terminal 2 generates various images for presenting the housing business operator information to the user, and executes control for displaying the images on a display unit (not shown) of the user terminal 2.

[0068] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0069] Here, the advantages unique to this system will be further explained. This system generates vectors based on an image selected by the user (e.g., a favorite image), and can further recommend house images with high similarity. Therefore, unlike existing systems, the user does not need to clearly verbalize the atmosphere, features, etc., and can automatically receive recommendations for new house images and housing businesses simply by selecting a house image that they like. In addition, this system is updated each time the user selects a new house image as a favorite image, thereby improving the accuracy of the selection. These points are the distinctive advantages of this system.

[0070] Although the present system has been described as using the generated vector to determine the similarity, the present system is not limited to this. For example, the present system may generate any criteria or numerical values ​​based on the tagging results, and select new home images or recommend home improvement companies based on the criteria or numerical values.

[0071] In the above embodiment, the favorite images are selected from house images pre-installed in the system, but this is not limited to this. The system may have a function for accepting uploads of favorite house images from users, for example. In this case, the system may appropriately tag newly uploaded house images.

[0072] In the above embodiment, the present system has been described as automatically assigning tags based on the results of learning by statistical machine learning, but this is not limited thereto. For example, the present system may assign tags based on any criteria, regardless of whether statistical machine learning is applied. In addition, the present system may simply accept input of the results of tagging performed by a person and acquire them as information.

[0073] In the above embodiment, the present system has been described as adopting five criteria, namely, "location," "style," "material," "item," and "emotion," as tagging criteria, but is not limited thereto. The present system may adopt any other criteria as tagging criteria.

[0074] In the above embodiment, the system has been described as generating a user vector by considering all of the criteria of "location", "style", "material", "item", and "emotion", but this is not limited thereto. For example, the system may generate a user vector by considering only any of the criteria of "location", "style", "material", "item", and "emotion".

[0075] In the above embodiment, the present system has described examples of criteria and tags to be added when tagging based on the criteria of "location," "style," "material," "item," and "emotion," but these are all examples and are not limiting. The present system may perform tagging based on any criteria other than the above examples, or may perform tagging by adopting any tags other than the above examples.

[0076] Although not described in the above embodiment, the timing when the user terminal 2 transmits favorite image information and the timing when tagging and the like are performed based on the favorite image information acquired by the server 1 are both arbitrary. For example, the present system may adopt a specification in which various processes are performed when a display area (not shown) is clicked by the user, or a specification in which various processes are always performed when various conditions are updated by the user's operation.

[0077] Although the above-mentioned embodiments (particularly the embodiments in Figs. 5 and 6) have been described simply, the present system can adopt results obtained by hierarchical learning, particularly in tagging of locations. Specifically, the present system can improve the learning accuracy by performing learning while taking into account the tagging contents including the hierarchy shown in the examples in Figs. 5 and 6.

[0078] Although the above-mentioned embodiments (particularly the embodiments in Fig. 4 and Fig. 8) have been described simply, the present system may have more substantial user management functions in addition to simply allowing the addition and deletion of favorite images. Specifically, for example, the present system may have a customer information management function, a work progress management function, a reservation function for events, etc., a communication function such as SNS, a function for accepting comments from users (information regarding preferences and requests for housing, planned construction site, budget, etc.), etc.

[0079] Although not described in the above embodiment, the tagging criteria of the present system do not need to be treated equally, and weighting may be adopted. For example, in terms of emotion criteria, the priority of a tag such as "family gathering, warmth" may be set relatively high compared to the priority of a tag such as "calming."

[0080] Although not described in the above embodiment, the system may have a function for interacting with the user. Specifically, for example, the system may present a question or the like to the user and have the user input an answer to the question, thereby improving the accuracy of matching.

[0081] Although the above embodiment is merely a simplified explanation, the vectors generated by the present system may be quantified in any manner. Specifically, the present system may have numerical values ​​as more specific tag items, such as location vectors such as living room (0.7), dining room (0.6), kitchen, etc., style vectors such as Western style (1.0), Japanese style (0.0), and emotion vectors such as cool (1.0), cute (0.0), etc.

[0082] Although not described in the above embodiment, the present system may use the following elements, for example, to select a new home image or recommend a home builder. (1) The system may use image features, for example, to select new home images or recommend home builders. (2) The system may use text (such as letters and numbers) contained in an image, for example, to select a new home image or to recommend a home builder. When using these elements, the system may use any of them as a criterion for tagging.

[0083] Furthermore, the above-described series of processes can be executed by hardware or software.

[0084] In other words, the functional configurations in FIG. 3 and the like are merely examples and are not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and the type of functional block used to realize this function is not limited to the example in Fig. 3 etc. Furthermore, the location of the functional block is not limited to the example in Fig. 3 etc. and may be arbitrary.

[0085] Furthermore, one functional block may be configured as a single piece of hardware, may be configured as a single piece of software, or may be configured as a combination of both.

[0086] Furthermore, the number of different hardware components constituting this system and the number of users are optional, and the system may also include other hardware components.

[0087] For example, the various hardware components and the number of users constituting the present system are not limited to the above examples, and may be any number. The present system may be configured by including and combining not only the various hardware components described above, but also other hardware components.

[0088] Also, for example, the present system may be configured to execute various processes via the Web, etc., as in a so-called cloud-type system, and send the results of the processes to various hardware.

[0089] Furthermore, when the series of processes is executed by software, the program constituting the software is installed into a computer or the like from a network or a recording medium.

[0090] The computer may be a computer that is built into dedicated hardware, or a computer that is capable of executing various functions by installing various programs.

[0091] That is, for example, any computer, any mobile terminal such as a smartphone, etc. may be freely adopted for the various hardware in the above-described embodiments.

[0092] Furthermore, the recording medium containing such a program may not only be constituted by a removable medium (not shown) provided separately from the device main body in order to provide the program to a user, etc., but may also be constituted by a recording medium, etc. that is provided to the user in a state in which it is pre-installed in the device main body.

[0093] In this specification, the steps of describing the program to be recorded on the recording medium may be executed in parallel or individually, as well as in chronological order. Also, some of the steps described in the above embodiment may be omitted.

[0094] In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc.

[0095] In addition, even when these other embodiments are adopted, the effects of the present embodiment are exhibited. In addition, the present embodiment can be appropriately combined with other embodiments, and other embodiments can be appropriately combined with each other.

[0096] In other words, it is sufficient for an information processing device to which the present invention is applied to have the following configuration, and various embodiments can be adopted.

[0097] That is, a first aspect of the present invention is an information processing device for processing images related to a house, comprising: a storage means for storing information regarding an image including the house from a construction company related to the house as first image information; a second image information acquisition means for acquiring information regarding an image including the house from a user related to the house as second image information; and a classification information acquisition means for acquiring information regarding a classification based on predetermined criteria for the second image information as classification information.

[0098] In addition, in the first aspect, the information processing device may further include a first determination means for determining a construction business operator that can be recommended to the user based on the classification information acquired by the classification information acquisition means.

[0099] In addition, in the first aspect, the information processing device may further include a second determination means for determining new house image candidates that may be presented to the user based on the classification information acquired by the classification information acquisition means.

[0100] In addition, in the first aspect, it is possible to acquire information classified based on at least one of a location-related criterion, a style-related criterion, a material-related criterion, an item-related criterion, and an emotion-related criterion.

[0101] Also, in the first aspect, the classification information acquisition means can acquire information about a tag assigned to an image related to the second image information, based on a result of learning using a statistical method.

[0102] In addition, in the first aspect, the classification information acquisition means can generate a numerical value that serves as a criterion for decision making, taking into consideration the results of learning using a statistical method.

[0103] In addition, in the first aspect, the classification information acquisition means can perform learning including a learning method that utilizes a hierarchical structure.

[0104] Moreover, a second aspect of the present invention is an information processing method executed by a computer for processing images relating to a house, the information processing method including a storage means for storing information relating to an image including the house from a construction company relating to the house as first image information, a second image information acquisition means for acquiring information relating to an image including the house from a user relating to the house as second image information, and a classification information acquisition means for acquiring information relating to a classification based on predetermined criteria for the second image information as classification information.

[0105] Furthermore, a third aspect of the present invention is a program for causing a computer for processing images relating to a house to execute processing including a storage means for storing information relating to an image including the house from a construction company relating to the house as first image information, a second image information acquisition means for acquiring information relating to an image including the house from a user relating to the house as second image information, and a classification information acquisition means for acquiring information relating to a classification of the second image information based on predetermined criteria as classification information.

[0106] This allows the user to intuitively and easily select a home builder of their choice.

[0107] As described above, in this embodiment, it is possible to realize various services for proposing housing businesses and new home images that match the user's tastes based on a home image selected by the user. Various aspects of these services will be described below. Note that illustrations and explanations of parts common to the above-described embodiments will be omitted as appropriate, or the same reference numerals will be used to describe them.

[0108] [Example of information processing system configuration] Fig. 11 is a block diagram showing an example of a configuration of an information processing system IS1. The information processing system IS1 corresponds to the information processing system shown in Fig. 1 and Fig. 3. The server 1 corresponds to the server 1 shown in Fig. 1 to Fig. 3. The user terminal 2 corresponds to the user terminal 2 shown in Fig. 1 and Fig. 3.

[0109] The information processing system IS1 is composed of a server 1 and a user terminal 2. For example, the server 1 and the user terminal 2 are connected via a predetermined network N (see FIG. 1). Note that communication between these devices is performed using wired communication or wireless communication. Furthermore, communication between these devices may be performed directly between the devices in addition to communication via the network N.

[0110] [Server configuration example] The server 1 includes a communication unit 310, a control unit 320, and a storage unit 330. The communication unit 310, the control unit 320, and the storage unit 330 correspond to the units with the same names shown in Figs.

[0111] The communication unit 310 exchanges various information with other devices by using at least one of wired communication and wireless communication. For example, the communication unit 310 executes a reception process for receiving various information transmitted from the user terminal 2, a transmission process for transmitting various information to the user terminal 2, and the like.

[0112] The control unit 320 includes an information acquisition unit 321 , a tag generation unit 322 , a vector generation unit 323 , an extraction unit 324 , and an information provision unit 325 .

[0113] The information acquisition unit 321 receives various pieces of information transmitted from the user terminal 2, and supplies the received pieces of information to each unit.

[0114] The tag generating unit 322 generates a tag for the information (for example, a house image, other images) output from the information acquiring unit 321, and outputs information (tag information) on the generated tag to the vector generating unit 323. For example, when the information output from the information acquiring unit 321 is a house image stored in an image information DB (Data Base) 340, the tag generating unit 322 acquires tag information 343 (see FIG. 13) associated with the house image. When the information output from the information acquiring unit 321 is another image, the tag generating unit 322 can generate a tag based on the tag information stored in the tag information DB 360. When the information output from the information acquiring unit 321 is another image, the tag generating unit 322 can generate a tag using an AI (Artificial Intelligent) model. The tag generating method will be described in detail with reference to FIG. 12 and the like.

[0115] The vector generation unit 323 generates a user vector according to the preferences of the user of the user terminal 2 based on the tag information (vector components) output from the tag generation unit 322, and outputs the generated user vector to the extraction unit 324. Note that a method for generating a user vector will be described in detail with reference to FIG. 15 etc.

[0116] The extraction unit 324 extracts desired information from each DB stored in the storage unit 330 based on the information output from each unit, and outputs each extracted information to the information provision unit 325. For example, when a user vector is output from the vector generation unit 323, the extraction unit 324 compares the user vector with a housing business operator vector stored in the housing business operator DB 350, and extracts a housing business operator according to the preference of the user of the user terminal 2 based on the similarity of each vector. Also, for example, when the provision of a new housing image is requested from the user terminal 2, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the tag information 343 (vector) stored in the image information DB 340, and extracts a housing image according to the preference of the user of the user terminal 2 based on the similarity of each vector. The method of generating each of these user vectors will be described in detail with reference to FIG. 15 and the like.

[0117] The information providing unit 325 executes transmission control for transmitting information output from each unit to the user terminal 2.

[0118] The storage unit 330 is a storage medium that stores various types of information. For example, the storage unit 330 stores various types of information (e.g., a control program, image information 340 (see FIG. 13), a housing business operator DB 350 (see FIG. 14), a tag information DB 360, and an AI model) required for the control unit 320 to perform various processes. The storage unit 330 also stores various types of information acquired via the communication unit 310. For example, a ROM, a RAM, a static random access memory (SRAM), a HDD, an SSD, or a combination of these can be used as the storage unit 330.

[0119] The image information DB 340 is a database for storing house images provided by housing developers in association with feature quantities of a plurality of tags generated based on the images. The image information DB 340 will be described in detail with reference to FIG. 13.

[0120] The housing business operator DB 350 is a database for storing various types of information related to housing businesses in association with each other. The housing business operator DB 350 will be described in detail with reference to FIG.

[0121] The tag information DB 360 is a database for storing information used when tag information is generated by the tag generation unit 322. For example, when tag information is generated using an AI model, information about the AI ​​model is stored.

[0122] [Example of user terminal configuration] The user terminal 2 includes a communication unit 410, a position information acquisition unit 420, an image acquisition unit 430, a sound acquisition unit 440, a control unit 450, a UI (User Interface) unit 460, and a storage unit 470. The user terminal 2 is realized by, for example, an information processing device such as a smartphone, a tablet terminal, a personal computer, or an electronic device. Note that each unit included in the user terminal 2 is an example, and some of these may be omitted, or other members may be added. For example, when the user terminal 2 is a personal computer, the position information acquisition unit 420 and the image acquisition unit 430 can be omitted or can be external devices.

[0123] The communication unit 410, under the control of the control unit 450, exchanges various types of information with other devices using wired or wireless communication.

[0124] The location information acquisition unit 420 acquires location information on the location where the user terminal 2 is located, and outputs the acquired location information to the control unit 450. The location information acquisition unit 420 can be realized, for example, by a GPS receiver that receives a GPS (Global Positioning System) signal and calculates location information based on the GPS signal. The calculated location information includes data on the location such as latitude, longitude, and altitude at the time of receiving the GPS signal. A location information acquisition device that acquires location information by other location information acquisition methods may be used. For example, a location information acquisition device that derives location information using access point information by wireless LAN present in the vicinity and acquires this location information may be used. For example, a location information acquisition device that derives location information using location information of a base station used for calls and communications and acquires this location information may be used. For example, a location information acquisition device that derives location information using a location estimation technique using a navigation function and acquires this location information may be used. For example, the location of the device itself can be estimated based on sensor information from various sensors (for example, an acceleration sensor, a gyro sensor) and map information.

[0125] The image acquisition unit 430 captures an image of a subject and generates an image (image data) based on the control of the control unit 450, and outputs the generated image to the control unit 450. The image acquisition unit 430 is configured, for example, with an imaging element (image sensor) into which light from the subject collected by a lens (not shown) is incident, and an image processing unit that performs predetermined image processing on the image data generated by the imaging element, etc. As the imaging element, for example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal Oxide Semiconductor) type imaging element can be used.

[0126] The sound acquisition unit 440 acquires sounds around the user terminal 2 under the control of the control unit 450, and outputs sound information related to the acquired sounds to the control unit 450. As the sound acquisition unit 440, for example, one or more microphones or sound acquisition sensors can be used.

[0127] The control unit 450 controls each unit based on various programs stored in the storage unit 470. The control unit 450 is realized by a processing device such as a CPU. Each process executed by the control unit 450 will be described in detail with reference to Figs. 15 to 18 and the like.

[0128] The storage unit 470 is a storage medium that stores various information. For example, the storage unit 470 stores various information (for example, a control program, an image information DB 800 (see FIG. 22), a housing business application, and device identification information) required for the control unit 450 to perform various processes. The storage unit 470 also stores various information acquired via the communication unit 410. For example, a ROM, a RAM, an SRAM, a HDD, an SSD, or a combination of these can be used as the storage unit 470. The storage unit 470 may be a removable storage unit or may be a storage unit built into the user terminal 2. The user terminal 2 may be provided with both a removable storage unit and a built-in storage unit.

[0129] The UI unit 460 includes a reception unit 461 and an output unit 462 .

[0130] The reception unit 461 receives various operations from the user and outputs the received operation contents to the control unit 450. The reception unit 461 and the output unit 462 may be configured as a touch panel that allows the user to input operations by touching or approaching the display surface with their finger, or may be configured as a separate user interface. When configured as a separate user interface, various operation members such as switches, buttons, and keyboards can be used as the reception unit 461. Also, a user interface may be configured in which an image corresponding to the operation unit is projected onto a wall or the like, and the image is used (for example, the user points) to perform operations. Also, the sound acquisition unit 440 or the like that receives various operations based on the voice uttered by the user may be used as the reception unit 461.

[0131] The output unit 462 outputs various information based on the control of the control unit 450. For example, the output unit 462 can be configured with a display unit and a sound output unit. This display unit displays various images based on an instruction from the control unit 450. As the display unit, for example, a display panel such as an organic EL (Electro Luminescence) panel or an LCD (Liquid Crystal Display) panel can be used. The sound output unit outputs various sounds based on an instruction from the control unit 450. As the sound output unit, for example, one or more speakers can be used. Note that the display unit, the sound output unit, the sound acquisition unit 440, and the reception unit 461 are examples of user interfaces, and some of them may be omitted, or other user interfaces may be used.

[0132] [Tagging example] Here, tagging applied to image information stored in image information DB 340 will be described. For example, image information uploaded to server 1 by a housing developer using the service in this embodiment is stored in image information DB 340. That is, image information uploaded to server 1 by the housing developer is registered as a house image. When a house image is registered in this manner, a tagging process is executed for the house image. Then, the registered house image and the tag generated for it are stored in image information DB 340 in association with each other.

[0133] Here, as described above, a tag is a criterion for classifying images (house images) based on the properties of each image, and also means an item that serves as a criterion for the classification. In other words, tagging of house images can be referred to as giving meaning to the house images. Furthermore, a tag means an item that serves as a criterion when extracting the features (feature amount) of an image (house image). Furthermore, a tag can also be referred to as information for identifying an image (house image). Note that a label described below also means information that represents the features, attributes, etc. of an image (house image). In this way, a tag and a label have approximately the same or similar meanings, and therefore, hereinafter, a tag and a label may be used to mean the same thing.

[0134] In addition, in this embodiment, information in which a tag and a corresponding feature (feature amount) are associated with each other will be referred to as tag information. However, tag information can also be referred to as a tag dictionary, metadata, associated information, additional information, etc. In addition, in this embodiment, an example will be described in which a quantified feature amount is used as a feature corresponding to a tag.

[0135] 12(A) shows an example of tags that are added to a house image. As described above, the example shows the use of the tags "location", "style", "material", "item", and "emotion".

[0136] A location tag means a label indicating a location included in a house image. Note that a single house image may contain multiple locations. For example, if a single house image contains an "entrance" and a "staircase", multiple locations will be included. Also, a hierarchical structure may be formed having a certain location (e.g., living room) and another location (a split-level floor) linked to it. For example, location tags relating to locations included in a house image may include the exterior of the entire house, the entrance, the stairs, the living room, the kitchen, the bathroom, rooms, etc. Note that a location tag may be expressed in multiple levels (e.g., five levels from 0 to 4, a range from 0 to 1), for example.

[0137] A style tag means a label that indicates the style of a place included in a house image. Note that a house or the like included in the same house image may have multiple styles. For example, if "natural" and "Japanese style" are included in the same house image, multiple styles will be included. Also, for example, if the exterior of a house included in a house image is a wooden exterior that is close to nature, information such as "natural" and "woody" is assigned as a style tag. Also, as described above, it is possible to adopt various style standards known in various fields of architecture, such as "urban," "dark tone," "woody," "natural," "Japanese style," "vintage," "old folk house," "modern," and "simple," as style tags. Note that style tags can be expressed in multiple stages (for example, five stages from 0 to 4, a range of 0 to 1). For example, when expressed in a range of 0 to 1, if a modern house image contains woody elements, evaluation values ​​such as modern: 0.7 and woody: 0.4 can be assigned.

[0138] A material tag means a label that indicates a material contained in a house image. Note that the same house image may contain multiple materials. For example, if the same house image contains "wood" and "glass," multiple materials will be included. For example, information on characteristic materials such as "wood," "wood grain," "tile," "concrete / mortar," "steel / iron," "designed wallpaper," "glass," and "stainless steel" is added as a material tag as a material contained in the house image. Note that the material tag can be expressed in multiple stages (for example, five stages from 0 to 4, a range from 0 to 1).

[0139] An item tag means a label representing an item such as a wood stove or a solar panel. Note that a plurality of items may be included in the same house image. For example, if a "wood stove" and a "solar panel" are included in the same house image, a plurality of items will be included. For example, if a wood stove is included in the house image, an item tag indicating "wood stove present" is added as a wood stove item, and if a solar panel is included in the house image, an item tag indicating "solar panel present" is added as a solar panel item. On the other hand, if a wood stove is not included in the house image, an item tag indicating "no wood stove" is added as a wood stove item, and if a solar panel is not included in the house image, an item tag indicating "no solar panel" is added as a solar panel item. Note that it is also possible to add a label representing an item other than a wood stove or a solar panel as an item tag. Also, the item tag can be expressed in multiple stages (for example, five stages from 0 to 4, a range from 0 to 1).

[0140] An emotion tag means a label that expresses the mood and appearance of a house image. This emotion tag can be expressed in multiple stages (for example, five stages from 0 to 4, ranging from 0 to 1). For example, emotion standards such as "family gathering / warmth," "openness," "luxury / elegance," "calm," "cool / chic," "flashy," "casual / pop," "solid," "sophisticated / stylish," and "cute" are adopted as emotions related to house images. Then, for example, information on the results of evaluating each of these emotion standards in the range from 0 to 1 for a house image is assigned as an emotion tag.

[0141] [Example of tagging done by humans] When a housing developer uploads a house image to the server 1, it is possible to execute a tagging process in which tags are manually assigned to the house image. For example, a person can judge the location, style, material, items, and emotions related to the house image contained in the house image, and assign the above-mentioned tag information to the house image based on the judgment result. However, when tags are assigned manually in this manner, personnel with specialized knowledge are required at the time of registration, and it is often difficult to obtain such personnel. Therefore, an example of executing the tagging process using artificial intelligence (AI) or the like is shown below. Note that in this embodiment, an example of assigning tags mainly using AI is described.

[0142] [Example of tagging using AI] [Examples of images and teacher labels used for learning] FIG. 12(B) shows an example of image IMG1 used for learning. Note that image IMG1 is an example of an image (e.g., a house image) used for learning. That is, when executing a tagging process for adding tags to house images using AI, various images such as image IMG1 can be used as training data. Note that image IMG1 is a simplified image for ease of explanation.

[0143] In addition, in this embodiment, an example of generating an AI model for each tag pattern (location, style, material, item, emotion) is shown. That is, an example of generating five AI models that have learned each of location, style, material, item, and emotion is shown. Note that the generation of AI models is not limited to this, and one AI model may be generated for all tag patterns (location, style, material, item, emotion), or multiple AI models may be generated for two to four patterns out of all tag patterns (location, style, material, item, emotion).

[0144] When learning image IMG1, specific tags are assigned as teacher labels to image IMG1. Then, the tag patterns (location, style, material, item, emotion) that correspond to each AI model to be learned are extracted and used for learning.

[0145] An example of a teacher label for image IMG1 is shown below. That is, an example of a tag to be added as a teacher label to image IMG1 used for learning is shown below. Note that the contents ((1) to (5)) shown below are simplified contents for ease of explanation, and are not limited thereto. (1) Location tag: Living room: Other, split floor (2) Style Tags: Natural, Woody (3) Material tags: wood, wood grain, carpet (4) Item tag: Not applicable (5) Emotion tags: Family gathering / warmth: 0.875 High-class, elegant, luxurious: 0.125 Cool / chic: 0.375 Casual Pop: 0.5 Sophisticated / Stylish: 0.75 Openness: 0.375 Calming: 0.75 Flashy:0 Weight: 0 Cute: 0.375

[0146] As shown in FIG. 12(A), the tags to be assigned are determined in advance. Each tag is a multi-label that allows overlapping. For example, each of the location tag, style tag, and material tag is set to "1" if the corresponding tag is included in the image IMG1, and set to "0" if the corresponding tag is not included in the image IMG1. That is, in the above example, since the image IMG1 contains "living room" and "skid floor", (1) the location tag "living room" is assigned a "1", the other location tag "skid floor" is assigned a "1", and the other location tags are assigned a "0". Furthermore, since the image IMG1 contains "natural" and "woody", (2) the style tags "natural" and "woody" are assigned a "1", and the other style tags are assigned a "0". Furthermore, since the image IMG1 contains "wood / wood grain" and "carpet", (3) the material tags "wood / wood grain" and "carpet" are assigned a "1", and the other material tags are assigned a "0".

[0147] Also, for example, the emotion tag is a tag that quantifies the degree of applicability between 0 and 1. In this case, an emotion tag of 0 means that the degree of the corresponding emotion is the lowest, and an emotion tag of 1 means that the degree of the corresponding emotion is the highest.

[0148] Also, for example, an item tag is a tag that indicates whether or not a corresponding thing exists in an image. That is, the item tag is set to "1" when the corresponding thing is contained in image IMG1, and is set to "0" when the corresponding thing is contained in image IMG1. That is, in the above example, since image IMG1 does not contain an object that corresponds to the item tag, (4) "0" is assigned to each item of the item tag. That is, it is assigned that image IMG1 does not contain an object that corresponds to the item tag.

[0149] Similarly, for images other than the image IMG1 used for learning (for example, an image of a house), predetermined tags are added as teacher labels and used for learning.

[0150] In this example, the location tag, style tag, material tag, and item tag are each assigned a value of 0 or 1, but at least one of these tags can be expressed in multiple levels (for example, in the range of 0 to 1) in the same way as the emotion tag. As described above, for example, a modern house image with woody elements can be given style tags such as modern: 0.7, woody: 0.4, etc.

[0151] [Example of generating an AI model for learning] An AI model (for example, a machine learning model generated by machine learning) is generated using the above-mentioned teacher data (for example, image IMG1) and teacher labels (for example, each tag information). Note that the learning shown in this embodiment means finding the regularity behind a large amount of data based on these data. Also, the AI ​​model generated by the learning shown in this embodiment is generated by various learning algorithms. For example, it is possible to load teacher data (for example, image IMG1) and teacher labels (for example, each tag information) and learn in advance, and use this AI model for tagging processing. Note that, for example, CNN (Convolutional Neural Network), ViT (Vision Transformer), etc. can be used as the AI ​​model. Note that these can be executed by installing a predetermined application on the server 1.

[0152] [Example of output from a trained AI model] As described above, an AI model is generated for each tag pattern (location, style, material, item, emotion), so five AI models are prepared for each tag pattern. For this reason, house images registered in the server 1 are input to each of the five AI models. As a result, the five AI models output the results of tagging for location, style, material, item, and emotion, respectively. In this way, using the five AI models, it is possible to output the input image with tag information related to the set tags attached. It is also possible to output the input image other than the house image with tag information related to the set tags attached.

[0153] [Image information DB configuration example] FIG. 13 is a simplified diagram showing the contents of the image information stored in the image information DB 340. As shown in FIG.

[0154] The image information DB 340 is a database for storing house images provided by housing developers in association with the feature amounts of a plurality of tags generated based on the images.

[0155] Specifically, image information 342 , tag information 343 , and home builder identification information 344 are stored in image information DB 340 in association with image identification information 341 .

[0156] The image identification information 341 is identification information for identifying each house image prepared by the housing developer to be provided to the user. Note that, for ease of explanation, in Fig. 13, an example is shown in which only a numerical value is stored as the identification information in the image identification information 341, but other identification information may be used.

[0157] The image information 342 is information about an image prepared by the housing developer to be provided to the user. In Fig. 13, for ease of explanation, an example is shown in which only the image is stored in the image information 342, but various attribute information stored in association with the image may also be included.

[0158] The tag information 343 is tag information generated for the image stored in the image information 342. Specifically, tag information assigned by the above-mentioned tag assignment process is stored. Among this tag information, emotion and style are used when generating a user vector. Among this tag information, location, material, and item can be used to narrow down searches for house images, housing businesses, etc.

[0159] For example, when house images stored in the image information DB 340 are displayed on the UI unit 460 (see FIG. 11) of the user terminal 2, it is possible to use it when narrowing down house images that the user likes. For example, if a user is particular about living rooms, it is possible to display house images limited to "living rooms" on the UI unit 460 based on the location tag associated with the house images. In this case, the user can select a house image that he / she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" added to the location tag "living room"). This makes it possible to generate a user vector (described later) that better reflects his / her preferences. Also, for example, if a user likes rooms with stoves, it is possible to display house images limited to house images with "stoves" on the UI unit 460 based on the item tag associated with the house images. In this case, the user can select a house image that he / she likes from among the house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" added to the item tag "stove"). This makes it possible to generate a user vector that better reflects his / her preferences. Also, for example, if a user likes rooms with exposed concrete, it is possible to display on the UI unit 460 house images limited to house images with "concrete" based on the material tag associated with the house image. In this case, the user can select a house image of his / her preference from among house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" added to the material tag "concrete"). This makes it possible to generate a user vector that better reflects his / her preference. Also, for example, if a user wants to be recommended only home builders who deal with solar panels, it is possible to display on the UI unit 460 house images limited to house images with "solar panels" based on the item tag associated with the house image. In this case, the user can select a house image of his / her preference from among house images displayed on the UI unit 460 (house images with "1 (or a number greater than 0)" added to the item tag "solar panels").This makes it possible to more appropriately recommend home builders who deal in solar panels. Note that illustrations and other explanations regarding the narrowing down of searches for home images, home builders, etc. are omitted.

[0160] The housing developer identification information 344 is identification information for identifying the housing developer who provided the image stored in the image information 342. Note that, for ease of explanation, Fig. 13 shows an example in which the name of the housing developer is stored in the housing developer identification information 344, but other information capable of identifying the housing developer (for example, a serial number or code) may be used.

[0161] Note that the information shown in FIG. 13 is an example of information stored in the image information DB 340, and other information may be stored in association with the information, and some of the information may be omitted as necessary.

[0162] [Example of the housing business information DB configuration] 14 is a simplified diagram showing the contents of the housing developer information stored in the housing developer DB 350. The housing developer DB 350 corresponds to the housing developer DB 400 shown in FIG.

[0163] The housing business DB 350 is a database for storing various pieces of information relating to housing businesses in association with each other.

[0164] Specifically, housing business operator identification information 351, profile information 352, and tag information 353 are stored in association with each other in the housing business operator DB 350.

[0165] The housing developer identification information 351 is identification information for identifying a housing developer. The housing developer identification information 351 corresponds to the housing developer identification information 344 shown in Fig. 13. As with the example shown in Fig. 13, in order to facilitate explanation, Fig. 14 shows an example in which the name of the housing developer is stored in the housing developer identification information 351, but other information capable of identifying a housing developer (for example, a serial number, a code) may be used.

[0166] The profile information 352 is various information related to the housing business operator whose identification information is stored in the housing business operator identification information 351. For example, the address of the housing business operator, introduction information of the housing business operator, etc. This profile information can be edited by the housing business operator as appropriate.

[0167] The tag information 353 is tag information assigned to a housing developer whose housing developer identification information is stored in the housing developer identification information 351. Only information related to "style" and "emotion" out of the tag information 343 shown in FIG. 13 is stored as this tag information. In addition, this tag information may be appropriately set by the housing developer, or may be stored based on the calculation results of each tag information assigned to a plurality of housing images uploaded to the server 1 by the housing developer. For example, it is possible to store the average value of each element related to "style" and "emotion" out of each tag information assigned to a plurality of housing images uploaded to the server 1 as each element related to "style" and "emotion". In addition, the calculation method of each element related to "style" and "emotion" may be the same as the calculation method of the user vector described later. This tag information is used when selecting a housing developer to be recommended to a user.

[0168] Note that the information shown in FIG. 14 is an example of information stored in the housing business operator DB 350, and other information may be stored in association with the information, and some of the information may be omitted as necessary.

[0169] [Example of selecting favorite images] 15 is a diagram showing an example of a selection screen 500 displayed on the UI unit 460 of the user terminal 2. The selection screen 500 is displayed on the UI unit 460 based on the control of the control unit 450.

[0170] The selection screen 500 displays a house image display area 501 that displays a plurality of house images 503 to 508, and a decision button 502. Specifically, the information providing unit 325 of the server 1 transmits selection screen information for displaying the selection screen 500 to the user terminal 2 based on a request from the user terminal 2. When the user terminal 2 receives the selection screen information, the control unit 450 causes the UI unit 460 to display the selection screen 500 based on the received selection screen information. Note that other house images can be displayed in the house image display area 501 based on a user operation.

[0171] Furthermore, when a user operation is received by the receiving unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. For example, when a selection operation (e.g., a touch operation) for selecting a favorite house image displayed in the house image display area 501 is performed by the user, the favorite house image selected by the selection operation is set to a selected state. For example, the favorite house image selected by the selection operation is displayed in a different manner from other house images (e.g., the favorite button 509 (star region) is painted black, surrounded by a thick line, or given a specific color) so that the user can understand the selected state. FIG. 15 shows an example in which the favorite button 509 (star region) is painted black to set the house image 503 to a selected state. Note that after the user performs a selection operation, it is possible to display only the house image selected by the user. This example is shown in FIG. 4. In this case, it is also possible to cancel the selected state of the house image by performing a selection operation (e.g., a click operation) on the favorite button (star region) as shown in FIG. 4.

[0172] Furthermore, when the user performs a selection operation (e.g., a touch operation) to select the decision button 502 after one or more house images are selected by the user, the control unit 450 transmits image selection information related to the one or more house images selected by the user to the server 1. When the server 1 receives the image selection information, the information acquisition unit 321 outputs the received image selection information to the tag generation unit 322 and the vector generation unit 323.

[0173] [Example of generating user vectors] The tag generation unit 322 acquires, from the image information DB 340, tag information associated with one or more house images corresponding to the image selection information transmitted from the user terminal 2. Specifically, the tag generation unit 322 acquires, from tag information 343 of the image information DB 340, one or more pieces of tag information associated with image identification information 341 corresponding to one or more house images corresponding to the image selection information, and outputs the acquired tag information to the vector generation unit 323.

[0174] Next, the vector generation unit 323 generates a user vector related to the user who transmitted the image selection information, based on the tag information output from the tag generation unit 322. Note that when there is one house image corresponding to the image selection information transmitted from the user terminal 2, the tag information associated with the one house image can be used as the user vector.

[0175] Here, in order to properly acquire the preferences of a user who has viewed a house image, it is considered that the appearance of the house image and the impression obtained from the house image are important. Therefore, in this embodiment, an example is shown in which a user vector is generated according to the user's preferences using information on style tags and emotion tags from among the various tag information. As described above, the information on the location tags, material tags, and item tags can be used to narrow down the search. The example of generating a user vector shown here is just an example, and a user vector may be generated using at least one of the style tags, emotion tags, and other tag information (tag information on the location tags, material tags, and item tags).

[0176] For example, the vector generation unit 323 performs an addition process on the elements constituting each of the style tags and emotion tags among the vectors corresponding to the tag information output from the tag generation unit 322, and divides the addition result by the number of vectors to be added, thereby generating a user vector.

[0177] For example, assume that the user selects three house images. Among the tag information (tag vector) associated with the three selected house images, the elements constituting the style tag are [a1, ... , an1], [b1, ... , bn1], [c1, ... , cn1], and the elements constituting the emotion tag are [x1, ... , xn2], [y1, ... , yn2], [z1, ... , zn2]. Here, n1 means the number of items in the style tag, and n2 means the number of items in the emotion tag. The components (numeric values) of each element are values ​​in the range of 0 to 1.

[0178] In this case, the vector generating unit 323 generates a value [(a1+b1+c1) / 3, ..., (an1+bn1+cn1) / 3] obtained by adding up the components of the three multidimensional vectors ([a1, ..., an1], [b1, ..., bn1], [c1, ..., cn1]) and dividing the sum by 3 as a user vector related to the style tag. Similarly, the vector generating unit 323 generates a value [(x1+y1+z1) / 3, ..., (xn2+yn2+zn2) / 3] obtained by adding up the components of the three multidimensional vectors ([x1, ..., xn2], [y1, ..., yn2], [z1, ..., zn2]) and dividing the sum by 3 as a user vector related to the emotion tag. Then, the vector generating unit 323 sets a combination of the user vector related to the style tag and the user vector related to the emotion tag as a user vector. Specifically, the user vector is [(a1+b1+c1) / 3, ..., (an1+bn1+cn1) / 3, (x1+y1+z1) / 3, ..., (xn2+yn2+zn2) / 3]. In this case, each element of the user vector corresponds to each element of the carrier vector.

[0179] In this way, the average value of a plurality of house images selected by the user can be used as the user vector. However, this embodiment is not limited to this, and the user vector can be generated by other calculation methods. For example, it is possible to adopt a calculation method in which the average is calculated after using a softmax function only for the style tag of each house image. Also, for example, it is possible to adopt a calculation method in which the average is calculated using a Max function. Also, in this embodiment, an example is shown in which the user vector is generated using elements constituting each of the style tag and the emotion tag, but this is not limited to this, and the user vector may be generated using at least one element other than the style tag and the emotion tag, or the user vector may be generated using other tag information.

[0180] [Example of housing business selection] Based on the user vector generated by the vector generation unit 323, the extraction unit 324 extracts a housing provider that meets the user's preferences corresponding to the user vector, from among the housing providers whose housing provider information is stored in the housing provider DB 350. For example, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the provider vector stored in the tag information 353 of the housing provider DB 350, and extracts a housing provider that meets the user's preferences based on the comparison result.

[0181] As described above, the business vector (tag information 353) is composed of a combination of a style vector and an emotion vector. Here, an example is shown in which the business vector (tag information 353) of a certain business is expressed as [S1, ..., Sn1, E1, ..., En2]. Here, n1 means the number of items in the style tag, and n2 means the number of items in the emotion tag. Also, the components (numeric values) of each element correspond to the values ​​of the tag information 353.

[0182] In this case, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the operator vector (tag information 353) [S1, ..., Sn1, E1, ..., En2] of a certain operator, and determines the similarity between them based on the comparison result. For example, if the user vector is [(a1+b1+c1) / 3, ..., (an1+bn1+cn1) / 3, (x1+y1+z1) / 3, ..., (xn2+yn2+zn2) / 3], the extraction unit 324 compares the user vector with the operator vector [S1, ..., Sn1, E1, ..., En2] of a certain operator.

[0183] For example, the extraction unit 324 calculates the cosine similarity between the user vector and each business vector for each business vector. A known calculation method can be used to calculate the cosine similarity. When calculating the cosine similarity, it is preferable to normalize each vector. Then, the extraction unit 324 extracts the housing business corresponding to the business vector with the highest cosine similarity (i.e., the smallest cosine value with the user vector) as the housing business according to the user's preference. Note that a high cosine similarity means that the distance is short in the vector space. In addition, when multiple housing businesses are provided to the user, the extraction unit 324 can extract a predetermined number (for example, about 2 to 10) of housing businesses with high calculated cosine similarity. The number of housing businesses extracted in this way may be specified based on a user operation using the user terminal 2.

[0184] The process of extracting housing businesses may be performed using other calculation methods. For example, the extraction unit 324 calculates the difference (for each corresponding component) between each component of the user vector and each component of each business vector (component corresponding to each component of the user vector) for each business vector, and calculates the sum of these difference values ​​for each business vector. The extraction unit 324 may then extract the housing business corresponding to the business vector with the smallest sum of the difference values ​​as the housing business that meets the user's preferences.

[0185] In this way, one housing business operator corresponding to the business vector most similar to the user vector may be selected, or multiple housing business operators corresponding to a predetermined number of business vectors having high similarity to the user vector may be selected in descending order of similarity. In other words, one housing business operator corresponding to the business vector closest to the user vector may be selected, or multiple housing business operators corresponding to business vectors having close distances may be selected in descending order.

[0186] The extraction unit 324 also outputs the extracted information related to the housing business owner (for example, the profile information 352 and the image information 342) to the information provision unit 325.

[0187] Then, the information providing unit 325 transmits to the user terminal 2 selected housing provider information (for example, the profile information 352, the image information 342) relating to the extracted housing provider.

[0188] [Example of selected business information display] 16 is a diagram showing an example of a selected business operator information screen 510 displayed on the UI unit 460 of the user terminal 2. The selected business operator information screen 510 is displayed on the UI unit 460 based on the control of the control unit 450.

[0189] The selected business operator information screen 510 displays information about the housing business operator extracted by the extraction unit 324 of the server 1 (for example, the profile information 352, the image information 342). For example, the selected business operator information screen 510 displays a profile information display area 511, a housing image display area 512 displaying a plurality of housing images 516 to 518 associated with the housing business operator extracted by the extraction unit 324 of the server 1, a re-propose new image button 514, and a decision button 515. Specifically, the information providing unit 325 of the server 1 transmits screen information for displaying the selected business operator information screen 510 to the user terminal 2. When the user terminal 2 receives the screen information, the control unit 450 causes the UI unit 460 to display the selected business operator information screen 510 based on the received screen information. In addition, the housing image display area 512 can display other housing images associated with the extracted housing business operator based on a user operation.

[0190] Furthermore, when a user operation is received by the receiving unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. For example, after checking the profile displayed in the profile information display area 511 and the multiple house images 516 to 518 displayed in the house image display area 512, when the user decides on the housing builder of the profile displayed in the profile information display area 511 as the preferred housing builder, the user performs a selection operation (e.g., a touch operation) to select the decision button 515. When the selection operation of the decision button 515 is performed, the control unit 450 transmits decision information indicating that the selection operation has been performed to the server 1. When the server 1 receives the decision information, the control unit 320 executes a predetermined process for enabling some kind of exchange between the housing builder corresponding to the received decision information and the user. As this predetermined process, for example, at least one of a transmission process of transmitting information about the user (information with the user's consent) to the housing provider corresponding to the determined information, and a transmission process of transmitting information for accessing the housing provider corresponding to the determined information (for example, access information such as a telephone number, email address, URL, SNS, etc.) to the user is executed. This allows the user to easily and quickly access a housing provider according to his / her preferences.

[0191] Here, it is assumed that after checking the profile displayed in the profile information display area 511 and the multiple home images 516 to 518 displayed in the home image display area 512, the user may request a re-proposal of the home builder of the profile displayed in the profile information display area 511. For example, it is conceivable that a user who has checked the address included in the profile displayed in the profile information display area 511 may request a re-proposal because the location is far away. It is also conceivable that a user who has checked the multiple home images 516 to 518 displayed in the home image display area 512 may request a re-proposal because there is no image to his / her liking.

[0192] In this way, when the user requests a re-proposal of the housing developer of the profile displayed in the profile information display area 511, the user performs a selection operation (for example, a touch operation) to select the re-proposal button 514 for a new image. When the selection operation of the re-proposal button 514 for a new image is performed, the control unit 450 transmits re-proposal request information indicating that the selection operation has been performed to the server 1. When the server 1 receives the re-proposal request information, the extraction unit 324 extracts a house image to be newly provided to the user from the image information DB 340 based on the user vector generated by the vector generation unit 323. For example, the extraction unit 324 compares the user vector generated by the vector generation unit 323 with the tag information (style, emotion) stored in the tag information 343 of the image information DB 340, and based on the comparison result, newly extracts a house image according to the user's preference. Specifically, the extraction unit 324 calculates a difference value between each component constituting the user vector and each component constituting each tag information (each vector) for each tag information, and calculates a total value of the difference values ​​for each tag information. Then, the extraction unit 324 extracts a house image corresponding to a predetermined number of tag information pieces with a small total value of the difference values ​​as a new house image according to the user's preference. Note that, by using the cosine similarity of each vector to be compared, a house image corresponding to a predetermined number of tag information pieces with a high cosine similarity may be extracted as a new house image according to the user's preference. Also, a new house image according to the user's preference may be extracted using another calculation method.

[0193] In addition, the extraction unit 324 outputs to the information provision unit 325 house image information (for example, image information 342) relating to the newly extracted house image.

[0194] [Example of new home image selection] 17 is a diagram showing an example of a selection screen 520 displayed on the UI unit 460 of the user terminal 2. The selection screen 520 is displayed on the UI unit 460 based on the control of the control unit 450.

[0195] The selection screen 520 displays a house image display area 521 that displays a plurality of house images 523 to 528, and a decision button 522. The selection screen 520 is a display screen corresponding to the selection screen 500 shown in Fig. 15, and the house image display area 521 and the decision button 522 correspond to the house image display area 501 and the decision button 502. Therefore, detailed explanations of these will be omitted. Furthermore, the selection operation and decision operation of each image are similar to the example shown in Fig. 15, and detailed explanations of these will be omitted.

[0196] Furthermore, when the user performs a selection operation (e.g., a touch operation) to select the decision button 522 after one or more house images have been selected by the user, the control unit 450 transmits image selection information related to the one or more house images selected by the user to the server 1. Note that the tag generation process by the tag generation unit 322, the vector generation process by the vector generation unit 323, and the housing business operator extraction process by the extraction unit 324 using the image selection information are the same as in the above-mentioned example.

[0197] In this way, a new house image can be selected and provided to the user using a user vector generated based on a house image (favorite image) selected by the user. In addition, by performing each of these processes multiple times, it is possible to refine the recommendation to the user. In addition, a new house image may be selected using a past calculation result (for example, a user vector from one or several times ago).

[0198] [Server operation example] Fig. 18 is a flow chart showing an example of a housing builder selection process by the server 1. This housing builder selection process is executed by the control unit 320 (see Fig. 11) based on a program stored in the storage unit 330 (see Fig. 11). This housing builder selection process is constantly executed at each control cycle. This housing builder selection process will be described with appropriate reference to Figs. 1 to 17.

[0199] In step S601, the information providing unit 325 transmits house image information to the user terminal 2 based on a house image request from the user terminal 2. This house image information is information for displaying a selection screen 500 (see FIG. 15 ) on the UI unit 460 of the user terminal 2. For example, the user terminal 2 may be able to receive the provision of the service in this embodiment by installing a predetermined application (housing business operator application) in the user terminal 2. In this case, the control unit 450 of the user terminal 2 transmits a house image request to the server 1 by activating the housing business operator application in the user terminal 2.

[0200] In step S602, the information acquisition unit 321 determines whether or not image selection information (favorite image information) has been received from the user terminal 2. For example, when the decision button 502 is selected after the user has performed a process of selecting a house image on the selection screen 500 (see FIG. 15), the control unit 450 of the user terminal 2 transmits the image selection information to the server 1. If the image selection information has been received, the process proceeds to step S603. On the other hand, if the image selection information has not been received, monitoring continues.

[0201] In step S603, the tag generation unit 322 acquires tag information associated with one or more house images corresponding to the image selection information received in step S602 from the image information DB 340. Note that the method of acquiring this tag information is the same as the acquisition method described above.

[0202] In step S604, the vector generation unit 323 generates a user vector related to the user of the user terminal 2 that transmitted the image selection information, based on the tag information acquired in step S603. Note that the method of generating this user vector is the same as the above-mentioned generation method.

[0203] In step S605, based on the user vector generated in step S604, the extraction unit 324 extracts and selects a housing provider that meets the preferences of the user corresponding to the user vector from among the housing providers whose housing provider information is stored in the housing provider DB 350. The selection method (extraction method) of this housing provider is the same as the extraction method described above.

[0204] In step S606, the information providing unit 325 transmits selected housing business operator information (e.g., profile information 352, image information 342) relating to the housing business operator selected in step S605 to the user terminal 2. When the user terminal 2 receives this selected housing business operator information, the control unit 450 of the user terminal 2 causes the UI unit 460 to display a selected business operator information screen 510 (see FIG. 16).

[0205] In step S607, the information acquisition unit 321 determines whether or not decision information has been received from the user terminal 2 that transmitted the selected housing business information in step S606. For example, when the user selects the decision button 515 on the selected business information screen 510 (see FIG. 16), the control unit 450 of the user terminal 2 transmits the decision information to the server 1. If the decision information has been received, the process proceeds to step S608. On the other hand, if the decision information has not been received, the process proceeds to step S609.

[0206] In step S608, the control unit 320 executes a predetermined process for introducing the housing provider corresponding to the decision information received in step S607 (selected housing provider) to the user of the user terminal 2. This predetermined process is similar to the predetermined process described above.

[0207] In step S609, the information acquisition unit 321 determines whether or not re-proposal request information has been received from the user terminal 2 that transmitted the selected housing business operator information in step S606. For example, when the user selects the re-propose button 514 for a new image on the selected business operator information screen 510 (see FIG. 16), the control unit 450 of the user terminal 2 transmits the re-proposal request information to the server 1. If the re-proposal request information has been received, the process proceeds to step S610. On the other hand, if the re-proposal request information has not been received, the process returns to step S607.

[0208] In step S610, the extraction unit 324 extracts, from the image information DB 340, a house image to be newly provided to the user, based on the user vector generated in step S604.

[0209] In step S611, the information providing unit 325 transmits house image information (e.g., image information 342) related to the new house image extracted in step S611 to the user terminal 2. This house image information is information for displaying the selection screen 520 (see FIG. 17) on the UI unit 460 of the user terminal 2.

[0210] [Example of effect] Here, it is assumed that although the user has an abstract ideal image and preferences regarding a preferred home, it is difficult to clearly verbalize the image and desires of the home. For this reason, if the user is asked to answer questions such as the image and desires of the home, and a housing business is proposed based on the user's answers, there is a risk that an appropriate housing business may not be proposed. Therefore, in this embodiment, it is possible to appropriately acquire the user's preferences by selecting a housing image according to the user's preferences from among a plurality of housing images displayed on the user terminal 2. Then, a user vector is generated using tag information related to the housing image according to the user's preferences, and a housing business is proposed to the user based on the comparison result between this user vector and the housing business vector, so that it is possible to propose an appropriate housing business according to the user's preferences. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0211] [Examples of user edited house images] Here, it is assumed that none of the house images provided to the user match the user's preferences. In this case, it is assumed that the user will have difficulty selecting a preferred house image. Therefore, it is conceivable that the house images provided to the user can be temporarily edited to match the user's preferences, making it easier for the user to select a preferred house image.

[0212] [Item information DB configuration example] 19 is a diagram showing, in a simplified form, the stored contents of the item information stored in the item information DB 370. The item information DB 370 may be stored in the storage unit 330 of the server 1, or may be stored in an external device other than the server 1 for use.

[0213] The item information DB 370 is a database for storing items for editing images provided by the housing developer and the feature amounts of a plurality of tags generated for the items in association with each other.

[0214] Specifically, image information 372 and tag information 373 are stored in item information DB 370 in association with item identification information 371 .

[0215] The item identification information 371 is identification information for identifying each item prepared by the service provider. In Fig. 19, for ease of explanation, an example is shown in which only the name indicating the item is stored as identification information in the item identification information 371, but other identification information may be used.

[0216] Image information 372 is information about an item image prepared by a service provider so that the user can edit it. In Fig. 19, for ease of explanation, an example is shown in which only an image is stored in image information 372, but various attribute information (e.g., size information) stored in association with the image may also be included.

[0217] Tag information 373 is tag information generated for the item image stored in image information 372. This tag information is used when generating a user vector. Note that, although Fig. 19 shows an example of storing tag information having the same content as tag information 343 shown in Fig. 13, only the necessary information may be stored and other information may be omitted.

[0218] [Example of a user editing a house image] 20 is a diagram showing an example of an edit screen 700 displayed on the UI unit 460 of the user terminal 2. The edit screen 700 is displayed on the UI unit 460 under the control of the control unit 450.

[0219] The editing screen 700 displays a house image display area 701 that displays a plurality of house images 703 to 706, a decision button 702, and an item display area 710. Specifically, the information providing unit 325 of the server 1 transmits editing screen information for displaying the editing screen 700 to the user terminal 2. When the user terminal 2 receives the editing screen information, the control unit 450 causes the UI unit 460 to display the editing screen 700 based on the received editing screen information. Note that the house image display area 701 corresponds to the house image display area 501 shown in FIG. 15 .

[0220] The item display area 710 is an area for displaying each of the item images 711 to 718 whose item information is stored in the item information DB 370 (see FIG. 19). The user can generate an image of the user's preference by performing a moving operation to move a desired item displayed in the item display area 710 to a desired house image displayed in the house image display area 701. For example, when the UI unit 460 is configured with a touch panel, the user can perform an operation (e.g., a drag & drop operation) to move a desired item while touching it to the position of the desired house image. For example, the item image 712 showing a foliage plant can be arranged in the house image 703 by the user performing an operation to move the item image 712 to a desired position in the house image 703. The transition of this movement is shown typically by an arrow 720. In addition, the house image in which the item image is arranged (the edited house image) is set to a selected state. In addition, the image synthesis process in which the item image is arranged in the house image and synthesized can use a known image processing. In addition, various image processes (e.g., zooming in and out, adjusting the three-dimensional angle, additional editing of the image) may be performed on each item image based on a user operation. For each of these image processes, known image processing may be used. In addition, the item image arranged in the house image may be appropriately transformed in relation to other components in the house image using known image processing techniques. For example, it is possible to zoom in and out on the item image based on the size of each object in the house image, or to adjust the angle of the item image based on the inclination of each object in the house image.

[0221] In this way, when a home image that matches the user's preferences is not displayed in the home image display area 701, it is possible to generate a home image that matches the user's preferences by editing a part of each home image displayed in the home image display area 701. Note that the home image generated by this editing is an image that is generated temporarily in order to select a home builder, and is deleted after the home builder selection process is completed.

[0222] In this way, when a user operation is accepted by the accepting unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. For example, when a selection operation (e.g., a touch operation) is performed by the user to select a house image to be edited from among the house images displayed in the house image display area 701, the preferred house image selected by the selection operation is placed in a selected state.

[0223] Furthermore, when the user executes a selection operation (e.g., a touch operation) of selecting the decision button 702 after editing one or more house images, the control unit 450 transmits edited image information regarding the one or more house images edited by the user to the server 1. When the server 1 receives the edited image information, the information acquisition unit 321 outputs the received edited image information to the tag generation unit 322 and the vector generation unit 323.

[0224] [Example of generating user vectors] The tag generation unit 322 acquires tag information for one or more edited images corresponding to the edited image information transmitted from the user terminal 2 from the image information DB 340 and the item information DB 370. Specifically, the tag generation unit 322 acquires tag information associated with the original house image of each edited image from the image information DB 340, and acquires tag information associated with items added to each edited image by editing from the item information DB 370. Then, the tag generation unit 322 outputs each piece of acquired tag information to the vector generation unit 323.

[0225] Next, the vector generation unit 323 generates a user vector related to the user who transmitted the edited image information based on the tag information output from the tag generation unit 322. In this case, the method of generating a user vector is the same as that described above, except that the tag information associated with the item is used, and therefore detailed description here will be omitted.

[0226] Here, it is considered that the user's preference is reflected more in the edited image in which an item is added by the user than in the unedited house image. Therefore, the user vector may be generated by weighting the tag information associated with the edited image in which an item is added by the user more than other tag information. It is also assumed that the edited image has a plurality of items added. In the case of such an edited image, it is considered that the user's preference is further reflected. Therefore, for example, the user vector may be generated by increasing the weighting for the edited image and the weighting for the added items according to the number of items added to the edited image. That is, the user vector may be generated by changing the weighting for the edited image and the weighting for the added items based on various information related to the items added to the edited image (for example, the number of items, the ratio of item images in the house image). This makes it possible to generate a user vector that reflects the user's preference.

[0227] Here, as shown in Fig. 20, when a house image is edited by adding an item to the house image, it is assumed that the style and emotion will differ depending on the position of the item in the house image. Therefore, using the above-mentioned AI model related to style and emotion, the tag generation unit 322 may generate new tag information related to style and emotion for the edited image. In this case, it is possible to generate more appropriate tag information according to the editing result edited by the user.

[0228] [Example of user terminal operation] Fig. 21 is a flow chart showing an example of a house image editing process by the user terminal 2. This house image editing process is executed by the control unit 450 (see Fig. 11) based on a program stored in the storage unit 470 (see Fig. 11). This house image editing process is constantly executed at each control cycle. This house image editing process will be described with appropriate reference to Figs. 1 to 20.

[0229] In step S751, the control unit 450 transmits editing screen request information for requesting editing screen information to the server 1. For example, when a user operation for requesting an editing screen is performed, the control unit 450 transmits the editing screen request information to the server 1.

[0230] In step S752, the control unit 450 determines whether or not editing screen information has been received from the server 1. If editing screen information has been received, the process proceeds to step S753. On the other hand, if editing screen information has not been received, the process continues to monitor.

[0231] In step S753, control unit 450 causes UI unit 460 to display an editing screen (for example, editing screen 700 (see FIG. 20)) based on the editing screen information received in step S752.

[0232] In step S754, the control unit 450 determines whether or not a user operation on the editing screen displayed in step S753 has been accepted by the accepting unit 461. If a user operation has been accepted, the process proceeds to step S755. On the other hand, if a user operation has not been accepted, the process continues to monitor.

[0233] In step S755, the control unit 450 determines whether or not the user operation accepted in step S754 is an editing operation for editing the house image displayed on the editing screen. For example, when a moving operation is performed to move a desired item displayed in the item display area 710 (see FIG. 20) to a desired house image displayed in the house image display area 701 (see FIG. 20), it is determined that an editing operation has been performed as the user operation. If the user operation is an editing operation, the process proceeds to step S756. On the other hand, if the user operation is not an editing operation, the process proceeds to step S757.

[0234] In step S756, the control unit 450 executes an editing process to edit the house image based on the user operation (editing operation) accepted in step S754. For example, when a move operation is performed to move an item displayed in the item display area 710 to the house image displayed in the house image display area 701, image processing is executed to move the item in response to the move operation and composite it with the house image. Also, for example, when a delete operation is performed to delete an item moved to the house image, image processing is executed to delete the item in response to the delete operation.

[0235] In step S757, the control unit 450 determines whether or not the user operation accepted in step S754 is a decision operation that is performed after one or more house images are edited. For example, when a selection operation (e.g., a touch operation) for selecting the decision button 702 (see FIG. 20) is performed, it is determined that the decision operation is performed as the user operation. If the user operation is the decision operation, the process proceeds to step S759. On the other hand, if the user operation is not the decision operation, the process proceeds to step S758.

[0236] In step S758, the control unit 450 executes a predetermined process based on the user operation accepted in step S754. For example, when a selection operation is performed to select the house image displayed in the house image display area 701, image processing is executed to select the house image in response to the selection operation.

[0237] In step S759, the control unit 450 transmits to the server 1 edited image information regarding one or more house images edited by the user.

[0238] In this way, by allowing the user to edit a home image according to the user's preferences from among the multiple home images displayed on the user terminal 2, it is possible to more appropriately express the user's preferences based on the user's subjective editing. Therefore, it is possible to propose appropriate housing developers according to the user's preferences based on the user's editing results. In other words, it is possible to support the user's home selection in an intuitive and easy-to-understand manner.

[0239] In the above, an example of editing a house image using an image (item image) has been shown, but a house image may be edited using text information input by a user (for example, manual input, voice input, gesture input). For example, it is possible to input "I want to put a piano" or "I want to put a sofa" as text information. In this case, for example, it is possible to reflect the input text information on the house image in the selected state. For example, it is possible to generate an image related to the input text information using a generative AI (for example, an image generation AI (for example, Stable Diffusion, Midjourney)) that has learned about each style. Then, it is possible to generate a desired edited image by synthesizing the generated image (image related to the text information) with the house image in the selected state. In this case, for example, a known image synthesis technique can be used. Thereby, it is possible to generate a desired edited image by following the atmosphere of the style related to the input text information. In this case, for example, a user vector may be generated by increasing the weight associated with the edited image and the weight associated with the input text information according to the weight of the input text information (for example, the number of characters). Thereby, it is possible to generate a user vector that reflects the user's preferences.

[0240] [Example using user-acquired images] As described above, it is possible that none of the house images provided to the user match the user's preferences. In this case, it is possible that the user has difficulty selecting a preferred house image. Therefore, it is possible to select a housing business operator that matches the user's preferences by generating a user vector that matches the user's preferences using an image acquired by the user.

[0241] [Image information DB configuration example] FIG. 22 is a simplified diagram showing the contents of the image information stored in the image information DB 800. As shown in FIG.

[0242] The image information DB 800 is a database stored in the storage unit 470 (see FIG. 11) of the user terminal 2. The image information DB 800 also stores image information acquired by the image acquisition unit 430. For example, when a smartphone is used as the user terminal 2, a camera mounted on the smartphone corresponds to the image acquisition unit 430.

[0243] When an image acquisition operation (for example, a still image shooting operation or a video shooting operation) is performed by the user, the control unit 450 of the user terminal 2 records the image acquired by the image acquisition unit 430 based on the acquisition operation as an image file (still image file, video file) in the image information DB 800. For example, as shown in FIG. 22, the image file is stored in the image information 802.

[0244] In this case, the control unit 450 associates the location information (e.g., latitude and longitude) acquired by the location information acquisition unit 420 of the user terminal 2 with the image file and stores it based on the timing of the acquisition operation. For example, as shown in Fig. 22, the acquired location information is stored in the location information 803 in association with the image file. For example, when a smartphone is used as the user terminal 2, a GPS device mounted on the smartphone corresponds to the location information acquisition unit 420.

[0245] Furthermore, the control unit 450 stores the audio information acquired by the audio acquisition unit 440 of the user terminal 2 in association with the image file based on the timing of the acquisition operation. For example, as shown in FIG. 22, the acquired audio information is stored in the audio information 804 in association with the image file. For example, when a smartphone is used as the user terminal 2, a microphone mounted on the smartphone corresponds to the audio acquisition unit 440. When a still image is captured, audio information of a predetermined time based on the timing of the capture can be stored in association with the still image. Furthermore, when a video is captured, a representative image of the video (for example, the first frame, the frame with the highest feature) may be used as a still image using the audio information acquired.

[0246] For example, a user uses the user terminal 2 to take pictures of a house, the interior of the house, scenery, and other subjects that suit the user's taste, and the images that suit the user's taste are stored in the image information DB 800. Thus, it is possible to generate a user vector according to the user's taste using the image information stored in the image information DB 800. When a stationary device such as a personal computer is used as the user terminal 2, it is possible that the position information acquisition unit 420 and the image acquisition unit 430 are not provided. In this case, an external device may be used as the position information acquisition unit 420 and the image acquisition unit 430, and image information acquired using an imaging device (for example, a smartphone, a digital still camera, a digital video camera (for example, a camera-integrated recorder)) is acquired and stored in the image information DB 800, and this can be used.

[0247] [Example of user selection of preferred image] 23 is a diagram showing an example of a selection screen 810 displayed on the UI unit 460 of the user terminal 2. The selection screen 810 is displayed on the UI unit 460 based on the control of the control unit 450.

[0248] The selection screen 810 displays an image display area 820 that displays a plurality of images 821 to 826, a selected image display area 830 that displays a selected image, a position information designation area 811, an audio information designation area 812, and a decision button 813. Specifically, the information providing unit 325 of the server 1 transmits selection screen information for displaying the selection screen 810 to the user terminal 2. When the user terminal 2 receives the selection screen information, the control unit 450 causes the UI unit 460 to display the selection screen 810 based on the received selection screen information.

[0249] The image display area 820 is an area for displaying each of the images 821 to 826 whose image information is stored in the image information DB 800 of the user terminal 2. That is, the control unit 450 acquires each piece of image information stored in the image information DB 800, and displays it in the image display area 820 of the selection screen 810. Note that the images to be displayed here are not limited to house images, and various kinds of images can be displayed.

[0250] The user can select a desired image by performing a moving operation to move a desired image displayed in the image display area 820 to the selected image display area 830. For example, when the UI unit 460 is configured with a touch panel, the user can perform an operation (e.g., a drag-and-drop operation) to move the desired image to the selected image display area 830 while touching it. For example, the user can place the image 822 in the selected image display area 830 by performing an operation to move the image 822 including a coffee cup to the selected image display area 830. The transition of this movement is shown typically by an arrow 840. Note that the image already displayed in the selected image display area 830 may be composited with another image using a known image processing technique to create a composite image. Also, one or more images may be edited using each of the above-mentioned editing techniques.

[0251] The selected image display area 830 is an area for displaying each of the images 831, 832, and 822 selected based on a user operation. The selected image display area 830 may display each of the house images selected on the screens shown in Fig. 15 and Fig. 17, each of the edited images edited on the editing screen 700 shown in Fig. 20, and the like. When various images are displayed in this manner, it may be possible to appropriately perform operations for deleting, adding, and editing various images based on a user operation.

[0252] FIG. 23 shows an example in which house images 831 and 832 that are in a selected state in FIG.

[0253] The position information designation area 811 is an area that is operated when designating whether or not to use the position information associated with the image displayed in the image display area 820. For example, when a selection operation of selecting the pull-down button of the position information designation area 811 is performed after the image moved to the selected image display area 830 is selected, a selection area for selecting use or non-use is displayed, and the user performs the operation desired in this selection area. For example, when designating to use the position information associated with the selected image, it is possible to generate a user vector using the position information associated with the image. On the other hand, when designating not to use the position information associated with the selected image, it is possible to generate a user vector without using the position information associated with the image.

[0254] The audio information designation area 812 is an area that is operated when designating whether or not to use audio information associated with an image displayed in the image display area 820. For example, when a selection operation is performed to select a pull-down button in the audio information designation area 812 after an image moved to the selected image display area 830 is selected, a selection area for selecting use or non-use is displayed, and the user performs a desired operation in this selection area. For example, when designating to use audio information associated with the selected image, it is possible to generate a user vector using the audio information associated with the image. On the other hand, when designating not to use audio information associated with the selected image, it is possible to generate a user vector without using the audio information associated with the image.

[0255] In this way, when a user operation is accepted by the accepting unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. Furthermore, when the user executes a selection operation (e.g., a touch operation) of selecting the decision button 813 after moving one or more images to the selected image display area 830, the control unit 450 transmits selected image information regarding one or more images selected by the user to the server 1. This selected image information includes image information, position information (if use is specified), and audio information (if use is specified). When the server 1 receives the selected image information, the information acquiring unit 321 outputs the received selected image information to the tag generating unit 322 and the vector generating unit 323.

[0256] [Example of generating user vectors] The tag generating unit 322 generates tag information for one or more selected images corresponding to the selected image information transmitted from the user terminal 2. When a house image stored in the image information DB 340 is selected as the selected image, the tag generating unit 322 acquires the tag information of the house image from the image information DB 340.

[0257] Furthermore, when an image stored in the image information DB 800 of the user terminal 2 is a selected image, the tag generation unit 322 generates tag information related to style and emotion for the image. For example, it is possible to generate tag information related to style and emotion for the image using the above-mentioned AI model related to style and emotion.

[0258] Furthermore, when at least one of location information (when use is specified) and voice information (when use is specified) is included in the selected image information, it is possible to generate tag information related to style and emotion for each piece of information. For example, a database that associates location information (e.g., latitude and longitude) with tag information related to style and emotion (e.g., information previously set for each region) is prepared in advance in the storage unit 330 of the server 1, and tag information corresponding to the location information included in the selected image information can be obtained using this database.

[0259] Also, for example, a database that associates location information (e.g., latitude and longitude) with the characteristics and environmental images of a certain location (e.g., Okinawa is "hot" and Kamakura is "ancient capital" and "calm") can be prepared in advance in the storage unit 330 of the server 1, and this database can be used to obtain the characteristics and environmental images (e.g., Okinawa is "hot" and Kamakura is "ancient capital" and "calm") corresponding to the location information included in the selected image information, and text information related to these characteristics and environmental images (e.g., "hot", "ancient capital", "calm"). In this case, tag information can be generated from the text information using an AI model described later.

[0260] In addition, for example, a database that associates audio information with tag information regarding style and emotion (for example, information pre-set for each type of sound (e.g., "noise," "quiet," "river babbling," "child's voice")) can be prepared in advance in the memory unit 330 of the server 1, and this database can be used to obtain tag information corresponding to the audio information included in the selected image information.

[0261] Also, for example, it is possible to convert voice information into text information. In this case, it is possible to generate tag information from the text information using an AI model described later. Also, tag information may be generated using an AI model that has been trained using location information, voice information, and the like.

[0262] Next, the vector generation unit 323 generates a user vector related to the user who transmitted the image selection information, based on the tag information output from the tag generation unit 322. Note that the method of generating this user vector is similar to the method of generating the user vector described above, and therefore a detailed description thereof will be omitted here.

[0263] In the above, an example has been shown in which tag information is generated using an AI model that has been previously learned and generated, and a user vector is generated using this tag information, but other AI models may be used. For example, a large language model (LLM) may be used as the AI ​​model. For example, ChatGPT (Generative Pre-trained Transformer), Bard, Llama2 (Large Language Model Meta AI 2), multimodal LLM, etc. may be used as the LLM. These are only examples, and other AI models may be used. For example, tag information may be generated using a multimodal LLM for various images. It is also possible that text information is attached to a house image (for example, a property image). In such a case, tag information related to the text information may be generated using an AI model, and the tag information obtained by combining this tag information with tag information related to the house image (for example, a property image) may be used to generate a user vector.

[0264] [Example of user terminal operation] Fig. 24 is a flowchart showing an example of image selection processing by the user terminal 2. This image selection processing is executed by the control unit 450 (see Fig. 11) based on a program stored in the storage unit 470 (see Fig. 11). This image selection processing is constantly executed for each control period. This image selection processing will be described with appropriate reference to Figs. 1 to 23.

[0265] In step S851, control unit 450 transmits selection screen request information for requesting selection screen information to server 1. For example, when a user operation for requesting a selection screen is performed, control unit 450 transmits the selection screen request information to server 1.

[0266] In step S852, the control unit 450 determines whether or not selection screen information has been received from the server 1. If selection screen information has been received, the process proceeds to step S853. On the other hand, if selection screen information has not been received, the process continues monitoring.

[0267] In step S853, control unit 450 acquires, from image information DB 800 (see FIG. 22), image information (eg, image information 802) to be displayed on the editing screen that is displayed based on the selection screen information received in step S852.

[0268] In step S854, control unit 450 causes UI unit 460 to display a selection screen (for example, selection screen 810 (see FIG. 23)) based on the selection screen information received in step S852.

[0269] In step S855, the control unit 450 determines whether or not a user operation on the selection screen displayed in step S854 has been accepted by the accepting unit 461. If a user operation has been accepted, the process proceeds to step S856. On the other hand, if a user operation has not been accepted, the process continues to monitor.

[0270] In step S856, control unit 450 determines whether or not the user operation accepted in step S855 is a selection operation for selecting an image displayed on the selection screen. For example, when a move operation is performed to move an image displayed in image display area 820 (see FIG. 23) to selected image display area 830 (see FIG. 23), it is determined that a selection operation has been performed as the user operation. If the user operation is a selection operation, the process proceeds to step S857. On the other hand, if the user operation is not a selection operation, the process proceeds to step S858.

[0271] In step S857, control unit 450 executes a selection process for selecting an image based on the user operation (selection operation) accepted in step S855. For example, when a move operation is performed to move an image displayed in image display area 820 (see FIG. 23) to selected image display area 830 (see FIG. 23), image processing is executed to move the image in response to the move operation and set it as a selected image. Also, for example, when a delete operation is performed to delete an image moved to selected image display area 830 as a selected image, image processing is executed to delete an image in response to the delete operation from selected image display area 830.

[0272] In step S858, control unit 450 determines whether the user operation accepted in step S855 is a decision operation that is performed after one or more images are selected. For example, when a selection operation (e.g., a touch operation) for selecting decision button 813 (see FIG. 23) is performed, it is determined that the decision operation is performed as the user operation. If the user operation is the decision operation, the process proceeds to step S860. On the other hand, if the user operation is not the decision operation, the process proceeds to step S859.

[0273] In step S859, control unit 450 executes a predetermined process based on the user operation accepted in step S855. For example, when a selection operation is performed to select the house image displayed in selected image display area 830, image processing is executed to select the house image in response to the selection operation.

[0274] In step S860, control unit 450 transmits to server 1 selected image information relating to one or more images selected by the user.

[0275] In this way, by allowing the user to select images according to the user's preferences from among images acquired using the user terminal 2, it is possible to more appropriately express the user's preferences based on the user's subjective image collection. Therefore, it is possible to propose appropriate housing businesses according to the user's preferences based on the user's collection results. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0276] [Example using text information] In the above, an example of selecting a housing business operator using mainly house images has been shown. However, it is assumed that the house image that the user prefers may not be presented to the user. Therefore, in the following, an example of selecting a house image using text information will be shown.

[0277] [Example of generating user vectors based on text information] 25 is a diagram showing an example of a text input screen 900 displayed on the UI unit 460 of the user terminal 2. The text input screen 900 is displayed on the UI unit 460 under the control of the control unit 450.

[0278] The text input screen 900 displays a house image display area 901 that displays a plurality of house images 902 to 904, a text information input field 905, a re-suggest button 906, and a decision button 907. Specifically, the information providing unit 325 of the server 1 transmits screen information for displaying the text input screen 900 to the user terminal 2 based on a request from the user terminal 2. When the user terminal 2 receives the screen information, the control unit 450 causes the UI unit 460 to display the text input screen 900 based on the received screen information. It is possible to display other house images on the text input screen 900 based on a user operation.

[0279] Furthermore, when a user operation is received by the reception unit 461 of the UI unit 460, the control unit 450 executes various processes based on the user operation. Here, it is assumed that the house image display area 901 does not display the user's favorite house image. In such a case, it is possible for the user to input characters related to the favorite house into the text information input field 905 by inputting (for example, manual operation or voice input). FIG. 25 shows an example in which the user inputs "I like houses with a warm feeling of wood." Note that this is just one example, and characters related to the favorite house may be, for example, "traditional English-style house," "house like Buckingham Palace," "I like houses with a more modern design," etc.

[0280] Furthermore, when the user executes a selection operation (e.g., a touch operation) to select the re-suggest button 906 after characters are input in the text information input field 905, the control unit 450 transmits text information corresponding to the characters input by the user to the server 1. When the server 1 receives the text information, the information acquisition unit 321 outputs the received text information to the tag generation unit 322 and the vector generation unit 323. Note that when one or more house images are selected by the user, the control unit 450 transmits image selection information related to the selected one or more house images to the server 1 together with the text information.

[0281] [Example of generating user vectors] The tag generation unit 322 generates tag information corresponding to the text information transmitted from the user terminal 2. When one or more house images are selected by the user, image selection information regarding the selected one or more house images is transmitted from the user terminal 2 together with the text information. In this case, it is possible to generate a user vector taking into consideration the image selection information. Examples of this are shown in Figs. 28 and 29.

[0282] For example, the tag generating unit 322 can generate tag information corresponding to text information by using a predetermined database (for example, a dictionary database) for converting text information into tag information. For example, a dictionary database in which character strings are associated with features of each element constituting tag information (for example, features related to style and emotion) can be used. Then, the tag generating unit 322 extracts character strings included in the dictionary database from each character string included in the sentence corresponding to the text information transmitted from the user terminal 2, and extracts features of tag information corresponding to the extracted character strings from the dictionary database. Also, when multiple character strings are extracted, the tag generating unit 322 extracts features of tag information corresponding to each character string. Then, the tag generating unit 322 counts (for example, calculates an average value) the features of one or more pieces of extracted tag information to generate tag information corresponding to the text information transmitted from the user terminal 2. Then, the tag generating unit 322 outputs the generated tag information to the vector generating unit 323.

[0283] Also, it is possible to generate tag information (user vector) from text information using an AI model. For example, it is possible to use an AI model that has learned a plurality of pieces of text information to which predetermined tags that associate sentences with features of each element that constitutes tag information (for example, features related to style and emotion) are attached as teacher labels. As for this learning method, it is possible to apply the learning method shown in FIG. 12.

[0284] Also, for example, LLM can be used as the AI ​​model. For example, ChatGPT, Bard, Llama2, etc. can be used as the LLM. Note that these are only examples, and other AI models may be used. For example, based on text information input by a user, condition information to the effect that features related to a house image are output for each of a plurality of tags can be input to the LLM as instruction information (prompt), and the output information in response to this can be used as tag information. Here, the plurality of tags are tags related to style and emotion, as described above.

[0285] Here, an example of a prompt that suggests tags using ChatGPT is shown. For example, it is possible to input instruction information (prompt) to output scores (0 to 1) regarding the relevance with each of a plurality of tags (style tags, emotion tags) using sentences constituting the text information transmitted from the user terminal 2 to the LLM, and to use the output information in response to this as the feature amount for each of the plurality of tags. The style tags may be the same as those described above, and may have elements such as Western style, Japanese style, modern, American, antique, country, simple, natural, Nordic, industrial, woody, etc. The emotion tags may be the same as those described above, and may have elements such as family gathering, warmth, high-class, elegant, luxury, cool, chic, casual, pop, refined, stylish, open, calm, flashy, profound, cute, etc.

[0286] Next, the vector generation unit 323 generates a user vector related to the user who transmitted the text information, based on the tag information output from the tag generation unit 322. In this case, the tag information generated by the tag generation unit 322 can be used as the user vector. Note that, when there are multiple pieces of text information transmitted from the user terminal 2, tag information is generated for each piece of text information, so that it is possible to generate a user vector using multiple pieces of tag information, similar to the example shown in FIG.

[0287] [Example of house image selection based on text information] Next, the extraction unit 324 extracts a house image to be newly provided to the user from the image information DB 340 based on the user vector generated by the vector generation unit 323. This extraction method is similar to the method of extracting a new house image when the "Re-suggest new image" button 514 (see FIG. 16) is selected, and therefore a description thereof will be omitted here.

[0288] Furthermore, the extraction unit 324 outputs house image information (e.g., image information 342) relating to the extracted house image to the information provision unit 325. Then, the information provision unit 325 transmits the house image information to the user terminal 2, and the user terminal 2 displays the house image information.

[0289] [Example of new home image selection] Fig. 26 is a diagram showing a display example of a selection screen 910 displayed on the UI unit 460 of the user terminal 2. The selection screen 910 is displayed on the UI unit 460 under the control of the control unit 450. Note that the selection screen 910 is a modified version of a part of the message at the top of the selection screen 520 shown in Fig. 17, and the rest is common to the selection screen 520. For this reason, the parts common to the selection screen 520 are denoted by the same reference numerals.

[0290] In this way, it is possible to select a new house image and provide it to the user using the user vector generated based on the text information input by the user. Note that a text information input field 905 (see FIG. 25) may be provided in the selection screen 910 to allow further input of text information. In this case, when the user inputs text information, it is possible to re-suggest a new house image to the user based on the text information. Furthermore, by performing each of these processes multiple times, it is possible to refine the recommendations to the user.

[0291] In the above, an example has been shown in which text information is received by a user when the user does not like any of the house images presented to the user, when the user has a request for a particular house image, etc. However, this is not limiting. For example, it is possible to receive text information by a user before the house images are presented, and to select a house image to present to the user using a user vector generated based on this text information.

[0292] [Server operation example] Fig. 27 is a flow chart showing an example of a housing provider selection process by the server 1. This housing provider selection process is a partial modification of the housing provider selection process shown in Fig. 18. Specifically, steps S931 to S934 have been added. Other than this addition, the process is the same as the housing provider selection process shown in Fig. 18, so the same reference numerals are used to designate parts that are common to Fig. 18, and description thereof will be omitted.

[0293] In step S931, the information acquisition unit 321 determines whether or not text information has been received from the user terminal 2. For example, when the user performs an input process of text information on the text input screen 900 (see FIG. 25) and then selects the re-suggest button 906, the control unit 450 of the user terminal 2 transmits the text information to the server 1. If the text information has been received, the process proceeds to step S932. On the other hand, if the text information has not been received, the process returns to step S602.

[0294] In step S932, the tag generation unit 322 and the vector generation unit 323 generate tag information (user vector) based on the text information received in step S931. The method of generating this user vector is the same as the above-mentioned generation method.

[0295] In step S933, the extraction unit 324 extracts, from the image information DB 340, a house image to be newly provided to the user, based on the user vector generated in step S932.

[0296] In step S934, the information providing unit 325 transmits house image information (e.g., image information 342) related to the new house image extracted in step S933 to the user terminal 2. This house image information is information for displaying the selection screen 910 (see FIG. 26) on the UI unit 460 of the user terminal 2.

[0297] [Example of house image re-recommendation based on house image and text information] In the above, an example of presenting a house image to a user using a user vector generated based on text information has been shown. Here, after presenting a plurality of house images to a user, it is assumed that the user selects a house image that the user likes from among them, and receives requests from the user regarding those house images through text information. Therefore, in the following, an example of generating a new user vector based on the house image selected by the user and the user's request regarding the house image presented to the user will be shown.

[0298] [Example of generating user vectors based on selected house images and text information] Fig. 28 is a diagram showing a display example of a text input screen 950 displayed on the UI unit 460 of the user terminal 2. The text input screen 950 is similar to the text input screen 900 shown in Fig. 25, and differs from the text input screen 900 in that the content entered in the text information input field 905 and that house images 902 and 903 are in a selected state (favorite buttons 951 and 952 are blacked out), and the rest is the same as the text input screen 900. For this reason, the parts common to the text input screen 900 are denoted by the same reference numerals. Furthermore, the process of transmitting each piece of information to the server 1 is also the same as the example shown in Fig. 25.

[0299] As described above, it is possible that the house image display area 901 does not display a house image that the user likes, that there are few house images that the user likes, or that a house image that is slightly different from the house image that the user likes is displayed. In such cases, it is possible for the user to input text related to the preferred house in the text information input field 905 by inputting (e.g., manual operation or voice input). FIG. 28 shows an example in which the user has input "Please recommend a house with a brighter atmosphere." Note that this is just one example, and text related to the preferred house may be, for example, "Please make the original recommendation result blank and recommend an image of a house with a brighter atmosphere."

[0300] [Example of generating user vectors] The tag generation unit 322 generates tag information corresponding to each of the image selection information and text information transmitted from the user terminal 2. The generation of tag information based on image selection information is similar to the example shown in FIG. 15 and the like. The generation of tag information based on text information is similar to the example shown in FIG. 25. The generation of user vectors based on tag information generated by the tag generation unit 322 (user vectors based on image selection information, user vectors based on text information) is also similar to the generation examples described above.

[0301] [Example of determining the ratio of image selection information and text information based on user requests] Here, an example is shown in which a user vector C for selecting a new house image is generated using a user vector (user vector A) generated based on image selection information and a user vector (user vector B) generated based on text information.

[0302] For example, it is possible to generate user vector C using a preset ratio (reflection ratio of user vector A and user vector B). For example, it is possible to obtain user vector C by the following formula 1 using reflection ratio α of user vector A and user vector B. Note that the calculation method according to formula 1 is only one example, and user vector C may be obtained by other calculation methods. User vector C = (1-α) × User vector A + α × User vector B … Equation 1

[0303] Furthermore, the vector generation unit 323 generates a user vector A and a user vector B based on the tag information output from the tag generation unit 322, and generates a user vector C based on the reflection ratio α and the user vector A and the user vector B. The reflection ratio α may be set by the user or may be set by the service provider in this embodiment.

[0304] The reflection ratio α may be set based on at least one of the text information. For example, it is possible to set the reflection ratio α using an AI model. For example, it is possible to use an AI model that has learned a plurality of text information to which a predetermined tag associated with the reflection ratio and the request sentence is added as a teacher label.

[0305] Also, for example, LLM can be used as the AI ​​model. For example, ChatGPT, Bard, Llama2, etc. can be used as the LLM. Note that these are only examples, and other AI models may be used. For example, based on text information input by a user, condition information for outputting the reflection ratio α can be input to the LLM as instruction information (prompt), and the output information in response to this can be the reflection ratio α.

[0306] Here, an example of a prompt to output the reflection ratio α using ChatGPT is shown. For example, it is possible to input instruction information (prompt) to the LLM to determine the extent to which the content of the sentence constituting the text information transmitted from the user terminal 2 should be reflected, and output the ratio (0 to 1), using the sentence, and to set the output information in response to this as the reflection ratio α. In this case, instruction information (prompt) to output the estimated basis for the reflection ratio α may be input to the LLM.

[0307] For example, as an example of how to determine the reflection ratio α (example of the estimation basis of the reflection ratio α), when a user inputs "Please blank out the original recommendation result and recommend an image of a house with a bright atmosphere," it is possible to estimate the reflection ratio α by setting the appropriate reflection ratio of the content of the text to 1.0, since the user inputs "Please recommend a house with a brighter atmosphere," it is possible to estimate the reflection ratio α by setting the appropriate reflection ratio of the content of the text to about 0.3, since the user inputs "Please recommend a house with a brighter atmosphere," it is possible to estimate the reflection ratio α by setting the appropriate reflection ratio of the content of the text to about 0.3, since the user inputs "I like houses with a bright atmosphere," it is difficult to determine whether the original recommendation result should be maintained or not, so it is possible to estimate the reflection ratio α by setting the appropriate reflection ratio of the content of the text to about 0.5.

[0308] [Example of house image selection based on text information] Next, the extraction unit 324 extracts a house image to be newly provided to the user from the image information DB 340 based on the user vector C generated by the vector generation unit 323. This extraction method is similar to the method of extracting a new house image when the "re-suggest new image" button 514 (see FIG. 16) is selected, and therefore a description thereof will be omitted here.

[0309] Furthermore, the extraction unit 324 outputs house image information (e.g., image information 342) relating to the extracted house image to the information provision unit 325. Then, the information provision unit 325 transmits the house image information to the user terminal 2, and the user terminal 2 displays the house image information.

[0310] In the above, an example has been shown in which a home image to be newly provided to the user is extracted based on the user vector C, but a home builder according to the user's preferences may be extracted from the home builder DB 350 based on the user vector C. In this case, selected home builder information (e.g., profile information 352, image information 342) related to the extracted home builder is transmitted to the user terminal 2 and displayed on the user terminal 2. For example, a selected builder information screen 510 (see FIG. 16) is displayed on the UI unit 460.

[0311] Also, a text information input field 905 (see FIGS. 25 and 28) may be provided on the selected business operator information screen 510 (see FIG. 16), and a user vector B may be generated according to the user's preferences using text information inputted into the text information input field 905, and a process of extracting home images, extracting home business operators, etc. may be executed using the already generated user vector A (user vector generated based on image selection information) and user vector B. In this case, each piece of extracted information is transmitted to the user terminal 2 and displayed on the user terminal 2.

[0312] [Example of new home image selection] Fig. 29 is a diagram showing a display example of a selection screen 910 displayed on the UI unit 460 of the user terminal 2. The selection screen 960 is displayed on the UI unit 460 under the control of the control unit 450. Note that the selection screen 960 is a modified version of a part of the message at the top of the selection screen 520 shown in Fig. 17, and the rest is common to the selection screen 520. For this reason, the parts common to the selection screen 520 are denoted by the same reference numerals.

[0313] In this way, it is possible to select a new house image and provide it to the user using the user vector C generated based on the text information input by the user and the house image selected by the user. Note that a text information input field 905 (see Figures 25 and 28) may be provided in the selection screen 960 to allow further input of text information. In this case, when text information is input by the user, it is possible to provide the user with a further new house image based on that text information. Furthermore, by performing each of these processes multiple times, it is possible to refine the recommendations to the user.

[0314] [Server operation example] As an example of the operation of the server 1, it is possible to apply the housing business operator selection process shown in FIG. 27. However, in step S602, it is determined whether only image selection information has been received. Also, in step S932, a user vector is generated based on the image selection information and text information (or only the text information). That is, a user vector C for selecting a new housing image is generated using a user vector (user vector A) generated based on the image selection information and a user vector (user vector B) generated based on the text information. In this case, it is possible to generate the user vector C using the above-mentioned formula 1.

[0315] 18, 21, 23, and 27 are examples for implementing this embodiment, and the order of some of the steps may be changed within the scope of implementing this embodiment, some of the steps may be omitted, and other steps may be added. Also, in each of these steps, an example has been shown in which the control unit 450 controls the display state of the display screen displayed on the user terminal 2 based on information transmitted from the server 1, but the display state of the display screen displayed on the user terminal 2 may be controlled based on control on the server 1 side.

[0316] In this way, the user can request improvements to the proposed house image from the server 1, generate a desired vector (user vector B) based on the content of the request, and use this desired vector to update the user vector. The updated user vector is user vector C. In this way, by updating the user vector based on the content of the user's request, the user can specifically communicate his or her own desires, and can more directly influence the recommendation results of house images. As a result, it is possible to improve user satisfaction and maintain the user's interest.

[0317] [Example of providing relevant information to housing businesses] Here, when a housing builder is selected as matching the user's preferences, information to that effect is sent to that housing builder. However, some information on the user's preferences (e.g., statistical information, user tendency information) may also be provided to housing builders that were not selected. For example, information on the type of the selected housing image and the number of selected housing images may be provided to a housing builder associated with a housing image selected by a user. Also, for example, in the example shown in FIG. 20, information on the type of edited housing image, the items used in the edit, and the number of edits may be provided to a housing builder associated with an edited housing image. These may be used as statistics on the overall preferences of housing builders, or may be used for analyzing user tendencies, etc.

[0318] [Example of executing processing on other devices or systems] In the above, an example has been shown in which tag information generation processing, vector generation processing, selection processing, etc. are executed in the server 1 (or information processing system IS1), but all or part of each of these processes may be executed in other devices. In this case, the information processing system is configured by each device that executes part of each of these processes. For example, at least part of each process can be executed using information processing devices managed by other service companies, devices available to users (e.g., user terminal 2, smartphone, tablet terminal, personal computer), various information processing devices such as servers connectable via a predetermined network such as the Internet, and various electronic devices. Also, the information stored in each DB may be managed in an external storage device and retrieved from the external storage device for use.

[0319] In addition, a part (or the whole) of the information processing system capable of executing the functions of the server 1 (or the information processing system IS1) may be provided by an application that can be provided via a predetermined network such as the Internet. This application is, for example, SaaS (Software as a Service).

[0320] [Example of the configuration and effects of this embodiment] The information processing system IS1 is an information processing system including a user terminal 2 (an example of an electronic device) used by a user, and a server 1 (an example of an information processing device) capable of providing image information to the user terminal 2. The information processing system IS1 includes a housing contractor DB350 (an example of a housing contractor database) that associates and stores housing contractors with housing characteristics related to the style and emotion of housing constructed by the housing contractor for each of a plurality of housing contractors, an information acquisition unit 321 (an example of an acquisition unit) that acquires from the user terminal 2 an image selected according to the user's preferences and text information indicating a request for the image, and generates respective characteristics (first characteristics) of the style and emotion related to the user's preferences based on the text information, and based on the image, calculates what is included in the image (for example, a characteristic related to the house). The image processing apparatus includes a tag generating unit 322 that generates image features (second features) related to the style and emotion of an object, and generates weights (e.g., reflection ratio α) for the first and second features based on the text information, a vector generating unit 323 (an example of a feature generating unit), and an extraction unit 324 (an example of a selection unit) that selects a housing business operator according to the user's preferences from among a plurality of housing businesses based on a comparison result obtained by comparing feature information (e.g., user vector C) indicating the user's preferences generated based on the weights, the first feature, and the second feature with the features of the house (e.g., housing business operator vector). The first feature is tag information (or a user vector A corresponding thereto) generated based on image information, and the second feature is tag information (or a user vector B corresponding thereto) generated based on text information. A user vector C is calculated by Equation 1 based on the user vector A, the user vector B, and the reflection ratio α.

[0321] According to this configuration, it is possible to select a housing provider according to the user's preferences using a user vector C generated based on the text information input by the user and the house image selected by the user. Therefore, it is possible to propose a suitable housing provider according to the user's preferences. In other words, it is possible to support the user's home selection in an intuitive and easy-to-understand manner.

[0322] The information processing system IS1 is an information processing system including a user terminal 2 (an example of an electronic device) used by a user, and a server 1 (an example of an information processing device) capable of providing image information to the user terminal 2. The information processing system IS1 includes a housing business operator DB350 (an example of a housing business operator database) that stores housing business operators and housing characteristics related to the style and emotion of a house constructed by the housing business operator in association with each of a plurality of housing businesses, an information acquisition unit 321 (an example of an acquisition unit) that acquires an image according to the user's preferences from the user terminal 2, a tag generation unit 322 that generates image characteristics related to the style and emotion of an object included in the image (e.g., an object related to the house) based on the image, a vector generation unit 323 (an example of a feature generation unit), and an extraction unit 324 (an example of a selection unit) that selects a housing business operator according to the user's preferences from among a plurality of housing businesses based on a comparison result obtained by comparing the image characteristics (e.g., tag information, user vector) with the housing characteristics (e.g., housing business operator vector) in the housing business operator DB350. The information processing method according to this embodiment is an information processing method including each of these processes. The program according to the present embodiment is a program that causes a computer to execute each of these processes. In other words, the program according to the present embodiment is a program that causes a computer to realize each of the functions that the server 1 can execute.

[0323] According to this configuration, it is possible to obtain an image of the user's preference from the user terminal 2, and based on the result of comparing the image features related to the style and emotion of the house etc. contained in the obtained image with the house features in the housing provider DB 350, select a housing provider that matches the user's preference from among a plurality of housing providers. Therefore, it is possible to propose a suitable housing provider that matches the user's preference. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0324] Moreover, the information acquiring unit 321 (an example of an acquiring unit) acquires a plurality of images according to the user's preferences from the user terminal 2 (an example of an electronic device). The information processing system IS1 further includes a vector generating unit 323 (an example of a feature information generating unit) that generates a user vector (an example of feature information) indicating the user's preferences regarding style and emotion based on the features of each of the plurality of images. Moreover, the extracting unit 324 (an example of a selecting unit) selects a housing provider according to the user's preferences based on a comparison result obtained by comparing the user vector with the features of the house (housing provider vector) in the housing provider DB 350.

[0325] According to this configuration, it is possible to acquire a plurality of images that are preferred by the user from the user terminal 2, and select a housing provider that matches the user's preferences from among a plurality of housing providers based on the results of comparing a user vector generated based on the characteristics of each of the acquired images with the characteristics of the housing in the housing provider DB 350. This makes it possible to propose a housing provider that is more appropriate for the user's preferences. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0326] The information processing system IS1 further includes an information providing unit 325 that transmits display information for displaying housing business information related to the housing business selected by the extraction unit 324 (an example of a selection unit) to the user terminal 2 (an example of an electronic device). The user terminal 2 also includes a control unit 450 that executes display control to cause a UI unit 460 (an example of a display unit) to display the housing business information based on the display information.

[0327] According to this configuration, the user can easily visually confirm a housing provider that meets the user's preferences, selected from among a plurality of housing providers, on the basis of an image of the user's preference, on the UI unit 460 of the user terminal 2. In other words, it is possible to support the user in selecting a home in an intuitive and easy-to-understand manner.

[0328] The information processing system IS1 further includes an image information DB340 (image information database) that stores housing image information on each of the housing developers constructed by each of the multiple housing developers in association with each of the multiple housing developers, and an information providing unit 325 that transmits display information for displaying the housing image information stored in the image information DB340 to a user terminal 2 (an example of an electronic device).The information acquiring unit 321 (an example of an acquiring unit) acquires from the user terminal 2, as an image according to the user's preference, a housing image selected based on a user operation from among the housing images displayed on the user terminal 2 (an example of an electronic device) based on the housing image information.

[0329] According to this configuration, it is possible to display the house image information stored in the image information DB 340 on the UI unit 460 of the user terminal 2. Therefore, the user can easily visually check the house image information displayed on the UI unit 460 of the user terminal 2, and can easily select the house image information that the user prefers from among them. In other words, it is possible to support the user in selecting a house in an intuitive and easy-to-understand manner.

[0330] Moreover, the user terminal 2 (an example of an electronic device) includes a control unit 450 that executes display control for displaying a house image corresponding to the house image information on the UI unit 460 (an example of a display unit) based on display information transmitted from the server 1 (an example of an information processing device), and transmission control for transmitting edited image information relating to an edited house image obtained by editing the house image displayed on the UI unit 460 based on a user operation to the server 1. Moreover, the information acquisition unit 321 (an example of an acquisition unit) of the information processing system IS1 acquires an image corresponding to the edited image information transmitted from the user terminal 2 as an image according to the user's preference.

[0331] According to this configuration, by enabling editing of the house image displayed on the user terminal 2 according to the user's preferences, it is possible to more appropriately express the user's preferences based on the user's subjective editing. Therefore, it is possible to propose appropriate housing businesses according to the user's preferences based on the user's editing results. In other words, it is possible to support the user's home selection in an intuitive and easy-to-understand manner.

[0332] Moreover, the user terminal 2 (an example of an electronic device) includes an image information DB800 (an example of an image information database) that stores images acquired by the image acquisition unit 430, and a control unit 450 that executes display control for displaying images stored in the image information DB800 on a UI unit 460 (an example of a display unit), and transmission control for transmitting to the server 1 (an example of an information processing device) selected image information related to an image selected based on a user operation from among the images displayed on the UI unit 460. Moreover, the information acquisition unit 321 (an example of an acquisition unit) of the information processing system IS1 acquires an image corresponding to the selected image information transmitted from the user terminal 2 as an image according to the user's preference.

[0333] According to this configuration, by allowing the user to select an image according to the user's preference from among images acquired using the user terminal 2 (or other external device), it is possible to more appropriately express the user's preference based on the user's subjective image collection. Therefore, it is possible to propose an appropriate housing business operator according to the user's preference based on the user's collection results. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0334] In addition, the tag generation unit 322 (an example of a feature generation unit) of the server 1 (an example of an information processing device) extracts image features related to style and emotion using an AI model that extracts features related to what is contained in the target image.

[0335] According to this configuration, it is possible to extract image features related to style and emotion from any image according to the user's preferences transmitted from the user terminal 2. This makes it possible to propose appropriate housing businesses according to the user's preferences. In other words, it is possible to support the user's housing selection in an intuitive and easy-to-understand manner.

[0336] The information processing system IS1 further includes an image information DB340 (image information database) that stores housing image information on each of the housing businesses constructed by each of the housing businesses in association with each of the housing businesses, and an information providing unit 325 that transmits display information for displaying the housing image information stored in the image information DB340 to the user terminal 2 (an example of an electronic device). The information acquiring unit 321 (an example of an acquiring unit) acquires from the user terminal 2 a housing image selected based on a user operation by the user from among the housing images displayed on the UI unit 460 of the user terminal 2 based on the housing image information as an image according to the user's preferences. When a re-proposal request is transmitted from the user terminal 2 requesting a re-proposal of a new housing business other than the selected housing business, the information providing unit 325 transmits display information for displaying the housing image information extracted from the housing image information stored in the image information DB340 as a housing image according to the user's preferences to the user terminal 2 based on the image features (e.g., tag information, user vector) generated by the tag generating unit 322 and the vector generating unit 323 (an example of a feature generating unit).

[0337] According to this configuration, the user can use the user terminal 2 to make a re-proposal request to request a re-proposal of a new housing provider other than the one proposed to the user. This makes it possible to easily request a re-proposal of a housing provider that meets the user's preferences even if a housing provider that is not to the user's taste is proposed. In this case, since the housing image used to select a housing provider can be selected based on the characteristics of an image once selected by the user, it is possible to provide the user with a housing image that is close to the user's preferences. This makes it possible to propose a more appropriate housing provider that meets the user's preferences.

[0338] The server 1 (an example of an information processing device) includes an information acquisition unit 321 (an example of an image acquisition unit) that acquires an image according to a user's preference, a tag generation unit 322 that acquires features related to the image, a vector generation unit 323 (an example of a feature acquisition unit), and an extraction unit 324 (an example of a selection unit) that selects a housing company according to the user's preference from among a plurality of housing companies based on a comparison result obtained by comparing the features related to the image with features of each housing company related to houses constructed by each of the plurality of housing companies. The information processing method according to the present embodiment is an information processing method including each of the processes. The program according to the present embodiment is a program that causes a computer to execute each of the processes. In other words, the program according to the present embodiment is a program that causes a computer to realize each function that the server 1 can execute. For example, the tag generation unit 322 acquires features (tag information) related to the image acquired by the information acquisition unit 321. The vector generation unit 323 generates a user vector based on the tag information generated by the tag generation unit 322.

[0339] Each process in this embodiment is executed based on a program for causing a computer to execute various processing procedures. This embodiment can also be understood as an embodiment of a program for implementing the function of executing each process, and a recording medium for storing the program. For example, the program can be stored in each storage device of each device by an update process for adding a new function to an information processing device, electronic device, etc. This makes it possible to cause each updated device to execute each process shown in this embodiment.

[0340] Although the embodiments of the present invention have been described above, the above-mentioned embodiments merely show some of the application examples of the present invention, and it is not intended that the technical scope of the present invention be limited to the specific configurations of the above-mentioned embodiments.

[0341] For example, the following configurations (configuration examples 1 to 21) are also possible. [Configuration example 1] An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers; an acquisition unit that acquires from the electronic device an image selected according to the user's preference and text information indicating a request for the image; a generation unit that generates a first feature related to a preference of the user based on the text information, generates a second feature that is a feature of the image related to what is included in the image based on the image, and generates weights for the first feature and the second feature based on the text information; a selection unit that selects a housing business operator according to the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing feature information indicating the user's preference generated based on the weight, the first feature, and the second feature with the feature of the house; and An information processing system comprising: [Configuration example 2] The information processing system according to configuration example 1, The housing business database stores, as the characteristics of the house, characteristics related to the style and emotion of the house; The generating unit generates, as the first features, features of style and emotion related to the user's preferences, and generates, as the second features, features related to style and emotion of an object included in the image. Information processing system. [Configuration example 3] An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers; an image information database for storing images of each of the houses constructed by each of the plurality of housing developers; an acquisition unit that acquires an image according to the user's preference from the electronic device; A feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image with the characteristics of the house; an information providing unit that, when a new image re-proposal is requested by an operation of the user, causes the electronic device to display the house image extracted from the image information database based on the features of the image acquired by the feature acquiring unit; An information processing system comprising: [Configuration example 4] An information processing system including an electronic device used by a user and an information processing device capable of providing image information to the electronic device, a housing developer database that stores the characteristics of houses constructed by a housing developer and the housing developer in association with each of a plurality of housing developers; an acquisition unit that acquires an image according to the user's preference from the electronic device; A feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image with the characteristics of the house; an information providing unit that causes the electronic device to display housing business operator information related to the housing business operator selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected; An information processing system comprising: [Configuration example 5] The information processing system according to configuration example 3 or 4, The housing business database stores, as the characteristics of the house, characteristics related to the style and emotion of the house; The feature acquisition unit generates and acquires features related to style and emotion of the image as features of the image. Information processing system. [Configuration example 6] The information processing system according to configuration example 5, The acquisition unit acquires a plurality of images according to a preference of the user from the electronic device; a feature information generating unit configured to generate feature information indicating the user's preferences regarding the style and the emotion based on features of each of the plurality of images; The selection unit selects a housing business operator according to the user's preferences based on a comparison result of the characteristic information and the characteristics of the house. Information processing system. [Configuration example 7] The information processing system according to configuration example 3, the information providing unit transmits to the electronic device display information for displaying housing business operator information related to the housing business operator selected by the selection unit; The electronic device includes a control unit that executes display control to display the housing business information on a display unit based on the display information. Information processing system. [Configuration example 8] The information processing system according to configuration example 4, An image information database is further provided that stores images of houses constructed by each of the plurality of housing developers in association with each of the plurality of housing developers, the information providing unit transmits display information for displaying the house image stored in the image information database to the electronic device; The acquisition unit acquires, from the electronic device, a house image selected based on a user operation by the user from among the house images displayed on the electronic device based on the display information, as an image according to the user's preference. Information processing system. [Configuration example 9] The information processing system according to configuration example 8, The electronic device includes: display control for displaying the house image on a display unit based on the display information; a control unit that executes a transmission control to transmit edited image information regarding an edited house image in which the house image displayed on the display unit is edited based on a user operation to the information processing device; The acquisition unit acquires an image corresponding to the edited image information transmitted from the electronic device as an image according to the user's preference. Information processing system. [Configuration example 10] The information processing system according to configuration example 3 or 4, The electronic device includes: an image database for storing images acquired by the image acquisition unit; a control unit that executes a display control for displaying images stored in the image database on a display unit, and a transmission control for transmitting selected image information related to an image selected based on a user operation from among the images displayed on the display unit to the information processing device, The acquisition unit acquires an image corresponding to the selected image information transmitted from the electronic device as an image according to the user's preference. Information processing system. [Configuration example 11] The information processing system according to configuration example 5, The feature acquisition unit extracts and acquires features of the image related to the style and the emotion using an AI model that extracts features related to what is included in the target image. Information processing system. [Configuration example 12] The information processing system according to configuration example 8, When a re-proposal request for a new image is transmitted from the electronic device in response to a user operation using the operation image, the information providing unit transmits to the electronic device display information for displaying a house image extracted as an image according to the user's preference from among the house images stored in the image information database, based on the features of the image acquired by the feature acquiring unit. Information processing system. [Configuration example 13] An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, an acquisition unit that acquires an image selected according to a user's preference and text information indicating a request for the image; a generation unit that generates a first feature related to a preference of the user based on the text information, generates a second feature that is a feature of the image related to what is included in the image based on the image, and generates weights for the first feature and the second feature based on the text information; a selection unit that selects a housing business operator according to the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing feature information indicating the user's preference generated based on the weight, the first feature, and the second feature with the feature of the house; and An information processing device comprising: [Configuration Example 14] An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of houses constructed by each of the plurality of housing developers, An acquisition unit that acquires an image according to a user's preference; A feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image with the characteristics of the house; an information providing unit that displays, on a display unit, the house image extracted from the image information database based on the features of the image acquired by the feature acquiring unit, when a new image re-proposition is requested by an operation of the user; An information processing device comprising: [Configuration Example 15] An information processing device capable of handling a database that stores characteristics of houses constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, An acquisition unit that acquires an image according to a user's preference; A feature acquisition unit that acquires features of the image based on the image; a selection unit that selects a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result obtained by comparing the characteristics of the image with the characteristics of the house; an information providing unit that displays on a display unit housing business operator information related to the housing business operator selected by the selection unit and an operation image for performing a user operation to request re-proposition of a new image to be selected; An information processing device comprising: [Configuration Example 16] An information processing method to be executed by a computer capable of handling a database that associates and stores characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, comprising: a generation process of generating a first feature related to the user's preference based on the text information when an image selected according to a user's preference and text information indicating a request for the image are acquired, generating a second feature that is a feature of the image related to what is included in the image based on the image, and generating weights for the first feature and the second feature based on the text information; a selection process for selecting a housing business operator according to the user's preference from among the plurality of housing businesses based on a comparison result of comparing feature information indicating the user's preference generated based on the weight, the first feature, and the second feature with the feature of the house; An information processing method comprising: [Configuration Example 17] An information processing method to be executed by a computer capable of handling a database that stores characteristics of homes constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of homes constructed by each of the plurality of housing developers, comprising: an acquisition process for acquiring an image according to a user's preference based on the image and acquiring a feature of the image; a selection process for selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics; an information providing process for displaying, on a display unit, the house image extracted from the image information database based on the features of the image acquired in the acquisition process, when a new image proposal is requested by an operation of the user; An information processing method comprising: [Configuration Example 18] An information processing method to be executed by a computer capable of handling a database that associates and stores characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, comprising: an acquisition process for acquiring an image according to a user's preference based on the image and acquiring a feature of the image; a selection process for selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics; an information providing process in which housing business operator information related to the housing business operator selected in the selection process and an operation image for performing a user operation to request re-proposition of a new image to be selected are displayed on a display unit; An information processing method comprising: [Configuration Example 19] A program to be executed by a computer capable of handling a database that associates and stores the characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, a generation step of generating a first feature related to the user's preference based on the text information when an image selected according to a user's preference and text information indicating a request for the image are acquired, generating a second feature that is a feature of the image related to what is included in the image based on the image, and generating weights for the first feature and the second feature based on the text information; a selection step of selecting a housing business operator according to the user's preference from among the plurality of housing businesses based on a comparison result of comparing feature information indicating the user's preference generated based on the weight, the first feature, and the second feature with the feature of the house; A program that causes a computer to execute the following. [Configuration Example 20] A program to be executed by a computer capable of handling a database that stores characteristics of homes constructed by a housing developer in association with the housing developer for each of a plurality of housing developers, and an image information database that stores images of homes constructed by each of the plurality of housing developers, an acquisition step of acquiring an image according to a user's preference based on the image and acquiring a feature of the image; a selection step of selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics; an information providing step of displaying, on a display unit, the house image extracted from the image information database based on the features of the image acquired in the acquisition step, when a new image re-proposal is requested by an operation of the user; A program that causes a computer to execute the following. [Configuration Example 21] A program to be executed by a computer capable of handling a database that associates and stores the characteristics of houses constructed by a housing developer with the housing developer for each of a plurality of housing developers, an acquisition step of acquiring an image according to a user's preference based on the image and acquiring a feature of the image; a selection step of selecting a housing business operator that meets the user's preference from among the plurality of housing businesses based on a comparison result of comparing the image characteristics with the housing characteristics; an information providing step of displaying on a display unit housing business operator information related to the housing business operator selected in the selection step and an operation image for performing a user operation to request re-proposition of a new image to be selected; A program that causes a computer to execute the following. [Explanation of symbols]

[0342] 1...server, 11...control unit, 60...image information acquisition unit, 62...tag information acquisition unit, 64...vector generation unit, 66...housing business selection unit, 400...housing business DB, 2...user terminal, 200...control unit, 220...display image management unit, 222...image selection reception unit, 224...housing business information acquisition unit, 3-1 to 3-n...housing business terminal, IS1...information processing system, 310...communication unit, 320...control unit , 321...information acquisition unit, 322...tag generation unit, 323...vector generation unit, 324...extraction unit, 325...information provision unit, 330...storage unit, 340...image information DB, 350...housing business operator DB, 360...tag information DB, 410...communication unit, 420...location information acquisition unit, 430...image acquisition unit, 440...sound acquisition unit, 450...control unit, 460...UI unit, 461...reception unit, 462...output unit, 470...storage unit

Claims

1. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; An information processing device having a control unit that, when a new image re-proposal is requested by a user operation, causes a plurality of new images selected based on the selected image to be displayed on the display unit.

2. An information processing device having a control unit that displays on a display unit housing provider information related to a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request re-proposition of a new image to be selected.

3. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; An information processing device having a control unit that, when a new proposal from a new housing developer is requested by user operation, displays a new plurality of images selected based on the selected image on the display unit.

4. An information processing device having a control unit that displays on a display unit housing provider information related to a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request a re-proposal of a new housing provider.

5. Acquire an edited image edited based on a user operation from among the plurality of images; obtaining characteristics of the edited image; Displaying on a display unit housing business information related to a housing business selected from a plurality of housing businesses based on the characteristics of the edited image. An information processing device having a control unit.

6. acquiring an image selected from the plurality of images based on a user operation and text information indicating a request for the image; generating feature information related to the user's preferences based on the image and the text information; Displaying on a display unit housing business information related to a housing business selected from among a plurality of housing businesses based on the characteristic information. An information processing device having a control unit.

7. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; When a new image is requested by a user operation, a plurality of new images selected based on the selected image are displayed on the display unit; An information processing method for causing a computer to execute the above.

8. An information processing method that causes a computer to display on a display unit housing provider information regarding a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request re-proposition of a new image to be selected.

9. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; When a new proposal from a new housing business operator is requested by a user operation, a new plurality of images selected based on the selected image are displayed on the display unit. An information processing method for causing a computer to execute the above.

10. An information processing method that causes a computer to display on a display unit housing provider information regarding a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request a re-proposal of a new housing provider.

11. Acquire an edited image edited based on a user operation from among the plurality of images; obtaining characteristics of the edited image; displaying on a display unit housing business information related to a housing business selected from among a plurality of housing businesses based on the characteristics of the edited image. An information processing method executed by a computer.

12. acquiring an image selected from the plurality of images based on a user operation and text information indicating a request for the image; generating feature information related to the user's preferences based on the image and the text information; and displaying on a display unit housing business information related to a housing business selected from among a plurality of housing businesses based on the characteristic information. An information processing method executed by a computer.

13. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; When a new image is requested by a user operation, a plurality of new images selected based on the selected image are displayed on the display unit; A program that causes a computer to execute the following.

14. A program that causes a computer to display on a display unit housing provider information regarding a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request re-proposition of a new image to be selected.

15. displaying on a display unit housing business operator information related to a housing business operator selected from the plurality of housing businesses based on a selected image selected from the plurality of images based on a user operation; When a new proposal from a new housing business operator is requested by a user operation, a new plurality of images selected based on the selected image are displayed on the display unit. A program that causes a computer to execute the following.

16. A program that causes a computer to display on a display unit housing provider information regarding a housing provider selected from a plurality of housing providers based on a selected image selected from a plurality of images based on user operation, and an operation image for performing a user operation to request a re-proposal of a new housing provider.

17. Acquire an edited image edited based on a user operation from among the plurality of images; obtaining characteristics of the edited image; Displaying on a display unit housing business information related to a housing business selected from a plurality of housing businesses based on the characteristics of the edited image. A program that causes a computer to execute the following.

18. acquiring an image selected from the plurality of images based on a user operation and text information indicating a request for the image; generating feature information related to the user's preferences based on the image and the text information; Displaying on a display unit housing business information related to a housing business selected from among a plurality of housing businesses based on the characteristic information. A program that causes a computer to execute the following.

Images