Computer program, information processing method, and information processing apparatus

Abstract

To provide a program or the like capable of presenting a member matching a user's image.SOLUTION: A computer program causes a computer to execute a process of acquiring an image and a word representing an image of a member in a house, deriving a feature amount calculated from the acquired image and word, specifying a recommended member in the house based on the derived feature amount, and outputting the specified recommended member.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device. [Background technology]

[0002] Technologies for supporting interior coordination have been proposed. For example, Patent Document 1 discloses an interior coordination system that combines personal information and property information of a user to propose an indoor environment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-318358 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 has a problem in that the image of the user is not taken into consideration when identifying components for a house.

[0005] The present disclosure aims to provide a program or the like that can present components that match the user's image. [Means for solving the problem]

[0006] A computer program according to one aspect of the present disclosure causes a computer to execute a process of acquiring images and phrases representing the image of components in a house, deriving features calculated from the acquired images and phrases, identifying recommended components for the house based on the derived features, and outputting the identified recommended components.

[0007] A computer program according to one aspect of the present disclosure causes a computer to execute a process of acquiring images or phrases representing an image of components in a house, deriving features calculated from the acquired images or phrases, identifying recommended components for the house based on the derived features, and outputting the identified recommended components.

[0008] An information processing method according to one aspect of the present disclosure involves a computer executing a process to acquire images and phrases representing the image of components in a house, derive features calculated from the acquired images and phrases, identify recommended components for the house based on the derived features, and output the identified recommended components.

[0009] An information processing device according to one aspect of the present disclosure includes a control unit that executes a process of acquiring images and phrases representing an image of components in a house, deriving features calculated from the acquired images and phrases, identifying recommended components in the house based on the derived features, and outputting the identified recommended components. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to present components that match the user's image. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram illustrating an example of the configuration of an information processing device. [Figure 2] FIG. 10 is a diagram showing an example of information stored in a component DB. [Figure 3] 10 is a flowchart showing an example of a process for generating a component DB. [Figure 4] 10 is a flowchart illustrating an example of a procedure for specifying a recommended member. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of a reception screen displayed on a display unit. [Figure 6] FIG. 10 is a schematic diagram showing an example of a result screen displayed on a display unit. [Figure 7]10 is a flowchart illustrating an example of a processing procedure executed by an information processing apparatus according to a second embodiment. [Figure 8] FIG. 10 is a schematic diagram showing an example of a result screen displayed on a display unit according to the second embodiment. [Figure 9] FIG. 10 is a block diagram illustrating an example of the configuration of an information processing apparatus according to a third embodiment. [Figure 10] 11 is a flowchart illustrating an example of a processing procedure executed by an information processing apparatus according to a third embodiment. [Figure 11] FIG. 2 is a schematic diagram showing an example of three-dimensional CAD data. [Figure 12] FIG. 1 is a schematic diagram illustrating an example of a presentation board. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present disclosure will be specifically described with reference to the drawings showing embodiments thereof.

[0013] (First embodiment) FIG. 1 is a block diagram showing an example of the configuration of an information processing device 1. The information processing device 1 of this embodiment is an information processing device capable of various information processing and information transmission and reception, such as a server computer, a personal computer, or a quantum computer. The information processing device 1 is used by a proposer, such as a designer or coordinator at a housing development company, who proposes recommended components to a client. Based on images and phrases representing the image of the components in the client's home, the information processing device 1 identifies, as recommended components, components from among multiple components that match the client's image, and presents the identified recommended components. The client is an example of a user. The user may be the client's family member, a housemate, a user of the home, or the like. Note that in this embodiment, for convenience of explanation, it is assumed that the person who determines the image and the person who determines the phrases (to be identified) are the same person, but these roles may be shared by different people.

[0014] The components to be identified include various components used in houses, and may be, for example, building materials including interior and exterior materials. Specific examples of the components include wall materials, floor materials, etc.

[0015] In recent years, in negotiations for the construction, purchase, or renovation of detached houses or condominiums, the client is increasingly taking the lead in planning, rather than the designer, coordinator, or other contractor. In client-led planning, the client may propose an image of the house or components based on specific images obtained through web services, social networking services (SNS), or the like. The proposer must select components from the many components available in their company that match the images and image presented by the client. This selection process requires time and effort, placing a heavy burden on the proposer. Furthermore, component selection is left to the proposer's knowledge and experience, creating issues in terms of efficiency and reproducibility. The information processing device 1 of this embodiment effectively supports the proposer's planning work by automatically extracting recommended components that match the images and phrases received from the client, based on the images and phrases received from the client, and providing them to the proposer or the client.

[0016] 1, the information processing device 1 includes a control unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15. The information processing device 1 may be a single computer, or may be a computer system configured with multiple computers and peripheral devices. The information processing device 1 may be a virtual machine whose entity is virtualized, or may be a cloud.

[0017] The control unit 11 includes one or more arithmetic processing devices such as a central processing unit (CPU) or a graphics processing unit (GPU). The control unit 11 controls each component unit and executes processing using built-in memories such as a read-only memory (ROM) or a random access memory (RAM), a clock, a counter, etc. Each functional unit of the control unit 11 may be realized by software, or part or all of it may be realized by hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0018] The storage unit 12 includes a non-volatile memory such as a hard disk or a flash memory. The storage unit 12 may be separate from the information processing device 1 and may be one or more external storage devices externally connected. The storage unit 12 stores computer programs and data referenced by the control unit 11. The storage unit 12 of this embodiment stores a program 1P for causing a computer to execute processing related to identifying recommended components, and a component DB (Data Base) 121.

[0019] A computer program (program product) including program 1P may be provided by a non-transitory recording medium 1A on which the computer program is readably recorded. The storage unit 12 stores the computer program read from the recording medium 1A by a reading device (not shown). The recording medium 1A is, for example, a magnetic disk, an optical disk, or a semiconductor memory. The computer program may also be provided via communication. Program 1P may be a single computer program or may be composed of multiple computer programs. Program 1P may also be executed on a single computer or may be executed cooperatively by multiple computers.

[0020] The communication unit 13 includes a communication device that performs communication via a network (not shown). The control unit 11 can send and receive data via the communication unit 13 to and from an external device (not shown).

[0021] The display unit 14 includes a display device such as a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The display unit 14 displays various information including the identification results of recommended members in accordance with instructions from the control unit 11. The display unit 14 may also be an audio output unit including a speaker.

[0022] The operation unit 15 is an interface that accepts operations from the proposer. The operation unit 15 includes, for example, a keyboard, a mouse, etc. The operation unit 15 may be a touch panel built into the display unit 14. The operation unit 15 may be an audio input unit including a microphone. The operation unit 15 accepts operation input from the proposer and sends a control signal according to the operation content to the control unit 11.

[0023] The information processing device 1 may be configured to receive operations via an externally connected computer (for example, a proposer terminal device used by the proposer) and output information to be notified to the external computer. In this case, the information processing device 1 may omit the display unit 14 and the operation unit 15.

[0024] 2 is a diagram showing an example of the content of information stored in the component DB 121. The component DB 121 is a database that stores information about components that can be recommended components. The component DB 121 includes a component information table 1211 that stores component information related to the details of the component and usage history information related to the usage history of the component, and a feature amount table 1212 that stores feature amounts of the component. The component information table 1211 and the feature amount table 1212 are associated with each other by a component ID for identifying the component.

[0025] The component information table 1211 stores records that link component information, including component image, component type, product number, caption, pattern name, features, and manufacturer name of the component, with usage history information, including history ID, component use date, and client attribute information, using, for example, a component ID as a key. The component type is information indicating the classification according to the component's intended use, and includes, for example, wall material, floor material, etc. The component use date is the date the component was used or selected. The client attribute information is attribute information of the client who used the component, and includes, for example, the client's age, occupation, household composition, number of years in residence, place of residence, etc. The usage history information may include information related to multiple uses.

[0026] The feature table 1212 stores feature components for each of a plurality of feature component items, for example, using a component ID as a key. In this embodiment, a feature is a feature vector, which is a multi-dimensional vector made up of a plurality of feature components (vector components) shown in the feature table 1212. In the example shown in Fig. 2, the feature table 1212 includes a plurality of vector component columns corresponding to a plurality of predefined vector component items, and each vector component column stores a vector component value normalized to a range of 0 to 1.

[0027] The vector components in the feature vector include vector components relating to the color of the image of the component and vector components relating to words indicating the characteristics of the component. Examples of vector components relating to the color of the image of the component include R (red) value, G (green) value, and B (blue) value. Examples of vector components relating to words indicating the characteristics of the component include glossiness, texture, transparency, and warm / cold feeling. Note that the vector components are not limited to values ​​between 0 and 1, and may be any appropriate numerical value depending on the content of the vector component.

[0028] The vector components in the feature vector of this embodiment further include vector components related to the client's attribute information and vector components related to component trends. Examples of vector components related to the client's attribute information include the client's age, occupation, household composition, years of occupancy, and place of residence. Examples of vector components related to trends include the total number of times a component has been used and the number of times a component has been used in a recent fixed period. The contents stored in component DB 121 are not limited to the example shown in FIG. 2. The data storage method shown in FIG. 2 is merely an example, and other storage formats are possible as long as the relationships between the data are maintained. For example, component information and usage history information may be stored in separate tables, and these tables may be associated by component IDs.

[0029] The information processing device 1 acquires component information and feature quantities related to a large number of components that can be recommended components, and stores the acquired information in the component DB 121. Components that can be recommended components are components that can be offered to clients as products, and may be, for example, components from a manufacturer that the proposer handles.

[0030] 3 is a flowchart showing an example of a process for generating the component DB 121. The processes in the following flowcharts are executed by the control unit 11 in accordance with a program 1P stored in the storage unit 12 of the information processing device 1.

[0031] The control unit 11 of the information processing device 1 acquires component images and phrases related to the component for which a feature vector is to be generated (step S11). For example, the control unit 11 acquires component images by extracting image data from a product catalog, and also acquires phrases related to the component by extracting the component's type, product number, caption, pattern name, features, manufacturer name, etc. using text mining or the like. The control unit 11 further acquires phrases representing the client's attribute information and trends as phrases related to the component. The client's attribute information and trends may be acquired by analyzing the past construction history in which the target component was used. The control unit 11 may acquire component images and phrases by accepting input of information based on the proposer's operation, or by receiving component images and phrases transmitted from an external device via communication.

[0032] The control unit 11 performs feature analysis based on the acquired component image as image data to derive a feature vector of the component image (step S12). The control unit 11 performs feature analysis based on the acquired phrase as text data to derive a feature vector of the phrase (step S13). The processing of steps S12 and S13 may be reversed in order or may be executed in parallel.

[0033] The method for deriving the feature vectors of component images and phrases is not particularly limited. For example, the control unit 11 may vectorize the component images using a learning model trained to output a feature vector for a component image when the component image is input. The learning model may be, for example, a convolutional neural network (CNN), a type of neural network. The learning model may be generated by preparing training data in which labels indicating the values ​​of each vector component constituting the feature vector are associated with the component image and then using the training data to train an untrained neural network. The vector components corresponding to the component images may be calculated using, for example, the method described in Japanese Patent Publication No. 3160020. In this case, the control unit 11 may calculate the values ​​of vector components for each predefined vector component item from the physical quantities of the component using a predetermined model function, and generate a multidimensional feature vector consisting of the calculated values ​​of each vector component.

[0034] The control unit 11 also vectorizes the component-related words using a known vectorization method such as Word2Vec. Both the component image-based feature vector and the word-based feature vector are vectors of the same dimension, each consisting of vector components corresponding to all vector component items defined in the feature amount table 1212.

[0035] The control unit 11 derives a feature vector of the component based on the obtained feature vector based on the component image and the feature vector based on the phrase (step S14). For example, the control unit 11 calculates the average value of the vector components in each of the component image feature vector and the phrase feature vector for each vector component item, and uses the obtained average value as the vector component to generate one feature vector related to the component.

[0036] The method for generating the feature vector of a component is not limited to the above example. The control unit 11 may, for example, use a multimodal learning model (e.g., GPT (Generative Pre-trained Transformer)-4) that can input multiple types of data collectively to process the component image and phrases in an integrated manner, thereby deriving one feature vector based on the component image and phrases at once. Note that the control unit 11 may separately obtain a feature vector based on the component image and a feature vector based on the phrase as the feature vector of the component, without integrating the feature vector of the component image and the feature vector of the phrase.

[0037] The control unit 11 associates the component ID of the component for which a feature vector is to be generated, the acquired component image and phrase, and the derived feature vector, and stores them in the component DB 121 (step S15). In detail, the control unit 11 associates the component ID with the component image and phrase and stores them in the component information table 1211, and also associates the component ID with each vector component of the feature vector and stores them in the feature amount table 1212. The control unit 11 then ends the process.

[0038] By performing the above-described vectorization process for each of a plurality of components that can be recommended components, a feature vector corresponding to each of the plurality of components is generated and stored in the component DB 121. The component DB 121 may be updated as needed. For example, the information processing device 1 may acquire new attribute information and trends at appropriate intervals and update the feature vectors of the components by deriving feature vectors based on the acquired new attribute information and trends. The information processing device 1 uses the component DB 121 constructed as described above to identify recommended components that match the information on the components imagined by the client.

[0039] 4 is a flowchart showing an example of a procedure for specifying a recommended component. The information processing device 1 starts the following process in response to, for example, receiving a request to specify a recommended component.

[0040] The control unit 11 of the information processing device 1 displays a reception screen for receiving information about components on the display unit 14 (step S21). Using the displayed reception screen, the control unit 11 acquires an image representing the client's component image, request information, and client attribute information based on the operation of the operation unit 15 by the proposer (step S22).

[0041] 5 is a schematic diagram showing an example of a reception screen 21 displayed on the display unit 14. The reception screen 21 includes, for example, an image reception unit 211 that receives an image, a request reception unit 212 that receives request information, and an attribute reception unit 213 that receives attribute information. The image is image data, and the request information and attribute information are text data.

[0042] The image receiving unit 211 is configured to be able to receive an image. For example, the proposer registers an image specified by the client in the image receiving unit 211. The image may be provided to the information processing device 1 through a client terminal device used by the client.

[0043] An image image is an image that represents the client's image of the components, and is, for example, an image showing one or more components that the client desires or that are close to the client's desires. The image image is, for example, an image that is made public through a web service, SNS, or the like. The image image may be an image that includes multiple types of components in a house (for example, a spatial image that shows the entire room), or an image that includes a specific type of component (for example, an image of only wall materials, an image of only floor materials, etc.). In FIG. 5, an image that includes a desired floor material and a desired wall material is shown as an example of the image image. The image receiving unit 211 may be configured to be able to receive multiple image images.

[0044] The request receiving unit 212 has an input field for request information. The request information includes words and phrases that express the client's image of the component. The request information may also include additional information for the image image. The proposer can use the operation unit 15 to input words, sentences, numbers, etc. that are in line with the image of the component obtained from the client. In the example of FIG. 5, a sentence such as "I would like the overall appearance to be a little more subdued" is accepted as request information. The request receiving unit 212 may be configured to be able to accept request information by type of component.

[0045] 5, words and phrases related to component trends are received through the request receiving unit 212. The proposer can receive the client's request regarding component trends and input them into the request receiving unit 212. Note that component trends are not limited to being included as part of the request information, and may be received as separate information.

[0046] The attribute receiving unit 213 has an input field for attribute information. The proposer can input attribute information including the age, occupation, household composition, place of residence, and number of years of occupancy of the client using the operation unit 15.

[0047] As shown in Fig. 5, the reception screen 21 may further include an importance reception unit 214 that receives the importance of each element in component selection. The importance reception unit 214 is configured to be able to receive, for example, multiple levels of importance for each of multiple elements whose importance is to be identified. The elements whose importance is to be identified may be appropriately selected from vector component items in the feature vector. The importance is used as a weight index in the calculation of similarity, which will be described later.

[0048] When the proposer presses the extract button 215 with the image, request information, attribute information, and importance level input into the image receiving unit 211, request receiving unit 212, attribute receiving unit 213, and importance level receiving unit 214, respectively, the image, request information, attribute information, and importance level are received by the control unit 11. Note that the control unit 11 may be configured to receive only some of the image, request information, attribute information, and importance level.

[0049] Returning to FIG. 4, the description will be continued. The control unit 11 acquires a component area in the acquired image (step S23). In step S23, the control unit 11 may acquire the type of component in association with the component area. A component area is an area in the image that includes the component to be analyzed. The method for acquiring a component area in the image is not particularly limited. For example, the control unit 11 acquires the component area by accepting a manual drawing of the component area. The control unit 11 may acquire the component area in the image using a predetermined object detection model constructed by machine learning. If multiple components are shown in the image, the control unit 11 may extract multiple component areas corresponding to each component from the image. When the component areas are acquired, the control unit 11 may display the acquired component areas in a recognizable manner. For example, as shown in FIG. 5, the control unit 11 visibly displays the acquired component areas on the image displayed on the image accepting unit 211.

[0050] In step S23, the control unit 11 may extract and acquire a component region using an edge detection technique. The control unit 11 may also acquire a component region by manually accepting designation of an outline forming a component region for an outline in an image extracted by a technique such as edge detection. The control unit 11 may acquire a component region using large-scale language models (LLMs). For example, the control unit 11 may create a prompt to request acquisition of a component region, and provide the created prompt and the image to the large-scale language model, thereby acquiring a component region via the large-scale language model.

[0051] In the image shown in FIG. 5, a first component region corresponding to a wall material and a second component region corresponding to a floor material are acquired. The control unit 11 displays an image on the image receiving unit 211 after image processing such as framing and masking has been performed on the image regions corresponding to the acquired component regions. In this case, it is preferable to vary the display mode of the image region corresponding to each component region depending on the type of component region, such as by changing the display color and line type of the component region depending on the type of component region. In FIG. 5, the first component region is hatched downward to the right, and the second component region is hatched upward to the right. For example, in the image, an image region of a vertically extending wall surface that is not covered by fixtures or the like and where the wall material is exposed is identified as the first component region corresponding to the wall material. In addition, in the image, an image region of a horizontally extending floor surface that is not covered by fixtures or the like and where the floor material is exposed is identified as the second component region corresponding to the floor material. In addition, if the image contains multiple component areas corresponding to the same category (for example, if it contains a component area corresponding to a first wall material and a component area corresponding to a second wall material), the component areas may be further subdivided.

[0052] The control unit 11 may be capable of accepting corrections to the acquired component area. The proposer can input a request to correct the component area, for example, by specifying the area change button 216 shown in FIG. 5. When the control unit 11 accepts a request to correct the component area, it may display screen information for accepting drawing of the component area and accept manual drawing of the component area. Alternatively, the control unit 11 may execute automatic extraction of the component area.

[0053] Returning to FIG. 4, the control unit 11 derives a feature vector of the component region in the image based on the component region in the image (step S24). The control unit 11 derives a feature vector of the phrase based on the phrase as text data including the acquired request information and attribute information (step S25). The control unit 11 derives a feature vector of the component based on the feature vector based on the acquired image and the feature vector based on the phrase (step S26). The vectorization processes in steps S24 to S26 are similar to steps S12 to S14, respectively. In step S25, the control unit 11 may derive a feature vector using only information corresponding to the type of component in the component region, from among the information included in the request information.

[0054] Based on the information stored in the feature table 1212 of the component DB 121, the control unit 11 calculates the similarity between the feature vector acquired from the client's image and the feature vector for each recommended component in the feature table 1212 (step S27). In step S27, the control unit 11 preferably calculates the similarity by comparing each vector component of the acquired feature vector with each vector component of the feature vector in the feature table 1212 for each vector component item. For example, the control unit 11 determines the similarity between each of the recommended components and the client's image by performing a data distance calculation between the feature vector of each of the recommended components and the acquired feature vector. The data distance calculation may use, for example, Euclidean distance, cosine similarity, or the like. The control unit 11 may also use other methods to determine the similarity between the acquired feature vector and each feature vector in the feature table 1212.

[0055] In calculating the similarity, the control unit 11 may assign weights according to each vector component of the feature vector. For example, the control unit 11 increases or decreases the weight of each vector component corresponding to each element depending on the level of importance of each element received through the importance receiving unit 214, multiplies the weight increased or decreased depending on the importance by each vector component corresponding to the importance, and calculates the similarity by using the multiplication result in the above-mentioned data distance calculation (for example, Euclidean distance, cosine similarity, etc.).

[0056] Based on the calculated similarity with each component, the control unit 11 identifies the component whose similarity satisfies a preset extraction condition as a recommended component (step S28). In other words, the control unit 11 extracts the component whose similarity satisfies the preset extraction condition from the component DB 121 as a recommended component. The extraction condition may be that the similarity is equal to or greater than a preset threshold, or that the calculated similarity is ranked within a preset rank from the top.

[0057] The control unit 11 executes the processes of steps S24 to S28 for each component region in the image, thereby identifying recommended components for each type of component included in the image. Note that if the image contains multiple types of components classified into the same category (for example, if the image contains two types of wall materials, two types of floor materials, etc.), the control unit 11 may identify recommended components for each component.

[0058] The control unit 11 generates a result screen showing the identified recommended components (step S29), and displays the generated result screen on the display unit 14 (step S30). The identified one or more recommended components are displayed on the result screen.

[0059] The control unit 11 receives the selection of one of the recommended components displayed on the result screen, thereby acquiring the recommended component finally selected by the client (step S31). The control unit 11 stores the usage history information, including the reception date and the client's attribute information acquired in step S22, in association with the component ID of the acquired recommended component in the component information table 1211 of the component DB 121 (step S32). The control unit 11 ends the series of processes.

[0060] In the above-described processing, the control unit 11 may update the feature amount table 1212 by re-deriving the feature vectors related to the components selected by the client based on the newly collected usage history information.

[0061] In the above process, the control unit 11 may determine the similarity based on the feature vector based on the image and the feature vector based on the phrase, respectively, without integrating the feature vector based on the image and the feature vector based on the phrase. When determining the similarity based on each feature vector, the control unit 11 may identify, as a recommended component, a component for which all of the multiple similarities based on each feature vector satisfy a preset extraction condition.

[0062] In the above, an example has been described in which a feature vector based on request information including trends and attribute information of a client is generated as a phrase-based feature vector, but the text data is not limited to being processed together. For example, the control unit 11 may separately vectorize request information other than trends, attribute information, and trends.

[0063] The vector components of the feature vector may be configured not to include at least one of the client's attribute information and trends. In this case, the phrase-based feature vector is generated from phrases other than at least one of the attribute information and trends.

[0064] The method for identifying recommended components is not limited to the method using similarity, and a method other than calculating similarity, such as a machine learning method, may be used.

[0065] In this embodiment, the identification results are configured to be output (displayed) through the display unit 14, but the identification results may be output to an external device other than the information processing device 1. The identification results may be output to a printing device or the like and presented to the proposer or the client on paper media.

[0066] Fig. 6 is a schematic diagram showing an example of a result screen 22 displayed on the display unit 14. Fig. 6 shows an example of a screen showing the flooring material identification results (extraction results). The result screen 22 includes a component information display field 221 that displays component information of, for example, recommended components, and a component selection section 222 for selecting the type of component to be displayed.

[0067] The component information display field 221 displays a list of component information for the identified recommended components. When the information processing device 1 identifies recommended components, it references the component information table 1211, reads out component information for each identified recommended component, and displays the read-out component information in a list in the component information display field 221. The information processing device 1 displays the recommended components in descending order of the calculated similarity in the component information display field 221. The component information display field 221 shown in Fig. 6 displays a component image, product number, pattern name, and features for each identified recommended component.

[0068] Each piece of component information displayed in the component information display field 221 is configured to be selectable. The component information display field 221 also functions as a component selection field that accepts the selection of component information relating to any of the components displayed in the component information display field 221. The component information display field 221 includes, for example, a plurality of check boxes 223 that accept selection inputs associated with each identified recommended component. The proposer can use the operation unit 15 to select (specify) the check box 223 corresponding to any of the components and press the decision button 224, thereby finally inputting the recommended component selected by the client.

[0069] The proposer can switch the display content of the result screen 22 by switching the selection of the component type using the operation unit 15. When the selection of the component type is accepted through the component selection unit 222, the information processing device 1 displays the identification results of the component corresponding to the selected component type in the component information display field 221. Note that the result screen 22 may be configured to display the identification results of each of multiple types of components together.

[0070] According to this embodiment, components that match the client's image can be presented based on images and words that express the client's image. By using the feature quantities of the images and words, recommended components can be identified efficiently and accurately. Since recommended components can be identified automatically, the workload of the proposer in identifying components can be reduced and efficiency and reproducibility can be improved. By calculating feature quantities using both images and words, the feature quantities of the components that the client imagines can be determined more accurately. By storing feature quantities of candidate components in a database in advance, recommended components can be easily identified based on comparison with these stored feature quantities. By treating the various characteristics of components as vector components, the characteristics of the components can be appropriately reflected in the feature vector.

[0071] (Second embodiment) In the second embodiment, a configuration for re-identifying recommended components will be described. In the following embodiment, differences from the first embodiment will be mainly described, and components common to the first embodiment will be assigned the same reference numerals and detailed descriptions thereof will be omitted.

[0072] 7 is a flowchart showing an example of a processing procedure executed by the information processing device 1 of the second embodiment. The control unit 11 of the information processing device 1 executes the same processes as steps S21 to S30 of the first embodiment, and displays a result screen showing the identified recommended components on the display unit 14.

[0073] The control unit 11 determines whether or not there is additional desired information (step S41). For example, if it is determined that there is no additional desired information because no additional desired information has been entered on the result screen (S41: YES), the control unit 11 executes the same processes as steps S31 to S32.

[0074] For example, when it is determined that there is additional desired information because additional desired information has been input on the result screen (S41: NO), the control unit 11 acquires the additional desired information (step S42).

[0075] Fig. 8 is a schematic diagram showing an example of a result screen 22 displayed on the display unit 14 in the second embodiment. The result screen 22 in the second embodiment includes a component information display field 221 and a component selection section 222 similar to those in Fig. 6. The component information display field 221 displays the identification results of recommended components identified based on the initially received image and phrase.

[0076] The result screen 22 in the second embodiment further includes an additional request receiving section 225 for receiving additional request information, and a re-extraction button 226 for specifying re-extraction (re-identification) of recommended components.

[0077] The additional request receiving unit 225 has an input field for additional request information. The additional request information includes words and phrases that express the client's further image and requests for the identified recommended components, improvements to the identified results, etc. The proposer can input words, sentences, numbers, etc. that align with the additional request using the operation unit 15. In the example of Figure 8, a sentence such as "I would like it to feel a little more subdued. A little brighter" is accepted as request information.

[0078] When the proposer designates the re-extract button 226 with the additional requested information input to the additional request receiving unit 225, the control unit 11 receives the additional requested information.

[0079] Returning to FIG. 7, after acquiring the additional desired information, the control unit 11 returns the process to step S25 and repeatedly executes the processes from deriving the feature vector to identifying and displaying the recommended components. Specifically, the control unit 11 derives a feature vector of a phrase based on a phrase as text data including the acquired additional desired information. The control unit 11 executes the same process as step S26 to derive a feature vector related to the component based on the feature vector based on the image and the feature vector based on the newly derived phrase. The control unit 11 executes the processes from step S27 onward to re-identify the recommended component taking the additional desired information into account. The control unit 11 repeatedly executes the above process until no additional desired information is acquired.

[0080] According to this embodiment, recommended components are searched again in response to additional request information, so the accuracy of identifying recommended components is improved, and components that are more suited to the client's image can be presented.

[0081] (Third embodiment) In the third embodiment, the method of presenting the identification results of recommended components differs from that in the first embodiment. In the following embodiments, differences from the first embodiment will be mainly described, and the same reference numerals will be used to designate components common to the first embodiment, and detailed descriptions thereof will be omitted.

[0082] 9 is a block diagram showing an example of the configuration of an information processing device 1 according to a third embodiment. The information processing device 1 according to the third embodiment is connected to a CAD system 31 and a presentation board generation system 32 via a wired or wireless connection. The CAD system 31 is a system for generating three-dimensional CAD data according to recommended components. The presentation board generation system 32 is a system for generating a presentation board (hereinafter also referred to as a presentation board) in which the recommended components are arranged in two dimensions. At least one of the CAD system 31 and the presentation board generation system 32 may be integrated into the information processing device 1.

[0083] 10 is a flowchart showing an example of a processing procedure executed by the information processing device 1 of the third embodiment. The control unit 11 of the information processing device 1 executes the same processes as steps S21 to S28 of the first embodiment to identify recommended members.

[0084] The control unit 11 outputs information about the identified recommended components to the CAD system 31 and the presentation board generation system 32 (step S51). Based on the information about the recommended components received from the information processing device 1, the CAD system 31 and the presentation board generation system 32 generate three-dimensional CAD data in which the recommended components are arranged in a three-dimensional space and a presentation board in which the recommended components are arranged in a two-dimensional space, respectively, using a known method.

[0085] The control unit 11 acquires three-dimensional CAD data and a presentation board based on the identified recommended components through the CAD system 31 and the presentation board generation system 32 (step S52). The control unit 11 displays the acquired three-dimensional CAD data and presentation board on the display unit 14 (step S53). Thereafter, the control unit 11 executes the same processes as in steps S31 to S32.

[0086] In the above-described process, the control unit 11 may output information about the recommended components to either the CAD system 31 or the presentation board generation system 32, and acquire and display either the three-dimensional CAD data or the presentation board.

[0087] 11 is a schematic diagram showing an example of three-dimensional CAD data. In the three-dimensional CAD data, the identified recommended components are arranged in appropriate positions according to the components. The three-dimensional CAD data may further display the product numbers, etc., of the identified recommended components.

[0088] 12 is a schematic diagram showing an example of a presentation board, on which information about the identified recommended components is laid out according to a template prepared in advance.

[0089] According to this embodiment, the results of identifying recommended components can be presented in a variety of ways, thereby further improving convenience for the user.

[0090] The information processing device 1 may acquire only either an image representing the client's image of the components or request information, which is a phrase, and identify recommended components based on feature quantities calculated from either the acquired image or request information. When acquiring only request information, the acquired request information may include, for example, a sentence describing the overall characteristics of the room that match the construction image, and a sentence describing the characteristics of the components that match the construction image. For example, the information processing device 1 may acquire, as request information, phrases such as "The room is in the style of an XX hotel. The color tone is based on great natural wood and is urban. The walls are finished in concrete. The floor is dark gray flooring."

[0091] The information processing device 1 can derive a feature vector for identifying recommended components based on a feature vector based on the acquired image or a feature vector based on a phrase. When only request information is acquired, instead of acquiring a component area in the image, a term indicating a component type in the request information may be acquired, and a feature vector of a sentence including the acquired term indicating the component type may be derived as a feature vector corresponding to the component of the component type. With the above configuration, recommended components can be identified from either an image or a phrase, simplifying processing and improving convenience.

[0092] The following additional notes are provided regarding the above-described embodiments. (Appendix 1) Acquire images and phrases that represent the image of components in a house; Deriving feature amounts calculated from the acquired images and phrases; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components A computer program that causes a computer to perform a process. (Appendix 2) Acquire an image or phrase that represents an image of a component in a house; Deriving feature amounts calculated from the acquired image or phrase; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components A computer program that causes a computer to perform a process. (Appendix 3) storing the feature amount for each recommended component; Identifying the recommended component whose stored feature quantity is similar to the derived feature quantity 10. The computer program of claim 1 or 2. (Appendix 4) Calculating the similarity between the feature amount of each recommended component and the derived feature amount; The recommended member is identified based on the calculated similarity that satisfies a predetermined condition. 4. The computer program of claim 3. (Appendix 5) The feature amount includes a plurality of feature amount components relating to a color of the image and a plurality of words indicating characteristics of the component. 5. A computer program according to any one of claims 1 to 4. (Appendix 6) The feature amount includes a feature amount component calculated from attribute information of the user. 6. A computer program according to any one of claims 1 to 5. (Appendix 7) The feature amount is calculated for each type of the member. 7. A computer program according to any one of claims 1 to 6. (Appendix 8) Identifying a plurality of the recommended components based on the feature amounts; The plurality of recommended components identified are displayed in a selectable manner. 8. A computer program according to any one of claims 1 to 7. (Appendix 9) Identifying a plurality of the recommended components based on the feature amounts; acquiring a recommended component selected by the user from the identified plurality of recommended components; The acquired recommended components are stored in association with the attribute information of the user. 9. A computer program according to any one of claims 1 to 8. (Appendix 10) Acquire CAD data or a presentation board corresponding to the identified recommended component. 10. A computer program according to any one of claims 1 to 9. (Appendix 11) Acquire a phrase for the identified recommended component; Deriving a feature amount calculated from the acquired phrase for the recommended component; Based on the derived feature amount, recommended components for the house are identified again. 11. A computer program according to any one of claims 1 to 10.

[0093] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and the scope equivalent to the claims. The sequences shown in each embodiment are not limited, and the order of each process may be changed within a range consistent with the present invention, and multiple processes may be executed in parallel. The entity that performs each process is not limited, and the process of each device may be executed by another device within a range consistent with the present invention.

[0094] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any combination, regardless of the reference format. Furthermore, although the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limited to this format. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used. [Explanation of symbols]

[0095] 1. Information processing equipment 11 Control section 12 Storage section 13 Communications Department 14 Display section 15 Control section 121 Component DB 1P Program 1A Recording Media

Claims

1. Acquire images and phrases that represent the image of components in a house; Deriving feature amounts calculated from the acquired images and phrases; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components A computer program that causes a computer to perform a process.

2. storing the feature amount for each recommended component; Identifying the recommended component whose stored feature quantity is similar to the derived feature quantity 2. The computer program of claim 1.

3. Calculating the similarity between the feature amount of each recommended component and the derived feature amount; The recommended member is identified based on the calculated similarity that satisfies a predetermined condition.

3. A computer program according to claim 2.

4. The feature amount includes a plurality of feature amount components relating to a color of the image and a plurality of words indicating characteristics of the component.

3. A computer program according to claim 1 or claim 2.

5. The feature amount includes a feature amount component calculated from attribute information of the user.

3. A computer program according to claim 1 or claim 2.

6. The feature amount is calculated for each type of the member.

3. A computer program according to claim 1 or claim 2.

7. Identifying a plurality of the recommended components based on the feature amounts; The plurality of recommended components identified are displayed in a selectable manner.

3. A computer program according to claim 1 or claim 2.

8. Identifying a plurality of the recommended components based on the feature amounts; acquiring a recommended component selected by the user from the identified plurality of recommended components; The acquired recommended components are stored in association with the attribute information of the user.

3. A computer program according to claim 1 or claim 2.

9. Acquire CAD data or a presentation board corresponding to the identified recommended component.

3. A computer program according to claim 1 or claim 2.

10. Acquire a phrase for the identified recommended component; Deriving a feature amount calculated from the acquired phrase for the recommended component; Based on the derived feature amount, recommended components for the house are identified again.

3. A computer program according to claim 1 or claim 2.

11. Acquire an image or phrase that represents an image of a component in a house; Deriving feature amounts calculated from the acquired image or phrase; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components A computer program that causes a computer to perform a process.

12. Acquire images and phrases that represent the image of components in a house; Deriving feature amounts calculated from the acquired images and phrases; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components An information processing method in which processing is performed by a computer.

13. Acquire images and phrases that represent the image of components in a house; Deriving feature amounts calculated from the acquired images and phrases; Identifying recommended components for the house based on the derived feature amount; Output the identified recommended components Equipped with a control unit that executes processing Information processing device.

Images