Furniture recommendation method and related device

Abstract

The invention discloses a furniture recommendation method and a related device, and relates to the technical field of artificial intelligence. Comprising the steps that search elements are input into a furniture recommendation model, and search feature vectors are obtained; obtaining a plurality of feature vectors from a feature vector library as a plurality of target feature vectors, wherein the similarity between the feature vectors and the search feature vectors is greater than or equal to a preset similarity threshold value; and taking the furniture corresponding to the plurality of target feature vectors as the furniture recommended to the user. The search feature vector generated on the basis of the furniture recommendation model is a feature vector capable of reflecting the feature of the target furniture and the feature vector of the relative position relation of the target furniture and other furniture placed in the use environment; therefore, the furniture product corresponding to the target feature vector generated based on the search feature vector is a furniture product which can better meet the use requirements of the user.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a furniture recommendation method and related devices. Background Art

[0002] In the home improvement area of ​​the bank's mobile points mall, there are a large number of furniture of different sizes, styles and materials for users to choose from. Traditional solutions rely on users' search texts, historical browsing records and purchase behaviors to recommend furniture products to users. Since it is difficult for ordinary users to accurately describe furniture features, such as furniture style, furniture name and furniture function, and historical browsing records and purchase behaviors can only reflect users' historical behavioral preferences; therefore, it is difficult for traditional solutions to accurately recommend furniture products that users really need. Therefore, how to accurately recommend furniture products that can better meet users' usage needs has become one of the technical problems that need to be solved urgently. Summary of the invention

[0003] Based on the above problems, the present application provides a furniture recommendation method to accurately recommend furniture products that can better meet the user's usage needs.

[0004] The embodiments of the present application disclose the following technical solutions:

[0005] The first aspect of the present application provides a furniture recommendation method, comprising:

[0006] Determine search elements; the search elements include search images and search texts;

[0007] Inputting the search element into a furniture recommendation model to obtain a search feature vector; the furniture recommendation model is a model pre-trained based on sample images and used to generate feature vectors; the sample images are images indicating features of target furniture and the relative positional relationship between the target furniture and other furniture placed in a usage environment;

[0008] Acquire multiple feature vectors whose similarity with the search feature vector is greater than or equal to a preset similarity threshold from the feature vector library as multiple target feature vectors;

[0009] The multiple sets of furniture corresponding to the multiple target feature vectors are used as furniture products recommended to the user.

[0010] In an optional implementation, the step of training the furniture recommendation model includes:

[0011] Acquire multiple sets of training data and corresponding multiple sets of label data; the training data includes the sample image and the sample text corresponding to the sample image; the label data is the serial numbers of multiple furniture products obtained after manually annotating the training data;

[0012] Obtain a model to be trained; the model to be trained includes M layers, each layer includes an image encoder and a text encoder; M is an integer greater than or equal to 2;

[0013] Based on the multiple groups of training data and the multiple groups of label data, the model to be trained is trained to obtain the furniture recommendation model.

[0014] In an optional implementation, the training of the model to be trained based on the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model includes:

[0015] An image encoder and a text encoder in the same layer of the model to be trained are used as sub-models to be trained;

[0016] For each of the sub-models to be trained, training the sub-model to be trained based on the multiple sets of training data to obtain a trained sub-model;

[0017] Substituting each of the trained sub-models into the model to be trained to obtain a target model to be trained;

[0018] The target model to be trained is trained using the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model.

[0019] In an optional implementation, the furniture recommendation model includes a first image encoder, a second image encoder, a first text encoder, and a second text encoder; inputting the search element into the furniture recommendation model to obtain a search feature vector includes:

[0020] generating a first image feature vector based on the first image encoder and the search image;

[0021] generating a second image feature vector based on the second image encoder and the first image feature vector;

[0022] Generate a first text feature vector based on the first text encoder and the search text;

[0023] Generate a second text feature vector based on the second text encoder and the first text feature vector;

[0024] The first image feature vector, the second image feature vector, the first text feature vector, and the second text feature vector are subjected to feature fusion to generate the search feature vector.

[0025] In an optional implementation, the method for acquiring the feature vector library includes:

[0026] Based on the multiple groups of training data and the furniture recommendation model, generating a feature vector corresponding to each group of training data;

[0027] The feature vector library is generated based on the feature vector corresponding to each group of the training data.

[0028] In an optional implementation, the method further includes:

[0029] The furniture corresponding to each target feature vector is displayed in descending order of similarity.

[0030] A second aspect of the present application provides a furniture recommendation device, comprising:

[0031] A search element determination module, used to determine the search element; the search element includes a search image and a search text;

[0032] A search feature vector determination module is used to input the search element into a furniture recommendation model to obtain a search feature vector; the furniture recommendation model is a model pre-trained based on sample images and used to generate feature vectors; the sample images are images indicating features of target furniture and relative positional relationships between the target furniture and other furniture placed in a usage environment;

[0033] A target feature vector determination module is used to obtain, from a feature vector library, a plurality of feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold, as a plurality of target feature vectors;

[0034] The furniture determination module is used to use the multiple sets of furniture corresponding to the multiple target feature vectors as furniture products recommended to the user.

[0035] In an optional implementation, the device further includes:

[0036] A training data acquisition module is used to acquire multiple sets of training data and corresponding multiple sets of label data; the training data includes the sample image and the sample text corresponding to the sample image; the label data is the serial numbers of multiple furniture products obtained after manually annotating the training data;

[0037] A model acquisition module to be trained, used to acquire a model to be trained; the model to be trained includes M layers, each layer includes an image encoder and a text encoder; M is an integer greater than or equal to 2;

[0038] The furniture recommendation model acquisition module is used to train the model to be trained based on the multiple groups of training data and the multiple groups of label data to obtain the furniture recommendation model.

[0039] A third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method introduced in any implementation manner of the first aspect.

[0040] A fourth aspect of the present application provides an electronic device, including:

[0041] a memory having a computer program stored thereon;

[0042] A processor is used to execute the computer program in the memory to implement the steps of the method introduced in any implementation manner of the first aspect.

[0043] Compared with the prior art, this application has the following beneficial effects:

[0044] The present application discloses a furniture recommendation method, comprising: determining a search element; inputting the search element into a furniture recommendation model to obtain a search feature vector; obtaining from a feature vector library a plurality of feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold as a plurality of target feature vectors; and using the furniture corresponding to the plurality of target feature vectors as furniture recommended to the user.

[0045] The furniture recommendation model in this application is a model that is pre-trained based on sample images and is used to generate feature vectors. Since the sample image is an image that indicates the characteristics of the target furniture and the relative position relationship between the target furniture and other furniture placed in the use environment; therefore, the search feature vector generated based on the search element and the furniture recommendation model is a feature vector that can reflect the characteristics of the furniture in the search image and the relative position relationship between the furniture and other furniture placed in the use environment. Therefore, the furniture product corresponding to the target feature vector generated based on the search feature vector is a furniture product that can better meet the user's usage needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0047] Figure 1 A flowchart of a furniture recommendation method provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of processing search elements by a furniture recommendation model provided in an embodiment of the present application;

[0049] Figure 3A schematic diagram of a text encoder provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of an image encoder provided in an embodiment of the present application;

[0051] Figure 5A A schematic diagram of a method for acquiring training data provided in an embodiment of the present application;

[0052] Figure 5B A schematic diagram of training a sub-model to be trained provided in an embodiment of the present application;

[0053] Figure 6 A structural schematic diagram of a furniture recommendation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In the home decoration area of ​​the bank's mobile points mall, there are a large number of furniture products of different sizes, styles and materials for users to choose from. The current traditional furniture recommendation system generally recommends furniture products that may meet the user's needs through the user inputting search text.

[0055] However, recommending furniture products to users based on the search text entered by the users requires the users to accurately describe the name, features or functions of the target furniture products, which is difficult for users who are not familiar with furniture terminology, resulting in the users being unable to obtain satisfactory search results. Even if the users can accurately enter the search terms, the traditional furniture recommendation system will return a large number of search results, and the users will still waste a lot of time selecting the target furniture products from the large number of search results.

[0056] Based on the shortcomings of the above solutions, some current furniture recommendation systems rely on the user's historical browsing history and historical purchase history in addition to the search text entered by the user to recommend furniture products that may meet the user's usage needs. However, historical browsing history and historical purchase history can only reflect the user's historical behavioral preferences, and it is difficult to accurately recommend the furniture products that the user needs.

[0057] Therefore, how to accurately recommend furniture products that can better meet users' usage needs has become one of the technical problems that need to be solved urgently.

[0058] The present application discloses a furniture recommendation method, including: determining a search element; inputting the search element into a furniture recommendation model to obtain a search feature vector; obtaining multiple feature vectors whose similarity with the search feature vector is greater than or equal to a preset similarity threshold from a feature vector library as multiple target feature vectors; and using the furniture corresponding to the multiple target feature vectors as furniture recommended to the user. The furniture recommendation model in the present application is a model that is pre-trained based on sample images and is used to generate feature vectors. Since the sample image is an image that indicates the characteristics of the target furniture and the relative position relationship between the target furniture and other furniture placed in the use environment; therefore, the search feature vector generated based on the search element and the furniture recommendation model is a feature vector that can reflect the characteristics of the furniture in the search image and the relative position relationship between the furniture and other furniture placed in the use environment. Therefore, the furniture product corresponding to the target feature vector generated based on the search feature vector is a furniture product that can better meet the user's usage needs.

[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] Figure 1 A flowchart of a furniture recommendation method provided in an embodiment of the present application. Figure 1 As shown, the furniture recommendation method disclosed in this application includes:

[0061] S101, determining a search element.

[0062] The search elements in this application include search images and search texts.

[0063] The search images include real scene images and artificial images.

[0064] The real image is a picture of furniture products placed in a living room, dining room or furniture store showroom in a certain layout with other furniture products; or a picture showing the layout of various furniture in a living room, dining room or other usage scenarios.

[0065] Artificial images are pictures artificially generated through image processing tools, showing furniture products and other furniture products placed in a living room, dining room or furniture store showroom in a certain layout; or artificially generated pictures showing the layout of various furniture in a living room, dining room or other usage scenarios.

[0066] The search text is the descriptive language entered by the user to describe the characteristics, performance and use effects of the furniture products searched.

[0067] S102: input the search element into a furniture recommendation model to obtain a search feature vector.

[0068] The furniture recommendation model in this application is a model that is pre-trained based on sample images and is used to generate feature vectors. The sample images in this application are images that indicate the features of the target furniture and the relative position relationship between the target furniture and other furniture placed in the use environment.

[0069] Figure 2 A schematic diagram of processing search elements by a furniture recommendation model provided in an embodiment of the present application. Figure 2 The furniture recommendation model in provides users with furniture products that they may need through a comprehensive understanding of the query image and query text.

[0070] Combination Figure 2 As shown, the furniture recommendation model is a multimodal model including a first image encoder, a second image encoder, a first text encoder and a second text encoder. After the search image and the search text are input into the furniture recommendation model, the first image encoder processes the search image to generate a first image feature vector, i.e., the shallow features of the image; the second image encoder processes the first image feature vector to generate a second image feature vector; at the same time, the first text encoder processes the search text to generate a first text feature vector, i.e., the shallow features of the text; the second text encoder processes the first text feature vector to generate a second text feature vector. After the first image feature vector, the second image feature vector, the first text feature vector and the second text feature vector are feature fused, the feature dimension reduction is carried out at the mapping layer to obtain the search feature vector.

[0071] Figure 2 The furniture recommendation model in the invention adopts a progressive fusion method to process data of different modalities (search images and search texts), establishes connections between different levels of the furniture model, and enables shallow fusion information (shallow features of images and shallow features of texts) to be used by deep fusion information (second image feature vectors and second text feature vectors). In this way, information loss is avoided and the advantages of post-fusion are maintained. The subsequent embodiments of this application will introduce the training steps of the furniture recommendation model in detail.

[0072] In this application, one of Bidirectional Encoder Representations from Transformers (BERT), roberta or xlnet is selected as the text encoder. The type of text encoder is not limited in this application.

[0073] Figure 3 A schematic diagram of a text encoder provided in an embodiment of the present application. Figure 3 Select BERT as the text encoder. Combined with Figure 3 As shown in the figure: After the search text is segmented, it is embedded in the embedding layer to obtain word embedding. After being input into Bert, the high-level features of the text are extracted by multi-layer self-attention, and the vector output by the CLS position of the last layer is used as the text feature vector.

[0074] Figure 4 A schematic diagram of an image encoder provided in an embodiment of the present application. Figure 4 Select VisionTransformer (ViT for short) as the image encoder. Figure 4 As shown in the figure, the search image is divided into blocks, and each block is divided into blocks of a certain size. The divided blocks are combined with position embedding to form a sequence, which constitutes the input of the VIT model. Finally, after passing through multiple layers of Transformer Encoder and fully connected layers, the image feature vector is obtained.

[0075] S103, acquiring from the feature vector library a plurality of feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold as a plurality of target feature vectors.

[0076] The feature vector library in this application is generated based on the furniture recommendation model. Multiple sets of training data are input into the furniture recommendation model to generate feature vectors corresponding to each set of training data; a database constructed by the feature vectors corresponding to each set of training data is used as the feature vector library. The training data in this application includes sample images and sample texts corresponding to the sample images.

[0077] After the search feature vector is obtained, the similarity between each feature vector in the feature vector library and the search feature vector can be calculated, and then a plurality of feature vectors whose similarity is greater than or equal to a preset similarity threshold are used as target feature vectors.

[0078] It should be noted that those skilled in the art may use methods such as Euclidean distance or cosine similarity to calculate the similarity between two feature vectors; the numerical value of the similarity threshold is not limited in this application, and those skilled in the art may set the similarity threshold according to actual needs.

[0079] S104: Use the multiple sets of furniture corresponding to the multiple target feature vectors as furniture products recommended to the user.

[0080] Each target feature vector corresponds to a specific furniture product. Multiple sets of furniture corresponding to multiple target feature vectors are furniture products recommended to users.

[0081] In an optional implementation, the furniture corresponding to each target feature vector may be displayed in descending order of similarity, thereby facilitating the user to find a furniture product that better suits the user's needs from among multiple recommended furniture products.

[0082] When it is understandable, users can provide evaluation feedback on the furniture products displayed in order of similarity from large to small to improve the accuracy of retrieval and sorting.

[0083] In summary, the present application discloses a furniture recommendation method, including: determining a search element; inputting the search element into a furniture recommendation model to obtain a search feature vector; obtaining from a feature vector library multiple feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold as multiple target feature vectors; and using the furniture corresponding to the multiple target feature vectors as furniture recommended to the user.

[0084] The furniture recommendation model in this application is a model that is pre-trained based on sample images and is used to generate feature vectors. Since the sample image is an image that indicates the characteristics of the target furniture and the relative position relationship between the target furniture and other furniture placed in the use environment; therefore, the search feature vector generated based on the search element and the furniture recommendation model is a feature vector that can reflect the characteristics of the furniture in the search image and the relative position relationship between the furniture and other furniture placed in the use environment. Therefore, the furniture product corresponding to the target feature vector generated based on the search feature vector is a furniture product that can better meet the user's usage needs.

[0085] It can be understood that the furniture recommendation method in this application has the following advantages:

[0086] First, the furniture recommendation model in this application introduces an image encoder, which improves the furniture recommendation model's ability to understand images. This solution allows users to directly use photos of their living environment as query conditions, reducing reliance on professional furniture terminology. Even if the user cannot accurately describe the style or name of the furniture they need, as long as they can provide photos of the usage environment, they can recommend suitable furniture to the user based on the environmental features in the picture, making the furniture selection process more intuitive and friendly.

[0087] Second, the furniture recommendation model in this application can effectively filter and accurately match user needs, greatly reducing the number of furniture that needs to be displayed to users. When users are faced with a small number of furniture products with high relevance, they do not need to spend a lot of time screening from irrelevant products, which improves selection efficiency and shopping experience.

[0088] Third, the furniture recommendation model in this application combines the environment pictures uploaded by the user and the descriptions in the search text to accurately capture the spatial characteristics of the use environment, the style and color characteristics of the living environment. Compared with the recommendation method that only relies on text search and historical behavior in traditional solutions, this solution can more accurately understand the user's needs for matching furniture with their living environment, and can recommend the most suitable furniture for their actual living environment to users, and can provide users with more personalized recommendation results.

[0089] Fourth, in this application, not only can furniture recommendations be made based on search images and search texts, but after the recommended furniture is displayed to the user, the parameters in the furniture recommendation model can be adjusted according to the user's behavior and feedback, such as clicking on furniture products, collecting furniture products, and purchasing furniture products, thereby optimizing the furniture recommendation model so that users can get recommended products that increasingly meet their needs.

[0090] In an optional implementation, the furniture recommendation model in the present application is trained by the following steps, specifically:

[0091] The first step is to obtain multiple sets of training data and corresponding sets of label data.

[0092] The training data includes sample images and sample texts corresponding to the sample images. The label data is the serial numbers of multiple furniture products obtained after manually labeling the training data.

[0093] There are three sources of sample images in the training phase. The first is images that indicate the placement relationships between furniture in real usage scenarios. The second is images generated by a cutout tool that show the placement relationships between a few pieces of furniture and other furniture. The third is when the furniture recommendation method is executed, multiple furniture products are displayed to a specific user. The user evaluates each furniture product based on his or her actual situation and provides evaluation information that meets or does not meet the screening requirements.

[0094] Figure 5A A schematic diagram of a method for obtaining training data provided in an embodiment of the present application. Figure 5A As shown, on-site real-life scene collection (real images) and user feedback data (users' opinions on whether the recommended furniture products meet their usage requirements) are used as real-scene data; Internet data is artificially synthesized to obtain artificially synthesized data; after manual review of the above data, training set, validation set and test set are divided into 6:2:2.

[0095] It is understandable that user feedback data is the user's opinion on whether the recommended furniture products meet their usage needs; there is no user feedback data during the model training process; only when the model is trained and the furniture recommendation model is used in actual work to recommend furniture products to users, user feedback data will be generated. This data is important data for fine-tuning the parameters of furniture recommendation products, making the recommendation function of furniture recommendation products more accurate and complete.

[0096] The second step is to obtain a model to be trained. The model to be trained in this application includes M layers, each layer includes an image encoder and a text encoder, where M is an integer greater than or equal to 2.

[0097] In one possible implementation, the structure of the model to be trained is Figure 2 The structure of the furniture recommendation model shown in is similar to that of the previous one, except that there is a classification layer after the embedding in the model to be trained.

[0098] The third step is to train the model to be trained based on the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model. This step includes the following contents:

[0099] (1) An image encoder and a text encoder at the same layer in the model to be trained are used as sub-models to be trained; for each sub-model to be trained, the sub-model to be trained is trained based on multiple sets of training data to obtain a trained sub-model.

[0100] Figure 5B A schematic diagram of training a sub-model to be trained provided in an embodiment of the present application. Figure 5B As shown, the image encoder and text encoder at the same layer in the model to be trained, such as the first image encoder and the first text encoder, are used as a sub-model to be trained. The model to be trained is trained by using a comparative learning method through sample images in the training data and text images corresponding to the sample images.

[0101] During training, assume that there are N sample text image pairs in a batch, that is, N sample images and N sample texts. Take one sample text image pair from the N sample text image pairs. The paired sample text image pair is a natural positive sample. For the sample text in the positive sample, the other N-1 sample images are negative samples and are unpaired. Figure 5BThe calculation of the sample encoder (text encoder) and image encoder (image encoder) shown in the figure calculates the similarity of the inner product of the text feature vector and the image feature vector, and obtains a matrix of N rows and N columns. The values ​​on the diagonal are the inner products of the paired features. If the word vector and the corresponding sentence vector are more similar, then its value is larger, and vice versa. Set the label of the positive sample to 1 and the label of the negative sample to 0, use cross entropy for training, and try to make the inner product of the label 1 larger, so that they are more similar, thereby achieving alignment of the text feature vector and the image feature vector.

[0102] (2) Substituting each trained sub-model into the model to be trained to obtain a target model to be trained; and training the target model to be trained using multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model.

[0103] Substitute the sub-model trained in the previous step into the model to be trained to obtain the target model to be trained; train the target model to be trained based on multiple training data and label data corresponding to each group of training data, that is, compare the furniture label output by the target model to be trained with the label data, and adjust the parameters in the target model to be trained based on the difference between the two. Stop training until the preset training stop condition is met, and then delete the classification layer in the model to be trained to obtain a furniture recommendation model.

[0104] It should be noted that since the furniture recommendation model does not include a classification layer, the output of the furniture recommendation model is a feature vector.

[0105] It should be noted that although the training data used in the process of training the furniture recommendation model are sample images, the sample images include the relative position relationship between the target furniture and other furniture in a specific application environment and parameters such as the coloring and features of the target furniture; however, during use, the search image input into the furniture recommendation model may not contain the features of the target furniture, but only include information such as the color, furniture layout, style elements, etc. of other furniture in the scene.

[0106] Based on the furniture recommendation method disclosed in the aforementioned embodiment, the present application also discloses a furniture recommendation device. Figure 6 A structural schematic diagram of a furniture recommendation device provided in an embodiment of the present application.

[0107] Combination Figure 6 As shown, the furniture recommendation device 600 disclosed in this application includes:

[0108] A search element determination module 601 is used to determine a search element; the search element includes a search image and a search text;

[0109] A search feature vector determination module 602 is used to input the search element into a furniture recommendation model to obtain a search feature vector; the furniture recommendation model is a model pre-trained based on sample images and used to generate feature vectors; the sample images are images indicating features of target furniture and relative positional relationships between the target furniture and other furniture placed in a usage environment;

[0110] A target feature vector determination module 603 is used to obtain, from a feature vector library, a plurality of feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold, as a plurality of target feature vectors;

[0111] The furniture determination module 604 is used to use the multiple sets of furniture corresponding to the multiple target feature vectors as furniture products recommended to the user.

[0112] In an optional implementation, the furniture recommendation device 600 further includes:

[0113] A training data acquisition module is used to acquire multiple sets of training data and corresponding multiple sets of label data; the training data includes the sample image and the sample text corresponding to the sample image; the label data is the serial numbers of multiple furniture products obtained after manually annotating the training data;

[0114] A model acquisition module to be trained, used to acquire a model to be trained; the model to be trained includes M layers, each layer includes an image encoder and a text encoder; M is an integer greater than or equal to 2;

[0115] The furniture recommendation model acquisition module is used to train the model to be trained based on the multiple groups of training data and the multiple groups of label data to obtain the furniture recommendation model.

[0116] In an optional implementation, the furniture recommendation model acquisition module includes:

[0117] A sub-model to be trained determining unit, used for taking an image encoder and a text encoder in the same layer in the model to be trained as sub-models to be trained;

[0118] A training unit for sub-models to be trained, used for training each sub-model to be trained based on multiple groups of training data to obtain a trained sub-model;

[0119] A target model to be trained acquisition unit, used for substituting each of the trained sub-models into the model to be trained to obtain a target model to be trained;

[0120] The furniture recommendation model determination unit is used to train the target model to be trained by using the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model.

[0121] In an optional implementation, the furniture recommendation model includes a first image encoder, a second image encoder, a first text encoder, and a second text encoder; the search feature vector determination module 602 includes:

[0122] A first feature vector generating unit, configured to generate a first image feature vector based on the first image encoder and the search image;

[0123] A second feature vector generating unit, configured to generate a second image feature vector based on the second image encoder and the first image feature vector;

[0124] A third feature vector generating unit, configured to generate a first text feature vector based on the first text encoder and the search text;

[0125] a fourth feature vector generating unit, configured to generate a second text feature vector based on the second text encoder and the first text feature vector;

[0126] The search feature vector generating unit is used to perform feature fusion on the first image feature vector, the second image feature vector, the first text feature vector and the second text feature vector to generate the search feature vector.

[0127] In an optional implementation, the furniture recommendation device 600 further includes:

[0128] A feature vector generating unit, configured to generate a feature vector corresponding to each set of the training data based on the multiple sets of the training data and the furniture recommendation model;

[0129] The feature vector library generating unit is used to generate the feature vector library based on the feature vector corresponding to each group of the training data.

[0130] In an optional implementation, the furniture recommendation device 600 further includes:

[0131] The display model is used to display the furniture corresponding to each target feature vector in descending order of similarity.

[0132] Based on the furniture recommendation method and device provided in the aforementioned embodiments, the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, some or all of the steps in the aforementioned furniture recommendation method are implemented.

[0133] Based on the furniture recommendation method and device provided in the above embodiments, the present application further provides an electronic device, including:

[0134] a memory having a computer program stored thereon;

[0135] A processor is used to execute the computer program in the memory to implement part or all of the steps in the furniture recommendation method provided in the above embodiment.

[0136] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the unit described as a separate component may or may not be physically separated, and the component prompted as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A furniture recommendation method, characterized in that: The method comprises: Determine search elements; the search elements include search images and search texts; Inputting the search element into a furniture recommendation model to obtain a search feature vector; the furniture recommendation model is a model pre-trained based on sample images and used to generate feature vectors; the sample images are images indicating features of target furniture and the relative positional relationship between the target furniture and other furniture placed in a usage environment; Acquire multiple feature vectors whose similarity with the search feature vector is greater than or equal to a preset similarity threshold from the feature vector library as multiple target feature vectors; The multiple sets of furniture corresponding to the multiple target feature vectors are used as furniture products recommended to the user.

2. The method according to claim 1, characterized in that The training steps of the furniture recommendation model include: Acquire multiple sets of training data and corresponding multiple sets of label data; the training data includes the sample image and the sample text corresponding to the sample image; the label data is the serial numbers of multiple furniture products obtained after manually annotating the training data; Obtain a model to be trained; the model to be trained includes M layers, each layer includes an image encoder and a text encoder; M is an integer greater than or equal to 2; Based on the multiple groups of training data and the multiple groups of label data, the model to be trained is trained to obtain the furniture recommendation model.

3. The method according to claim 2, characterized in that The step of training the model to be trained based on the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model includes: An image encoder and a text encoder in the same layer of the model to be trained are used as sub-models to be trained; For each of the sub-models to be trained, training the sub-model to be trained based on the multiple sets of training data to obtain a trained sub-model; Substituting each of the trained sub-models into the model to be trained to obtain a target model to be trained; The target model to be trained is trained using the multiple sets of training data and the multiple sets of label data to obtain the furniture recommendation model.

4. The method according to claim 1, characterized in that: The furniture recommendation model includes a first image encoder, a second image encoder, a first text encoder, and a second text encoder; the step of inputting the search element into the furniture recommendation model to obtain a search feature vector includes: generating a first image feature vector based on the first image encoder and the search image; generating a second image feature vector based on the second image encoder and the first image feature vector; Generate a first text feature vector based on the first text encoder and the search text; Generate a second text feature vector based on the second text encoder and the first text feature vector; The first image feature vector, the second image feature vector, the first text feature vector, and the second text feature vector are subjected to feature fusion to generate the search feature vector.

5. The method according to claim 2, characterized in that: The method for acquiring the feature vector library comprises: Based on the multiple groups of training data and the furniture recommendation model, generating a feature vector corresponding to each group of training data; The feature vector library is generated based on the feature vector corresponding to each group of the training data.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The furniture corresponding to each target feature vector is displayed in descending order of similarity.

7. A furniture recommendation device, characterized in that: The device comprises: A search element determination module, used to determine the search element; the search element includes a search image and a search text; A search feature vector determination module is used to input the search element into a furniture recommendation model to obtain a search feature vector; the furniture recommendation model is a model pre-trained based on sample images and used to generate feature vectors; the sample images are images indicating features of target furniture and relative positional relationships between the target furniture and other furniture placed in a usage environment; A target feature vector determination module is used to obtain, from a feature vector library, a plurality of feature vectors whose similarity to the search feature vector is greater than or equal to a preset similarity threshold, as a plurality of target feature vectors; The furniture determination module is used to use the multiple sets of furniture corresponding to the multiple target feature vectors as furniture products recommended to the user.

8. The device according to claim 7, characterized in that The device also includes: A training data acquisition module is used to acquire multiple sets of training data and corresponding multiple sets of label data; the training data includes the sample image and the sample text corresponding to the sample image; the label data is the serial numbers of multiple furniture products obtained after manually annotating the training data; A model acquisition module to be trained, used to acquire a model to be trained; the model to be trained includes M layers, each layer includes an image encoder and a text encoder; M is an integer greater than or equal to 2; The furniture recommendation model acquisition module is used to train the model to be trained based on the multiple groups of training data and the multiple groups of label data to obtain the furniture recommendation model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images