Multi-mode home decoration recommendation method and system, storage medium and electronic device

Abstract

The invention discloses a multi-mode home decoration recommendation method and system, a storage medium and an electronic device, and relates to the technical field of smart home, and the method comprises the steps: obtaining multi-mode user home decoration demand input content; and analyzing the home decoration demand input content of the user by using a pre-trained artificial intelligence large model to obtain a home decoration demand analysis result of the user. And generating at least one home decoration effect picture according to the home decoration demand analysis result, and pushing the home decoration effect picture to the user side. Thus, the obtained home decoration demand input content of the user can comprise various formats such as characters, pictures, videos and voices, the purchase demand of the user can be known more flexibly and accurately, the home decoration demand analysis result of the user is obtained by using the pre-trained artificial intelligence large model, the home decoration purchase demand of the user is fully analyzed, and the user experience is improved. And a plurality of home decoration effect pictures are generated to more intuitively recommend home decoration effects to the user, so that actual home decoration requirements under various different application scenes are met.

Description

Technical Field

[0001] The present application relates to the field of smart home, and more specifically, to a multimodal home decoration recommendation method, system, storage medium and electronic device. Background Art

[0002] As a huge market, the competition situation within the home improvement industry is constantly changing. With the development of the Internet platform, the standardization of the home improvement industry has been further improved, and the home improvement market is shifting from offline to online service platforms. This requires industry companies to not only provide customers with intelligent and personalized one-stop services, but also to accurately meet customers' actual purchasing needs and better recommend home improvement products to customers.

[0003] At present, sales staff in the home improvement industry generally obtain customer needs by printing some home improvement brochures for customers to view before verbal communication with them or according to sales needs in advance, and recommend products in the store based on customer needs. However, this offline sales method is not only inefficient, but also has a high customer return rate because customers cannot clearly determine whether the plan is suitable for them. Online sales methods such as mobile applications that connect to floor plan databases and real estate information databases provide more abundant home improvement product data and home improvement plans for sales staff to use, but the program maintenance cost required for this sales method is extremely high, and because customers cannot intuitively see the actual effects of home improvement products, it cannot meet the needs of customers in most application scenarios.

[0004] Accordingly, a new solution is needed in the art to solve the above problems. Summary of the invention

[0005] This application aims to solve the above-mentioned technical problem, namely, how to meet the actual home improvement needs of different customers in a variety of different application scenarios.

[0006] In a first aspect, the present application provides a multimodal home improvement recommendation method, comprising:

[0007] Acquire multimodal user home improvement demand input content; wherein the user home improvement demand input content includes at least two of text input content, voice input content, picture input content, and video input content;

[0008] Utilize the pre-trained artificial intelligence model to analyze the user's home improvement demand input content to obtain the user's home improvement demand analysis result;

[0009] At least one home decoration rendering is generated according to the home decoration demand analysis result, and the home decoration rendering is pushed to the user end.

[0010] In a technical solution of the above-mentioned multimodal home improvement recommendation method, the use of a pre-trained artificial intelligence large model to parse the user's home improvement demand input content to obtain the user's home improvement demand analysis result includes:

[0011] If the user's home improvement demand input content includes the text input content, the text input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a text parsing result;

[0012] If the user's home improvement demand input content includes the voice input content, the voice input content is analyzed using the natural language processing function of the artificial intelligence big model to obtain a voice analysis result;

[0013] If the user's home improvement demand input content includes the picture input content, the picture input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a picture parsing result;

[0014] If the user's home improvement demand input content includes the video input content, the video input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a video parsing result;

[0015] The multimodal capability of the artificial intelligence large model is utilized to integrate at least two of the text analysis results, voice analysis results, image analysis results, and video analysis results, and the home improvement demand analysis results are generated based on the integration results.

[0016] In a technical solution of the above multi-modal home improvement recommendation method, generating the home improvement demand analysis result according to the integration result includes:

[0017] Extracting preset demand content from the integration result to form the home improvement demand analysis result;

[0018] The preset demand content may include at least one of the user's house type information, house internal dimension data, purchase demand type, decoration demand type and expected price of demand.

[0019] In a technical solution of the above multi-modal home improvement recommendation method, at least one home improvement effect picture is generated according to the home improvement demand analysis result, including:

[0020] Creating a three-dimensional model of the user's house according to the user's house layout information;

[0021] Selecting corresponding home improvement products according to the home improvement demand analysis results, and determining the decorative positions of the home improvement products in the home improvement renderings;

[0022] According to the internal dimension data of the user's house and the specific data of the home improvement product, obtaining the dimension ratio data of the user's house and the home improvement product;

[0023] The home decoration effect diagram is generated according to the decoration position and the size ratio data.

[0024] In a technical solution of the above-mentioned multimodal home improvement recommendation method, selecting corresponding home improvement products according to the home improvement demand analysis results and determining the decoration position of the home improvement products in the user's house includes:

[0025] Searching for home improvement products corresponding to the home improvement demand analysis results in a search engine associated with the artificial intelligence big model;

[0026] According to the database associated with the artificial intelligence big model, the decorative position of the home improvement product in the home improvement rendering is determined, and the artificial intelligence big model is associated with at least one of the floor plan database, the big model database and the home improvement rendering database.

[0027] In a technical solution of the above multi-modal home decoration recommendation method, generating the home decoration effect picture according to the decoration position and the size ratio data includes:

[0028] Adding a corresponding information tag to each home decoration product in the home decoration rendering, wherein the information tag is used to display detailed information of the home decoration product to the user;

[0029] The detailed information includes at least one of the function introduction, installation price, decoration style and home decoration color of the home decoration product.

[0030] In a technical solution of the above multi-modal home improvement recommendation method, the method further includes modifying the home improvement products in the home improvement rendering through the following steps, including:

[0031] After obtaining the user replacement instruction, determining the target home improvement product to be replaced in the user replacement instruction;

[0032] Selecting home improvement products of the same type as the target home improvement product from a preset home improvement product database, and generating a home improvement product replacement list for the user to select;

[0033] According to the selection result of the user, replacing the three-dimensional model of the target home improvement product in the home improvement rendering; and / or,

[0034] The method further comprises purchasing the home improvement products in the home improvement renderings through the following steps, including:

[0035] After obtaining the user's one-key order instruction, determining all home improvement products in the home improvement rendering according to the user's one-key order instruction;

[0036] Combine all the home improvement products into one purchase order, and jump to the supply platform to submit the purchase order.

[0037] In a second aspect, a multimodal home decoration recommendation system is provided, the system comprising: a user home decoration demand acquisition module, a home decoration demand analysis module and a home decoration recommendation module;

[0038] The user home improvement demand acquisition module is used to acquire multi-modal user home improvement demand input content; wherein the user home improvement demand input content includes at least two of text input content, voice input content, picture input content, and video input content;

[0039] The home improvement demand analysis module is used to analyze the user's home improvement demand input content using a pre-trained artificial intelligence model to obtain the user's home improvement demand analysis result;

[0040] The home decoration recommendation module is used to generate at least one home decoration rendering according to the home decoration demand analysis result, and push the home decoration rendering to the user end.

[0041] In a third aspect, an electronic device is provided, comprising at least one processor and at least one memory, wherein the memory is suitable for storing a plurality of program codes, and the program codes are suitable for being loaded and run by the processor to execute the multimodal home improvement recommendation method described in any one of the technical solutions of the multimodal home improvement recommendation method.

[0042] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the multimodal home improvement recommendation method described in any one of the technical solutions of the multimodal home improvement recommendation method.

[0043] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:

[0044] In the case of adopting the above technical solution, the present application provides a multimodal home improvement recommendation method, including: first, obtaining multimodal user home improvement demand input content; wherein the user home improvement demand input content includes at least two of text input content, voice input content, picture input content, and video input content. Then, the user's home improvement demand input content is parsed using a pre-trained artificial intelligence large model to obtain the user's home improvement demand analysis results. Finally, based on the home improvement demand analysis results, at least one home improvement rendering is generated, and the home improvement rendering is pushed to the user end. Through the above configuration method, the present application obtains multimodal user home improvement demand input content, and the user's home improvement demand input content can include text, pictures, videos, voice and other formats, so as to understand the user's purchasing needs more flexibly and accurately. Secondly, the multimodal capabilities and natural language understanding capabilities of the pre-trained artificial intelligence large model are used to extract the user's home improvement demand analysis results from the user's home improvement demand input content, and generate a home improvement rendering. In this way, the artificial intelligence big model can be used to fully understand the user's home improvement purchasing needs, and the home improvement renderings can be used to more intuitively display and recommend home improvement effects to users, thereby more personally meeting the customer's actual home improvement needs in a variety of different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] Figure 1 is a flowchart of the main steps of a multimodal home improvement recommendation method according to an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the main steps of an implementation method for generating a home decoration effect diagram in an embodiment of the present application;

[0049] Figure 3 It is a schematic flow chart of the main steps of a multi-modal home improvement recommendation method in one implementation of the present application;

[0050] Figure 4 It is a hardware environment diagram of an interaction method of a smart device according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] According to one aspect of the embodiments of the present application, a multi-modal home improvement recommendation method is provided. Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a multi-modal home improvement recommendation method according to an embodiment of the present application. Figure 1 As shown, the multimodal home improvement recommendation method includes steps S101 to S103:

[0054] Step S101: Obtain multi-modal user home improvement demand input content.

[0055] Among them, multimodality refers to information of multiple modes, which may include text, images, videos, and audios. That is to say, the user's home improvement demand input content may include at least two of text input content, voice input content, picture input content, and video input content. For example, the user's house floor plan and the user's home improvement demand description voice can be used as the user's home improvement demand input content at the same time; the user's house interior environment video and the user's home improvement demand description document can also be used as the user's home improvement demand input content at the same time.

[0056] In this embodiment, the user can describe his or her home improvement needs in various formats such as text, voice, video or pictures to obtain a list of suitable home improvement product recommendations. In actual application scenarios, the home improvement sales program applied to the mobile client can also be used to obtain the user's voice when the salesperson or AI intelligent voice communicates with the customer; the salesperson can also upload floor plans, pictures or videos of the environmental conditions of each space in the house, and text or voice describing the customer's home improvement purchase needs as the user's home improvement needs input content.

[0057] Step S102: Utilize the pre-trained artificial intelligence model to parse the user's home improvement demand input content and obtain the user's home improvement demand analysis result.

[0058] In this embodiment, the Artificial Intelligence Large Model (AI Large Model) is a machine learning algorithm based on a neural network. These models are usually based on deep learning technology, can process large amounts of data and learn complex patterns and relationships from them, can automatically adjust the model structure and parameters according to changes in input data to meet the needs of different scenarios, and play an important role in natural language processing, image recognition, speech recognition and other fields. The basic principle of the AI ​​Large Model is a complex neural network model obtained through large-scale data training, which usually contains millions to billions of parameters for complex calculations and reasoning on input data. In addition, the AI ​​Large Model can not only be quickly applied to new fields and tasks through technologies such as transfer learning and fine-tuning, reducing development costs and time, but also can continuously improve the performance and accuracy of the model through continuous training and optimization.

[0059] In one implementation, step S102 may further include steps S1021 to S1025:

[0060] Step S1021: If the user's home improvement demand input content includes text input content, the natural language processing function of the artificial intelligence large model is used to parse the text input content to obtain a text parsing result.

[0061] In this embodiment, the artificial intelligence big model can understand the semantic content of the text input content through a series of technologies and methods, including lexical semantics, syntactic analysis, semantic role labeling and named entity recognition.

[0062] In this embodiment, the text analysis results can be set according to the preset result analysis granularity of the artificial intelligence model. For example, the text analysis results can include the customer's house location, house type, house decoration status information, house internal environment information, home decoration purchase budget price and other customer home decoration demand information.

[0063] Step S1022: If the user's home improvement demand input content includes voice input content, the natural language processing function of the artificial intelligence large model is used to parse the voice input content to obtain a voice analysis result.

[0064] In this embodiment, parsing voice input content using the natural language processing function of the artificial intelligence large model can specifically include: using speech recognition technology to capture the acoustic features in the voice input content and converting it into text, then performing semantic understanding and intent recognition on the converted text, analyzing the grammar, syntax and semantics of the text, so as to understand the meaning and context of the voice input content.

[0065] Step S1023: If the user's home improvement demand input content includes picture input content, the natural language processing function of the artificial intelligence large model is used to parse the picture input content to obtain the picture analysis result.

[0066] In this embodiment, using the natural language processing function of the artificial intelligence big model to parse the image input content can specifically include: using the natural speech processing technology of the artificial intelligence big model combined with computer vision technology to extract visual information such as the target, direction, length data and environment from the image input content, and then identifying and processing the text information in the image input content, extracting keywords, phrases or sentences, and then understanding the semantic content in the image, integrating the text information and image features to form an analysis result of the image.

[0067] In this embodiment, if the image input content is a floor plan or a picture of the interior environment of a house, the image analysis result may include information such as the house type, decoration style, house area and specific size data of the interior of the house.

[0068] Step S1024: If the user's home improvement demand input content includes video input content, the natural language processing function of the artificial intelligence large model is used to parse the video input content to obtain a video analysis result.

[0069] In this embodiment, parsing the video input content using the natural language processing function of the artificial intelligence large model can specifically include: using the intelligent video analysis technology in the natural language processing technology to automatically generate a concise and accurate text description based on the video content, and using optical character recognition (OCR) technology to recognize the text content in the video image to obtain the parsed content of the video input content.

[0070] Step S1025: Utilize the multimodal capabilities of the artificial intelligence large model to integrate at least two of the text analysis results, voice analysis results, image analysis results, and video analysis results, and generate a home improvement demand analysis result based on the integration result.

[0071] In this embodiment, multimodal capability refers to the ability of the artificial intelligence model to process, understand and generate data across multiple perceptual modalities (such as text, pictures, videos and sounds, etc.). Through the fusion processing of multi-modal data, the artificial intelligence model can use multiple different data inputs at the same time, so that it is closer to human cognitive ability in complexity and understands and processes information more comprehensively. For example, by using image and voice input content at the same time, the characteristics and environmental background of the user's house can be more accurately identified and understood, thereby improving the accuracy of image recognition and voice recognition, and then improving the accuracy of parsing the user's home improvement demand input content, and obtaining a home improvement demand analysis result that is more in line with the user's actual needs.

[0072] In one implementation, step S1025 may further include:

[0073] Step S10251: extracting preset demand content from the integration results to form a home improvement demand analysis result.

[0074] In this embodiment, the preset demand content can be set according to the result analysis granularity of the artificial intelligence large model, and the result analysis granularity can be set according to the detail level of parsing the input content according to the home improvement sales demand.

[0075] In one implementation, the preset demand content may include at least one of the user's house type information, house internal dimension data, purchase demand type, decoration demand type, and expected price of demand.

[0076] In this embodiment, the house type information may include the number of rooms and living rooms, room orientations, house area and other information; the house internal dimension data includes the length, width and height of the room, the length, width and height of the living room, and the length, width and height of the balcony; the purchase demand type may include the purchase of a complete home improvement plan or the purchase of home appliances; the decoration demand type may include simple decoration, fine decoration or rough decoration.

[0077] Step S103: Generate at least one home decoration rendering according to the home decoration demand analysis result, and push the home decoration rendering to the user end.

[0078] In this embodiment, the artificial intelligence model can be combined with the floor plan database, the network search engine, the real estate information database and the home decoration effect database to generate 1-10 home decoration effect pictures for users to choose from.

[0079] In this embodiment, pushing the home decoration effect picture to the user terminal is used to display the generated home decoration effect picture to the user. The user terminal is used to be installed on a common client and needs to cooperate with the server terminal to run, access the resources and services provided by the server terminal through the network and provide local services to the user. For example, the user terminal may include a personal computer or a mobile device.

[0080] In one embodiment, please refer to the attached Figure 2 , Figure 2 1 is a schematic diagram of the main steps of an implementation method of generating a home decoration effect diagram according to an embodiment of the present application. Figure 2 As shown, step S103 can generate a home decoration effect diagram according to the following steps: step S1031 to step S1034:

[0081] Step S1031: Create a three-dimensional model of the user's house according to the user's house layout information.

[0082] In this embodiment, the three-dimensional model of the house can be created by three-dimensional modeling software, including but not limited to building a basic frame of the house, adding decorations inside the house, wall textures, doors and windows, etc. In some embodiments, the three-dimensional model of the house can be reduced or enlarged according to manual operations of the user.

[0083] Step S1032: Select corresponding home decoration products according to the home decoration demand analysis results, and determine the decorative positions of the home decoration products in the home decoration renderings.

[0084] In this embodiment, the home decoration product may be a home decoration product to be sold stored in a home decoration database of a seller, or may be a three-dimensional model of a home decoration product design case in an existing home decoration effect database.

[0085] In one implementation, step S1032 may further include step S10321 to step S10322:

[0086] Step S10321: Search for home improvement products corresponding to the home improvement demand analysis results in the search engine associated with the artificial intelligence big model.

[0087] Step S10322: Determine the decorative position of the home improvement product in the home improvement rendering according to the database associated with the artificial intelligence large model.

[0088] In this embodiment, the artificial intelligence big model is associated with at least one of the floor plan database, the big model database and the home decoration effect database.

[0089] Among them, the floor plan database is an online database that centrally manages residential building product data. It covers corporate information, regional information, project coefficients, residential types, plane forms, width, depth, width-to-depth ratio, building area, common area, number of elevators and households, number of rooms, halls, bathrooms, utility rate and comfort coefficient, etc. The large model database refers to a database that uses natural language processing, machine learning, graphs and other technical means to obtain and organize data from multiple data sources. It can cover knowledge in various fields, including science, history, culture, medicine, engineering, etc. It is used to integrate scattered information on the Internet to form a complete knowledge network, providing more accurate and comprehensive knowledge support for artificial intelligence systems. The home decoration effect database is a database containing home decoration design cases, materials, models and other resources. It aims to provide designers and customers with rich home decoration design references and inspirations. It usually includes design cases, material samples, furniture models, design ideas, material selection and color matching information of various types of houses.

[0090] Step S1033: According to the internal dimension data of the user's house and the specific data of the home decoration products, the dimension ratio data of the user's house and the home decoration products are obtained.

[0091] Step S1034: Generate a home decoration rendering according to the decoration position and size ratio data.

[0092] In this embodiment, the home decoration rendering may include multiple home decoration products for users to select, modify and place orders.

[0093] In one embodiment, each 3D model of a home improvement product may be provided with a link so that a user can jump to a display page of specific information of the home improvement product by clicking on the home improvement product in the home improvement rendering. In other embodiments, each home improvement product may also be provided with a label pointing to the corresponding home improvement product for roughly displaying basic information of the home improvement product.

[0094] As an example, step S1034 may further include:

[0095] Step S10341: Add corresponding information tags for each home decoration product in the home decoration rendering.

[0096] In this embodiment, the information tag is used to display detailed information of the home decoration product to the user. The detailed information may include at least one of the function introduction, installation price, decoration style and home decoration color of the home decoration product.

[0097] In this embodiment, the user can also modify or change the home furnishing products in the home furnishing renderings by issuing a user replacement command, or can place an order for the home furnishing products involved in the home furnishing renderings with one click by issuing a user order command. In other embodiments, the user can also change the size data or color of the home furnishing products by selecting the home furnishing products in the home furnishing renderings during the interaction process.

[0098] In one implementation, the home decoration products in the home decoration rendering may be modified by the following steps, including steps S201 to S203:

[0099] Step S201: after obtaining the user replacement instruction, determining the target home improvement product to be replaced in the user replacement instruction.

[0100] Step S202: selecting home improvement products of the same type as the target home improvement product from a preset home improvement product database, and generating a home improvement product replacement list for user selection.

[0101] Step S203: according to the selection result of the user, replace the three-dimensional model of the target home decoration product in the home decoration rendering.

[0102] In one implementation, after the user confirms the plan in the home decoration rendering, the user can place an order for all the home decoration products in the home decoration rendering with one click, and the finalized plan list can be shared with the customer.

[0103] For example, the home decoration products in the home decoration renderings can be purchased through the following steps, including steps S301 to S302:

[0104] Step S301: after obtaining the user's one-key order instruction, all home improvement products in the home improvement rendering are determined according to the user's one-key order instruction.

[0105] Step S302: combine all home improvement products into one purchase order, and jump to the supply platform to submit the purchase order.

[0106] In one embodiment, please refer to the attached Figure 3 , Figure 3 FIG. 1 is a flow chart of the main steps of a multi-modal home improvement recommendation method in one embodiment of the present application. Figure 2 As shown, the multimodal home improvement recommendation method may further include:

[0107] Step S401: Obtain customer input content according to preset requirements.

[0108] In this embodiment, if Figure 2 As shown, the input content may be in a multi-modal form, for example, input content including text, audio, video, and picture modalities may be obtained.

[0109] In this embodiment, the preset requirements can be data information on multiple aspects of customer needs that the home improvement salesperson needs to understand in advance, or it can be data information on various aspects contained by the customer to express his or her own home improvement purchase intentions, neither of which affects the normal implementation of the embodiments of the present application.

[0110] For example, preset requirements may include the following aspects: which aspect of the sales plan to consult, such as "smart home improvement" or "smart appliances"; the size of the customer's house, such as "115 square meters"; the floor plan of the customer's house, such as "three bedrooms and one living room", "two bedrooms and one living room, two bedrooms facing south, and the master bedroom has a balcony" or "large open space", etc.; whether the existing decoration in the customer's house is rough, simple or fine; the decoration style preferred by the customer; purchase type, such as "complete plan purchase" or "purchase of appliances"; expected price and other special requirements, etc.

[0111] In one implementation, the customer's input content may be obtained through a mobile client, and then uploaded to a cloud server for storage through the client.

[0112] Step S402: Upload the acquired input content to the cloud server.

[0113] In this embodiment, the cloud server is a server based on cloud computing technology, which provides efficient, stable and secure cloud computing services. Users can access the cloud server through the Internet to store input content, build and deploy AI models, and perform data analysis.

[0114] In one implementation, a trained AI model can be pre-built in a cloud server to analyze input content to obtain the customer's home improvement purchase needs. The customer's home improvement purchase needs may include the customer's apartment type information, the environment information in the house, the decoration information (rough, simple, fine, etc.), the community information, and the geographical location.

[0115] Step S403: Control the cloud server to input the input content into the AI ​​big model to analyze the customer's purchase intention.

[0116] In this embodiment, the AI ​​big model can integrate information from modalities such as text, audio, video, and pictures to fully understand the input content. Through the natural language understanding ability of the AI ​​big model, it can understand the customer's text or voice information. Through the multimodal capabilities of the AI ​​big model, it can understand the uploaded picture and video information, and analyze the customer's home improvement purchase needs or home improvement consultation intentions.

[0117] In one implementation, the granularity of the result analysis of the AI ​​big model can be defined as needed.

[0118] Step S404: The AI ​​big model fuses multimodal information to generate and output a home improvement plan.

[0119] In one implementation, 1 to 10 home decoration effect pictures can be generated based on the home decoration needs of the customer in combination with the home decoration database stored in the house type diagram database, the large model information database, the real estate information database, and the home decoration effect database. In other implementations, the home decoration effect pictures can also be generated in combination with the home decoration products stored in the cloud server and the internal environment of the customer's house, which does not affect the normal implementation of this embodiment.

[0120] Step S405: Display the home improvement plan output by the AI ​​large model to the user through at least one of voice response and text display, plan rendering display or 3D space display.

[0121] In this embodiment, through the home decoration rendering display method including but not limited to voice response and text display, solution rendering display or 3D space display, customers can consult all information related to smart home decoration in real time, including the information of all products in the home decoration rendering, product function introduction, product price, recommendation of similar products and decoration information.

[0122] Step S406: Obtain the result of whether the user is satisfied. If the user is not satisfied, proceed to step S407; if the user is satisfied, proceed to step S408.

[0123] Step S407: Obtain the modification instruction or rendering regeneration instruction issued by the user, and transmit it to the cloud server so that the cloud server regenerates the home improvement plan.

[0124] In this embodiment, customers can make real-time modifications to the displayed home decoration renderings, including but not limited to replacing the home decoration products in the home decoration renderings, changing the color or size of the home decoration products, and changing the display method of the home decoration renderings, etc.

[0125] Step S408: Perform one-click sharing or one-click ordering operation.

[0126] In this embodiment, if Figure 2 As shown. The one-click ordering operation may include: generating a one-click purchase order instruction, filling in customer information and payment, and then completing the one-click ordering operation to generate a home improvement purchase order. In some other implementations, it may also include steps such as the process progress and time update of the subsequent delivery and installation of home improvement products.

[0127] In this embodiment, if Figure 2As shown, before the one-click order operation, a one-click sharing operation can also be selected. The one-click sharing operation may include: generating a one-click sharing instruction, generating a QR code, and sharing through a mini program on a mobile client, and then continuing the one-click order operation.

[0128] Through the above steps S101 to S103, this embodiment can obtain multi-modal user home improvement demand input content, which can include text, pictures, videos, voice and other formats, so as to understand the user's purchasing needs more flexibly and accurately. In addition, it can also fully understand the user's home improvement purchase needs through the artificial intelligence big model, and use the home improvement effect map to more intuitively recommend home improvement products to the user, so as to more personally meet the actual home improvement needs of customers in various different application scenarios.

[0129] According to one aspect of an embodiment of the present application, a multimodal home decoration recommendation system is also provided, which includes: a user home decoration demand acquisition module, a home decoration demand analysis module and a home decoration recommendation module.

[0130] The user home improvement demand acquisition module is used to obtain multi-modal user home improvement demand input content.

[0131] Among them, the user's home improvement demand input content includes at least two of text input content, voice input content, picture input content, and video input content.

[0132] The home improvement demand analysis module is used to use the pre-trained artificial intelligence large model to analyze the user's home improvement demand input content and obtain the user's home improvement demand analysis results.

[0133] The home decoration recommendation module is used to generate at least one home decoration rendering according to the home decoration demand analysis results, so as to recommend home decoration products in the home decoration rendering to users.

[0134] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium may be configured to store a program for executing the multimodal home improvement recommendation method of the above-mentioned method embodiment, and the program may be loaded and run by a processor to implement the above-mentioned multimodal home improvement recommendation method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a memory device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0135] According to one aspect of an embodiment of the present application, an electronic device is provided. In an electronic device embodiment according to the present application, the electronic device includes a processor and a memory. The memory can be configured to store a program for executing the multimodal home improvement recommendation method of the above method embodiment. The processor can be configured to execute the program in the memory, which includes but is not limited to the program for executing the multimodal home improvement recommendation method of the above method embodiment. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The electronic device can be an electronic device device formed by various electronic devices.

[0136] In the embodiment of the present application, the electronic device may be an electronic device including various electronic devices. In some possible implementations, the electronic device may include multiple memories and multiple processors. The program for executing the multimodal home improvement recommendation method of the above method embodiment may be divided into multiple subprograms, each of which may be loaded and run by a processor to execute different steps of the multimodal home improvement recommendation method of the above method embodiment. Specifically, each subprogram may be stored in different memories, and each processor may be configured to execute programs in one or more memories to jointly implement the multimodal home improvement recommendation method of the above method embodiment, that is, each processor executes different steps of the multimodal home improvement recommendation method of the above method embodiment to jointly implement the multimodal home improvement recommendation method of the above method embodiment.

[0137] The above-mentioned multiple processors may be processors deployed on the same device. For example, the above-mentioned electronic device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors may also be processors deployed on different devices. For example, the above-mentioned electronic device may be a server cluster, and the above-mentioned multiple processors may be processors on different servers in the server cluster.

[0138] According to one aspect of the embodiment of the present application, a method for interacting with a smart home device is also provided. The method for interacting with the smart home device is widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the method for interacting with the smart home device can be applied to Figure 3 In the hardware environment composed of the terminal device 102 and the server 104 shown in FIG. Figure 3As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.

[0139] The network may include but is not limited to at least one of the following: a wired network, a wireless network. The wired network may include but is not limited to at least one of the following: a wide area network, a metropolitan area network, a local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 102 may not be limited to a PC, a mobile phone, a tablet computer, etc.

[0140] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multimodal home improvement recommendation method, characterized in that: The method comprises: Acquire multimodal user home improvement demand input content; wherein the user home improvement demand input content includes at least two of text input content, voice input content, picture input content, and video input content; Utilize the pre-trained artificial intelligence model to analyze the user's home improvement demand input content to obtain the user's home improvement demand analysis result; At least one home decoration rendering is generated according to the home decoration demand analysis result, and the home decoration rendering is pushed to the user end.

2. The multimodal home improvement recommendation method according to claim 1, characterized in that: The method of using the pre-trained artificial intelligence model to analyze the user's home improvement demand input content to obtain the user's home improvement demand analysis result includes: If the user's home improvement demand input content includes the text input content, the text input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a text parsing result; If the user's home improvement demand input content includes the voice input content, the voice input content is analyzed using the natural language processing function of the artificial intelligence big model to obtain a voice analysis result; If the user's home improvement demand input content includes the picture input content, the picture input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a picture parsing result; If the user's home improvement demand input content includes the video input content, the video input content is parsed using the natural language processing function of the artificial intelligence big model to obtain a video parsing result; The multimodal capability of the artificial intelligence large model is utilized to integrate at least two of the text analysis results, voice analysis results, image analysis results, and video analysis results, and the home improvement demand analysis results are generated based on the integration results.

3. The multimodal home improvement recommendation method according to claim 2, characterized in that: Generating the home improvement demand analysis result according to the integration result includes: Extracting preset demand content from the integration result to form the home improvement demand analysis result; The preset demand content may include at least one of the user's house type information, house internal dimension data, purchase demand type, decoration demand type and expected price of demand.

4. The multimodal home improvement recommendation method according to claim 3, characterized in that: According to the home improvement demand analysis result, at least one home improvement effect picture is generated, including: Creating a three-dimensional model of the user's house according to the user's house layout information; Selecting corresponding home improvement products according to the home improvement demand analysis results, and determining the decorative positions of the home improvement products in the home improvement renderings; According to the internal dimension data of the user's house and the specific data of the home improvement product, obtaining the dimension ratio data of the user's house and the home improvement product; The home decoration effect diagram is generated according to the decoration position and the size ratio data.

5. The multimodal home improvement recommendation method according to claim 4, characterized in that: The selecting corresponding home improvement products according to the home improvement demand analysis result, and determining the decoration position of the home improvement products in the user's house, includes: Searching for home improvement products corresponding to the home improvement demand analysis results in a search engine associated with the artificial intelligence big model; According to the database associated with the artificial intelligence big model, the decorative position of the home improvement product in the home improvement rendering is determined, and the artificial intelligence big model is associated with at least one of the floor plan database, the big model database and the home improvement rendering database.

6. The multimodal home improvement recommendation method according to claim 4, characterized in that: The generating the home decoration effect diagram according to the decoration position and the size ratio data includes: Adding a corresponding information tag to each home decoration product in the home decoration rendering, wherein the information tag is used to display detailed information of the home decoration product to the user; The detailed information includes at least one of the function introduction, installation price, decoration style and home decoration color of the home decoration product.

7. The multimodal home improvement recommendation method according to claim 1, characterized in that: The method further comprises modifying the home decoration products in the home decoration rendering by the following steps, including: After obtaining the user replacement instruction, determining the target home improvement product to be replaced in the user replacement instruction; Selecting home improvement products of the same type as the target home improvement product from a preset home improvement product database, and generating a home improvement product replacement list for the user to select; According to the selection result of the user, replacing the three-dimensional model of the target home improvement product in the home improvement rendering; and / or, The method further comprises purchasing the home improvement products in the home improvement renderings through the following steps, including: After obtaining the user's one-key order instruction, determining all home improvement products in the home improvement rendering according to the user's one-key order instruction; Combine all the home improvement products into one purchase order, and jump to the supply platform to submit the purchase order.

8. A multimodal home improvement recommendation system, characterized in that: The system includes: a user home decoration demand acquisition module, a home decoration demand analysis module and a home decoration recommendation module; The user home improvement demand acquisition module is used to acquire multi-modal user home improvement demand input content; wherein the user home improvement demand input content includes text input content, voice input content, picture input content, At least two of the video input contents; The home improvement demand analysis module is used to analyze the user's home improvement demand input content using a pre-trained artificial intelligence model to obtain the user's home improvement demand analysis result; The home decoration recommendation module is used to generate at least one home decoration rendering according to the home decoration demand analysis result, and push the home decoration rendering to the user end.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the multimodal home improvement recommendation method according to any one of claims 1 to 7 when running.

10. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to execute the multimodal home improvement recommendation method according to any one of claims 1 to 7 through the computer program.

Images