Display device and wearing and overlapping recommendation method

Abstract

The invention relates to a display device and a wearing and building recommendation method. The device comprises: a display configured to display content from a broadcast system or a network and / or a user interface, and to display a user interface of a smart wear application in response to a startup operation triggered by a user for the smart wear application; an image collector configured to collect a user image of a user; the controller is configured to construct wearing and building prompt information for a user based on the user image; presenting a wearing and building scheme determined on the basis of the wearing and building prompt information on a user interface; the wearing scheme is presented through the upper body effect of the target clothes; the putting-on and putting-off prompt information comprises intention information representing the intention of the user and respective article information of a plurality of pieces of candidate clothes matched with the intention of the user; the plurality of candidate clothes are one part of clothes owned by the user; the target clothes are obtained by screening the multiple pieces of candidate clothes according to the intention information through the wearing and building model. By adopting the scheme of the invention, the accuracy of wearing and building recommendation can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and an outfit recommendation method. Background Art

[0002] With the rapid development of technology, more and more additional functions have emerged on the basis of the original functions of display devices, such as information search function, outfit recommendation function, etc. Integrating the outfit recommendation function in the display device can show users the effect of clothing trying on the human model.

[0003] In traditional technology, users are provided with dressing solutions based on the sharing of fashion bloggers or the listing of shopping websites. Since these reference information are relatively fixed and universal, the dressing solutions provided based on these reference information are also universal and difficult to adapt to the personalized needs of different users.

[0004] Therefore, using traditional technology will result in inaccurate outfit recommendation results. Summary of the invention

[0005] The present application provides a display device and an outfit recommendation method to solve the problem of inaccurate outfit recommendation results.

[0006] In a first aspect, some embodiments provide a display device, including:

[0007] A display, configured to display content from a broadcast system or network and / or a user interface, and, in response to a start operation triggered by a user for the smart dressing application, display a user interface of the smart dressing application;

[0008] An image collector is configured to collect a user image of the user;

[0009] The controller is configured as:

[0010] Based on the user image, construct outfit prompt information for the user; the outfit prompt information includes intention information representing the user's intention and item information of each of a plurality of candidate clothes matching the user's intention; the plurality of candidate clothes are part of the clothes owned by the user;

[0011] In the user interface, a dressing plan determined based on the dressing prompt information is presented; the dressing plan is presented through the effect of the target clothing on the body; the target clothing is screened from the multiple candidate clothing by the dressing model according to the intention information.

[0012] In a second aspect, some embodiments further provide an outfit recommendation method, which is applied to the display device provided in the first aspect. The method comprises:

[0013] In response to a start operation triggered by a user on a smart dressing application, displaying a user interface of the smart dressing application;

[0014] collecting a user image of the user;

[0015] Based on the user image, construct outfit prompt information for the user; the outfit prompt information includes intention information representing the user's intention and item information of each of a plurality of candidate clothes matching the user's intention; the plurality of candidate clothes are part of the clothes owned by the user;

[0016] In the user interface, a dressing plan determined based on the dressing prompt information is presented; the dressing plan is presented through the effect of the target clothing on the body; the target clothing is screened from the multiple candidate clothing by the dressing model according to the intention information.

[0017] In a third aspect, some embodiments further provide an outfit recommendation device, which is applied to the display device provided in the first aspect. The device comprises:

[0018] A user interface display module, configured to display a user interface of the smart dressing application in response to a start-up operation triggered by a user on the smart dressing application;

[0019] An image acquisition module, used to acquire a user image of the user;

[0020] A prompt information construction module is used to construct outfit prompt information for the user based on the user image; the outfit prompt information includes intention information representing the user's intention and item information of each of a plurality of candidate clothes matching the user's intention; the plurality of candidate clothes are part of the clothes owned by the user;

[0021] A dressing plan presentation module is used to present a dressing plan determined based on the dressing prompt information on the user interface; the dressing plan is presented through the effect of the target clothing on the body; the target clothing is screened from the multiple candidate clothing by the dressing model according to the intention information.

[0022] In a fourth aspect, some embodiments further provide a projection device. The projection device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the outfit recommendation method provided in the second aspect when executing the computer program.

[0023] In a fifth aspect, some embodiments further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the outfit recommendation method provided in the second aspect.

[0024] In a sixth aspect, some embodiments further provide a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the outfit recommendation method provided in the second aspect.

[0025] It can be seen from the above technical solution that the present application displays the user interface of the smart dressing application in response to the startup operation triggered by the user for the smart dressing application, and then constructs dressing prompt information for the user based on the collected user image, and presents the dressing scheme determined based on the dressing information in the user interface. Among them, the dressing prompt information includes intention information representing the user's intention, and the item information of multiple candidate clothes that match the user's intention, and the multiple candidate clothes are part of the clothes owned by the user. Filter multiple candidate clothes that match the user's intention from the clothes owned by the user, and then combine the item information of these candidate clothes and the intention information representing the user's intention into prompt information to guide the dressing model to generate a dressing scheme, which is equivalent to generating a dressing scheme based on the user's intention by referring to external information that is suitable for the user and has a high degree of matching. It can provide personalized dressing schemes for different users, which is conducive to improving the accuracy of the dressing recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0028] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0029] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0030] Figure 4 A schematic diagram of the process of outfit recommendations provided in some embodiments of the present application;

[0031] Figure 5 A schematic diagram of a display interface of a smart dressing application provided in some embodiments of the present application;

[0032] Figure 6 An interactive flow chart of an outfit recommendation method provided in some embodiments of the present application;

[0033] Figure 7 A flowchart of an outfit recommendation method provided in some embodiments of the present application. DETAILED DESCRIPTION

[0034] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0035] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0036] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0037] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0038] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0039] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0040] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0041] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0042] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0043] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0044] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0045] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0046] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0047] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface. In some optional embodiments, the display device may also include a touch component connected to the display 260, which may be a touch pad, a touch screen, etc.

[0048] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0049] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0050] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0051] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0052] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0053] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0054] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0055] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0056] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0057] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0058] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the applications in the application layer include but are not limited to the above examples.

[0059] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0060] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0061] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0062] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0063] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0064] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0065] In some embodiments, a display device is provided, the display device comprising: a display configured to display content from a broadcast system or network and / or a user interface, and, in response to a start-up operation triggered by a user for the smart dressing application, display a user interface of the smart dressing application; an image collector configured to collect a user image of the user. The display device also includes a controller configured to: construct dressing prompt information for the user based on the user image; and present a dressing plan determined based on the dressing prompt information in the user interface.

[0066] The smart dressing application refers to an application for providing dressing solutions. The smart dressing application is deployed in a display device.

[0067] The outfit suggestion information includes intention information representing the user's intention and item information of each of multiple candidate clothes matching the user's intention.

[0068] User intent refers to the purpose and needs expressed by users during interaction, which reflects the goals that users hope to achieve in a specific situation. Specifically in the outfit recommendation scenario, user intent reflects the characteristics of the outfit plan that users expect to obtain.

[0069] Intent information is used to characterize user intent. The intent information can specifically include dimensional information of multiple intent dimensions such as style, color, material, and scene, and can characterize user intent from multiple dimensions. Among them, styles can include retro style, minimalist style, sweet style, lady style, etc.; colors can include warm colors, cool colors, Modilan colors, etc.; materials can include cotton, linen, chiffon, etc.; scenes can include salons, dinners, meetings, etc. Exemplarily, the intent information can be "I need to attend a dinner party and require warm-colored Chinese style clothing." The user intent represented is: I expect to obtain a dressing plan with "warm colors", "Chinese style" and "dinner party" scenes.

[0070] The multiple candidate clothes are part of the clothes owned by the user. The clothes owned by the user refer to the clothes that the user currently owns, which can be understood as the clothes stored in the user's wardrobe. The clothing types to which these clothes belong may specifically include clothes, pants, shoes, hats, etc. The source of these clothes can be purchased online or in offline stores. Specifically, an item database containing the item information of each item of clothing can be constructed for the clothes owned by the user. The item information may include pictures, materials, sizes, colors, brands, categories, and other information.

[0071] A dressing model refers to a neural network model that has professional knowledge in the field of dressing and is used to provide dressing solutions. The specific network structure of the dressing model is not unique, and can include a fully connected neural network (FCNN), a convolutional neural network (CNN), a recurrent neural network (RNN), or a deep neural network (DNN).

[0072] In an optional embodiment, the dressing model is a large language model (LLM) that has been trained through professional knowledge learning and instruction fine-tuning in the field of dressing. Specifically, on the basis of the open source large model, firstly, through incremental pre-training, a large amount of dressing expertise and information can be instilled into the basic LLM, increasing the professional ability and content creation ability of the large model in the field of dressing, thereby providing a capability foundation for the large model to generate dressing solutions. Then, by adding an appropriate amount of precisely labeled data, the instructions of the pre-trained LLM are fine-tuned, so that the dressing content generation and output style of the LLM are professional and consistent. Finally, by constructing a professional evaluation set to verify the professional ability of the LLM in the field of dressing, the model is further optimized for the feedback issues, and finally a dressing model with strong professional ability can be obtained.

[0073] Specifically, the user can trigger a startup operation for the smart wear application. The triggering method of the startup operation is not unique. Optionally, the display of the display device can display the application logo of the smart wear application, and the user can trigger the startup operation for the smart wear application by selecting the application logo. Optionally, the display device may include a sound collector, and the user can trigger the startup operation for the smart wear application by voice.

[0074] When the user triggers the startup operation for the smart wear application, the display can respond to the startup operation and display the user interface of the smart wear application. The user interface (UI) refers to the medium for interaction and information exchange between the system and the user, which can include a graphical user interface (GUI), a command line interface (CLI), a touch interface, a voice user interface (VUI) and other forms.

[0075] The image collector can collect the user's user image so as to recommend a dressing plan for the user later. Specifically, the controller can control the image acquisition device to collect the user image. The user image at least includes the user's facial image. In an optional embodiment, the user image can include the user's facial image and torso image, wherein the facial image can be used to identify the user's identity and makeup style, and the torso image can be used to determine the user's current body shape.

[0076] After collecting the user image, the controller can, on the one hand, identify the user based on the user image, and determine the clothes the user owns based on the user identity; on the other hand, it can determine the user's makeup style, user emotions, and user body shape based on the user image, and then extract the intent information including style characteristics, emotional characteristics, and body shape characteristics. Then, the controller can filter out multiple candidate clothes that match the user's intent from the clothes owned by the user based on the intent information. Then, the controller can fuse the intent information and the item information of each candidate clothing to construct dressing prompt information for the user. Among them, the specific methods of fusing intent information and item information can include splicing, semantic-based information fusion, and so on.

[0077] For example, when the intent information is "I need to attend a dinner party, and warm-colored traditional Chinese style clothing is required", the dressing prompt information can be "I need to attend a dinner party, and warm-colored traditional Chinese style clothing is required. Please select appropriate clothing from the following candidate clothing to determine the dressing plan: item information A, item information B..."

[0078] The constructed dressing prompt information will be input into the dressing model, so that the dressing model can filter the target clothing from multiple candidate clothing according to the intention information, and further determine the dressing scheme including each target clothing. The dressing scheme is presented by the upper body effect of the target clothing. Optionally, the controller can present the upper body effect of the target clothing based on the virtual character image. Optionally, based on the user image including the torso image, the controller can also present the upper body effect of the target clothing based on the user image corresponding to the user image.

[0079] In an optional embodiment, the dressing model can be deployed on the server side of the smart dressing application, which provides service support for multiple display devices and determines the dressing plan through interaction between the display device and the server side. Specifically, the display device can send the dressing prompt information to the server side, so that the dressing model deployed on the server side can determine the dressing plan provided for the user under the guidance of the dressing prompt information, and feed the dressing plan back to the display device. The display device then presents the dressing plan on the user interface.

[0080] In an optional embodiment, when the computing power of the display device meets the requirements, the dressing model can also be deployed on the display device, and the display device can determine and present the dressing plan alone.

[0081] The above technical solution has the following advantages or beneficial effects: in response to the startup operation triggered by the user for the smart dressing application, the user interface of the smart dressing application is displayed, and then based on the collected user image, the dressing prompt information for the user is constructed, and the dressing scheme determined based on the dressing information is presented in the user interface. Among them, the dressing prompt information includes intention information representing the user's intention, and the item information of each of the multiple candidate clothes that match the user's intention, and the multiple candidate clothes are part of the clothes owned by the user. Multiple candidate clothes that match the user's intention are screened from the clothes owned by the user, and then the item information of these candidate clothes and the intention information representing the user's intention are combined into prompt information to guide the dressing model to generate a dressing scheme, which is equivalent to generating a dressing scheme based on the user's intention by referring to external information that is suitable for the user and has a high degree of matching, and can provide personalized dressing schemes for different users, which is conducive to improving the accuracy of the dressing recommendation results.

[0082] In some embodiments, when the controller executes the step of constructing dressing tip information for the user based on the user image, the controller is further configured to: construct intention information representing the user's intention based on the user image; filter out multiple candidate clothes that match the user's intention from the clothes owned by the user; and splice the intention information and the item information of each candidate clothing to obtain dressing tip information for the user.

[0083] Specifically, the controller can perform user intent recognition based on the user image and construct intent information representing the user intent. User intent recognition based on the user image includes but is not limited to recognizing the user's makeup style, recognizing the user's current emotion, recognizing the user's body shape, etc.

[0084] Then, the controller determines the clothes owned by the user based on the user identity corresponding to the user image, and selects a plurality of candidate clothes that match the user's intention from the clothes owned by the user.

[0085] There is no single specific way to screen candidate clothing items.

[0086] Optionally, the controller can match the dimensional information of the user's clothes in the intent dimension represented by the intent information with the dimensional information of the intent dimension in the intent information, and filter candidate clothes with matching dimensional information from the user's clothes. Among them, the intent dimension may include style, color, material, scene, etc. Matching two dimensional information may mean that the two dimensional information are the same or similar. Taking color as an example, light blue, lake blue, and dark blue may be similar.

[0087] Optionally, the controller can map the intention information and the item information of the clothes owned by the user to the same feature space respectively to obtain the intention feature and the item feature. Then, for each item feature, the feature similarity between the item feature and the intention feature is determined. Finally, the clothes whose feature similarity meets the similarity condition are determined as candidate clothes. Among them, the data form of the item feature and the intention feature can be a matrix or a feature. The feature similarity can be represented by Euclidean distance, cosine similarity, Hamming distance, etc. The feature similarity meets the similarity condition, which can mean that the feature similarity is greater than or equal to the similarity threshold.

[0088] After determining the intention information and the candidate clothes, the controller can obtain the dressing prompt information for the user by splicing the intention information and the item information of each candidate clothes. Exemplarily, the controller can splice the dressing prompt information in a way that the intention information comes first and the item information comes second, or in a way that the intention information comes second and the item information comes first, and can also insert the item information into the intention information.

[0089] Take the case where the intent information is "I need to attend a dinner party, and I require warm-colored traditional Chinese style clothing", and the item information is "item information A, item information B" as an example. The dressing prompt information can be "I need to attend a dinner party, and I require warm-colored traditional Chinese style clothing. Please select suitable clothing from the following candidate clothing to determine the dressing plan: item information A, item information B"; it can also be "Please select suitable clothing from the following candidate clothing to determine the dressing plan: item information A, item information B. I need to attend a dinner party, and I require warm-colored traditional Chinese style clothing"; it can also be "I need to attend a dinner party, please select suitable clothing from the following candidate clothing to determine the dressing plan: item information A, item information B. Warm-colored traditional Chinese style clothing is required."

[0090] The above technical solution has the following advantages or beneficial effects: first, candidate clothes that match the user's intention are screened from the clothes owned by the user, and then the intention information and the item information of each candidate intention are spliced ​​together to obtain personalized dressing prompt information that is adapted to the user's intention, which can further improve the accuracy of the dressing recommendation results.

[0091] In some embodiments, the display device further includes: a sound collector configured to obtain user voice. In the case of this embodiment, when the controller executes the step of constructing the intention information representing the user's intention based on the user image, it is further configured to: query the user's character portrait information from the character portrait database based on the user identity represented by the user image; determine the dressing reference information including the user's voice, user image and character portrait information; use the pre-trained large language model to perform multimodal intention understanding on the dressing reference information, and construct the intention information representing the user's intention.

[0092] Among them, the startup operation of the smart dressing application is triggered by user voice.

[0093] Specifically, the user can wake up the dressing application service through voice. The user voice can be, for example, "give me a suitable outfit recommendation" or "open the dressing application service".

[0094] After the dressing application service is awakened, the controller can call the image collector to capture the user's current image, obtain the user image, and match the user image with the user image in the character portrait database to determine the user's identity. Then, based on the user identity, the controller can query the user's character portrait information from the character portrait database.

[0095] Among them, character portrait information may include but is not limited to the following information: gender, age, selfie photos, height, weight, personality and historical dressing records, etc.

[0096] After obtaining the user's character portrait information, the controller can determine the dressing reference information including the user's voice, user image and character portrait information. Then, the pre-trained large language model is used to perform multimodal intent understanding on the dressing reference information and construct intent information that represents the user's intent.

[0097] Among them, the use of a pre-trained large language model to perform multimodal intent understanding of dressing reference information can include: recognizing user emotions expressed by user voice; recognizing the makeup style of user images; combining user images and character portrait information to determine the user's body shape, preferences, etc.

[0098] It should be noted that if the user's character portrait information cannot be queried from the character portrait database based on the user identity represented by the user image, it means that the user is a new user. At this time, the display device can guide the user to supplement basic information such as height, weight, age, gender, etc., and use this basic information as the user's character portrait information and add it to the character portrait database. Optionally, the display device can output the query information through an audio output device, or display the query information in the user interface of the display to guide the user to supplement the basic information.

[0099] In an optional embodiment, if Figure 4 As shown, the user can input user voice into the display device, and the voice recognition application deployed in the display device converts the voice into text to wake up the smart dressing application service. The user voice can be, for example: "Give me a suitable outfit recommendation." After voice wake-up, the camera application is first called to capture the user's current image, and match it with the user in the character portrait database to determine whether the user's character portrait exists. If it exists, it is an old user, and continue to ask for some advanced data, such as travel occasions, current mood, weather, etc.; if it does not exist, it is a new user, and before asking for advanced data, the user will be asked to add some basic data, such as height, weight, age, gender, etc., and the new user portrait will be saved in the character portrait database.

[0100] The above technical solution has the following advantages or beneficial effects: based on user voice, user image and character portrait information, multimodal dressing reference information is determined, and then a pre-trained large language model is used to perform multimodal intention understanding on the dressing reference information, and construct intention information that represents user intention. It can mine user intention from information of multiple modalities, thereby improving the intention representation capability of intention information, which is conducive to further improving the accuracy of dressing plans determined based on intention information.

[0101] In some embodiments, the display device further includes: an audio output device configured to output dressing scene inquiry information. In the case of this embodiment, the sound collector is also configured to obtain the user's response voice feedback to the dressing scene inquiry information. When the controller executes the step of determining the dressing reference information including the user voice, user image and character portrait information, it is further configured to: identify the dressing scene information expressed by the response voice; determine the dressing reference information including the user voice, user image, character portrait information and dressing scene information.

[0102] The dressing scene inquiry information is used to inquire about the dressing scene. The dressing scene inquiry information may be, for example, "What kind of occasion do you need to dress for?" or "Please describe the dressing scene."

[0103] Specifically, when the character portrait information is obtained through the query, the display device can further inquire about some advanced information. The advanced information may include dressing scene information, current mood, etc. Then, the display device can obtain the user's response voice for the dressing scene information feedback from the sound collector, and further identify the dressing scene information expressed by the response voice. The dressing scene information may include time, location, event, etc., which are used to describe the dressing scene. Finally, the controller determines the dressing reference information including user voice, user image, character portrait information and dressing scene information.

[0104] Optionally, the controller can determine the weather information based on the time and location of the dressing scene, and can also determine the dressing style that matches the event based on the event of the dressing scene, thereby obtaining dressing reference information including dimensions such as weather and style.

[0105] The above technical solution has the following advantages or beneficial effects: the display device further obtains dressing reference information by interacting with the user, can enrich the content of the dressing reference information, and further improve the user's intention expression ability based on the intention information determined by the dressing reference information, which is conducive to further improving the accuracy of the dressing plan determined based on the intention information.

[0106] In some embodiments, when the controller executes the step of filtering out multiple candidate clothes that match the user's intention from the clothes owned by the user, the controller is further configured to: determine the clothing type and item features of each of the multiple clothes owned by the user, and the intention features of the user's intention; for each clothing type, based on the feature similarity between the item features corresponding to the clothing type and the intention features, filter candidate clothes from the clothes belonging to the clothing type.

[0107] The clothing type refers to the type determined according to the purpose, and the clothing type may specifically include clothes, pants, shoes, hats, etc.

[0108] The item feature is information used to represent the characteristics of clothing. The specific data form of the item feature can be a vector or a matrix. The intention feature is information used to identify the intention feature. The specific data form of the intention feature can be a vector or a matrix. It can be understood that the data form of the item feature and the intention feature is the same, that is, when the item feature is an item vector, the intention feature is an intention vector; when the item feature is an item matrix, the intention feature is an intention vector.

[0109] Specifically, the controller may determine the clothing types to which the plurality of clothing items owned by the user respectively belong, and the item characteristics of the respective clothing items, and determine the intention characteristics of the user's intention.

[0110] Optionally, an item database may be constructed for multiple clothes owned by the user. The item database may include item information, clothing type, and item features of each piece of clothing owned by the user. Thus, the controller may query the clothing type and item features of each piece of clothing owned by the user from the item database.

[0111] Optionally, the item database may include item information and clothing types. The controller may extract features from the item information to obtain item features for each clothing item. Specific methods for feature extraction include, for example, using algorithms such as K-means clustering or support vector machines to extract item features, and using deep learning technology to extract item features. Exemplarily, the item information may be input into a convolutional neural network, and the output of the convolutional neural network may be used as the item feature.

[0112] Optionally, natural language processing technology can be used to extract features from intent information to obtain intent features. Exemplarily, the controller can use a text classification model, a sequence tagging model (such as LSTM and BERT), and a named entity recognition (NER) model to extract intent features from intent information. The controller can also use the BERT language model for feature extraction and capture intent features through the word embedding representation output by the Transformer encoding layer.

[0113] Optionally, a machine learning method may be used to extract keywords, phrases, and other information from the intent information, and to construct intent features based on the extracted information.

[0114] After obtaining the clothing type, item features, and intention features, the controller can determine the feature similarity between the item features corresponding to the clothing type and the intention features for each clothing type, and then select candidate clothing whose feature similarity meets the similarity condition from the clothing belonging to the clothing type. Thus, candidate clothing of the clothing type can be obtained for each clothing type. Exemplarily, the candidate clothing may include: candidate tops A1, candidate tops A2, and candidate tops A3; candidate pants B1, candidate pants B2, and candidate pants B3; candidate shoes C1, candidate shoes C2.

[0115] Furthermore, the feature similarity may be represented by Euclidean distance, cosine similarity, Hamming distance, etc. The feature similarity satisfies the similarity condition, which may mean that the feature similarity is greater than or equal to a similarity threshold.

[0116] The above technical solution has the following advantages or beneficial effects: for each clothing type, candidate clothing is screened from the clothing belonging to the clothing type based on the feature similarity between the features of each item corresponding to the clothing type and the intended features, and the corresponding candidate clothing can be screened for different clothing types respectively, thereby achieving fine-grained dressing plan recommendations, which is conducive to further improving the accuracy of the recommendation results.

[0117] In some embodiments, when the controller executes the step of determining the item features of each of the multiple clothes owned by the user, it is further configured to: obtain the item information of each of the multiple clothes; for each piece of clothing, label the clothing with features according to the item information of the clothing, and determine the parameter values ​​of the clothing corresponding to different dressing parameters; map each parameter value to a feature space containing each dressing parameter to obtain the item features of the clothing.

[0118] Among them, dressing parameters refer to the various factors and standards that need to be considered in the process of clothing matching, including but not limited to size, style, fabric, color, etc.

[0119] Among them, the size can be determined based on anthropometric data, which may include waist circumference, chest circumference, shoulder width, sleeve length, etc.

[0120] Styles can be divided according to cultural background, occasion or design concept, etc. For example, according to different cultural backgrounds, dressing styles can include classic style, classical style, British style, etc.; according to different occasions, dressing styles can include sports style, casual style, OL (office occupation) style, etc.; according to different design concepts, dressing styles can include gorgeous style, simple style, street style, etc.

[0121] Fabrics can be divided according to type, characteristics, or source, etc. For example, according to fabric type, clothing fabrics can be divided into cotton, wool, silk, linen, etc.; according to fabric characteristics, clothing fabrics can include soft fabrics, glossy fabrics, etc.

[0122] Colors can be classified according to basic colors such as primary colors, secondary colors and complex colors, or they can be classified more finely according to hue, purity and lightness. Taking hue as an example, it can be divided into warm colors, cool colors and intermediate colors.

[0123] Specifically, the controller can obtain the item information of each of the multiple clothes. Then, for each piece of clothing, the clothing is characterized according to the item information of the clothing, and the parameter values ​​of the clothing corresponding to different dressing parameters are determined. For example, a certain piece of clothing is warm-colored, cotton, and sports-style clothing. Then, the controller maps each parameter value to a feature space containing each dressing parameter to obtain the item feature of the clothing.

[0124] Furthermore, the method of mapping each parameter value to the feature space containing each dressing parameter can be based on a pre-set corresponding relationship or the mapping can be achieved by using an Embedding model.

[0125] In an optional embodiment, for clothing purchased online, the controller can actively obtain the link of the purchased clothing by applying for online application permissions, and automatically process the item information in the link to obtain the picture, material and other descriptive information of the clothing to obtain the item information.

[0126] In an optional embodiment, it is supported to obtain the item information of purchased clothing by copying and pasting or manually entering a link.

[0127] In an optional embodiment, if Figure 4 As shown in the figure, for clothes purchased in offline stores, you can scan the product code of the purchased clothes or take a photo of the item to obtain the item information. Figure 4 As shown, the item information may include clothing picture information, clothing product links, clothing text descriptions, etc. Among them, for items that are only photographed, LLM needs to be called additionally in the data processing pipeline to generate information descriptions of pictures, etc. After the item information is processed and classified in the data processing pipeline, the professional dressing field LLM will be used again to annotate the data to obtain the parameter values ​​of the clothing corresponding to different dressing parameters. Finally, the embedding model is used to realize the vectorization conversion of the data and store it in the item database.

[0128] The above technical solution has the following advantages or beneficial effects: clothing is labeled based on dressing parameters, and the labeled information is further mapped to the feature space containing various dressing parameters to obtain the item features of the clothing. This can ensure that the extracted item features are the key factors that need to be considered in the dressing recommendation process, which is conducive to further improving the accuracy of the dressing recommendation results.

[0129] In some embodiments, when the controller executes the step of screening candidate clothing from each clothing type based on the feature similarity between each item feature corresponding to the clothing type and the intention feature, the controller is further configured to: screen candidate clothing whose feature similarity meets the similarity condition from each clothing type according to the feature similarity between each item feature corresponding to the clothing type and the intention feature; use the re-ranking model to determine the fitness of each candidate clothing corresponding to the user's intention; and screen candidate clothing whose fitness meets the fitness condition from each candidate clothing.

[0130] Among them, the specific limitations on feature similarity can be found above and will not be repeated here.

[0131] The re-ranking model is a model used to re-rank the initial ranking results in an information retrieval or recommendation system. Its main purpose is to improve the relevance and accuracy of the final results through more sophisticated evaluation and ranking. The re-ranking model can specifically be a multi-objective ranking model, which considers multiple objectives (such as conversion rate, collection rate, etc.) at the same time and ranks the candidate clothes through a multi-objective ranking algorithm; it can also be a deep learning model, which uses deep learning models such as transformer and Bi-LSTM to predict conversion rate or collection rate, etc., and re-rank the candidate clothes according to the predicted results. In an optional embodiment, the re-ranking model is a lightweight Re-rank model.

[0132] Among them, the conversion rate of alternative clothing may refer to the ratio between the number of times a user accepts a dressing plan including the alternative clothing and the number of times a dressing plan including the alternative clothing is recommended to the user.

[0133] The collection rate of an alternative clothing item may refer to the ratio between the number of collections of outfit plans including the alternative clothing item and the number of times the outfit plans including the alternative clothing item are recommended to the user.

[0134] It can be understood that the higher the conversion rate or collection rate, the more compatible the candidate clothing is with the user's intention.

[0135] Specifically, the controller can select candidate clothes whose feature similarity meets the similarity condition from the clothes belonging to the clothing type according to the feature similarity between the features of each item corresponding to the clothing type and the intended features. Then, the re-ranking model is used to determine the fitness of each candidate clothing corresponding to the user's intention. Finally, the candidate clothing whose fitness meets the fitness condition is selected from the candidate clothing. Furthermore, the fitness meets the fitness condition, which means that the fitness is greater than or equal to the fitness threshold.

[0136] In an optional embodiment, multiple clothing samples carrying fitness labels for the intent samples can be obtained, and the initial model can be trained using each clothing sample to obtain a re-ranking model for determining the fitness between the candidate clothing and the user's intent. The fitness labels of the clothing samples for the intent samples can be determined based on the conversion rate or collection rate of the clothing samples in the outfit recommendation scenario corresponding to the intent sample, or can be determined based on the expert experience in the outfit field.

[0137] The above technical solution has the following advantages or beneficial effects: from each clothing type, select alternative clothing whose feature similarity meets the similarity conditions, and then use the re-ranking model to determine the fitness of each alternative clothing corresponding to the user's intention, and then select candidate clothing whose fitness meets the fitness conditions from each alternative clothing. Before inputting the dressing model, the number of candidate clothing can be further reduced, thereby reducing the requirements of the dressing recommendation process on the device computing power, thereby saving computing resources.

[0138] In some embodiments, when the controller executes the step of splicing the intention information and the item information of each candidate clothing to obtain dressing prompt information for the user, it is further configured to: sort the item information of each candidate clothing according to the clothing type and fit of each candidate clothing to obtain item reference information; splicing the intention information and the item reference information to obtain dressing prompt information for the user.

[0139] The order of the item information in the item reference information is used to represent the priority of the candidate clothing corresponding to the item information.

[0140] Specifically, in the process of splicing the dressing prompt information, the priority of the candidate clothes can be considered. The controller can sort the item information of each candidate clothing according to the clothing type and fitness of each candidate clothing to obtain item reference information, and then splice the intention information and the item reference information to obtain the dressing prompt information for the user.

[0141] For example, the outfit reminder message may be "I need to attend a dinner party, which requires warm-colored traditional Chinese style clothing. Please select appropriate clothing from the following candidate clothing items in order of priority from front to back to determine the outfit plan: item information A, item information B".

[0142] The above technical solution has the following advantages or beneficial effects: in the process of splicing dressing prompt information, the priority of candidate clothing is considered, and the priority is positively correlated with the degree of fit between the candidate clothing and the user's intention, which can guide the dressing model to provide a more accurate dressing plan.

[0143] In some embodiments, when the controller executes the step of presenting a dressing plan determined based on the dressing prompt information on the user interface, the controller is further configured to: display a virtual character image on the user interface; and present a dressing plan determined based on the dressing prompt information and descriptive information for the dressing plan corresponding to the virtual character image.

[0144] Among them, the virtual character image is generated based on the user's character portrait information.

[0145] Specifically, the controller can generate and display a virtual character image matching the user on the user interface based on the user's character portrait information. Then, the virtual character image presents the effect of the target clothing on the body and the description information of the dressing scheme. The description information may include, for example, matching ideas, explanations of item information for each part, etc.

[0146] In some optional embodiments, the controller may output descriptive information about the dressing plan in an audio format by calling an audio output device.

[0147] In an optional embodiment, the controller can automatically call a 3D modeling tool to construct a virtual character image based on the character portrait information, thereby realizing the screen display of the 3D character clothing portrait.

[0148] In an optional embodiment, the calling function of the 3D modeling tool can be integrated into the dressing model. Specifically, during the model training process, professional 3D modeling tools and calling interfaces can be collected, and data such as tool capabilities, tool information, and interface function introductions can be processed; at the same time, a large amount of 3D character dressing data can be collected, and the data can be pre-processed, including image standardization, feature extraction, calling tools, calling interfaces, and professional function calling links, etc., to complete the training of the 3D modeling calling function. Therefore, during the application process, the dressing model can support the output of 3D character dressing portraits based on the determination of the dressing method.

[0149] The above technical solution has the following advantages or beneficial effects: corresponding to the virtual character image matching the user, presenting the dressing plan and its description information, can intuitively present the upper body effect of the target clothing to the user, which is conducive to improving the user experience.

[0150] In some embodiments, the controller is further configured to: obtain feedback information from the user regarding the dressing plan; if the feedback information indicates that the user is dissatisfied with the dressing plan, obtain supplementary information input by the user; adjust the dressing plan based on the supplementary information, and obtain and present an updated dressing plan on the user interface.

[0151] The supplementary information may include the unsatisfactory part and the adjustment direction for the unsatisfactory part, etc.

[0152] After the dressing plan is presented, the plan can be continuously improved and adjusted through interaction between the user and the display device until a dressing plan that satisfies the user is obtained.

[0153] Specifically, the controller can obtain feedback information from the user on the dressing scheme. The feedback information can be text or voice, or it can be information formed by triggering a selection operation on a feedback control. The feedback control can include a "satisfied" control and a "dissatisfied" control. The feedback information can indicate that the user is very satisfied with the dressing scheme, or it can indicate that the user is dissatisfied with the dressing scheme.

[0154] When the feedback information indicates that the user is satisfied with the outfit plan, the controller can end the outfit recommendation process and store the outfit plan in the character portrait database for subsequent reference.

[0155] When the feedback information indicates that the user is not satisfied with the outfit plan, the user's inputted supplementary information can be obtained through voice interaction, text interaction, touch screen interaction, etc. Then, the outfit plan is adjusted according to the supplementary information, and an updated outfit plan is obtained and presented on the user interface.

[0156] The above technical solution has the following advantages or beneficial effects: for dressing plans that users are not satisfied with, local modifications can be made through multiple rounds of interactions, ultimately achieving user satisfaction, which is conducive to further improving the accuracy of dressing recommendation results.

[0157] In some embodiments, the controller is further configured to: input the intention information into the dressing model so that the dressing model, under the guidance of the intention information, outputs a dressing reference plan that matches the user's intention; and present the dressing reference plan and the clothing information of each online clothing item corresponding to the virtual character image.

[0158] Among them, dressing suggestions are presented through the wearing effects of online clothes. Online clothes refer to clothing products that users do not currently own and are sold through Internet channels.

[0159] The clothing information may include clothing images, links, and the like.

[0160] Specifically, the intention information representing the user's intention can be input into the dressing model, so that the dressing model outputs a dressing reference scheme that matches the user's intention based on online clothing under the guidance of the intention information. Then, the dressing reference scheme and the clothing information of each online clothing included in the dressing reference scheme are presented corresponding to the virtual character image.

[0161] In an optional embodiment, when feedback information indicates that the user is dissatisfied with the dressing plan, the controller may output a dressing reference plan that matches the user's intention based on online clothing, so as to provide the user with a shopping reference, thereby enriching the user's item database and facilitating subsequent dressing recommendations.

[0162] In an optional embodiment, if Figure 5As shown, the display device can simultaneously present in the user interface a dressing plan 501 based on the clothes owned by the user and a dressing reference plan 502 based on online clothes, so that the user can determine whether to purchase online clothes through comparison.

[0163] The above technical solution has the following advantages or beneficial effects: based on the online clothing presentation, a dressing reference plan that matches the user's intention can be provided to the user, thereby enriching the user's item database, forming a virtuous circle, and helping to improve the accuracy of subsequent dressing recommendation results.

[0164] In some embodiments, the training process of the dressing model includes: obtaining an open source big model and sample information of each of a plurality of clothing samples; using the sample information to perform incremental pre-training on the open source big model to obtain a clothing recognition model with clothing recognition capabilities; using dressing samples carrying dressing professionalism labels to perform incremental training on the clothing recognition model to obtain an initial dressing model with dressing domain knowledge; performing instruction fine-tuning training on the initial dressing model to obtain a dressing model.

[0165] Among them, the open source large model can be a large language model.

[0166] The clothing sample is clothing used as a sample. The sample information refers to item information of the clothing sample.

[0167] In actual applications, the server of the smart dressing application can perform the training process of the dressing model. Specifically, the server can obtain the open source big model and the sample information of multiple clothing samples. Then, the open source big model is incrementally trained using the sample information to obtain a clothing recognition model with clothing recognition capabilities.

[0168] Then, the server uses the dressing samples with dressing professionalism labels and adopts ORPO (Odds Ratio Preference Optimization) or reinforcement learning to perform incremental training on the clothing recognition model to obtain an initial dressing model with professional dressing knowledge.

[0169] Finally, the initial dressing model is trained with fine-tuning instructions to align the content output format and obtain an applicable dressing model.

[0170] In an optional embodiment, the current fashion trends and styles can be analyzed by collecting professional data information in the field of dressing, such as fashion websites, TOP bloggers, well-known brands, professional books, etc. On the basis of the open source big model, firstly, through the method of incremental pre-training, a large amount of dressing professional knowledge and information is instilled into the basic LLM, and the professional ability and content creation ability of the big model in the field of dressing are increased, thereby providing a capability foundation for the big model to generate professional guidance suggestions. Then, by adding an appropriate amount of refined and labeled data, the instructions of the pre-trained LLM are fine-tuned to make the dressing content generation and output style of the LLM professional and consistent. Finally, by constructing a professional evaluation set to verify the professional ability of the LLM in the field of dressing, the model is further optimized for the feedback problems, and finally a dressing model with strong professional ability can be obtained.

[0171] In an optional embodiment, the generation capability of 3D character dressing portraits can also be specially trained and improved based on the initial dressing model. First, professional 3D modeling tools and calling interfaces are collected, and data such as tool capabilities, tool information, and interface function introductions are processed; at the same time, a large amount of 3D character dressing data is collected, and the data is pre-processed, including image standardization, feature extraction, calling tools, calling interfaces, and professional function call links, etc., to complete the incremental pre-training of the model. Then, through the refined part of the fashion dressing-related 3D character model data, such as professional function call links, etc., instruction fine-tuning training is performed to align the content output form of LLM, etc. Finally, a complete 3D character dressing evaluation set is constructed, and further targeted capacity improvement and optimization are carried out for the problems reported in the evaluation set test, so as to finally achieve the purpose of the dressing model being able to well support the output of 3D character dressing portraits.

[0172] The above technical solution has the following advantages or beneficial effects: a dressing model is obtained by adopting a step-by-step and multiple training method, which significantly reduces the requirement for the amount of sample data in the training process, thereby reducing the difficulty of model training and improving work efficiency.

[0173] In an optional embodiment, if Figure 4 and Figure 6 As shown, the outfit recommendation process based on the display device includes the following steps:

[0174] Step 1: The user wakes up the smart dressing application service through the voice function. For example, "Give me a suitable outfit recommendation." After voice wake-up, the camera application is first called to take a picture of the user's current image, and then matched with the users in the character portrait database to determine whether the user is an old user. If the user is an old user, some advanced data will continue to be asked, such as travel occasions, current mood, etc.; if the user is a new user, before asking for advanced data, the user will be asked to add some basic data, such as height, weight, age, gender, etc., and the new user portrait will be saved in the character portrait database.

[0175] Step 2: After the character portrait information is transmitted to the professional dressing model (i.e., dressing model), the image processing and information organization capabilities of LLM are used to analyze the user's intentions, and the data is processed and combined in a unified manner, such as: identification of the makeup style of the user's selfie, analysis of the height and weight, etc., and finally outputting the intention information containing the user's intention tendency. Among them, the dressing model is trained on the basis of LLM. For the specific training process of the dressing model, please refer to the above, which will not be repeated here.

[0176] Step 3: After slicing the intent information, extract key information (such as style, color, scene, etc.), vectorize it through the Embedding model, and search and match it with the data in the item database to obtain more suitable clothing for different types of clothing (such as clothes, pants, shoes, hats, etc.). Then send the obtained item information to the Re-rank model for re-ranking, and then distinguish the suitability of different clothing.

[0177] Step 4: Reassemble the generated intent information and the item information obtained after reordering into a valid prompt and send it to the dressing model. The dressing model analyzes and processes the input information based on its professional capabilities in the dressing field, and finally outputs a suitable dressing plan, and automatically calls the 3D modeling tool to realize the screen display of the 3D character dressing portrait. After the 3D portrait is drawn, relying on the LLM's image and text understanding capabilities, the voice application explains and describes the dressing plan awards.

[0178] Step 5: If you are not satisfied with some parts of your outfit, you can use voice interaction to feedback the existing problems and provide detailed information. After receiving the supplementary information, the smart dressing application will redesign the plan for the unsatisfactory parts and generate a modified 3D character outfit portrait. After the user confirms that the portrait is satisfactory, the final portrait information will be stored in the character portrait database for future reference.

[0179] Step 6: This step is synchronized with the above step 3. After LLM outputs the intention information containing the user's intention tendency, it will directly output a suitable outfit suggestion based on its professional outfit field capabilities, and based on the suggestion information, actively call the AI ​​search interface to obtain relevant clothing images and links from major professional online shopping malls. Finally, by calling the 3D modeling tool, LLM can realize the screen display of the optimal 3D character outfit portrait generated by itself, and simultaneously list the product details of each part of the outfit.

[0180] In the above-mentioned outfit recommendation process, multiple rounds of interactions can be used to provide users with real-time outfit suggestions that fit the user's actual situation, which is specifically reflected in the following aspects: it can provide appropriate guiding opinions based on the clothes the user owns; it can complete 3D character modeling and display outfits based on the user's personal portrait, such as portrait, body shape, makeup style, weather conditions, travel occasions and other information; for outfit suggestions that the user is not satisfied with, it can make local modifications through multiple rounds of interactions to ultimately achieve user satisfaction; it will simultaneously utilize LLM's own professional outfit capabilities, without relying on the local vector database, to independently generate an outfit guidance suggestion, and display various outfits to users in the form of clickable products, providing a reference for later user shopping.

[0181] Based on the same inventive concept, the embodiment of the present application also provides a method for recommending outfits applied to the above-mentioned display device. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above-mentioned display device embodiment, so the specific limitations in one or more embodiments of the method for recommending outfits provided below can refer to the limitations on the display device above, and will not be repeated here.

[0182] In some embodiments, the present application also provides a method for recommending outfits, which is applied to the above-mentioned display device. Figure 7 As shown, the outfit recommendation method includes the following steps:

[0183] Step S701, in response to a start operation triggered by a user on the smart dressing application, displaying a user interface of the smart dressing application;

[0184] Step S702, collecting a user image of the user;

[0185] Step S703, constructing dressing tips for the user based on the user image;

[0186] The outfit prompt information includes intention information representing the user's intention and item information of multiple candidate clothes matching the user's intention; the multiple candidate clothes are part of the clothes owned by the user;

[0187] Step S704, presenting the dressing plan determined based on the dressing prompt information on the user interface;

[0188] Among them, the dressing plan is presented through the effect of wearing the target clothing; the target clothing is selected from multiple candidate clothing by the dressing model according to the intention information.

[0189] In some embodiments, step S703 includes: constructing intention information representing the user's intention based on the user image; screening out multiple candidate clothes that match the user's intention from the clothes owned by the user; splicing the intention information and the item information of each candidate clothing to obtain dressing prompt information for the user.

[0190] In some embodiments, the start operation is triggered by user voice. In the case of this embodiment, the outfit recommendation method also includes: obtaining user voice. Based on the user image, constructing intention information representing the user's intention, including: based on the user identity represented by the user image, querying the user's character portrait information from the character portrait database; determining the outfit reference information including the user voice, user image and character portrait information; using the pre-trained large language model to perform multimodal intention understanding on the outfit reference information, and constructing the intention information representing the user's intention.

[0191] In some embodiments, the outfit recommendation method further includes: outputting outfit scene query information; obtaining a response voice of the user in response to the outfit scene query information. In the case of this embodiment, determining the outfit reference information including the user voice, user image, and character portrait information includes: identifying the outfit scene information expressed by the response voice; determining the outfit reference information including the user voice, user image, character portrait information, and outfit scene information.

[0192] In some embodiments, multiple candidate clothes that match the user's intention are screened out from the clothes owned by the user, including: determining the clothing type and item features of each of the multiple clothes owned by the user, and the intention features of the user's intention; for each clothing type, based on the feature similarity between each item feature corresponding to the clothing type and the intention feature, candidate clothes are screened out from each clothing belonging to the clothing type.

[0193] In some embodiments, determining the item features of each of multiple clothes owned by a user includes: obtaining the item information of each of the multiple clothes; for each piece of clothing, labeling the clothing with features according to the item information of the clothing, and determining the parameter values ​​of the clothing corresponding to different dressing parameters; mapping each parameter value to a feature space containing each dressing parameter, and obtaining the item features of the clothing.

[0194] In some embodiments, based on the feature similarities between the features of each item corresponding to the clothing type and the intended features, candidate clothing is screened from the clothing items belonging to the clothing type, including: based on the feature similarities between the features of each item corresponding to the clothing type and the intended features, from the clothing items belonging to the clothing type, candidate clothing whose feature similarities satisfy a similarity condition; using a reranking model to respectively determine the fitness of each candidate clothing item corresponding to the user's intent; and from the candidate clothing items, screening candidate clothing whose fitness satisfies the fitness condition.

[0195] In some embodiments, the intention information and the item information of each candidate clothing are spliced ​​to obtain dressing prompt information for the user, including: sorting the item information of each candidate clothing according to the clothing type and fitness of each candidate clothing to obtain item reference information; the order of the item information in the item reference information is used to characterize the priority of the candidate clothing corresponding to the item information; splicing the intention information and the item reference information to obtain dressing prompt information for the user.

[0196] In some embodiments, step S704 includes: displaying a virtual character image on a user interface; generating the virtual character image based on the user's character portrait information; and presenting a dressing plan determined based on the dressing prompt information and description information for the dressing plan corresponding to the virtual character image.

[0197] In some embodiments, the outfit recommendation method also includes: obtaining feedback information from users regarding the outfit plan; when the feedback information indicates that the user is dissatisfied with the outfit plan, obtaining supplementary information input by the user; adjusting the outfit plan based on the supplementary information, and obtaining and presenting an updated outfit plan on the user interface.

[0198] In some embodiments, the dressing recommendation method also includes: inputting intention information into a dressing model, so that the dressing model outputs a dressing reference plan that matches the user's intention under the guidance of the intention information; the dressing reference plan is presented through the wearing effect of online clothing; online clothing refers to clothing products that the user does not currently own and are sold through Internet channels; corresponding to the virtual character image, the dressing reference plan and the clothing information of each online clothing are presented.

[0199] In some embodiments, the training process of the dressing model includes: obtaining an open source big model and sample information of each of multiple clothing samples; using each sample information to perform incremental pre-training on the open source big model to obtain a clothing recognition model with clothing recognition capabilities; using dressing samples carrying dressing professionalism labels to perform incremental training on the clothing recognition model to obtain an initial dressing model with dressing field knowledge; performing instruction fine-tuning training on the initial dressing model to obtain a dressing model.

[0200] Based on the same inventive concept, the embodiment of the present application also provides a dressing recommendation device for implementing the dressing recommendation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more dressing recommendation device embodiments provided below can refer to the limitations of the dressing recommendation method above, and will not be repeated here.

[0201] In some embodiments, the present application further provides a dressing recommendation device, which is applied to the above-mentioned display device. In this embodiment, the dressing recommendation device includes:

[0202] A startup module, configured to display a user interface of the smart dressing application in response to a startup operation triggered by a user on the smart dressing application;

[0203] An image acquisition module, used to acquire a user image of a user;

[0204] A prompt information building module is used to build dressing prompt information for users based on user images;

[0205] The outfit prompt information includes intention information representing the user's intention and item information of multiple candidate clothes matching the user's intention; the multiple candidate clothes are part of the clothes owned by the user;

[0206] A dressing plan presentation module, used to present a dressing plan determined based on the dressing prompt information on a user interface;

[0207] Among them, the dressing plan is presented through the effect of wearing the target clothing; the target clothing is selected from multiple candidate clothing by the dressing model according to the intention information.

[0208] In some embodiments, the prompt information construction module includes: an intention information construction component, which is used to construct intention information representing the user's intention based on the user image; a candidate clothing screening component, which is used to screen out multiple candidate clothing that matches the user's intention from the clothing owned by the user; and a splicing component, which is used to splice the intention information and the item information of each candidate clothing to obtain dressing prompt information for the user.

[0209] In some embodiments, the outfit recommendation device further includes: a voice acquisition module for acquiring user voice; the start operation is triggered by the user voice. In the case of this embodiment, the intention information construction component includes: a character portrait information query unit for querying the user's character portrait information from the character portrait database based on the user identity represented by the user image; an outfit reference information determination unit for determining outfit reference information including user voice, user image and character portrait information; a clothing information construction unit for using a pre-trained large language model to perform multimodal intent understanding on the outfit reference information and construct intent information representing the user's intention.

[0210] In some embodiments, the outfit recommendation device further includes: an audio output module for outputting outfit scene inquiry information. In the case of this embodiment, the voice acquisition module is also used to acquire the user's response voice to the outfit scene inquiry information feedback; the outfit reference information determination unit is specifically used to: identify the outfit scene information expressed by the response voice; and determine the outfit reference information including the user voice, user image, character portrait information, and outfit scene information.

[0211] In some embodiments, the candidate clothing screening component includes: a feature extraction unit, used to determine the clothing type and item features of each of the multiple clothing items owned by the user, as well as the intention features of the user's intention; a candidate clothing screening unit, used to screen candidate clothing from the clothing items belonging to the clothing type based on the feature similarity between the item features corresponding to the clothing type and the intention features.

[0212] In some embodiments, the feature extraction unit is specifically used to: obtain item information of each of a plurality of clothing items; for each clothing item, perform feature labeling on the clothing item according to the item information of the clothing item, and determine the parameter values ​​of the clothing item corresponding to different dressing parameters; map each parameter value to a feature space containing each dressing parameter, and obtain the item features of the clothing item.

[0213] In some embodiments, the candidate clothing screening unit is specifically used to: screen candidate clothing whose feature similarities meet similarity conditions from each clothing belonging to the clothing type according to the feature similarities between each item feature corresponding to the clothing type and the intended feature; use the re-ranking model to determine the fitness of each candidate clothing corresponding to the user's intention; and screen candidate clothing whose fitness meets the fitness conditions from each candidate clothing.

[0214] In some embodiments, the splicing component is specifically used to: sort the item information of each candidate clothing according to the clothing type and suitability of each candidate clothing to obtain item reference information; the order of the item information in the item reference information is used to characterize the priority of the candidate clothing corresponding to the item information; splicing the intention information and the item reference information to obtain dressing prompt information for the user.

[0215] In some embodiments, the dressing plan presentation module is specifically used to: display a virtual character image on a user interface; generate a virtual character image based on the user's character portrait information; and present a dressing plan determined based on the dressing prompt information and descriptive information for the dressing plan corresponding to the virtual character image.

[0216] In some embodiments, the dressing plan presentation module is also used to: obtain user feedback information on the dressing plan; when the feedback information indicates that the user is dissatisfied with the dressing plan, obtain supplementary information input by the user; adjust the dressing plan according to the supplementary information, and obtain and present an updated dressing plan on the user interface.

[0217] In some embodiments, the dressing plan presentation module is also used to: input intention information into the dressing model, so that the dressing model outputs a dressing reference plan that matches the user's intention under the guidance of the intention information; the dressing reference plan is presented through the wearing effect of online clothing; online clothing refers to clothing products that the user does not currently own and are sold through Internet channels; corresponding to the virtual character image, the dressing reference plan and the clothing information of each online clothing are presented.

[0218] In some embodiments, the outfit recommendation device also includes a model training module, which is used to: obtain an open source large model and sample information of each of multiple clothing samples; use the sample information to incrementally pre-train the open source large model to obtain a clothing recognition model with clothing recognition capabilities; use outfit samples carrying outfit professionalism labels to incrementally train the clothing recognition model to obtain an initial outfit model with outfit field knowledge; perform instruction fine-tuning training on the initial outfit model to obtain an outfit model.

[0219] Each module in the above-mentioned outfit recommendation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0220] In some embodiments, a display device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method steps are implemented when the processor executes the computer program.

[0221] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0222] In some embodiments, a computer program product is provided, comprising a computer program, which implements the above method steps when executed by a processor.

[0223] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0224] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0225] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: The device comprises: A display configured to display content from a broadcast system or network and / or a user interface, and, in response to a start operation triggered by a user for a smart dressing application, display a user interface of the smart dressing application; An image collector is configured to collect a user image of the user; The controller is configured as: Based on the user image, construct outfit prompt information for the user; the outfit prompt information includes intention information representing the user's intention and item information of each of a plurality of candidate clothes matching the user's intention; the plurality of candidate clothes are part of the clothes owned by the user; In the user interface, a dressing plan determined based on the dressing prompt information is presented; the dressing plan is presented through the effect of the target clothing on the body; the target clothing is screened from the multiple candidate clothing by the dressing model according to the intention information.

2. The device according to claim 1, characterized in that When the controller executes the step of constructing dressing prompt information for the user based on the user image, the controller is further configured to: Based on the user image, constructing intention information representing the user's intention; Filtering a plurality of candidate clothes that match the user's intention from the clothes owned by the user; The intention information and the item information of each of the candidate clothing items are spliced ​​together to obtain dressing prompt information for the user.

3. The device according to claim 2, characterized in that The device also includes: A sound collector is configured to obtain user voice; the start operation is triggered by the user voice; When the controller executes the step of constructing intention information representing the user's intention based on the user image, the controller is further configured to: Based on the user identity represented by the user image, querying the character portrait information of the user from a character portrait database; Determine dressing reference information including the user voice, the user image and the character portrait information; A pre-trained large language model is used to perform multimodal intent understanding on the dressing reference information and construct intent information that represents the user's intent.

4. The device according to claim 3, characterized in that The device also includes: an audio output device configured to output dressing scene inquiry information; The sound collector is further configured to obtain the user's response voice for the information feedback of the dressing scene; When the controller executes the step of determining the dressing reference information including the user voice, the user image and the character portrait information, the controller is further configured to: Recognize the dressing scene information expressed by the response voice; Determine dressing reference information including the user voice, the user image, the character portrait information and the dressing scene information.

5. The device according to claim 2, characterized in that When the controller performs the step of screening out a plurality of candidate clothes matching the user's intention from the clothes owned by the user, the controller is further configured to: Determining clothing types and item features of each of a plurality of clothing items owned by the user, and an intention feature of the user's intention; For each of the clothing types, candidate clothing is screened from clothing belonging to the clothing type based on feature similarities between the features of each item corresponding to the clothing type and the intended features.

6. The device according to claim 5, characterized in that When the controller executes the step of determining the item characteristics of each of the plurality of clothes owned by the user, the controller is further configured to: Get item information of multiple clothes respectively; For each of the clothing items, feature-label the clothing items according to the item information of the clothing items, and determine parameter values ​​of the clothing items corresponding to different dressing parameters; Each of the parameter values ​​is mapped to a feature space containing each of the dressing parameters to obtain the item features of the clothing.

7. The device according to claim 5, characterized in that When the controller performs the step of screening candidate clothing from clothing belonging to the clothing type based on the feature similarities between the features of each item corresponding to the clothing type and the intended feature, the controller is further configured to: According to the feature similarities between the features of each item corresponding to the clothing type and the intended feature, select the candidate clothing whose feature similarities meet the similarity condition from the clothing belonging to the clothing type; Using a re-ranking model, determining the degree of suitability of each of the candidate clothing items corresponding to the user's intention; From the candidate clothes, select the candidate clothes whose fit degree meets the fit condition.

8. The device according to claim 7, characterized in that When the controller executes the step of combining the intention information and the item information of each of the candidate clothes to obtain the dressing prompt information for the user, the controller is further configured as follows: According to the clothing types and fits of the candidate clothing, the item information of each candidate clothing is sorted to obtain item reference information; the order of the item information in the item reference information is used to represent the priority of the candidate clothing corresponding to the item information; The intention information and the item reference information are combined to obtain dressing prompt information for the user.

9. The device according to any one of claims 1 to 8, characterized in that When the controller executes the step of presenting, on the user interface, a dressing scheme determined based on the dressing prompt information, the controller is further configured to: Displaying a virtual character image on the user interface; the virtual character image is generated according to the character portrait information of the user; Corresponding to the virtual character image, a dressing plan determined based on the dressing prompt information and description information for the dressing plan are presented.

10. The device according to claim 9, characterized in that The controller is also configured to: Obtaining feedback information from the user regarding the dressing plan; When the feedback information indicates that the user is dissatisfied with the dressing plan, obtaining supplementary information input by the user; The dressing plan is adjusted according to the supplementary information to obtain and present an updated dressing plan on the user interface.

11. The device according to claim 9, characterized in that The controller is also configured to: Inputting the intention information into the dressing model, so that the dressing model outputs a dressing reference plan that matches the user's intention under the guidance of the intention information; The dressing reference plan is presented through the wearing effect of online clothing; the online clothing refers to clothing products that the user does not currently own and are sold through Internet channels; Corresponding to the virtual character image, the dressing reference plan and the clothing information of each of the online clothing items are presented.

12. The device according to any one of claims 1 to 8, characterized in that The training process of the dressing model includes: Obtain the open source large model and sample information of multiple clothing samples; Using each of the sample information to incrementally pre-train the open source large model to obtain a clothing recognition model with clothing recognition capability; Using outfit samples with outfit professionalism labels, incremental training is performed on the clothing recognition model to obtain an initial outfit model with outfit domain knowledge; The initial dressing model is fine-tuned and trained according to instructions to obtain a dressing model.

13. A method for recommending outfits, characterized in that: Applied to the display device according to any one of claims 1 to 12, the method comprises: In response to a start operation triggered by a user on a smart dressing application, displaying a user interface of the smart dressing application; collecting a user image of the user; Based on the user image, construct outfit prompt information for the user; the outfit prompt information includes intention information representing the user's intention and item information of each of a plurality of candidate clothes matching the user's intention; the plurality of candidate clothes are part of the clothes owned by the user; In the user interface, a dressing plan determined based on the dressing prompt information is presented; the dressing plan is presented through the effect of the target clothing on the body; the target clothing is screened from the multiple candidate clothing by the dressing model according to the intention information.

Images