Display device, virtual fitting system and method

Abstract

The application provides a display device, a virtual fitting system and a method. The method can collect user image data in real time in use, and send the user image data to a server to generate a human body model. Clothing material is added to the human body model to synthesize and render a model, and the action of a person is extracted in real time, so that the model posture of the rendered model is adjusted according to the action parameters, and a fitting picture is formed. The method can realize a dynamic 3D virtual fitting function, and can display the action of the user in real time through the rendered model, so that the clothes follow the action of the user, and the use effect of dressing recommendation is achieved, and the problem that the traditional virtual fitting method cannot display the fitting effect in real time is solved.

Description

Technical Field

[0001] This application relates to the field of intelligent display device technology, and in particular to a display device, virtual fitting system and method. Background Technology

[0002] A virtual try-on system is a service platform integrating hardware and software. It uses technologies such as Augmented Reality (AR), Artificial Intelligence (AI), and 3D vision to construct virtual models and generate display images based on these models. Virtual try-on systems provide a 360-degree natural fit and responsive user experience, allowing for wide application in online shopping, everyday dressing, and other life scenarios.

[0003] Virtual try-on systems can be built into smart devices such as smart terminals and display devices. Users can control the smart device to run the virtual try-on system application. By inputting image data such as photos, the virtual try-on system can overlay a static image of the user's face or offline animation onto a pre-built body model through image compositing. By changing the clothing on the body model, a composite image is created and displayed through the smart device, achieving the purpose of virtual try-on.

[0004] However, this virtual try-on method, which uses image synthesis to create the effect image, is only suitable for static image display. It cannot display the try-on effect in real time, nor can it display the effect under various wearing postures and viewing angles. As a result, the final effect presented by this virtual try-on system is poor, the try-on accuracy is low, and the user experience is reduced. Summary of the Invention

[0005] This application provides a display device, a virtual fitting system, and a method to solve the problem that traditional virtual fitting methods cannot display the fitting effect in real time.

[0006] In a first aspect, this application provides a display device, including: a display, a camera, a communicator, and a controller. The display is configured to display a user interface; the camera is configured to acquire user image data in real time; the communicator is configured to establish a communication connection with a server, the server having a built-in model reconstruction application for generating a human body model based on the user image data; the controller is configured to execute the following program steps:

[0007] Obtain the user image data;

[0008] The user image data is sent to the server to trigger the server to generate and feed back a human body model based on the user image data;

[0009] Add clothing materials to the human body model to composite and render the model;

[0010] Motion parameters are extracted from the user image data, and the model pose of the rendering model is adjusted according to the motion parameters to render the fitting screen.

[0011] Secondly, this application also provides a virtual fitting system, including: a display device, an image acquisition device, and a server; wherein the image acquisition device is connected to the display device, and the display device establishes a communication connection with the server;

[0012] The image acquisition device is configured to acquire user images in real time and perform image signal processing on the user images to generate user image data; the image acquisition device is also configured to send the user image data to the display device.

[0013] The display device is configured to acquire the user image data and send the user image data to the server;

[0014] The server has a built-in model reconstruction application, and is configured to receive the user image data and run the model reconstruction application; generate a human body model based on the user image data, and the server is also configured to feed the human body model back to the display device;

[0015] The display device is also configured to add clothing materials to the human body model to synthesize a rendering model; and to extract motion parameters from the user image data and adjust the model posture of the rendering model according to the motion parameters to render a fitting scene.

[0016] Thirdly, this application also provides a virtual try-on method applied to a virtual try-on system, the virtual try-on system comprising: a display device, an image acquisition device, and a server; wherein, the image acquisition device is connected to the display device, and the display device establishes a communication connection with the server; the virtual try-on method includes the following steps:

[0017] The image acquisition device acquires user images in real time and performs image signal processing on the user images to generate user image data;

[0018] The display device acquires the user image data and sends the user image data to the server;

[0019] The server receives the user image data and generates a human body model based on the user image data;

[0020] The display device adds clothing materials to the human body model to synthesize and render the model;

[0021] The display device extracts motion parameters from the user image data and adjusts the model posture of the rendering model according to the motion parameters to render the fitting screen.

[0022] As can be seen from the above technical solutions, the display device, virtual try-on system, and method provided in this application can collect user image data in real time during use and send the user image data to the server to generate a human body model. Clothing materials are then added to the human body model to synthesize a rendered model, and the user's movements are extracted in real time. The model's posture is then adjusted according to the movement parameters to form a try-on screen. This method can achieve dynamic 3D virtual try-on functionality and display user movements in real time through the rendered model, achieving the effect of clothing moving with the user and providing outfit recommendations, thus solving the problem that traditional virtual try-on methods cannot display the try-on effect in real time. Attached Figure Description

[0023] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a diagram illustrating the use case of the virtual try-on system in the embodiments of this application;

[0025] Figure 2 This is a hardware configuration diagram of the display device in the embodiments of this application;

[0026] Figure 3 This is a schematic diagram of the structure of a home smart wardrobe in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the display device structure with a built-in camera in an embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the display device structure of the external image acquisition device in the embodiments of this application;

[0029] Figure 6a This is a schematic diagram of the virtual try-on interface in an embodiment of this application;

[0030] Figure 6b This is a diagram illustrating the display effect of clothing option categories in an embodiment of this application.

[0031] Figure 6c This is a schematic diagram of the clothing identification interface in an embodiment of this application;

[0032] Figure 6d This is a schematic diagram of the interface for selecting clothing color in an embodiment of this application;

[0033] Figure 6e This is a schematic diagram showing the effect of the purchase link in the embodiments of this application;

[0034] Figure 6f This is a schematic diagram showing the purchase interface in an embodiment of this application;

[0035] Figure 6g This is a schematic diagram illustrating the effect of the smart fitting mirror displaying the sales location of goods in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of the virtual try-on method in the embodiments of this application;

[0037] Figure 8 This is a software configuration diagram of the server in an embodiment of this application;

[0038] Figure 9 This is a schematic diagram of a fitting application interface in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of the keyframe-based action-driven process in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of the matching and association of clothing in an embodiment of this application;

[0041] Figure 12 This is a schematic diagram of the expression matching process in an embodiment of this application;

[0042] Figure 13 This is a timing diagram of data interaction in the virtual try-on system of this application embodiment. Detailed Implementation

[0043] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0044] In this application embodiment, the virtual fitting system is a service platform that integrates hardware and software. It can construct virtual models through technologies such as augmented reality (AR), artificial intelligence (AI), and 3D vision, and generate display screens based on the virtual models.

[0045] Figure 1 This is a usage scenario diagram of the virtual try-on system in the embodiments of this application. For example... Figure 1As shown, the virtual try-on system provided in this application may include a control device 100, a display device 200, a smart terminal 300, and a server 400. The virtual try-on system can achieve virtual try-on functionality through the collaborative work of multiple devices. The control device 100 and the smart terminal 300 can be used for user interaction, and can input control commands to the display device 200 based on the virtual try-on user interface provided by the display device 200. Both the display device 200 and the server 400 have data processing capabilities. The display device 200 is deployed locally, and the server 400 is deployed in the cloud; the display device 200 and the server 400 can interact with each other.

[0046] It should be noted that the display device 200 generally refers to devices with data processing and image display capabilities, including but not limited to smart TVs, personal computers, display terminals, smart advertising screens, cube screens, virtual reality devices, augmented reality devices, smart wearable devices, robot assistants, smart fitting mirrors in shopping malls, and smart wardrobes in homes. In some embodiments, such as Figure 2 As shown, the display device 200 includes one or more combinations of functional modules such as a power supply 210, a communicator 220, a memory 230, an interface module 240, a controller 250, and a display 260.

[0047] The power supply 210 provides power to the display device 200, enabling all functional modules to operate. The communicator 220 establishes a communication connection between the display device 200 and the server 400. The memory 230 stores various information and application data. The interface module 240 connects the display device 200 to peripherals, enabling the input or output of specific types of signals. The controller 250 controls the operation of the display device 200 and responds to user operations by running various software control programs stored in the memory 230.

[0048] The display 260 is used to present a user interface, enabling the display device 200 to display images. The display device 200 can display specific images on the display 260 by running the display process application program stored in the memory 230. Examples include playback interfaces, user interfaces, and application interfaces.

[0049] For different types of display devices 200, the monitor 260 can take different forms and have different display ratios. For display devices 200 such as smart TVs, personal computers, and display terminals, the monitor 260 can be a shape corresponding to the standard display ratio. For example, the display resolution of a smart TV is 3840×2160; the monitor resolution of a personal computer is 1920×1080; and the display resolution of a display terminal is 2400×1080. However, for display devices 200 such as virtual reality devices, smart fitting mirrors in shopping malls, and smart wardrobes in the home, the monitor 260 included therein can be designed with a shape and ratio appropriate to its actual use. For example, the monitor 260 of a virtual reality device includes two square screens on the left and right sides; such as... Figure 3 As shown, the display 260 of the smart wardrobe is a long strip display with a width and height less than or equal to the width and height of the wardrobe door.

[0050] Since the virtual fitting system can composite images based on the user's portrait to generate a fitting effect image, the display device 200 should be able to acquire the user's portrait image in order to perform image compositing. In some embodiments, the display device 200 can acquire the portrait image through a built-in image acquisition module 270; that is, in addition to the various functional modules mentioned above, the display device 200 also includes an image acquisition module 270. For example, as... Figure 4 As shown, the image acquisition module is a camera installed at the top or bottom of the display device 200.

[0051] In some embodiments, the display device 200 can acquire a human image through an external image acquisition device 500. That is, Figure 5 As shown, the display device 200 can be connected to the image acquisition device 500 via the interface module 240. The image acquisition device 500 has a built-in camera and transmission circuit, which can capture images of the user through the camera and then send the captured images or videos to the display device 200 for display via the transmission circuit and the interface module 240.

[0052] In a virtual fitting system, the display device 200 serves as the user's direct interaction device. It can receive the user's control commands and perform data processing according to the commands to form a user interface containing different content, which is then presented through the display 260. In some embodiments, the display device 200 can be used as a dedicated device for the virtual fitting system, meaning it is only used to run the virtual fitting program and present the virtual fitting interface. For example, the display device 200 can be applied to a robot assistant in a shopping mall environment, allowing users to achieve virtual fitting functionality through voice interaction with the robot assistant.

[0053] In some embodiments, the display device 200 can also be an implementation device of a virtual try-on system, meaning that the display device 200 has numerous functions, and the virtual try-on function is one of them. For example, the display device 200 can be a personal computer, and a user can install a virtual try-on application on the personal computer to enable the virtual try-on function.

[0054] In some embodiments, users can install various applications on the display device 200 to achieve specific functions. The installed applications can be system applications or third-party applications. For example, a user can download and install a "virtual try-on" application from an application store provided by the display device 200 operator. The "virtual try-on" application may have pre-built clothing materials. The user can then control the display device 200 to run the application and input their image. The display device 200 can then composite the user's input image with the pre-built clothing materials and display the composite effect, achieving the purpose of trying on clothes.

[0055] Virtual try-on applications can be standalone applications or integrated into a specific application. For example, in shopping applications, to allow users to see how clothing they want to buy will look on them while shopping online, users can activate the "virtual try-on" function. As with standalone applications, after enabling the "virtual try-on" function, the shopping application can guide the user to input their image through a prompt interface and call up the model of the clothing to be purchased. The "virtual try-on" function then composites the user image with the model to output a virtual try-on effect.

[0056] In implementing the virtual try-on function, the display device 200 can perform image synthesis processing based on the acquired user portrait image. In some embodiments, the display device 200 combines the portrait image with clothing images by adding virtual clothing patterns to achieve virtual try-on. For example, after acquiring the user's portrait image, the display device 200 can perform feature recognition on the portrait image to identify wearable locations of the portrait pattern, including upper limbs, lower limbs, hands, feet, neck, and top of the head. Then, virtual clothing materials are extracted from a clothing material library, and corresponding clothing materials are added to each wearable location to complete the virtual try-on.

[0057] In some embodiments, the display device 200 can also composite the image by adding a human portrait image based on the virtual clothing pattern. For example, a virtual try-on application can add a human portrait display area to the head area corresponding to the virtual clothing pattern, and display the head image from the acquired human portrait image in the display area, thereby compositing a virtual try-on image.

[0058] The virtual clothing images can be obtained from clothing materials stored in the memory 230 or the cloud server 400. For example, the operator of the virtual fitting system can generate virtual clothing materials by taking multi-angle images and performing 3D modeling based on currently popular clothing items. The generated virtual clothing materials can be stored in the server 400 of the virtual fitting system. When the display device 200 activates the virtual fitting application and selects clothing to be tried on, the display device 200 can request virtual clothing materials from the server 400 according to the user's selection.

[0059] After acquiring virtual clothing materials, the display device 200 can cache the requested virtual clothing materials based on the usage of its own memory 230. Therefore, when a user subsequently uses the virtual try-on system, after selecting clothing to try on, the system can first match the corresponding virtual clothing materials in the local cache of memory 230. If the local cache contains the corresponding virtual clothing materials, they can be retrieved. If the local cache does not contain the corresponding virtual clothing materials, the user can request the corresponding virtual clothing materials from the server 400 again.

[0060] In some embodiments, virtual clothing materials may include multiple independent parameters, including clothing style, color, and material. The display device 200 can present various clothing styles through the arrangement and combination of parameters across different dimensions. For example, the same virtual clothing material can be used for clothing of the same style, while adjusting parameters such as color and material during virtual try-on to obtain different clothing appearances. Therefore, the display device 200 can generate multiple clothing materials with less model data, reducing the amount of clothing model construction and data transmission when requesting clothing materials.

[0061] To create a virtual try-on effect, in some embodiments, the display device 200 can present a virtual try-on interface to the user after running a virtual try-on application. For example... Figure 6a As shown, the virtual try-on interface can include a display window, clothing options, control options, and outfit recommendations. Users can select different options on the virtual try-on interface to control the creation of a virtual try-on image. For example, the user's portrait image captured by the image acquisition device 500 can be displayed in real time in the display window. When the user selects "Clothing A" from multiple clothing options, the display device 200 can retrieve the virtual clothing material corresponding to "Clothing A" from its local cache or server 400 and display it through the display window. Similarly, when the user selects the option "Clothing B," the display device 200 can retrieve the virtual clothing material corresponding to "Clothing B" and display it through the display window.

[0062] The clothing options in the virtual try-on interface can be categorized according to their actual wearing position, such as tops, bottoms, shoes, hats, and accessories. Clothing options from different categories can be selected simultaneously, but clothing options within the same category cannot be selected multiple times. For example, ... Figure 6b As shown, after selecting "Top A" from the top category, a user can then select "Pants B" from the bottom category. The display device 200 can then simultaneously display the corresponding images of the selected top "Top A" and bottom "Pants B" in the display window. However, for tops "Top A" and "Top B" that are both tops, they cannot be selected simultaneously; instead, the later-selected clothing item will replace the first-selected clothing item.

[0063] Different categories of virtual clothing materials can use different clothing styles, therefore the display device 200 needs to call different virtual clothing materials. In some embodiments, due to the large variety of clothing types, virtual clothing materials of the same category can be further divided into more detailed categories. For example, the categories of tops can be further divided according to the wearing season into spring wear, summer wear, autumn wear, winter wear, etc.; or they can be further divided according to the wearing position into coats, shirts, etc. Similarly, for different further subdivided categories, the display device 200 also needs to call different virtual clothing materials.

[0064] When a user interacts with the display device 200 for virtual try-on, in addition to selecting clothing through the aforementioned clothing options, they can also select clothing through other interactive methods. In some embodiments, the user can input information such as the clothing item name and product number to indicate the selection of clothing to be tried on. For this type of interaction, the display device 200 can search the clothing material library based on the user information to retrieve clothing materials related to the input information.

[0065] In some embodiments, the display device 200 can also identify image-based information such as displayed images, physical images, and barcodes on tags of clothing based on image recognition technology to determine the clothing to be tried on. For example, Figure 6a , Figure 6c As shown, the virtual try-on interface may include a "Recognize Clothing" option. When the user selects the "Recognize Clothing" option, the display device 200 can automatically activate the image acquisition device 500 to acquire clothing images, recognize the clothing images, and retrieve virtual clothing materials based on the image recognition results. The basis for recognizing the clothing images can be by calculating image similarity between the captured clothing image and a preset standard image. The standard image with the highest similarity is taken as the recognition result, and the corresponding virtual clothing material is retrieved from the database.

[0066] In the virtual try-on interface, each clothing option corresponds to a type of virtual try-on material. After a user selects a clothing option, they can adjust the clothing parameters using the control options. For example... Figure 6d As shown, after selecting the "Clothing A" option, the display device 200 can display color control options in the virtual try-on interface. Users can select any color from the preset multiple color control options to control the current color of the virtual clothing screen displayed in the display window.

[0067] In addition to controlling the color of the selected clothing material, the control options can also be used to control and adjust the display screen of the display window, such as rotating the display angle, zooming in on specific areas, adjusting brightness, and using beauty filters. Users can select the corresponding options and input parameters to adjust the interactive actions and control the presentation effect of the display device 200 on the display window.

[0068] The outfit recommendation option enables or disables the outfit recommendation function of the virtual try-on application. This function automatically displays recommended outfits in the display window after the user selects any clothing item. The outfit recommendation function uses a specific style algorithm to match clothing items from the virtual clothing library that are not in the selected category, based on the user's selected clothing item. The outfit recommendation algorithm can perform a unified matching calculation based on dimensions such as color, type, and style. For example, after the user selects a black, formal top, the outfit recommendation algorithm can automatically match black, formal bottoms and shoes, and call up the corresponding virtual clothing models, displaying them together with the selected top's virtual clothing model in the display window.

[0069] In addition to the aforementioned display window, clothing options, control options, and outfit recommendation options, in some embodiments, the virtual try-on interface presented by the display device 200 may also include prompts and link options. For example, the virtual try-on interface may also include a "Save Outfit" option. After a user completes the try-on by selecting multiple clothing options, they can click the "Save Outfit" option to save the pattern currently displayed in the display window. Simultaneously, the virtual try-on application can automatically display purchase links or shopping guides for the corresponding clothing items after the user clicks "Save Outfit," allowing the user to purchase the appropriate clothing items based on the virtual try-on results.

[0070] Obviously, the prompts and link options displayed on the virtual try-on interface may differ depending on the type of display device 200. For example, as... Figure 6e , Figure 6fAs shown, when a shopping application is running on a computer, after the user selects "Save Outfit," the display device 200 can present product links corresponding to the virtual clothing images displayed in the current window for the user to choose from. After the user selects any product link, the display device 200 can jump from the virtual fitting interface to the product details interface. When using a smart fitting mirror in a shopping mall, if the user selects "Save Outfit," the virtual fitting interface can display the location of the corresponding store selling the clothing in the current window, such as "Clothing A can be purchased at 306××, Section A, 3rd Floor." Figure 6g As shown.

[0071] As can be seen, both virtual try-on and shopping applications require user image input to display the try-on effect. Since different applications present different formats of the final try-on effect, users need to input their images in different ways. Specifically, in some embodiments, to synthesize a 2D, static try-on image, the user can open a file manager when prompted to input an image, select an image file in a specific save path, and use that image file as the user image input. Alternatively, the user can take a picture using the built-in camera of the display device 200 or an external image acquisition device, and input the captured image into the application.

[0072] However, 2D static image displays cannot show the real-time effect of trying on clothes, nor can they demonstrate the effect under various wearing postures and viewing angles, resulting in a poor final effect presented by such virtual try-on applications. Therefore, in some embodiments, virtual try-on applications can also provide dynamic try-on images, allowing users to input their images by uploading video files or recording video files in real time. The input video images can be displayed in a specific area of ​​the application interface to achieve a better composite effect.

[0073] To achieve a better virtual try-on effect, in some embodiments, the display device 200 can also use augmented reality technology to add virtual clothing material models to the user-uploaded video footage. By tracking the person in the video, the clothing material model can move in sync with the person's movements. However, this dynamic try-on method is limited by AR technology, resulting in poor integration between the clothing material and the person in the video, and the clothing cannot accurately fit the person. Furthermore, there is a significant delay in the display effect of the clothing following the person's movements, thus resulting in a poor interactive experience for the virtual try-on function.

[0074] To achieve better virtual try-on effects and interactive experiences, some embodiments of this application provide a virtual try-on method. This method can be applied to a virtual try-on system to display the try-on effect in real time. The virtual try-on system includes a display device 200, a server 400, and an image acquisition device 500. The image acquisition device 500 is used to acquire user images in real time and process the images to form user image data. The server 400 can have a built-in model reconstruction application that can generate a human body model based on the user image data. The display device 200 is used to run the virtual try-on application and render the human body model to synthesize and display the virtual try-on image. Specifically, the virtual try-on method includes the following:

[0075] The image acquisition device 500 acquires user images in real time. The image acquisition device 500 may include a camera and a data processing module. The camera captures images of the user's environment to obtain user images. The data processing module performs image signal processing on the user images to generate user image data, which is then input as an image signal into the display device 200.

[0076] In some embodiments, the image acquisition device 500 may be a functional module built into the display device 200, forming an integrated device with the display device 200. For example, such as Figure 4 As shown, the image acquisition device 500 is a camera on the display device 200, which can be uniformly controlled by the controller 250 in the display device 200, and can directly send the acquired user images to the controller 250. Obviously, to facilitate the acquisition of user images, the camera should be placed at a specific location on the display device 200. For example, for a display device 200 such as a smart TV, the camera can be placed on the top of the smart TV, and the camera's shooting direction is the same as the direction of light emitted from the smart TV screen, thereby enabling it to capture images of users located in front of the screen.

[0077] In some embodiments, the image acquisition device 500 can also be an external device of the external display device 200, i.e., Figure 5As shown, the display device 200 may be equipped with an interface module 240. The interface module 240 can be one of the following: an HDMI interface, an analog or high-definition component input interface, a CVBS interface, a USB interface, and an RGB port, supporting specific data transmission methods. After connecting to the interface module 240, the image acquisition device 500 can send the acquired user image to the display device 200 through the specific data transmission method supported by that interface. For example, after connecting to a smart TV, the image acquisition device 500 can communicate with the smart TV through Open Natural Interaction (OpenNI) to send the acquired user image to the smart TV.

[0078] To achieve a better dynamic virtual try-on effect, the user image data sent from the image acquisition device 500 to the display device 200 may include not only image content but also image recognition results, skeletal parameters, facial expression parameters, gesture recognition results, and other data. Therefore, in some embodiments, the data processing module of the image acquisition device 500 may also have a built-in image processing application. After the camera acquires the user image, the data processing module can run the image processing application to recognize the user image.

[0079] For example, the image acquisition device 500 can identify "image-depth" data, 3D human skeleton key point coordinates, human head recognition position, and human target tracking points in the user image by running image processing applications with different functions. Therefore, when sending the identified content to the display device 200, it can provide the display device 200 with functional support such as RGBD image data, limb driving, human tracking, face reconstruction materials, and expression driving materials. Thus, this data can be combined with the user image to constitute user image data.

[0080] To obtain the corresponding data, the image acquisition device 500 needs specific hardware support. For example, in order to identify the image depth in an image, the image acquisition device 500 can be a camera group with multiple lenses. The multiple lenses can detect the same target from different positions, and the depth information of the target can be calculated by comparing the image capture results and angle differences of the multiple lenses.

[0081] Obviously, the image acquisition device 500 can also integrate sensor elements with specific functions to include data for 3D modeling in the user images it acquires. For example, the image acquisition device 500 can have a built-in lidar element that can emit lasers into the shooting area, detect the reflected laser signals to measure the distance between the target in the shooting area and the image acquisition device 500, and associate the scanning results with each frame of the image captured by the camera to generate user image data with image depth.

[0082] like Figure 7 As shown, after acquiring a user image and generating user image data, the image acquisition device 500 can send the user image data to the display device 200. The display device 200 then obtains the user image data as data input for the virtual try-on application. After the user initiates the virtual try-on application, the display device 200 can send a start command to the image acquisition device 500, controlling the image acquisition device 500 to begin acquiring the user image and perform processing on the user image to obtain user image data.

[0083] In some embodiments, depending on the visual effects presented by the virtual try-on application launched by the display device 200, the display device 200 may acquire user image data including different content from the image acquisition device 500. For example, when the user controls the display device 200 to launch a virtual try-on application that can only display 2D static effects, the display device 200 may set the type of user image data to include only image content in the launch command sent to the image acquisition device 500. However, when the user controls the display device 200 to launch a virtual try-on application that can display 3D dynamic effects, the launch command needs to specify that the user image data includes "image-depth" data, 3D human skeleton key point coordinates, and other data content.

[0084] Similarly, for different virtual try-on applications, the image format in the user image data acquired by the display device 200 can also be different. For example, for virtual try-on applications with static effects, user image data in the form of an image can be acquired from the image acquisition device 500; while for virtual try-on applications with dynamic effects, user image data in the form of a video can be acquired from the image acquisition device 500.

[0085] After acquiring user image data, display device 200 can render a fitting result screen based on the user image data. To present a better clothing blending effect, display device 200 can obtain the fitting screen based on a human body model. Therefore, after acquiring user image data, display device 200 can create a human body model based on the user image data. However, due to the huge amount of data processing involved in creating the human body model and the limitations of the display device 200's hardware configuration, the human body model created locally by display device 200 is of coarse quality and has poor accuracy. Therefore, in some embodiments, display device 200 can send the user image data to server 400 for modeling processing after acquiring the user image data.

[0086] Therefore, a model reconstruction application can be built into the server 400. This application can generate human body models based on user image data. The server 400 can be pre-configured with multiple initial human body models, which can be set according to factors such as age and gender. When the display device 200 sends user image data to the server 400, the server 400 can first identify the images in the user image data to determine the age, gender, and other information of the person in the image, and then, based on this information, call upon the appropriate initial human body model from among the multiple preset models.

[0087] After calling the initial human body model, the server 400 can also read "image-depth" data, 3D human skeleton key point coordinates, and other data from the user image data. Based on this data, the server 400 can set, modify, and adjust the initial human body model, gradually imbuing it with the human facial features found in the user image data. For example, the server 400 can extract the ratio between the user's head width and shoulder width from the user image data and adjust the head and shoulder widths in the initial model accordingly.

[0088] In some embodiments, such as Figure 8 As shown, the human body model created by server 400 can include multiple parts, each of which can be individually configured and adjusted. Furthermore, it supports replacement using a pre-set material library to quickly generate the human body model. For example, server 400 can pre-store various hairstyle materials. After receiving user image data, server 400 can match similar models from the hairstyle material library based on the hairstyle and color in the user image data, and add the hair model to the head of the human body model as the initial hairstyle.

[0089] In subsequent use, when using virtual try-on applications, users can choose to change the hairstyle of the person in the virtual try-on effect screen. After the user selects to change to a specific hairstyle, the display device 200 can request the server 400 to rebuild the model. The server 400 can then select a new hairstyle material from the hairstyle material library and add it to the human body model to rebuild the human body model.

[0090] To create more detailed human body models, in some embodiments, multiple modeling modules can be built into the server 400, including a head reconstruction module, a body reconstruction module, an expression recognition module, and a trial firing module. Each modeling module can construct a corresponding virtual model through a specific model reconstruction method. For example, head modeling can further include geometric reconstruction and texture reconstruction units. The geometric reconstruction unit can perform point cloud denoising, triangulation, point cloud smoothing, and point cloud fusion processing based on the user image data to gradually make the shape of the head model consistent with the human target in the user image data. The texture reconstruction unit can perform human face segmentation, skin color transfer, and skin color fusion processing on the head model to gradually make the appearance of the head model consistent with the human target in the user image data.

[0091] Similarly, the body reconstruction module can also generate body models based on specific modeling methods. For example, the body reconstruction module can create a body model using a Skinned Multi-Person Linear Model (SMPL). Therefore, the body reconstruction module can include an SMPL model generation unit, a parametric mapping unit, and a stitching unit. When reconstructing the body model, the server 400 can generate an initial model using the SMPL model generation unit, then extract human body parameters from the user image data to map the SMPL model to the parametric model, obtaining a parametric body model that conforms to the user image data. Finally, the stitching unit stitches the body model with the head model to obtain the human body model.

[0092] In some embodiments, to obtain a human body model, the display device 200 may, while sending user image data to the server 400, identify a human figure from the user image data and add skeletal key points to the human figure based on the identification results to generate skeletal parameters. The skeletal parameters are then sent to the server 400 so that the server 400 can set the joint positions of the human body model according to the skeletal parameters. The human body model with added skeletal parameters can change according to the rules corresponding to the skeletal parameters, thereby simulating a model posture that more closely resembles a realistic human figure.

[0093] Furthermore, the facial expression recognition module can also establish a base model for facial expressions and estimate facial parameters through its built-in processing unit, setting specific facial expressions for the human model based on the facial expressions in the user's image data. As for the hair profiling module, preset functional units can be used to set hair matching and hair penetration processing functions, creating more realistic hair models.

[0094] Since the human body model generated or reconstructed by server 400 is used for the subsequent virtual fitting process, in some embodiments, server 400 may also include an application for generating clothing models, in addition to the application for generating human body models. Server 400 can run this application to perform clothing modeling, fabric simulation, and clothing deformation processing to create clothing models based on clothing image data.

[0095] Similar to the human body model, when creating a clothing model, the server 400 can also receive clothing image data from the display device 200. This clothing image data can be obtained by taking pictures of the clothing from multiple angles. For example, six views of the clothing can be obtained from the clothing's product interface: front view, back view, left view, right view, bottom view, and top view. After sending images from different angles to the server 400, the server 400 can automatically model and generate clothing patterns based on these images. Then, through fabric simulation, garment deformation, and other rendering processes, a clothing model that matches the real-world effect is obtained.

[0096] After generating the human body model, server 400 can send the generated human body model back to display device 200. Upon receiving the human body model, display device 200 renders it to produce a fitting effect image. During the rendering process, display device 200 can add clothing materials to the human body model to composite the rendered model.

[0097] During the compositing and rendering process, the display device 200 can also optimize the human body model fed back by the server 400. For example, the display device 200 can perform head compositing and restoration processing on the human body model based on the Unity engine, including head shape processing using the meshfilter and meshrender tools; texture processing using the texture and normalmap tools; and processing of facial expression parameters and adjusting hair to fit the head shape using the blendshape / morph tool's head parametric model algorithm.

[0098] In some embodiments, to facilitate user comparison of fitting room effects, the display device 200 can display the image portion of the user image data while simultaneously sending the user image data to the server 400. For example, ... Figure 9 As shown, the interface of a virtual fitting application can include a virtual fitting window and an original window. The virtual fitting window can display the rendering results of the human body model and clothing materials, while the original window displays the image portion of the user's image data, namely the image captured in real time by the image acquisition device 500.

[0099] After synthesizing the rendered model, the display device 200 can extract motion parameters from the user's image data and adjust the model's posture according to the motion parameters to render the fitting room image. That is, the display device 200 can determine the user's actions based on the user's image data and control the rendered model to follow the user's actions corresponding to the image data. The actions that can be presented in the fitting room image include body movements, head movements, facial expressions, gestures, and clothing movements. The motion parameters can be calculated by the image acquisition device 500 through head detection, 3D human skeleton key point detection, facial expression detection, and gesture recognition detection.

[0100] The display device 200 can detect user actions by comparing multiple frames of images in the user image data. Specifically, in some embodiments, during the step of adjusting the model pose of the rendered model according to motion parameters, the display device 200 can traverse the skeletal keypoints of each frame in the user image data; then compare the positions of the skeletal keypoints in adjacent frames to obtain the movement distance of each skeletal keypoint, and move the joint positions of the human model according to the movement distance. Through the coordinated adjustment of multiple skeletal keypoints, the display device 200 can adjust the pose of the character in the rendered model to match the same actions as the content of each frame in the user image data, achieving a motion-following effect.

[0101] To achieve motion-following effects, the display device 200 can use skinning, i.e., adding bones to the model, to enable the human body model to behave in a reasonable manner. For example, after acquiring the human body model, the display device 200 can skin the human body model, clothing model, and provide a texture for the cloth simulation algorithm, and use the blendshape / morph tool to process the skeletal skinning animation of the model.

[0102] As can be seen, in the above embodiments, the virtual try-on method can acquire user image data in real time through the image acquisition device 500, generate a human body model based on the user image data through the server 400, and add clothing materials to the human body model on the display device 200 to form a rendered model. Simultaneously, the display device 200 can also obtain user actions based on the user image data and adjust the model posture of the rendered model in real time according to the user actions, realizing virtual avatar driving. The method can achieve a 3D, dynamic, real-time, and multi-angle display of the try-on scene through the human body model. Furthermore, by having the server 400 perform model creation and reconstruction, it can reduce the data processing load of the display device 200, which is beneficial for generating more detailed and realistic character models and improving the user experience.

[0103] Since the display device 200 is deployed locally and the server 400 is deployed in the cloud, data communication between the display device 200 and the server 400 is affected by factors such as network latency. This can cause a delay in receiving the human body model after the user's image data is sent to the server 400. This prolongs the waiting time for displaying the fitting effect. Therefore, during the presentation of the final fitting image, the number of data interactions between the display device 200 and the server 400 should be minimized to reduce latency. In some embodiments, the display device 200 can monitor the human figure after initially obtaining it, thus using the initially obtained human body model unless significant changes in features occur. That is, as... Figure 10 As shown, after sending user image data to server 400, display device 200 extracts initial keyframe image from user image data and extracts image depth parameters from initial keyframe image. Then, it sends both initial keyframe and image depth parameters to server 400 so that server 400 can set human body model parameters according to image depth parameters and feed back human body model to display device 200.

[0104] After obtaining the human body model, the display device 200 can extract real-time keyframe images from the user image data in subsequent applications. A keyframe can be an image frame corresponding to a specific point in time. For example, the display device 200 can extract an image frame from the user image data every 1 second as a keyframe. A keyframe can also be an image frame obtained at intervals of a certain number of frames; for example, the display device 200 can extract an image frame from the user image data at 20-frame intervals as a keyframe. A keyframe can also be an image frame with a clearly defined human figure, obtained through image recognition of image frames. For example, by inputting user image data frame by frame into a human face recognition model, when the model determines that an image frame contains a human figure, it is marked as a keyframe.

[0105] After extracting keyframes, the display device 200 can compare the initial keyframe image and the real-time keyframe image to obtain image similarity. The initial keyframe is the image frame used to extract image depth parameters to generate the human body model. The real-time keyframe is the keyframe extracted by the display device 200 in real time from the user's image data. The image similarity can be determined by calculating the histograms of the two images separately, and then calculating the normalized correlation coefficient of the two histograms, such as Bach distance or histogram intersection distance. Since the content of the two keyframe images is the same when the user is stationary, but differs when the user is in motion, and the greater the amplitude of the motion, the lower the similarity between the two frames, the user's motion state can be detected through image similarity.

[0106] After comparing the initial keyframe and real-time keyframe images to obtain image similarity, the display device 200 can compare the image similarity with a preset similarity threshold. If the image similarity is greater than or equal to the preset similarity threshold, that is, the user's actual movement amplitude in the two keyframes is small or in a static state, there is no need to remodel. The original human body model can still be used to obtain a better fitting screen display effect. That is, the display device 200 can perform the step of extracting motion parameters from the user image data to drive the human body model to generate motion according to the motion parameters.

[0107] If the image similarity is less than the preset similarity threshold, that is, the content differences between the two keyframes are large, in order to obtain a more realistic model effect, the display device 200 can extract the image depth parameters from the real-time keyframe image and send the extracted image depth parameters to the server 400 to trigger the server 400 to reconstruct the human body model based on the image depth parameters.

[0108] As the image acquisition device 500 continuously acquires user images, the content contained in the images can gradually differ with user actions. That is, the similarity between the initial keyframe and subsequently extracted implementation keyframes gradually decreases. Therefore, to maintain the consistency of the display device 200's action-driven model and reduce the number of remodeling operations, in some embodiments, the display device 200 can extract image depth parameters from the initial keyframe image and then record the initial keyframe image. Furthermore, after performing the step of extracting action parameters from the user image data, or after sending the image depth parameters to the server, the recorded initial keyframe image is replaced with a real-time keyframe image.

[0109] For example, after a user launches a virtual try-on application, the display device 200 can first extract an initial keyframe T0 from the user's image data and send it to the server 400 to generate a human body model. Furthermore, during subsequent rendering of the human body model's try-on screen, the display device 200 can continuously acquire real-time keyframes, i.e., T1. After acquiring the real-time keyframe T1, the display device 200 can compare the initial keyframe T0 and the real-time keyframe T1 to calculate the similarity S01 between the two frames. When the similarity S01 is greater than or equal to a similarity threshold S, motion parameters are extracted from the real-time keyframe T1 or the user's image data to drive the human body model to generate actions that follow the user's movements. After extracting the motion parameters, the display device 200 can replace the initial keyframe T0 with the real-time keyframe T1 as the initial keyframe for subsequent action judgment processes. That is, when obtaining the real-time keyframe T2, the similarity between keyframe T1 and keyframe T2 can be compared to continue tracking user actions or reconstructing the human body model.

[0110] As can be seen from the above technical solutions, the above embodiments can continuously analyze the user image captured by the image acquisition device 500 through initial keyframes and real-time keyframes, thereby tracking the user's actual actions. Furthermore, the display device 200 can determine whether it is necessary to reconstruct the human body model by comparing the image similarity between two adjacent keyframes, so as to reduce the number of model reconstructions while ensuring timely synchronization of the human body model, thereby improving the response speed of virtual try-on applications.

[0111] In some embodiments, the virtual try-on application, while providing real-time dynamic try-on functionality, can also offer outfit recommendations to users, such as... Figure 11 As shown, during the process of adding clothing materials to a human body model, the display device 200 can obtain selection instructions input by the user for selecting clothing. For example, in a try-on application interface, multiple clothing options can be set, and the user can select any option on the interface through interactive operations to input selection instructions. The user can also select multiple target garments from multiple clothing options, that is, the selection instruction specifies at least one target garment. For example, the user can select both tops and bottoms simultaneously to input selection instructions.

[0112] After the user inputs a selection command, the display device 200 can respond to the selection command and extract the target clothing material from the clothing material library. Since clothing can be divided into multiple categories, such as tops, bottoms, shoes, hats, and bags, these categories can be combined to create the final dressing effect. Therefore, when the user does not select all categories in the selection command, the virtual try-on application can automatically match other suitable clothing types based on the characteristics of the selected clothing to present a better virtual try-on effect.

[0113] Therefore, while extracting the target clothing material, the display device 200 can match related clothing materials in the clothing material library according to preset outfit recommendation rules. These preset outfit recommendation rules can be comprehensively set based on categories such as color, usage, style, and applicable age. For example, when a user selects a blue and white fresh-style top, the display device 200 can recommend related clothing materials such as blue and white fresh-style bottoms and shoes according to the preset outfit recommendation rules.

[0114] After obtaining the target clothing material and related clothing materials, the display device 200 can add the target clothing material and related clothing materials to the human body model to form a rendered model with suggested outfit effects. Obviously, to recommend more types of outfit styles to users, the display device 200 can pre-set multiple outfit recommendation rules. Each rule can match suitable related clothing materials under that rule based on the target clothing material selected by the user, and preview them through multiple windows to improve the user experience.

[0115] Because of the high complexity of actual user expressions, the amount of parameter data that needs to be modified in the human body model under different expressions is enormous. This means that try-on applications need to process a large amount of data to modify facial features in the human body model during expression tracking. This slows down the application's response time. Therefore, in some embodiments, the display device 200 can quickly switch expressions using preset standard expression templates. For example... Figure 12 As shown, when adjusting the model pose of the rendering model according to the motion parameters, the display device 200 can identify the head region in the user image data, detect the user's expression in the head region, and then match the expression model with the same expression type in the preset expression library according to the expression type of the user's expression, thereby replacing the facial region in the rendering model with the expression model.

[0116] For example, the display device 200 can identify the head region in a user's image by detecting the target shape in the user's image data and the layout characteristics within that shape. Then, from the head region, an expression recognition model is used to identify the current user's expression. This expression recognition model can be obtained by training an artificial intelligence model with sample data. Specifically, a large amount of sample data with expression labels is input into the initial model, and the model output is set to the probability of an image belonging to a specific expression category. The error between the classification probability and the expression label is then calculated, and the model parameters are adjusted by backpropagation according to this error, so that the model output gradually matches the label results, thus obtaining the expression recognition model.

[0117] After the facial expression recognition model outputs the classification probability of a certain expression for the current image, the expression with the highest classification probability can be taken as the user's expression, such as a smile. Then, the recognized user expression is matched against a pre-set database with a facial expression model of the same type, namely the standard smile model, and the facial area in the rendering model is replaced with the standard smile model so that the rendering model can display a smiling expression.

[0118] As can be seen, the above embodiments can quickly replace facial areas in the rendered model by matching standard expression models, so that the display device 200 does not need to make extensive modifications to the model's facial parameters, thus reducing the amount of data processing. Furthermore, it can achieve better facial expression tracking efficiency and improve the user experience.

[0119] Based on the above virtual try-on method, some embodiments of this application also provide a display device 200 including: a display 260, a camera, a communicator 220, and a controller 250. The display 260 is configured to display a user interface; the camera is configured to acquire user image data in real time; the communicator 220 is configured to establish a communication connection with a server 400, the server 400 having a built-in model reconstruction application for generating a human body model based on the user image data; the controller 250 is configured to execute the following program steps:

[0120] Obtain the user image data;

[0121] The user image data is sent to the server to trigger the server to generate and feed back a human body model based on the user image data;

[0122] Add clothing materials to the human body model to composite and render the model;

[0123] Motion parameters are extracted from the user image data, and the model pose of the rendering model is adjusted according to the motion parameters to render the fitting screen.

[0124] like Figure 13 As shown, in some embodiments, this application also provides a virtual fitting system, including: a display device 200, an image acquisition device 500, and a server 400; wherein, the image acquisition device 500 is connected to the display device 200, and the display device 200 establishes a communication connection with the server 400.

[0125] The image acquisition device 500 is configured to acquire user images in real time and perform image signal processing on the user images to generate user image data; the image acquisition device 500 is also configured to send the user image data to the display device 200.

[0126] The display device 200 is configured to acquire the user image data and send the user image data to the server 400;

[0127] The server 400 has a built-in model reconstruction application, which is configured to receive the user image data and run the model reconstruction application; generate a human body model based on the user image data, and the server 400 is also configured to feed the human body model back to the display device 200.

[0128] The display device 200 is also configured to add clothing materials to the human body model to synthesize a rendering model; and to extract motion parameters from the user image data and adjust the model posture of the rendering model according to the motion parameters to render a fitting scene.

[0129] As can be seen, in the above embodiments, the entire virtual try-on system may include: a cloud server 400, a local display device 200, and an image acquisition device 500. The cloud server 400 can be responsible for human body reconstruction, including basic algorithm modules such as head reconstruction, body reconstruction, expression recognition, and hair try-on, providing the display device 200 with functions such as human body models, expression-driven parameters, and hair try-on. The server 400 can rely on data collected by the image acquisition device 500 as input and output the modeling processing results to the display device 200 to support applications.

[0130] The image acquisition device 500 is responsible for providing acquisition data, including ISP debugging, RGBD streams, 3D human skeleton key points, human head detection, and multi-target tracking data. This data is then transmitted via OpenNI to the display device 200, providing functional support for RGBD image data, limb-driven animation, human body tracking, face reconstruction materials, and expression-driven materials. The display device 200 is responsible for rendering the human model, displaying and rendering clothing materials, motion-driven animation, and adjusting local parameters.

[0131] As can be seen from the above technical solution, the virtual try-on system provided in this application can collect user image data in real time during use and send the user image data to the server 400 to generate a human body model. Clothing materials are then added to the human body model to synthesize and render the model, and the user's movements are extracted in real time. The model's posture is then adjusted according to the movement parameters to form a try-on screen. The virtual try-on system can realize dynamic 3D virtual try-on functionality and display user movements in real time through the rendered model, achieving the effect of clothing moving with the user and providing outfit recommendations, thus solving the problem that traditional virtual try-on methods cannot display the try-on effect in real time.

[0132] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: The monitor is configured to display the user interface; The camera is configured to capture user image data in real time; A communicator is configured to establish a communication connection with a server, which has a built-in model reconstruction application for generating human body models based on user image data. The controller is configured as follows: Obtain the user image data; The user image data is sent to the server, and the initial keyframe image is extracted from the user image data; Extract image depth parameters from the initial keyframe image; The image depth parameters are sent to the server, so that the server generates human body model parameters based on the image depth parameters and feeds back the human body model. Real-time keyframe images are extracted from the user image data. When the image similarity between the initial keyframe image and the real-time keyframe image is less than a preset similarity threshold, the human body model reconstructed by the server based on the image depth parameters corresponding to the real-time keyframe image is received. Add clothing materials to the human body model to composite and render the model; The motion parameters are extracted from the user image data, and the model pose of the rendering model is adjusted according to the motion parameters to render the fitting scene; Wherein, after the step of extracting image depth parameters from the initial keyframe image, the initial keyframe image is recorded; After performing the step of extracting motion parameters from the user image data, or after sending the image depth parameters to the server, the recorded initial keyframe image is replaced with the real-time keyframe image.

2. The display device according to claim 1, characterized in that, The controller is further configured to extract real-time keyframe images from the user image data, and to: Compare the initial keyframe image and the real-time keyframe image to obtain image similarity; If the image similarity is greater than or equal to a preset similarity threshold, the step of extracting action parameters from the user image data is executed.

3. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of sending the user image data to the server, human targets are identified from the user image data; Add skeletal key points to the human figure target to generate skeletal parameters; The skeletal parameters are sent to the server so that the server can set the joint positions of the human body model according to the skeletal parameters.

4. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of adding clothing materials to the human body model, a selection instruction for selecting clothing is obtained from user input, wherein the selection instruction specifies at least one target garment. In response to the selection instruction, the target clothing material is extracted from the clothing material library; According to preset outfit recommendation rules, match related clothing materials in the clothing material library based on the target clothing material; Add the target clothing material and the associated clothing material to the human body model.

5. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of adjusting the model pose of the rendering model according to the action parameters, the skeletal key points of each frame of the user image data are traversed. Compare the positions of skeletal key points in two adjacent frames to obtain the movement distance of each skeletal key point; The joint positions of the human body model are moved according to the moving distance.

6. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of adjusting the model pose of the rendered model according to the action parameters, the head region is identified in the user image data; Detect user facial expressions in the head area; Based on the type of the user's expression, match an expression model of the same type from a preset expression library; Replace the facial area in the rendered model with the expression model.

7. A virtual fitting system, characterized in that, include: The system comprises a display device, an image acquisition device, and a server; wherein the image acquisition device is connected to the display device, and the display device establishes a communication connection with the server. The image acquisition device is configured to acquire user images in real time and perform image signal processing on the user images to generate user image data; the image acquisition device is also configured to send the user image data to the display device. The display device is configured to acquire the user image data and send the user image data to the server; the user image data includes initial keyframe images and real-time keyframe images; The server has a built-in model reconstruction application, which is configured to receive the user image data and run the model reconstruction application; generate a human body model based on the initial keyframe image in the user image data; the server is also configured to feed the human body model back to the display device; the server is further configured to reconstruct the human body model based on the image depth parameters corresponding to the real-time keyframe image when the image similarity between the initial keyframe image and the real-time keyframe image is less than a preset similarity threshold, and feed the reconstructed human body model back to the display device. The display device is also configured to add clothing materials to the human body model to synthesize a rendering model; and to extract motion parameters from the user image data and adjust the model posture of the rendering model according to the motion parameters to render a fitting scene. The display device is further configured to record the initial keyframe image after the step of extracting image depth parameters from the initial keyframe image; After performing the step of extracting motion parameters from the user image data, or after sending the image depth parameters to the server, the recorded initial keyframe image is replaced with the real-time keyframe image.

8. A virtual try-on method, characterized in that, An application is made in a virtual try-on system, the virtual try-on system comprising: a display device, an image acquisition device, and a server; wherein, the image acquisition device is connected to the display device, and the display device establishes a communication connection with the server; the virtual try-on method includes: The image acquisition device acquires user images in real time and performs image signal processing on the user images to generate user image data; The display device acquires the user image data and sends the user image data to the server; The display device extracts an initial keyframe image from the user image data, extracts an image depth parameter from the initial keyframe image, and sends the image depth parameter to the server. The server receives the user image data, generates human body model parameters based on the image depth parameters, and feeds back the human body model to the display device. The display device extracts real-time keyframe images from the user image data, and when the image similarity between the initial keyframe image and the real-time keyframe image is less than a preset similarity threshold, it sends the image depth parameters corresponding to the real-time keyframe image to the server. The server reconstructs a human body model based on the image depth parameters; The display device adds clothing materials to the human body model to synthesize and render the model; The display device extracts motion parameters from the user image data and adjusts the model posture of the rendering model according to the motion parameters to render the fitting screen. After the step of extracting image depth parameters from the initial keyframe image, the display device records the initial keyframe image; after performing the step of extracting motion parameters from the user image data, or after sending the image depth parameters to the server, the real-time keyframe image replaces the recorded initial keyframe image.

Images