A fitting method, device, terminal equipment, storage medium and system

Abstract

The application is suitable for the field of intelligent devices, and provides a fitting method, a fitting device, a terminal device, a storage medium and a fitting system. The fitting method comprises the following steps: collecting point cloud data and user images of a user, and establishing an initial three-dimensional model of the user based on the point cloud data and the user images; obtaining clothing data of clothes to be changed currently worn by the user; correcting the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user; and fusing a clothing model of clothes to be tried on and the target three-dimensional model, and displaying the fused fitting model. The embodiment of the application can make the fitting effect more close to the actual situation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent devices, and particularly relates to a fitting method and device, a terminal device, a storage medium and a system. BACKGROUND

[0002] With the progress of laser radar, 3D scanning and detection technology, virtual fitting technology based on 3D scanning is continuously developed and landed. Some large shopping malls begin to use virtual fitting technology to help merchants promote commodities, improve customer fitting efficiency and enhance user experience.

[0003] The existing virtual fitting technology is often based on a 3D scan to obtain a human three-dimensional model, and fuses the human three-dimensional model with a clothing model of clothes sold by a merchant. However, it is found in actual application that the fitting model fused in this way does not fit the actual situation, resulting in a decrease in user experience and promotion effect of the merchant. SUMMARY

[0004] Embodiments of the present application provide a fitting method, device, terminal device and storage medium, which can make the fitting model more realistic.

[0005] The first aspect of the embodiments of the present application provides a fitting method, comprising:

[0006] acquiring point cloud data and user images of a user, and establishing an initial three-dimensional model of the user based on the point cloud data and the user images;

[0007] obtaining clothing data of clothes to be changed currently worn by the user;

[0008] correcting the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user;

[0009] fusing a clothing model of clothes to be tried on with the target three-dimensional model, and displaying the fused fitting model.

[0010] The second aspect of the embodiments of the present application provides a fitting device, comprising:

[0011] a modeling unit configured to acquire point cloud data and user images of a user, and establish an initial three-dimensional model of the user based on the point cloud data and the user images;

[0012] an obtaining unit configured to obtain clothing data of clothes to be changed currently worn by the user;

[0013] a correction unit configured to correct the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user;

[0014] The display unit is configured to fuse a garment model of the clothes to be tried on and the target three-dimensional model, and display the fused garment model.

[0015] The third aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of the above method.

[0016] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0017] The fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a terminal device, causes the terminal device to implement the steps of the method.

[0018] The sixth aspect of the embodiments of the present application provides a fitting system, including a point cloud processing device, a central server, a display device, a camera and a laser radar; wherein the camera is configured to collect a user image of a user in a first collection area; the laser radar is configured to collect point cloud data of the user in the first collection area; the point cloud processing device is configured to generate the target three-dimensional model according to the user image and the point cloud data; the central server is configured to store the target three-dimensional model uploaded by the point cloud processing device; and the display device is configured to obtain the target three-dimensional model stored in the central server, fuse a garment model of clothes to be tried on and the target three-dimensional model, and display the fused garment model.

[0019] In the embodiments of the present application, by establishing an initial three-dimensional model of a user based on point cloud data and user images of the user, and correcting the initial three-dimensional model by using garment data of clothes to be replaced currently worn by the user, a target three-dimensional model of the user is obtained, and then a garment model of clothes to be tried on is fused with the target three-dimensional model, and the fused garment model is displayed, so that the fitting degree of the target three-dimensional model and the garment model in the garment model is improved, and the garment model is more fitted to the actual situation, and the user experience and the promotion effect of the merchant are improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flowchart of an implementation process of a fitting method provided by an embodiment of the present application;

[0022] Figure 2 is a flowchart of a specific implementation process of step S102 provided by an embodiment of the present application;

[0023] Figure 3 is a flowchart of a specific implementation process of step S103 provided by an embodiment of the present application;

[0024] Figure 4 is a structural diagram of a fitting device provided by an embodiment of the present application;

[0025] Figure 5 is a structural diagram of a terminal device provided by an embodiment of the present application;

[0026] Figure 6 is a structural diagram of a fitting system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0028] With the progress of laser radar, 3D scanning and detection technology, virtual fitting technology based on 3D scanning is continuously developed and landed. Some large shopping malls begin to use virtual fitting technology to help merchants promote goods, improve customer fitting efficiency and enhance user experience.

[0029] The existing virtual fitting technology is often based on a 3D human model obtained by 3D scanning, and fuses the 3D human model with a clothing model of clothes sold by a merchant. However, it is found in actual application that the clothing model is often designed according to a 3D human model without clothes, so when a user wears more clothes, the fitting degree of the clothing model and the 3D human model is reduced.

[0030] For example, when a user who actually fits M-size clothes wears more clothes, the fitting model obtained by fusing the scanned 3D human model with a clothing model of L-size clothes may look more fitted.

[0031] Therefore, there is a need for a fitting method that makes the fitting model more fitted to the actual situation.

[0032] In order to illustrate the technical solutions of the present application, the following will be described by specific embodiments.

[0033] Figure 1 An implementation flowchart of a fitting method provided by an embodiment of the present application is shown, which can be applied to a terminal device and can be suitable for a situation in which a fitting model needs to be more fitted to reality. The terminal device can be a smart phone, a computer, a tablet device, or other smart devices with display function and computing processing capability, such as a guide device or a robot set in a shopping mall.

[0034] Specifically, the fitting method can include the following steps S101 to S104.

[0035] In step S101, point cloud data and user images of a user are collected, and an initial three-dimensional model of the user is established based on the point cloud data and the user images.

[0036] In an embodiment of the present application, a staff member can install sensors for collecting point cloud data and user images near a preset first collection area, and when a user enters the first collection area, a terminal device can obtain sensor data from the sensors, and establish an initial three-dimensional model of the user based on the point cloud data and the user images in the sensor data.

[0037] In some embodiments of the present application, a staff member can plan a detection channel at an entrance of a shopping mall, and arrange cameras and laser radars on both sides of the detection channel. When a user passes through the detection channel, a terminal device can obtain user images collected by the cameras and point cloud data obtained by the laser radars, and establish an initial three-dimensional model using conversion parameters previously calibrated between the cameras and the laser radars.

[0038] In some specific embodiments, a staff member may install a first laser radar and a first camera 5 meters in front of the entrance of the inspection channel to collect front-facing user images and front-facing point cloud data of users passing through the inspection channel. A second laser radar and a second camera may also be installed at the entrance of the inspection channel to collect back-facing user images and back-facing point cloud data of users passing through the inspection channel. After installation, the terminal device may calibrate the first laser radar, the second laser radar, the first camera, and the second camera. For example, the coordinate systems of the first laser radar and the first camera may be calibrated to obtain a first transformation matrix and a first translation matrix; the coordinate systems of the second laser radar and the second camera may be calibrated to obtain a second transformation matrix and a second translation matrix; and the coordinate systems of the first laser radar and the second laser radar may be calibrated to obtain a third transformation matrix and a third translation matrix. At this point, with the coordinate system of the first laser radar as the primary coordinate system, the coordinates of the first camera, the second laser radar, and the second camera may be converted to the coordinate system of the first laser radar using the first transformation matrix, the first translation matrix, the second transformation matrix, the second translation matrix, the third transformation matrix, and the third translation matrix. It should be noted that the above is only an example. In actual applications, different three combinations can be selected for calibration, and the coordinate system of any sensor can be used as the main coordinate system. Ultimately, the coordinates of other sensors can be converted to the coordinates of the main coordinate system. After calibration, the plane where the X-axis and Y-axis of the main coordinate system are located is used as the ground, and the Z-axis is used as the height. The ground points in the point cloud data can be rotated and translated to the plane where the XY axes are located. At the same time, the terminal device can calibrate the first acquisition area in the main coordinate system according to the coordinates set by the staff.

[0039] By calibrating the coordinate systems of the first and second LiDARs to obtain the third transformation matrix and third translation matrix, the terminal device can convert the coordinates of the 3D point cloud scanned by the second LiDAR into the coordinates of the first LiDAR. The terminal device then uses a point cloud object detection algorithm to determine the bounding box of the 3D point cloud of the human body. Clustering and denoising algorithms are then used to remove background and noise from the point cloud. By registering continuously acquired point cloud frames, the initial 3D model can be fused.

[0040] Furthermore, the terminal device can decompose the initial three-dimensional model into sub-models of various body parts of the human body through a point cloud skeleton extraction algorithm, which is convenient for subsequent correction and fusion.

[0041] Step S102: Acquire clothing data of the clothing to be replaced that the user is currently wearing.

[0042] The clothing data may refer to data such as thickness and length of each position of the clothing to be replaced.

[0043] Specifically, such as Figure 2As shown, the step S102 can specifically include steps S201-S203.

[0044] In step S201, the clothing type and clothing identifier of the clothing to be replaced are identified according to the user image.

[0045] The clothing type is used to represent the category of the clothing to be replaced, for example, the clothing type can be short-sleeved, long-sleeved, etc. The clothing identifier is used to identify the clothing to be replaced.

[0046] Specifically, the manner in which the terminal device performs clothing recognition on the user image can be selected according to actual conditions. In some embodiments, the terminal device can input the user image into a trained recognition model to obtain the clothing type and clothing identifier output by the recognition model. In other embodiments, the terminal device can also recognize the contour, pattern, trademark, etc. of the clothing to be replaced in the user image, and determine the clothing type and clothing identifier of the clothing to be replaced based on these information.

[0047] In step S202, the body shape data of the user is obtained.

[0048] The body shape data can include the weight data and / or height data of the user.

[0049] In some embodiments of the present application, the staff can lay weight sensors on the ground of the first collection area, and calibrate the coordinate region of each weight sensor in the main coordinate system during the aforementioned coordinate system calibration process. The terminal device can determine the weight data of the user passing through the first collection area by using the weight sensors installed on the ground of the first collection area.

[0050] Specifically, the terminal device can determine a collection position in the first collection area corresponding to the initial three-dimensional model, determine a target weight sensor from the weight sensors installed in the first collection area according to the collection position, and obtain the moving speed and moving acceleration of the user at the collection position. At this time, the terminal device can determine the weight data of the user according to the weight data collected by the target weight sensor, the moving speed and the moving acceleration.

[0051] In some specific embodiments, the terminal device can determine the position of the user passing through the first collection area when collecting the user image and the point cloud data based on the point cloud data, and take the position of the user passing through the first collection area as the collection position. At this time, the terminal device can take each weight sensor within a preset distance range as a target weight sensor with the collection position as the center, and determine the moving speed and moving acceleration of the user at the collection position by using multiple frames of user images and the collection time of each frame of user image. Furthermore, the terminal device can calculate the static weight of the user by using the weight data collected by each target weight sensor, the moving speed and the moving acceleration, and a mathematical model.

[0052] In the embodiments of the present application, the terminal can model the user in motion without requiring the user to stay at a certain position to wait for modeling, and can avoid inaccurate body weight data caused by user motion by calculating the static body weight by combining the weight data collected by each target weight sensor, the moving speed and the moving acceleration.

[0053] In some other embodiments of the present application, the terminal device can determine the height data of the user by using the initial three-dimensional model of the user, and specifically, can determine the height data of the user passing through the first collection area according to the coordinates of the head feature point of the user, or can determine the height data by using the image or point cloud data of the user associated with the initial three-dimensional model of the user.

[0054] In step S203, the clothing data of the clothing to be replaced is determined based on the body shape data, the clothing type and the clothing identifier.

[0055] Specifically, if the clothing identifier is an unknown identifier, it indicates that the clothing to be replaced currently worn by the user is from an unknown source, at this time, the terminal device cannot directly obtain the clothing data of the clothing to be replaced, and the terminal device can obtain a plurality of standard clothing data set in advance, and the standard clothing data matched with the body shape data and the clothing type is taken as the clothing data of the clothing to be replaced. If the clothing identifier is a known identifier, it indicates that the clothing to be replaced currently worn by the user is from a known source, for example, the clothing to be replaced is clothing once sold in a mall, and the terminal device can take the reference clothing data matched with the body shape data and the clothing identifier as the clothing data of the clothing to be replaced.

[0056] That is, if the clothing to be replaced belongs to clothing from an unknown source, the terminal device can obtain the standard clothing data of the same clothing type as the clothing to be replaced and meeting the size required by the body shape data, and take the standard clothing data as the clothing data of the clothing to be replaced. If the clothing to be replaced belongs to clothing from a known source, the terminal device can directly obtain the clothing data of the clothing to be replaced stored in advance by the size required by the body shape data and the clothing identifier of the clothing to be replaced.

[0057] In the embodiments of the present application, the body shape data, the clothing type and the clothing identifier can be combined to determine accurate clothing data, which is convenient for subsequent correction of the initial three-dimensional model.

[0058] In step S103, the initial three-dimensional model is corrected by using the clothing data to obtain the target three-dimensional model of the user.

[0059] In some embodiments of the present application, as shown in FIG. 3, Figure 3 The above step S103 can specifically include the following steps S301 to S303.

[0060] Step S301, determine the clothing pixel points of the clothing to be replaced in the user image.

[0061] Step S302, take the points associated with the clothing pixel points in the initial three-dimensional model as the points to be corrected.

[0062] Specifically, the terminal device can perform semantic analysis on the user image to determine whether each pixel point is a clothing pixel point of the clothing to be replaced in the user image, and mark the clothing pixel points. Then, the terminal device can obtain the pre-calibrated camera intrinsic parameters, combine the conversion matrix and translation matrix between the camera and the laser radar, determine the points associated with the clothing pixel points in the initial three-dimensional model, and mark these points as points to be corrected.

[0063] Step S303, correct the points to be corrected using clothing data to obtain a target three-dimensional model of the user.

[0064] Specifically, the above-mentioned clothing data can refer to the thickness values at each position point of the clothing to be replaced, and the terminal device associates each point to be corrected with a point to be corrected. At this time, the thickness value at a certain position can be taken as the correction amount of the point to be corrected associated with the position, and each point to be corrected is corrected based on the correction amount.

[0065] In some embodiments of the present application, the terminal device can decompose the initial three-dimensional model into sub-models of each body part of the user by a point cloud skeleton extraction algorithm, and correct the points to be corrected on each sub-model one by one based on the above-mentioned correction amount to obtain a target three-dimensional model of the user.

[0066] It should be noted that if the number of clothing to be replaced is greater than one, the terminal device can calculate the correction amount at a certain position based on the clothing data of different clothing to be replaced, and then correct each point to be corrected based on the correction amount.

[0067] Step S104, fuse the clothing model of the clothing to be tried on and the target three-dimensional model, and display the fused fitting model.

[0068] Wherein, the clothing to be tried on can be a recommended clothing set by the merchant, or a clothing selected by the user.

[0069] Specifically, the terminal device can decompose the target three-dimensional model into sub-models of each body part of the user, fuse the clothing model of the clothing to be tried on with the sub-model associated with the clothing to be tried on, and fuse the sub-models that have not been fused with the corresponding standard clothing model to obtain the fused fitting model.

[0070] For example, when the clothes to be tried on is a top, the terminal device can fuse the upper body sub-model with the clothes model of the clothes to be tried on, and fuse the lower body sub-model with the standard trousers model set in advance, to obtain the fused fitting model.

[0071] If there is no sub-model that is not fused after the clothes model of the clothes to be tried on is fused with the sub-model associated with the clothes to be tried on, the fused fitting model can be directly displayed.

[0072] The fusion process can refer to moving the clothes model to the target three-dimensional model according to the matching relationship between the set position in the clothes model and the corresponding position in the target three-dimensional model, so that the matching positions are fitted.

[0073] In the embodiments of the present application, an initial three-dimensional model of the user is established based on the point cloud data and the user image of the user, and the initial three-dimensional model is corrected using the clothes data of the clothes to be replaced that the user is currently wearing, to obtain a target three-dimensional model of the user. Then, the clothes model of the clothes to be tried on is fused with the target three-dimensional model, and the fused fitting model is displayed, so that the fitting degree of the target three-dimensional model and the clothes model in the fitting model is improved, and the fitting model is more fitted to the actual situation, improving the user experience and the promotion effect of the merchant.

[0074] In some embodiments of the present application, the terminal device can perform face recognition based on the user image, extract face features, and generate a user identifier associated with the face features. Then, the user identifier is bound to the target three-dimensional model and stored. When the user enters the second collection area, the terminal can obtain the face image of the user entering the second collection area, and recognize the face image of the user to obtain the face features of the user. At this time, the terminal can obtain the user identifier associated with the face features, and fuse the clothes model of the clothes to be tried on with the target three-dimensional model bound to the user identifier.

[0075] The second collection area can be a different area from the first collection area. In some embodiments, the second collection area can refer to an area within a preset distance range centered on the display screen of the terminal device.

[0076] In actual application, the number of users entering the second collection area can be greater than one. Therefore, the terminal device can obtain the target three-dimensional model of the nearest user based on the distance between each user and the display screen, fuse the clothes model of the clothes to be tried on with the target three-dimensional model of the user, and display the fused fitting model.

[0077] In some embodiments of the present application, the terminal device can comprise a plurality of display screens, and the terminal device can acquire the real-time position of each user entering the third collection area, and then display the fitting model corresponding to the user closest to the user on each display screen.

[0078] In some embodiments of the present application, the third collection area can refer to the entire shopping mall. For example, the terminal device can acquire the real-time position of each user in the shopping mall through the monitoring device, and when one of the users approaches a display screen, the display screen can display the fitting model obtained by fusing the target three-dimensional model of the user and the clothing model.

[0079] In addition, when the terminal device can comprise a plurality of display screens, the clothing data used by each display screen can be different. Specifically, the terminal device can determine the display merchant corresponding to the display screen according to the position of the display screen, and fuse and display the target three-dimensional model of the user and the clothing model of the display merchant corresponding to the display screen.

[0080] In actual application, considering that there can be a cooperative relationship between merchants, the terminal device can associate the merchant corresponding to the display screen with other merchants having a cooperative relationship with the merchant as the display merchant corresponding to the display screen. After displaying the fitting model, the positions of the merchants corresponding to each of the clothes to be tried on in the fitting model can also be displayed.

[0081] In order to make the display effect of the fitting model more realistic, the terminal device can also determine whether there is a target clothing having the same clothing type as the clothing type of the clothing to be tried on in the clothing currently worn by the user according to the clothing type of the clothing to be tried on.

[0082] If there is, the terminal device can take the target clothing and the clothing located outside the target clothing as the clothes to be changed, and perform steps S102 to S104. At this time, the fitting model comprises the target three-dimensional model combining the human body and the clothing located inside the target clothing, and the clothing model of the clothing to be tried on.

[0083] If there is no target clothing having the same clothing type as the clothing type of the clothing to be tried on in the clothing currently worn by the user, in some embodiments, the terminal device can determine the number of levels to which the clothing to be tried on belongs. If the total number of levels of the clothing currently worn by the user is less than the number of levels to which the clothing to be tried on belongs, the initial three-dimensional model can be taken as the target three-dimensional model, i.e. the initial three-dimensional model can not be modified. If the total number of levels of the clothing currently worn by the user is greater than or equal to the number of levels to which the clothing to be tried on belongs, the clothing having a level number greater than or equal to the number of levels to which the clothing to be tried on belongs in the clothing currently worn by the user can be taken as the clothes to be changed, and steps S102 to S104 can be performed.

[0084] Specifically, the terminal device can take the total number of clothes currently worn by the user as the total number of layers of clothes currently worn by the user.

[0085] Since the user cannot accurately estimate the total number when wearing multiple clothes, the terminal device can also determine the type of the outermost clothes among the clothes currently worn by the user, determine the number of layers to which the outermost clothes belong according to a pre-stored mapping table between the type of clothes and the number of layers, and take the number of layers to which the outermost clothes belong as the total number of layers of clothes currently worn by the user.

[0086] For example, when the user wears an innerwear, a long-sleeved shirt and a coat from inside to outside, it can be obtained from the mapping table that the number of layers to which the coat belongs is 3, and if the clothes to be tried on is a jacket, the number of layers to which the jacket belongs is also 3, then the coat can be taken as the clothes to be replaced, and steps S102 to S104 are executed, and finally the fitting model displayed includes the user, the innerwear, the long-sleeved shirt worn by the user and the jacket to be tried on.

[0087] In some other embodiments of the present application, if there is no target clothes with the same type of clothes to be tried on among the clothes currently worn by the user, the terminal device can also directly take all the clothes currently worn by the user as the clothes to be replaced, and execute steps S102 to S104.

[0088] In addition, in some other embodiments of the present application, after the fused fitting model is displayed, the terminal device can also obtain the gaze duration of the user on the display screen for displaying the fitting model; if the gaze duration is less than the duration threshold, it means that the user is not interested in the fitting model, then the terminal device can obtain a new clothes model, fuse the new clothes model with the target three-dimensional model to obtain a new fitting model, and update and display the fitting model.

[0089] If the gaze duration is greater than or equal to the duration threshold, it means that the user is interested in the fitting model, then the terminal device can update the fitting model by fusing the clothes model with the same type of clothes and the same style of clothes with the target three-dimensional model after a preset duration, or directly display the detailed information of the clothes to be replaced in the fitting model.

[0090] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences.

[0091] As Figure 4 Fig. 1 shows a structure schematic diagram of a fitting device 400 provided by an embodiment of the present application, which is configured on a terminal device.

[0092] Specifically, the fitting device 400 can include:

[0093] a modeling unit 401 configured to collect point cloud data and a user image of a user, and establish an initial three-dimensional model of the user based on the point cloud data and the user image;

[0094] an acquisition unit 402 configured to acquire clothing data of a to-be-changed clothing currently worn by the user;

[0095] a correction unit 403 configured to correct the initial three-dimensional model by using the clothing data, to obtain a target three-dimensional model of the user;

[0096] a display unit 404 configured to fuse a clothing model of the to-be-fitted clothing and the target three-dimensional model, and display the fused fitting model.

[0097] In some embodiments of the present application, the correction unit 403 can be specifically configured to: determine clothing pixel points of the to-be-changed clothing in the user image; take points associated with the clothing pixel points in the initial three-dimensional model as to-be-corrected points; correct the to-be-corrected points by using the clothing data, to obtain the target three-dimensional model of the user.

[0098] In some embodiments of the present application, the acquisition unit 402 can be specifically configured to: identify a clothing type and a clothing identifier of the to-be-changed clothing according to the user image; acquire body shape data of the user; and determine the clothing data of the to-be-changed clothing based on the body shape data, the clothing type and the clothing identifier.

[0099] In some embodiments of the present application, the body shape data can include weight data and / or height data of the user; the acquisition unit 402 can be further specifically configured to: determine a collection position corresponding to the initial three-dimensional model in a first collection area; determine a target weight sensor from weight sensors installed in the first collection area according to the collection position; acquire a moving speed and a moving acceleration of the user at the collection position; and determine the weight data of the user according to weight data collected by the target weight sensor, the moving speed and the moving acceleration; and the acquisition unit 402 can also be specifically configured to: determine the height data of the user according to the initial three-dimensional model of the user.

[0100] In some embodiments of the present application, the acquisition unit 402 can be further configured to: if the clothing identifier is an unknown identifier, acquire standard clothing data matching the body shape data and the clothing type as the clothing data of the clothing to be replaced; and if the clothing identifier is a known identifier, acquire reference clothing data matching the body shape data and the clothing identifier as the clothing data of the clothing to be replaced.

[0101] In some embodiments of the present application, the display unit 404 can be configured to: decompose the target three-dimensional model into sub-models of each body of the user; fuse the clothing model of the clothing to be tried on with the sub-models associated with the clothing to be tried on, and fuse the sub-models not fused with the corresponding standard clothing model to obtain a fused fitting model.

[0102] In some embodiments of the present application, the display unit 404 can be further configured to: when the user enters the second acquisition area, recognize a face image of the user to obtain a face feature of the user; acquire a user identifier associated with the face feature, and fuse the clothing model of the clothing to be tried on with a target three-dimensional model bound with the user identifier.

[0103] In some embodiments of the present application, the fitting device 400 further comprises an updating unit configured to acquire a gaze duration of the user on a display screen used to display the fitting model; and update and display the fitting model if the gaze duration is less than a duration threshold.

[0104] It should be noted that, for the convenience and brevity of description, the specific working process of the fitting device 400 can refer to Figures 1 to 3 the corresponding process of the method, which will not be repeated here.

[0105] As Figure 5 shown, a schematic diagram of a terminal device provided by an embodiment of the present application is shown. The terminal device 5 can include a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a fitting program. The processor 50 implements the steps in each of the above fitting method embodiments when executing the computer program 52, such as steps S101-S104 shown in Figure 1 . Alternatively, the processor 50 implements the functions of each module / unit in each of the above device embodiments when executing the computer program 52, such as the modeling unit 401, the acquisition unit 402, the correction unit 403, and the display unit 404 shown in Figure 4 .

[0106] The computer program can be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the terminal device.

[0107] For example, the computer program can be divided into a modeling unit, an acquisition unit, a correction unit and a display unit. The specific functions of each unit are as follows: the modeling unit is configured to collect point cloud data and user images of a user, and establish an initial three-dimensional model of the user based on the point cloud data and the user images; the acquisition unit is configured to acquire clothing data of a piece of clothing currently worn by the user; the correction unit is configured to correct the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user; and the display unit is configured to fuse a clothing model of a piece of clothing to be tried on with the target three-dimensional model, and display the fused fitting model.

[0108] The terminal device can include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art can understand that, Figure 5 The terminal device is only an example and does not constitute a limitation on the terminal device, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the terminal device can also include an input / output device, a network access device, a bus, etc.

[0109] The processor 50 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0110] The memory 51 can be an internal storage unit of the terminal device, such as a hard disk or a memory of the terminal device. The memory 51 can also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. Further, the memory 51 can also include both the internal storage unit and the external storage device of the terminal device. The memory 51 is used to store the computer program and other programs and data required by the terminal device. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0111] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0112] The above embodiments are described for a single terminal device to execute the above fitting method, and in actual application, the steps of the fitting method can be executed by multiple devices respectively, and these devices form a fitting system.

[0113] Specifically, Figure 6 A fitting system provided by the present application is shown, which can include a point cloud processing device 61, a central server 62, a display device 63, a camera and a laser radar.

[0114] The camera is used to collect a user image of a user in a first collection area; the laser radar is used to collect point cloud data of the user in the first collection area; the point cloud processing device 61 is used to generate a target three-dimensional model in the foregoing fitting method according to the user image and the point cloud data; the central server 62 is used to store the target three-dimensional model uploaded by the point cloud processing device 61; and the display device 63 is used to obtain the target three-dimensional model stored in the central server 62, fuse a clothing model of clothes to be tried on with the target three-dimensional model, and display the fused fitting model.

[0115] The first collection area can be different from a second collection area where the display device 63 is located.

[0116] In some embodiments of the present application, the fitting system further comprises a weight sensor 66 installed on the ground of the first collection area, and the weight sensor 66 is used to collect weight data at the location where the weight sensor 66 is located.

[0117] In a specific embodiment, the camera can specifically include a first camera 641 and a second camera 642, and the fitting system can capture a front user image of the user passing through the first collection area through the first camera 641 and capture a back user image of the user passing through the first collection area through the second camera 642. The laser radar can specifically include a first laser radar 651 and a second laser radar 652, and the fitting system can scan front point cloud data of the user passing through the first collection area through the first laser radar 651 and scan back point cloud data of the user passing through the first collection area through the second laser radar 652. In some embodiments of the present application, the first collection area can specifically refer to the entrance area of the mall, that is, the camera and the laser radar are both installed in the entrance area of the mall and are used to collect the user image and the point cloud data of the user entering the mall.

[0118] The point cloud processing device 61 can calibrate the first camera 641, the second camera 642, the first laser radar 651 and the second laser radar 652, and receive the user image and the point cloud data uploaded by the first camera 641, the second camera 642, the first laser radar 651 and the second laser radar 652, and then establish an initial three-dimensional model. Then, the point cloud processing device 61 can determine the height data of the user by using the user image or the point cloud data, and obtain the sensor data uploaded by the weight sensor 66, determine the weight data of the user based on the sensor data, and then correct the initial three-dimensional model to obtain a target three-dimensional model. Then, by recognizing the facial features of the user and generating a user identification, the target three-dimensional model is bound with the user identification and uploaded to the central server 62.

[0119] After the central server 62 receives the target three-dimensional model and the user identification uploaded by the point cloud processing device 61, the central server 62 can store the target three-dimensional model and the user identification.

[0120] The display device 63 can include a camera and a display screen. When a user enters a second collection area where the display device 63 is located, the display device 63 can capture a user image through the camera and identify a face feature, obtain a target three-dimensional model corresponding to a user identifier associated with the face feature from the central server, fuse a clothing model of the clothes to be tried on with the target three-dimensional model, and display the fused fitting model on the display screen. The display device 63 can be arranged at the entrance of a merchant, and the second collection area can specifically refer to an area within a preset distance range centered on the display device 63.

[0121] In some embodiments of the present application, the above-mentioned fitting system can include a plurality of display devices 63, each of which is arranged at the entrance of a different merchant in a shopping mall. Any one of the display devices 63 can determine the distance between the user and itself through the sensor installed thereon. Based on the distance between the user and itself, the display device 63 can obtain the target three-dimensional model of the user closest to itself, fuse the clothing model of the clothes to be tried on with the target three-dimensional model of the user, and display the fusion result.

[0122] Specifically, when a user enters a second collection area where a certain display device 63 is located, the display device 63 can obtain a face image of the user entering the second collection area, identify the face image of the user, obtain a face feature of the user, and determine a user identifier associated with the face feature. Then, the display device 63 can obtain a target three-dimensional model bound to the user identifier from the central server 62, and fuse the clothing model of the clothes to be tried on with the target three-dimensional model bound to the user identifier, and then display the fitting model.

[0123] In some embodiments of the present application, the above-mentioned fitting system can include a plurality of display devices 63, each of which is arranged at the entrance of a different merchant in a shopping mall. Any one of the display devices 63 can determine the distance between the user and itself through the sensor installed thereon. Based on the distance between the user and itself, the display device 63 can obtain the target three-dimensional model of the user closest to itself, fuse the clothing model of the clothes to be tried on with the target three-dimensional model of the user, and display the fusion result.

[0124] In some embodiments of the present application, each display device 63 can determine the corresponding display merchant according to the location where it is located, and fuse and display the target three-dimensional model of the user with the clothing model of the corresponding display merchant.

[0125] In actual application, considering that there can be a cooperation relationship between the merchant and another merchant, the display device 63 can also associate the merchant with which the display device 63 is associated and another merchant having a cooperation relationship with the merchant as a display merchant corresponding to the display screen. After displaying the fitting model, the positions of the merchants corresponding to each of the clothes to be tried on in the fitting model can also be displayed.

[0126] In order to improve the fitting experience of the user, in some embodiments of the present application, the display device 63 can also acquire the gaze duration of the user on the display device 63. If the gaze duration is less than the duration threshold, it indicates that the user has a low degree of interest in this fitting model, and then the display device 63 can acquire a new clothes model, fuse the new clothes model with the target three-dimensional model to obtain a new fitting model, and update and display the fitting model.

[0127] The specific working processes of the devices of the fitting system described above can also be referred to Figures 1 to 3 The corresponding processes of the fitting method are not described here.

[0128] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in a certain embodiment can be referred to the related description of other embodiments.

[0129] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software mode depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0130] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal device and method can be implemented by other means. For example, the apparatus / terminal device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be through some interface, indirect coupling or communication connection between the devices or units, and can be electrical, mechanical or other forms.

[0131] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0132] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0133] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0134] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of fitting, characterized by, The method comprises the following steps: collecting point cloud data and user images of a user, and establishing an initial three-dimensional model of the user based on the point cloud data and the user images; if there is no target clothing of the same type as the clothing type of the clothing to be tried on in the clothing currently worn by the user, determining the hierarchical number to which the clothing to be tried on belongs; if the total hierarchical number of the clothing currently worn by the user is less than the hierarchical number to which the clothing to be tried on belongs, taking the initial three-dimensional model as a target three-dimensional model; if the total hierarchical number of the clothing currently worn by the user is greater than or equal to the hierarchical number to which the clothing to be tried on belongs, taking the clothing currently worn by the user and having a hierarchical number greater than or equal to the hierarchical number to which the clothing to be tried on belongs as clothing to be replaced; obtaining clothing data of the clothing to be replaced currently worn by the user; modifying the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user; fusing a clothing model of the clothing to be tried on and the target three-dimensional model, and displaying the fused fitting model.

2. The fitting method according to claim 1, wherein The step of modifying the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user comprises the following steps: determining clothing pixel points of the clothing to be replaced in the user images; taking points associated with the clothing pixel points in the initial three-dimensional model as points to be modified; modifying the points to be modified by using the clothing data to obtain a target three-dimensional model of the user.

3. The fitting method according to claim 1 or 2, wherein The step of obtaining clothing data of the clothing to be replaced currently worn by the user comprises the following steps: identifying clothing types and clothing identifiers of the clothing to be replaced according to the user images; obtaining body shape data of the user; determining clothing data of the clothing to be replaced based on the body shape data, the clothing types and the clothing identifiers.

4. The fitting method according to claim 3, wherein The body shape data comprises weight data and / or height data of the user. The step of obtaining body shape data of the user comprises the following steps: determining a collection position corresponding to the initial three-dimensional model in a first collection area; determining a target weight sensor from weight sensors installed in the first collection area according to the collection position; obtaining a moving speed and a moving acceleration of the user at the collection position; determining weight data of the user according to weight data collected by the target weight sensor, the moving speed and the moving acceleration; and / or determining height data of the user according to the initial three-dimensional model of the user.

5. The fitting method according to claim 3, wherein The step of determining clothing data of the clothing to be replaced based on the body shape data, the clothing types and the clothing identifiers comprises the following steps: if the clothing identifier is an unknown identifier, taking standard clothing data matched with the body shape data and the clothing types as the clothing data of the clothing to be replaced; if the clothing identifier is a known identifier, taking reference clothing data matched with the body shape data and the clothing identifier as the clothing data of the clothing to be replaced.

6. The fitting method according to claim 1 or 2, wherein The step of fusing a clothing model of the clothing to be tried on and the target three-dimensional model comprises the following steps: decomposing the target three-dimensional model into sub-models of each body part of the user; Fuse the clothing model of the clothing to be tried on and the sub-model associated with the clothing to be tried on, and fuse the unfused sub-model with the corresponding standard clothing model to obtain a fused fitting model.

7. The fitting method according to claim 1 or 2, wherein The clothing model of the clothing to be tried on and the target three-dimensional model are fused, including: When the user enters the second collection area, the face image of the user is recognized to obtain the face feature of the user; Obtain the user identifier associated with the face feature, and fuse the clothing model of the clothing to be tried on and the target three-dimensional model bound to the user identifier.

8. The fitting method according to claim 1 or 2, wherein After the fused fitting model is displayed, including: Obtain the gaze duration of the user on the display screen for displaying the fitting model; If the gaze duration is less than the duration threshold, update and display the fitting model.

9. A fitting device, characterized by Including: The modeling unit is configured to collect point cloud data and user images of a user, and establish an initial three-dimensional model of the user based on the point cloud data and the user images; The fitting device is further configured to: if there is no target clothing of the same clothing type as the clothing type of the clothing to be tried on in the clothing currently worn by the user, determine the hierarchical number to which the clothing to be tried on belongs; if the total hierarchical number of the clothing currently worn by the user is less than the hierarchical number to which the clothing to be tried on belongs, take the initial three-dimensional model as a target three-dimensional model; if the total hierarchical number of the clothing currently worn by the user is greater than or equal to the hierarchical number to which the clothing to be tried on belongs, take the clothing of the user currently worn and having a hierarchical number greater than or equal to the hierarchical number to which the clothing to be tried on belongs as a clothing to be replaced; The obtaining unit is configured to obtain clothing data of the clothing to be replaced currently worn by the user; The correction unit is configured to correct the initial three-dimensional model by using the clothing data to obtain a target three-dimensional model of the user; The display unit is configured to fuse the clothing model of the clothing to be tried on and the target three-dimensional model, and display the fused fitting model.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

12. A fitting system characterized in that, The point cloud processing device, the central server, the display device, the camera and the laser radar are included. The camera is configured to collect a user image of a user in a first collection area; The laser radar is configured to collect point cloud data of the user in the first collection area; The point cloud processing device is configured to generate a target three-dimensional model according to the user image and the point cloud data, as claimed in any one of claims 1 to 8; The central server is configured to store the target three-dimensional model uploaded by the point cloud processing device; The display device is configured to obtain the target three-dimensional model stored by the central server, fuse the clothing model of the clothing to be tried on and the target three-dimensional model, and display the fused fitting model.

Images