A fitting method, device, system and storage medium

Abstract

The present application is applicable to the field of smart devices, and provides a fitting method, device, system and storage medium. Among them, the above-mentioned fitting method is applied to a fitting system, and the fitting system includes multiple guide screens, and a collection device facing a collection area enclosed by the multiple guide screens. The fitting method specifically includes: in response to detecting that a user enters the collection area, outputting guidance information to obtain point cloud data respectively collected by a collection device set at a fixed position when the user sequentially turns different sides toward the collection device; determining a first three-dimensional model of the user according to the point cloud data corresponding to different sides of the user; and determining and displaying a fitting model based on the first three-dimensional model and the first clothing model of the clothing to be tried on. The embodiments of the present application can reduce the hardware cost of the fitting system.

Description

Technical Field

[0001] The present application relates to the field of smart devices, and in particular to a fitting method, device, system and storage medium. Background Art

[0002] At present, the fitting rooms in stores are basically traditional fitting rooms, which require manual picking up of clothes, fitting on of clothes, returning of clothes, and sorting by store staff. This method is inefficient and easily causes unnecessary damage to clothes. Virtual fitting rooms in smart scenarios have great market demand. With the advancement of LiDAR, 3D scanning and detection technology, some stores have begun to use virtual fitting technology to promote products, improve customers' fitting efficiency, and enhance user experience.

[0003] Existing virtual fitting technologies often use circular rails to drive sensors to rotate, vertical rails to drive sensors to rotate, turntables to drive sensors to rotate, etc., or need to set sensors at different angles to achieve three-dimensional modeling of users. These methods have high hardware costs, complex structures, and are not conducive to application implementation. Summary of the invention

[0004] The embodiments of the present application provide a fitting method, device, system and storage medium, which can reduce the cost of hardware.

[0005] A first aspect of an embodiment of the present application provides a fitting method, which is applied to a fitting system, wherein the fitting system includes a plurality of guide screens and a collection device facing a collection area enclosed by the plurality of guide screens. The fitting method includes

[0006] In response to detecting that the user enters the acquisition area, outputting guidance information to obtain point cloud data respectively acquired by the acquisition device when the user sequentially turns different sides toward the acquisition device;

[0007] Determining a first three-dimensional model of the user according to the point cloud data corresponding to different sides of the user;

[0008] Based on the first three-dimensional model and a first clothing model of the clothing to be tried on, a fitting model is determined and displayed.

[0009] A second aspect of an embodiment of the present application provides a fitting device, which is configured in a fitting system, wherein the fitting system includes a plurality of guide screens and a collection device facing a collection area enclosed by the plurality of guide screens, wherein the fitting device includes:

[0010] an information output unit, configured to output guidance information in response to detecting that a user enters the collection area, so as to obtain point cloud data respectively collected by the collection device when the user sequentially turns different sides toward the collection device;

[0011] A three-dimensional modeling unit, configured to determine a first three-dimensional model of the user according to point cloud data corresponding to different sides of the user;

[0012] The fitting unit is used to determine and display a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on.

[0013] A third aspect of an embodiment of the present application provides a fitting system, comprising a computing device and a collection device arranged at a fixed position, the computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the above-mentioned fitting method when executing the computer program.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned fitting method are implemented.

[0015] A fifth aspect of an embodiment of the present application provides a computer program product, which, when executed on a system, enables the system to implement the steps of the fitting method.

[0016] In an embodiment of the present application, in response to detecting that a user has entered a collection area formed by multiple guide screens, the fitting system can output guidance information to obtain point cloud data collected by the collection device when the user turns different sides toward the collection device in turn, and then, according to the point cloud data corresponding to different sides of the user, the first three-dimensional model of the user is determined, and based on the first three-dimensional model and the first clothing model of the clothes to be tried on, the fitting model is determined and displayed, so that the fitting system does not need to install laser radars at different positions, nor does it need to configure a complex rotating structure for the laser radar. By using a collection device installed in a fixed position, point cloud data from different sides of the user can be collected, which reduces the hardware cost of the fitting system and facilitates the application of the fitting system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 This is a schematic diagram of an implementation flow of a fitting method provided in an embodiment of the present application;

[0019] Figure 2 is a first structural schematic diagram of a fitting system provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of a specific implementation process of displaying guidance information provided by an embodiment of the present application;

[0021] Figure 4 This is a schematic diagram of a first specific implementation flow of step S103 provided in an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of a second specific implementation flow of step S103 provided in an embodiment of the present application;

[0023] Figure 6 It is a third specific implementation flow diagram of step S103 provided in an embodiment of the present application;

[0024] Figure 7 is a top view of a fitting system provided in an embodiment of the present application;

[0025] Figure 8 is a schematic structural diagram of a fitting device provided in an embodiment of the present application;

[0026] Fig. 9 It is a second structural schematic diagram of the fitting system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0028] Existing virtual fitting technologies often use circular rails to drive sensors to rotate, vertical rails to drive sensors to rotate, turntables to drive sensors to rotate, etc., or need to set sensors at different angles to achieve three-dimensional modeling of users. These methods have high hardware costs, complex structures, and are not conducive to application implementation.

[0029] In view of the above background and problems, the present application provides a low-cost fitting method and fitting system, which can facilitate the application of the fitting system.

[0030] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0031] Figure 1 A schematic diagram of an implementation flow of a fitting method provided in an embodiment of the present application is shown. The method can be applied to a fitting system and is suitable for situations where hardware costs need to be reduced.

[0032] In some embodiments of the present application, Figure 2 As shown, the fitting system may include a computing device, a plurality of guide screens, and a collection device facing a collection area formed by the plurality of guide screens. The collection device may collect information of users entering the collection area, and the computing device may process the information collected by the collection device.

[0033] The acquisition equipment may include sensors such as laser radar and camera. The laser radar may include a planar laser radar, including but not limited to a multi-line laser radar, a single-line laser radar, and a combination of a single-line laser radar and a gimbal. The computing device may be a smart device with certain computing capabilities such as a smartphone, a computer, a tablet device, or a server device.

[0034] It should be noted that the number of guide screens can be set according to actual conditions. In some specific implementations, such as Figure 2 As shown, the fitting system can be composed of four guide screens.

[0035] Specifically, the above fitting method may include the following steps S101 to S103.

[0036] Step S101, in response to detecting that a user enters a collection area, outputting guidance information to obtain point cloud data collected by a collection device when the user sequentially turns different sides toward the collection device.

[0037] In the implementation of the present application, in order to obtain an accurate three-dimensional model of the user, it is necessary to collect point cloud data from different sides of the user. Therefore, when the fitting system detects that the user has entered the collection area, it can output guidance information and control the collection device to collect information from the user, so as to obtain point cloud data collected by the collection device when the user turns different sides toward the collection device in turn.

[0038] In some embodiments of the present application, the orientation of the acquisition device is the same as the orientation of one of the multiple guide screens, and the user will turn different sides toward the acquisition device when facing different guide screens.

[0039] At this time, the fitting system can display guidance information on the guide screen that the user is facing when the user faces each guide screen in turn, so as to obtain point cloud data collected by the collection device when the user turns different sides toward the collection device in turn.

[0040] The above-mentioned guidance information can be used to guide the user to complete the collection of point cloud data currently facing the side of the collection device, and can also be used to guide the user to turn to another guidance screen.

[0041] That is to say, when the user faces a certain guide screen, guidance information can be displayed on the guide screen to guide the user, assisting the collection device to collect point cloud data facing the side of the collection device, or the user can be guided to turn, assisting the collection device to collect point cloud data facing the side of the collection device after turning, until the collection device completes the collection of point cloud data on different sides of the user.

[0042] Specifically, Figure 3 As shown, the above-mentioned displaying guidance information may specifically include the following steps S301 to S304.

[0043] Step S301: determining a current guide screen, which the user is currently facing, among multiple guide screens.

[0044] In some embodiments of the present application, multiple guide screens may surround a collection area. When a user enters the collection area, the user will face any guide screen. The fitting system may photograph the user in the collection area through a camera and perform image recognition to determine the guide screen the user is currently facing.

[0045] Step S302, obtaining point cloud data corresponding to the current guide screen collected by the collection device.

[0046] When the user faces the current guide screen, he or she will turn one side of himself or herself toward the acquisition device accordingly. Therefore, the acquisition device can collect point cloud data of this side, and the fitting system can obtain the point cloud data corresponding to the current guide screen collected by the acquisition device.

[0047] In order to ensure the reliability of point cloud data, the fitting system can also detect whether the user meets the point cloud collection conditions. If not, the posture guidance information is output on the current guide screen, and the user is re-detected to see if the user meets the point cloud collection conditions until it is detected that the user meets the point cloud collection conditions, and the point cloud data corresponding to the current guide screen collected by the collection device is obtained.

[0048] Among them, the posture guidance information is used to prompt the user to adjust his or her own posture.

[0049] In some embodiments of the present application, the fitting system can scan the user's body point cloud data through a laser radar, and convert the point cloud data based on the conversion parameters calibrated in advance to obtain the coordinate value of the three-dimensional space. At this time, the fitting system can extract the center of mass O(x, y, z) of the user's point cloud, and calculate the distance between the user and the current guide screen based on the difference vector Ct[x, y, z] between the installation position A(x, y, z) of the current guide screen and the center of mass O(x, y, z). If the distance between the user and the current guide screen is not within the reference range, it can be confirmed that the user does not meet the point cloud acquisition conditions, and the posture guidance information carrying the user's three-dimensional coordinate position, the distance the user needs to move, and the direction, etc. is fed back in real time on the current guide screen.

[0050] In some other embodiments of the present application, the fitting system can also be modeled according to the currently collected point cloud data, and according to the established model M a And the sub-model set M1 corresponding to each part of the human body = [Msub1, Msub2, ..., Msub n ], determine the first modeling completeness W1(M1) of each part and the second modeling completeness W2(M a ). The first modeling completeness W1(M1) and the second modeling completeness W2(M a ) Combined with the remaining action algorithm S(W1(M1), W2(M a )) is processed to obtain the actions required to model the remaining parts. At this time, the fitting system can display the posture guidance information carrying the action posture information in the current guidance screen.

[0051] Among them, Msub is the sub-model of each part of the human body model, n is the number of human body parts, W1 and W2 are completeness calculation functions, and S is the remaining action algorithm.

[0052] In the implementation manner of the present application, the fitting system provides real-time feedback of the user's three-dimensional coordinate position, the distance to be moved, the completeness of the current modeling, and other information through the current guide screen, thereby visually assisting the user in completing the modeling.

[0053] Step S303: When point cloud data corresponding to the current guide screen is acquired and there is a guide screen that the user is not facing among the multiple guide screens, turn guidance information is displayed on the current guide screen.

[0054] After completing the acquisition of the point cloud data corresponding to the current guide screen, the fitting system can detect whether there is a guide screen among the multiple guide screens that the user is not facing. If there is a guide screen among the multiple guide screens that the user is not facing, it means that the fitting system has not collected the point cloud data on the side of the user. At this time, the turning guidance information can be displayed on the current guide screen.

[0055] The above-mentioned turning guidance information may include text, graphics, etc., which are used to prompt the user to turn to any non-directional guidance screen.

[0056] In some embodiments of the present application, the fitting system can determine whether there is a guide screen that the user is not facing based on the number of times point cloud data is collected and the total number of guide screens. It can also save the information of the guide screen that the user is facing after each acquisition of the point cloud data corresponding to the current guide screen, and then determine whether there is a guide screen that the user is not facing.

[0057] Step S304, re-determine the current guide screen that the user is currently facing among the multiple guide screens, obtain the point cloud data corresponding to the current guide screen collected by the collection device, until the point cloud data corresponding to the current guide screen is obtained, and when there is no guide screen that the user is not facing among the multiple guide screens, obtain the point cloud data corresponding to different sides of the user.

[0058] The fitting system can re-photograph the user in the collection area through the camera to determine whether the user is facing another guide screen. If so, the current guide screen that the user is currently facing among the multiple guide screens can be re-determined, and the point cloud data corresponding to the current guide screen collected by the collection device can be obtained, and so on, until the point cloud data corresponding to the current guide screen is obtained, and when there is no guide screen among the multiple guide screens that the user is not facing, the point cloud data corresponding to different sides of the user are obtained.

[0059] like Figure 2 As shown, in some embodiments, the fitting system may include guide screen A, guide screen B, guide screen C, and guide screen D. Guide screen A can guide the user to collect data on the front of the human body, guide screen B can guide the user to collect data on the left side of the human body, guide screen C can guide the user to collect data on the back of the human body, and guide screen D can guide the user to collect data on the back of the human body. When the user faces guide screen A and completes data collection on the front of the human body, a turning guidance message may be displayed on guide screen A, prompting the user to turn to one of guide screens B, C, and D, and the point cloud data may be collected again after the user turns. Finally, the fitting system can obtain point cloud data on the front, left, back, and right sides of the human body, respectively.

[0060] In other embodiments of the present application, the fitting system may also broadcast guidance information through a voice output device to obtain point cloud data collected by the collection device when the user sequentially turns different sides toward the collection device.

[0061] Step S102: determining a first three-dimensional model of the user based on point cloud data corresponding to different sides of the user.

[0062] In the implementation of the present application, after collecting the point cloud data corresponding to each side of the user, the fitting system can denoise and segment the background of the point cloud data to obtain the point cloud data of the foreground part, and perform point cloud splicing and point cloud registration according to the characteristic points of the human body trunk, head, etc. in the foreground part, and then segment it into point clouds corresponding to the limbs through the point cloud segmentation algorithm. The point cloud of each part is processed separately to obtain the sub-model of each part, and then the sub-models of each part are combined using the combination algorithm to obtain the user's three-dimensional human body model.

[0063] It should be noted that since the point cloud data is generally obtained by scanning when the user is wearing clothes, the fitting system can determine the second three-dimensional model based on the point cloud data corresponding to different sides of the user, and obtain the clothing category of the clothes to be replaced that the user is currently wearing, so as to determine the reduction value of each point on the second three-dimensional model according to the clothing category, and then, correct the second three-dimensional model based on the reduction value to obtain the first three-dimensional model of the user.

[0064] Specifically, the fitting system can use the image data captured by the camera and the point cloud data scanned by the laser radar to identify the type of clothing, perform algorithmic processing on the identification results, determine the parts of the human body covered by the clothing, and then match them with the second three-dimensional model to obtain the points that need to be corrected. At the same time, the morphological characteristics of the clothing can be judged, and combined with the clothing category, it can be determined whether the clothing is in a contracted state or an expanded state. By matching the above-mentioned clothing category, morphological characteristics and other data with the preset reduction value library, the reduction value A of each point can be obtained.

[0065] In addition, the fitting system can also identify key nodes of the human skeleton, such as shoulders, elbows, hips, knees, etc., and then identify the height of the human body. By matching the height with the preset value library of human body shape, the body shape reference value B can be obtained and the minimum point cloud model range Bf can be calculated.

[0066] The second three-dimensional model is reduced according to the reduction value A and the reference value B, and then judged in combination with the minimum point cloud model range Bf, and the human body parts that do not meet the minimum point cloud model range Bf are continued to be repaired until the minimum point cloud model range Bf is met, thereby obtaining the first three-dimensional model of the unclothed human body.

[0067] Step S103: determining and displaying a fitting model based on the first three-dimensional model and the first clothing model of the clothing to be tried on.

[0068] In an embodiment of the present application, after obtaining the first three-dimensional model of the user, the fitting system may enter a virtual fitting session of the user.

[0069] Specifically, the fitting system can pre-store clothing models of clothing in stores, and according to the selection operation of the user or the merchant on the clothing to be tried on, merge the first clothing model of the clothing to be tried on that matches the user's body shape and the first three-dimensional model of the user to obtain the fitting model. Then, the fitting model can be displayed on the guide screen.

[0070] In an embodiment of the present application, in response to detecting that a user has entered a collection area formed by multiple guide screens, the fitting system can output guidance information to obtain point cloud data collected by the collection device when the user turns different sides toward the collection device in turn, and then, according to the point cloud data corresponding to different sides of the user, the first three-dimensional model of the user is determined, and based on the first three-dimensional model and the first clothing model of the clothes to be tried on, the fitting model is determined and displayed, so that the fitting system does not need to install laser radars at different positions, nor does it need to configure a complex rotating structure for the laser radar. By using a collection device installed in a fixed position, point cloud data from different sides of the user can be collected, which reduces the hardware cost of the fitting system and facilitates the application of the fitting system.

[0071] In the implementation manner of the present application, after entering the virtual fitting session, the fitting system can continue to collect the user's image and point cloud data to perform diversified fitting demonstrations.

[0072] In order to provide users with a better demonstration effect, in some implementations of the present application, the above step S103 may also include the following steps S401 to S403.

[0073] Step S401, obtaining the user's posture data at multiple collection moments.

[0074] Specifically, the fitting system can detect the human body posture based on the point cloud data and user image collected by the acquisition device through the human body posture detection model Ma, and obtain the human body posture Z. According to the detection results of multiple frames before and after, the posture data of the user at multiple acquisition times is determined to obtain the human body posture sequence [Z1, Z2, ..., Zn].

[0075] At the same time, the fitting system can also use the user behavior algorithm Mb to process the human posture sequence and determine the user's current behavior.

[0076] Step S402: Correct the first three-dimensional model and the first clothing model according to the posture data to obtain a third three-dimensional model corresponding to the first three-dimensional model and a second clothing model corresponding to the first clothing model.

[0077] In an embodiment of the present application, based on the user's posture data at multiple acquisition moments, the fitting system can respectively determine the points that need to be corrected on the first three-dimensional model and the first clothing model, as well as the correction amount of each point that needs to be corrected.

[0078] Specifically, the fitting system can determine the actual position of the point cloud of each part in the spatial coordinate system based on the posture data and the point cloud segmentation algorithm, and then determine the points that need to be corrected in the first three-dimensional model and the first clothing model, as well as the correction amount of each point that needs to be corrected. Then, based on the demonstration effect algorithm corresponding to the user's current behavior, the points that need to be corrected and the corresponding correction amount to achieve the display effect of the clothing model are further determined, so as to correct the first three-dimensional model and the first clothing model.

[0079] For example, if the user's current behavior is squatting, the fitting system can determine the actual position of the point cloud of each part in the spatial coordinate system based on the posture data and the point cloud segmentation algorithm Mc. The points of each part of the first three-dimensional model and the first clothing model are rotated and translated to the corresponding spatial coordinate system positions. In addition, the clothing model can be scaled to adapt to the three-dimensional model of the human body. Then, according to the posture data and the wrinkle algorithm Md corresponding to the squatting behavior, the points and corresponding correction amounts required to achieve the wrinkle change of the clothing model are determined. After the correction, the third three-dimensional model and the second clothing model corresponding to the user squatting can be obtained.

[0080] For another example, if the user's current behavior is walking, the fitting system can determine the actual position of the point cloud of each part in the spatial coordinate system based on the posture data and the point cloud segmentation algorithm Mc. The points of each part of the first three-dimensional model and the first clothing model are rotated and translated to the corresponding spatial coordinate system positions, and the clothing model can be scaled to fit the three-dimensional model of the human body. Then, according to the swaying algorithm Me corresponding to the posture data and the walking behavior, the points and corresponding correction amounts required to achieve the swinging change of the clothing model are determined. After the correction, the third three-dimensional model and the second clothing model corresponding to the user's walking can be obtained.

[0081] Step S403: merge the third three-dimensional model and the second clothing model to obtain and display a fitting model.

[0082] In an implementation manner of the present application, by acquiring the user's posture data at multiple acquisition times and correcting the first three-dimensional model and the first clothing model based on the posture data, the fitting system can adjust the model according to the user's current posture, achieve a display effect corresponding to the user's posture, and improve the user's fitting experience.

[0083] In actual applications, based on the guidance screen of the fitting system, the fitting system can also perform human-computer interaction with the user.

[0084] Specifically, the fitting system can recognize the user's gestures at multiple acquisition moments, obtain gesture recognition results, and update the fitting model according to the gesture recognition results.

[0085] In some embodiments of the present application, the fitting system can perform algorithmic processing on the user image, segment the human hand image t, and use continuous multiple frames of images to form a gesture image sequence tl = [t1, t2, ..., t n ]. Similarly, the fitting system can process the point cloud data, segment the point cloud d of the human hand, and form a point cloud sequence dl = [d1, d2, ..., d n ]. The gesture image sequence tl and the point cloud sequence dl are fused using the fusion algorithm Fa and processed by the gesture recognition algorithm Fb to obtain the gesture result R.

[0086] The working process of the fusion algorithm Fa and the gesture recognition algorithm Fb includes: using the calibrated external parameter matrix between the camera coordinate system and the laser coordinate system to convert the laser coordinate system to the camera coordinate system, and then fusing the user image rgb data with the point cloud data xyz data to obtain the xyzrgb six-dimensional data td. Based on the six-dimensional data of multiple consecutive frames, the six-dimensional sequence [td1, td2, ..., td n ]. Then, the displacement values ​​w at adjacent acquisition moments are determined according to the position difference between the previous and next frames of the six-dimensional sequence, and merged into the six-dimensional sequence to obtain a seven-dimensional data sequence tdw containing rgbxyz and displacement values, and then the feature matrix Mtdw is obtained. For example, if 50 frames of point cloud data and user images are collected, a 50*7 feature matrix Mtdw can be formed.

[0087] The sample feature matrix is ​​one-hot encoded through the pre-designed N gesture categories to obtain the gesture category vector S. The output of the model to be trained is connected to the gesture category vector S for iterative training to obtain the trained neural network model Ms. Then the feature matrix Mtdw is input into the trained neural network model Ms to obtain the gesture recognition result R.

[0088] At this time, based on the gesture recognition result, the fitting system can update the fitting model. For example, the clothing model pointed to by the gesture recognition result can be obtained, and the user's first three-dimensional model and the clothing model pointed to by the gesture recognition result can be merged to obtain a new fitting model, so as to update the fitting model and achieve the effect of changing clothes.

[0089] At the same time, the fitting system can also realize human-computer interaction functions such as page turning, zooming in and out based on the gesture recognition result R.

[0090] In some embodiments of the present application, the fitting system may also provide different display modes based on multiple guide screens to meet the user's diverse display effects.

[0091] Specifically, in some embodiments of the application, when the orientation of the acquisition device is the same as the orientation of one of the plurality of guide screens, such as Figure 5 As shown, the above-mentioned display of the fitting model may include the following steps S501 to S503.

[0092] Step S501: determining a current guide screen, which the user is currently facing, among multiple guide screens.

[0093] The method of determining the current guide screen that the user is currently facing can be found in the description of the aforementioned step S301, and this application will not elaborate on this.

[0094] Step S502: Determine a first display angle for displaying the fitting model on the current guide screen.

[0095] Among them, the first display angle can be used to rotate the fitting model, thereby displaying the side of the acquisition device that the user is currently facing in the fitting model.

[0096] Step S503: displaying a fitting model in the current guide screen according to the first display angle.

[0097] In some embodiments, the fitting system can store the mapping relationship between the guide screen, the side facing the acquisition device, and the corresponding display angle in the memory in advance. After determining the current guide screen that the user is currently facing, the fitting system can obtain the first display angle according to the mapping relationship.

[0098] In other embodiments, the fitting system may also capture user images through a capture device, and then identify target feature points on the side of the user currently facing the capture device, and then determine the required rotation angle based on the positional relationship between the reference feature points and the target feature points on the fitting model, and use the angle as the first display angle.

[0099] The reference feature point may be the center of the front of the user's body or the center of the front of the user's face.

[0100] The fitting system rotates the fitting model according to the first display angle, and then displays the rotated fitting model on the current guide screen.

[0101] In the implementation manner of the present application, by displaying the rotated fitting model, the user can see the fitting effect facing the side of the acquisition device on the current guide screen. Therefore, when the user needs to view the fitting effect on the left, right or back, he only needs to face the corresponding guide screen, which can improve the user's fitting experience.

[0102] In some embodiments of the present application, Figure 6 As shown, the above-mentioned display of the fitting model may include the following steps S601 to S603.

[0103] Step S601: determining a target guide screen located within a user's field of view among multiple guide screens.

[0104] In some embodiments of the present application, the fitting system can capture the user's image through a camera, and determine the orientation of the user's face based on the user image to determine the user's field of view, and then select a target guide screen within the user's field of view from multiple guide screens to display the fitting model.

[0105] It should be noted that the number of the target guide screens may be one or more. If the number of the target guide screens is multiple, the fitting model may be displayed on each target guide screen.

[0106] Step S602: determining a second display angle of the fitting model on each target guide screen.

[0107] The second display angle is also used to rotate the fitting model. Unlike the first display angle, the second display angle is used to display the side of the corresponding target guide screen that the user is currently facing in the fitting model.

[0108] Step S603: displaying a fitting model on each target guide screen according to the second display angle.

[0109] Specifically, the fitting system may calculate the angle between each target guide screen and the user's face orientation and determine the second display angle of the fitting model on each target guide screen according to the angle.

[0110] The fitting system rotates the fitting model according to the second display angle, and then displays the rotated fitting model on the corresponding guide screen. By displaying the rotated fitting model, the user can see the fitting effect facing the side of the guide screen on the guide screen.

[0111] Figure 7 Shows Figure 2 The top view of the fitting system shown in FIG. Figure 7 For example, when the user's field of view is POQ, the fitting system can Figure 2 The guide screen A and guide screen B in the figure are determined as the target guide screens, and the angle α between the user's face direction OO' and the guide screen A and the angle β between the user's face direction OO' and the guide screen B are calculated. At this time, it can be obtained that the display angle of the fitting model displayed on the guide screen A is 90°-α, the fitting model is rotated 90°-α, and the rotated fitting model is displayed on the guide screen A. Similarly, it can be obtained that the display angle of the fitting model displayed on the guide screen B is 90°-β, the fitting model is rotated 90°-β, and the rotated fitting model is displayed on the guide screen B.

[0112] In addition, when there are multiple clothes to be tried on, the fitting system can also merge the first three-dimensional model with the first clothing model of each clothing to be tried on, to obtain a fitting model corresponding to each clothing to be tried on, and display different fitting models corresponding to different clothing to be tried on each guide screen.

[0113] For example, a fitting model corresponding to top A may be displayed on guide screen A, and a fitting model corresponding to top B may be displayed on guide screen B.

[0114] Combination Figure 6 With the display method shown, the user can display different fitting models in multiple target guide screens within the field of view, thereby improving the user's fitting efficiency.

[0115] And, combined with Figure 4 In the display mode shown, the fitting system can also determine whether the user is in different postures in multiple acquisition time periods. If so, it can perform model correction and model fusion based on the posture data of multiple acquisition moments in each acquisition time period to obtain the fitting model corresponding to each posture, and then display the fitting models corresponding to different postures in each guide screen.

[0116] In order to improve the shopping efficiency of users, the fitting system can also display payment information on the guide screen. When the user performs the payment operation, the computing device of the fitting system will receive the order information. At the same time, the computing device can send the order information to the front desk device to remind the staff to ship the goods and complete the shopping process.

[0117] Among them, the user's payment operations may include but are not limited to: paying by scanning the QR code on the guide screen through payment terminals such as smartphones and tablets, paying through devices that support near field communication technology (NFC), and paying through the point of sales terminal (POS) information management system configured in the fitting system.

[0118] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0119] like Figure 8 The figure is a schematic structural diagram of a fitting device 800 provided in an embodiment of the present application. The fitting device 800 is configured on a fitting system. The fitting system may include a plurality of guide screens and a collection device facing a collection area enclosed by the plurality of guide screens.

[0120] Specifically, the fitting device 800 may include:

[0121] An information output unit 801 is used to output guidance information in response to detecting that a user enters the collection area, so as to obtain point cloud data respectively collected by the collection device when the user sequentially points different sides toward the collection device;

[0122] A three-dimensional modeling unit 802, configured to determine a first three-dimensional model of the user according to point cloud data corresponding to different sides of the user;

[0123] The fitting unit 803 is used to determine and display a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on.

[0124] In some embodiments of the present application, the orientation of the acquisition device is the same as the orientation of one of the multiple guide screens; accordingly, the fitting unit 803 can be specifically used to: determine a current guide screen to which the user is currently facing among the multiple guide screens; determine a first display angle for displaying the fitting model on the current guide screen, the first display angle being used to display the side of the acquisition device to which the user is currently facing in the fitting model; and display the fitting model in the current guide screen according to the first display angle.

[0125] In some embodiments of the present application, the fitting system may include multiple guide screens; the fitting unit 803 may be specifically used to: determine a target guide screen located within the user's field of view among the multiple guide screens; determine a second display angle of the fitting model on each of the target guide screens, the second display angle being used to display the side of the corresponding target guide screen that the user is currently facing in the fitting model; and display the fitting model on each of the target guide screens according to the second display angle.

[0126] In some embodiments of the present application, there are multiple models of the first clothing; the fitting unit 803 can be specifically used to: fuse each of the first clothing models with the first three-dimensional model respectively to obtain a fitting model corresponding to each of the first clothing models; and display different fitting models corresponding to the first clothing models on each of the multiple guide screens respectively.

[0127] In some embodiments of the present application, the orientation of the acquisition device is the same as the orientation of one of the multiple guide screens; the information output unit 801 can be specifically used to: display guidance information on the guide screen that the user is facing when the user faces each guide screen in turn, so as to obtain point cloud data collected by the acquisition device when the user turns different sides toward the acquisition device in turn.

[0128] In some embodiments of the present application, the above-mentioned information output unit 801 can be specifically used to: determine the current guide screen that the user is currently facing among the multiple guide screens; obtain point cloud data corresponding to the current guide screen collected by the acquisition device; when the point cloud data corresponding to the current guide screen is obtained, and there is a guide screen that the user is not facing among the multiple guide screens, display turning guidance information on the current guide screen, and the turning guidance information is used to prompt the user to turn to any guide screen that the user is not facing; re-determine the current guide screen that the user is currently facing among the multiple guide screens, and obtain point cloud data corresponding to the current guide screen collected by the acquisition device, until the point cloud data corresponding to the current guide screen is obtained, and there is no guide screen that the user is not facing among the multiple guide screens, then obtain point cloud data corresponding to different sides of the user.

[0129] In some embodiments of the present application, the above-mentioned information output unit 801 can be specifically used to: detect whether the user meets the point cloud acquisition conditions; if not, output posture guidance information on the current guide screen, and the posture guidance information is used to prompt the user to adjust his or her posture; re-detect whether the user meets the point cloud acquisition conditions until it is detected that the user meets the point cloud acquisition conditions, and obtain the point cloud data corresponding to the current guide screen collected by the acquisition device.

[0130] In some embodiments of the present application, the above-mentioned three-dimensional modeling unit 802 can be specifically used to: determine the second three-dimensional model based on the point cloud data corresponding to different sides of the user; obtain the clothing category of the clothes to be replaced currently worn by the user, and determine the reduction value of each point on the second three-dimensional model according to the clothing category; correct the second three-dimensional model based on the reduction value to obtain the first three-dimensional model of the user.

[0131] In some embodiments of the present application, the fitting unit 803 may be specifically used to: obtain posture data of the user at multiple acquisition times; according to the posture data, respectively correct the first three-dimensional model and the first clothing model to obtain a third three-dimensional model corresponding to the first three-dimensional model and a second clothing model corresponding to the first clothing model; merge the third three-dimensional model and the second clothing model to obtain and display the fitting model.

[0132] In some embodiments of the present application, the fitting unit 803 may be specifically used to: recognize the user's gestures at multiple acquisition moments to obtain gesture recognition results; and update the fitting model according to the gesture recognition results.

[0133] It should be noted that, for the convenience and simplicity of description, the specific working process of the fitting device 800 can be referred to Figures 1 to 7 The corresponding process of the method will not be repeated here.

[0134] like Fig. 9 Shown is a schematic diagram of a fitting system provided in an embodiment of the present application.

[0135] The fitting system may include: a plurality of guide screens 90 , a collection device 91 facing a collection area enclosed by the plurality of guide screens, and a computing device 92 .

[0136] The computing device 92 may include a processor 920, a memory 921, and a computer program 922 stored in the memory 921 and executable on the processor 920, such as a fitting program. When the processor 920 executes the computer program 922, the steps in the above-mentioned fitting method embodiments are implemented, such as Figure 1 Alternatively, when the processor 920 executes the computer program 922, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 8 The information output unit 801, the three-dimensional modeling unit 802 and the fitting unit 803 are shown.

[0137] The computer program may be divided into one or more modules / units, which are stored in the memory 921 and executed by the processor 920 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the system.

[0138] For example, the computer program may be divided into: an information output unit, a three-dimensional modeling unit, and a fitting unit.

[0139] The specific functions of each unit are as follows: an information output unit, used to output guidance information in response to detecting that a user enters the collection area, so as to obtain the point cloud data collected by the collection device when the user sequentially turns different sides toward the collection device; a three-dimensional modeling unit, used to determine a first three-dimensional model of the user according to the point cloud data corresponding to different sides of the user; a fitting unit, used to determine and display a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on.

[0140] The system may include, but is not limited to, a processor 920 and a memory 921. Those skilled in the art will appreciate that Fig. 9It is only an example of a fitting system and does not constitute a limitation of the fitting system. The fitting system may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the system may also include input and output devices, network access devices, buses, etc.

[0141] For example, in some embodiments of the present application, the fitting system may further include a plurality of guide screens, each of which may be used to display guide information and a fitting model.

[0142] The processor 920 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0143] The memory 921 may be an internal storage unit of the system, such as a hard disk or memory of the system. The memory 921 may also be an external storage device of the system, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the system. Further, the memory 921 may also include both an internal storage unit of the system and an external storage device. The memory 921 is used to store the computer program and other programs and data required by the system. The memory 921 may also be used to temporarily store data that has been output or is to be output.

[0144] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0145] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0147] In the embodiments provided in the present application, it should be understood that the disclosed devices / systems and methods can be implemented in other ways. For example, the device / system embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0151] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for fitting clothes, characterized in that: Applied to a fitting system, the fitting system comprises a plurality of guide screens, and a collection device facing a collection area enclosed by the plurality of guide screens, wherein the direction of the collection device is the same as the direction of one of the plurality of guide screens; The fitting method comprises: In response to detecting that the user enters the acquisition area, outputting guidance information to obtain point cloud data respectively acquired by the acquisition device when the user sequentially turns different sides toward the acquisition device; Determining a first three-dimensional model of the user according to the point cloud data corresponding to different sides of the user; Determine and display a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on; The output guidance information is used to obtain point cloud data respectively collected by the collection device when the user turns different sides toward the collection device in turn, including: determining a current guide screen that the user is currently facing among the multiple guide screens; obtaining point cloud data corresponding to the current guide screen collected by the collection device; when the point cloud data corresponding to the current guide screen is obtained and there is a guide screen that the user is not facing among the multiple guide screens, displaying turning guidance information on the current guide screen, the turning guidance information is used to prompt the user to turn toward any guide screen that is not facing; re-determining the current guide screen that the user is currently facing among the multiple guide screens, and obtaining point cloud data corresponding to the current guide screen collected by the collection device, until the point cloud data corresponding to the current guide screen is obtained, and there is no guide screen that the user is not facing among the multiple guide screens, then the point cloud data corresponding to different sides of the user are obtained.

2. The fitting method according to claim 1, wherein: The direction of the acquisition device is the same as the direction of one of the plurality of guide screens; The display fitting model comprises: Determine a current guide screen, among the plurality of guide screens, to which the user is currently pointing; Determine a first display angle for displaying the fitting model on the current guide screen, wherein the first display angle is used to display the side of the fitting model where the user is currently facing the acquisition device; The fitting model is displayed in the current guide screen according to the first display angle.

3. The fitting method according to claim 1, wherein: The display fitting model further comprises: Determine a target guide screen located within the user's field of view among the plurality of guide screens; Determining a second display angle of the fitting model on each of the target guide screens, wherein the second display angle is used to display the side of the corresponding target guide screen that the user in the fitting model is currently facing; The fitting model is displayed on each of the target guide screens according to the second display angle.

4. The fitting method according to claim 1, wherein: The number of the first clothing models is multiple, and the determining and displaying the fitting model based on the first three-dimensional model and the first clothing model of the clothing to be tried on further includes: fusing each of the first clothing models with the first three-dimensional model to obtain a fitting model corresponding to each of the first clothing models; A different fitting model corresponding to the first clothing model is displayed on each of the plurality of guide screens.

5. The fitting method according to claim 1, wherein: The acquiring the point cloud data corresponding to the current guide screen acquired by the acquisition device includes: Detecting whether the user meets the point cloud acquisition conditions; If not, the posture guidance information is output on the current guidance screen, and the posture guidance information is used to prompt the user to adjust his / her posture; Re-detecting whether the user meets the point cloud acquisition conditions until it is detected that the user meets the point cloud acquisition conditions, and acquiring point cloud data corresponding to the current guide screen collected by the acquisition device.

6. The fitting method according to any one of claims 1 to 5, characterized in that: Determining the first three-dimensional model of the user according to the point cloud data corresponding to different sides of the user includes: Determining a second three-dimensional model based on the point cloud data corresponding to different sides of the user; Acquire the clothing category of the clothing to be replaced that the user is currently wearing, and determine the reduction value of each point on the second three-dimensional model according to the clothing category; The second three-dimensional model is modified based on the reduction value to obtain a first three-dimensional model of the user.

7. The fitting method according to any one of claims 1 to 5, characterized in that: The determining and displaying a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on comprises: Acquiring posture data of the user at multiple collection moments; According to the posture data, respectively modifying the first three-dimensional model and the first clothing model to obtain a third three-dimensional model corresponding to the first three-dimensional model and a second clothing model corresponding to the first clothing model; The third three-dimensional model and the second clothing model are merged to obtain and display the fitting model.

8. The fitting method according to any one of claims 1 to 5, characterized in that: After determining and displaying the fitting model, the method further includes: Recognizing the gestures of the user at multiple collection moments to obtain gesture recognition results; The fitting model is updated according to the gesture recognition result.

9. A fitting device, characterized in that: Configured in a fitting system, the fitting system comprises a plurality of guide screens, and a collection device facing a collection area enclosed by the plurality of guide screens, the collection device facing the same direction as one of the plurality of guide screens; The fitting device comprises: an information output unit, configured to output guidance information in response to detecting that a user enters the collection area, so as to obtain point cloud data respectively collected by the collection device when the user sequentially turns different sides toward the collection device; A three-dimensional modeling unit, configured to determine a first three-dimensional model of the user according to point cloud data corresponding to different sides of the user; a fitting unit, configured to determine and display a fitting model based on the first three-dimensional model and a first clothing model of the clothing to be tried on; The information output unit is used to: determine the current guide screen that the user is currently facing among the multiple guide screens; obtain point cloud data corresponding to the current guide screen collected by the acquisition device; when the point cloud data corresponding to the current guide screen is obtained and there is a guide screen that the user is not facing among the multiple guide screens, display turning guidance information on the current guide screen, and the turning guidance information is used to prompt the user to turn to any guide screen that the user is not facing; re-determine the current guide screen that the user is currently facing among the multiple guide screens, and obtain point cloud data corresponding to the current guide screen collected by the acquisition device, until the point cloud data corresponding to the current guide screen is obtained and there is no guide screen that the user is not facing among the multiple guide screens, then obtain point cloud data corresponding to different sides of the user.

10. A fitting system, characterized in that: The method comprises a computing device, a plurality of guide screens, and a collection device facing a collection area enclosed by the plurality of guide screens. The computing device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the fitting method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the fitting method according to any one of claims 1 to 8 are implemented.

Images