Digital retail system based on human-computer interaction and operation method

Abstract

The invention is suitable for the technical field of information and the technical field of artificial intelligence, and provides a digital retail system based on human-computer interaction and an operation method.The digital retail system based on human-computer interaction comprises a cloud server platform and AR equipment used for human-computer interaction, and the cloud server platform stores a virtual commodity module and a virtual mall module. The AR device is connected with the cloud server platform and the virtual human body module; the virtual shopping mall module is used for processing the building data through an artificial intelligence engine, generating a virtual shopping mall model, determining a placement score of the virtual model of the target commodity through a scoring model, and deploying the virtual model of the target commodity when the placement score meets a preset condition; and the AR equipment is used for importing the human body model of the user from the personal data storage system in an offline mode, mapping the body feature data of the user to the virtual try-on model, and obtaining a panoramic image of the target commodity try-on by the user. The panoramic image of the target commodity tried by the user can be generated.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of information technology and artificial intelligence, and particularly relates to a digital retail system based on human-computer interaction and a running method. BACKGROUND

[0002] The target commodity refers to a commodity promoted by an enterprise. The target commodity has a high demand for try-on, and a user needs to personally experience the fitting degree of the target commodity with the user.

[0003] However, the digital retail system mainly relies on two-dimensional pictures and simple video displays when presenting the target commodity. The simple video display usually only follows a predetermined script and process to perform a regular display of the target commodity, and cannot provide panoramic images of the user trying on the target commodity, which is not conducive to improving the user's shopping experience. Therefore, how the digital retail system generates panoramic images of the user trying on the target commodity is a technical problem to be solved. SUMMARY

[0004] The purpose of the embodiments of the application is to provide a digital retail system based on human-computer interaction, which aims to solve the technical problem of how the digital retail system generates panoramic images of the user trying on the target commodity.

[0005] In a first aspect, the embodiments of the application provide a digital retail system based on human-computer interaction, which comprises a cloud server platform and an AR device for human-computer interaction, the cloud server platform stores a virtual commodity module and a virtual mall module, and the AR device is connected to the cloud server platform and a virtual human body module respectively; The virtual commodity module is configured to generate a first matching degree by using a predefined first matching model, select a virtual model of the target commodity as a modeling model according to the first matching degree, and send a running instruction to the virtual mall; The virtual mall module is configured to obtain building data of a physical mall according to the running instruction, call an artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data by the artificial intelligence engine, generate a virtual mall model, determine a placement score of the virtual model of the target commodity by using a predefined scoring model, and deploy the virtual model of the target commodity when the placement score meets a preset condition. The virtual human body module is configured to generate a second matching degree by using a predefined second matching model, select an initial human body model as a user human body model according to the second matching degree, and transmit the user human body model to a personal data storage system; The AR device is configured to load the virtual model of the target commodity from a store of the virtual mall model, import the user human body model from the personal data storage system in an offline manner, determine a virtual try-on model of the target commodity, map the body feature data of the user to the virtual try-on model, and obtain panoramic images of the user trying on the target commodity.

[0006] In the embodiments of the present application, the beneficial effects are in two aspects. On the one hand, the user body model is imported from the personal data storage system in an offline manner to determine the virtual fitting model of the target commodity, and the body feature data of the user is mapped to the virtual fitting model to obtain the panoramic image of the user wearing the target commodity, thereby solving the problem of how the digital retail system generates the panoramic image of the user wearing the target commodity. On the other hand, since the body data of different individuals is significantly different, the body feature data of the user is mapped to the virtual fitting model to virtually display the target commodity according to the body feature data of the user, so that the target commodity can be fitted to the user's body in a real way, and the realism of the panoramic image of the user wearing the target commodity can be significantly enhanced.

[0007] In a possible implementation manner of the first aspect, the virtual commodity module is specifically configured to: obtain a modeling model of the target commodity from an output file of a three-dimensional modeling software, sample the modeling model according to a sampling density parameter to obtain a data point set of the modeling model, send the sampling density parameter to a three-dimensional scanner, control the three-dimensional scanner to scan the surface of the target commodity according to the sampling density parameter, obtain scanned point cloud data of the target commodity from an output file of the three-dimensional scanner, and use a predefined first matching model to process the least mean square error between the feature point set of the modeling model and the scanned point cloud data to generate a first matching degree. When the first matching degree is greater than a first preset value, the modeling model is selected as the virtual model of the target commodity, a running instruction is sent to the virtual mall, and the first matching degree is the matching degree between the feature point set of the modeling model and the scanned point cloud data. The value range of the first matching degree is 0 to 1. The closer the first matching degree is to 1, the more accurate the matching between the feature point set of the modeling model and the scanned point cloud data is, and the higher the applicability of the modeling model is. The farther the first matching degree is from 1, the less accurate the matching between the feature point set of the modeling model and the scanned point cloud data is, and the lower the applicability of the modeling model is.

[0008] In a possible implementation manner of the first aspect, the virtual human body module is specifically configured to receive an instruction indication, obtain initial human body parameter information according to the instruction indication, obtain an initial height parameter, an initial weight parameter and an initial three-circumference parameter from the initial human body parameter information, construct an initial human body model according to the initial height parameter, the initial weight parameter and the initial three-circumference parameter, obtain an initial human body vector of the initial human body model, send a collection instruction to the 3D modeling device, control the 3D modeling device to collect the height, the weight, the bust circumference, the waist circumference and the hip circumference of the user according to the collection instruction, obtain a current human body vector of the user from an output file of the 3D modeling device, process the initial human body vector and the current human body vector by using a predefined second matching model, generate a second matching degree, when the second matching degree is greater than a second preset value, select the initial human body model as a user human body model, save the user human body model to a storage file, and send a running instruction to the virtual mall module, the second matching degree is a matching degree between the initial human body vector and the current human body vector, the second matching degree ranges from 0 to 1, the closer the second matching degree is to 1, the more accurate the matching between the initial human body vector and the current human body vector is, and the higher the applicability of the initial human body model is, and the farther the second matching degree is from 1, the less accurate the matching between the initial human body vector and the current human body vector is, and the worse the applicability of the initial human body model is.

[0009] In a possible implementation manner of the first aspect, the virtual mall module is specifically configured to receive a running instruction, obtain building data of a physical mall according to the running instruction, call an artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data by using the artificial intelligence engine, generate a virtual mall model, import a virtual model of a target commodity into a store of the virtual mall model, obtain a visibility score, an orientation score and a proximity score of the virtual model of the target commodity, add the visibility score, the orientation score and the proximity score by using a predefined score model, to obtain a placement score of the virtual model of the target commodity, when the placement score is greater than a preset score, deploy the virtual model of the target commodity, the proximity score is a score of proximity between the virtual model of the target commodity and a customer flow, and the customer flow is a regular moving track of the customer. In a possible implementation manner of the first aspect, the AR device is specifically configured to access the store of the virtual mall model, when a try-on instruction is received, load the virtual model of the target commodity from the store of the virtual mall model, read the user human body model from the storage file, fuse the virtual model of the target commodity and the user human body model to obtain a virtual try-on model of the target commodity, collect body feature data of the user in real time by using a built-in motion capture system, map the body feature data of the user to the virtual try-on model to obtain an updated virtual try-on model, and perform material processing, lighting processing and animation processing on the updated virtual try-on model by using a real-time rendering engine to obtain a panoramic image of the user trying on the target commodity.

[0010] In a possible implementation manner of the first aspect, the artificial intelligence engine is a three-dimensional modeling engine based on artificial intelligence, the body feature data includes body posture data, joint angle data and gait data, the building data includes building structure diagrams and decoration diagrams, the target commodity includes clothing commodities and make-up commodities, and the personal data storage system includes a U disk, a local private cloud or a personal network disk.

[0011] In a possible implementation manner of the first aspect, the first matching model is: ; is a first matching degree, is a minimum mean square error, is a maximum error.

[0012] In a possible implementation manner of the first aspect, the second matching model is: ; is a second matching degree; is an initial human body vector, is a current human body vector.

[0013] In a possible implementation manner of the first aspect, the scoring model is: ; is a placement score, V is a visibility score, A is a direction score, D is a proximity score, α is a first weight coefficient, β is a second weight coefficient, gamma is a third weight coefficient, V , A , D and all have a value range of 0 to 100.

[0014] In the second aspect, the embodiments of the present application provide a running method of a digital retail system based on human-computer interaction, including: The virtual commodity module adopts a predefined first matching model to process a minimum mean square error between a feature point set of a modeling model and scanned point cloud data, to generate a first matching degree, and when the first matching degree is greater than a first preset value, the modeling model is selected as a virtual model of a target commodity, and a running instruction is sent to a virtual mall; The virtual mall module calls an artificial intelligence engine according to the building data of the obtained physical mall, inputs the building data into the artificial intelligence engine, processes the building data through the artificial intelligence engine, generates a virtual mall model, adopts a predefined scoring model, adds the visibility score, the orientation score and the proximity score to obtain a placement score of the virtual model of the target commodity, and deploys the virtual model of the target commodity when the placement score is greater than a preset score; The virtual human body module is configured to generate a second matching degree by using a predefined second matching model, select an initial human body model as a user human body model according to the second matching degree, and transmit the user human body model to the personal data storage system; The AR device is configured to load the virtual model of the target commodity from a store of the virtual mall model, import the user human body model from the personal data storage system in an offline manner, determine a virtual try-on model of the target commodity, map the body feature data of the user to the virtual try-on model, obtain a panoramic image of the user trying on the target commodity, display the virtual try-on image through a display window, and control the haptic feedback module to output a feedback signal according to the material characteristics of the target commodity.

[0015] In the embodiments of the present application, the beneficial effects are in two aspects. On the one hand, the user human body model is imported from the personal data storage system in an offline manner, the virtual try-on model of the target commodity is determined, the body feature data of the user is mapped to the virtual try-on model, and the panoramic image of the user trying on the target commodity is obtained, which solves the problem of how the digital retail system generates the panoramic image of the user trying on the target commodity. In addition, since the body data of different individuals is significantly different, the body feature data of the user is mapped to the virtual try-on model, which can virtually display the target commodity according to the body feature data of the user. In this way, the target commodity can be fitted to the user's body in a real way, which can significantly enhance the realism of the panoramic image of the user trying on the target commodity. On the other hand, the haptic feedback module outputs a feedback signal according to the material characteristics of the target commodity, which enhances the immersion of the user trying on the target commodity. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a first structural block diagram of a digital retail system provided by the embodiments of the present application; Figure 2 is a second structural block diagram of a digital retail system provided by the embodiments of the present application; Figure 3 is an implementation flowchart of a running method provided by the embodiments of the present application. DETAILED DESCRIPTION

[0017] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application. Embodiment one

[0018] Reference Figure 1 , Figure 1 is the first structural diagram of the digital retail system provided by the embodiment of the present application, which is described in detail as follows: A digital retail system based on human-computer interaction, the digital retail system comprising a cloud server platform and an AR device for human-computer interaction, the cloud server platform storing a virtual commodity module and a virtual mall module, the AR device being connected to the cloud server platform and the virtual human body module respectively; The virtual commodity module is configured to generate a first matching degree by using a predefined first matching model, select a modeling model as a virtual model of a target commodity according to the first matching degree, and send a running instruction to the virtual mall; The virtual mall module is configured to obtain building data of a physical mall according to the running instruction, call an artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data by the artificial intelligence engine, generate a virtual mall model, determine a placement score of the virtual model of the target commodity by a predefined scoring model, and deploy the virtual model of the target commodity when the placement score meets a preset condition. The virtual human body module is configured to generate a second matching degree by using a predefined second matching model, select an initial human body model as a user human body model according to the second matching degree, and transmit the user human body model to a personal data storage system; The AR device is configured to load the virtual model of the target commodity from a store of the virtual mall model, import the user human body model from the personal data storage system in an offline manner, determine a virtual try-on model of the target commodity, map the user's body feature data to the virtual try-on model, and obtain a panoramic image of the user trying on the target commodity.

[0019] Among them, the AR device, the full name is Augmented Reality (Augmented Reality) device, is a kind of intelligent hardware that can fuse virtual digital information with real physical environment in real time.

[0020] Among them, the artificial intelligence engine is a three-dimensional modeling engine based on artificial intelligence, the body feature data includes body posture data, joint angle data and gait data, the building data includes building structure diagram and decoration diagram, the target commodity includes clothing commodity and makeup commodity, and the personal data storage system includes U disk, local private cloud or personal network disk.

[0021] The three-dimensional modeling engine based on artificial intelligence is a new tool system integrating artificial intelligence technology and traditional three-dimensional graphics. The three-dimensional modeling engine based on artificial intelligence reduces the technical threshold of three-dimensional modeling and improves the efficiency of three-dimensional modeling.

[0022] The body posture data refers to the relative position relationship and geometric arrangement characteristics of the head, neck, torso and limbs of the human body in three-dimensional space under static or quasi-static conditions.

[0023] The body posture data refers to a set of multi-dimensional information collected and quantified by sensors, cameras or motion capture systems in real time, which reflects the overall or local position, angle and motion trend of the human body in space. These data usually include the relative position, inclination angle, rotation amplitude and symmetry of each part of the body, which can objectively reflect the posture characteristics in standing, sitting, walking or exercising.

[0024] The joint angle data refers to a set of quantitative values used to accurately describe the relative position and motion state of each joint in the human body in space. The joint angle data is collected in real time by sensors and can reflect the specific angle changes of the joint in flexion, rotation, abduction and adduction.

[0025] The gait data refers to the dynamic change information of each joint and limb during walking. The gait data includes step length, step frequency and step speed.

[0026] In the embodiments of the present application, the beneficial effects are in two aspects. On the one hand, the user's body model is imported from the personal data storage system in an offline manner to determine the virtual fitting model of the target commodity, map the user's body feature data to the virtual fitting model, and obtain the panoramic image of the user trying on the target commodity, solving the problem of how the digital retail system generates the panoramic image of the user trying on the target commodity. On the other hand, since the body data of different individuals differs significantly, mapping the user's body feature data to the virtual fitting model can virtually display the target commodity according to the user's body feature data, so that the target commodity can fit the user's body in a real way, significantly enhancing the realism of the panoramic image of the user trying on the target commodity. Embodiment Two

[0027] Reference Figure 2 , Figure 2 The second structural diagram of the digital retail system provided by the embodiments of the present application is shown in FIG. 2, and the details are as follows: A digital retail system based on human-computer interaction, the digital retail system comprising a cloud server platform and an AR device for human-computer interaction, the cloud server platform storing a virtual commodity module, a virtual human body module connected to the virtual commodity module, and a virtual mall module connected to the virtual human body module; The virtual commodity module is specifically configured to obtain a modeling model of the target commodity from an output file of a three-dimensional modeling software, sample the modeling model according to a sampling density parameter to obtain a data point set of the modeling model, send the sampling density parameter to a three-dimensional scanner, control the three-dimensional scanner to scan a surface of the target commodity according to the sampling density parameter, obtain scanned point cloud data of the target commodity from an output file of the three-dimensional scanner, and use a predefined first matching model to process a minimum mean square error between a feature point set of the modeling model and the scanned point cloud data to generate a first matching degree. When the first matching degree is greater than a first preset value, the modeling model is selected as a virtual model of the target commodity, a running instruction is sent to a virtual mall, and the first matching degree is a matching degree between the feature point set of the modeling model and the scanned point cloud data. The first matching degree has a value range of 0 to 1. The closer the first matching degree is to 1, the more accurate the matching between the feature point set of the modeling model and the scanned point cloud data is, and the higher the applicability of the modeling model is. The farther the first matching degree is from 1, the less accurate the matching between the feature point set of the modeling model and the scanned point cloud data is, and the lower the applicability of the modeling model is.

[0028] The first matching model is: ; The first matching degree is The minimum mean square error is The maximum error is

[0029] For ease of illustration, the following examples are given: For example, the plurality of feature points in the feature point set of the modeling model are , , ; =(100 mm,200 mm,300 mm), =(120 mm,220 mm,320 mm), and =(130 mm,230 mm,330 mm); The plurality of feature points in the scanned point cloud data of the target commodity are , , ; =(102 mm,198 mm,301 mm), =(118 mm,223 mm,319 mm), =(131 mm,229 mm,328 mm), mm represents millimeter, the maximum acceptable error =100 set by the system, so ; ; When the first preset value is 0.9, greater than 0.9, confirming that the modeling model is effective, selecting the modeling model as the virtual model of the target commodity. It is shown that The virtual human module is specifically configured to receive an indication instruction, obtain initial human parameter information according to the indication instruction, obtain an initial height parameter, an initial weight parameter and an initial three-circumference parameter from the initial human parameter information, construct an initial human model according to the initial height parameter, the initial weight parameter and the initial three-circumference parameter, obtain an initial human vector of the initial human model, send a collection instruction to the 3D modeling device, control the 3D modeling device to collect the height, weight, chest circumference, waist circumference and hip circumference of the user according to the collection instruction, obtain a current human vector of the user from an output file of the 3D modeling device, process the initial human vector and the current human vector by using a predefined second matching model, and generate a second matching degree. When the second matching degree is greater than a second preset value, the initial human model is selected as a user human model, the user human model is saved to a storage file, and a running instruction is sent to the virtual mall module. The second matching degree is a matching degree between the initial human vector and the current human vector, the value range of the second matching degree is 0 to 1, the closer the second matching degree is to 1, the more accurate the matching between the initial human vector and the current human vector is, and the higher the applicability of the initial human model is. The farther the second matching degree is from 1, the less accurate the matching between the initial human vector and the current human vector is, and the worse the applicability of the initial human model is.

[0030] The second matching model is: ; The second matching degree is: The initial human vector is The current human vector is.

[0031] For the convenience of description, the following examples are given: For example: = [175,65,90,70,90], the units of 175, 65, 90, 70 and 90 are centimeters, kilograms, centimeters, centimeters and centimeters respectively; = [173.2,67.5,88.6,71.8,92.1], the units of 173.2, 67.5, 88.6, 71.8 and 92.1 are centimeters, kilograms, centimeters, centimeters and centimeters respectively, and and are substituted into the second matching model, ; When the second preset value is 0.8, greater than 0.8, confirming that the initial human body model is accurate and effective, and selecting the initial human body model as the user human body model.

[0032] The virtual mall module is specifically configured to receive a running instruction, obtain building data of a physical mall according to the running instruction, call an artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data through the artificial intelligence engine, generate a virtual mall model, import a virtual model of a target commodity into a store of the virtual mall model, obtain a visibility score, an orientation score and a proximity score of the virtual model of the target commodity, add the visibility score, the orientation score and the proximity score by using a predefined scoring model to obtain a placement score of the virtual model of the target commodity, and deploy the virtual model of the target commodity when the placement score is greater than a preset score. The proximity score is a score of the proximity between the virtual model of the target commodity and a customer flow, and the customer flow is a regular moving track of a customer. Optionally, the visibility score, the orientation score and the proximity score of the virtual model of the target commodity can be given by an expert group hired by the mall.

[0033] Exemplarily, obtaining the visibility score, the orientation score and the proximity score of the virtual model of the target commodity comprises: obtaining position coordinates and orientation information of the virtual model of the target commodity in the store of the virtual mall model, submitting the position coordinates and the orientation information to the expert group, obtaining first score data given by the expert group based on the position coordinates, obtaining second score data given by the expert group based on the orientation information, obtaining the visibility score of the virtual model of the target commodity and the proximity score of the virtual model of the target commodity from the first score data, and obtaining the orientation score of the virtual model of the target commodity from the second score data; or obtaining a visible range, orientation information and proximity information of the virtual model of the target commodity in the store of the virtual mall model, respectively processing the visible range, the orientation information and the proximity information by using a deep learning model to respectively obtain the visibility score, the orientation score and the proximity score of the virtual model of the target commodity.

[0034] The scoring model is: ; is the placement score, V is the visibility score, A is the orientation score, D is the proximity score, α is a first weight coefficient, β is a second weight coefficient, gamma is a third weight coefficient, V , A , D and all have a value range of 0 to 100.

[0035] For example: V For 80, A For 70, D For 60, the placement score is: ; When the preset score is 70, greater than 70, it is confirmed that the placement of the virtual model of the target commodity meets the requirements, and the virtual model of the target commodity is deployed.

[0036] The AR device is specifically configured to access a store of a virtual mall model, load a virtual model of a target commodity from the store of the virtual mall model when receiving a try-on instruction, read a user body model from a storage file, fuse the virtual model of the target commodity and the user body model to obtain a virtual try-on model of the target commodity, collect body feature data of the user in real time through a built-in motion capture system, map the body feature data of the user to the virtual try-on model to obtain an updated virtual try-on model, and perform material processing, lighting processing and animation processing on the updated virtual try-on model through a real-time rendering engine to obtain a panoramic image of the user trying on the target commodity. For ease of illustration, the specific implementation scheme is as follows: Step one: generating a virtual model The target commodity is digitally modeled using a three-dimensional modeling software; after the target commodity is actually produced, a global three-dimensional scanner is used to calibrate the modeling result in terms of precision, to ensure that the target commodity is highly consistent with the actual object in terms of size, appearance and details.

[0037] Optionally, the three-dimensional modeling software includes CLO, Style3D, 3DMAX and SolidWorks. CLO, Style3D, 3dsMax and SolidWorks are all professional software in the field of 3D design. The full Chinese name of CLO is 3D clothing design software.

[0038] The full Chinese name of Style3D is digital fashion 3D design platform.

[0039] The full Chinese name of 3ds Max is three-dimensional animation modeling and rendering software.

[0040] The full Chinese name of SolidWorks is three-dimensional computer-aided design software.

[0041] The initial body parameter information provided by the user is imported into an AI-driven body visualization tool to generate a preliminary initial body model; then, data supplement and model correction are performed in combination with a handheld three-dimensional scanner (see Figure 3 ), to realize high-precision one-to-one modeling of the body shape of the consumer.

[0042] Optionally, the body shape visualization tool uses Body Visualizer. The full Chinese name of Body Visualizer is Body Simulator. Body Visualizer can help users intuitively understand body shape changes and body structures.

[0043] Step 2: Build a virtual mall A. Import basic design data such as the building structure diagram and interior decoration diagram of the physical mall into the artificial intelligence engine. The artificial intelligence engine automatically generates a virtual mall model highly consistent with the real scene, ensuring the accurate restoration of the structural layout.

[0044] For example: 1. Based on the parameters on the construction drawings of a department store building, including floor height, number of floors, number of shops on each floor, and their corresponding length, width, and height, use the artificial intelligence engine to build a 3D rough model of the department store mall; Optionally, the artificial intelligence engine uses PartCrafter. The full Chinese name of PartCrafter is: Structured 3D Mesh Generation Model.

[0045] 2. Collect 3D models of interior effects during the decoration of all merchants and public areas; 3. Use modeling and rendering software to embed the 3D models of interior effects corresponding to the shops and public areas of the rough model of the department store mall into its rough model, and then perform color rendering to achieve the accurate restoration of the mall's structural layout.

[0046] B. Based on the shop numbers, allocate independent management permissions and virtual space coordinates to each settled shop. Merchants can independently adjust the position, orientation, and arrangement of the virtual models of target products through the human-computer interaction interface provided by the system to complete the virtual display layout.

[0047] C. Deploy the complete virtual mall model to the cloud server platform and set up an AI-driven product information administrator. The system can achieve 7×24-hour automatic review, update, listing, and delisting management to ensure that product data is always synchronized with the actual status.

[0048] The implementation process of the AI-driven shop information administrator is as follows: 1. The system receives the virtual models of target products uploaded by merchants and their supporting description information; 2. Based on the object recognition models of the YOLO series trained with big data, automatically recognize and classify the virtual models of target products, and extract their key feature tags; 3. Perform semantic parsing on the description information provided by merchants through the natural language processing model; 4. Compare the model identification result with the description information: if the matching degree is above the preset threshold, automatically pass the audit and update the product model information to the cloud server platform, if the matching degree does not meet the standard, trigger the abnormal feedback mechanism, the system automatically rejects uploading and prompts the merchant to modify the information or re-model.

[0049] Step three: load the AR device A. Before the AR device is connected to the digital retail system server, the system will prompt the user to clear the local old virtual human body model data, to avoid the possible data leakage risk.

[0050] B. After the AR device is connected to the cloud server platform, the user can choose to automatically or manually download / update the virtual mall product data; after the update is completed, the system automatically disconnects the cloud server platform connection, so that the AR device enters the offline state, and the possibility of user data exposure in the online state is eliminated from the source.

[0051] C. When the AR device is in the offline state, the user can import the virtual human body model into the device through a U disk, a local private cloud, or a personal private network disk, etc., to realize personalized try-on, try-out, etc. without the need for networking, and to effectively protect the privacy and data security of the user.

[0052] Among them, the virtual mall supports multi-person interaction, and the same AR host can connect multiple AR devices such as AR glasses and scene simulators, and multiple people can walk together in the virtual mall and show the try-on in real time to the same person, so that the try-on person can get real-time suggestions.

[0053] D. Support multi-person interaction, when the AR device has multi-person access function, it can enter the simulated mall at the same time, and the same person can watch the dynamic change process of the try-on, try-out in real time when trying on clothes and trying out makeup, and can restore the scene of trying on clothes and trying out makeup in the real world with high fidelity. The change process is provided by the 3D physics engine for real-time scene rendering and updating.

[0054] The use process of the AR device is as follows: 1. The user imports the personal virtual human body model into the AR device system; 2. The user wears the brain-computer interface of the AR device and puts on the full-body touch experience kit. Through the built-in motion capture system, the user's body posture, joint angle and gait data are collected in real time, and these data are mapped to the virtual human body model in real time, realizing high-precision correspondence between user actions and virtual try-on process; at the same time, the integrated tactile feedback module can apply electric stimulation or vibration feedback to different parts of the user's body according to the material properties of the virtual clothes, so as to simulate the real feeling of the contact between the clothes and the skin in the dressing process, and improve the immersion and judgment accuracy of the try-on; 3. After entering the virtual mall, the system will propose clothes of corresponding style for reference according to the personal preferences of the user; 4. When trying on clothes in the virtual mall, real dressing feeling can be achieved on the trunk and limbs; 5. The satisfied clothes are put into the virtual shopping cart, and after shopping and clearing the personal body virtual model, online ordering can be performed like in the mall.

[0055] In the embodiment of the present application, since the body data of different individuals has significant differences, mapping the body feature data of the user to the virtual fitting model can perform virtual display of the target commodity according to the body feature data of the user, so that the target commodity can be fitted to the user's body in a real way, which can significantly enhance the realism of the panoramic image of the user trying on the target commodity, thereby significantly improving the decision accuracy and trust of the user in the purchase process.

[0056] Embodiment three Reference Figure 3 , Figure 3 is the implementation flowchart of the running method provided by the embodiment of the present application, which is described in detail as follows: S301, the virtual commodity module adopts a predefined first matching model to process the minimum mean square error between the feature point set of the modeling model and the scanned point cloud data, generates a first matching degree, and when the first matching degree is greater than a first preset value, selects the modeling model as a virtual model of the target commodity, and sends a running instruction to the virtual mall; S302, the virtual mall module acquires the building data of the entity mall, calls an artificial intelligence engine, inputs the building data into the artificial intelligence engine, processes the building data through the artificial intelligence engine, generates a virtual mall model, adopts a predefined scoring model, adds the visibility score, the orientation score and the proximity score to obtain a placement score of the virtual model of the target commodity, and when the placement score is greater than a preset score, deploys the virtual model of the target commodity; S303, the virtual human body module is used to generate a second matching degree by adopting a predefined second matching model, and according to the second matching degree, an initial human body model is selected as a user human body model, and the user human body model is transmitted to a personal data storage system; Optionally, the virtual mall module acquires the building data of the entity mall, calls an artificial intelligence engine, inputs the building data into the artificial intelligence engine, processes the building data through the artificial intelligence engine, generates a virtual mall model, adopts a predefined scoring model, adds the visibility score, the orientation score and the proximity score to obtain a placement score of the virtual model of the target commodity, and when the placement score is greater than a preset score, deploys the virtual model of the target commodity, and sends a start instruction to the virtual human body module; The virtual human module generates a second matching degree according to the starting instruction and a predefined second matching model, selects an initial human model as a user human model according to the second matching degree, and transmits the user human model to the personal data storage system.

[0057] In S304, the AR device is configured to load a virtual model of a target product from a store of a virtual mall model, import a user human model from the personal data storage system in an offline manner, determine a virtual try-on model of the target product, map the user's body feature data to the virtual try-on model to obtain a panoramic image of the user trying on the target product, display the virtual try-on image through a display window, and control a tactile feedback module to output a feedback signal according to the material characteristics of the target product.

[0058] For example, loading the virtual model of the target product from the store of the virtual mall model includes: Opening the virtual mall, using a Verify_Customer instruction to obtain the user's identity information, comparing the user's identity information through a database, returning a verification result, and when the verification result is a pass result, loading the virtual model of the target product from the store of the virtual mall model.

[0059] In the Chinese, Verify_Customer instruction means user verification instruction.

[0060] When the user information is verified, it means that the authenticity and legality of the user are confirmed, which can effectively intercept the risk behavior of account theft, ensure the safety of the mall transaction environment from the source, and avoid losses such as false orders and fraudulent payments.

[0061] In the embodiments of the present application, the beneficial effects are in two aspects. On the one hand, the user human model is imported from the personal data storage system in an offline manner, the virtual try-on model of the target product is determined, the user's body feature data is mapped to the virtual try-on model, and the panoramic image of the user trying on the target product is obtained, which solves the problem of how the digital retail system generates the panoramic image of the user trying on the target product. In addition, since the body data of different individuals is significantly different, the user's body feature data is mapped to the virtual try-on model, which can virtually display the target product according to the user's body feature data. In this way, the target product can be fitted to the user's body in a real way, which can significantly enhance the realism of the panoramic image of the user trying on the target product. On the other hand, the tactile feedback module is controlled to output a feedback signal according to the material characteristics of the target product, which enhances the immersion of the user trying on the target product.

[0062] Those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary universal hardware from the above description of the embodiments. The programs can be stored in a readable storage medium, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, and the like. The storage medium is located in the memory, and the processor reads the information in the memory and executes the methods described in various embodiments of the present application in combination with the hardware.

[0063] The processing unit of the present application can be a central processing unit (CPU), and the processor 20 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0064] The above description and drawings sufficiently show the embodiments of the present disclosure to enable a person skilled in the art to practice them. Other embodiments can include structural, logical, electrical, process, and other changes. The embodiments only represent possible changes. Unless explicitly required, individual components and functions are optional, and the order of operations can be changed. Some parts and sub-samples of some embodiments can be included or replaced by parts and sub-samples of other embodiments. Moreover, the words used in the present application are only used to describe the embodiments and not to limit the claims.

[0065] In this document, each embodiment can focus on the differences from other embodiments, and the same or similar parts between various embodiments can be referred to each other. For the methods, products, and the like disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, the relevant part can be referred to the description of the method part.

[0066] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner can depend on specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0067] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of units can be merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms. The units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to implement the embodiments. In addition, the functional units in the embodiments of the present disclosure can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0068] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (media) having exact sequences of instructions, program, code segments, or computer instructions, which is designed and configured to direct, manage, and control the functions of one or more computers or computer systems, and is recorded on a computer- readable medium (media) to prepare a machine, such that the instructions, program, code segments, or computer instructions recorded on the computer-readable medium (media) are used to implement various functions of the system and the method according to the present disclosure.

Claims

1. A digital retail system based on human-computer interaction, characterized in that, The digital retail system includes a cloud server platform and AR devices for human-computer interaction. The cloud server platform stores virtual product modules and virtual mall modules, and the AR devices are connected to the cloud server platform and the virtual human body module respectively. The virtual product module is used to generate a first matching degree using a predefined first matching model, select the modeling model as the virtual model of the target product based on the first matching degree, and send a running instruction to the virtual mall. The virtual mall module is used to acquire the building data of the physical mall according to the running instructions, call the artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data through the artificial intelligence engine, generate a virtual mall model, determine the placement score of the virtual model of the target product through a predefined scoring model, and deploy the virtual model of the target product when the placement score meets the preset conditions; The virtual human body module is used to generate a second matching degree using a predefined second matching model, select an initial human body model as the user human body model based on the second matching degree, and transmit the user human body model to the personal data storage system; AR devices are used to load virtual models of target products from shops in a virtual shopping mall model, import user human body models from personal data storage systems offline, determine virtual try-on models of target products, map user body feature data to virtual try-on models, and obtain panoramic images of users trying on target products.

2. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The virtual product module is specifically used to obtain the modeling model of the target product from the output file of the 3D modeling software, sample the modeling model according to the sampling density parameter to obtain the data point set of the modeling model, send the sampling density parameter to the 3D scanner, control the 3D scanner to scan the surface of the target product according to the sampling density parameter, obtain the scanned point cloud data of the target product from the output file of the 3D scanner, use a predefined first matching model to process the minimum mean square error between the feature point set of the modeling model and the scanned point cloud data, generate a first matching degree, when the first matching degree is greater than a first preset value, select the modeling model as the virtual model of the target product, and send a running command to the virtual mall. The first matching degree is the matching degree between the feature point set of the modeling model and the scanned point cloud data.

3. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The virtual human body module is specifically used to receive instructions, obtain initial human body parameter information according to the instructions, obtain initial height, initial weight, and initial body measurements from the initial human body parameter information, construct an initial human body model based on the initial height, initial weight, and initial body measurements, obtain the initial human body vector of the initial human body model, send a collection instruction to the 3D modeling device, control the 3D modeling device to collect the user's height, weight, chest circumference, waist circumference, and hip circumference according to the collection instruction, obtain the user's current human body vector from the output file of the 3D modeling device, process the initial human body vector and the current human body vector using a predefined second matching model, generate a second matching degree, select the initial human body model as the user's human body model when the second matching degree is greater than a second preset value, save the user's human body model to a storage file, and send a run instruction to the virtual mall module. The second matching degree is the matching degree between the initial human body vector and the current human body vector.

4. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The virtual mall module is specifically used to receive operation instructions, obtain the building data of the physical mall according to the operation instructions, call the artificial intelligence engine, input the building data into the artificial intelligence engine, process the building data through the artificial intelligence engine, generate a virtual mall model, import the virtual model of the target product into the shop of the virtual mall model, obtain the visibility score, orientation score, and proximity score of the virtual model of the target product, add the visibility score, orientation score, and proximity score using a predefined scoring model to obtain the placement score of the virtual model of the target product, and deploy the virtual model of the target product when the placement score is greater than the preset score. The proximity score is the score of the proximity between the virtual model of the target product and the customer's movement path, and the customer's movement path is the customer's normal movement trajectory.

5. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The AR device is specifically used to access shops in a virtual shopping mall model. When a try-on instruction is received, it loads a virtual model of the target product from the shop in the virtual shopping mall model, reads the user's human body model from a storage file, merges the virtual model of the target product and the user's human body model to obtain a virtual try-on model of the target product, collects the user's body feature data in real time through a built-in motion capture system, maps the user's body feature data to the virtual try-on model to obtain an updated virtual try-on model, and performs material processing, lighting processing and animation processing on the updated virtual try-on model through a real-time rendering engine to obtain a panoramic image of the user trying on the target product.

6. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The artificial intelligence engine is an AI-based 3D modeling engine. The body feature data includes body posture data, joint angle data, and gait data. The architectural data includes architectural structural drawings and decoration drawings. The target products include clothing and cosmetics. The personal data storage system includes USB flash drives, local private clouds, or personal cloud storage.

7. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The first matching model is: ; The first match degree, To minimize the mean square error, This represents the maximum error.

8. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The second matching model is: ; The second degree of matching; For the initial human body vector, This is the current human body vector.

9. The digital retail system based on human-computer interaction as described in claim 1, characterized in that, The scoring model is as follows: ; Rate the placement. V Score visibility A For the orientation rating, D Rate the proximity. α It is the first weighting coefficient. β It is the second weighting coefficient. γ It is the third weighting coefficient. V , A , D and The values ​​range from 0 to 100.

10. A method for operating the system according to any one of claims 1 to 9, characterized in that, include: The virtual product module uses a predefined first matching model to process the minimum mean square error between the feature point set of the modeling model and the scanned point cloud data, and generates a first matching degree. When the first matching degree is greater than a first preset value, the modeling model is selected as the virtual model of the target product, and a running instruction is sent to the virtual mall. The virtual mall module obtains the building data of the physical mall, calls the artificial intelligence engine, inputs the building data into the artificial intelligence engine, processes the building data through the artificial intelligence engine, generates a virtual mall model, and uses a predefined scoring model to add the visibility score, orientation score, and proximity score to obtain the placement score of the virtual model of the target product. When the placement score is greater than the preset score, the virtual model of the target product is deployed. The virtual human body module is used to generate a second matching degree using a predefined second matching model, select an initial human body model as the user's human body model based on the second matching degree, and transmit the user's human body model to the personal data storage system. The AR device is used to load virtual models of target products from shops in a virtual shopping mall model, import user human body models from personal data storage systems offline, determine virtual try-on models of target products, map user body feature data to virtual try-on models, obtain panoramic images of users trying on target products, display virtual try-on images through display windows, and control the haptic feedback module to output feedback signals according to the material characteristics of target products.

Images