Jewelry virtual wearing method, device, computer equipment and storage medium

Abstract

The present application relates to a method, device, computer equipment and storage medium for virtual wearing of jewelry, the method comprising collecting a full-body image of a user, and using a computer vision algorithm to identify full-body key point information in the full-body image, and then calculating the relative position of the jewelry according to the full-body key point information and the corresponding jewelry wearing position; obtaining a corresponding local image from the full-body image of the user according to the jewelry wearing position information, and extracting local feature points of the local image, and then obtaining the final wearing position according to the local feature points and the relative position; obtaining the user's dynamic action data, adjusting the display effect of the jewelry based on the dynamic action data and AR / VR technology, and simulating the physical performance of the jewelry during wearing; obtaining the user's real-time feedback information during the simulation process, and analyzing the user's preference according to the real-time feedback information, and then generating recommendation information according to the user's preference. The present application has the effect of improving the accuracy of virtual wearing display.

Description

Technical Field

[0001] The present application relates to the technical field of virtual wearing of jewelry, and in particular to a method, device, computer equipment and storage medium for virtual wearing of jewelry. Background Art

[0002] At present, the virtual try-on technology of jewelry is widely used in e-commerce platforms and virtual reality (VR) and augmented reality (AR) environments, aiming to provide users with a more convenient and realistic try-on experience. This technology usually uses the user's camera or sensor to obtain the user's image information, and virtually superimposes and displays the jewelry through image processing algorithms.

[0003] Existing virtual jewelry try-on technology usually relies on facial feature points or local wearing parts for virtual display. However, since only local information is considered, the wearing effect is often difficult to accurately restore the natural effect of actual wearing. Especially under dynamic user movements, jewelry dislocation and penetration often occur, affecting the user experience.

[0004] The above-mentioned existing technical solutions have the following defects: the existing virtual try-on technology mostly relies on local images and facial feature points for display, ignoring the coordination of whole body posture and dynamic movements, resulting in the virtual wearing effect being not accurate enough, so there is room for improvement. Summary of the invention

[0005] In order to improve the accuracy of virtual wearing display, the present application provides a jewelry virtual wearing method, device, computer equipment and storage medium.

[0006] The above-mentioned invention objective of the present application is achieved through the following technical solutions:

[0007] A method for virtually wearing jewelry, the method comprising:

[0008] Collecting a full-body image of the user, and using a computer vision algorithm to identify full-body key point information in the full-body image, and then calculating the relative position of the worn jewelry based on the full-body key point information and the corresponding jewelry wearing position;

[0009] Acquire a corresponding partial image from the full-body image of the user according to the jewelry wearing position information, extract local feature points of the partial image, and then obtain a final wearing position according to the local feature points and the relative position;

[0010] Acquire the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing;

[0011] Real-time feedback information from users during the simulation process is obtained, and user preferences are analyzed based on the real-time feedback information, and then recommendation information is generated based on the user preferences.

[0012] By adopting the above technical solution, by collecting the user's full-body image and using the computer vision algorithm to identify the full-body key point information in the full-body image, the user's body posture and relative position can be accurately obtained, thereby ensuring that the virtual effect of wearing jewelry matches the actual wearing effect; by obtaining the corresponding local image from the user's full-body image according to the jewelry wearing position information, and extracting the local feature points of the local image, the jewelry wearing position can be accurately located, thereby avoiding the deviation of the jewelry display position during virtual trial wearing and improving the naturalness of the wearing effect; by obtaining the user's dynamic action data and adjusting the jewelry display effect based on the data and AR / VR technology, the jewelry display effect can be dynamically adjusted according to the user's actual action, so that the jewelry display during the wearing process is more consistent with the actual wearing effect; by obtaining the user's real-time feedback information during the simulation process and analyzing the user's preferences, personalized recommendation information can be generated according to the user's feedback, thereby improving the user experience and the accuracy of personalized jewelry recommendation.

[0013] In one example, the present application may be further configured as follows: the whole body key point information in the whole body image is identified by using a computer vision algorithm, and then the relative position of the worn jewelry is calculated according to the whole body key point information and the corresponding jewelry wearing position, specifically including:

[0014] Using the computer vision algorithm to identify key points of the user's whole-body image to obtain coordinate information of key body parts, and performing feature extraction and position correction on the whole-body image through a multi-layer neural network to obtain the whole-body key point information;

[0015] Combined with the spatial relationship between the jewelry wearing position and the key points of the whole body, a geometric transformation algorithm is used to calculate the relative position of the jewelry and the wearing part to ensure that the jewelry position matches the posture and angle of the user when actually wearing it.

[0016] By adopting the above technical solution, by using a computer vision algorithm to identify key points of the user's full-body image, the key coordinate information of the user's body parts can be accurately extracted, thereby providing an accurate spatial basis for the subsequent calculation of the position of the jewelry being worn; by using a multi-layer neural network to extract features and correct the position of the full-body image, the positioning error in the image can be eliminated, thereby ensuring the accuracy of the full-body key point information and improving the accuracy of the virtual effect of wearing jewelry.

[0017] In one example, the present application may be further configured as follows: obtaining a corresponding partial image from the full-body image of the user according to the jewelry wearing position information, extracting local feature points of the partial image, and then obtaining the final wearing position according to the local feature points and the relative position, specifically including:

[0018] According to the jewelry wearing position information, extract the local image area corresponding to the wearing position from the whole body image, and use computer vision technology to process the local image to identify and extract the local feature points;

[0019] Based on the local feature points and the relative position, the relative position of the jewelry is adjusted through geometric calculation and a preset wearing ratio to obtain the final wearing position.

[0020] By adopting the above technical solution, by extracting the local image area corresponding to the wearing part from the whole body image according to the jewelry wearing position information, the detailed image data of the wearing area can be efficiently obtained, thereby ensuring the accuracy of the local wearing effect; by using computer vision technology to process the local image and identify the local feature points, the key features of the wearing part can be accurately extracted, thereby improving the fit between the virtual jewelry and the wearing part and enhancing the authenticity of the wearing effect; by adjusting the wearing position of the jewelry based on the local feature points and relative positions, the jewelry can be accurately docked with the user's wearing part, thereby avoiding misalignment between the jewelry and the wearing part during virtual trial wearing and improving the naturalness of the trial wearing effect.

[0021] In one example, the present application may be further configured as follows: adjusting the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulating the physical performance of the jewelry during wearing, specifically including:

[0022] Combining the dynamic motion data with the AR / VR technology to analyze the dynamic motion data in real time and store and memorize it;

[0023] Using AR technology, the position and angle data of the corresponding jewelry model are adjusted according to the user's real-time action changes;

[0024] In VR mode, the virtual reality engine is used to render the three-dimensional effect of the user wearing jewelry and simulate the dynamic performance of the jewelry during wearing.

[0025] By adopting the above technical solution, by combining dynamic motion data with AR / VR technology and performing real-time analysis, it is possible to track the user's dynamic motion in real time and adjust the jewelry display effect, making the jewelry display more natural and smooth; by utilizing AR technology and adjusting the position and angle of the jewelry model according to the user's real-time motion changes, it is possible to ensure dynamic adjustment of the jewelry during wearing, thereby avoiding unnatural dislocation or penetration during jewelry display; by rendering the three-dimensional effect of the user wearing jewelry in VR mode, it is possible to accurately simulate the dynamic performance of the jewelry wearing process, thereby enhancing the user's sense of immersion and the realism of the wearing experience.

[0026] In one example, the present application may be further configured as follows: the real-time analysis and storage of the dynamic action data specifically includes:

[0027] Extracting features from the dynamic motion data using a recognition algorithm to identify the motion patterns of the user in the process of wearing the jewelry;

[0028] According to the action mode, a dynamic action model is established, and the dynamic action model is combined with the specific position and posture data of the jewelry wearing part for memory storage.

[0029] By adopting the above technical solution, through real-time analysis and storage of dynamic action data, the action patterns generated by the user in the process of wearing jewelry can be recorded, and the virtual jewelry display effect can be adjusted according to the user's wearing habits, thereby improving the accuracy of virtual jewelry try-on; by combining the dynamic action model with the specific position and posture data of the jewelry wearing parts and storing them in memory, the user's wearing action and the actual position of the jewelry can be accurately simulated, thereby reducing the misalignment or penetration of jewelry during the wearing process and improving the accuracy of the virtual wearing effect.

[0030] In one example, the present application may be further configured as follows: analyzing the user preference according to the real-time feedback information, and then generating recommendation information according to the user preference, specifically including:

[0031] Performing data mining on the real-time feedback information using a data mining algorithm to obtain corresponding mining data, and scoring the mining data, thereby obtaining the user preference according to the corresponding score;

[0032] The user's wearing records, selection history and user preferences are obtained, a short-term memory model of the user's preferences is established, the user's preference changes in multiple wearing scenarios are analyzed, and the recommendation information is generated.

[0033] By adopting the above technical solution and using data mining algorithms to mine real-time feedback information, key preference data can be extracted from user feedback, thereby accurately capturing the user's jewelry preferences; by combining the user's wearing records, selection history and user preferences and establishing a short-term memory model, the user's jewelry preference data can be updated in real time, and personalized jewelry recommendation information can be generated, thereby improving the accuracy and personalization of jewelry recommendations and increasing user purchase intentions and satisfaction.

[0034] The second object of the invention is achieved by the following technical solutions:

[0035] A jewellery virtual wearing device, the jewellery virtual wearing device comprising:

[0036] An image acquisition module is used to acquire a full-body image of the user, and identify full-body key point information in the full-body image using a computer vision algorithm, and then calculate the relative position of the worn jewelry based on the full-body key point information and the corresponding jewelry wearing position;

[0037] A local image extraction module, used to obtain a corresponding local image from the user's full-body image according to the jewelry wearing position information, and extract local feature points of the local image, and then obtain the final wearing position according to the local feature points and the relative position;

[0038] A motion data acquisition module is used to obtain the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing;

[0039] The real-time feedback collection module is used to obtain real-time feedback information from users during the simulation process, analyze user preferences based on the real-time feedback information, and then generate recommendation information based on the user preferences.

[0040] By adopting the above technical solution, by collecting the user's full-body image and using the computer vision algorithm to identify the full-body key point information in the full-body image, the user's body posture and relative position can be accurately obtained, thereby ensuring that the virtual effect of wearing jewelry matches the actual wearing effect; by obtaining the corresponding local image from the user's full-body image according to the jewelry wearing position information, and extracting the local feature points of the local image, the jewelry wearing position can be accurately located, thereby avoiding the deviation of the jewelry display position during virtual trial wearing and improving the naturalness of the wearing effect; by obtaining the user's dynamic action data and adjusting the jewelry display effect based on the data and AR / VR technology, the jewelry display effect can be dynamically adjusted according to the user's actual action, so that the jewelry display during the wearing process is more consistent with the actual wearing effect; by obtaining the user's real-time feedback information during the simulation process and analyzing the user's preferences, personalized recommendation information can be generated according to the user's feedback, thereby improving the user experience and the accuracy of personalized jewelry recommendation.

[0041] The third objective of the present application is achieved through the following technical solutions:

[0042] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for virtual wearing of jewelry when executing the computer program.

[0043] The fourth objective of the present application is achieved through the following technical solutions:

[0044] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned jewelry virtual wearing method.

[0045] In summary, this application includes the following beneficial technical effects:

[0046] 1. By collecting the user's full-body image and using computer vision algorithms to identify the full-body key point information in the full-body image, the user's body posture and relative position can be accurately obtained, thereby ensuring that the virtual effect of wearing jewelry matches the actual wearing effect; by obtaining the corresponding local image from the user's full-body image according to the jewelry wearing position information, and extracting the local feature points of the local image, the jewelry wearing position can be accurately located, thereby avoiding the deviation of the jewelry display position during virtual try-on and improving the naturalness of the wearing effect; by obtaining the user's dynamic action data and adjusting the jewelry display effect based on the data and AR / VR technology, the jewelry display effect can be dynamically adjusted according to the user's actual action, so that the jewelry display during the wearing process is more consistent with the actual wearing effect; by obtaining the user's real-time feedback information during the simulation process and analyzing the user's preferences, personalized recommendation information can be generated based on the user's feedback, thereby improving the user experience and the accuracy of personalized jewelry recommendations;

[0047] 2. By using computer vision algorithms to identify key points of the user's full-body image, the key coordinate information of the user's body parts can be accurately extracted, thereby providing an accurate spatial basis for the subsequent calculation of the position of the jewelry being worn; by using a multi-layer neural network to extract features and correct the position of the full-body image, the positioning error in the image can be eliminated, thereby ensuring the accuracy of the key point information of the whole body and improving the accuracy of the virtual effect of wearing jewelry;

[0048] 3. By extracting the local image area corresponding to the wearing part from the whole body image according to the jewelry wearing position information, the detailed image data of the wearing area can be efficiently obtained, thereby ensuring the accuracy of the local wearing effect; by using computer vision technology to process the local image and identify the local feature points, the key features of the wearing part can be accurately extracted, thereby improving the fit between the virtual jewelry and the wearing part and enhancing the authenticity of the wearing effect; by adjusting the wearing position of the jewelry based on the local feature points and relative position, the jewelry can be accurately docked with the user's wearing part, thereby avoiding misalignment between the jewelry and the wearing part during virtual trial wearing and improving the naturalness of the trial wearing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a method for virtual wearing of jewelry in one embodiment of the present application;

[0050] Figure 2 is a flowchart for implementing step S10 in the jewelry virtual wearing method in one embodiment of the present application;

[0051] Figure 3 is a flowchart for implementing step S20 in the jewelry virtual wearing method in one embodiment of the present application;

[0052] Figure 4is a flowchart for implementing step S30 in the jewelry virtual wearing method in one embodiment of the present application;

[0053] Figure 5 is a flowchart for implementing step S31 in the jewelry virtual wearing method in one embodiment of the present application;

[0054] Figure 6 is a flowchart for implementing step S40 in the jewelry virtual wearing method in one embodiment of the present application;

[0055] Figure 7 This is a principle block diagram of a jewelry virtual wearable device in one embodiment of the present application;

[0056] Figure 8 It is a schematic diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The present application is further described in detail below in conjunction with the accompanying drawings.

[0058] In one embodiment, if Figure 1 As shown, the present application discloses a method for virtual wearing of jewelry, which specifically includes the following steps:

[0059] S10: Collect a full-body image of the user, and use a computer vision algorithm to identify full-body key point information in the full-body image, and then calculate the relative position of the worn jewelry based on the full-body key point information and the corresponding jewelry wearing position.

[0060] Specifically, the user's full-body image is processed using a deep learning algorithm, and feature extraction is performed through a convolutional neural network. The coordinate information of key body parts, such as the head, neck, shoulders, wrists, etc., is extracted from the image, and the spatial positions of these key points in the image are calculated. Then, the actual spatial positions of these key points are further corrected through a multi-layer neural network to ensure the accuracy of recognition. Then, the relative position of the jewelry wearing position is obtained by calculating the spatial relationship between the part where the jewelry is worn and these key points, thereby ensuring that the jewelry model can be displayed correctly and naturally in the user's image to avoid misalignment with the wearing part.

[0061] S20: Obtain a corresponding partial image from the user's full-body image according to the jewelry wearing position information, extract local feature points of the partial image, and then obtain the final wearing position according to the local feature points and the relative position.

[0062] Specifically, according to the position where the jewelry is worn, the local area corresponding to the position is extracted, such as local image areas such as the neck and wrist, and then the local image is processed using computer vision technology. The feature points in the local image are identified and extracted through image processing methods such as Harris corner detection or SIFT algorithm, and then these local feature points are combined with the whole body key point information obtained previously. The placement of the jewelry in the local image is adjusted through a geometric transformation algorithm to ensure that the final wearing position of the jewelry is natural and ergonomic, and to avoid errors that do not conform to the actual wearing effect.

[0063] S30: Acquire the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing.

[0064] Specifically, by using acquisition devices such as motion capture devices, depth sensors or cameras, the dynamic motion data of the user when wearing jewelry is obtained in real time, such as the motion data of the head, arms, fingers and other parts. Based on these data, AR technology is used to adjust the display position and angle of the jewelry in the user's actual environment in real time to ensure that the jewelry is naturally adjusted as the user's movements change. At the same time, VR technology is used to render the three-dimensional effect of jewelry in a virtual environment, and the virtual engine is used to simulate the physical performance of jewelry as it changes with movement during wearing, such as the swing of jewelry, gloss changes, etc., to further enhance the user's wearing experience and ensure that the dynamic performance of jewelry in a virtual environment is consistent with the effect when it is actually worn.

[0065] S40: obtaining real-time feedback information from users during the simulation process, analyzing user preferences based on the real-time feedback information, and then generating recommendation information based on the user preferences.

[0066] Specifically, during the simulated wearing process, user feedback on the jewelry display is collected in real time, including user preference evaluations on jewelry style, color, size, etc., and user operational behaviors during the virtual try-on process. The feedback information is deeply analyzed through data mining algorithms, and the user's preference trend is evaluated through a scoring system for feedback data. A short-term memory model of user preferences is established, which can analyze the changes in users in different wearing scenarios. The model is used to generate personalized recommendation information, thereby recommending to users the jewelry style or configuration that best suits their preferences.

[0067] In one embodiment, if Figure 2 As shown, in step S10, the whole body key point information in the whole body image is identified by using a computer vision algorithm, and then the relative position of the worn jewelry is calculated according to the whole body key point information and the corresponding jewelry wearing position, which specifically includes:

[0068] S11: Use computer vision algorithm to identify key points of the user's whole body image to obtain the coordinate information of key body parts, and use a multi-layer neural network to extract features and correct the position of the whole body image to obtain the whole body key point information.

[0069] Specifically, a convolutional neural network is first used to process the user's full-body image, extract multidimensional features in the image, such as edges, contours, and color distribution, and identify the user's key body parts. Then, the identified full-body image is corrected using a trained multi-layer neural network to correct the positional deviation in the image so that the extracted coordinates of the full-body key points can accurately reflect the user's actual body shape and posture.

[0070] S12: Based on the spatial relationship between the jewelry wearing position and the key points of the whole body, a geometric transformation algorithm is used to calculate the relative position of the jewelry and the wearing part to ensure that the jewelry position matches the user's actual wearing posture and angle.

[0071] Specifically, after obtaining the coordinates of the user's whole body key points, the jewelry wearing position is spatially aligned with the user's actual wearing part through a geometric transformation algorithm, such as an affine transformation or a projection transformation algorithm. According to the distance and angle relationship between the key points, the position and angle of the jewelry model are adjusted to ensure that the jewelry is accurately connected to the wearing part in the virtual image and matches the user's actual posture and movement, avoiding misalignment of the virtual jewelry and the wearing part or unnatural visual effects.

[0072] In one embodiment, if Figure 3 As shown, in step S20, the corresponding local image is obtained from the user's full-body image according to the jewelry wearing position information, and the local feature points of the local image are extracted, and then the final wearing position is obtained according to the local feature points and the relative position, which specifically includes:

[0073] S21: According to the jewelry wearing position information, extract the local image area corresponding to the wearing position from the whole body image, and use computer vision technology to process the local image to identify and extract local feature points.

[0074] Specifically, computer vision technology is used to analyze the extracted local images to identify and extract key feature points of the local areas. These feature points may include feature points of the center of the neck, around the wrist, etc. Then, based on these local feature points, the relative position of the jewelry and the wearing part is further calculated to ensure that the jewelry model can be correctly placed in the user's local image.

[0075] S22: Based on the local feature points and the relative positions, the relative positions of the jewelry are adjusted through geometric calculations and preset wearing ratios to obtain the final wearing position.

[0076] Specifically, after obtaining the local feature points, the size and angle of the jewelry model are adjusted through geometric calculation methods in combination with the wearing ratio to match the actual position and size of the local wearing part, ensuring that the jewelry appears natural on the user's wearing part and avoiding position deviation or incoordination of the virtual jewelry.

[0077] In one embodiment, if Figure 4 As shown, in step S30, the display effect of the jewelry is adjusted based on the dynamic motion data and AR / VR technology, and the physical performance of the jewelry during wearing is simulated, which specifically includes:

[0078] S31: Combine dynamic motion data with AR / VR technology to analyze dynamic motion data in real time and store it in memory.

[0079] Specifically, the user's dynamic motion data, such as the motion trajectory and angle changes of the hands, head, neck and other parts, are collected through motion sensors or depth cameras, and these dynamic data are processed in real time using real-time analysis algorithms such as time series analysis or motion trajectory prediction algorithms. The data are stored in a database for subsequent use, and the dynamic performance of the jewelry during wearing is adjusted according to changes in the user's motion patterns to ensure that each movement can produce an accurate jewelry display effect.

[0080] S32: Using AR technology, and adjusting the position and angle data of the corresponding jewelry model according to the user's real-time action changes.

[0081] Specifically, AR technology is used to dynamically track the user's wearing parts, update the display position and angle of the jewelry in real time, and ensure that the jewelry model rotates or swings naturally when the user wears it, synchronized with the user's movements, avoiding unnatural misalignment or distortion when the jewelry is displayed.

[0082] S33: In VR mode, the virtual reality engine is used to render the three-dimensional effect of the user wearing jewelry and simulate the dynamic performance of the jewelry during wearing.

[0083] Specifically, in VR mode, the jewelry is three-dimensionally modeled and rendered through a virtual reality engine, allowing users to view the effect of wearing jewelry in a virtual environment, simulating the jewelry's dynamic performance such as gloss changes, swing amplitude, and the influence of gravity, ensuring that the display of the jewelry in the virtual world is as close as possible to the actual wearing effect.

[0084] In one embodiment, if Figure 5 As shown, in step S31, the dynamic action data is analyzed in real time and stored in memory, which specifically includes:

[0085] S311: Extract features from dynamic motion data using a recognition algorithm to identify the motion patterns of the user while wearing jewelry.

[0086] Specifically, through algorithms such as feature selection algorithms and clustering algorithms, feature extraction is performed on the action data generated by the user when wearing jewelry, and the user's common wearing behavior patterns are identified, such as slight head movements, hand adjustments, etc., so as to provide data support for subsequent jewelry display adjustments.

[0087] S312: A dynamic action model is established according to the action mode, and the dynamic action model is combined with the specific position and posture data of the jewelry wearing part for memory storage.

[0088] Specifically, a dynamic motion model of the user is established based on the extracted motion patterns, and stored and memorized in combination with the parts where the jewelry is worn, such as the neck and hands, as well as the user's actual wearing posture. The model is then used to make precise adjustments each time the jewelry is virtually tried on to avoid unnatural distortion in the jewelry display.

[0089] In one embodiment, if Figure 6 As shown, in step S40, the user preference is analyzed according to the real-time feedback information, and then recommendation information is generated according to the user preference, which specifically includes:

[0090] S41: Perform data mining on the real-time feedback information using a data mining algorithm to obtain corresponding mining data, and score the mining data, and then obtain user preferences based on the corresponding scores.

[0091] Specifically, by using data mining techniques such as association rule mining and cluster analysis, we analyze users' real-time feedback data to discover their preference trends, such as jewelry style preferences and wearing size preferences, and score the data to establish user preference profiles.

[0092] S42: Obtain the user's wearing records, selection history and user preferences, establish a short-term memory model of user preferences, analyze the changes in user preferences in multiple wearing scenarios, and generate recommendation information.

[0093] Specifically, the user's wearing history records, jewelry selection records and their feedback preference information are integrated to establish a short-term memory model. This model can analyze the changing trends of users' wearing preferences in different scenarios, combine real-time data, generate personalized jewelry recommendation information, and display it to users.

[0094] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0095] In one embodiment, a jewelry virtual wearing device is provided, and the jewelry virtual wearing device corresponds to the jewelry virtual wearing method in the above embodiment. Figure 7 As shown, the jewelry virtual wearable device includes an image acquisition module, a local image extraction module, an action data acquisition module, and a real-time feedback acquisition module. The detailed description of each functional module is as follows:

[0096] An image acquisition module is used to acquire a full-body image of the user and identify full-body key point information in the full-body image using a computer vision algorithm, and then calculate the relative position of the jewelry worn based on the full-body key point information and the corresponding jewelry wearing position;

[0097] The local image extraction module is used to obtain the corresponding local image from the user's full-body image according to the jewelry wearing position information, and extract the local feature points of the local image, and then obtain the final wearing position according to the local feature points and relative positions;

[0098] The motion data collection module is used to obtain the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing;

[0099] The real-time feedback collection module is used to obtain real-time feedback information from users during the simulation process, analyze user preferences based on the real-time feedback information, and then generate recommendation information based on user preferences.

[0100] Optionally, the image acquisition module specifically includes:

[0101] The key point recognition submodule is used to recognize key points of the user's whole body image using a computer vision algorithm to obtain coordinate information of key body parts, and to extract features and correct positions of the whole body image through a multi-layer neural network to obtain whole body key point information;

[0102] The geometric transformation submodule is used to combine the spatial relationship between the jewelry wearing position and the key points of the whole body, and use the geometric transformation algorithm to calculate the relative position of the jewelry and the wearing part to ensure that the jewelry position matches the user's actual wearing posture and angle.

[0103] Optionally, the local image extraction module specifically includes:

[0104] The local image extraction submodule is used to extract the local image area corresponding to the wearing part from the whole body image according to the jewelry wearing position information, and use computer vision technology to process the local image to identify and extract local feature points;

[0105] The position adjustment submodule is used to adjust the relative position of the jewelry based on the local feature points and relative positions through geometric calculation and preset wearing ratios to obtain the final wearing position.

[0106] Optionally, the motion data collection module specifically includes:

[0107] The motion data analysis submodule is used to combine dynamic motion data with AR / VR technology to analyze the dynamic motion data in real time and store it in memory;

[0108] The AR adjustment submodule is used to use AR technology to adjust the position and angle data of the corresponding jewelry model according to the user's real-time action changes;

[0109] The VR rendering submodule is used to render the three-dimensional effect of the user wearing jewelry through the virtual reality engine in VR mode, and simulate the dynamic performance of the jewelry during the wearing process.

[0110] Optionally, the motion data analysis submodule specifically includes:

[0111] A motion data analysis unit, used to extract features from dynamic motion data using a recognition algorithm, and identify the motion patterns of the user in the process of wearing jewelry;

[0112] The motion modeling unit is used to establish a dynamic motion model according to the motion pattern, and combine the dynamic motion model with the specific position and posture data of the jewelry wearing part for memory storage.

[0113] Optionally, the real-time feedback collection module specifically includes:

[0114] The real-time feedback collection submodule is used to perform data mining on the real-time feedback information using a data mining algorithm to obtain corresponding mining data, and to score the mining data, and then obtain the user preference according to the corresponding score;

[0115] The user preference analysis submodule is used to obtain the user's wearing records, selection history and user preferences, establish a short-term memory model of user preferences, analyze the changes in user preferences in multiple wearing scenarios, and generate recommendation information.

[0116] The specific definition of the jewelry virtual wearing device can be found in the definition of the jewelry virtual wearing method above, which will not be repeated here. Each module in the above-mentioned jewelry virtual wearing device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0117] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for virtual wearing of jewelry is implemented.

[0118] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0119] Collect the user's full-body image, and use computer vision algorithms to identify the full-body key point information in the full-body image, and then calculate the relative position of the jewelry worn according to the full-body key point information and the corresponding jewelry wearing position;

[0120] Obtaining a corresponding partial image from the user's full-body image according to the jewelry wearing position information, extracting local feature points of the partial image, and then obtaining the final wearing position according to the local feature points and relative positions;

[0121] Obtain the user's dynamic motion data, adjust the display effect of jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of jewelry during wearing;

[0122] Obtain real-time feedback information from users during the simulation process, analyze user preferences based on the real-time feedback information, and then generate recommendation information based on user preferences.

[0123] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0124] Collect the user's full-body image, and use computer vision algorithms to identify the full-body key point information in the full-body image, and then calculate the relative position of the jewelry worn according to the full-body key point information and the corresponding jewelry wearing position;

[0125] Obtaining a corresponding partial image from the user's full-body image according to the jewelry wearing position information, extracting local feature points of the partial image, and then obtaining the final wearing position according to the local feature points and relative positions;

[0126] Obtain the user's dynamic motion data, adjust the display effect of jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of jewelry during wearing;

[0127] Obtain real-time feedback information from users during the simulation process, analyze user preferences based on the real-time feedback information, and then generate recommendation information based on user preferences.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0130] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for virtual wearing of jewelry, characterized in that: The jewelry virtual wearing method comprises: Collecting a full-body image of the user, and using a computer vision algorithm to identify full-body key point information in the full-body image, and then calculating the relative position of the worn jewelry according to the full-body key point information and the corresponding jewelry wearing position. The method of using a computer vision algorithm to identify the full-body key point information in the full-body image, and then calculating the relative position of the worn jewelry according to the full-body key point information and the corresponding jewelry wearing position, specifically includes: using the computer vision algorithm to perform key point recognition on the full-body image of the user to obtain coordinate information of key body parts, and performing feature extraction and position correction on the full-body image through a multi-layer neural network to obtain the full-body key point information; combining the spatial relationship between the jewelry wearing position and the full-body key points, using a geometric transformation algorithm to calculate the relative position of the jewelry and the wearing part, to ensure that the jewelry position matches the posture and angle of the user when actually wearing it; According to the jewelry wearing position information, a corresponding partial image is obtained from the user's full-body image, and local feature points of the partial image are extracted, and then the final wearing position is obtained according to the local feature points and the relative position. The method of obtaining a corresponding partial image from the user's full-body image according to the jewelry wearing position information, and extracting local feature points of the partial image, and then the final wearing position is obtained according to the local feature points and the relative position, specifically includes: according to the jewelry wearing position information, extracting a local image area corresponding to the wearing part from the full-body image, and processing the partial image using computer vision technology to identify and extract the local feature points; based on the local feature points and the relative position, adjusting the relative position of the jewelry through geometric calculation and preset wearing ratio, and then obtaining the final wearing position; Acquire the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing; Real-time feedback information from users during the simulation process is obtained, and user preferences are analyzed based on the real-time feedback information, and then recommendation information is generated based on the user preferences.

2. The method for virtual wearing of jewelry according to claim 1, characterized in that: The adjusting the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulating the physical performance of the jewelry during wearing, specifically includes: Combining the dynamic motion data with the AR / VR technology to analyze the dynamic motion data in real time and store and memorize it; Using AR technology, the position and angle data of the corresponding jewelry model are adjusted according to the user's real-time action changes; In VR mode, the virtual reality engine is used to render the three-dimensional effect of the user wearing jewelry and simulate the dynamic performance of the jewelry during wearing.

3. The method for virtual wearing of jewelry according to claim 2, characterized in that: The real-time analysis and storage of the dynamic action data specifically includes: Extracting features from the dynamic motion data using a recognition algorithm to identify the motion patterns of the user in the process of wearing the jewelry; According to the action mode, a dynamic action model is established, and the dynamic action model is combined with the specific position and posture data of the jewelry wearing part for memory storage.

4. The method for virtual wearing of jewelry according to claim 1, characterized in that: Analyzing the user preference according to the real-time feedback information, and then generating recommendation information according to the user preference, specifically includes: Performing data mining on the real-time feedback information using a data mining algorithm to obtain corresponding mining data, and scoring the mining data, thereby obtaining the user preference according to the corresponding score; The user's wearing records, selection history and user preferences are obtained, a short-term memory model of the user's preferences is established, the user's preference changes in multiple wearing scenarios are analyzed, and the recommendation information is generated.

5. A virtual jewelry wearing device, characterized in that: The jewellery virtual wearing device comprises: An image acquisition module is used to acquire a full-body image of the user, and identify full-body key point information in the full-body image using a computer vision algorithm, and then calculate the relative position of the worn jewelry based on the full-body key point information and the corresponding jewelry wearing position; A local image extraction module, used to obtain a corresponding local image from the user's full-body image according to the jewelry wearing position information, and extract local feature points of the local image, and then obtain the final wearing position according to the local feature points and the relative position; A motion data acquisition module is used to obtain the user's dynamic motion data, adjust the display effect of the jewelry based on the dynamic motion data and AR / VR technology, and simulate the physical performance of the jewelry during wearing; A real-time feedback collection module is used to obtain real-time feedback information from users during the simulation process, analyze user preferences based on the real-time feedback information, and then generate recommendation information based on the user preferences; The image acquisition module specifically includes: A key point recognition submodule, for using the computer vision algorithm to perform key point recognition on the user's full-body image to obtain coordinate information of key body parts, and performing feature extraction and position correction on the full-body image through a multi-layer neural network to obtain the full-body key point information; A geometric transformation submodule is used to combine the spatial relationship between the jewelry wearing position and the key points of the whole body, and use a geometric transformation algorithm to calculate the relative position of the jewelry and the wearing part, so as to ensure that the jewelry position matches the posture and angle of the user when actually wearing it; The local image extraction module specifically includes: A local image extraction submodule is used to extract a local image area corresponding to the wearing position from the whole body image according to the jewelry wearing position information, and process the local image using computer vision technology to identify and extract the local feature points; The position adjustment submodule is used to adjust the relative position of the jewelry through geometric calculation and preset wearing ratio based on the local feature points and the relative position, so as to obtain the final wearing position.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the jewelry virtual wearing method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the jewelry virtual wearing method as claimed in any one of claims 1 to 4 are implemented.

Images