A cosmetic effect analysis method and system based on image analysis and dialogue interaction

Abstract

This application discloses a method and system for beauty effect analysis based on image analysis and dialogue interaction, belonging to the field of data processing technology. The method includes: acquiring a user's skin image; comparing and analyzing the user's skin image with a reference skin image to obtain a beauty effect analysis result; generating personalized dialogue content adapted to the user's information based on user information and the beauty effect analysis result; and presenting the beauty effect analysis result and personalized dialogue content to the user through an electronic device. This application obtains a quantitative beauty effect analysis result by comparing and analyzing the user's skin image with a reference skin image, and generates personalized dialogue content by combining user information and the beauty effect analysis result, thereby realizing dynamic comparative analysis and personalized presentation of beauty effects, allowing users to intuitively understand the beauty effects and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and system for analyzing beauty effects based on image analysis and dialogue interaction. Background Technology

[0002] Currently, the evaluation methods for the beauty effects of beauty devices are mainly divided into two categories: one is user self-judgment, which means that users judge the beauty effect based on their own subjective feelings during the use of beauty devices; the other is professional institution testing, which means that through professional testing equipment equipped by beauty institutions, professionals operate to collect images and combine them with model algorithms to quantitatively analyze multiple indicators such as skin texture, pores, and pigmentation to achieve the evaluation of beauty effects.

[0003] However, existing technologies have significant drawbacks: on the one hand, user judgment relies on subjective feelings and lacks objective comparative analysis of beauty effects, making it impossible to intuitively and accurately assess the actual beauty effects of beauty devices; on the other hand, professional testing equipment in beauty institutions requires operation and interpretation by professionals, resulting in a high barrier to entry, and is generally designed for static analysis of single images, failing to achieve dynamic comparative analysis of beauty effects and making it difficult to intuitively present the results, thus affecting user trust and repurchase rates. Therefore, how to achieve dynamic comparative analysis and presentation of beauty effects is a problem that urgently needs to be solved. Summary of the Invention

[0004] This paper provides a method and system for analyzing beauty effects based on image analysis and dialogue interaction to solve the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for beauty effect analysis based on image analysis and dialogue interaction. The method includes: acquiring a user's skin image; comparing and analyzing the user's skin image with a reference skin image to obtain a beauty effect analysis result; generating personalized dialogue content adapted to the user's information based on the user information and the beauty effect analysis result; and presenting the beauty effect analysis result and the personalized dialogue content to the user through an electronic device.

[0006] As one possible implementation, comparing and analyzing a user's skin image with a reference skin image to obtain a cosmetic effect analysis result includes: performing skin feature analysis on the user's skin image to obtain skin feature data of the user's skin image, the skin feature data including at least one skin feature index data; querying the corresponding reference skin image from the user database based on the user's skin image and extracting the skin feature data of the reference skin image; and comparing and analyzing the skin feature data of the user's skin image and the reference skin image to obtain a cosmetic effect analysis result.

[0007] As one possible implementation, the process involves retrieving a corresponding reference skin image from a user database based on a user's skin image, and extracting skin feature data from the reference skin image. This includes: retrieving a set of historical skin images corresponding to the user information in the user database based on the user information corresponding to the user's skin image; selecting a reference skin image from the set of historical skin images according to a preset comparison period; and extracting skin feature data from the reference skin image.

[0008] As one possible implementation, before extracting skin feature data of the reference skin image based on the reference skin image, the method further includes: identifying the user skin image and the selected reference skin image; and when the identification result indicates that the user skin image and the reference skin image belong to the same user, extracting skin feature data of the reference skin image.

[0009] As one possible implementation, the cosmetic effect analysis results include a skin radar map. The analysis involves comparing and analyzing skin feature data from a user's skin image and a reference skin image to obtain the cosmetic effect analysis results. This includes: constructing a multi-dimensional coordinate system based on various skin feature indicators in the skin feature data, with each dimension axis of the multi-dimensional coordinate system corresponding to each skin feature indicator; plotting coordinate points on the multi-dimensional coordinate system and connecting them based on the skin feature indicator data of the user's skin image; calculating the variation range of each skin feature indicator data in the user's skin image and the reference skin image, and extracting skin feature indicators whose absolute variation range is greater than a set threshold as target skin feature indicators; and displaying the corresponding dimension axes of the multi-dimensional coordinate system differently based on the target skin feature indicators to obtain a skin radar map.

[0010] As one possible implementation, the beauty effect analysis results include a skin heatmap; comparing and analyzing the skin feature data of the user's skin image and a reference skin image to obtain the beauty effect analysis results includes: using the user's skin image as a base map and dividing the base map into multiple regions; analyzing the skin feature index data of the user's skin image and the reference skin image in each region based on the skin feature data of the user's skin image and the reference skin image; calculating the variation range of each region based on the skin feature index data of the user's skin image and the reference skin image in each region; mapping the variation range to a fill color according to a preset mapping rule to obtain the fill color of each region; and coloring the corresponding region of the base map according to the fill color to obtain the skin heatmap.

[0011] As one possible implementation method, personalized dialogue content adapted to user information is generated based on user information and beauty effect analysis results, including: extracting user keywords based on beauty effect analysis results; matching dialogue templates from a pre-set dialogue template library based on user keywords, supplementing the dialogue templates based on beauty effect analysis results, and generating dialogue content; selecting a personalized interaction method based on user information, and generating personalized dialogue content adapted to user information based on the personalized interaction method.

[0012] As one possible implementation, the beauty effect analysis results and personalized dialogue content are presented to the user through an electronic device, including: displaying the beauty effect analysis results, the user's skin image, and a reference skin image through an electronic device; determining the target analysis area based on the user's actions on the displayed user skin image; extracting skin feature data from the user's skin image and the reference skin image in the target analysis area and performing comparative analysis to obtain the beauty effect analysis results of the target analysis area, and presenting them to the user through an electronic device.

[0013] As one possible implementation, the method further includes: pushing a first user operation task to the electronic device before comparing and analyzing the user's skin image with a reference skin image to obtain a cosmetic effect analysis result; and when the electronic device is detected to have completed the first user operation task, comparing and analyzing the user's skin image with the reference skin image to obtain a cosmetic effect analysis result.

[0014] As one possible implementation, the method further includes: after presenting the beauty effect analysis results and personalized dialogue content to the user through an electronic device, detecting whether the electronic device issues a request to unlock the care plan; when the electronic device issues a request to unlock the care plan, pushing a second user operation task to the electronic device; when the electronic device completes the second user operation task, matching the corresponding care plan in the preset care plan library according to the personalized dialogue content and pushing it.

[0015] As one possible implementation method, the method further includes: after presenting the beauty effect analysis results and personalized dialogue content to the user through an electronic device, matching the corresponding product information in a pre-set product push library according to the personalized dialogue content and pushing it.

[0016] As one possible implementation, when the electronic device is a user terminal, the user terminal is used to acquire a user skin image. Acquiring a user skin image includes: calling the user terminal's shooting component; loading AR markers on the shooting interface of the shooting component to prompt the user to adjust to a preset shooting angle and shooting distance; performing light detection on the current environment, and controlling the shooting component to continuously capture multiple frames of time-series images after the light detection is qualified; performing time-series alignment and fusion noise reduction processing on the multiple frames of time-series images to obtain a noise-reduced image; and performing skin color correction on the noise-reduced image to obtain the user skin image.

[0017] Secondly, this application provides a beauty effect analysis system based on image analysis and dialogue interaction, comprising: an image acquisition module for acquiring user skin images; an image analysis module for comparing and analyzing user skin images with reference skin images to obtain beauty effect analysis results; a dialogue generation module for generating personalized dialogue content adapted to user information based on user information and beauty effect analysis results; and an analysis presentation module for presenting beauty effect analysis results and personalized dialogue content to the user through an electronic device.

[0018] Compared with existing technologies, the beauty effect analysis method and system based on image analysis and dialogue interaction provided in this application obtains quantitative beauty effect analysis results by comparing and analyzing user skin images with reference skin images. At the same time, it generates personalized dialogue content by combining user information and beauty effect analysis results, realizing dynamic comparative analysis and personalized presentation of beauty effects, enabling users to intuitively understand beauty effects and improving user experience. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0021] Figure 1 Illustration of the application scenarios provided in the embodiments of this application Figure 1 ; Figure 2 Illustration of the application scenarios provided in the embodiments of this application Figure 2 ; Figure 3 The flowchart of the cosmetic effect analysis method provided in the embodiments of this application Figure 1 ; Figure 4 An architecture diagram of the AI ​​model provided in the embodiments of this application; Figure 5 Skin radar map provided for embodiments of this application; Figure 6 The flowchart of the cosmetic effect analysis method provided in the embodiments of this application Figure 2 ; Figure 7 The flowchart of the cosmetic effect analysis method provided in the embodiments of this application Figure 3 ; Figure 8 This is a structural diagram of the beauty effect analysis system provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0023] In the embodiments of this application, "at least one" refers to one or more; "multiple" refers to two or more. In the description of this application, the terms "first," "second," "third," etc., are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.

[0024] References such as “one embodiment” or “some embodiments” as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the terms “comprising,” “including,” “having,” and variations thereof, as used in this specification, mean “including, but not limited to,” unless otherwise specifically emphasized.

[0025] It should be noted that in the embodiments of this application, "and / or" describes the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. In addition, the character " / ", unless otherwise specified, generally indicates that the associated objects before and after it are in an "or" relationship.

[0026] First, the application scenarios of the beauty effect analysis method based on image analysis and dialogue interaction provided in the embodiments of this application will be described.

[0027] Please see Figure 1As shown, in one possible application scenario, the beauty effect analysis method based on image analysis and dialogue interaction can be applied to electronic devices in beauty salons, including personal computers, tablets, or other devices composed of electronic components, powered by electricity, and capable of performing data processing / communication / control operations. In specific applications, before a beauty treatment, salon staff can use image acquisition devices, such as professional dermoscopic imaging equipment or digital cameras, to capture skin images of target areas like the user's face and neck under a preset acquisition environment. These images are then used as reference skin images and stored in the user's database. After the user completes the beauty treatment, the same image acquisition device is used to capture skin images of the target areas again, serving as the user's post-treatment skin image. Ideally, the same image acquisition device should be used for both acquisitions under the same acquisition environment to minimize errors caused by the acquisition device and environment. Furthermore, it is preferable to complete the acquisition of the user's skin images on the same day after the beauty treatment to provide more accurate image data for comparative analysis. Based on the analysis method of this application, the electronic device performs comparative analysis on the user's skin images before and after the beauty treatment, generating beauty effect analysis results and personalized dialogue content, which are then presented through the electronic device to assist beauty salon staff in interpreting the beauty effects of the treatment.

[0028] Please see Figure 2 As shown, in another possible application scenario, the beauty effect analysis method based on image analysis and dialogue interaction can be applied to user-side electronic devices, i.e., user terminals, including smartphones, tablets, and other devices with image acquisition and data processing / communication / control operation functions. Before using the home beauty device, the user can use the built-in camera component of the user terminal to acquire skin images under the required acquisition environment; after using the beauty device for a period of time, the user can acquire skin images again using the camera component under the required acquisition environment. The two acquisitions can preferably be performed using the same user terminal under the same acquisition environment to reduce errors caused by the acquisition device and environment. It is also preferable to complete the acquisition of user skin images on the same day after using the beauty device to provide more accurate image data for comparative analysis. Based on the analysis method of this application, the user terminal performs comparative analysis on the user's skin images before and after using the home beauty device, generating beauty effect analysis results and personalized dialogue content, which are presented through the user terminal to allow the user to intuitively understand the beauty effect of the beauty device.

[0029] It is understood that the above application scenarios are merely exemplary descriptions to facilitate understanding of the technical solutions of this application by those skilled in the art, and do not constitute a limitation on the scope of protection or application of this application.

[0030] Please see Figure 3As shown in the figure, the beauty effect analysis method based on image analysis and dialogue interaction provided in this application includes steps S100 to S400, which will be described in detail below.

[0031] Step S100: Obtain the user's skin image.

[0032] The methods for acquiring user skin images include, but are not limited to, two scenarios: self-capture by electronic devices and import from external image acquisition devices, to adapt to the image acquisition needs of different application scenarios. In the scenario of self-capture by electronic devices, electronic devices with built-in shooting components, such as smartphones and tablets, can be used to capture user skin images by calling the built-in shooting components through a mini-program or app. In the scenario of importing from external image acquisition devices, external image acquisition devices such as professional dermatoscopes and digital cameras can be used to capture user skin images and upload them to the electronic device, thus completing the acquisition of user skin images.

[0033] It should be noted that the process of collecting user skin images is carried out with the user's knowledge and authorization, and does not involve any user privacy.

[0034] As a possible implementation, when the electronic device is a user terminal, the user terminal is used to acquire the user's skin image. Step S100 may include steps S110 to S150, which will be described in detail below.

[0035] Step S110: Call the built-in camera component of the user terminal. Specifically, call the corresponding function interface of the built-in camera component of the user terminal, for example, calling it on the mini-program side. <camera>The component, which calls the Camera2 API on the Android App side, captures user skin images through the user terminal's built-in shooting component.

[0036] Step S120: Load AR markers on the shooting interface of the shooting component to prompt the user to adjust to the preset shooting angle and shooting distance. Specifically, AR markers are loaded through a pre-set AR marker library. The AR markers include virtual positioning lines such as forehead alignment lines and nose alignment lines, which are superimposed on the shooting interface in real time to guide the user to align facial features with the virtual positioning lines, so that the shooting angle is kept at the preset front or 45° side view, and the shooting distance is kept in the optimal acquisition range of 20-30cm, ensuring the standardization of shooting angle and shooting distance.

[0037] Step S130: Perform light detection on the current environment. If the light detection is satisfactory, control the shooting component to continuously capture multiple frames of time-series images. Specifically, the ambient light sensor of the user terminal collects the current ambient brightness data and combines it with a preset brightness threshold to determine whether the light is sufficient. If the light is insufficient, a prompt message is output to remind the user to adjust the ambient brightness, or the screen is used to supplement the light until the light is sufficient. Once the light is sufficient, the shooting component is triggered to continuously capture multiple frames of time-series images, such as 3, 4, 5, 6, 7, or 8 frames of time-series images.

[0038] Step S140: Perform temporal alignment and fusion noise reduction processing on multiple frames of temporal images to obtain a denoised image. Specifically, a temporal alignment algorithm is used to perform inter-frame registration on continuously acquired multiple frames of temporal images to eliminate image offset and blur caused by user hand tremors. Then, a multi-frame synthesis algorithm is used to perform pixel fusion on the registered images to suppress image noise and improve the recognition accuracy of fine lines, pores, spots and other detailed features.

[0039] Those skilled in the art will understand that the above-mentioned temporal alignment algorithm can adopt ORB (Oriented Fast and Rotated BRIEF) feature point registration algorithm, optical flow alignment algorithm or other image registration methods, and the multi-frame synthesis algorithm can adopt guided filter fusion algorithm, median fusion algorithm, etc., and this application does not impose specific limitations.

[0040] Step S150: Perform skin color correction on the denoised image to obtain the user's skin image. Specifically, based on Retinex theory, an illumination separation model is constructed to decompose the denoised image into illumination and reflection components, eliminating image brightness deviations caused by uneven ambient lighting and inherent differences in skin color, restoring the true skin feature information, and obtaining a standardized user skin image.

[0041] In the above implementation, steps S110 to S150 involve using the user terminal's camera to acquire user skin images. First, AR markers and ambient light detection are used to control the shooting angle, shooting distance, and lighting conditions, achieving standardized image acquisition. Second, multi-frame temporal alignment fusion noise reduction and skin tone correction processes are combined to eliminate interference from hand shakiness, environmental noise, and uneven lighting on image quality, resulting in more realistic user skin images. These steps provide standardized, high-precision base images for subsequent user skin image comparison and analysis, effectively improving the accuracy of the comparison analysis.

[0042] Step S200: Compare and analyze the user's skin image with the reference skin image to obtain the cosmetic effect analysis results.

[0043] Step S200 first analyzes the skin feature data of the user's skin image by calling an AI model. This skin feature data includes multiple skin feature indicators, such as hydration, pore area, sebum secretion, skin brightness, redness, number and depth of fine lines, skin roughness, number and size of blemishes, etc. The skin feature indicators used in the comparative analysis can be set as needed. For example, when analyzing the effects of anti-aging beauty devices / treatments, the number and depth of fine lines can be compared; when analyzing the effects of hydrating beauty devices / treatments, hydration and sebum secretion can be compared, allowing for targeted analysis of beauty effects. Next, a corresponding reference skin image is determined based on the user's skin image, and the skin feature indicator data of the reference skin image are extracted. The reference skin image can be any historical user skin image taken by the user, or the historical user skin image taken most recently. Finally, by comparing and analyzing the data of various skin feature indicators between the user's skin image and the reference skin image, the variation range of each skin feature indicator data is quantified to obtain the results of the beauty effect analysis, so as to intuitively present the beauty effects of beauty projects and beauty instruments.

[0044] As a possible implementation, step S200 may include steps S210 to S240, which will be described in detail below.

[0045] Step S210: Perform skin feature analysis on the user's skin image to obtain skin feature data of the user's skin image. The skin feature data includes at least one skin feature index.

[0046] In this application, skin feature analysis of user skin images can be performed by calling an AI model. The AI ​​model can be an existing model that supports image and text analysis, and this application does not impose any restrictions on it.

[0047] As an example of an AI model, such as Figure 4 As shown, the top layer of the AI ​​model is the application layer that enables intelligent interaction; below it is the inference engine and task scheduling layer, which undertakes the scheduling central functions such as dynamic routing requests, modality adaptation, multi-task joint invocation, and prompting engineering triggers; below it is the distributed modal encoder layer, which includes visual encoders, text encoders, and speech encoders, each of which runs independently and in parallel; below it is the multimodal alignment and fusion layer, which maps the modal information of different modal encoders to a unified semantic space based on cross-modal attention mechanisms, modal shared representations, and dynamic gating fusion technology; below it is the general large model base, which includes dense models with hundreds of billions of parameters and hybrid expert architectures, and completes the basic capability construction through multi-task pre-training, SFT (Supervised Fine-Tuning), and RLHF (Reinforcement Learning from Human Feedback); the bottom layer is an infrastructure and deployment platform composed of GPU (Graphics Processing Unit) clusters, high-speed network optimization, distributed storage systems, and intelligent scheduling systems, which provides computing power and resource support for the upper-layer capabilities.

[0048] Taking a user's skin image in the face region as an example, the process of using an AI model to analyze the skin features of the user's skin image is as follows: Input the analysis request at the application layer. The analysis request includes the user's skin image and the request text. The request text includes the skin feature indicators to be analyzed and the output rules. For example, the request text is "Please analyze the hydration, pore area, sebum secretion, and skin brightness in the user's skin image, and output the hydration score, pore area score, sebum secretion score, and skin brightness score according to 0-100". The analysis request first enters the inference engine and task scheduling layer, where the dynamic routing module identifies that the input contains both visual and textual content. The user's skin image is then distributed to the visual encoder, and the request text to the text encoder. The visual encoder converts the user's skin image into image embedding vectors, and the text encoder encodes the skin feature indicators in the request text into text embedding vectors. Both image and text embedding vectors are simultaneously transmitted to the multimodal alignment and fusion layer. The fusion layer maps the two types of vectors to a unified semantic space through a cross-attention mechanism, generating multimodal semantic information. This multimodal semantic information is then input into a general large model base. The base, based on its multimodal understanding capabilities, interprets the multimodal semantic information and outputs hydration scores, pore area scores, sebum secretion scores, and skin brightness scores, thus obtaining the skin feature data of the user's skin image.

[0049] In addition, the request text can also request that the user's skin image be divided into multiple regions, and that the skin feature indicators of each region be analyzed to obtain the skin feature indicator data for each region, thus supplementing the skin feature data. For skin feature indicators with regional boundaries, such as the number of fine lines, redness areas, and spots, the request text can also request that the fine line areas, redness areas, and spot areas be marked on the user's skin image to supplement the skin feature data.

[0050] Step S220: Query the corresponding reference skin image from the user database based on the user's skin image, and extract the skin feature data of the reference skin image.

[0051] Each user's captured skin image is stored in a user database. A reference skin image is extracted from this database based on the captured image. This reference image can be any of the user's historical skin images, or the one most recently captured from their historical images. Once the reference image is determined, its skin feature data is extracted. If the reference image's skin feature data is already stored in the user database, it is directly extracted and used. If the database does not contain this data, an AI model is used to extract it, providing benchmark data for subsequent comparative analysis.

[0052] Furthermore, as a possible implementation of step S220, firstly, based on the user information corresponding to the user's skin image, a set of historical skin images corresponding to the user information can be queried from the user database. The set of historical skin images includes multiple historical skin images. Then, according to a preset comparison period, a reference skin image is selected from the set of historical skin images. For example, if the preset comparison period is one week, a historical skin image collected one week ago is selected as the reference skin image; if the preset comparison period is one month, a historical skin image collected one month ago is selected as the reference skin image. Based on the reference skin image, skin feature data of the reference skin image is extracted to achieve effect tracking of beauty projects / beauty instruments.

[0053] Furthermore, when both the reference skin image and the user skin image represent facial regions, the user skin image and the selected reference skin image can be identified before extracting the skin feature data of the reference skin image. This identification process can determine whether the user skin image and the reference skin image belong to the same user. For example, similarity calculations can be performed by extracting features of key points such as eyes, nose, and mouth to obtain the identification result. If the identification result indicates that the user skin image and the reference skin image belong to the same user, then the skin feature data of the reference skin image can be extracted to avoid analysis errors.

[0054] Step S230: Compare and analyze the skin feature data of the user's skin image and the reference skin image to obtain the cosmetic effect analysis results. The cosmetic effect analysis results compare various skin feature indicators in the skin feature data of the user's skin image and the reference skin image, and visualize the changes through image reports or graphic reports. The specific form of visualization can be selected as needed, such as using skin radar maps or skin heat maps, to present the cosmetic effect from multiple perspectives.

[0055] Taking skin radar image as an example, step S230 may include steps S231 to S234, which will be explained in detail below.

[0056] Step S231: Construct a multidimensional coordinate system based on the various skin feature indicators in the skin feature data. Each dimension of the multidimensional coordinate system corresponds to a skin feature indicator. One dimension of the multidimensional coordinate system corresponds to one skin feature indicator. Each dimension has a reasonable data range, which can be determined based on large-scale population skin data. The constructed multidimensional coordinate system provides the foundation for subsequent coordinate point plotting and change comparison.

[0057] Step S232: Plot coordinate points and connect them on a multidimensional coordinate system based on the skin feature index data of the user's skin image. Specifically, based on the skin feature index data of the user's skin image, a first set of coordinate points is plotted on the multidimensional coordinate system. The first set of coordinate points is then connected sequentially according to a fixed dimensional order to form a first polygonal region, thus obtaining a skin radar map. The map can visually present the skin feature index data of the user's skin image.

[0058] Furthermore, in order to intuitively present the changes in various skin feature index data of the user's skin image, a second set of coordinate points can be drawn on a multi-dimensional coordinate system based on the various skin feature index data of the reference skin image. The second set of coordinate points are then connected sequentially in a fixed dimensional order to form a second polygonal region. Through the first polygonal region and the second polygonal region, the changes in various skin feature index data of the user's skin image and the reference skin image can be intuitively presented.

[0059] Step S233: Calculate the variation range of each skin feature index data in the user skin image and the reference skin image, and extract the skin feature indexes whose absolute value of the variation range is greater than a set threshold as the target skin feature indexes. The formula for calculating the variation range is: ; In the formula, Let j be the range of change of the j-th skin characteristic index. Let j be the j-th skin feature index data of the user's skin image. Referencing the j-th skin feature index data from the skin image, valid changes are extracted based on a set threshold, for example, a threshold of 5%. When the absolute value of the change in a certain skin feature index displayed on the radar chart is greater than 5%, the change is considered to exceed the range of natural fluctuations or measurement errors and is regarded as a valid change, and the skin feature index is regarded as the target skin feature index. If the absolute value of the change in a certain skin feature index is less than or equal to 5%, it is considered to be a small fluctuation or measurement error affected by short-term factors (such as water intake, sweating, shooting angle, ambient light, etc.), and the skin feature index is regarded as a non-target skin feature index.

[0060] It should be noted that different threshold values ​​can be set for each skin characteristic indicator to accurately extract the effective changes of each skin characteristic indicator. The threshold value of 5% mentioned above is only an example. The specific value of the threshold value can be determined based on the average change of each skin characteristic indicator before and after the use of beauty treatments and beauty instruments.

[0061] Step S234: Based on the target skin feature indicators, differentiate the corresponding dimensional axes of the multi-dimensional coordinate system to obtain a skin radar map. Specifically, for target skin feature indicators whose changes exceed a set threshold, the corresponding dimensional axes of the target skin feature indicator data are differentiated in the radar map. Differentiation methods include, but are not limited to, highlighting, bolding, and setting symbols on the dimensional axes to visually present the effective change indicators.

[0062] Furthermore, text annotations can be added to the dimensional axis corresponding to the target skin feature index to indicate the magnitude of change. The magnitude of change is obtained through the calculation formula in step S233, for example, text annotations such as -12% or +8% to present the changes in the index.

[0063] Furthermore, the color of the axis corresponding to the target skin feature indicator can be modified to indicate whether the change in that skin feature indicator data is positive or negative. Specifically, the positive or negative nature of the change can be determined based on the magnitude of the change and the meaning of the skin feature indicator. For example, increased hydration and decreased pore area are positive changes, while decreased hydration and increased pore area are negative changes. To visually represent whether the change in skin feature indicator data is positive or negative, the axis corresponding to the skin feature indicator can be set to green to indicate a positive change and red to indicate a negative change.

[0064] Building upon this, the degree of change can be further represented by the intensity of color, with the color of the dimensional axis deepening as the absolute value of the change increases. For example, a darker green can be used for the dimensional axis when the change in the number of spots is -12%, a lighter green for -7%, a lighter red for +8%, and a darker red for +13%, visually representing whether the change in the indicator is positive or negative, and the degree of change.

[0065] As an example, please refer to Table 1, which contains skin feature data for a user's skin image and a reference skin image.

[0066] Table 1

[0067] According to Table 1, when the threshold is set to 5%, hydration and pore area are considered as target skin characteristic indicators. Figure 5 As shown, the dashed lines in the skin radar map represent the various skin feature indicators of the reference skin image, while the dashed lines represent the various skin feature indicators of the user's skin image. The dimensions corresponding to hydration and pore area are distinguished by bolding and using symbols. Text labels are also added to the dimensions corresponding to hydration and pore area to visually present the effective change indicators and the magnitude of their changes.

[0068] In the above implementation, steps S231 to S234 first construct a multi-dimensional coordinate system that corresponds one-to-one with the skin feature indicators to realize the visualization of multi-dimensional skin data; secondly, by combining the set threshold to filter the target skin feature indicators with effective change range, the influence of natural fluctuations and measurement errors on the indicator data is eliminated, and the indicators reflecting the improvement effect or the deterioration effect are extracted; finally, by distinguishing the axis of the target indicator, the indicators reflecting the improvement effect or the deterioration effect are presented intuitively.

[0069] Taking skin thermography as an example, step S230 may include steps S235 to S239, which will be explained in detail below.

[0070] Step S235: Use the user's skin image as a base map and divide the base map into multiple regions. The base map can be divided into multiple regions according to the T, U, V classification method commonly used in the beauty field. The T region refers to the T-shaped area in the center of the face, covering the forehead, nose, and chin; the U region refers to the U-shaped area extending downwards from the top of the face, covering both cheeks, cheekbones, temples, and temples; the V region refers to the contour area of ​​the lower half of the face, covering the jawline, chin, and ear base. Skin anatomy partitioning rules or other custom partitioning rules can also be used. This application does not specifically limit the partitioning rules, as long as the skin features of each region can be accurately analyzed.

[0071] Step S236: Based on the skin feature data of the user skin image and the reference skin image, analyze the skin feature index data of the user skin image and the reference skin image in each region.

[0072] Specifically, various skin feature indicators, such as hydration, pore area, sebum secretion, skin brightness, and redness, are extracted from user skin images and reference skin images in each region to provide a data foundation for subsequent calculation of regional changes.

[0073] Step S237: Calculate the variation range of each region based on the skin feature index data of the user's skin image and the reference skin image in each region. Specifically, the variation range of each region is calculated based on individual skin feature index data to present the changes of each individual skin feature index in each region.

[0074] The formula for calculating the variation range of each region is as follows: ; in, Let j be the variation range of the j-th skin feature index in the i-th region. For the j-th skin feature index data of the i-th region in the user's skin image, The reference is the j-th skin feature index data of the i-th region in the skin image.

[0075] Step S238: Map the change amplitude to fill color according to the preset mapping rules to obtain the fill color of each region. Specifically, for each skin feature index, the change amplitude of each region corresponding to the skin feature index is mapped to the fill color of each region according to the mapping rules to obtain the fill color of the skin feature index in each region.

[0076] The mapping rules can be specifically set as follows: when the absolute value of the change in a region is less than or equal to a set threshold, it is considered a change caused by short-term fluctuations, measurement errors, or other interference factors, and the fill color of that region is set to gray; when the absolute value of the change in a region is greater than the set threshold and the change indicates a positive change, the fill color of that region is set to green, and the color depth increases with the increase of the absolute value of the change to indicate the degree of improvement; when the absolute value of the change in a region is greater than the set threshold and the change indicates a negative change, the fill color of that region is set to red, and the color depth increases with the increase of the absolute value of the change to indicate the degree of deterioration. Specifically, the sign of the change and the meaning of skin characteristic indicators can be used to determine whether the change is positive or negative. For example, increased hydration and reduced pore area are considered positive changes, while decreased hydration and increased pore area are considered negative changes.

[0077] Step S239: Color the corresponding areas of the base map according to the fill color to obtain a skin heatmap. Specifically, for each skin feature indicator, color each area of ​​the base map according to the fill color of that skin feature indicator in each area. This can be done by using an overlay of transparency to preserve the skin contour and texture features. After coloring all areas, a skin heatmap of that skin feature indicator is generated to visually present the improvement or deterioration of that skin feature indicator in each area.

[0078] Multiple skin feature indicators are colored on multiple base maps to form multiple skin heat maps, which show the improvement or deterioration of each skin feature indicator in each region.

[0079] In the above implementation, steps S235 to S239 divide the base map into multiple regions using partitioning rules, extract various skin feature index data of the user's skin image and the reference skin image in each region, calculate the variation range of each skin feature index in each region, and map the variation range to the fill color of the region using mapping rules to draw a skin heatmap to present the changes of each skin feature index in each region. The mapping rules visually represent improvement and deterioration through color, while eliminating interference factors such as short-term fluctuations and measurement errors, and visually present the degree of improvement and deterioration through changes in color depth.

[0080] Step S300: Based on user information and beauty effect analysis results, generate personalized dialogue content adapted to user information.

[0081] As a possible implementation, step S300 may include steps S310 to S330, which will be described in detail below.

[0082] Step S310: Extract user keywords based on the beauty effect analysis results. Specifically, user keywords are extracted from the beauty effect analysis results by identifying skin feature indicators whose absolute value of change exceeds a set threshold, along with the signs of their changes. For skin radar maps, user keywords can be extracted based on the distinguished parts; for skin heatmaps, user keywords can be extracted based on their fill color.

[0083] Step S320: Match a dialogue template from a pre-built dialogue template library based on user keywords, supplement the dialogue template according to the beauty effect analysis results, and generate dialogue content. The dialogue template library can be built based on NLP (Natural Language Processing) technology, storing the correspondence between user keywords and dialogue templates. The mapping relationship between user keywords and dialogue templates is established using NLP semantic classification and matching algorithms. The dialogue template contains placeholders, which are filled with the variation range of skin feature indicators corresponding to the user keywords.

[0084] For example, a beauty effect analysis result shows hydration +18.3% and pore area -22.5%. Extracting the user keywords "hydration +" and "pore area -", the matching script template from the script template library is "Hydration detected". "It is recommended to maintain moisturizing care" and "The area of ​​pores has been detected". "Keep oil-controlling skincare routine at night." This indicates a placeholder; the generated dialogue content is "Hydration level detected +18.3%, moisturizing care recommended; Pore area detected -22.5%, oil control care needed at night." For example, please refer to Table 2, which exemplarily shows the correspondence between user keywords and dialogue templates in the dialogue template library.

[0085] Table 2

[0086] It should be noted that for certain skin feature indicators that are related, such as the number and depth of fine lines, the number and size of spots, these skin feature indicators are usually analyzed in association and can be combined as user keywords.

[0087] In addition, since the script template library built by NLP technology is quite large, in order to ensure that the generated dialogue content is safe, compliant and unambiguous, compliance verification can also be performed on the generated dialogue content. For example, by querying the prohibited word database, prohibited words that imply medical efficacy or false advertising can be avoided in the dialogue content.

[0088] Step S330: Select a personalized interaction method based on user information, and generate personalized dialogue content adapted to the user information based on the personalized interaction method. User information may include: user age, user gender, etc. This user information can be obtained from user information entered into a user database, or by inputting the user's skin image into an AI model and combining it with prompt word engineering. Personalized interaction methods can be different styles of voice packs, such as a calm voice pack or a lively voice pack. Select the corresponding voice pack based on the user information; for example, select a lively voice pack for female users aged 20-35, and a calm voice pack for female users over 35. Output dialogue content based on the selected voice pack to achieve personalized interaction.

[0089] In the above implementation, steps S310 to S330 generate dialogue content by extracting user keywords and matching the dialogue template, ensuring that the dialogue content corresponds to the analysis results. Personalized interaction methods are selected based on user information to achieve personalized adaptation of dialogue content, which is in line with the interaction preferences of different user groups and helps to enhance user experience and user stickiness.

[0090] Step S400: Present the beauty effect analysis results and personalized dialogue content to the user via an electronic device. The beauty effect analysis results are presented in a visual and interactive report display page. The report display page supports sliding to compare before and after images, and clicking on areas to view detailed indicators, enhancing the user's perception of changes in skin condition and improving the interactive experience.

[0091] As a possible implementation, step S400 may include steps S410 to S430, which will be described in detail below.

[0092] Step S410: Display the cosmetic effect analysis results, user skin image, and reference skin image via an electronic device. Specifically, the cosmetic effect analysis results (such as skin radar map, skin heat map, etc.), user skin image, and reference skin image are integrated into the same report display page and visualized via an electronic device. For example, the user skin image and reference skin image are located in the same window of the report display page, responding to the user's left and right swipes to switch between viewing the user skin image and the reference skin image for a direct comparison of the cosmetic effect; the cosmetic effect analysis results can be located in another window of the report display page to present the quantitative results of the cosmetic effect.

[0093] Step S420: Determine the target analysis area based on the user's actions on the displayed user skin image. These user actions include, but are not limited to, clicking or selecting specific areas of the user skin image on the report display page; the area containing these selected areas is recorded as the target analysis area. The division of the area can follow the T, U, V classification method commonly used in the beauty industry.

[0094] Step S430: Extract skin feature data from the user's skin image and the reference skin image in the target analysis area, and perform comparative analysis to obtain the cosmetic effect analysis results for the target analysis area. This result is then presented to the user via an electronic device. Specifically, skin feature index data for each area are extracted from the cosmetic effect analysis results and presented in a pop-up window to allow users to understand the quantitative data of the cosmetic effect in each area.

[0095] In the above implementation, steps S410 to S430 present the beauty effect analysis results through the report display page, combined with the function of sliding to compare before and after images, to intuitively show the changes in the skin before and after the intervention of beauty projects / beauty instruments; in addition, detailed indicators of the area can be viewed by the user, presenting the local beauty effect analysis results, realizing a comprehensive display of beauty effects from the whole to the part.

[0096] The beauty effect analysis method based on image analysis and dialogue interaction provided in this application embodiment may further include: pushing a first user operation task to an electronic device before comparing and analyzing the user's skin image with a reference skin image to obtain the beauty effect analysis result; and when the electronic device is detected to have completed the first user operation task, comparing and analyzing the user's skin image with the reference skin image to obtain the beauty effect analysis result.

[0097] Please see Figure 6 As shown, when the analysis method of this application is applied to a user terminal, after obtaining the user skin image in step S100 and before performing image comparison analysis between the user skin image and the reference skin image in step S200, a first user operation task can be pushed to the user terminal. The first user operation task can be a payment task, an advertisement click task, a questionnaire filling task, etc., and this application does not impose specific limitations. When it is detected that the user terminal has completed the first user operation task, the content of step S200 is executed. When it is detected that the user terminal has not completed the first user operation task, the user skin image is stored in the user database for the next comparison analysis.

[0098] In the above implementation, by pushing the first user operation task, the image comparison analysis in step S200 is triggered after the user completes the first user operation task, which can effectively filter invalid analysis requests and reduce the system's computing power consumption.

[0099] Please see Figure 7 As shown in the embodiments of this application, the beauty effect analysis method based on image analysis and dialogue interaction may further include: after presenting the beauty effect analysis results and personalized dialogue content to the user through an electronic device, detecting whether the electronic device issues a request to unlock the care plan; when the electronic device issues a request to unlock the care plan, pushing a second user operation task to the electronic device; when the electronic device completes the second user operation task, matching the corresponding care plan in the preset care plan library according to the personalized dialogue content and pushing it.

[0100] The second user operation task can be a payment task, an ad click task, a questionnaire completion task, etc., and this application does not impose specific restrictions. When the electronic device is detected to have completed the second user operation task: First, the NLP semantic extraction algorithm is used to extract indicators and indicator change trends from the personalized dialogue content to obtain keywords; for example, from the dialogue content "Hydration level +18.3% detected, it is recommended to maintain moisturizing care; pore area -22.5% detected, oil control care is needed at night", the keywords "hydration level +" and "pore area -" are extracted. Second, a pre-set care plan library stores the correspondence between keywords and care plans. Keywords can include single indicators and multiple related indicators, such as hydration level +, pore area -, fine line number - & fine line depth -, etc.; each keyword corresponds to a set of pre-set care plans, which can include care procedures, applicable products, care frequency, precautions, etc. Finally, the care plans in the care plan library are matched according to the keywords and pushed to the electronic device for presentation via instant messaging methods such as SMS and WeChat.

[0101] Through the above implementation methods, the nursing plan is pushed to users by actively triggering the execution of the nursing plan. The nursing plan is matched with the beauty effect analysis, and the nursing plan is adapted to the beauty effect analysis, realizing a closed-loop service from beauty effect analysis to solution.

[0102] Furthermore, the beauty effect analysis method based on image analysis and dialogue interaction provided in this application embodiment may further include: after presenting the beauty effect analysis results and personalized dialogue content to the user through an electronic device, matching the corresponding product information in a preset product push library according to the personalized dialogue content and pushing it.

[0103] First, NLP semantic extraction algorithms are used to extract negative changes from personalized dialogue content to obtain keywords. For example, the keyword "pore area -" is extracted from the dialogue content "Hydration detected +18.3%, moisturizing care recommended; pore area detected -22.5%, oil control care needed at night." Second, a pre-built product push library stores various product information and corresponding product keywords. Product information can include the product name and description. For example, the product information of a certain product in the product push library is "XX beauty essence can shrink pore area," and its corresponding product keyword is "pore area -." Finally, based on the keywords extracted from personalized dialogue content, the corresponding product keywords are matched from the pre-built product library, and the product information is pushed to electronic devices via instant messaging methods such as SMS and WeChat.

[0104] Through the above implementation method, after the analysis results are presented, corresponding product information is pushed by matching the personalized dialogue content. The product information is adapted to the beauty effect analysis results, providing users with applicable product references.

[0105] Please see Figure 8 As shown in the embodiments of this application, a beauty effect analysis system based on image analysis and dialogue interaction is also provided. The system includes: an image acquisition module, an image analysis module, a dialogue generation module, and an analysis presentation module. The image acquisition module is used to acquire user skin images; the image analysis module is used to compare and analyze the user skin images with reference skin images to obtain beauty effect analysis results; the dialogue generation module is used to generate personalized dialogue content adapted to user information based on user information and the beauty effect analysis results; and the analysis presentation module is used to present the beauty effect analysis results and personalized dialogue content to the user through an electronic device.

[0106] It should be noted that the beauty effect analysis system based on image analysis and dialogue interaction provided in this application embodiment is used to implement the beauty effect analysis method based on image analysis and dialogue interaction provided in the above embodiment. It corresponds one-to-one with the method and achieves the corresponding effect. For details, please refer to the above method embodiment section, which will not be elaborated here.

[0107] It is understood that the beauty effect analysis method and system based on image analysis and dialogue interaction provided in this application obtains quantitative beauty effect analysis results by comparing and analyzing user skin images with reference skin images. At the same time, it generates personalized dialogue content by combining user information and beauty effect analysis results, realizing dynamic comparative analysis and personalized presentation of beauty effects, allowing users to intuitively understand beauty effects and improving user experience. In addition, this application also calls AI models for comparative analysis to realize personalized customization analysis based on AI models.

[0108] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0109] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although this application has disclosed preferred embodiments as above, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.< / camera>

Claims

1. A method for analyzing cosmetic effects based on image analysis and dialogue interaction, characterized in that, The method includes: Obtain the user's skin image; The user's skin image is compared and analyzed with a reference skin image to obtain the cosmetic effect analysis results; Based on user information and the beauty effect analysis results, generate personalized dialogue content adapted to the user information; The results of the beauty effect analysis and the personalized dialogue content are presented to the user via an electronic device.

2. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The step of comparing and analyzing the user's skin image with a reference skin image to obtain the cosmetic effect analysis result includes: Perform skin feature analysis on the user's skin image to obtain skin feature data of the user's skin image, wherein the skin feature data includes at least one skin feature index data; Based on the user's skin image, a corresponding reference skin image is retrieved from the user database, and skin feature data of the reference skin image is extracted. The skin feature data of the user's skin image and the reference skin image are compared and analyzed to obtain the cosmetic effect analysis results.

3. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 2, characterized in that, The step of querying a corresponding reference skin image from the user database based on the user's skin image and extracting skin feature data from the reference skin image includes: Based on the user information corresponding to the user's skin image, query the set of historical skin images corresponding to the user information from the user database; According to a preset comparison period, a reference skin image is selected from the set of historical skin images; Based on the reference skin image, extract the skin feature data of the reference skin image.

4. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 3, characterized in that, Before extracting skin feature data from the reference skin image, the method further includes: The user skin image and the selected reference skin image are identified. When the identification result indicates that the user skin image and the reference skin image belong to the same user, the skin feature data of the reference skin image is extracted.

5. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 2, characterized in that, The cosmetic effect analysis results include a skin radar map; the comparison and analysis of skin feature data between the user's skin image and the reference skin image to obtain the cosmetic effect analysis results includes: A multidimensional coordinate system is constructed based on each of the skin feature indicators in the skin feature data, and each axis of the multidimensional coordinate system corresponds to each of the skin feature indicators. Based on the skin feature index data of the user's skin image, coordinate points are plotted and connected on the multidimensional coordinate system; Calculate the variation range of each skin feature index data in the user skin image and the reference skin image, and extract the skin feature index whose absolute value of the variation range is greater than a set threshold as the target skin feature index; Based on the target skin feature indicators, the corresponding dimensional axes of the multidimensional coordinate system are displayed differently to obtain the skin radar map.

6. A method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 2 or 5, characterized in that, The cosmetic effect analysis results include a skin heatmap; the comparison and analysis of skin feature data between the user's skin image and the reference skin image to obtain the cosmetic effect analysis results includes: The user's skin image is used as a base image, and the base image is divided into multiple regions; Based on the skin feature data of the user skin image and the reference skin image, analyze the skin feature index data of the user skin image and the reference skin image in each region; Based on the skin feature index data of the user skin image and the reference skin image in each region, calculate the change range of each region; The change amplitude is mapped to fill color according to the preset mapping rules to obtain the fill color of each region; The corresponding area of ​​the base map is colored according to the fill color to obtain the skin heat map.

7. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The step of generating personalized dialogue content adapted to the user information based on the user information and the beauty effect analysis results includes: Extract user keywords based on the beauty effect analysis results; Based on the user keywords, a dialogue template is matched from a pre-set dialogue template library. Based on the beauty effect analysis results, the dialogue template is supplemented to generate dialogue content. Based on the user information, a personalized interaction method is selected, and personalized dialogue content adapted to the user information is generated based on the personalized interaction method.

8. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The process of presenting the beauty effect analysis results and the personalized dialogue content to the user via an electronic device includes: The beauty effect analysis results, the user's skin image, and the reference skin image are displayed via an electronic device; The target analysis area is determined based on the user's actions on the displayed user skin image; Skin feature data of the user's skin image and the reference skin image are extracted from the target analysis area and compared and analyzed to obtain the beauty effect analysis result of the target analysis area, which is then presented to the user through the electronic device.

9. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The method further includes: Before comparing and analyzing the user's skin image with a reference skin image to obtain the cosmetic effect analysis result, a first user operation task is pushed to the electronic device. When the electronic device is detected to have completed the first user operation task, the user's skin image is compared and analyzed with a reference skin image to obtain the cosmetic effect analysis result.

10. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The method further includes: After presenting the beauty effect analysis results and the personalized dialogue content to the user via an electronic device, it is detected whether the electronic device issues a request to unlock the care plan; When a request to unlock a care plan is detected from the electronic device, a second user operation task is pushed to the electronic device. When the electronic device is detected to have completed the second user operation task, a corresponding nursing plan is matched in the preset nursing plan library according to the personalized dialogue content and pushed.

11. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, The method further includes: After the beauty effect analysis results and the personalized dialogue content are presented to the user through an electronic device, the corresponding product information is matched with the pre-set product push library according to the personalized dialogue content and pushed.

12. The method for analyzing beauty effects based on image analysis and dialogue interaction according to claim 1, characterized in that, When the electronic device is a user terminal, the user terminal is used to acquire the user skin image, wherein acquiring the user skin image includes: Invoke the camera component of the user terminal; AR markers are loaded on the shooting interface of the shooting component to prompt the user to adjust to the preset shooting angle and shooting distance; Perform light detection on the current environment, and control the imaging component to continuously capture multiple frames of time-series images after the light detection is qualified; The multi-frame temporal images are subjected to temporal alignment and fusion noise reduction processing to obtain a denoised image; Skin color correction is performed on the denoised image to obtain the user's skin image.

13. A beauty effect analysis system based on image analysis and dialogue interaction, characterized in that, include: The image acquisition module is used to acquire images of the user's skin. The image analysis module is used to compare and analyze the user's skin image with a reference skin image to obtain the cosmetic effect analysis results; The dialogue generation module is used to generate personalized dialogue content adapted to the user information based on the user information and the beauty effect analysis results. The analysis and presentation module is used to present the beauty effect analysis results and the personalized dialogue content to the user through an electronic device.

Images