Hand skin state analysis method based on ai image recognition and related device

Abstract

This application provides a method and related equipment for analyzing hand skin condition based on AI image recognition. The method includes: acquiring a hand image to be analyzed; extracting features from the hand image to obtain hand visual features, and inputting the hand visual features into a preset hand image classification model to obtain preliminary analysis results of hand skin condition; inputting the hand visual features and preliminary analysis results into a preset skincare semantic language model to obtain corresponding skincare semantic description text; comparing the skincare semantic description text with skincare common sense rules in a preset skincare knowledge base to obtain skincare deviation comparison results; and correcting the preliminary analysis results based on the skincare deviation comparison results to obtain the target analysis result of hand skin condition in the hand image. This application can make the hand skin condition analysis results more accurate, providing effective data support for subsequent customized skincare product formulations.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, specifically to a method and related equipment for analyzing hand skin condition based on AI image recognition. Background Technology

[0002] With the growing demand for personalized skincare products, computer vision-based skin condition analysis technology is being widely applied in user skin assessment scenarios.

[0003] Related technologies utilize deep neural networks to learn features from visual information such as color, texture, and contour in user hand images, and output classification or scoring results for hand skin conditions such as dryness, fine lines, and roughness. However, the hand skin condition analysis results obtained in this way are often not accurate enough and cannot provide effective data support for subsequent customized skincare product formulations. Summary of the Invention

[0004] The embodiments of this application provide a method and related equipment for analyzing the skin condition of the hands based on AI image recognition, which aims to make the analysis results of the skin condition of the hands more accurate and provide effective data support for subsequent customized skin care product formulations.

[0005] In a first aspect, embodiments of this application provide a method for analyzing hand skin condition based on AI image recognition, the method comprising:

[0006] Obtain the hand image to be analyzed;

[0007] Feature extraction is performed on the hand image to obtain hand visual features, and the hand visual features are input into a preset hand image classification model to obtain preliminary analysis results of hand skin condition;

[0008] The hand visual features and the preliminary analysis results are input into a preset skin care semantic big language model to obtain the corresponding skin care semantic description text;

[0009] The skincare semantic description text is compared with the skincare common sense rules in the preset skincare knowledge base to obtain the skincare deviation comparison results.

[0010] Based on the skin care deviation comparison results, the preliminary analysis results are corrected to obtain the target analysis results of the hand skin condition in the hand image.

[0011] In the above embodiments, feature extraction is performed on hand images to obtain hand visual features. These features are then combined with a hand image classification model to obtain preliminary analysis results. The hand visual features and preliminary analysis results are then input into a skincare semantic language model to obtain skincare semantic description text. This text is then compared with skincare common sense rules in a skincare knowledge base for consistency. Corrections are made based on the skincare deviation comparison results. This approach allows for joint verification of hand skin condition using visual features, semantic description text, and skincare common sense rules, resulting in more accurate hand skin condition analysis results and providing effective data support for subsequent customized skincare product formulations.

[0012] In one embodiment, the step of extracting features from the hand image to obtain hand visual features includes:

[0013] The hand image is normalized in size and converted to a preset color gamut to obtain a hand image with a target color gamut.

[0014] Semantic segmentation is performed on the target color gamut hand image to obtain the region of interest hand image;

[0015] The hand image of the region of interest is filtered, denoised, and illuminated to obtain an enhanced hand image;

[0016] Skin color detection is performed on the enhanced hand image to extract a pure hand mask for the pure skin region;

[0017] A preprocessed hand image is generated based on the clean hand mask and the enhanced hand image;

[0018] Feature extraction is performed on the preprocessed hand image to obtain the visual features of the hand.

[0019] In the above embodiments, by performing size normalization, preset color gamut conversion, semantic segmentation, filtering and denoising, illumination equalization and skin color detection on the hand image in sequence, and combining it with a clean hand mask to generate a pre-processed hand image, the hand skin area can be highlighted, the influence of background area, noise information and illumination differences on feature extraction can be reduced, and the correspondence between the pre-processed hand image and the real hand skin area can be improved.

[0020] In one embodiment, the step of extracting features from the preprocessed hand image to obtain the visual features of the hand includes:

[0021] Extracting shallow hand features from the preprocessed hand image, wherein the shallow hand features include at least one of hand skin color information, hand skin texture information, and hand skin morphology information;

[0022] The preprocessed hand image is input into a preset depth visual coding model to obtain deep hand features;

[0023] The shallow hand features and the deep hand features are dimensionally aligned to obtain initial two-dimensional features;

[0024] Based on a preset attention mechanism, the feature weights of each feature in the initial two-dimensional features are obtained, and multiple features in the initial two-dimensional features are weighted and fused according to the feature weights to obtain fused hand features.

[0025] Based on the fused hand features, feature dimensionality reduction processing is performed to obtain the hand visual features.

[0026] In the above embodiments, by extracting shallow hand features from the preprocessed hand image and combining them with a deep visual coding model to obtain deep hand features, and then performing dimension alignment, feature weight acquisition, weighted fusion and feature dimensionality reduction processing, it is possible to simultaneously represent hand skin color information, hand skin texture information and deep semantic information, thereby reducing the information loss problem caused by single feature extraction.

[0027] In one embodiment, the hand image classification model is trained through the following steps:

[0028] Obtain the fused hand sample features and corresponding manually labeled data from each sample image in a preset set of hand skin sample images;

[0029] Based on the feature weights of each sample feature in the fused hand sample features, the correlation between the fused hand sample features and the manually labeled tags is determined;

[0030] Sample images corresponding to the fused hand sample features with a correlation degree less than a preset correlation threshold are designated as anomaly samples.

[0031] The labeled abnormal samples are input into a preset label prediction model to obtain the corrected label of the labeled abnormal samples, and the corresponding manual label is replaced with the corrected label to obtain a set of replaced hand skin sample images;

[0032] The hand image classification model is obtained by training the model based on the replaced set of hand skin sample images.

[0033] In the above embodiments, by determining the correlation between the hand sample features and the manually labeled features based on the feature weights of the fused hand sample features, and by filtering out the labeled abnormal samples, the label prediction model is used to replace and correct the manually labeled features. This can reduce the impact of abnormal labeled samples on the quality of training data and improve the consistency between the replaced hand skin sample image set and the real skin condition.

[0034] In one embodiment, training the model based on the replaced set of hand skin sample images to obtain the hand image classification model includes:

[0035] The set of replaced hand skin sample images is divided into a training sample set and a validation sample set;

[0036] The fused hand sample features of each sample image in the training sample set are input into a preset initial network model to obtain the training classification result;

[0037] Based on the training classification results, the corrected labeling labels corresponding to each sample image, and the preset parameter regularization term, the training loss value of the initial network model is calculated.

[0038] Based on the training loss value, the model parameters of the initial network model are adjusted to obtain an iterative network model;

[0039] Obtain the current verification accuracy value of the verification sample set on the iterative network model, and when the number of consecutive times the current verification accuracy value does not increase is equal to a preset stopping round threshold, determine the iterative network model as the hand image classification model.

[0040] In the above embodiments, by dividing the set of replaced hand skin sample images into training sample set and validation sample set, adjusting the model parameters of the initial network model based on the training loss value, and then combining the current validation accuracy value and the preset stopping round threshold to determine the hand image classification model, it is possible to take into account both the model parameter optimization process and the validation results, and reduce the classification bias caused by overtraining.

[0041] In one embodiment, after obtaining the target analysis result of the hand skin state in the hand image, the method further includes:

[0042] Extract the initial predicted probability of the hand skin condition from the target analysis results;

[0043] At least two target probability calibration algorithms are selected from a plurality of preset candidate probability calibration algorithms, wherein the candidate probability calibration algorithms include at least one of linear scaling algorithm, Pratt scaling algorithm and Bayesian calibration algorithm;

[0044] The at least two target probability calibration algorithms are spliced ​​together to obtain a combined probability calibration algorithm.

[0045] The initial prediction probability is calibrated using the combined probability calibration algorithm to obtain the calibrated prediction probability.

[0046] The initial predicted probability in the target analysis result is replaced with the calibrated predicted probability to obtain the optimized skin condition analysis result.

[0047] In the above embodiments, by extracting the initial prediction probability from the target analysis results, selecting at least two target probability calibration algorithms for process splicing, and then performing probability calibration on the initial prediction probability, the processing results of different candidate probability calibration algorithms can be integrated, the probability bias caused by a single probability calibration algorithm can be reduced, and the matching degree between the calibrated prediction probability and the classification result can be improved.

[0048] In one embodiment, after obtaining the optimized skin condition analysis results, the method further includes:

[0049] The calibrated prediction probability is extracted from the optimized skin condition analysis results;

[0050] Based on the calibration prediction probability, the state quantification index corresponding to each hand skin state is determined;

[0051] Based on the quantitative indicators of each state, determine the corresponding ingredient requirement indicators for the skin condition of the hands;

[0052] Using the ingredient requirement index, corresponding candidate hand care ingredients and candidate concentration ranges of the candidate hand care ingredients are determined from a preset skin care ingredient database.

[0053] The calibration prediction probability, the candidate hand care ingredients, and the candidate concentration range are concatenated to obtain the formula derivation condition text.

[0054] Using a preset formula parameter generation model, the formula combination and proportion calculation are performed on the formula derivation condition text to obtain the target hand care ingredient formula scheme.

[0055] In the above embodiments, by extracting the calibration prediction probability from the optimized skin condition analysis results, and determining the condition quantification index and ingredient requirement index accordingly, and then combining the skin care ingredient database to determine the candidate hand care ingredients and candidate concentration ranges, and generating the target hand care ingredient formulation scheme, the skin condition analysis results can be converted into the basis for ingredient selection and concentration determination, thereby improving the correspondence between the formulation generation results and the skin condition.

[0056] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the AI ​​image recognition-based hand skin state analysis methods described in the application.

[0057] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the AI ​​image recognition-based hand skin state analysis methods described in the present application.

[0058] Fourthly, embodiments of this application provide a hand skin state analysis system based on AI image recognition, including a computer program that, when executed by a processor, implements the steps of any of the hand skin state analysis methods based on AI image recognition. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart illustrating a hand skin condition analysis method based on AI image recognition provided in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0062] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. In addition, in the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0063] In a first aspect, embodiments of this application provide a hand skin condition analysis method based on AI (Artificial Intelligence) image recognition, applied to an AI image recognition-based hand skin condition analysis system. The AI ​​image recognition-based hand skin condition analysis system can run on an electronic device, and during operation, it can be used to execute the steps of any embodiment of the AI ​​image recognition-based hand skin condition analysis method.

[0064] Specifically, refer to Figure 1 A hand skin condition analysis method based on AI image recognition may include:

[0065] S101. Obtain the hand image to be analyzed.

[0066] In this embodiment of the application, a hand image refers to an image containing the skin surface of a human hand area. This hand image can be captured and uploaded by a user using a camera on a specific device, and its pixel information reflects the color, texture, and shape of the hand skin under specific lighting conditions.

[0067] S102. Extract features from the hand image to obtain hand visual features, and input the hand visual features into the preset hand image classification model to obtain preliminary analysis results of the hand skin condition.

[0068] In the embodiments of this application, hand visual features refer to a set of numerical vectors calculated from hand images that can quantitatively characterize the state of hand skin.

[0069] A hand image classification model is a pre-trained artificial intelligence model that can output a classification prediction of the condition of the hand skin based on the input visual features of the hand.

[0070] Preliminary analysis results refer to the initial judgments about the skin condition of the hand output by the hand image classification model, such as skin condition labels like dryness, sensitivity, and wrinkle level, and their corresponding initial predicted probabilities.

[0071] In some embodiments of this application, in step S102, multi-dimensional features such as color histogram, local binary pattern texture features, and scale-invariant feature transform keypoints can be extracted from the hand image. These features collectively constitute the hand visual features. Next, the hand visual feature vector is input into a trained hand image classification model. The multi-layer neural network structure within the hand image classification model performs nonlinear transformations and mapping calculations on this feature vector, ultimately outputting a probability distribution vector. The preliminary analysis results of the hand skin state include the skin state label with the highest initial predicted probability in this vector and its corresponding initial predicted probability.

[0072] S103. Input the visual features of the hand and the preliminary analysis results into the preset skin care semantic big language model to obtain the corresponding skin care semantic description text.

[0073] In this embodiment, the skincare semantic large language model refers to a large language model that has been fine-tuned and trained with a large amount of text data in the skincare field, capable of understanding and generating natural language descriptions related to skincare. The skincare semantic description text refers to the text content generated by this model, which uses natural language sentences to explain and expand upon the preliminary analysis results.

[0074] In some embodiments of this application, in step S103, key values ​​from the visual features of the hand can be combined with skin condition labels from the preliminary analysis results to construct a structured text prompt. For example, the prompt could be "Hand skin features are [color feature value, texture feature value], the model initially judges it to be moderately dry, please generate descriptive text." This text prompt is then input into the skincare semantic language model. Based on its internal semantic knowledge, the skincare semantic language model can generate coherent skincare semantic descriptive text such as "Hand skin shows slight peeling, and the texture is slightly rough, which matches the typical characteristics of moderate dryness."

[0075] In the specific engineering deployment phase, the skincare semantic large language model or formula parameter generation model can be selected from a large-scale language model that has been licensed as open source. To enable it to have professional skincare knowledge derivation capabilities, it can also be combined with the constructed skincare mechanism relation library and QLoRA (Quantized Low-Rank Adaptation) or PTuning (Prompt-tuning) techniques to perform lightweight fine-tuning of its model parameters. This reduces computational consumption while maintaining the generalization ability of the basic large model language, and is suitable for the formula derivation application in the subsequent embodiments of this application.

[0076] S104. Compare the skincare semantic description text with the skincare common sense rules in the preset skincare knowledge base to obtain the skincare deviation comparison results.

[0077] In this embodiment, the preset skincare knowledge base refers to a database storing structured knowledge such as skin physiology, common skin problems, and corresponding care principles. Skincare common sense rules refer to logical rules stored in the knowledge base in "if-then" format, used to describe the generally accepted correlation between skin condition and appearance characteristics. Skincare deviation comparison results refer to a quantitative indicator used to reflect the degree of logical conflict or matching between the description in the skincare semantic description text and the skincare common sense rules.

[0078] In some embodiments of this application, in step S104, all common-sense skincare rules related to the skin condition labels in the preliminary analysis results can be retrieved from a preset skincare knowledge base. Then, natural language processing technology is used to perform semantic similarity calculation or logical consistency checks on the skincare semantic description text and these rules. For example, if a rule is "dry skin is often accompanied by dandruff," while the generated description text is "smooth skin without dandruff," then the distance between the two in the semantic vector space is calculated or directly determined to be contradictory. Finally, the results of comparing all rules are combined to generate a score representing overall consistency or a list containing specific conflicting entries, as the skincare deviation comparison result.

[0079] S105. Based on the skin care deviation comparison results, the preliminary analysis results are corrected to obtain the target analysis results of the hand skin condition in the hand image.

[0080] In this embodiment of the application, the target analysis result refers to the final hand skin condition judgment conclusion obtained after consistency verification and logical correction, and its credibility is higher than that of the preliminary analysis result.

[0081] In some embodiments of this application, in step S105, if the skincare deviation comparison result indicates that the skincare semantic description text is highly consistent with skincare common sense rules, the preliminary analysis result can be directly used as the target analysis result. If the skincare deviation comparison result indicates a logical conflict, the preliminary analysis result can be corrected according to the severity of the conflict. For example, the initial prediction probability of the conflicting skin condition label can be lowered, or a more reasonable candidate category can be recommended based on skincare common sense rules. The correction process can be completed based on a preset rule engine or a lightweight calibration network. Finally, the corrected skin condition label and the corresponding calibrated probability value are output as the target analysis result.

[0082] As can be seen, this embodiment of the application extracts features from hand images to obtain hand visual features, and combines them with a hand image classification model to obtain preliminary analysis results. Then, the hand visual features and preliminary analysis results are input into a skin care semantic language model to obtain skin care semantic description text, which is then compared with skin care common sense rules in the skin care knowledge base for consistency. Based on the skin care deviation comparison results, corrections are made. This approach can use visual features, semantic description text, and skin care common sense rules to jointly verify the state of hand skin, thereby making the hand skin state analysis results more accurate and providing effective data support for subsequent customized skin care product formulations.

[0083] To reduce the impact of background areas, noise, and lighting differences on feature extraction, some embodiments of this application refine the logic for extracting hand visual features. Specifically, feature extraction is performed on a hand image to obtain hand visual features, including:

[0084] S201. Normalize the size of the hand image and convert it to the preset color gamut to obtain the hand image in the target color gamut.

[0085] In this embodiment, size normalization refers to the process of adjusting hand images from different sources and with different resolutions to a uniform pixel size. Preset color gamut conversion refers to converting the hand image from its original color space to a color space more suitable for skin region analysis. The target color gamut hand image refers to the intermediate image after size normalization and preset color gamut conversion, whose pixel distribution is more conducive to subsequent region segmentation and skin color recognition.

[0086] In some embodiments of this application, in step S201, the original width and height of the hand image can be read first, and the hand image can be scaled to a fixed size, such as 224 pixels by 224 pixels or 256 pixels by 256 pixels, using a bilinear interpolation algorithm. Subsequently, the scaled hand image can be converted from the Red Green Blue (RGB) color space to the Lab color space or the YCbCr color space to enhance the color difference between the skin area and the background area, thereby obtaining a hand image in the target color gamut.

[0087] S202. Perform semantic segmentation on the target color gamut hand image to obtain the region of interest hand image.

[0088] In this embodiment, semantic segmentation refers to the process of classifying each pixel in an image into categories to distinguish between the hand skin region and non-hand regions. The Region of Interest (ROI) hand image refers to the main hand region extracted from the target color gamut hand image, which primarily retains image content related to hand skin condition analysis.

[0089] In some embodiments of this application, in step S202, the target color gamut hand image can be input into a pre-trained semantic segmentation neural network, such as U-Net. The semantic segmentation neural network can output a segmentation mask of the same size as the input image, wherein the pixel values ​​of the hand region can be set to 1, and the pixel values ​​of the background region can be set to 0. Subsequently, the segmentation mask can be multiplied pixel-by-pixel with the target color gamut hand image, retaining only the pixels of the hand region, thereby obtaining the region of interest hand image.

[0090] S203. Filter and denoise the hand image in the region of interest and perform illumination equalization to obtain an enhanced hand image.

[0091] In this embodiment, denoising refers to using digital filtering to suppress random noise, compressed noise, or localized particle interference in an image. Illumination equalization refers to adjusting the brightness distribution of the image to maintain a more consistent brightness across the hand skin area under different shooting conditions. Enhancing the hand image refers to the intermediate image after denoising and illumination equalization processing.

[0092] In some embodiments of this application, in step S203, Gaussian filtering or median filtering can be applied to the hand image of the region of interest to reduce local noise interference. After filtering and denoising, a contrast-limited adaptive histogram equalization algorithm can be further used to enhance the local brightness and adjust the contrast of the hand image of the region of interest, so as to improve the situation of excessively dark shadow areas or excessively strong local highlights, thereby obtaining an enhanced hand image.

[0093] S204. Perform skin color detection on the enhanced hand image to extract a pure hand mask for the pure skin region.

[0094] In this embodiment, skin color detection refers to the process of identifying skin regions from an enhanced hand image based on the distribution pattern of skin pixels in a specific color space. A clean hand mask is a binary image used to mark the locations of pure skin regions, where skin regions and non-skin regions are clearly distinguished. The clean hand mask is used to preserve the main skin area of ​​the hand and exclude nail areas, jewelry areas, or residual background areas.

[0095] In some embodiments of this application, in step S204, the enhanced hand image can be converted to the YCbCr color space, and each pixel can be determined to belong to the skin region based on the value range of each chromaticity component. For example, a skin color ellipse model or skin color threshold range obtained based on sample statistics can be preset, and pixels falling within the range can be selected to obtain a skin color mask. Subsequently, morphological opening or closing operations can be performed on the skin color mask to remove scattered noise and repair local void areas, resulting in a clean hand mask.

[0096] S205. Generate a preprocessed hand image based on the clean hand mask and the enhanced hand image.

[0097] In this embodiment, the preprocessed hand image refers to an image obtained by filtering from an enhanced hand image using a clean hand mask, primarily retaining the pure skin area of ​​the hand. The preprocessed hand image has less interference from background and non-skin areas.

[0098] In some embodiments of this application, in step S205, a pixel-by-pixel logical AND operation or a pixel-by-pixel multiplication operation can be performed between the clean hand mask and the enhanced hand image, retaining only the pixel values ​​corresponding to the positions marked as skin areas in the clean hand mask, and setting the remaining positions to preset background values, such as black or zero, thereby obtaining a pre-processed hand image.

[0099] S206. Extract features from the preprocessed hand image to obtain the visual features of the hand.

[0100] In the embodiments of this application, hand visual features refer to the multidimensional feature results extracted from the preprocessed hand image to characterize the state of the hand skin.

[0101] In some embodiments of this application, in step S206, features such as color histogram features, local binary pattern texture features, and edge morphology features can be extracted from the preprocessed hand image and then stitched together to obtain the visual features of the hand.

[0102] As can be seen, the embodiments of this application perform size normalization, preset color gamut conversion, semantic segmentation, filtering and denoising, illumination equalization and skin color detection on the hand image in sequence, and generate a pre-processed hand image by combining a clean hand mask. This can highlight the hand skin area, reduce the influence of background area, noise information and illumination differences on feature extraction, and improve the correspondence between the pre-processed hand image and the real hand skin area.

[0103] To more comprehensively characterize the state of hand skin, some embodiments of this application detail the specific process of extracting hand visual features from preprocessed hand images. Specifically, feature extraction is performed on the preprocessed hand image to obtain hand visual features, including:

[0104] S301. Extract shallow hand features from the preprocessed hand image, wherein the shallow hand features include at least one of hand skin color information, hand skin texture information, and hand skin morphology information.

[0105] In this embodiment, shallow hand features refer to low-level visual features directly calculated based on the pixel distribution and local structure of the pre-processed hand image. Hand skin color information is used to characterize the hue distribution, brightness distribution, or overall color variation of the skin region. Hand skin texture information is used to describe the roughness, fine line distribution, and local regularity of the skin surface. Hand skin morphology information is used to record the boundary shape, local contour, or geometric relationships of the hand region.

[0106] In some embodiments of this application, in step S301, multiple types of shallow features can be extracted in parallel from the preprocessed hand image. For example, a color histogram of the skin region can be statistically analyzed in the HSV (Hue, Saturation, Value) color space or the Lab color space to serve as the hand skin color information; the hand skin texture information can be extracted using local binary mode algorithms, gray-level co-occurrence matrix algorithms, or Gabor filtering algorithms; and the hand skin morphology information can be extracted using edge detection algorithms, contour extraction algorithms, or convex hull statistics.

[0107] Furthermore, after extracting the aforementioned superficial hand features such as skin color, texture, and morphology, feature filtering can be performed using analysis of variance or mutual information calculation. By evaluating the correlation between each feature dimension and the hand skin condition, invalid or redundant features with weak correlations are eliminated, retaining only features strongly correlated with skin problems. This reduces the computational burden of subsequent feature fusion and improves feature representation capabilities.

[0108] S302. Input the preprocessed hand image into the preset depth visual coding model to obtain deep hand features.

[0109] In this embodiment, the deep visual coding model refers to a pre-trained image feature coding network, such as a Residual Network (ResNet) or a Vision Transformer (ViT) pre-trained model, used to extract high-level semantic representations from pre-processed hand images. Deep hand features refer to the high-dimensional feature vectors output by the deep visual coding model, which not only contain local appearance information but also the combination relationships and semantic associations between different regions.

[0110] In some embodiments of this application, in step S302, the preprocessed hand image can be adjusted to a fixed input size required by the deep visual coding model, such as 224 pixels by 224 pixels. Subsequently, the preprocessed hand image can be input into the deep visual coding model, and the convolutional feature map before the global average pooling layer or the feature vector before the fully connected layer can be selected as the deep hand features.

[0111] S303. Align the shallow hand features and deep hand features dimensionally to obtain the initial two-dimensional features.

[0112] In this embodiment, dimension alignment refers to the processing operation of mapping feature results from different sources and with different dimensions to the same or compatible dimension space. The initial two-dimensional feature refers to the merged feature result obtained after dimension alignment, which retains both shallow hand features and deep hand features.

[0113] In some embodiments of this application, in step S303, independent mapping layers can be set for projection processing based on the dimensional differences between shallow hand features and deep hand features. For example, when the shallow hand features are 128-dimensional and the deep hand features are 2048-dimensional, they can be mapped to the same preset dimension, such as 512-dimensional. Subsequently, the two 512-dimensional vectors can be concatenated along the channel dimension to obtain an initial two-dimensional feature of 1024 dimensions.

[0114] S304. Based on the preset attention mechanism, obtain the feature weights of each feature in the initial two-dimensional features, and perform weighted fusion of multiple features in the initial two-dimensional features according to the feature weights to obtain the fused hand features.

[0115] In this embodiment, the attention mechanism refers to a processing method that evaluates the importance of different parts of the input features and assigns weights. Feature weights refer to the importance scores corresponding to each feature item in the initial two-dimensional features. Fusing hand features refers to the comprehensive feature result obtained after weighting the initial two-dimensional features according to the feature weights, which simultaneously retains shallow appearance information and deep semantic information.

[0116] In some embodiments of this application, in step S304, the initial two-dimensional features can be input into the attention weight calculation network. The attention weight calculation network can consist of one or more fully connected layers and non-linear activation layers, and output a weight vector with the same dimension as the initial two-dimensional features. Subsequently, the initial two-dimensional features can be multiplied element-wise with the weight vector, so that the feature terms with higher weights are strengthened and the feature terms with lower weights are suppressed, thereby obtaining the fused hand features.

[0117] In some embodiments of this application, the attention weight calculation network may adopt a Squeeze-and-Excitation Network (SENet) structure or a Convolutional Block Attention Module (CBAM) structure. Taking the SENet structure as an example, the attention weight calculation network includes a globally average pooling layer, a first fully connected dimensionality reduction layer (with ReLU activation function), a second fully connected dimensionality increase layer, and a sigmoid activation layer connected in sequence, used to output normalized attention weights that correspond one-to-one with the feature channels.

[0118] S305. Based on the fusion of hand features, feature dimensionality reduction processing is performed to obtain hand visual features.

[0119] In this embodiment, feature dimensionality reduction refers to the process of transforming high-dimensional fused hand features into low-dimensional feature representations through mathematical projection or feature compression. Hand visual features refer to the feature vectors obtained after dimensionality reduction.

[0120] In some embodiments of this application, in step S305, Principal Components Analysis (PCA) or Linear Discriminant Analysis (LDA) algorithms can be used to select feature dimensions that retain the main discriminative information based on the distribution of fused hand features in the training samples, and compress the high-dimensional fused hand features into low-dimensional feature vectors to obtain hand visual features that are more compact in dimensions and have stronger discriminative ability.

[0121] As can be seen, the embodiments of this application extract shallow hand features from preprocessed hand images, combine them with a deep visual coding model to obtain deep hand features, and then perform dimension alignment, feature weight acquisition, weighted fusion and feature dimensionality reduction processing. This can simultaneously represent hand skin color information, hand skin texture information and deep semantic information, reducing the information loss problem caused by single feature extraction.

[0122] To reduce the impact of manual labeling errors on model training results, some embodiments of this application refine the training steps of the hand image classification model. Specifically, the hand image classification model is trained through the following steps:

[0123] S401. Obtain the fused hand sample features and corresponding manually labeled tags of each sample image in the preset set of hand skin sample images.

[0124] In this embodiment, the hand skin sample image set refers to the dataset used for model training, which includes multiple collected and stored hand images, i.e., sample images. Fusion of hand sample features refers to fusing hand features for each sample image in the hand skin sample image set according to the aforementioned steps S301 to S304. Manually labeled tags refer to category or level identifiers assigned by relevant technicians based on the state of the hand skin in the sample images.

[0125] In some embodiments of this application, in step S401, a set of hand skin sample images stored in a database or training sample library can be loaded first. Then, preprocessing, shallow hand feature extraction, deep hand feature extraction, dimension alignment, and weighted fusion processing can be sequentially performed on each sample image in the set to obtain a fixed-dimensional fused hand sample feature. For example, each sample image can ultimately correspond to a 1024-dimensional or 2048-dimensional fused hand sample feature vector. Simultaneously, the manually labeled tags of the corresponding sample images can be read from the annotation information table, and the fused hand sample features and manually labeled tags can be organized into one-to-one data pairs.

[0126] S402. Based on the feature weights of each sample feature in the fused hand sample features, determine the correlation between the fused hand sample features and the manually labeled tags.

[0127] In this embodiment, feature weight refers to the importance score of each sample feature obtained through attention processing during the formation of fused hand sample features. Correlation refers to the quantitative result used to measure the degree of matching between the fused hand sample features and the manually labeled tags. A higher correlation indicates a more consistent performance between the manually labeled tags and the sample image features; a lower correlation indicates a potential discrepancy between the two.

[0128] In some embodiments of this application, in step S402, the fused hand sample features can first be input into a preset lightweight classifier to obtain predicted label results based on the fused hand sample features. Then, the degree of difference between the predicted label results and the manually labeled labels can be calculated, and the concentration of sample features in the current fused hand sample features can be calculated in combination with the feature weight distribution. Finally, the label difference and the concentration of sample features can be weighted and combined to obtain the correlation between the fused hand sample features and the manually labeled labels.

[0129] In some embodiments of this application, the preset lightweight classifier may include a Multilayer Perceptron (MLP), a Support Vector Machine (SVM), or a Random Forest classifier. If a Multilayer Perceptron is used, its network structure may include an input layer, at least one hidden layer (with a Rectified Linear Unit (ReLU) activation function and a Dropout mechanism), and an output layer based on a Softmax activation function to quickly perform preliminary label determination calculations.

[0130] S403. The sample images corresponding to the fused hand sample features with a correlation degree less than the preset correlation degree threshold are labeled as abnormal samples.

[0131] In this embodiment, the preset correlation threshold refers to a preset judgment threshold used to distinguish between labeled normal samples and labeled abnormal samples. Labeled abnormal samples refer to sample images whose correlation between the fused hand sample features and the manually labeled images is lower than the preset correlation threshold. Labeled abnormal samples typically indicate a significant deviation between the manually labeled images and the image feature representation.

[0132] S404. Input the labeled abnormal samples into the preset label prediction model to obtain the corrected label of the labeled abnormal samples, and replace the corresponding manual label with the corrected label to obtain the replaced hand skin sample image set.

[0133] In this embodiment, the label prediction model refers to a prediction model used to re-predict skin state labels for anomaly-labeled samples. The corrected label refers to the alternative label result output by the label prediction model for the anomaly-labeled samples. The replaced set of hand skin sample images refers to the set of hand skin sample images obtained after replacing the original manually labeled labels corresponding to the anomaly-labeled samples with the corrected label.

[0134] In some embodiments of this application, in step S404, a pre-trained label prediction model can be pre-set, and labeled anomalous samples can be input into it. The input can be the original sample image corresponding to the labeled anomalous sample, or the fused hand sample features corresponding to the labeled anomalous sample. The label prediction model can output the predicted probability distribution of multiple candidate categories, and determine the category with the highest predicted probability as the corrected label. After the label replacement of all labeled anomalous samples is completed, the set of replaced hand skin sample images is obtained.

[0135] In some embodiments of this application, the label prediction model that has completed preliminary training can use a network backbone structure that is the same as or similar to the aforementioned hand image classification model, such as ResNet or a Visual Geometry Group (VGG) backbone network supplemented by a fully connected classification head. This label prediction model is a reference teacher model that has been pre-trained and converged from a batch of clean feature subsets that do not contain labeled anomalous samples and have been manually verified twice. It can be used to provide more reliable pseudo-label prediction values ​​for anomalous feature samples.

[0136] S405. Train the model based on the replaced set of hand skin sample images to obtain a hand image classification model.

[0137] In this embodiment of the application, since a set of replaced hand skin sample images is used during the model training process, the resulting hand image classification model has higher consistency with the real sample state.

[0138] As can be seen, the embodiments of this application determine the correlation between the hand sample features and the manually labeled samples by using feature weights based on the fused hand sample features, and then filter out the labeled abnormal samples. The label prediction model is then used to replace and correct the manually labeled samples, which can reduce the impact of abnormal labeled samples on the quality of training data and improve the consistency between the replaced hand skin sample image set and the real skin state.

[0139] To balance learning ability and generalization performance during model training, some embodiments of this application detail the specific model training process for the hand image classification model. Specifically, the model is trained based on a set of replaced hand skin sample images to obtain the hand image classification model, including:

[0140] S501. Divide the set of replaced hand skin sample images into a training sample set and a validation sample set.

[0141] In this embodiment, the training sample set refers to a subset of samples used to directly update model parameters. The validation sample set refers to a subset of samples that is retained separately during training, does not participate in parameter updates, and is only used to evaluate the current performance of the model.

[0142] In some embodiments of this application, in step S501, after obtaining the replaced hand skin sample image set, a stratified random sampling method can be used for partitioning. For example, the replaced hand skin sample image set can be divided into a training sample set and a validation sample set according to a ratio of 80% and 20%, respectively. During the partitioning process, the sample proportion of each hand skin state label in the training sample set and the validation sample set can be made approximately consistent with the category proportion in the original set.

[0143] S502. Input the fused hand sample features of each sample image in the training sample set into the preset initial network model to obtain the training classification result.

[0144] In the embodiments of this application, the initial network model refers to the classification network before the start of the current training round, and its parameters can be randomly initialized parameters or initial parameters obtained based on the pre-training task.

[0145] The training classification result refers to the prediction result output by the initial network model based on the fused hand sample features corresponding to each sample image in the training sample set. The training classification result may include, for example, the predicted probability distribution corresponding to each skin state label, or the skin state label with the highest probability.

[0146] In some embodiments of this application, in step S502, fused hand sample features corresponding to multiple sample images can be read in batches from the training sample set, and these fused hand sample features are input into the initial network model. For example, the initial network model can be a multilayer perceptron, whose input layer neuron number matches the dimension of the fused hand sample features. After the fused hand sample features pass through the linear transformation layer, nonlinear activation layer, and output layer in the initial network model, an output vector corresponding to the number of skin state labels can be obtained. Further, the softmax function can be used to convert the output vector into a predicted probability distribution corresponding to each skin state label, and this predicted probability distribution is the training classification result.

[0147] In some embodiments of this application, the initial network model can be a deep feedforward neural network, whose network topology includes: an input layer that receives fused hand sample features, multiple hidden feature mapping layers connected in series (e.g., including fully connected layers, batch normalization (BN) layers, and leaky-ReLU activation layers), and a final classification output layer. The number of neurons in this output layer is the same as the preset total number of hand skin state categories, and it outputs the predicted probability distribution corresponding to each skin state label via a Softmax function.

[0148] S503. Based on the training classification results, the corrected labels corresponding to each sample image, and the preset parameter regularization term, calculate the training loss value of the initial network model.

[0149] In this embodiment, the training loss value refers to a scalar result that comprehensively reflects the difference between the training classification result and the corrected label, as well as the model complexity constraints. The parameter regularization term refers to the parameter constraint term introduced into the loss function, which is used to limit the size of the model parameters and reduce the possibility of the model learning overly complex decision boundaries.

[0150] In some embodiments of this application, in step S503, the main loss term can be calculated first based on the training classification result and the corrected label. For example, if the corrected label adopts one-hot encoding, the difference between the training classification result and the corrected label can be calculated using the binary cross entropy (BCE) loss function. Subsequently, a L2 regularization term can be constructed for all weight parameters in the initial network model, and the main loss term can be added to the parameter regularization term to obtain the training loss value.

[0151] Furthermore, data augmentation methods can be introduced when acquiring training data and calculating parameter regularization terms. For example, random rotation, flipping, cropping, and brightness or contrast adjustments can be performed on hand skin sample images to expand the diversity of training samples. Correspondingly, in addition to using L2 regularization, the preset parameter regularization terms can also combine L1 regularization and Dropout mechanisms to further limit the model parameter size and prevent the network model from overfitting.

[0152] S504. Adjust the model parameters of the initial network model according to the training loss value to obtain the iterative network model.

[0153] In this embodiment, the iterative network model refers to the model state obtained after updating the parameters once based on the current training loss value. By adjusting the model parameters of the initial network model, the model output can gradually approach the corrected label.

[0154] In some embodiments of this application, in step S504, after calculating the training loss value, the backpropagation (BP) algorithm can be used to calculate the gradient value of the training loss value with respect to each model parameter layer by layer. Subsequently, a stochastic gradient descent (SGD) optimizer, a momentum gradient descent optimizer, or an Adam optimizer can be used, combined with a preset learning rate, to update each model parameter. After completing one parameter update, the iterative network model for the current round is obtained.

[0155] S505. Obtain the current verification accuracy value of the verification sample set on the iterative network model, and determine the iterative network model as a hand image classification model when the number of consecutive times the current verification accuracy value does not increase is equal to the preset stopping round threshold.

[0156] In this embodiment, the current validation accuracy value refers to the classification accuracy obtained by the iterative network model on the validation sample set after the current training epoch. The number of consecutive epochs without improvement refers to the number of epochs in which the current validation accuracy value has not improved compared to the historical best validation accuracy value. The preset stopping epoch threshold is a threshold used to control the number of epochs at which training is stopped prematurely.

[0157] In some embodiments of this application, in step S505, after each round of complete training, the fused hand sample features from the validation sample set can be input into the current iterative network model, and the number of correctly predicted samples can be counted to calculate the current validation accuracy value. If the current validation accuracy value is higher than the historical best validation accuracy value, the historical best validation accuracy value can be updated, and the model parameters corresponding to the current iterative network model can be saved, while the number of consecutive periods without increment is cleared to zero. If the current validation accuracy value is not higher than the historical best validation accuracy value, the number of consecutive periods without increment can be incremented by one. When the number of consecutive periods without increment reaches a preset stopping round threshold, such as 10 rounds, the training process can be stopped, and the iterative network model with the model parameters corresponding to the historical best validation accuracy value can be determined as the hand image classification model.

[0158] As can be seen, this embodiment divides the set of replaced hand skin sample images into a training sample set and a validation sample set, adjusts the model parameters of the initial network model based on the training loss value, and then determines the hand image classification model by combining the current validation accuracy value and the preset stopping round threshold. This can take into account both the model parameter optimization process and the validation results, and reduce the classification bias caused by overtraining.

[0159] To improve the reliability of probability values ​​in the target analysis results, make the probability output closer to the true classification confidence level, and reduce the probability bias caused by a single probability calibration algorithm, some embodiments of this application further refine the subsequent processing procedures after obtaining the target analysis results of the hand skin state in the hand image. Specifically, after obtaining the target analysis results of the hand skin state in the hand image, the process further includes:

[0160] S601. Extract the initial predicted probability of hand skin condition from the target analysis results.

[0161] In this embodiment, the initial predicted probability refers to the original probability value assigned to each hand skin state label by the hand image classification model when outputting the target analysis result. The initial predicted probability is usually derived from the probability result after the output layer of the classification network is processed by the Softmax function. The initial predicted probability can reflect the degree of bias of the classification network towards each hand skin state label, but without calibration, its value may not accurately correspond to the true classification confidence level.

[0162] In some embodiments of this application, in step S601, the probability field corresponding to the hand skin condition label can be parsed from the data structure of the target analysis result. For example, if the target analysis result is "label is dry, probability is 0.85", then 0.85 can be used as the initial prediction probability; if the target analysis result contains complete probability vectors corresponding to multiple categories, then the entire probability vector can be extracted as the initial prediction probability.

[0163] S602. Select at least two target probability calibration algorithms from a plurality of preset candidate probability calibration algorithms, wherein the candidate probability calibration algorithms include at least one of linear scaling algorithm, Pratt scaling algorithm and Bayesian calibration algorithm.

[0164] In this embodiment, multiple candidate probability calibration algorithms refer to a set of algorithms used to correct the original probability output of the classification model. Specifically, the Linear Scaling algorithm is used to scale and translate the initial predicted probabilities overall using a linear mapping relationship. The Platt Scaling algorithm is used to fit the relationship between the probability output and the true frequency using a parameterized S-shaped mapping relationship. The Bayesian calibration algorithm is used to update the original probabilities posteriorly by combining prior distributions and observation results. The target probability calibration algorithm refers to at least two algorithms selected from the candidate probability calibration algorithms that are actually used in the current probability calibration process.

[0165] In some embodiments of this application, in step S602, a candidate probability calibration algorithm library can be pre-established, storing one or more of the linear scaling algorithm, Pratt scaling algorithm, and Bayesian calibration algorithm. Subsequently, at least two target probability calibration algorithms can be determined using a fixed selection method or a dynamic selection method based on validation data. For example, the linear scaling algorithm and Pratt scaling algorithm can be fixedly selected, or the expected calibration error or probability fitting effect of each candidate probability calibration algorithm can be evaluated on an independent validation set first, and then the two algorithms with better performance can be selected as the target probability calibration algorithms.

[0166] S603. Perform process concatenation on at least two target probability calibration algorithms to obtain a combined probability calibration algorithm.

[0167] In this embodiment, the process of concatenating multiple independent target probability calibration algorithms into a continuous processing chain in a preset order. The combined probability calibration algorithm refers to a composite calibration algorithm formed through the process concatenation, whose output combines the processing effects of multiple target probability calibration algorithms.

[0168] In some embodiments of this application, in step S603, if the linear scaling algorithm and the Pratt scaling algorithm have been selected as the target probability calibration algorithms, the output of the linear scaling algorithm can be connected to the input of the Pratt scaling algorithm, so that the initial predicted probability first undergoes linear scaling processing and then enters Pratt scaling processing. Alternatively, the Bayesian calibration algorithm can be connected after the linear scaling algorithm, so that the initial predicted probability first undergoes global scale correction and then completes a priori calibration by combining prior information. Through this process, multiple target probability calibration algorithms can be organized into a single, continuously executable combined probability calibration algorithm.

[0169] S604. Using a combined probability calibration algorithm, the initial prediction probability is calibrated to obtain the calibrated prediction probability.

[0170] In this embodiment, probability calibration processing refers to the process of inputting the initial predicted probability into a combined probability calibration algorithm and outputting a new probability value after multiple consecutive calibration steps. The calibrated predicted probability refers to the probability result obtained after correction by the combined probability calibration algorithm, which, relative to the initial predicted probability, is closer to the probability level corresponding to the classification accuracy in real samples.

[0171] In some embodiments of this application, in step S604, the initial predicted probability can be input into the combined probability calibration algorithm. Taking the aforementioned cascaded processing as an example, the initial predicted probability can first enter the linear scaling algorithm, and perform a global adjustment based on the preset scaling factor and bias term. Then, the linearly scaled output result is input into the Pratt scaling algorithm, and nonlinear correction is performed through the S-shaped mapping relationship to obtain the calibrated predicted probability.

[0172] S605. Replace the initial predicted probability in the target analysis results with the calibrated predicted probability to obtain the optimized skin condition analysis results.

[0173] In this embodiment, the optimized skin condition analysis result refers to the result that maintains the hand skin condition classification conclusion in the target analysis result unchanged, but only updates the initial predicted probability to the calibrated predicted probability. The optimized skin condition analysis result provides a more reliable probability expression while retaining the original classification label.

[0174] As can be seen, the embodiments of this application extract the initial prediction probability from the target analysis results, select at least two target probability calibration algorithms for process splicing, and then perform probability calibration on the initial prediction probability. This can integrate the processing results of different candidate probability calibration algorithms, reduce the probability bias caused by a single probability calibration algorithm, and improve the matching degree between the calibrated prediction probability and the classification result.

[0175] To further transform the optimized skin condition analysis results into an executable hand care ingredient formulation, some embodiments of this application provide logic for generating the hand care ingredient formulation. Specifically, after obtaining the optimized skin condition analysis results, the process also includes:

[0176] S701. Extract the calibration prediction probability from the optimized skin condition analysis results.

[0177] S702. Based on the calibration prediction probability, determine the state quantification index corresponding to each hand skin state.

[0178] In this embodiment, the state quantification index refers to the result of converting the calibration prediction probability into a specific state degree value, used to reflect the severity, intensity level, or demand level of a certain hand skin condition. Compared with directly using probability values, the state quantification index is more convenient for subsequent calculation of ingredient requirements and matching of formulation rules.

[0179] In some embodiments of this application, in step S702, a mapping rule table between calibration prediction probability and state level can be pre-established. For example, for a dry state, the state quantization index can be set to level 1 when the calibration prediction probability is in the range of 0 to 0.3; level 3 when it is in the range of 0.3 to 0.6; level 6 when it is in the range of 0.6 to 0.8; and level 9 when it is in the range of 0.8 to 1.0. If the calibration prediction probability of a certain dry state is 0.78, then the corresponding state quantization index can be determined to be level 6 according to the mapping rule table.

[0180] S703. Based on the quantitative indicators of each state, determine the ingredient requirement indicators corresponding to the state of hand skin.

[0181] In this embodiment of the application, the ingredient demand index refers to a numerical result used to quantify the intensity of the demand for different functional ingredients in the current state of the hand skin. The ingredient demand index can indicate the degree of demand for moisturizing ingredients, soothing ingredients, barrier repair ingredients, oil ingredients, or anti-wrinkle ingredients.

[0182] In some embodiments of this application, a demand transformation matrix can be pre-established in step S703. The demand transformation matrix defines the influence weights of different hand skin condition levels on the demand for various functional ingredients. For example, when the dryness level is high, the demand weights for moisturizing and oil-based ingredients are higher; when the sensitivity level is high, the demand weights for soothing and barrier-repairing ingredients are higher; and when the fine line level is high, the demand weights for anti-wrinkle and repairing ingredients are higher. Subsequently, the quantitative indicators of each condition can be combined into a condition input vector, and multiplied with the demand transformation matrix to obtain demand intensity scores corresponding to multiple ingredient categories, which serve as ingredient demand indicators.

[0183] S704. Using ingredient requirement indicators, determine the corresponding candidate hand care ingredients and the candidate concentration range of the candidate hand care ingredients in the preset skin care ingredient database.

[0184] In this embodiment, the skincare ingredient database refers to a structured database that stores various skincare ingredients and their efficacy categories, physicochemical properties, safe concentration ranges, and applicable conditions. Candidate handcare ingredients refer to a group of specific handcare ingredients suitable for use in the current formulation, selected based on ingredient requirement indicators. Candidate concentration ranges refer to the initially determined recommended concentration ranges for each candidate handcare ingredient.

[0185] In some embodiments of this application, in step S704, the ingredient demand index can be matched with a skincare ingredient database. For functional ingredient categories with a demand intensity higher than a preset threshold, the ingredient list under the corresponding functional category can be retrieved from the skincare ingredient database. For example, when the demand index for moisturizing ingredients is greater than 0.7, ingredients such as glycerin, sodium hyaluronate, and panthenol can be retrieved; when the demand index for barrier repair ingredients is high, ceramides, cholesterol, and fatty acids can be retrieved. Subsequently, the retrieved ingredients can be sorted or filtered according to the score of the ingredient demand index, and several top-ranked ingredients can be selected as candidate hand care ingredients. At the same time, the safe concentration range of each candidate hand care ingredient in the skincare ingredient database can be read, and the range can be fine-tuned in combination with the current demand intensity to obtain the candidate concentration range corresponding to each candidate hand care ingredient, such as glycerin 3% to 5% and panthenol 1% to 2%.

[0186] S705. The calibration prediction probability, candidate hand care ingredients and candidate concentration range are processed by text splicing to obtain the formula derivation condition text.

[0187] In this embodiment of the application, the formula derivation condition text refers to a structured natural language description formed by integrating the calibration prediction probability, candidate hand care ingredients, and candidate concentration range.

[0188] In some embodiments of this application, in step S705, a text template can be pre-established, and the calibration prediction probability, candidate hand care ingredient names, and candidate concentration ranges can be sequentially filled into the text template. For example, the following formulation derivation condition text can be generated: The current hand skin condition analysis probability is: dry 0.78, sensitive 0.15; suggested candidate hand care ingredients include glycerin, panthenol, and ceramide, with corresponding candidate concentration ranges of 3% to 5%, 1% to 2%, and 0.2% to 1.0%, respectively.

[0189] S706. Using the preset formula parameters to generate a model, perform formula combination and ratio calculations on the formula derivation condition text to obtain the target hand care ingredient formula scheme.

[0190] In this embodiment, the formula parameter generation model refers to a model used to understand the formula derivation condition text and output specific ingredient combinations and proportions. The target hand care ingredient formula scheme refers to the complete formula result output by the formula parameter generation model, which includes specific ingredient names and corresponding addition proportions.

[0191] In some embodiments of this application, in step S706, the formula derivation condition text can be input into a preset formula parameter generation model. The formula parameter generation model can first perform semantic parsing on the state probabilities, candidate hand care ingredients, and candidate concentration ranges in the text to determine which ingredients should be prioritized and the boundary conditions that each ingredient concentration should meet. Subsequently, the formula parameter generation model can combine and optimize the candidate hand care ingredients with pre-stored mature formula samples and ingredient compatibility rules, and calculate the specific addition ratio of each ingredient. For example, the formula parameter generation model can output a target hand care ingredient formula scheme such as "70% deionized water, 5% glycerin, 2% panthenol, 0.5% ceramide, and 8% shea butter".

[0192] As can be seen, the embodiments of this application extract calibration prediction probabilities from the optimized skin condition analysis results, determine the condition quantification index and ingredient requirement index accordingly, and then combine them with the skin care ingredient database to determine candidate hand care ingredients and candidate concentration ranges, and generate target hand care ingredient formulation schemes. This can transform the skin condition analysis results into a basis for ingredient selection and concentration determination, and improve the correspondence between the formulation generation results and the skin condition.

[0193] To ensure that the generated target hand care ingredient formulation meets the corresponding skin physiological needs, some embodiments of this application refine the processing procedure of the formulation parameter generation model. Specifically, using a preset formulation parameter generation model, the formulation derivation condition text is used to perform formulation combination and proportion calculations to obtain the target hand care ingredient formulation, including:

[0194] S801. Input the formula derivation condition text into the formula parameter generation model for semantic parsing processing to obtain the skin physiological attribute feature vector.

[0195] In this embodiment, semantic parsing refers to the process of analyzing and structurally representing the skin state description, ingredient information, and concentration constraints in the formula derivation condition text. The skin physiological attribute feature vector refers to the numerical vector result obtained after semantic parsing, and its different dimensions can correspond to attributes such as stratum corneum hydration requirements, barrier repair requirements, soothing requirements, or oil replenishment requirements.

[0196] In some embodiments of this application, in step S801, after the formula parameter generation model receives the input formula derivation condition text, it can first convert the words or phrases in the text into dense vector representations through a word embedding layer. Subsequently, a Transformer encoder, recurrent neural network, or other text encoding network can be used to perform context modeling on the entire text sequence, and the encoded text semantic representation can be mapped to a fixed-dimensional vector space through a fully connected projection layer to obtain the skin physiological attribute feature vector.

[0197] In some embodiments of this application, the formula parameter generation model further includes a joint output network based on a multi-task learning mechanism at the output end. The joint output network may include a classification decoder head and a regression decoder head. The classification decoder head is used to output the discrete selection states of the target component set. The regression decoder head may, for example, include a multi-layer fully connected network with a sigmoid activation function, which can be combined with the upper and lower limits of the safe concentration of the components for inverse normalization scaling, and is used to output continuous component concentration ratio values, thereby realizing the generation of formula parameters from textual semantics.

[0198] S802. Extract the causal derivation rules of skin care mechanism corresponding to the skin physiological attribute feature vector from the preset skin care mechanism relationship library.

[0199] In this embodiment, the skincare mechanism relation library refers to a knowledge base that pre-stores the relationships between skin conditions, physiological attributes, care directions, and ingredient interactions. The skincare mechanism causal derivation rules refer to a set of rules extracted from the skincare mechanism relation library that correspond to the current skin physiological attribute feature vector. These rules describe why a certain type of skin condition requires a certain type of care direction, and why a certain type of care direction should preferentially match certain ingredients.

[0200] In some embodiments of this application, in step S802, the high-weight dimensions in the skin physiological attribute feature vector can be matched with the rule premises in the skin care mechanism relation library. The matching process can be implemented through vector similarity calculation, rule tag retrieval, or threshold triggering. For example, when the stratum corneum water content requirement dimension and the barrier repair requirement dimension are high in the skin physiological attribute feature vector, causal derivation rules such as "insufficient stratum corneum water content requires the supplementation of moisturizing ingredients" and "when the barrier repair requirement is high, ceramides, cholesterol, and fatty acid ingredients are given priority" can be extracted from the skin care mechanism relation library.

[0201] S803. Using the causal deduction rules of skin care mechanism, perform logical reasoning operations on the feature vector of skin physiological attributes to obtain the parameters of care strategy.

[0202] In this embodiment, logical reasoning operation refers to the process of matching rules, triggering conditions, and summarizing conclusions on the feature vector of skin physiological attributes based on the causal deduction rules of skin care mechanisms. Nursing strategy parameters refer to the parameter results obtained after logical reasoning operation, used to guide the direction of formula construction, and can reflect different strategy directions of current hand care formulas, such as moisture absorption and moisturizing, occlusive moisturizing, barrier repair, immediate soothing, or long-term repair.

[0203] In some embodiments of this application, in step S803, a rule engine can be constructed, using the skin physiological attribute feature vector as the input fact and the causal derivation rules of skincare mechanisms as the rule knowledge of the rule engine. The rule engine can use a forward chain reasoning approach to determine which rules are triggered by the current skin physiological attribute feature vector. Subsequently, the conclusions of multiple triggered rules can be summarized, conflict resolved, and weights normalized to ultimately form care strategy parameters. For example, care strategy parameters may include moisturizing priority parameters, barrier repair priority parameters, and soothing control parameters, etc.

[0204] S804. According to the nursing strategy parameters, perform component mechanism conflict test on the candidate hand care components in the formula derivation condition text, and extract the component combinations that do not have mechanism conflicts as the target component set.

[0205] In this embodiment, the component mechanism conflict test refers to the process of judging whether there is efficacy cancellation, stability disruption, increased irritation, or conflict in nursing direction among candidate hand care components based on nursing strategy parameters and component attribute information. The target component set refers to the component combination that can jointly satisfy the current nursing strategy parameters after passing the component mechanism conflict test.

[0206] In some embodiments of this application, in step S804, a list of candidate hand care ingredients can be extracted from the formula derivation condition text, and the ingredient attributes, incompatibilities, and environmental compatibility conditions recorded in the skin care ingredient database or skin care mechanism relation database can be read. Subsequently, the candidate hand care ingredients can be checked one by one according to the care strategy parameters to see if there are any conflicts between pairs of candidate hand care ingredients or between combinations of multiple ingredients. If a candidate hand care ingredient is obviously incompatible with the selected ingredients in terms of efficacy direction, stability conditions, or compatibility, the candidate hand care ingredient can be removed from the combination. After the check, the ingredient combinations that do not have mechanistic conflicts can be determined as the target ingredient set.

[0207] S805. Using nursing strategy parameters, perform boundary calculations on the candidate concentration ranges in the formula derivation condition text to determine the target concentration value combination corresponding to the target component set.

[0208] In this embodiment, boundary calculation refers to the process of narrowing, shifting, or optimizing the candidate concentration ranges of each candidate hand care component under the guidance of nursing strategy parameters. The target concentration value combination refers to the specific concentration value determined for each component in the target component set. The target concentration value combination must not only satisfy the safe concentration range and candidate concentration range of each component, but also meet the constraints in the nursing strategy parameters regarding total proportion, component synergy, and efficacy focus.

[0209] In some embodiments of this application, in step S805, the candidate concentration ranges corresponding to each component in the target component set can be used as the initial search space, and the care strategy parameters can be transformed into numerical constraints. For example, the care strategy parameters may require the total amount of moisturizing components to be between 15% and 25%, and the total amount of barrier repair components to be between 1% and 5%. Subsequently, a concentration optimization model can be established, and the solution objectives are to meet the constraints, improve efficacy synergy, or improve the adaptability to the target state. Numerical optimization calculations are performed within the safe concentration range of each component to obtain the specific concentration value of each component. These specific concentration values ​​together constitute the target concentration numerical combination.

[0210] S806. Generate a target hand care ingredient formulation based on the combination of the target ingredient set and the target concentration value.

[0211] In this embodiment, the target hand care ingredient formulation scheme refers to the final hand care formula generated based on the combination of a set of target ingredients and target concentration values. This formula may include ingredient names, the specific addition ratios of each ingredient, and optional formulation organization forms. The target hand care ingredient formulation scheme can be directly used for subsequent formula evaluation, process development, or product design.

[0212] In some embodiments of this application, in step S806, the component names in the target component set can be associated and aligned with the specific concentration values ​​in the target concentration value combination item by item, and these component records can be organized according to the formula output format. For example, they can be sorted from high to low according to the amount added, or classified and organized according to aqueous phase components, oil phase components, and functional active ingredients. Furthermore, the base phase components can be supplemented to make the total concentration reach 100%, and finally a structured table or text formula document is formed as the target hand care component formula scheme.

[0213] As can be seen, the embodiments of this application obtain skin physiological attribute feature vectors by semantically parsing the formula derivation condition text, and perform logical reasoning operations by combining the skin care mechanism causal derivation rules in the skin care mechanism relation library, and then perform ingredient mechanism conflict detection and candidate concentration interval boundary calculation. This can improve the rationality of the combination of target ingredient set and target concentration value, and reduce ingredient combination conflicts and concentration setting imbalance problems.

[0214] In the above embodiments, the hand can be treated as a single region for extracting superficial hand features. However, the skin structure and common problems vary significantly across different regions of the hand. For example, the back of the hand is typically smoother, the knuckles are prone to wrinkles due to bending, and the palm edges or finger sides are more susceptible to roughness or thickening due to friction. Mixing these regions may result in the extracted superficial hand features failing to accurately reflect the unique characteristics of each region. Therefore, to enhance the distinguishability of features corresponding to different hand skin regions and reduce feature bias caused by region mixing, some embodiments of this application refine the specific logic for extracting superficial hand features from preprocessed hand images. Specifically, extracting superficial hand features from preprocessed hand images includes:

[0215] S901. Perform skeletal keypoint detection processing on the preprocessed hand image to determine the coordinates of multiple hand skeletal keypoints in the preprocessed hand image.

[0216] In this embodiment, skeletal keypoint detection processing refers to image analysis processing that locates key anatomical positions in a hand image. Hand skeletal keypoint coordinates refer to the set of coordinates in the two-dimensional plane of the image used to represent positions such as the wrist, finger roots, interphalangeal joints, and fingertips. These hand skeletal keypoint coordinates provide a geometric basis for subsequent region boundary delineation.

[0217] In some embodiments of this application, in step S901, the pre-processed hand image can be input into a pre-trained hand pose estimation model, such as the MediaPipe Hands or OpenPose hand keypoint detection network. This model can output a fixed number of hand skeletal keypoint coordinates, for example, 21 keypoint coordinates, used to characterize the wrist, the base of the five fingers, the middle joint position, and the fingertip position. Using these hand skeletal keypoint coordinates, the structural distribution of the current hand in the image can be recovered, providing a basis for subsequent anatomical rule-based region segmentation.

[0218] S902. Using the preset convex hull algorithm, perform boundary calculations on the coordinates of key points of each hand bone to obtain multiple topological boundaries of the hand skin.

[0219] In this embodiment, the convex hull algorithm refers to a processing algorithm that calculates the minimum convex polygon boundary that can enclose a discrete set of points. The topological boundary of the hand skin refers to multiple continuous boundary contours calculated based on the coordinates of key points of the hand bones, used to describe the spatial extent of the overall hand region and its local regions in the image.

[0220] In some embodiments of this application, in step S902, the coordinates of all key points of the hand skeleton can be used as the input point set, and the Graham scan algorithm, Andrew monotonic chain algorithm, or other convex hull calculation algorithms can be used to obtain the overall boundary contour surrounding all key points. Furthermore, convex hull calculation can be performed on the key point subsets corresponding to each finger separately to obtain the local boundaries of the fingers. These boundary contours can collectively constitute multiple topological boundaries of the hand skin, which can be used for subsequent functional region division of the preprocessed hand image.

[0221] S903. Based on the topological boundaries of each hand skin, the preprocessed hand image is divided into smooth area sub-images, folded area sub-images, and edge friction area sub-images.

[0222] In this embodiment, the smooth area sub-image refers to a local image of a region on the back of the hand or palm with relatively uniform texture and fewer wrinkles. The folded area sub-image refers to a local image of a region prone to physiological wrinkles, such as the joints of the fingers or the palm-finger junctions. The edge friction area sub-image refers to a local image of a region prone to contact friction, such as the edge of the palm, the sides of the fingers, or the web of the hand. Through this division, the pre-processed hand image can be segmented according to regional functions and differences in skin condition.

[0223] In some embodiments of this application, in step S903, the coordinate ranges of three regions can be determined based on the topological boundaries of the hand skin and the coordinates of key points of the hand bones, combined with preset anatomical rules. For example, the main area of ​​the back of the hand or the middle area of ​​the palm can be defined as a smooth region, the rectangular or polygonal areas around the key points of each interphalangeal joint can be defined as folded regions, and the strip-shaped areas near the outer edge of the palm, the finger edges, or the web of the hand can be defined as edge friction regions. Subsequently, image blocks corresponding to these three regions can be cropped from the preprocessed hand image to obtain sub-images of the smooth region, folded region, and edge friction regions.

[0224] S904. Extract shallow hand sub-features from the smooth region sub-image, folded region sub-image, and edge friction region sub-image, respectively.

[0225] In the embodiments of this application, shallow hand sub-features refer to the color, texture, or morphological features extracted for a specific sub-region. Shallow hand sub-features in smooth sub-images are more suitable for reflecting skin tone uniformity and smoothness, shallow hand sub-features in folded sub-images are more suitable for reflecting wrinkles and folds, and shallow hand sub-features in edge friction sub-images are more suitable for reflecting roughness and local thickening.

[0226] In some embodiments of this application, in step S904, feature extraction can be performed in parallel on the smooth region sub-image, the folded region sub-image, and the edge friction region sub-image. For the smooth region sub-image, the focus is on extracting color mean, color variance, brightness distribution, and fine-grained texture features; for the folded region sub-image, the focus is on extracting wrinkle direction distribution, texture response histogram, and local complexity features; for the edge friction region sub-image, the focus is on extracting edge density, gray-level co-occurrence matrix contrast, and roughness-related morphological features. After processing, shallow hand sub-feature vectors corresponding to the three regions can be obtained respectively.

[0227] S905. Combine the shallow hand sub-features in the smooth area sub-image, folded area sub-image, and edge friction area sub-image to obtain shallow hand features.

[0228] In this embodiment, the combination processing refers to the process of splicing, summarizing, or weighting the shallow hand sub-features corresponding to multiple regions in a preset order. The shallow hand features refer to the comprehensive feature vector formed after the combination processing, which retains the region-specific information of the smooth area sub-image, folded area sub-image, and edge friction area sub-image, and can also be used as a unified input to participate in subsequent deep feature fusion.

[0229] In some embodiments of this application, in step S905, after obtaining the shallow hand sub-feature vectors corresponding to the smooth area sub-image, folded area sub-image, and edge friction area sub-image, they can be concatenated dimensionally according to a fixed region order. For example, the feature vectors corresponding to the smooth area sub-image can be concatenated first, then the feature vectors corresponding to the folded area sub-image can be concatenated, and finally the feature vectors corresponding to the edge friction area sub-image can be concatenated to form a comprehensive vector. In another case, different combination weights can be assigned to different shallow hand sub-features according to the importance of the regions before integration to form a comprehensive vector. The final comprehensive vector is the shallow hand feature.

[0230] As can be seen, the embodiments of this application perform skeletal key point detection processing on the preprocessed hand image, and use the convex hull algorithm to obtain multiple topological boundaries of the hand skin. Then, the preprocessed hand image is divided into smooth area sub-images, folded area sub-images and edge friction area sub-images, and shallow hand sub-features are extracted and combined respectively. This can enhance the distinguishability of corresponding features of different hand skin regions and reduce feature deviation caused by region mixing.

[0231] In the above embodiments, texture-related features can be extracted from the folded area sub-image. However, the folded area sub-image mainly corresponds to the finger joint bending area. The initial texture representation in this area may simultaneously include static textures formed by dry skin, fine lines, or aging, as well as physiological wrinkles formed by changes in finger bending posture. If these two types of factors are not distinguished and the initial texture features in the folded area sub-image are directly used for skin condition assessment, wrinkles generated by normal activities may be misidentified as abnormal texture representations, thus introducing judgment bias. Therefore, in order to reduce the judgment bias caused by the influence of posture changes on the folded area texture features, some embodiments of this application refine the specific logic for extracting hand skin texture sub-information. Specifically, the shallow hand sub-features in the folded area sub-image include hand skin texture sub-information, which is extracted through the following steps:

[0232] S1001. Based on the coordinates of key points of the hand bones, determine the joint bending angle of each target finger joint within the folded area sub-image.

[0233] In this embodiment, the target knuckle refers to the joint position located within the folded area sub-image that directly affects the distribution of skin folds. The joint bending angle refers to the angle formed by connecting adjacent key points with the intermediate joint point corresponding to the target knuckle as the vertex, and is used to quantify the current degree of bending of the target knuckle.

[0234] In some embodiments of this application, in step S1001, for each target finger joint covered by the folded area sub-image, the coordinates of three key points constituting the target finger joint can be selected from the set of key point coordinates of the hand bones. Subsequently, two vectors pointing from the middle joint point to the two adjacent key points can be constructed respectively, and the angle between the two vectors can be calculated using the vector angle formula. The calculated angle value is the joint bending angle corresponding to the target finger joint.

[0235] S1002. Using a preset tension index function, the joint bending angle is converted to obtain the local skin tension index at the target finger joint.

[0236] In this embodiment, the tension exponential function refers to a preset mapping relationship used to map the joint bending angle to a numerical value of skin tension. The local skin tension index refers to the output of the tension exponential function, used to quantify the degree of theoretical stretching or compression of the skin at the target knuckle due to bending posture. The larger the local skin tension index, the more significant the effect of posture change on the skin at the target knuckle.

[0237] In some embodiments of this application, in step S1002, the tension index function can be set to a sigmoid function, a piecewise linear function, or other monotonic mapping functions. When the joint flexion angle is small, the local skin tension index is close to a low value; as the joint flexion angle increases, the local skin tension index gradually increases; when the joint flexion angle approaches the physiological upper limit, the local skin tension index approaches a high value. By substituting the joint flexion angle into the tension index function, the local skin tension index at the corresponding target finger joint can be obtained.

[0238] S1003. Extract the initial skin texture sub-features at the target knuckle in the folded region sub-image.

[0239] In this embodiment, the initial skin texture sub-feature refers to the original texture description result extracted directly from the local image region corresponding to the target knuckle before introducing bending state correction. The initial skin texture sub-feature is used to reflect the currently visible texture complexity, roughness, directionality, and wrinkle distribution at the target knuckle.

[0240] In some embodiments of this application, in step S1003, the coordinates of the intermediate joint point used to determine the joint bending angle can be used as the center point, and a fixed-size local image patch can be cropped from the folded area sub-image. Subsequently, texture features can be extracted from this local image patch, for example, by using a local binary mode algorithm to generate a texture encoding map and statistically analyze the texture histogram, or by using a gray-level co-occurrence matrix algorithm to calculate contrast, energy value, and correlation. After processing, a fixed-dimensional feature vector can be obtained, which is the initial skin texture sub-feature.

[0241] S1004. Multiply the local skin tension index with the initial skin texture sub-features to obtain the hand skin texture sub-information.

[0242] In this embodiment, the multiplication and product processing refers to the element-wise modulation of the feature values ​​of each dimension in the initial skin texture sub-feature vector using the local skin tension index. The hand skin texture sub-information refers to the new feature vector obtained after modulation processing, which, while retaining the original texture description results, further introduces the degree of influence of the target finger joint bending posture on the texture representation.

[0243] In some embodiments of this application, in step S1004, a local skin tension index and an initial skin texture sub-feature vector can be obtained. The local skin tension index is used as a scaling factor and multiplied by each dimension value in the initial skin texture sub-feature vector. When the local skin tension index is high, it indicates that the texture at the target knuckle is more affected by posture changes, and the corresponding modulation effect is more obvious; when the local skin tension index is low, the initial skin texture sub-feature is closer to static skin properties. After this element-wise multiplication and product process, the hand skin texture sub-information is obtained.

[0244] As can be seen, the embodiments of this application determine the joint bending angle of each target finger joint in the folded area sub-image based on the coordinates of key points of the hand bones, and obtain the local skin tension index using the tension index function. Then, by combining the initial skin texture sub-features for integration processing, the bending state at the target finger joint can be introduced into the hand skin texture sub-information, reducing the judgment bias caused by the influence of posture changes on the texture features of the folded area.

[0245] In the above embodiments, local features can be extracted from the smooth area sub-image and the edge friction area sub-image. However, especially in the edge friction area sub-image, there may be areas of thickened keratin due to long-term friction. A common manifestation of thickened keratin in images is strong highlight reflection on the surface, but this highlight reflection is easily confused with ordinary oiliness on the skin surface or reflections caused by environmental irradiation. If these two sources of highlight are not distinguished, ordinary reflection may be misidentified as thickened keratin, thus affecting the targeting of the thickened keratin sub-feature extraction. Therefore, in order to distinguish between highlight reflection areas and real thickened keratin areas and improve the targeting of the thickened keratin sub-feature extraction results, some embodiments of this application refine the specific logic for extracting thickened keratin sub-features. Specifically, the superficial hand sub-features in the smooth area sub-image and the edge friction area sub-image include thickened keratin sub-features, which are extracted through the following steps:

[0246] S1101. Perform dual-color reflection optical decoupling processing on the smooth area sub-image and the edge friction area sub-image to obtain the corresponding diffuse reflection component image and specular reflection component image, respectively.

[0247] In this embodiment, the dual-color reflection optical decoupling processing refers to the process of decomposing a skin image into diffuse reflection and specular reflection components based on a physical reflection model. The diffuse reflection component image mainly reflects the skin's color, pigment distribution, and texture structure. The specular reflection component image mainly reflects the direct reflection intensity of the skin surface to a light source, including highlights, shine, and local reflection information. By performing dual-color reflection optical decoupling processing, the true structural information of the skin can be separated from the surface reflection information at the optical level.

[0248] In some embodiments of this application, in step S1101, an image decomposition algorithm based on a two-color reflectance model can be applied to the smooth area sub-image and the edge friction area sub-image, respectively. This algorithm estimates the diffuse and specular reflection components of pixels based on pixel clustering distribution, brightness variation relationships, and color saturation differences in the RGB color space. After processing, a diffuse reflection component image retaining only the intrinsic color and texture of the skin, and a specular reflection component image mainly retaining highlight reflection information, can be obtained, respectively.

[0249] S1102. Extract the skin ridge micro-mesh from the diffuse reflection component image and extract the specular reflection region from the specular reflection component image.

[0250] In this embodiment, the dermal ridge micromesh refers to the micro-texture structure of the skin surface formed by the interweaving of dermal ridges and grooves, which can reflect the complete texture connectivity of the skin surface. The specular reflection region refers to a localized reflective area in the specular reflection component image where the brightness value is significantly higher than the surrounding area, used to characterize the location of strong specular reflection on the skin surface.

[0251] In some embodiments of this application, in step S1102, for the diffuse reflection component image, high-frequency enhancement filtering, Laplacian filtering, local contrast enhancement, or ridge detection algorithms can be used to highlight the fine texture structure of the skin surface. Further, image thinning algorithms, texture line connection algorithms, or ridge tracking algorithms can be used to integrate local texture lines into a grid-like representation, thereby extracting the micro-grid of the skin ridges. For the specular reflection component image, a global brightness threshold can be set or an adaptive threshold segmentation algorithm can be used for binarization processing; pixel areas with brightness values ​​higher than the threshold can be marked as specular reflection areas.

[0252] In other embodiments of this application, for the skin microtexture in the diffuse reflection component image, a Hessian matrix or a Frangi filter can be used to enhance the tubular or linear structures in the image. By calculating the local second derivative of the image and constructing the Hessian matrix, the dermal grooves and ridges formed by the directional intersections of normal skin can be accurately responded to and extracted, thereby obtaining a highly restored dermal ridge micromesh.

[0253] S1103. Determine the mesh connectivity density of the dermal ridge microgrid, and identify image regions with mesh connectivity density less than a preset density threshold as mesh breakage regions.

[0254] In this embodiment, mesh connectivity density refers to a numerical index used to quantify the degree of connectivity and integrity of the skin crest micromesh per unit area. A mesh breakage region refers to an image area where the connectivity of the skin crest micromesh is significantly reduced, and the local texture mesh structure is incomplete. Mesh breakage regions typically reflect damage, blurring, or local thickening and occlusion of the skin surface texture structure.

[0255] In some embodiments of this application, in step S1103, the diffuse reflection component image can be divided into multiple local image blocks, and the total length of the grid lines, the number of intersections, or the grid coverage ratio per unit area of ​​the dermal ridge microgrid can be statistically analyzed within each local image block. For example, the total length of the grid lines can be divided by the area of ​​the corresponding image block to obtain the grid connectivity density of that image block. Subsequently, the grid connectivity density corresponding to each image block can be compared with a preset density threshold. When the grid connectivity density of an image block is less than the preset density threshold, the region where the image block is located can be determined as a grid break region.

[0256] S1104. Perform a spatial Boolean intersection operation on the mesh fracture region and the high-reflection region to obtain the target keratin thickening region.

[0257] In this embodiment, spatial Boolean intersection operation refers to the image operation process of performing pixel-by-pixel logical AND processing on two region masks. The target keratin thickening region refers to the region that simultaneously satisfies the mesh breakage feature and the specular reflection feature. Compared with regions that only have specular reflection or only have texture breakage, it can better represent the true local keratin thickening state.

[0258] In some embodiments of this application, in step S1104, the mesh breakage area can be represented as a first binary mask, and the specular reflection area can be represented as a second binary mask. Subsequently, a pixel-by-pixel logical AND operation is performed on the first and second binary masks. Through this spatial Boolean intersection operation, ordinary reflective areas with only highlights but normal texture can be filtered out, as can shadow or smudge areas with only localized texture blurring but lacking surface highlights. Finally, the remaining connected regions can be identified as the target keratin thickening area.

[0259] S1105. Extract the thickened keratin sub-features from the target thickened keratin region.

[0260] In this embodiment, the keratin thickening sub-feature refers to the feature result extracted from the target keratin thickening region, used to quantitatively characterize the local keratin layer thickening state. The keratin thickening sub-feature may include area features, morphological features, brightness features, and texture intensity features, and is used as one of the important shallow hand sub-features in the smooth area sub-image and the edge friction area sub-image.

[0261] In some embodiments of this application, in step S1105, the corresponding pixel region can be located in the original smooth area sub-image or edge friction area sub-image using a target keratin thickening region mask, and descriptive features can be extracted from this region. For example, the proportion of the target keratin thickening region to the area of ​​the corresponding sub-image can be calculated as an area feature; the aspect ratio or boundary complexity of the bounding rectangle of the target keratin thickening region can be calculated as a morphological feature; the average brightness, brightness variance, and local texture roughness of the target keratin thickening region can also be calculated as brightness and texture features. Subsequently, these feature values ​​can be combined into a fixed-dimensional feature vector as a keratin thickening sub-feature.

[0262] As can be seen, the embodiments of this application perform dual-color reflective optical decoupling processing on the smooth area sub-image and the edge friction area sub-image, and combine the skin ridge micro-mesh, high-light reflection area, mesh break area and spatial Boolean intersection operation to extract the keratin thickening sub-features in the target keratin thickening area. This can distinguish the high-light reflection area from the real keratin thickening area and improve the pertinence of the keratin thickening sub-feature extraction results.

[0263] In the above embodiments, the correlation between the fused hand sample features and manually labeled features is determined based on the feature weights of the fused hand sample features. The accuracy of the correlation calculation directly affects the screening effect of anomaly labeled samples. If the correlation calculation is too coarse, it may not be able to accurately identify samples where the manually labeled features are significantly inconsistent with the image features. In order to more accurately correspond the fused texture feature vector with the manually labeled texture level and improve the effectiveness of the correlation calculation results in identifying anomaly labeled samples, some embodiments of this application refine the specific logic for determining the correlation. Specifically, the correlation between the fused hand sample features and manually labeled features is determined based on the feature weights of each sample feature in the fused hand sample features, including:

[0264] S1201, Extract the manually labeled texture level from the manually labeled tags.

[0265] In this embodiment, the manually labeled texture level refers to the grade assigned by professional annotators based on the texture roughness, fine line visibility, or wrinkle appearance in the hand skin sample image. The manually labeled texture level can serve as a true reference value in model training and can be used to subsequently evaluate the consistency between the automatic analysis results and the manually labeled results.

[0266] In some embodiments of this application, in step S1201, the manually labeled tags can be stored in a structured field format. For example, the manually labeled tags can be written as a record format of "drying level is 3, texture level is 5, sensitivity is 2", or stored in key-value pair format. Subsequently, the texture-related field content can be directly parsed from the manually labeled tags, and its corresponding value can be read. If the manually labeled tags are in text level format, the text level can be converted into a standard numerical level through a preset mapping table to obtain the manually labeled texture level.

[0267] S1202. Extract the fused texture feature vector from the fused hand features, and perform quantization decoding on the fused texture feature vector to obtain the physical prediction texture level.

[0268] In this embodiment, the fused texture feature vector refers to the feature sub-vector extracted from the fused hand features that is strongly correlated with skin texture. Quantization decoding refers to the process of mapping the high-dimensional fused texture feature vector to a scalar-level result consistent with the manually annotated texture level scale. Physically predicted texture level refers to the texture level result automatically predicted based on the fused texture feature vector.

[0269] In some embodiments of this application, in step S1202, feature dimensions related to shallow texture response and deep texture representation can be indexed from the fused hand features based on the dimensionality source relationship of the fused hand features, and these dimensions can be combined into a fused texture feature vector. Subsequently, the fused texture feature vector can be input into a pre-trained quantization decoder. This quantization decoder can be a linear regression model, a small neural network, or a single fully connected mapping layer, and its output is a scalar value consistent with the scale of manually labeled texture levels. For example, when the quantization decoder outputs 4.2, 4.2 can be determined as the physically predicted texture level.

[0270] S1203. Determine the level difference between the physically predicted texture level and the manually labeled texture level.

[0271] In this embodiment, the level difference refers to the numerical difference between the physically predicted texture level and the manually labeled texture level, used to quantify the degree of consistency between the automatic texture judgment result and the manually labeled result. The smaller the level difference, the more consistent the two are; the larger the level difference, the more obvious the discrepancy between the two in texture judgment.

[0272] In some embodiments of this application, in step S1203, the physically predicted texture level can be subtracted from the manually labeled texture level, and the absolute value of the difference can be taken. For example, if the physically predicted texture level is 4.2 and the manually labeled texture level is 5, then 0.8 can be obtained as the level difference by subtracting 5 from 4.2 and taking the absolute value.

[0273] S1204. Extract the target feature weights corresponding to the fused texture feature vector from the feature weights.

[0274] In this embodiment, the target feature weight refers to the weight value corresponding one-to-one with each dimension of the texture feature in the fused texture feature vector, or the representative weight result obtained after statistically summarizing these weight values. The target feature weight is used to characterize the degree of influence of texture features in the classification judgment of the current sample.

[0275] In some embodiments of this application, in step S1204, weight values ​​at the same index position can be extracted from the complete feature weight vector according to the dimension index used when extracting the fused texture feature vector. For example, if the fused texture feature vector corresponds to dimensions 101 to 160 in the fused hand features, then weight values ​​of dimensions 101 to 160 in the feature weight vector can be extracted simultaneously. Subsequently, these weight values ​​can be averaged, summed, or weighted to form target feature weights representing the overall importance of the texture features.

[0276] S1205. Determine the degree of correlation based on the target feature weights and level differences.

[0277] In this embodiment, the correlation degree refers to a comprehensive numerical result used to measure the consistency between the fused hand sample features and the manually labeled labels. The calculation of the correlation degree considers both the degree of difference between the physically predicted texture level and the manually labeled texture level, as well as the importance of texture-related features in sample discrimination. If the weight of texture-related features is high, but the difference between the physically predicted texture level and the manually labeled texture level is still large, it indicates that the manually labeled labels are more likely to be inconsistent with the true feature performance of the sample.

[0278] In some embodiments of this application, in step S1205, a correlation calculation function can be constructed to determine the correlation degree. For example, the correlation calculation function may include: Correlation degree = 1 / (1 + Level difference × Target feature weight). According to this form, the larger the level difference, the lower the correlation degree; simultaneously, the larger the target feature weight, the more significant the impact of the level difference on the correlation degree. By substituting the level difference and the target feature weight into the above correlation calculation function, the correlation degree of the current sample can be obtained.

[0279] As can be seen, the embodiments of this application extract the manually labeled texture level from the manually labeled labels and extract the fused texture feature vector from the fused hand features to obtain the physical predicted texture level. Then, the correlation degree is determined by combining the level difference and the target feature weight. This can correspond the fused texture feature vector with the manually labeled texture level and improve the effectiveness of the correlation degree calculation result for identifying labeled abnormal samples.

[0280] In the above embodiments, when performing feature dimensionality reduction based on fused hand features, different dimensions of the fused hand features often have different numerical scales and distribution ranges. For example, the values ​​of color-related features may be between 0 and 1, while the numerical range of some deep-coded features may reach tens or even hundreds. If dimensionality reduction projection is directly performed on fused hand features with large scale differences, the larger numerical dimensions are more likely to dominate the projection direction, thereby masking the information of smaller but more discriminative feature dimensions, thus affecting the discrimination effect of the dimensionality-reduced hand visual features. Therefore, in order to reduce the impact of differences in the numerical scales of different dimensional features on the dimensionality reduction results and improve the discriminative ability of the obtained hand visual features, some embodiments of this application refine the specific logic of feature dimensionality reduction processing. Specifically, feature dimensionality reduction processing based on fused hand features yields hand visual features, including:

[0281] S1301. Extract the mean and standard deviation of the fused hand features in the current dimension sequence.

[0282] In this embodiment, the feature mean refers to the average value obtained by statistically analyzing multiple samples along the same dimension using fused hand features, reflecting the central tendency of that dimension's features. The feature standard deviation is the quantified result of the dispersion of a feature within the same dimension relative to the feature mean, reflecting the fluctuation range of that dimension's features.

[0283] In some embodiments of this application, in step S1301, the aforementioned statistics can be calculated based on a representative calibration dataset. This calibration dataset may contain multiple preprocessed and feature-fused hand image samples. Assuming the fused hand features are a D-dimensional vector, for the d-th feature, the values ​​of all samples in the calibration dataset along that dimension can be iterated, and their arithmetic mean can be calculated as the feature mean of the d-th dimension; simultaneously, the standard deviation of the values ​​along that dimension can be calculated as the feature standard deviation of the d-th dimension. After repeating the above processing for all D dimensions, a feature mean vector of length D and a feature standard deviation vector of length D can be obtained.

[0284] S1302. Based on the feature mean and feature standard deviation, perform normalization calculation on the fused hand features to obtain the standardized fused features.

[0285] In this embodiment, normalization computation refers to the process of decentralizing and scaling each dimension of the fused hand features. The standardized fused feature refers to the new feature vector obtained after normalization computation. After normalization computation, the feature values ​​of different dimensions can be adjusted to a closer numerical scale range, thereby reducing the impact of differences in the original feature dimensions.

[0286] In some embodiments of this application, in step S1302, for the fused hand feature vector of any sample to be processed, normalization calculation can be performed dimension by dimension. Specifically, for the d-th dimension feature value x d It can be combined with the mean μ of the d-th dimension feature. d and the standard deviation σ of the d-th dimension feature d According to formula z d =(x d -μ d ) / σ d Perform the calculation. After completing the calculation for all D dimensions, a new D-dimensional vector [z1, z2, ..., z] can be obtained. D The D-dimensional vector is the standardized fusion feature. After this processing, the depth features, which originally had a large numerical range, and the color features, which had a small numerical range, can be unified to a similar scale.

[0287] S1303. Using a preset linear discriminant analysis algorithm, the standardized fusion features are subjected to dimensionality reduction projection processing to obtain hand visual features.

[0288] In some embodiments of this application, in step S1303, training samples labeled with hand skin condition categories can be used to calculate the class mean vectors of the standardized fusion features under each category, and further calculate the intra-class scatter matrix and inter-class scatter matrix. Subsequently, the generalized eigenvalue problem corresponding to the linear discriminant analysis algorithm can be solved to obtain the optimal projection matrix. Then, the standardized fusion feature vector can be multiplied by this optimal projection matrix, thereby projecting the original high-dimensional features into a low-dimensional discriminant subspace. For example, if the original standardized fusion features are 1024-dimensional and the number of categories is 5, they can be projected into 4-dimensional or other preset-dimensional hand visual features. This hand visual feature retains category discrimination information while reducing redundant feature dimensions.

[0289] As can be seen, the embodiments of this application extract the feature mean and feature standard deviation of the fused hand features in the current dimension sequence, and perform normal standardization calculation on the fused hand features accordingly to obtain standardized fused features. Then, the linear discriminant analysis algorithm is used for dimensionality reduction projection processing, which can reduce the influence of the difference in the numerical scale of features of different dimensions on the dimensionality reduction results and improve the discrimination ability of the obtained hand visual features.

[0290] Secondly, embodiments of this application provide a hand skin condition analysis system based on AI image recognition, which can run on an electronic device. The AI ​​image recognition-based hand skin condition analysis system includes a computer program that, when executed by a processor, implements the steps of the hand skin condition analysis method based on AI image recognition as described in any of the above embodiments.

[0291] Thirdly, embodiments of this application provide an electronic device. This electronic device may include a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the AI ​​image recognition-based hand skin state analysis method as described in any of the above embodiments.

[0292] Fourthly, embodiments of this application provide an electronic device capable of running a hand skin condition analysis system based on AI image recognition. For example... Figure 2 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0293] The electronic device includes a Central Processing Unit (CPU) 201, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 202 or a program loaded from storage portion 208 into Random Access Memory (RAM) 203, such as performing the methods described in the above embodiments. The RAM 203 also stores various programs and data required for system operation. The CPU 201, ROM 202, and RAM 203 are interconnected via a bus 204. An Input / Output (I / O) interface 205 is also connected to the bus 204.

[0294] The following components are connected to I / O interface 205: input section 206 including audio input devices, push-button switches, etc.; output section 207 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 208 including a hard disk, etc.; and communication section 209 including a network interface card such as a local area network (LAN) card, modem, etc. Communication section 209 performs communication processing via a network such as the Internet. Drive 210 is also connected to I / O interface 205 as needed. Removable media 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 210 as needed so that computer programs read from them can be installed into storage section 208 as needed.

[0295] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the computer program is executed by CPU 201, it performs the various functions defined in this application.

[0296] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0297] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0298] Specifically, the electronic device of this embodiment includes a processor and a memory. The memory is coupled to one or more processors and is used to store computer program code. The computer program code includes computer instructions. One or more processors call the computer instructions to cause the electronic device to perform the method provided in the above embodiment.

[0299] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the hand skin state analysis method based on AI image recognition as described in any of the preceding claims.

[0300] Sixthly, embodiments of this application provide a computer program product, including a computer program or instructions, which are executed by a processor to implement the steps of the hand skin state analysis method based on AI image recognition as described in any of the preceding claims.

[0301] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A hand skin state analysis method based on AI image recognition, characterized by, The method includes: Obtain the hand image to be analyzed; Feature extraction is performed on the hand image to obtain hand visual features, and the hand visual features are input into a preset hand image classification model to obtain preliminary analysis results of hand skin condition; The hand visual features and the preliminary analysis results are input into a preset skin care semantic big language model to obtain the corresponding skin care semantic description text; The skincare semantic description text is compared with the skincare common sense rules in the preset skincare knowledge base to obtain the skincare deviation comparison results. Based on the skin care deviation comparison results, the preliminary analysis results are corrected to obtain the target analysis results of the hand skin condition in the hand image.

2. The hand skin state analysis method based on AI image recognition as described in claim 1, characterized in that, The step of extracting features from the hand image to obtain hand visual features includes: The hand image is normalized in size and converted to a preset color gamut to obtain a hand image with a target color gamut. Semantic segmentation is performed on the target color gamut hand image to obtain the region of interest hand image; The hand image of the region of interest is filtered, denoised, and illuminated to obtain an enhanced hand image; Skin color detection is performed on the enhanced hand image to extract a pure hand mask for the pure skin region; A preprocessed hand image is generated based on the clean hand mask and the enhanced hand image; Feature extraction is performed on the preprocessed hand image to obtain the visual features of the hand.

3. The hand skin state analysis method based on AI image recognition as described in claim 2, characterized in that, The step of extracting features from the preprocessed hand image to obtain the visual features of the hand includes: Extracting shallow hand features from the preprocessed hand image, wherein the shallow hand features include at least one of hand skin color information, hand skin texture information, and hand skin morphology information; The preprocessed hand image is input into a preset depth visual coding model to obtain deep hand features; The shallow hand features and the deep hand features are dimensionally aligned to obtain initial two-dimensional features; Based on a preset attention mechanism, the feature weights of each feature in the initial two-dimensional features are obtained, and multiple features in the initial two-dimensional features are weighted and fused according to the feature weights to obtain fused hand features. Based on the fused hand features, feature dimensionality reduction processing is performed to obtain the hand visual features.

4. The hand skin state analysis method based on AI image recognition as described in claim 3, characterized in that, The hand image classification model is trained through the following steps: Obtain the fused hand sample features and corresponding manually labeled data from each sample image in a preset set of hand skin sample images; Based on the feature weights of each sample feature in the fused hand sample features, the correlation between the fused hand sample features and the manually labeled tags is determined; Sample images corresponding to the fused hand sample features with a correlation degree less than a preset correlation threshold are designated as anomaly samples. The labeled abnormal samples are input into a preset label prediction model to obtain the corrected label of the labeled abnormal samples, and the corresponding manual label is replaced with the corrected label to obtain a set of replaced hand skin sample images; The hand image classification model is obtained by training the model based on the replaced set of hand skin sample images.

5. The hand skin state analysis method based on AI image recognition as described in claim 4, characterized in that, The model training based on the replaced hand skin sample image set to obtain the hand image classification model includes: The set of replaced hand skin sample images is divided into a training sample set and a validation sample set; The fused hand sample features of each sample image in the training sample set are input into a preset initial network model to obtain the training classification result; Based on the training classification results, the corrected labeling labels corresponding to each sample image, and the preset parameter regularization term, the training loss value of the initial network model is calculated. Based on the training loss value, the model parameters of the initial network model are adjusted to obtain an iterative network model; Obtain the current verification accuracy value of the verification sample set on the iterative network model, and when the number of consecutive times the current verification accuracy value does not increase is equal to a preset stopping round threshold, determine the iterative network model as the hand image classification model.

6. The hand skin state analysis method based on AI image recognition as described in claim 1, characterized in that, After obtaining the target analysis results of the hand skin condition in the hand image, the method further includes: Extract the initial predicted probability of the hand skin condition from the target analysis results; At least two target probability calibration algorithms are selected from a plurality of preset candidate probability calibration algorithms, wherein the candidate probability calibration algorithms include at least one of linear scaling algorithm, Pratt scaling algorithm and Bayesian calibration algorithm; The at least two target probability calibration algorithms are spliced ​​together to obtain a combined probability calibration algorithm. The initial prediction probability is calibrated using the combined probability calibration algorithm to obtain the calibrated prediction probability. The initial predicted probability in the target analysis result is replaced with the calibrated predicted probability to obtain the optimized skin condition analysis result.

7. The hand skin state analysis method based on AI image recognition as described in claim 6, characterized in that, After obtaining the optimized skin condition analysis results, the process also includes: The calibrated prediction probability is extracted from the optimized skin condition analysis results; Based on the calibration prediction probability, the state quantification index corresponding to each hand skin state is determined; Based on the quantitative indicators of each state, determine the corresponding ingredient requirement indicators for the skin condition of the hands; Using the ingredient requirement index, corresponding candidate hand care ingredients and candidate concentration ranges of the candidate hand care ingredients are determined from a preset skin care ingredient database; The calibration prediction probability, the candidate hand care ingredients, and the candidate concentration range are concatenated to obtain the formula derivation condition text. Using a preset formula parameter generation model, the formula combination and proportion calculation are performed on the formula derivation condition text to obtain the target hand care ingredient formula scheme.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the hand skin state analysis method based on AI image recognition as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the hand skin state analysis method based on AI image recognition as described in any one of claims 1 to 7.

10. A hand skin condition analysis system based on AI image recognition, characterized in that, The method includes a computer program that, when executed by a processor, implements the steps of the hand skin state analysis method based on AI image recognition as described in any one of claims 1 to 7.

Images