Skin improvement method and system based on image processing technology
By using image processing technology of VGG16 network and BLSTM network in skin detection applications, the problem of inaccurate skin detection in the existing technology is solved, and efficient and personalized skin problem identification and treatment plan formulation is achieved, ensuring the privacy and security of user data.
Patent Information
- Application Number
- CN202510247797.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-04
- Publication Date
- 2025-05-30
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing skin detection applications lack intelligent image processing capabilities and cannot accurately identify skin conditions, resulting in limited improvement effects and insufficient image processing algorithms, which affects the skin improvement effects.
Image processing technology based on VGG16 network and BLSTM network is used to pre-process, detect, feature extraction and match real-time skin photos uploaded by users, combine neural network models to build a skin problem type recognition model, and formulate a personalized treatment plan.
It improves the accuracy and efficiency of skin problem detection, can provide users with personalized treatment plans, improves the pertinence and effectiveness of treatment effects, and sends the solutions through encrypted form to ensure the privacy and security of user data.
Smart Images
Figure CN120070246A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and specifically relates to a skin improvement method and system based on image processing technology. Background Art
[0002] Skin detection is a scientific test for the skin texture or condition. As an important indicator for evaluating the overall texture during the skin test, through the characterization of glossiness, systematic guidance suggestions can be provided for skin management.
[0003] With the development of technology, there are currently apps on the market for identifying users' skin problems, but there are still the following deficiencies and drawbacks: Existing products lack intelligent image processing capabilities, cannot accurately identify skin conditions, resulting in limited improvement effects, and the image processing algorithms are not advanced enough, resulting in low analysis accuracy and efficiency, affecting the skin improvement effect and making it difficult to meet the needs of staff. Summary of the Invention
[0004] To solve the above technical problems, a skin improvement method and system based on image processing technology are provided. This technical solution solves the problems in the above background art that existing products lack intelligent image processing capabilities, cannot accurately identify skin conditions, resulting in limited improvement effects, and the image processing algorithms are not advanced enough, resulting in low analysis accuracy and efficiency, affecting the skin improvement effect.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In the first aspect of the present invention, a skin improvement method based on image processing technology is provided, including: The user uploads a real-time skin photo taken in a mobile application, and the mobile application preprocesses the real-time skin photo using image processing technology. The preprocessing includes image denoising and image enhancement; Detect the real-time skin photo based on the VGG16 network and the BLSTM network to obtain the real-time skin problem area; Extract features from the detected real-time skin problem area based on the BLSTM layer to obtain real-time skin problem feature data, where the real-time skin problem feature data includes acne, freckles, blackheads, wrinkles, etc.; Based on the neural network model, and retrieve all historical user skin problem feature data and the corresponding skin type data of the user from the database to construct a skin problem type recognition model; Calculate the matching degree between the real-time skin problem feature data and all historical user skin problem feature data based on the skin problem type recognition model to determine the user's skin problem type; Based on the types of users' skin problems, a personalized treatment plan is formulated for the users, and the personalized treatment plan is sent to the users in an encrypted form.
[0006] Preferably, the image denoising specifically includes the following steps: Perform adaptive wavelet threshold denoising on the real-time skin photo; By constructing a probability model of wavelet coefficients, use Bayesian estimation to determine the optimal wavelet threshold; Perform adaptive threshold processing on the wavelet coefficients and perform wavelet reconstruction to obtain the denoised image.
[0007] Preferably, the image enhancement specifically includes the following steps: Perform adaptive histogram equalization processing on the denoised image; By calculating the gray histogram of the denoised image, determine the cumulative distribution function of the local area; Based on the cumulative distribution function, adaptively adjust the gray mapping relationship of the local area to obtain the enhanced image.
[0008] Preferably, the detection of the real-time skin photo based on the VGG16 network and the BLSTM network to obtain the real-time skin problem area specifically includes the following steps: Input the enhanced skin photo into the VGG16 network; Obtain the mapping of the convolutional layer in the VGG16 network and obtain features of size W×H×C, where W is the width of the picture, H is the height of the picture, and C is the number of picture channels; Use a sliding window to extract feature vectors from the convolutional layer; Input the extracted feature vectors into the BLSTM network for processing to obtain the real-time skin problem area.
[0009] Preferably, the feature extraction of the detected real-time skin problem area based on the BLSTM layer to obtain the real-time skin problem feature data specifically includes the following steps: Input the real-time skin problem area into the VGG16 network to generate several convolutional feature matrices; Each convolutional feature matrix corresponds to a rectangular area, and input the convolutional feature matrix into the BLSTM layer; Set the maximum time length of the BLSTM layer, and classify the result output by the BLSTM using the softmax function to obtain data such as acne, age spots, blackheads, wrinkles, etc.
[0010] Preferably, the construction of the skin problem type recognition model based on the neural network model and retrieving all historical users' skin problem feature data and the corresponding skin type data of the users from the database specifically includes the following steps: Based on a feedforward neural network, a skin problem type recognition model is constructed, and the input data of the skin problem type recognition model includes user skin problem feature data information; Based on machine learning, the multiple user skin problem feature data information is labeled and divided to obtain a training set, a validation set, and a test set; The training set, the validation set, and the test set are used to supervise the training, validation, and testing of the skin problem type recognition model, and the skin problem type recognition model with an accuracy rate meeting the preset accuracy rate requirement is obtained.
[0011] Preferably, the formula for calculating the matching degree between the real-time skin problem feature data and the skin problem feature data of all historical users is:
[0012] In the formula, is the similarity of the i-th feature between the real-time skin problem feature data and the historical user skin problem feature data, is the j-th feature index value of the i-th feature of the real-time skin problem feature data, is the j-th feature index value of the i-th feature of the historical user skin problem feature data, is the total number of the i-th feature index values.
[0013] Preferably, the specific steps of sending the personalized treatment plan to the user in an encrypted form are as follows: Encode and encrypt the personalized treatment plan and send it to the user's mobile application; The mobile application encodes and decodes the information received by the user, makes it display as garbled characters, and transmits it to the storage space for storage; When the user needs to consult the information, by performing a set operation through the mobile application, the decoding function is awakened, the information is decoded, and the information is displayed normally. If the set operation is not performed, the information is displayed as garbled characters.
[0014] In the second aspect of the present invention, a skin improvement system based on image processing technology is further provided, including: A preprocessing module, which is used for the user to upload the captured real-time skin photo in the mobile application, and the mobile application preprocesses the real-time skin photo by using image processing technology, and the preprocessing includes image denoising and image enhancement; A detection module, which is used to detect the real-time skin photo based on the VGG16 network and the BLSTM network to obtain the real-time skin problem area; A feature extraction module, which is used to extract features from the detected real-time skin problem area based on the BLSTM layer to obtain real-time skin problem feature data, where the real-time skin problem feature data includes acne, age spots, blackheads, wrinkles, etc.; A model construction module, which is used to construct a skin problem type recognition model based on a neural network model and retrieve all historical user skin problem feature data and the corresponding skin type data of the users from the database; A calculation module, which is used to calculate the matching degree between the real-time skin problem feature data and all historical user skin problem feature data based on the skin problem type recognition model to determine the skin problem type of the user; A sending module, which is used to formulate a personalized treatment plan for the user according to the skin problem type of the user and send the personalized treatment plan to the user in an encrypted form.
[0015] In the third aspect of the present invention, an electronic device is further provided. The electronic device has at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method of the first aspect of the present invention.
[0016] Compared with the prior art, the present invention provides a skin improvement method and system based on image processing technology, which has the following beneficial effects: By allowing users to upload real-time skin photos for analysis, the present invention enables users to obtain personalized skin analysis reports and treatment plans without leaving the application, which greatly improves the convenience and satisfaction of users. Using the VGG16 network and the BLSTM network for skin problem detection and combining image preprocessing technologies (such as denoising and enhancement) can improve the accuracy and efficiency of skin problem detection. Moreover, based on the matching between the real-time skin problem feature data and historical user data, a personalized treatment plan can be formulated for the user. This personalized method takes into account the unique skin conditions and needs of each user, improving the pertinence and effectiveness of the treatment effect. In addition, by sending the personalized treatment plan in an encrypted form, the privacy and security of user data are ensured, increasing the user's trust in the application and complying with relevant data protection regulations. Description of the Drawings
[0017] Figure 1 It is a schematic diagram of the skin improvement method based on image processing technology in the present invention; Figure 2 It is a schematic diagram of the method for image denoising in the present invention; Figure 3Schematic diagram of the method for image enhancement in the present invention; Figure 4 Schematic diagram of the method for obtaining real-time skin problem areas in the present invention; Figure 5 Schematic diagram of the method for obtaining real-time skin problem feature data in the present invention; Figure 6 Schematic diagram of the method for constructing a skin problem type recognition model in the present invention; Figure 7 Block diagram of an exemplary electronic device capable of implementing embodiments of the present invention; Among them, 700 is an electronic device, 701 is a computing unit, 702 is a ROM, 703 is a RAM, 704 is a bus, 705 is an I / O interface, 706 is an input unit, 707 is an output unit, 708 is a storage unit, and 709 is a communication unit. Detailed implementation manners
[0018] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations.
[0019] Embodiment 1 Please refer to Figure 1 As shown, in the first aspect of the present invention, a skin improvement method based on image processing technology is provided, including: S101. The user uploads a captured real-time skin photo in a mobile application, and the mobile application preprocesses the real-time skin photo using image processing technology. The preprocessing includes image denoising and image enhancement; S102. Detect the real-time skin photo based on the VGG16 network and the BLSTM network to obtain real-time skin problem areas; S103. Extract features from the detected real-time skin problem areas based on the BLSTM layer to obtain real-time skin problem feature data, where the real-time skin problem feature data includes acne, age spots, blackheads, wrinkles, etc.; S104. Based on a neural network model, retrieve all historical user skin problem feature data and the corresponding skin type data of the user from the database, and construct a skin problem type recognition model; S105. Calculate the matching degree between the real-time skin problem feature data and all historical user skin problem feature data based on the skin problem type recognition model to determine the user's skin problem type; S106. For the user's skin problem type, formulate a personalized treatment plan for the user and send the personalized treatment plan to the user in an encrypted form.
[0020] Those skilled in the art can understand that the present invention allows users to upload real-time skin photos for analysis, and users can obtain personalized skin analysis reports and treatment plans without leaving the application, which greatly improves the convenience and satisfaction of users. By using the VGG16 network and the BLSTM network for skin problem detection and combining image preprocessing techniques (such as denoising and enhancement), the accuracy and efficiency of skin problem detection can be improved. Moreover, based on the matching of real-time skin problem feature data and historical user data, personalized treatment plans can be formulated for users. This personalized method takes into account the unique skin conditions and needs of each user, improving the pertinence and effectiveness of the treatment effect. In addition, by sending the personalized treatment plan in an encrypted form, the privacy and security of user data are ensured, increasing users' trust in the application and complying with relevant data protection regulations.
[0021] Please refer to Figure 2 as shown, the image denoising specifically includes the following steps: S201. Perform adaptive wavelet threshold denoising on the real-time skin photo; S202. Determine the optimal wavelet threshold by constructing a probability model of wavelet coefficients and using Bayesian estimation; S203. Perform adaptive threshold processing on the wavelet coefficients and perform wavelet reconstruction to obtain the denoised image.
[0022] Those skilled in the art can understand that adaptive wavelet threshold denoising can dynamically adjust the threshold according to the noise level and features in the skin photo, thereby more effectively removing noise. This helps to retain detailed information in the skin image, such as texture, edges, etc., while removing unnecessary noise, improving the clarity and quality of the image. Moreover, in skin problem recognition and analysis, high-quality images are crucial. Through adaptive wavelet threshold denoising, the visual effect of the skin image can be improved, making subsequent skin problem recognition and analysis more accurate. When the adaptive wavelet threshold denoising method processes real-time skin photos, it can dynamically adjust according to different noise levels and image features, thereby optimizing the processing efficiency while ensuring the denoising effect. This helps to shorten the image processing time, improve the response speed of the overall service and the user experience. By constructing a probability model of wavelet coefficients and using Bayesian estimation to determine the optimal wavelet threshold, the denoising method can be made more flexible and robust. This helps to enhance the generalization ability of the model under different noise levels and image features, enabling the denoising method to be more widely applied to the processing and analysis of various skin images; Please refer to Figure 3 as shown, the image enhancement specifically includes the following steps: S301. Perform adaptive histogram equalization processing on the denoised image; S302. Determine the cumulative distribution function of the local region by calculating the grayscale histogram of the denoised image; S303. Based on the cumulative distribution function, adaptively adjust the grayscale mapping relationship of the local region to obtain an enhanced image.
[0023] Those skilled in the art can understand that adaptive histogram equalization can perform histogram equalization on the local regions of an image, thereby improving the contrast of the image. Compared with global histogram equalization, this method can better adapt to the brightness distribution of different regions in the image, avoiding over-enhancement or detail loss that may be caused by global processing; through the adjustment of the grayscale mapping relationship of the local region, adaptive histogram equalization can more finely control the enhancement degree of different regions in the image, which helps to retain the detail information in the image, such as edges, textures, etc., making the enhanced image clearer and more natural; before performing adaptive histogram equalization processing, the image has been denoised, which helps to reduce the impact of noise on the histogram equalization process, avoiding over-amplification of noise or introduction of new noise. Adaptive histogram equalization processing can significantly improve the visual effect of the image, making the details in the image more prominent and the overall brightness distribution more uniform. This helps users to more accurately identify and analyze the information in the image.
[0024] Please refer to Figure 4 As shown, detecting real-time skin photos based on the VGG16 network and the BLSTM network to obtain the specific real-time skin problem areas includes the following steps: S401. Input the enhanced skin photo into the VGG16 network; S402. Obtain the mapping of the convolutional layer in the VGG16 network and obtain features of size W×H×C, where W is the width of the picture, H is the height of the picture, and C is the number of picture channels; S403. Use a sliding window on the convolutional layer to extract feature vectors; S404. Input the extracted feature vectors into the BLSTM network for processing to obtain the real-time skin problem areas.
[0025] Those skilled in the art can understand that the specific Python code is as follows: import cv2 import numpy as np from tensorflow.keras.applications import VGG16 from tensorflow.keras.models import Model from tensorflow.keras.layers import Input, Bidirectional, LSTM, TimeDistributed, Dense, Flatten from tensorflow.keras.preprocessing import image from tensorflow.keras.applications.vgg16 import preprocess_input # Load the pre-trained VGG16 model and remove the top fully-connected layers base_model = VGG16(weights='imagenet', include_top=False, input_shape=(224, 224, 3)) # Freeze the convolutional layers of VGG16 so that their weights are not updated during the training of BLSTM for layer in base_model.layers: layer.trainable = False # Define the input layer input_layer = Input(shape=(224, 224, 3)) # Connect the input layer to the convolutional layers of VGG16 x = base_model(input_layer) # Assume we get the feature map from a certain convolutional layer of VGG16 (e.g., block5_conv3) # Here we don't directly obtain the output of a specific layer for now, but assume x is already the desired feature map # In actual applications, you need to specify a certain layer of base_model as the output and pass it to subsequent processing # For example: x = base_model.get_layer('block5_conv3').output # Since we assume x is already a feature map of size W×H×C, we can directly use it # Note that BLSTM requires sequential data, so we need to convert the feature map into a sequential form # This usually involves slicing or reshaping the feature map in the spatial dimensions # For simplicity, here we assume a fixed window size and reshape the feature map into the form of (num_windows, window_size, C) # In practical applications, you need to calculate num_windows and window_size based on the window size and stride window_size = 16 # Assumed window size step_size = 8 # Assumed stride C = x.shape[-1] # Number of channels of the feature map (here we assume x has been calculated, but in reality you need to obtain it from the model) # Note: The following code is conceptual and not directly runnable as it depends on the specific shape and dimensions of x # In practical applications, you need to adjust the reshaping and slicing operations according to the shape of x # Assume x has been calculated as a numpy array or a similar tensor form # num_windows = (W - window_size) / / step_size + 1 # Need to consider boundary cases during actual calculation # x_reshaped = x.reshape((-1, window_size, window_size, C)) # Assume reshaped into a 4D tensor, but this is usually not the input format for BLSTM # You need to further process x_reshaped to convert it into a sequence data format suitable for BLSTM # Since the direct conversion from the feature map to BLSTM is relatively complex, here we skip the specific reshaping and slicing steps # Instead, assume you already have the correct input format (num_samples, time_steps, features) # Where num_samples is the number of samples (in our case, it may be the number of sliding windows), time_steps is the window size (but flattened here), and features is the number of channels C # Note: The following code is for illustrative purposes only and is not a practical solution # Assume x_blstm_input is the input data already prepared for BLSTM # x_blstm_input =... # You need to fill in the code here to generate the correct input data # Define the BLSTM layer blstm_layer = Bidirectional(LSTM(64, return_sequences=True))(x_blstm_input) # Here, return_sequences=True is to preserve sequence information, but the specific setting depends on your task # You may also need to add additional layers to process the output of the BLSTM, such as fully connected layers, softmax layers, etc. # But for simplicity, we stop here # Since the above code is conceptual and there is no real x_blstm_input, we cannot build a complete model # In practical applications, you need to build your model according to the above guidance and compile and train it # Assume you already have the enhanced skin photos and have preprocessed them # enhanced_image =... # Your enhanced skin photo, which needs to be preprocessed and resized to the input size of VGG16 (224x224) # enhanced_image = cv2.resize(enhanced_image, (224, 224)) # enhanced_image = preprocess_input(enhanced_image) # enhanced_image = np.expand_dims(enhanced_image, axis=0) # Add batch dimension # If you have built a complete model (including the VGG16 and BLSTM parts), you can call it like this # model_output = your_complete_model.predict(enhanced_image) # Then, you can process the detection of real-time skin problem areas based on model_output # Note: The above code is a simplified framework to illustrate how to combine VGG16 and BLSTM to process skin photos # In practical applications, you need to adjust the code according to your specific requirements and data.
[0026] Please refer to Figure 5As shown, the feature extraction of the detected real-time skin problem area is performed based on the BLSTM layer to obtain the real-time skin problem feature data, which specifically includes the following steps: S501. Input the real-time skin problem area into the VGG16 network to generate several convolutional feature matrices; S502. Each convolutional feature matrix corresponds to a rectangular area, and the convolutional feature matrix is input into the BLSTM layer; S503. Set the maximum time length of the BLSTM layer, and classify the result output by the BLSTM using the softmax function to obtain data such as acne, age spots, blackheads, and wrinkles.
[0027] Those skilled in the art can understand that the specific Python code is as follows: import tensorflow as tf from tensorflow.keras.applications import VGG16 from tensorflow.keras.layers import Input, GlobalAveragePooling2D,Reshape, Permute, Bidirectional, LSTM, TimeDistributed, Dense, Softmax from tensorflow.keras.models import Model import numpy as np # Assume that we already have a preprocessed image input of the real-time skin problem area, with the shape (batch_size, img_height, img_width, channels) # Here we use a random array to simulate this input input_shape = (224, 224, 3) # The default input shape of VGG16 batch_size = 1 # Assume that we process one image at a time img_input = np.random.rand(batch_size, *input_shape).astype(np.float32) # Randomly generated simulated input # Load the pre-trained VGG16 model and remove the top fully connected layer vgg_base = VGG16(weights='imagenet', include_top=False, input_shape=input_shape) # Freeze the convolutional layers of VGG16 for layer in vgg_base.layers: layer.trainable = False # Define the input layer input_layer = Input(shape=input_shape) # Connect the input layer to the convolutional layers of VGG16 x = vgg_base(input_layer) # Assume we obtain the feature map from the last convolutional layer of VGG16 # Note: In practical applications, you may need to select other layers as the feature extraction layer last_conv_layer_name = 'block5_conv3' # Example: Select a certain convolutional layer of VGG16 as the feature extraction layer x = vgg_base.get_layer(last_conv_layer_name).output # Since BLSTM requires sequential data, we need to convert the feature map into a sequential form # Here we adopt a simple method: slice the feature map in the spatial dimension, and each slice is input into the BLSTM as a time step # Note: This method may not be optimal because it ignores the spatial structure information of the feature map # More complex methods may involve using convolutional LSTM or other types of sequential processing layers to process the entire feature map # Assume we slice the feature map into small blocks of size (height_slices, width_slices), and each block is a time step for the BLSTM # Here for simplicity, we assume that the height and width of the feature map are divisible by the slice size height_slices = 7 # Example: Assume we slice the feature map into 7 blocks in height width_slices = 7 # Example: Assume we slice the feature map into 7 blocks in width slice_height = x.shape[1] / / height_slices slice_width = x.shape[2] / / width_slices # Reshape the feature map to (batch_size, height_slices * width_slices, slice_height * slice_width * channels) # Note: The reshaping operation here is to match the input requirements of the BLSTM, but it may not be optimal as it destroys the spatial structure of the feature map x = Reshape((height_slices * width_slices, slice_height * slice_width* vgg_base.layers[-1].output_shape[-1]))(x) # Since the BLSTM expects the input shape to be (batch_size, time_steps, features) # We need to swap dimensions to match this shape x = Permute((1, 2, 0))(x) # Note: The Permute operation here may need to be adjusted according to the actual reshape result x = Reshape((height_slices * width_slices, -1))(x) # Remove the last dimension (batch size) and merge the feature dimensions # Now the shape of x should be (batch_size * height_slices * width_slices,slice_height * slice_width * channels) # But since we set batch_size = 1 before, the actual shape of x is (height_slices* width_slices, slice_height * slice_width * channels) # To match the expected input shape of the BLSTM, we need to add a batch dimension again (we temporarily keep this shape unchanged until later processing) # Note: In a real application, you may need to further process x here to match the input requirements of the BLSTM # Assume we already have the correct input shape, and now define the BLSTM layer # Note: Due to possible issues with our previous reshape and permute operations, the following BLSTM layer definition may need to be adjusted according to the actual situation blstm_layer = Bidirectional(LSTM(64, return_sequences=False))(x) # Here return_sequences=False because we only need the final output of the BLSTM # Since the shape of our input x may be incorrect, the above BLSTM layer definition may not run directly # In practical applications, you need to ensure that the shape of x matches the input requirements of the BLSTM # A possible solution is to flatten the feature map into a long vector sequence, where each vector corresponds to the features at a spatial position # However, this method will lose spatial structure information and may not be optimal # Due to the problems in the above code, we cannot directly build a complete model # But to illustrate the classification part, we assume that we already have a correct BLSTM output blstm_output # blstm_output =... # You need to fill in the code here to generate the correct BLSTM output # Assume the shape of blstm_output is (batch_size * num_sequences, features) # We need to reshape it to (batch_size, num_sequences, features) to match the input requirements of the softmax layer # But due to possible issues with our previous operations, we temporarily skip the reshape step here # In practical applications, you need to ensure that the shape of blstm_output is correct and perform the necessary reshape operations # Define the softmax layer for classification # Assume we have 4 classes: acne, age spots, blackheads, wrinkles num_classes = 4 softmax_layer = Dense(num_classes, activation='softmax')(blstm_output) # Note: The blstm_output here needs to be in the correct shape # Since there is a problem with the above code, we cannot directly build a complete model # But for illustration purposes, we assume that we already have a complete model and can compile and predict # model = Model(inputs=input_layer, outputs=softmax_layer) # model.compile(optimizer='adam', loss='categorical_crossentropy',metrics=['accuracy']) # Assume that we already have the correct labels y_true, and we can make predictions # y_pred = model.predict(img_input) # Note: The img_input here needs to be in the correct shape, and the model needs to be correct # Note: The above code is a simplified framework for illustrating how to combine VGG16 and BLSTM for classification tasks # In practical applications, you need to adjust the code according to your specific requirements and data # In particular, you need to ensure that the feature maps are correctly converted into sequence data, and the input and output shapes of the BLSTM are correct.
[0028] Please refer to Figure 6 As shown, based on the neural network model, and retrieving all historical user skin problem feature data and the corresponding skin type data of users from the database, constructing a skin problem type recognition model specifically includes the following steps: S601. Based on the feedforward neural network, construct a skin problem type recognition model, and the input data of the skin problem type recognition model includes user skin problem feature data information; S602. Based on machine learning, label and divide the skin problem feature data information of multiple users to obtain a training set, a validation set, and a test set; S603. Use the training set, the validation set, and the test set to perform supervised training, validation, and testing on the skin problem type recognition model to obtain a skin problem type recognition model with an accuracy rate meeting the preset accuracy rate requirements.
[0029] The formula for calculating the matching degree between the real-time skin problem feature data and the skin problem feature data of all historical users is as follows:
[0030] In the formula, is the similarity of the i-th feature between the real-time skin problem feature data and the skin problem feature data of historical users, is the j-th feature index value of the i-th feature of the real-time skin problem feature data, is the j-th feature index value of the i-th feature of the skin problem feature data of historical users, is the total number of the i-th feature index values.
[0031] Sending the personalized treatment plan to the user in an encrypted form specifically includes the following steps: Encode and encrypt the personalized treatment plan and send it to the user's mobile application; The mobile application encodes and decodes the information received by the user, making it display as garbled characters, and transmits it to the storage space for storage; When the user needs to consult the information, by performing the set operation through the mobile application, the decoding function is awakened, the information is decoded, and the information is displayed normally. If the set operation is not performed, the information is displayed as garbled characters.
[0032] In the second aspect of the present invention, a skin improvement system based on image processing technology is further provided, including: A preprocessing module, which is used for the user to upload the captured real-time skin photo in the mobile application, and the mobile application preprocesses the real-time skin photo by using image processing technology. The preprocessing includes image denoising and image enhancement; A detection module, which is used to detect the real-time skin photo based on the VGG16 network and the BLSTM network to obtain the real-time skin problem area; A feature extraction module, which is used to extract features from the detected real-time skin problem area based on the BLSTM layer to obtain the real-time skin problem feature data. The real-time skin problem feature data includes acne, freckles, blackheads, wrinkles, etc.; A model construction module, which is used to construct a skin problem type recognition model based on the neural network model and retrieve all historical users' skin problem feature data and the corresponding skin type data of the users from the database; A calculation module, which is used to calculate the matching degree between the real-time skin problem feature data and the skin problem feature data of all historical users based on the skin problem type recognition model to determine the user's skin problem type; A sending module, which is configured to formulate a personalized treatment plan for a user according to the type of the user's skin problem, and send the personalized treatment plan to the user in an encrypted form.
[0033] In a third aspect of the present invention, an electronic device is further provided.
[0034] Figure 7 FIG. shows a schematic block diagram of an electronic device 700 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.
[0035] The electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0036] A plurality of components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0037] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 executes the various methods and processes described above, such as methods S100 to S600. For example, in some embodiments, methods S101 to S106 can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of methods S101 to S106 described above can be executed. Alternatively, in other embodiments, the computing unit 701 can be configured to execute methods S101 to S106 by any other suitable means (e.g., by means of firmware).
[0038] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.
[0039] The program code for implementing the methods of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0040] In the context of the present invention, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0041] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).
[0042] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0043] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
[0044] The working principle and usage process of this device: Do not write the name The above has shown and described the basic principle, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.
Claims
1. A skin improvement method based on image processing technology, characterized in that: include: The user uploads the real-time skin photo taken in the mobile application, and the mobile application uses image processing technology to pre-process the real-time skin photo, wherein the pre-processing includes image denoising and image enhancement; Detect real-time skin photos based on VGG16 network and BLSTM network to obtain real-time skin problem areas; Based on the BLSTM layer, feature extraction is performed on the detected real-time skin problem area to obtain real-time skin problem feature data, where the real-time skin problem feature data includes acne, spots, blackheads, wrinkles, etc.; Based on the neural network model, all historical user skin problem feature data and user's corresponding skin type data are retrieved from the database to build a skin problem type recognition model; Based on the skin problem type recognition model, the real-time skin problem feature data and all historical user skin problem feature data are matched to determine the user's skin problem type; According to the user's skin problem type, a personalized treatment plan is formulated for the user, and the personalized treatment plan is sent to the user in encrypted form.
2. A skin improvement method based on image processing technology according to claim 1, characterized in that: The image denoising specifically comprises the following steps: Adaptive wavelet threshold denoising for real-time skin photos; By constructing a probability model of wavelet coefficients, the optimal wavelet threshold is determined using Bayesian estimation. The wavelet coefficients are processed with adaptive threshold and then reconstructed by wavelet to obtain a denoised image.
3. A skin improvement method based on image processing technology according to claim 2, characterized in that: The image enhancement specifically comprises the following steps: Performing adaptive histogram equalization processing on the denoised image; Determine the cumulative distribution function of the local area by calculating the grayscale histogram of the denoised image; Based on the cumulative distribution function, the grayscale mapping relationship of the local area is adaptively adjusted to obtain an enhanced image.
4. The skin improvement method based on image processing technology according to claim 3, characterized in that: The method of detecting the real-time skin photo based on the VGG16 network and the BLSTM network to obtain the real-time skin problem area specifically includes the following steps: Input the enhanced skin photo into the VGG16 network; Get the mapping of the convolutional layer in the VGG16 network and get the features of size W×H×C, where W is the image width, H is the image height, and C is the number of image channels; Extracting feature vectors from the convolutional layer using a sliding window; The extracted feature vector is input into the BLSTM network for processing to obtain the real-time skin problem area.
5. The skin improvement method based on image processing technology according to claim 4, characterized in that: The method of extracting features of the detected real-time skin problem area based on the BLSTM layer to obtain real-time skin problem feature data specifically includes the following steps: Input the real-time skin problem area into the VGG16 network to generate several convolution feature matrices; Each convolution feature matrix corresponds to a rectangular area, and the convolution feature matrix is input into the BLSTM layer; Set the maximum time length of the BLSTM layer, and classify the results of the BLSTM output using the softmax function to obtain data such as acne, spots, blackheads, and wrinkles.
6. The skin improvement method based on image processing technology according to claim 5, characterized in that: The neural network model is based on the skin problem feature data of all users and the skin type data corresponding to the users, and the skin problem type recognition model is constructed by retrieving the skin problem feature data and the skin type data corresponding to the users from the database, and the steps are as follows: Based on a feedforward neural network, a skin problem type recognition model is constructed, wherein the input data of the skin problem type recognition model includes user skin problem feature data information; Based on machine learning, the skin problem characteristic data information of the plurality of users is labeled and divided to obtain a training set, a validation set and a test set; The training set, validation set and test set are used to perform supervised training, validation and testing on the skin problem type recognition model to obtain the skin problem type recognition model whose accuracy meets the preset accuracy requirements.
7. The skin improvement method based on image processing technology according to claim 6, characterized in that: The formula for calculating the matching degree between the real-time skin problem feature data and the historical skin problem feature data of all users is: ; In the formula, is the similarity between the i-th feature of the real-time skin problem feature data and the historical user skin problem feature data, is the jth feature index value of the i-th feature of the real-time skin problem feature data, is the jth feature index value of the i-th feature of the historical user skin problem feature data, is the total number of characteristic index values of the ith feature.
8. The skin improvement method based on image processing technology according to claim 7, characterized in that: The step of sending the personalized treatment plan to the user in an encrypted form specifically includes the following steps: Encode and encrypt the personalized treatment plan and send it to the user's mobile application; The mobile application encodes and decodes the information received by the user, making it appear as garbled text, and transmits it to the storage space for storage; When the user needs to check the information, he performs the set operation through the mobile application to wake up the decoding function, decode the information, and display the information normally. If the set operation is not performed, the information will be displayed in garbled characters.
9. A skin improvement system based on image processing technology, used to implement a skin improvement method based on image processing technology as claimed in any one of claims 1 to 8, characterized in that: include: A preprocessing module, wherein the preprocessing module is used for users to upload real-time skin photos taken in a mobile application, and the mobile application uses image processing technology to preprocess the real-time skin photos, and the preprocessing includes image denoising and image enhancement; A detection module, which is used to detect real-time skin photos based on a VGG16 network and a BLSTM network to obtain real-time skin problem areas; A feature extraction module, which is used to extract features of the detected real-time skin problem area based on the BLSTM layer to obtain real-time skin problem feature data, wherein the real-time skin problem feature data includes acne, spots, blackheads, wrinkles, etc.; A model building module, which is used to build a skin problem type recognition model based on a neural network model and retrieve all historical user skin problem feature data and user corresponding skin type data from a database; A calculation module, the calculation module is used to calculate the matching degree between the real-time skin problem feature data and the skin problem feature data of all historical users based on the skin problem type recognition model to determine the skin problem type of the user; The sending module is used to formulate a personalized treatment plan for the user according to the type of skin problem of the user, and send the personalized treatment plan to the user in an encrypted form.
10. An electronic device comprising at least one processor; and a memory connected in communication with the at least one processor; characterized in that: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.