Nail color supplementing method and system, and nail color supplementing apparatus
This nail color correction method, which combines multi-angle photography and deep learning models with convolutional networks, solves the problem of labor-intensive and inaccurate nail color correction in existing technologies. It achieves automated and precise nail color correction and cleaning, improving the color correction effect and efficiency.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- CHAN PO IN
- Filing Date
- 2025-06-04
- Publication Date
- 2026-07-23
AI Technical Summary
Existing nail touch-up methods are labor-intensive and inaccurate, making it difficult to accurately detect missing nail areas and perform precise touch-up in two-dimensional image detection. Furthermore, the nail industry lacks automated cleaning devices.
The system acquires nail images from multiple angles, locates missing areas using a deep learning-based missing area detection model, extracts boundary and color features using a convolutional network for color correction, and achieves accurate color correction through 3D coordinate mapping. It is also equipped with an automatic cleaner for cleaning.
It achieves automated and precise nail touch-up and cleaning, reduces manual intervention, improves the accuracy and efficiency of touch-up, and ensures color consistency at fine boundaries and in three-dimensional positions.
Smart Images
Figure CN2025099104_23072026_PF_FP_ABST
Abstract
Description
A method, system, and device for nail color touch-up Technical Field
[0001] This invention relates to the field of computer technology, and more specifically, to a method, system, and device for nail color touch-up. Background Technology
[0002] Currently, after getting a manicure, nail polish often chips and fades due to daily activities, resulting in missing pieces of the nail design. In such cases, manual repainting is typically used for touch-ups. However, this is a repetitive and tedious task, too labor-intensive. Therefore, automated nail touch-ups are needed. However, due to the small size of nails, accurate detection of missing areas is difficult, and inaccurate boundary positioning results in small areas not being painted during the application process. Furthermore, because the detection relies on a two-dimensional image, applying a two-dimensional image-based touch-up method also leads to inaccurate coloring.
[0003] Furthermore, in the current nail industry, nail cleaning is primarily done manually, such as removing dirt and discarding unwanted nail designs. Therefore, a device that can automatically clean nails is needed. Summary of the Invention
[0004] The purpose of this invention is to provide a nail touch-up method, system, and device to solve the aforementioned problems existing in the prior art.
[0005] In a first aspect, embodiments of the present invention provide a method for nail color restoration, comprising:
[0006] Obtain a set of nail images; the set of nail images includes a frontal nail image, a left-side nail image, and a right-side nail image; the frontal nail image, the left-side nail image, and the right-side nail image are images of nails that include missing parts;
[0007] Based on the nail image set, the missing nail area is located using a missing nail detection model to obtain the missing nail region; the missing nail region represents the area where the nail has lost its color.
[0008] Based on the nail image set, the missing patterns are obtained; the missing patterns represent the patterns in the missing areas.
[0009] Control the nail color filler to fill in the missing color in the missing area.
[0010] Optionally, the step of locating nail defects based on the nail image set using a defect detection model to obtain the defect region includes:
[0011] The frontal fingernail image is input into the missing area detection model to obtain the frontal missing area region;
[0012] The left-side fingernail image is input into the missing area detection model to obtain the left-side missing area;
[0013] The right-side nail image is input into the missing area detection model to obtain the right-side missing area;
[0014] Based on the front missing area, the left side missing area, the right side missing area, and the nail image group, the missing area is obtained.
[0015] Optionally, the training method for the missing block detection model includes:
[0016] Obtain a training set; the training set includes multiple training images and labeled data; the labeled data includes labeled nail region images and labeled boundary positions; the labeled nail region images represent images where the missing nail area is covered by the same color; the labeled boundary positions represent the positions of the boundary between the missing nail area and the non-missing nail area;
[0017] Based on the training images, training segmentation information is obtained through an image segmentation network; the training segmentation information includes boundary position images, initial training nail-missing images, and training nail-missing regions.
[0018] The initial training image with missing nails and the image with labeled nail regions are used to obtain the initial training nail loss value through the cross-entropy loss function;
[0019] The boundary location image and the labeled boundary locations are used to obtain the boundary loss value through the cross-entropy loss function;
[0020] The training image with missing nails and the image with labeled nail regions are used to obtain the training loss value for missing nails through the cross-entropy loss function;
[0021] Obtain the total loss value; the total loss value is the sum of the initial training nail loss value, the boundary loss value, and the training nail loss value.
[0022] Obtain the current number of training iterations of the missing block detection model and the preset maximum number of training iterations of the missing block detection model;
[0023] Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained missing block detection model is obtained.
[0024] Optionally, obtaining training segmentation information based on the training images through an image segmentation network includes:
[0025] The training images are input into the backbone network to obtain the backbone feature map;
[0026] The backbone feature map is convolved through the first convolutional network to extract boundary features and obtain the boundary feature map.
[0027] The backbone feature map is convolved through a second convolutional network to perform image semantic segmentation, resulting in a segmentation feature map.
[0028] The boundary feature map is upsampled through deconvolution to reach the original image size, resulting in a boundary location image; the value in the boundary location image is the predicted existence of the boundary.
[0029] The segmentation feature map is upsampled through deconvolution to reach the original image size, resulting in an initial training image of missing nails; the initial training image of missing nails represents the predicted missing nail region.
[0030] Using the outermost layer of the nail region in the upper convolution boundary map and the upper convolution segmentation map as the boundary, we obtain the training image of missing nails.
[0031] Optionally, the step of obtaining the missing region based on the front missing region, the left side missing region, the right side missing region, and the nail image group includes:
[0032] Construct a three-dimensional coordinate system based on the nail image group; use the length of the frontal nail image as the x-axis and the length of the frontal nail image as the y-axis of the three-dimensional coordinate system, and use the width of the left side nail image as the x-axis of the three-dimensional coordinate system.
[0033] The position that exists in both the front missing area and the left missing area is taken as the three-dimensional position of the left missing area; the left missing area is obtained by taking the x-axis and y-axis coordinates from the front missing area and the z-axis coordinate from the left missing area.
[0034] The position that exists in both the front missing area and the right missing area is taken as the three-dimensional position of the right missing area; the three-dimensional position of the right missing area is obtained with the x-axis and y-axis coordinates of the front missing area and the z-axis coordinate of the right missing area.
[0035] The missing areas are obtained by traversing the front missing area, the left missing area, and the right missing area; the missing area is a region composed of multiple three-dimensional missing positions on the left and multiple three-dimensional missing positions on the right.
[0036] Optionally, obtaining the missing pattern based on the nail image set includes:
[0037] The color of the nail image group is detected, and clustering and classification are performed to obtain multiple color groups.
[0038] Based on the multiple color classification groups, a color histogram is obtained;
[0039] The color groups whose values in the color histogram are greater than those of other color groups are designated as color regions.
[0040] The target color corresponding to the color within the color region is found in the database and is taken as the missing suit.
[0041] Optionally, the method further includes:
[0042] Control the cleaner to clean your nails.
[0043] Secondly, embodiments of the present invention provide a nail touch-up device, comprising: a nail touch-up protective housing (1), a supporting base plate (2), a left-side camera (401), a right-side camera (402), a front-side camera (502), a nail touch-up device (501), a nail holder (301), a cleaner (504), and a processor, wherein the processor is connected to the left-side camera (401), the right-side camera (402), the front-side camera (502), and the nail touch-up device (501).
[0044] A nail support plate (302) is fixed on the support base plate (2), and a nail holder (301) is fixed on the nail support plate (302). A left-side camera (401) is fixed on the left side of the nail support plate (302), and a right-side camera (402) is fixed on the right side of the nail support plate (302). A movable rod (503) is fixed at the top of the nail color protection box. The movable rod (503) connects the front camera (502), the cleaner (504), and the nail color protector (501). The processor, the left-side camera (401), the right-side camera (402), the front camera (502), and the nail color protector (501) are fixed inside the nail color protection box (1).
[0045] The processor is configured to: obtain a nail image set; the nail image set includes a frontal nail image, a left-side nail image, and a right-side nail image; the frontal nail image, the left-side nail image, and the right-side nail image are images of nails containing missing parts; based on the nail image set, perform nail missing part localization using a missing part detection model to obtain a missing part region; the missing part region represents the area where the nail has lost color; based on the nail image set, obtain the missing pattern; the missing pattern represents the pattern of the missing part region; control the nail color filler (501) to fill the nail with color according to the missing pattern in the missing part region;
[0046] The front-facing camera (502) is used to obtain a front-facing nail image; the left-facing camera (401) is used to obtain a left-facing nail image; and the right-facing camera (402) is used to obtain a right-facing nail image.
[0047] The processor is also used to control the cleaner (504) to clean the nails.
[0048] Thirdly, embodiments of the present invention provide a nail color restoration system, comprising:
[0049] Acquisition module: Acquires a group of nail images; the group of nail images includes a frontal nail image, a left-side nail image, and a right-side nail image; the frontal nail image, left-side nail image, and right-side nail image are images of nails containing missing parts;
[0050] Missing nail detection module: Based on the nail image group, the missing nail is located using the missing nail detection model to obtain the missing nail region; the missing nail region represents the area where the nail has lost color;
[0051] Missing Pattern Module: Based on the nail image group, the missing pattern is obtained; the missing pattern represents the pattern of the missing area;
[0052] Color correction module: Controls the nail color correction device to correct the missing color in the missing area of the nail.
[0053] Optionally, the step of locating nail defects based on the nail image set using a defect detection model to obtain the defect region includes:
[0054] The frontal fingernail image is input into the missing area detection model to obtain the frontal missing area region;
[0055] The left-side fingernail image is input into the missing area detection model to obtain the left-side missing area;
[0056] The right-side nail image is input into the missing area detection model to obtain the right-side missing area;
[0057] Based on the front missing area, the left side missing area, the right side missing area, and the nail image group, the missing area is obtained.
[0058] Optionally, the training method for the missing block detection model includes:
[0059] Obtain a training set; the training set includes multiple training images and labeled data; the labeled data includes labeled nail region images and labeled boundary positions; the labeled nail region images represent images where the missing nail area is covered by the same color; the labeled boundary positions represent the positions of the boundary between the missing nail area and the non-missing nail area;
[0060] Based on the training images, training segmentation information is obtained through an image segmentation network; the training segmentation information includes boundary position images, initial training nail-missing images, and training nail-missing regions.
[0061] The initial training image with missing nails and the image with labeled nail regions are used to obtain the initial training nail loss value through the cross-entropy loss function;
[0062] The boundary location image and the labeled boundary locations are used to obtain the boundary loss value through the cross-entropy loss function;
[0063] The training image with missing nails and the image with labeled nail regions are used to obtain the training loss value for missing nails through the cross-entropy loss function;
[0064] Obtain the total loss value; the total loss value is the sum of the initial training nail loss value, the boundary loss value, and the training nail loss value.
[0065] Obtain the current number of training iterations of the missing block detection model and the preset maximum number of training iterations of the missing block detection model;
[0066] Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained missing block detection model is obtained.
[0067] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:
[0068] This invention also provides a nail color restoration method, system, and device. The method includes: obtaining a nail image set, which includes a frontal nail image, a left-side nail image, and a right-side nail image. The frontal, left-side, and right-side nail images are images of nails containing missing areas. Based on the nail image set, a missing area is located using a missing area detection model to obtain the missing area. The missing area represents the region where the nail has lost color. Based on the nail image set, the missing pattern is obtained. The missing pattern represents the pattern of the missing area. A nail color restorer is controlled to restore color to the nail in the missing area according to the missing pattern.
[0069] A detection method is used to identify the discolored missing areas. A convolutional network is trained to obtain the missing portion and the crack boundary between it and the colored nail. During convolution, the same backbone network is used, but different branch networks are employed to obtain the boundaries of the missing and colored portions simultaneously with the nail region. The boundary position image and the initial training image of the missing nail are jointly used for judgment. The outermost layer of the nail region in the upper convolution boundary map and the upper convolution segmentation map is used as the boundary. The outermost boundary also allows for better blending of boundary colors. During the coloring process, fine boundaries are blended and colored. Convolution is used to obtain the crack boundary because if only pixel-level judgment is used to obtain pixel abrupt change positions, color abrupt changes may also occur. Therefore, texture abrupt changes are used to obtain the crack boundary. The front and side views are merged to obtain the three-dimensional position, allowing for accurate color filling based on the three-dimensional region. Attached Figure Description
[0070] Figure 1 is a flowchart of a nail touch-up method provided by an embodiment of the present invention.
[0071] Figure 2 is a schematic diagram of the missing block detection model in a nail color correction method provided in an embodiment of the present invention.
[0072] Figure 3 is a schematic diagram of a nail color touch-up method device provided in an embodiment of the present invention.
[0073] Figure 4 is a block structure diagram of an electronic device provided in an embodiment of the present invention.
[0074] The diagram shows: 1. Nail touch-up protective box; 2. Support base plate; 301. Nail holder; 302. Nail support plate; 401. Left-side camera; 402. Right-side camera; 501. Nail touch-up device; 502. Front camera; 503. Movable rod; 504. Cleaner; 600. Bus; 601. Receiver; 602. Processor; 603. Transmitter; 604. Memory; 605. Bus interface. Detailed implementation method.
[0075] The present invention will now be described in detail with reference to the accompanying drawings.
[0076] Example 1
[0077] As shown in Figure 1, this embodiment of the invention provides a method for nail touch-up, the method comprising:
[0078] S101: Obtain a nail image group. The nail image group includes a frontal nail image, a left-side nail image, and a right-side nail image. The frontal nail image, left-side nail image, and right-side nail image are images of nails containing missing parts.
[0079] The frontal nail image refers to the image of the nail taken by the frontal camera, the left-side nail image refers to the image of the nail taken by the left-side camera, and the right-side nail image refers to the image of the nail taken by the right-side camera.
[0080] S102: Based on the nail image set, the missing nail area is located using a missing nail detection model to obtain the missing nail region. The missing nail region represents the area where the nail has lost its color.
[0081] The structural diagram of the missing block detection model is shown in Figure 2.
[0082] S103: Based on the nail image group, obtain the missing pattern. The missing pattern represents the pattern of the missing area.
[0083] S104: Controls the nail color filler to fill in the missing color in the missing area of the nail.
[0084] Optionally, the step of locating nail defects based on the nail image set using a defect detection model to obtain the defect region includes:
[0085] The frontal nail image is input into the missing area detection model to obtain the frontal missing area.
[0086] The process of obtaining the frontal missing area from the frontal nail image input missing area detection model is equivalent to the process of training the nail missing area using training images during training.
[0087] The left-side nail image is input into the missing area detection model to obtain the missing area on the left side.
[0088] The process of obtaining the left-side missing area from the missing nail image input into the missing nail detection model is equivalent to the process of training the missing nail area using the training image during training.
[0089] The right-side nail image is input into the missing area detection model to obtain the missing area on the right side.
[0090] The process of obtaining the right-side missing area from the right-side nail image input into the missing area detection model is equivalent to the process of training the nail missing area using the training image during training.
[0091] Based on the front missing area, the left side missing area, the right side missing area, and the nail image group, the missing area is obtained.
[0092] The above method fails to accurately color the nail due to its curved shape if only a flat plane is used. Therefore, using images from multiple angles to find 3D patches for coloring allows for more accurate and comprehensive nail coloring.
[0093] Optionally, the training method for the missing block detection model includes:
[0094] Obtain the training set. The training set includes multiple training images and labeled data. The labeled data includes labeled nail region images and labeled boundary positions. The labeled nail region images represent images where the missing nail area is covered by the same color. The labeled boundary positions represent the positions of the boundaries between the missing nail area and the non-missing nail area.
[0095] In this embodiment, the missing nail area in the marked nail region image is covered in red.
[0096] The location of the marked boundary is also recorded in color in the image of the nail area. In this embodiment, the marked boundary is recorded in black.
[0097] Based on the training images, training segmentation information is obtained through an image segmentation network. The training segmentation information includes boundary location images, initial training images of missing nails, and training images of missing nail regions.
[0098] The initial training image with missing nails and the image with labeled nail regions are used to obtain the initial training nail loss value through the cross-entropy loss function.
[0099] The boundary location image and labeled boundary locations are used to obtain the boundary loss value through the cross-entropy loss function.
[0100] The training image with missing nails and the image with labeled nail regions are used to obtain the training loss value for missing nails through the cross-entropy loss function.
[0101] Obtain the total loss value. The total loss value is the sum of the initial training nail loss value, the boundary loss value, and the training nail loss value.
[0102] Obtain the current number of training iterations of the missing block detection model and the pre-set maximum number of training iterations of the missing block detection model.
[0103] The maximum number of iterations for training the pre-set missing block detection model is 12,000.
[0104] Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained missing block detection model is obtained.
[0105] Using the above method, a missing nail detection model is used to separate the faded and unfaded parts of the nail, simultaneously determining the boundary positions. Unlike traditional boundary detection methods, deep learning-based detection methods do not require manual intervention. They can autonomously learn representative features and use features extracted through convolution of segmented regions to detect boundaries in the input image. The boundary positions can be more accurately determined, allowing for more detailed coloring of the boundary locations. Performing dual detection on the missing nail area using the boundary alone improves the accuracy of boundary location, thus enhancing the detection of missing nail areas. Furthermore, the network is trained using a loss function applied to the affected missing nail area, enabling the trained network to directly determine the location of the missing nail.
[0106] Optionally, obtaining training segmentation information based on the training images through an image segmentation network includes:
[0107] In this embodiment, the image segmentation network is an FCN network (Fully Convolutional Networks for Semantic Segmentation).
[0108] The training images are input into the backbone network to obtain the backbone feature map.
[0109] In this embodiment, the backbone network is the backbone network of the FCN network.
[0110] The backbone feature map is convolved through a first convolutional network to extract boundary features, thus obtaining a boundary feature map.
[0111] The first convolutional network that convolves the backbone feature map consists of multiple residual networks. The size of the boundary feature map is 7x7x2. Here, 7 (px) x 7 (px) represents the size of the boundary feature map, and 2 indicates that the boundary feature map is categorized as either a boundary or not.
[0112] The backbone feature map is convolved through a second convolutional network to perform image semantic segmentation, resulting in a segmentation feature map.
[0113] The second convolutional network that convolves the backbone feature map consists of multiple residual networks. The segmentation feature map has a size of 7x7x2. Here, 7 (px) x 7 (px) represents the size of the segmentation feature map, and 2 indicates that the segmentation feature map is categorized as either a nail-missing region or a non-nail-missing region. The 7x7x2 segmentation feature map size is obtained using two 1x1 convolutional kernels.
[0114] The boundary feature map is upsampled through deconvolution to reach the original image size, resulting in an upconvolution boundary map; the upconvolution boundary map shows the predicted boundary positions.
[0115] In this process, two convolutional kernels of the same size as the training image are used for deconvolution to obtain the upper convolution boundary map.
[0116] The segmentation feature map is upsampled through deconvolution to reach the original image size, resulting in an upconvolution segmentation map. The initial training image of the missing nail represents the predicted missing nail region.
[0117] In this process, deconvolution is performed using two convolution kernels of the same size as the training image to obtain the upper convolution segmentation map;
[0118] Using the outermost layer of the nail region in the upper convolution boundary map and the upper convolution segmentation map as the boundary, we obtain the nail missing region.
[0119] If the boundary position in the upper convolution boundary map does not exist in the nail missing region, the boundary in the upper convolution boundary map is used as the boundary of the nail missing region.
[0120] The above method uses a backbone network to extract image features. Then, convolutional layers are applied to the backbone feature map to extract features from the boundary locations and the missing nail region. During training, loss is calculated using labeled data, followed by backward convolution to train the parameters of the backbone network and the convolutional parameters for extracting the boundary locations and the missing nail region. Since the extracted features are different, the parameters are different, thus different first and second convolutional networks are used for feature extraction. Both the boundary locations and the nail region are up-convolved to obtain feature maps of the same size as the original image. The presence of the nail is indicated by predicting the features at each location. A dual detection is performed on the boundary for the missing nail region; separate detection on the boundary improves the accuracy of boundary location, thus making the detection of the missing nail region more accurate.
[0121] Optionally, the step of obtaining the missing region based on the front missing region, the left side missing region, the right side missing region, and the nail image group includes:
[0122] A three-dimensional coordinate system is constructed based on the nail image group. The length of the frontal nail image is used as the x-axis and y-axis of the three-dimensional coordinate system, and the width of the left-side nail image is used as the x-axis.
[0123] The values on the plane formed by the three-dimensional x-axis and z-axis correspond to the values in the left-side nail image. The values on the plane formed by the three-dimensional y-axis and z-axis correspond to the values in the right-side nail image. The values on the plane formed by the three-dimensional x-axis and y-axis correspond to the values in the front-facing nail image.
[0124] The location existing in both the front and left side missing regions is defined as the left-side missing 3D location. The left-side missing 3D location is determined by obtaining the x-axis and y-axis coordinates from the front missing region and the z-axis coordinate from the left-side missing region.
[0125] The location existing in both the front and right side missing regions is defined as the three-dimensional location of the right-side missing area. The x-axis and y-axis coordinates of this right-side missing area are derived from the front missing region, and the z-axis coordinate is derived from the right-side missing region.
[0126] The missing regions are obtained by traversing the front, left, and right missing regions. Each missing region consists of multiple three-dimensional missing locations on the left and right sides.
[0127] The above method produces a three-dimensional nail image, but the captured image is two-dimensional. While the entire length is visible from the front, the actual position shows curvature. The positions detected in the two-dimensional image are insufficient for accurate color correction based on three-dimensional regions. Establishing three-dimensional coordinates maps the positions of the missing areas on the front, left, and right sides onto the 3D nail, thus obtaining the missing areas. This allows for more accurate color application.
[0128] Optionally, obtaining the missing pattern based on the nail image and the missing area includes:
[0129] The color of the nail image group is detected, and clustering and classification are performed to obtain multiple color groups.
[0130] Clustering is performed using the K-means clustering algorithm.
[0131] Based on the multiple color classification groups, a color histogram is obtained.
[0132] The color groups whose values in the color histogram are greater than those of other color groups are designated as color regions.
[0133] The target color corresponding to the color within the color region is found in the database and is taken as the missing suit.
[0134] If a color exists in one color region in the database, that color is the target color. If multiple color regions exist in the database, a traversal and counting method is used to find the color that appears most frequently in each region. This most frequent color in the region is then used as the target color.
[0135] The database stores nail colors that can be used for color matching.
[0136] The above method addresses the issue that nails may contain other colors on top of their base color. However, nail color correction primarily involves using large areas of color to avoid creating a color discrepancy. Therefore, a clustering method is used to categorize colors, grouping them according to similarity to obtain multiple color groups. A histogram is then used to determine which group has the most colors. The colors from that group are then used as the base color in the database, which is used as the missing pattern. This ensures accurate color selection for the user.
[0137] Using the above method, since nail polish colors are not fixed, and the texture of the original nail polish that has lost color is similar and easy to detect, a detection method is adopted to detect the missing areas. A convolutional network is trained to obtain the missing parts and the crack boundaries between them and the colored nail. During convolution, the nail region and the crack are obtained simultaneously, and the boundary image and the missing image are jointly judged. Convolution is used to obtain the crack boundaries because if only pixel-level judgment is used to obtain pixel abrupt change locations, there may be abrupt changes in the color as well; therefore, texture abrupt changes are used to obtain the crack boundaries. The outermost layer of the nail region in the upper convolution boundary map and the upper convolution segmentation map is used as the boundary. The outermost boundary also allows for better blending of boundary colors, and fine boundaries are colored during the coloring process. The front and side fusion parts are 3D positioned for accurate color correction.
[0138] The method also includes controlling the cleaner to clean the nails.
[0139] Example 2
[0140] Based on the above-described nail touch-up method, this embodiment of the invention also provides a nail touch-up device. The device includes a nail touch-up protective housing (1), a support base plate (2), a left-side camera (401), a right-side camera (402), a front-side camera (502), a nail touch-up device (501), and a nail holder (301). It also includes a processor connected to the left-side camera (401), right-side camera (402), front-side camera (502), and nail touch-up device (501).
[0141] A nail support plate (302) is fixed on the support base plate (2), and a nail holder (301) is fixed on the nail support plate (302). A left-side camera (401) is fixed on the left side of the nail support plate (302), and a right-side camera (402) is fixed on the right side of the nail support plate (302). A movable rod (503) is fixed at the top of the nail touch-up protection box. The movable rod (503) connects the front camera (502), the cleaner (504), and the nail touch-up device (501). The processor is fixed inside the nail touch-up protection box (1) along with the left-side camera (401), the right-side camera (402), the front camera (502), and the nail touch-up device (501).
[0142] The processor is configured to: obtain a nail image set; the nail image set includes a frontal nail image, a left-side nail image, and a right-side nail image; the frontal nail image, left-side nail image, and right-side nail image are images of nails containing missing parts; based on the nail image set, perform nail missing part localization using a missing part detection model to obtain the missing part region; the missing part region represents the area where the nail has lost color; based on the nail image set, obtain the missing pattern; the missing pattern represents the pattern of the missing part region; and control a nail color filler to fill in the nail in the missing part region according to the missing pattern.
[0143] The front-facing camera (502) is used to obtain a front-facing nail image; the left-facing camera (401) is used to obtain a left-facing nail image; and the right-facing camera (402) is used to obtain a right-facing nail image.
[0144] The nail is placed on a nail support plate (302) for support. A nail fixator (301) is used to fix the nail, ensuring accurate detection of the same location for both missing and complementary color correction. The position of the front camera (502) is adjusted using a movable lever (503) until it corresponds to the area above the nail. Images from three different directions are obtained using the left camera (401), right camera (402), and front camera (502). The obtained images are transmitted to a computer, where a missing location is detected using a missing area detection model, enabling the detection of three-dimensional positions and thus improving the accuracy of complementary color correction. The front camera (502) is moved away from the nail using the movable lever (503), and the nail color corrector (501) is adjusted so that it performs complementary color correction based on the calculated location of the missing area.
[0145] The processor is also connected to the cleaner (504) and is also used to: control the cleaner (504) to clean the nails.
[0146] In this embodiment of the invention, the cleaner contains nail cleaning solution, which can be sprayed onto the nails and then rubbed directly to clean the nails.
[0147] Example 3
[0148] Based on the above-described nail color restoration method, this invention also provides a nail color restoration system, which includes an acquisition module, a missing patch detection module, a missing pattern module, and a color restoration control module.
[0149] The acquisition module is used to obtain a group of nail images. The nail image group includes a frontal nail image, a left-side nail image, and a right-side nail image. The frontal nail image, left-side nail image, and right-side nail image are images of the nail including the missing parts.
[0150] The missing nail detection module is used to locate missing nail areas based on the nail image set using a missing nail detection model, thereby obtaining the missing nail regions. The missing nail regions represent the areas where the nail has lost its color.
[0151] The missing pattern module is used to obtain the missing pattern based on the nail image group. The missing pattern represents the pattern of the missing area.
[0152] The color-correction control module is used to control the nail color corrector to correct the missing color in the missing area of the nail.
[0153] The specific manner in which each module performs its operations in the system described in the above embodiments has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0154] This invention also provides an electronic device, as shown in FIG4, including a memory 604, a processor 602, and a computer program stored in the memory 604 and executable on the processor 602. When the processor 602 executes the program, it implements the steps of any of the nail touch-up methods described above.
[0155] In Figure 4, a bus architecture (represented by bus 600) is shown. Bus 600 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 602 and memory represented by memory 604. Bus 600 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 605 provides an interface between bus 600 and receiver 601 and transmitter 603. Receiver 601 and transmitter 603 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 602 is responsible for managing bus 600 and general processing, while memory 604 can be used to store data used by processor 602 during operation. Embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods of the nail touch-up method described above, as well as the data involved above.
[0156] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of the invention.
[0157] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0158] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.
[0159] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0160] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
[0161] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0162] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.
Claims
1. A method for touch-up nail color, characterized in that, include: Obtain a set of nail images; the set of nail images includes a frontal nail image, a left-side nail image, and a right-side nail image; The frontal, left-side, and right-side nail images are images of nails that include the missing parts; Based on the nail image set, the missing nail area is located using a missing nail detection model to obtain the missing nail region; the missing nail region represents the area where the nail has lost its color. Based on the nail image set, the missing patterns are obtained; the missing patterns represent the patterns in the missing areas. Control the nail color filler to fill in the missing color in the missing area.
2. A method for touch-up nail color according to claim 1, characterized in that, Based on the nail image set, the missing nail region is located using a missing nail detection model, including: The frontal fingernail image is input into the missing area detection model to obtain the frontal missing area region; The left-side fingernail image is input into the missing area detection model to obtain the left-side missing area; The right-side nail image is input into the missing area detection model to obtain the right-side missing area; Based on the front missing area, the left side missing area, the right side missing area, and the nail image group, the missing area is obtained.
3. A method for touch-up nail color according to claim 1, characterized in that, The training method for the missing block detection model includes: Obtain a training set; the training set includes multiple training images and labeled data; the labeled data includes labeled nail region images and labeled boundary positions; the labeled nail region images represent images where the missing nail area is covered by the same color; the labeled boundary positions represent the positions of the boundary between the missing nail area and the non-missing nail area; Based on the training images, training segmentation information is obtained through an image segmentation network; the training segmentation information includes boundary position images, initial training nail-missing images, and training nail-missing regions. The initial training image with missing nails and the image with labeled nail regions are used to obtain the initial training nail loss value through the cross-entropy loss function; The boundary location image and the labeled boundary locations are used to obtain the boundary loss value through the cross-entropy loss function; The training image with missing nails and the image with labeled nail regions are used to obtain the training loss value for missing nails through the cross-entropy loss function; Obtain the total loss value; the total loss value is the sum of the initial training nail loss value, the boundary loss value, and the training nail loss value. Obtain the current number of training iterations of the missing block detection model and the preset maximum number of training iterations of the missing block detection model; Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained missing block detection model is obtained.
4. A method for touch-up nail color according to claim 3, characterized in that, The step of obtaining training segmentation information based on the training images through an image segmentation network includes: The training images are input into the backbone network to obtain the backbone feature map; The backbone feature map is convolved through the first convolutional network to extract boundary features and obtain the boundary feature map. The backbone feature map is convolved through a second convolutional network to perform image semantic segmentation, resulting in a segmentation feature map. The boundary feature map is upsampled through deconvolution to reach the original image size, resulting in a boundary location image; the value in the boundary location image is the predicted existence of the boundary. The segmentation feature map is upsampled through deconvolution to reach the original image size, resulting in an initial training image of missing nails; the initial training image of missing nails represents the predicted missing nail region. Using the outermost layer of the nail region in the upper convolution boundary map and the upper convolution segmentation map as the boundary, we obtain the training image of missing nails.
5. A method for touch-up nail color according to claim 2, characterized in that, The missing region is obtained based on the frontal missing region, the left side missing region, the right side missing region, and the nail image group, including: Construct a three-dimensional coordinate system based on the nail image group; use the length of the frontal nail image as the x-axis and the length of the frontal nail image as the y-axis of the three-dimensional coordinate system, and use the width of the left side nail image as the x-axis of the three-dimensional coordinate system. The position that exists in both the front missing area and the left missing area is taken as the three-dimensional position of the left missing area; the left missing area is obtained by taking the x-axis and y-axis coordinates from the front missing area and the z-axis coordinate from the left missing area. The position that exists in both the front missing area and the right missing area is taken as the three-dimensional position of the right missing area; the three-dimensional position of the right missing area is obtained with the x-axis and y-axis coordinates of the front missing area and the z-axis coordinate of the right missing area. The missing areas are obtained by traversing the front missing area, the left missing area, and the right missing area; the missing area is a region composed of multiple three-dimensional missing positions on the left and multiple three-dimensional missing positions on the right.
6. A method for touch-up nail color according to claim 1, characterized in that, The process of obtaining the missing pattern based on the nail image set includes: The color of the nail image group is detected, and clustering and classification are performed to obtain multiple color groups. Based on the multiple color classification groups, a color histogram is obtained; The color groups whose values in the color histogram are greater than those of other color groups are designated as color regions. The target color corresponding to the color within the color region is found in the database and is taken as the missing suit.
7. A nail color touch-up device, characterized in that, The device includes a nail touch-up protective box (1), a support base plate (2), a left-side camera (401), a right-side camera (402), a front-side camera (502), a nail touch-up device (501), a nail holder (301), a cleaner (504), and a processor. The processor is connected to the left-side camera (401), the right-side camera (402), the front-side camera (502), and the nail touch-up device (501). A nail support plate (302) is fixed on the support base plate (2), and a nail holder (301) is fixed on the nail support plate (302). A left-side camera (401) is fixed on the left side of the nail support plate (302), and a right-side camera (402) is fixed on the right side of the nail support plate (302). A movable rod (503) is fixed at the top of the nail touch-up protection box. The movable rod (503) connects the front camera (502), the nail touch-up device (501), and the cleaner (504). The processor, the left-side camera (401), the right-side camera (402), the front camera (502), and the nail touch-up device (501) are fixed inside the nail touch-up protection box (1). The processor is configured to: obtain a nail image set; the nail image set includes a frontal nail image, a left-side nail image, and a right-side nail image; the frontal nail image, the left-side nail image, and the right-side nail image are nail images containing missing parts; based on the nail image set, perform nail missing part localization using a missing part detection model to obtain the missing part region; the missing part region represents the area where the nail has lost color; Based on the nail image set, the missing pattern is obtained; the missing pattern represents the pattern of the missing area; the nail color filler (501) is controlled to fill the nail in the missing area according to the missing pattern; The front-facing camera (502) is used to obtain a front-facing nail image; the left-facing camera (401) is used to obtain a left-facing nail image; and the right-facing camera (402) is used to obtain a right-facing nail image.
8. The nail color touch-up device according to claim 7, characterized in that, The processor is also used to: control the cleaner (504) to clean the nails.
9. A nail color-correcting system, characterized in that, include: Acquisition module: Acquires a group of nail images; the nail image group includes a frontal nail image, a left-side nail image, and a right-side nail image; The frontal, left-side, and right-side nail images are images of nails that include the missing parts; Missing nail detection module: Based on the nail image group, the missing nail is located using the missing nail detection model to obtain the missing nail region; the missing nail region represents the area where the nail has lost color; Missing Pattern Module: Based on the nail image group, the missing pattern is obtained; the missing pattern represents the pattern of the missing area; Color correction module: Controls the nail color correction device to correct the missing color in the missing area of the nail.
10. A nail color-correcting system according to claim 9, characterized in that, The training method for the missing block detection model includes: Obtain a training set; the training set includes multiple training images and labeled data; the labeled data includes labeled nail region images and labeled boundary positions; the labeled nail region images represent images where the missing nail area is covered by the same color; the labeled boundary positions represent the positions of the boundary between the missing nail area and the non-missing nail area; Based on the training images, training segmentation information is obtained through an image segmentation network; the training segmentation information includes boundary position images, initial training nail-missing images, and training nail-missing regions. The initial training image with missing nails and the image with labeled nail regions are used to obtain the initial training nail loss value through the cross-entropy loss function; The boundary location image and the labeled boundary locations are used to obtain the boundary loss value through the cross-entropy loss function; The training image with missing nails and the image with labeled nail regions are used to obtain the training loss value for missing nails through the cross-entropy loss function; Obtain the total loss value; the total loss value is the sum of the initial training nail loss value, the boundary loss value, and the training nail loss value. Obtain the current number of training iterations of the missing block detection model and the preset maximum number of training iterations of the missing block detection model; Training stops when the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, and a trained missing block detection model is obtained.