Nail color complementing method and system and nail color complementing device

Abstract

The invention discloses a nail color complementing method, a nail color complementing system and a nail color complementing device. A nail image group is obtained. Based on the nail image group, through a missing block detection model, nail missing block positioning is carried out, and a missing block area is obtained. And obtaining a missing pattern based on the nail image group. And controlling the nail color complementing device to complement the color of the nail in the missing block area according to the missing color. The boundary position image and the initial training nail missing image are jointly judged, the outermost layer of the nail area in the upper convolution boundary image and the upper convolution integral cut image is used as the boundary, and the boundary of the outermost layer can further enable the boundary color to be better fused. The fracture boundary is obtained by convolution due to the fact that if only pixels are used for judgment to obtain pixel abrupt change positions, the situation of color abrupt change may occur, and texture abrupt change needs to be used for obtaining the fracture boundary. The front face and the side face are fused to obtain a three-dimensional position, so that color complementing is carried out according to a three-dimensional area when color complementing is carried out on the nails, and accurate color complementing is carried out.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a nail color correction method, system and nail color correction device. Background Art

[0002] At present, nail polish often falls off and fades due to daily activities after getting a manicure, resulting in missing pieces of the nail pattern. In such cases, the color is usually supplemented by manual re-coloring. However, since this type of work is repetitive and single, it is too labor-intensive. Therefore, it is necessary to automatically supplement the color of the nails, but because the size of the nails is too small, the missing area cannot be accurately detected, and the inaccurate boundary position will cause small borders to be unpainted during the coloring process. Because the detection is a two-dimensional image, the color is supplemented according to the method of two-dimensional images, which also causes inaccurate coloring.

[0003] In addition, in the current nail art industry, nail cleaning is mainly done manually, for example, dirt on the nails needs to be removed, nail art styles that users no longer like need to be removed, etc. For this reason, a device that can automatically clean nails is needed by people. Summary of the invention

[0004] The object of the present invention is to provide a nail color correction method, system and nail color correction device to solve the above problems existing in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for nail color correction, comprising:

[0006] Obtaining a nail image group; the nail image group includes a front nail image, a left side nail image, and a right side nail image; the front nail image, the left side nail image, and the right side nail image are images of the nail containing the missing block portion;

[0007] Based on the nail image group, the missing block of the nail is located by using a missing block detection model to obtain a missing block area; the missing block area indicates a discolored area of ​​the nail;

[0008] Based on the nail image group, a missing pattern is obtained; the missing pattern indicates the pattern of the missing area;

[0009] Control the nail color complementer to complement the nail color in the missing area according to the missing color.

[0010] Optionally, the locating the missing blocks of the nails by using a missing block detection model based on the nail image group to obtain the missing block area includes:

[0011] Inputting the front nail image into a missing block detection model to obtain a front missing block area;

[0012] Input the left side nail image into the missing block detection model to obtain the left side missing block area;

[0013] Inputting the right side nail image into a missing block detection model to obtain a right side missing block area;

[0014] Based on the front missing block area, the left missing block area, the right missing block area and the nail image group, a missing block area is obtained.

[0015] Optionally, the training method of the missing block detection model includes:

[0016] A training set is obtained; the training set includes a plurality of training images and annotated data; the annotated data includes an annotated nail region image and an annotated boundary position; the annotated nail region image represents an image in which a nail missing region is covered with the same color; the annotated boundary position represents the position of the boundary between the nail missing region and the nail non-missing region;

[0017] Based on the training image, obtaining training segmentation information through an image segmentation network; the training segmentation information includes a boundary position image, an initial training nail missing image, and a training nail missing area;

[0018] The initial training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the initial training nail missing loss value;

[0019] The boundary position image and the annotated boundary position are passed through the cross entropy loss function to obtain the boundary loss value;

[0020] The training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the training nail missing loss value;

[0021] Obtain a total loss value; the total loss value is the sum of the initial training nail missing loss value, the boundary loss value and the training nail missing loss value;

[0022] Obtaining the current training iteration number of the missing block detection model and the preset maximum iteration number of the missing block detection model training;

[0023] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained missing block detection model.

[0024] Optionally, obtaining training segmentation information based on the training image through an image segmentation network includes:

[0025] Inputting the training image into the backbone network to obtain a backbone feature map;

[0026] Convolving the backbone feature map through a first convolutional network to extract boundary features and obtain a boundary feature map;

[0027] The backbone feature map is convolved through a second convolutional network to perform image semantic segmentation to obtain a segmentation feature map;

[0028] The boundary feature map is upsampled by deconvolution to reach the original image size, and a boundary position image is obtained; the median value of the boundary position image is the predicted boundary existence;

[0029] The segmentation feature map is upsampled by deconvolution to reach the original image size, thereby obtaining an initial training nail missing image; the initial training nail missing image represents the predicted nail missing area;

[0030] The outermost layer of the nail area in the up-convolution boundary map and the up-convolution segmentation map is taken as the boundary to obtain the training nail missing image.

[0031] Optionally, obtaining the missing block area based on the front missing block area, the left missing block area, the right missing block area and the nail image group includes:

[0032] Constructing three-dimensional coordinates according to the nail image group; taking the length of the front nail image as the x-axis of the three-dimensional coordinates, the length of the front nail image as the y-axis of the three-dimensional coordinates, and the width of the left side nail image as the x-axis of the three-dimensional coordinates;

[0033] The position existing in the front missing block area and the left side missing block area is taken as the left side missing three-dimensional position; the left side missing three-dimensional position is obtained by using the front missing block area to obtain the x-axis and y-axis coordinates, and by using the left side missing block area to obtain the z-axis coordinate;

[0034] The position existing in the front missing block area and the right side missing block area is taken as the right side missing three-dimensional position; the right side missing three-dimensional position is obtained by using the front missing block area to obtain the x-axis and y-axis coordinates, and by using the right side missing block area to obtain the z-axis coordinate;

[0035] The missing block area is obtained by traversing the front missing block area, the left side missing block area, and the right side missing block area; the missing block area is an area composed of multiple left missing three-dimensional positions and multiple right missing three-dimensional positions.

[0036] Optionally, obtaining the missing patterns based on the nail image group includes:

[0037] Detecting the colors of the nail image group, performing clustering and classification, and obtaining a plurality of classified color groups;

[0038] Based on the multiple classified color groups, obtaining a color histogram;

[0039] The classified color groups whose values ​​in the color histogram are larger than those of other classified color groups are regarded as color regions;

[0040] The target color corresponding to the color in the color area is found in the database as the missing color.

[0041] Optionally, the method further includes:

[0042] Control the cleaner to clean the nails.

[0043] In a second aspect, an embodiment of the present invention provides a nail color correction device, comprising: a nail color correction protection box (1), a supporting base plate (2), a left camera (401), a right camera (402), a front camera (502), a nail color correction device (501), a nail fixer (301), a cleaner (504) and a processor, wherein the processor is connected to the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501).

[0044] A nail support plate (302) is fixed on the support base plate (2), a nail holder (301) is fixed on the nail support plate (302), a left camera (401) is fixed on the left side of the nail support plate (302), a right camera (402) is fixed on the right side of the nail support plate (302), a movable rod (503) is fixed on the top of the nail color correction protection box, and the movable rod (503) connects the front camera (502), the cleaner (504) and the nail color correction device (501); the processor, the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501) are fixed in the nail color correction protection box (1);

[0045] The processor is used to: obtain a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; the front nail image, the left side nail image and the right side nail image are images of the nail containing a missing block; based on the nail image group, locate the missing block of the nail through a missing block detection model to obtain a missing block area; the missing block area represents a nail discolored area; based on the nail image group, obtain a missing color; the missing color represents the color of the missing block area; control the nail color complementor (501) to complement the nail in the missing block area according to the missing color;

[0046] The front camera (502) is used to obtain a front nail image; the left camera (401) is used to obtain a left nail image; and the right camera (402) is used to obtain a right nail image.

[0047] The processor is also used to control the cleaner (504) to clean the nails.

[0048] In a third aspect, an embodiment of the present invention provides a nail color correction system, comprising:

[0049] Acquisition module: obtain a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; the front nail image, the left side nail image and the right side nail image are images of the nail containing the missing part;

[0050] Missing block detection module: Based on the nail image group, the missing block detection model is used to locate the missing block of the nail to obtain the missing block area; the missing block area indicates the area where the nail has faded;

[0051] Missing pattern module: based on the nail image group, obtains the missing pattern; the missing pattern indicates the pattern of the missing area;

[0052] Control color complement module: control the nail color complement device to complement the nail color in the missing area according to the missing color.

[0053] Optionally, the locating the missing blocks of the nails by using a missing block detection model based on the nail image group to obtain the missing block area includes:

[0054] Inputting the front nail image into a missing block detection model to obtain a front missing block area;

[0055] Input the left side nail image into the missing block detection model to obtain the left side missing block area;

[0056] Inputting the right side nail image into a missing block detection model to obtain a right side missing block area;

[0057] Based on the front missing block area, the left missing block area, the right missing block area and the nail image group, a missing block area is obtained.

[0058] Optionally, the training method of the missing block detection model includes:

[0059] A training set is obtained; the training set includes a plurality of training images and annotated data; the annotated data includes an annotated nail region image and an annotated boundary position; the annotated nail region image represents an image in which a nail missing region is covered with the same color; the annotated boundary position represents the position of the boundary between the nail missing region and the nail non-missing region;

[0060] Based on the training image, obtaining training segmentation information through an image segmentation network; the training segmentation information includes a boundary position image, an initial training nail missing image, and a training nail missing area;

[0061] The initial training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the initial training nail missing loss value;

[0062] The boundary position image and the annotated boundary position are passed through the cross entropy loss function to obtain the boundary loss value;

[0063] The training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the training nail missing loss value;

[0064] Obtain a total loss value; the total loss value is the sum of the initial training nail missing loss value, the boundary loss value and the training nail missing loss value;

[0065] Obtaining the current training iteration number of the missing block detection model and the preset maximum iteration number of the missing block detection model training;

[0066] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained missing block detection model.

[0067] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0068] The embodiment of the present invention also provides a nail color correction method, system and nail color correction device, the method comprising: obtaining a nail image group. The nail image group comprises a front nail image, a left side nail image and a right side nail image. The front nail image, the left side nail image and the right side nail image are images of the nail containing a missing block. Based on the nail image group, the missing block of the nail is located by a missing block detection model to obtain a missing block area. The missing block area represents an area where the nail is discolored. Based on the nail image group, a missing color is obtained. The missing color represents the color of the missing block area. Controlling the nail color corrector to perform color correction on the nail in the missing block area according to the missing color.

[0069] A detection method is used to detect the discolored missing block area. The convolution network is trained to obtain the missing part and the crack boundary between the colored nails. In the convolution process, the same main network and different branch networks are used to obtain the boundary of the missing block and the colored part while obtaining the nail area. The boundary position image and the initial training nail missing image are jointly judged, and the outermost layer of the nail area in the upper convolution boundary map and the upper convolution segmentation map is used as the boundary. The outermost boundary can also make the boundary color more integrated. The small boundary will be fused and colored during the coloring process. The crack boundary is obtained by convolution because if only pixels are used to judge the pixel mutation position, the color may also mutate, so the crack boundary is obtained by texture mutation. The front and side fusion parts are used to obtain the three-dimensional position, so that when the nail is supplemented, the color is supplemented according to the three-dimensional area, and the color is supplemented accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The present invention provides a flowchart of a nail color correction method.

[0071] Figure 2 It is a structural schematic diagram of a missing block detection model in a nail color complementation method provided by an embodiment of the present invention.

[0072] Figure 3 It is a structural schematic diagram of a nail color correction method and device provided by an embodiment of the present invention.

[0073] Figure 4 It is a schematic diagram of a block structure of an electronic device provided by an embodiment of the present invention.

[0074] Markings in the figure: nail color correction protection box 1; support base 2; nail fixer 301; fixed nail support plate 302; left camera 401; right camera 402; nail color correction device 501; front camera 502; movable rod 503; cleaner 504; bus 600; receiver 601; processor 602; transmitter 603; memory 604; bus interface 605.

[0075] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0076] Example 1

[0077] like Figure 1 As shown, an embodiment of the present invention provides a method for nail color correction, the method comprising:

[0078] S101: Obtain a nail image group. The nail image group includes a front nail image, a left side nail image, and a right side nail image. The front nail image, the left side nail image, and the right side nail image are images of the nails containing missing parts.

[0079] The front nail image represents the image of the front side of the nail taken by the front camera, the left side nail image represents the image of the left side of the nail taken by the left camera, and the right side nail image represents the image of the right side of the nail taken by the right camera.

[0080] S102: Based on the nail image group, the missing block detection model is used to locate the missing block of the nail to obtain the missing block area. The missing block area indicates the area where the nail has faded.

[0081] The structural diagram of the missing block detection model is as follows: Figure 2 shown.

[0082] S103: Based on the nail image group, obtain the missing pattern. The missing pattern indicates the pattern of the missing area.

[0083] S104: Controlling the nail color complementing device to complement the nail color in the missing area according to the missing color.

[0084] Optionally, the locating the missing blocks of the nails by using a missing block detection model based on the nail image group to obtain the missing block area includes:

[0085] The front nail image is input into the missing block detection model to obtain the front missing block area.

[0086] The process of obtaining the front missing block area by inputting the front nail image into the missing block detection model is equal to the process of training the nail missing area of ​​the training image in training.

[0087] The left side nail image is input into the missing block detection model to obtain the left side missing block area.

[0088] The process of obtaining the left side missing block area by inputting the left side nail image into the missing block detection model is equal to the process of training the nail missing area of ​​the training image in training.

[0089] The right side nail image is input into the missing block detection model to obtain the right side missing block area.

[0090] The process of obtaining the right side missing block area by inputting the right side nail image into the missing block detection model is equal to the process of training the nail missing area of ​​the training image in training.

[0091] Based on the front missing block area, the left missing block area, the right missing block area and the nail image group, a missing block area is obtained.

[0092] According to the above method, since the nail has an arc angle, if only the plane is judged, the nail cannot be colored accurately. Therefore, using images from multiple angles to find the 3D patch to color it can color the nail more accurately without missing any area.

[0093] Optionally, the training method of the missing block detection model includes:

[0094] A training set is obtained. The training set includes a plurality of training images and annotated data. The annotated data includes an annotated nail region image and an annotated boundary position. The annotated nail region image represents an image in which a nail missing region is covered with the same color. The annotated boundary position represents the position of the boundary between the nail missing region and the nail non-missing region.

[0095] In this embodiment, the nail missing area in the labeled nail area image is covered with red.

[0096] The position of the marked boundary is also recorded in color in the marked nail region image. In this embodiment, the marked boundary is recorded in black.

[0097] Based on the training image, training segmentation information is obtained through an image segmentation network. The training segmentation information includes a boundary position image, an initial training nail missing image, and a training nail missing area.

[0098] The initial training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the initial training nail missing loss value.

[0099] The boundary position image and the annotated boundary position are passed through the cross entropy loss function to obtain the boundary loss value.

[0100] The training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the training nail missing loss value.

[0101] A total loss value is obtained, which is the sum of the initial training nail missing loss value, the boundary loss value, and the training nail missing loss value.

[0102] The current training iteration number of the missing block detection model and the preset maximum iteration number of the missing block detection model training are obtained.

[0103] Among them, the preset maximum number of iterations for training the missing block detection model is 12,000 times.

[0104] When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained missing block detection model.

[0105] Through the above method, the missing block detection model is used to separate the faded part of the nail from the non-faded part, and the boundary position is obtained at the same time. Unlike the traditional boundary detection method, the detection method based on deep learning does not require manual intervention, can autonomously learn representative features, and can use the features extracted by convolution in the segmented area to detect the boundary of the input image. The boundary position can obtain the segmented area more accurately, so that the boundary position can be colored more carefully. The boundary is used to perform a double detection on the nail missing area. The boundary detection alone makes the boundary position more accurate, thereby making the detection of the nail missing area more accurate. And the network is trained by making a loss on the affected nail missing area, so that the trained network can directly obtain the nail missing position.

[0106] Optionally, obtaining training segmentation information based on the training image through an image segmentation network includes:

[0107] Among them, the image segmentation network described in this embodiment is an FCN network (Fully Convolutional Networks for Semantic Segmentation, fully convolutional neural network for semantic segmentation).

[0108] The training image is input into the backbone network to obtain a backbone feature map.

[0109] The backbone network described in this embodiment is the backbone network of the FCN network.

[0110] The backbone feature map is convolved through a first convolutional network to extract boundary features to obtain a boundary feature map.

[0111] The first convolution network for convolving the backbone feature map is a plurality of residual networks. The size of the boundary feature map is 7x7x2. The 7(px)x7(px) represents the size of the boundary feature map, and the 2 represents the category of the boundary feature map: boundary or not boundary.

[0112] The backbone feature map is convolved through the second convolutional network to perform image semantic segmentation to obtain a segmentation feature map.

[0113] The second convolution network for convolving the backbone feature map is a plurality of residual networks. The size of the segmentation feature map is 7x7x2. The 7(px)x7(px) represents the size of the segmentation feature map, and the 2 represents the category of the segmentation feature map: nail missing area and nail not missing area. The segmentation feature map size of 7x7x2 is obtained by a convolution kernel of 1x1 with a number of 2.

[0114] The boundary feature map is upsampled by deconvolution to reach the original image size, thereby obtaining an upconvolution boundary map; the upconvolution boundary map contains the predicted boundary positions.

[0115] Among them, deconvolution is performed using 2 convolution kernels with the same size as the training image to obtain the upper convolution boundary map.

[0116] The segmentation feature map is upsampled by deconvolution to reach the original image size, and the segmentation map is upconvolved. The initial training nail missing image represents the predicted nail missing area.

[0117] Among them, a convolution kernel with a size of 2 and the same size as the training image is used for deconvolution to obtain an upper convolution segmentation map;

[0118] The outermost layer of the nail area in the up-convolution boundary map and the up-convolution segmentation map is taken as the boundary to obtain the nail missing area.

[0119] If the boundary position in the upper convolution boundary map does not exist in the nail missing area, the boundary in the upper convolution boundary map is used as the boundary of the nail missing area.

[0120] Through the above method, the backbone network is used to extract the features of the image. Then, the backbone feature map is convolved to extract features of the boundary location and the nail missing area. During training, the loss is calculated by the labeled data, so as to perform backward convolution, train the parameters of the backbone network and the convolution parameters of the boundary location and the nail missing area. Since the extracted features are different, the parameters are different, and then different first convolution networks and second convolution networks are used to extract features. The boundary location and the nail area are both convolved to obtain a feature map of the same size as the original image, and the existence is indicated by predicting the features of each position. The boundary is used to perform a double detection on the nail missing area, and the left detection of the boundary alone makes the positioning of the boundary position more accurate, thereby making the detection of the nail missing area more accurate.

[0121] Optionally, obtaining the missing block area based on the front missing block area, the left missing block area, the right missing block area and the nail image group includes:

[0122] The three-dimensional coordinates are constructed according to the nail image group, with the length of the front nail image as the x-axis of the three-dimensional coordinates, the length of the front nail image as the y-axis of the three-dimensional coordinates, and the width of the left side nail image as the x-axis of the three-dimensional coordinates.

[0123] The values ​​on the plane formed by the three-dimensional coordinate x-axis and z-axis are equivalent to the values ​​in the left side nail image. The values ​​on the plane formed by the three-dimensional coordinate y-axis and z-axis are equivalent to the values ​​in the right side nail image. The values ​​on the plane formed by the three-dimensional coordinate x-axis and y-axis are equivalent to the values ​​in the front side nail image.

[0124] The position existing in the front missing block area and the left side missing block area is taken as the left side missing three-dimensional position. The left side missing three-dimensional position is obtained by obtaining the x-axis and y-axis coordinates of the front missing block area and the z-axis coordinate of the left side missing block area.

[0125] The position existing in the front missing block area and the right side missing block area is taken as the right side missing three-dimensional position. The right side missing three-dimensional position is obtained by obtaining the x-axis and y-axis coordinates of the front missing block area and the z-axis coordinate of the right side missing block area.

[0126] By traversing the front missing block area, the left side missing block area, and the right side missing block area, a missing block area is obtained. The missing block area is an area consisting of a plurality of left missing three-dimensional positions and a plurality of right missing three-dimensional positions.

[0127] Through the above method, the nail is in a three-dimensional state, but the captured image is a two-dimensional image. The entire length can be seen from the front, but the actual position is curved. The position detected by the two-dimensional image is not enough to make the nail coloring according to the three-dimensional area. Establish three-dimensional coordinates, map the positions of the front missing block area, the left side missing block area, and the right side missing block area to the 3D nail, and obtain the missing block area. This can make coloring more accurate.

[0128] Optionally, obtaining the missing pattern based on the nail image and the missing area includes:

[0129] The colors of the nail image group are detected, and clustered and classified to obtain a plurality of classified color groups.

[0130] Among them, clustering is performed using the K-means clustering algorithm.

[0131] Based on the multiple classified color groups, a color histogram is obtained.

[0132] Use the categorical color groups with greater values ​​than other categorical color groups in the color histogram as color regions.

[0133] The target color corresponding to the color in the color area is found in the database as the missing color.

[0134] If there is a color in one color area in the database, then this color is the target color. If there are multiple colors in the database, use the traversal counting method to get the most color in the color area. The most color in the color area is taken as the target color.

[0135] Wherein, the database stores nail colors that can be complemented.

[0136] Through the above method, since there may be other colors on the nail base color, the nail color is mainly complemented by the color of a large area to avoid color gap. Therefore, the clustering method is used to classify the colors, and the colors are grouped according to similar colors, so as to obtain multiple groups of classified color groups. The histogram is used to find which category of classified color groups is the largest. The colors in this group of classified color groups are used as colors, and the corresponding base colors in the database are used as missing colors. In this way, the color can be accurately selected for the user to complement the color.

[0137] Through the above method, since the nail color is not fixed, and the texture of the original nail that has faded is similar and easy to detect, a detection method is adopted to detect the missing area of ​​the faded nail. The convolution network is trained to obtain the missing part and the crack boundary between the colored nail. In the convolution process, the crack is obtained while the nail area is obtained, and the boundary image and the missing image are jointly judged. The crack boundary is obtained by convolution because if only pixels are used to judge the pixel mutation position, the color may also mutate, so the crack boundary is obtained by texture mutation. The outermost layer of the nail area in the upper convolution boundary map and the upper convolution segmentation map is used as the boundary. The outermost boundary can also make the boundary color more integrated, and the small boundary will be colored during the coloring process. The front and side fusion parts are three-dimensionally positioned for accurate color complementation.

[0138] The method further comprises controlling the cleaner to clean the nails.

[0139] Example 2

[0140] Based on the above-mentioned nail color correction method, an embodiment of the present invention further provides a nail color correction device, the device comprising a nail color correction protection box (1), a supporting base plate (2), a left camera (401), a right camera (402), a front camera (502), a nail color correction device (501) and a nail holder (301), and also comprising a processor, the processor being connected to the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501),

[0141] A nail support plate (302) is fixed on the support base plate (2), a nail holder (301) is fixed on the nail support plate (302), a left camera (401) is fixed on the left side of the nail support plate (302), a right camera (402) is fixed on the right side of the nail support plate (302), a movable rod (503) is fixed on the top of the nail color correction protection box, and the movable rod (503) connects the front camera (502), the cleaner (504) and the nail color correction device (501); the processor, the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501) are fixed in the nail color correction protection box (1);

[0142] The processor is used to: obtain a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; the front nail image, the left side nail image and the right side nail image are images of the nails containing missing parts; based on the nail image group, the missing blocks of the nails are located by a missing block detection model to obtain a missing block area; the missing block area represents an area where the nail has faded; based on the nail image group, a missing color is obtained; the missing color represents the color of the missing block area; and the nail color complementor is controlled to complement the nails in the missing block area according to the missing color.

[0143] The front camera (502) is used to obtain a front nail image; the left camera (401) is used to obtain a left side nail image; and the right camera (402) is used to obtain a right side nail image.

[0144] The nail is placed on a nail support plate (302) to support the nail. A nail holder (301) is used to fix the nail so that the same position can be accurately detected when detecting the missing position and complementing the color. The position of the front camera (502) is adjusted by a movable rod (503) until it corresponds to the top of the nail. Images in three different directions are obtained by using the left camera (401), the right camera (402) and the front camera (502). The obtained images are transmitted to a computer, and the missing position is detected by a missing block detection model, and a three-dimensional position can be detected, so that the color complementation is more accurate. The front camera (502) is moved away from the top of the nail by the movable rod (503), and the nail color complementor (501) is adjusted so that the color complementor performs color complementation according to the position corresponding to the missing block area returned by calculation.

[0145] The processor is also connected to the cleaner (504), and the processor is also used to control the cleaner (504) to clean the nails.

[0146] In the embodiment of the present invention, the nail cleaning liquid is arranged in the cleaner, and the nail cleaning liquid can be sprayed on the nails, and then directly rubbed to achieve the cleaning of the nails.

[0147] Example 3

[0148] Based on the above-mentioned nail color correction method, an embodiment of the present invention further provides a nail color correction system, which includes an acquisition module, a missing block detection module, a missing color module and a control color correction module.

[0149] The acquisition module is used to obtain a nail image group. The nail image group includes a front nail image, a left side nail image and a right side nail image. The front nail image, the left side nail image and the right side nail image are images of the nail containing the missing part.

[0150] The missing block detection module is used to locate the missing blocks of the nails based on the nail image group and the missing block detection model to obtain the missing block area. The missing block area indicates the area where the nails are discolored.

[0151] The missing pattern module is used to obtain the missing pattern based on the nail image group. The missing pattern indicates the pattern of the missing area.

[0152] The control color complement module is used to control the nail color complement device to complement the nail color in the missing block area according to the missing color.

[0153] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, it includes a memory 604, a processor 602, and a computer program stored in the memory 604 and executable on the processor 602. When the processor 602 executes the program, the steps of any one of the nail color correction methods described above are implemented.

[0155] Among them, Figure 4 In the embodiment, the bus architecture (represented by bus 600) may include any number of interconnected buses and bridges, and bus 600 links various circuits including one or more processors represented by processor 602 and memory represented by memory 604. Bus 600 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. Bus interface 605 provides an interface between bus 600 and receiver 601 and transmitter 603. Receiver 601 and transmitter 603 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices on a transmission medium. Processor 602 is responsible for managing bus 600 and general processing, and memory 604 may be used to store data used by processor 602 when performing operations. The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of any method of a nail color correction method described above and the data involved above are implemented.

[0156] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0157] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0158] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0159] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0160] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0161] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to an embodiment of the present invention. The present invention may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0162] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A method for nail color correction, characterized in that: include: Obtaining a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; The front nail image, the left side nail image and the right side nail image are images of the nails including the missing parts; Based on the nail image group, the missing block of the nail is located by using a missing block detection model to obtain a missing block area; the missing block area indicates a discolored area of ​​the nail; Based on the nail image group, a missing pattern is obtained; the missing pattern indicates the pattern of the missing area; Control the nail color complementer to complement the nail color in the missing area according to the missing color.

2. A nail color correction method according to claim 1, characterized in that: The method of locating the missing blocks of the nail based on the nail image group and obtaining the missing block area by using the missing block detection model includes: Inputting the front nail image into a missing block detection model to obtain a front missing block area; Input the left side nail image into the missing block detection model to obtain the left side missing block area; Inputting the right side nail image into a missing block detection model to obtain a right side missing block area; Based on the front missing block area, the left missing block area, the right missing block area and the nail image group, a missing block area is obtained.

3. A nail color correction method according to claim 1, characterized in that: The training method of the missing block detection model includes: A training set is obtained; the training set includes a plurality of training images and annotated data; the annotated data includes an annotated nail region image and an annotated boundary position; the annotated nail region image represents an image in which a nail missing region is covered with the same color; the annotated boundary position represents the position of the boundary between the nail missing region and the nail non-missing region; Based on the training image, obtaining training segmentation information through an image segmentation network; the training segmentation information includes a boundary position image, an initial training nail missing image, and a training nail missing area; The initial training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the initial training nail missing loss value; The boundary position image and the annotated boundary position are passed through the cross entropy loss function to obtain the boundary loss value; The training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the training nail missing loss value; Obtain a total loss value; the total loss value is the sum of the initial training nail missing loss value, the boundary loss value and the training nail missing loss value; Obtaining the current training iteration number of the missing block detection model and the preset maximum iteration number of the missing block detection model training; When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained missing block detection model.

4. A nail color correction method according to claim 3, characterized in that: The obtaining of training segmentation information based on the training image through an image segmentation network includes: Inputting the training image into the backbone network to obtain a backbone feature map; Convolving the backbone feature map through a first convolutional network to extract boundary features and obtain a boundary feature map; The backbone feature map is convolved through a second convolutional network to perform image semantic segmentation to obtain a segmentation feature map; The boundary feature map is upsampled by deconvolution to reach the original image size, and a boundary position image is obtained; the median value of the boundary position image is the predicted boundary existence; The segmentation feature map is upsampled by deconvolution to reach the original image size, thereby obtaining an initial training nail missing image; the initial training nail missing image represents the predicted nail missing area; The outermost layer of the nail area in the up-convolution boundary map and the up-convolution segmentation map is taken as the boundary to obtain the training nail missing image.

5. A nail color correction method according to claim 2, characterized in that: The obtaining of the missing block area based on the front missing block area, the left missing block area, the right missing block area and the nail image group includes: Constructing three-dimensional coordinates according to the nail image group; taking the length of the front nail image as the x-axis of the three-dimensional coordinates, the length of the front nail image as the y-axis of the three-dimensional coordinates, and the width of the left side nail image as the x-axis of the three-dimensional coordinates; The position existing in the front missing block area and the left side missing block area is taken as the left side missing three-dimensional position; the left side missing three-dimensional position is obtained by using the front missing block area to obtain the x-axis and y-axis coordinates, and by using the left side missing block area to obtain the z-axis coordinate; The position existing in the front missing block area and the right side missing block area is taken as the right side missing three-dimensional position; the right side missing three-dimensional position is obtained by using the front missing block area to obtain the x-axis and y-axis coordinates, and by using the right side missing block area to obtain the z-axis coordinate; The missing block area is obtained by traversing the front missing block area, the left side missing block area, and the right side missing block area; the missing block area is an area composed of multiple left missing three-dimensional positions and multiple right missing three-dimensional positions.

6. A nail color correction method according to claim 1, characterized in that: The step of obtaining missing patterns based on the nail image group includes: Detecting the colors of the nail image group, performing clustering and classification, and obtaining a plurality of classified color groups; Based on the multiple classified color groups, obtaining a color histogram; The classified color groups whose values ​​in the color histogram are larger than those of other classified color groups are regarded as color regions; The target color corresponding to the color in the color area is found in the database as the missing color.

7. A nail color correction device, characterized in that: The invention comprises a nail color correction protection box (1), a supporting base plate (2), a left camera (401), a right camera (402), a front camera (502), a nail color correction device (501), a nail fixer (301), a cleaner (504) and a processor, wherein the processor is connected to the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501). A nail support plate (302) is fixed on the support base plate (2), a nail holder (301) is fixed on the nail support plate (302), a left camera (401) is fixed on the left side of the nail support plate (302), a right camera (402) is fixed on the right side of the nail support plate (302), a movable rod (503) is fixed on the top of the nail color correction protection box, and the movable rod (503) connects the front camera (502), the nail color correction device (501) and the cleaner (504); the processor, the left camera (401), the right camera (402), the front camera (502) and the nail color correction device (501) are fixed in the nail color correction protection box (1); The processor is used to: obtain a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; the front nail image, the left side nail image and the right side nail image are images of the nail containing a missing block; based on the nail image group, the missing block of the nail is located by a missing block detection model to obtain a missing block area; the missing block area indicates a discolored area of ​​the nail; Based on the nail image group, a missing color is obtained; the missing color represents the color of the missing block area; and a nail color complementor (501) is controlled to complement the color of the nail in the missing block area according to the missing color. The front camera (502) is used to obtain a front nail image; the left camera (401) is used to obtain a left nail image; and the right camera (402) is used to obtain a right nail image.

8. The nail color correction device according to claim 7, characterized in that: The processor is also used to control the cleaner (504) to clean the nails.

9. A nail color correction system, characterized in that: include: Acquisition module: obtains a nail image group; the nail image group includes a front nail image, a left side nail image and a right side nail image; The front nail image, the left side nail image and the right side nail image are images of the nails including the missing parts; Missing block detection module: Based on the nail image group, the missing block detection model is used to locate the missing block of the nail to obtain the missing block area; the missing block area indicates the area where the nail has faded; Missing pattern module: based on the nail image group, obtains the missing pattern; the missing pattern indicates the pattern of the missing area; Control color complement module: control the nail color complement device to complement the nail color in the missing area according to the missing color.

10. A nail color correction system according to claim 9, characterized in that: The training method of the missing block detection model includes: A training set is obtained; the training set includes a plurality of training images and annotated data; the annotated data includes an annotated nail region image and an annotated boundary position; the annotated nail region image represents an image in which a nail missing region is covered with the same color; the annotated boundary position represents the position of the boundary between the nail missing region and the nail non-missing region; Based on the training image, obtaining training segmentation information through an image segmentation network; the training segmentation information includes a boundary position image, an initial training nail missing image, and a training nail missing area; The initial training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the initial training nail missing loss value; The boundary position image and the annotated boundary position are passed through the cross entropy loss function to obtain the boundary loss value; The training nail missing image and the labeled nail area image are passed through the cross entropy loss function to obtain the training nail missing loss value; Obtain a total loss value; the total loss value is the sum of the initial training nail missing loss value, the boundary loss value and the training nail missing loss value; Obtaining the current training iteration number of the missing block detection model and the preset maximum iteration number of the missing block detection model training; When the total loss value is less than or equal to the threshold or the number of training iterations reaches the maximum number of iterations, the training is stopped to obtain a trained missing block detection model.

Images