Method, apparatus and device for nail region detection and key point estimation based on cnn

Abstract

This invention discloses a CNN-based method, apparatus, device, and storage medium for nail region detection and keypoint estimation. The method includes: acquiring an image to be processed; preprocessing the image, which includes a hand region; inputting the preprocessed image into a pre-trained nail recognition model, outputting a nail segmentation mask, a horizontal map of each pixel, and a vertical map of each pixel; calculating the pixels of the nail region based on the nail segmentation mask to obtain the nail region; and calculating the nail's principal direction and other directions based on the horizontal and vertical maps to obtain the coordinates of keypoints on the nail edge. This method can solve two application scenarios with a single model, significantly reducing the computational requirements of the hardware.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, and device for nail region detection and keypoint estimation based on CNN. Background Technology

[0002] The main ideas behind existing nail segmentation or nail keypoint related algorithms are based on deep learning to train segmentation models for nail regions or to estimate the coordinates of key points on the nail edge contour. In practical applications, especially nail-related algorithms released by industrial internet companies, they can be divided into two categories according to different use cases: one is nail region image segmentation, which directly uses algorithms and models to distinguish pixels in the nail region, and then color can be applied to the identified nail area to achieve a manicure effect, similar to applying nail polish; the second is to use finger skeleton estimation or nail bounding box detection models to find the nail position, and then estimate the coordinates of multiple key points on the nail edge, such as identifying 8 key points on the nail edge, and then using these key point coordinates to apply manicure materials to achieve the effect of manicure materials.

[0003] However, existing technologies do not directly combine the two scenarios mentioned above for nail art, nor do they propose a solution that can simultaneously address both usage scenarios (i.e., pixel-level segmentation of the nail region and nail keypoint estimation). While some indirect solutions exist that simultaneously satisfy both scenarios—using models for different tasks, one for nail segmentation and the other for nail keypoint estimation; or interpolating the nail region pixels based on keypoint estimation—these solutions have two problems: first, the interpolated nail region edges may not perfectly match the actual nail contour; second, nail keypoint estimation models often perform poorly when using full-image estimation because the nail portion is often very small compared to the full image, generally requiring the nail location or bounding box to be identified first. This necessitates the use of additional hand skeleton models or nail detection models, making the overall solution insufficient for hardware requirements and not a good solution. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a method, apparatus and device for nail region detection and key point estimation based on CNN, so as to at least solve the problem of high hardware computing power requirements when simultaneously implementing two use cases in related technologies.

[0005] To achieve the above objectives, this invention provides a CNN-based method for nail region detection and keypoint estimation, the method comprising:

[0006] Acquire an image to be processed, and preprocess the image to be processed, wherein the image to be processed includes a hand region;

[0007] The preprocessed image to be processed is input into a pre-trained nail recognition model, which outputs a nail segmentation mask, a horizontal map of each pixel, and a vertical map of each pixel.

[0008] The nail region is obtained by calculating the pixels of the nail region based on the nail segmentation mask;

[0009] The main direction and other directions of the nail are calculated based on the horizontal and vertical maps to obtain the coordinates of key points on the nail edge.

[0010] Preferably, the step of calculating the pixels of the nail region based on the nail segmentation mask to obtain the nail region includes:

[0011] By traversing each pixel position in the nail segmentation mask, the probability value corresponding to each pixel position is obtained;

[0012] The probability value is compared with a preset value, and the pixel position that is greater than the preset value is taken as the pixel of the nail region, thus obtaining the nail region.

[0013] Preferably, the step of calculating the nail's principal direction and other directions based on the horizontal and vertical maps to obtain the coordinates of key points on the nail edge includes:

[0014] The vector value of the horizontal or vertical direction corresponding to each pixel position is obtained according to the horizontal map or the vertical map, and used as the direction vector;

[0015] The main direction of the nail region is obtained by summing and averaging the direction vectors of all pixel positions.

[0016] The pixel positions corresponding to the boundary of the length of the projection of the nail in the main direction are used as the first edge point and the second edge point;

[0017] The coordinates of the key points on the nail edge are obtained by interpolating the pixel positions in the other directions. The other directions include arbitrary angle directions divided according to the midpoint between the first edge point and the second edge point.

[0018] Preferably, the training process of the nail recognition model includes:

[0019] The neural network is trained according to a preset loss function to obtain the nail recognition model; the preset loss function includes a loss that supervises the horizontal direction map and the vertical direction map according to preset direction vector labels.

[0020] Preferably, the supervision based on preset direction vector labels includes:

[0021] Based on the direction vector label, the position of each pixel in the nail area is calculated with respect to the starting point of the vector, wherein the direction of the direction vector of each pixel position is taken from the starting point of the vector.

[0022] Preferably, the supervision based on preset direction vector labels includes:

[0023] Based on the direction vector label, the direction vector of each pixel position in the nail region is calculated according to the vector starting point to the vector ending point, wherein the direction of the direction vector of each pixel position is the same.

[0024] Preferably, the nail segmentation mask includes a binary mask or a multi-class mask; the binary mask includes one nail region and a background region; the multi-class mask includes multiple nail regions and background regions.

[0025] To achieve the above objectives, the present invention also provides a CNN-based nail region detection and keypoint estimation device, the device comprising:

[0026] An acquisition unit is used to acquire an image to be processed and to preprocess the image to be processed, wherein the image to be processed includes a hand region;

[0027] The output unit is used to input the preprocessed image to be processed into the pre-trained nail recognition model and output the nail segmentation mask, the horizontal map of each pixel, and the vertical map of each pixel.

[0028] The first calculation unit is used to calculate the pixels of the nail region based on the nail segmentation mask to obtain the nail region;

[0029] The second calculation unit is used to calculate the main direction and other directions of the nail based on the horizontal direction map and the vertical direction map, and obtain the coordinates of key points on the edge of the nail.

[0030] To achieve the above objectives, the present invention also proposes an apparatus comprising a processor, a memory, and a computer program stored in the memory, the computer program being executed by the processor to implement the steps of a CNN-based nail region detection and keypoint estimation method as described in the above embodiments.

[0031] To achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a computer program that is executed by a processor to implement the steps of a CNN-based nail region detection and keypoint estimation method as described in the above embodiments.

[0032] Beneficial effects:

[0033] The above solution inputs the image to be processed into a pre-trained nail recognition model. Based on the output nail segmentation mask, the pixel count of the nail region is calculated to obtain the nail region. Based on the output horizontal and vertical maps, the main direction and other directions of the nail are calculated to obtain the coordinates of the nail edge key points. This solution can solve the two usage scenarios above with one model, that is, obtain the nail region and the coordinates of the nail edge key points at the same time, which greatly reduces the computing power requirements of the hardware.

[0034] The above solution iterates through each pixel position in the nail segmentation mask, obtains the probability value of each pixel position, compares it with a preset value, and takes the pixel position with the higher probability value as the pixel of the nail area, thus obtaining the nail area. It has low computational cost and high accuracy, and can achieve better nail art results in practical applications. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a CNN-based method for nail region detection and keypoint estimation, provided as an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram of the main direction of the nail and key points of the nail outline provided in an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram illustrating two methods for creating direction vector labels according to an embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of a CNN-based nail region detection and key point estimation device according to an embodiment of the present invention.

[0040] The realization of the invention's objective, its functional characteristics, and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0043] The present invention will be described in detail below with reference to the embodiments.

[0044] Reference Figure 1 The diagram shown is a flowchart of a CNN-based nail region detection and key point estimation method provided in an embodiment of the present invention.

[0045] In this embodiment, the method includes:

[0046] S11, Obtain the image to be processed, and preprocess the image to be processed, wherein the image to be processed includes a hand region.

[0047] In this embodiment, the image to be processed refers to the image for which nail region detection and keypoint estimation are to be performed. Specifically, the image to be processed includes the hand region. First, the image to be processed undergoes data normalization and data augmentation preprocessing. For example, the mean and variance of each image channel are calculated to normalize the image pixel values ​​to between [-1,1] or [0,1]. Then, data augmentation methods such as adding random noise, random perturbation, random rotation and scaling of the image, and random cropping of the image are applied.

[0048] S12, input the preprocessed image to be processed into the pre-trained nail recognition model, and output the nail segmentation mask, the horizontal map of each pixel and the vertical map of each pixel.

[0049] The nail segmentation mask includes a binary mask or a multi-class mask; the binary mask includes one nail region and a background region; the multi-class mask includes multiple nail regions and background regions.

[0050] In this embodiment, the preprocessed image to be processed passes through the backbone module of the nail recognition model (this backbone module can be any common backbone network structure), outputting a nail segmentation mask. The nail segmentation mask can be a binary mask (i.e., nail region and background) or a multi-class mask (e.g., five different finger categories + background). Further explanation: if it is a multi-class mask referring to each nail region and background, the shape of the nail segmentation mask output by the model is [w, h, 6] (w represents the image width, h represents the image height, and 6 represents the nail map of the five fingers + the background map of the non-nails region). When traversing the image position (x, y), the corresponding 6 values ​​represent the probability values ​​of the thumb, index finger, middle finger, ring finger, little finger, and background, respectively. The index of the maximum value is the nail region or background represented by that pixel. Whether to use binary classification or multi-class classification depends on the requirements of the task scenario. At the same time, the model outputs two maps to represent the horizontal and vertical directions of each pixel, both normalized to the range of [-1,1]. In addition, the loss used during model training is the same loss used in the nail segmentation mask task, plus the supervision loss for the horizontal and vertical directions, which is not limited here.

[0051] S13, calculate the pixels of the nail region based on the nail segmentation mask to obtain the nail region.

[0052] The step of calculating the pixels of the nail region based on the nail segmentation mask to obtain the nail region includes:

[0053] S13-1, by traversing each pixel position in the nail segmentation mask, obtain the probability value corresponding to each pixel position;

[0054] S13-2, compare the probability value with a preset value, and take the pixel position that is greater than the preset value as the pixel of the nail region to obtain the nail region.

[0055] In this embodiment, the pixels of the nail region are calculated using the nail segmentation mask output by the model to obtain the nail region. The nail segmentation mask is a tensor with the same size as the original image. Each pixel position stores a probability value (between 0 and 1) of whether that pixel position is a nail. By comparing this probability value with a set threshold (e.g., 0.5), pixels with a value greater than or equal to the threshold are considered as nail region pixels, and others are considered as background. By traversing all pixel positions of the nail segmentation mask, the pixels of the nail region can be obtained, thus obtaining the nail region.

[0056] S14, calculate the main direction and other directions of the nail based on the horizontal direction map and the vertical direction map to obtain the coordinates of the key points of the nail edge.

[0057] The step of calculating the nail's main direction and other directions based on the horizontal and vertical maps to obtain the coordinates of key points on the nail edge includes:

[0058] S14-1, Obtain the vector value of the horizontal or vertical direction corresponding to each pixel position according to the horizontal direction map or the vertical direction map, and use it as the direction vector;

[0059] S14-2, sum and average the direction vectors of all pixel positions to obtain the main direction of the nail in the nail region;

[0060] S14-3, the pixel position corresponding to the boundary of the length of the projection of the nail in the main direction is used as the first edge point and the second edge point;

[0061] S14-4, calculate the pixel positions in the other directions by interpolation to obtain the coordinates of the key points of the nail edge. The other directions include arbitrary angle directions divided according to the midpoint between the first edge point and the second edge point.

[0062] In this embodiment, the value corresponding to each pixel position is found on the horizontal and vertical maps output by the model (since the model simultaneously outputs a nail segmentation mask, the corresponding vertical and horizontal maps, their pixel positions are one-to-one; for example, a pixel position (x, y) in the nail segmentation mask also has the same position (x, y) on both the horizontal and vertical maps). This value is used as the unit direction vector (u, v), and the value at that pixel position represents the vector value in the horizontal or vertical direction at that point. Then, by summing and averaging the direction vectors at each pixel position, the main direction of the nail in that nail region is obtained (equivalent to finding the direction the finger is pointing). Figure 2In (a), the direction A->B is the main direction of the nail. After obtaining the main direction of the nail region, the key points of the nail outline are further determined. The steps include: taking 8 key points as an example, find the pixels in the same direction as the nail region outline (the edge outline of the nail region can be calculated) along the main direction of the nail region. That is, find the pixel position corresponding to the boundary of the length of the nail projection in the A->B direction, i.e., find points A and B; then find the perpendicular direction of the midpoint of AB by interpolation, see [link to relevant documentation]. Figure 2 (b) Find the pixel position corresponding to the boundary of the nail region with the maximum length in this direction, and then similarly find the pixels in the AB45° and 135° directions, etc. See [link to relevant documentation]. Figure 2 (c); This gives us the coordinates of the eight edge key points of the nail outline without any supervision or training steps for the key point coordinates (equivalent to finding the pixel containing the corresponding direction vector value on the edge outline of the nail region).

[0063] The training process of the nail recognition model includes:

[0064] The neural network is trained according to a preset loss function to obtain the nail recognition model; the preset loss function includes a loss that supervises the horizontal direction map and the vertical direction map according to preset direction vector labels.

[0065] Furthermore, the supervision based on preset direction vector labels includes:

[0066] Based on the direction vector label, the position of each pixel in the nail area is calculated with respect to the starting point of the vector, wherein the direction of the direction vector of each pixel position is taken from the starting point of the vector.

[0067] Furthermore, the supervision based on preset direction vector labels includes:

[0068] Based on the direction vector label, the direction vector of each pixel position in the nail region is calculated according to the vector starting point to the vector ending point, wherein the direction of the direction vector of each pixel position is the same.

[0069] See Figure 3As shown, in this embodiment, two schemes are included when pre-creating data direction vector labels for supervision. A is the starting point of the vector and B is the ending point of the vector (points A and B can be considered as two key points on each nail along the direction of the finger). A->B is defined as the main direction of the nail. The first scheme is to calculate the direction A->(x,y) of all pixel positions (x,y) in the nail region for points A and B in the direction vector label data. That is, each pixel position in the nail region in the foreground nail segmentation mask is calculated by combining this position with A (x-x_a, y-y_a). The second scheme is to calculate the direction vector of all pixel positions (x,y) in the nail region as A->B. That is, each pixel position of the nail stores the same direction value, i.e. (x_b-x_a, y_b-y_a).

[0070] Reference Figure 4 The diagram shown is a schematic representation of a CNN-based nail region detection and key point estimation device according to an embodiment of the present invention.

[0071] In this embodiment, the device 40 includes:

[0072] Acquisition unit 41 is used to acquire an image to be processed and to preprocess the image to be processed, wherein the image to be processed includes a hand region;

[0073] The output unit 42 is used to input the preprocessed image to be processed into the pre-trained nail recognition model and output the nail segmentation mask, the horizontal map of each pixel and the vertical map of each pixel.

[0074] The first calculation unit 43 is used to calculate the pixels of the nail region based on the nail segmentation mask to obtain the nail region;

[0075] The second calculation unit 44 is used to calculate the main direction of the nail and other directions based on the horizontal direction map and the vertical direction map, and obtain the coordinates of key points on the edge of the nail.

[0076] Furthermore, the first computing unit 43 includes:

[0077] The probability value acquisition unit is used to obtain the probability value of each pixel position by traversing each pixel position in the nail segmentation mask;

[0078] The comparison and judgment unit is used to compare the probability value with a preset value, and to take the pixel position that is greater than the preset value as the pixel of the nail region, so as to obtain the nail region.

[0079] Furthermore, the second computing unit 44 includes:

[0080] The vector value acquisition unit is used to acquire the vector value of the horizontal or vertical direction corresponding to each pixel position according to the horizontal direction map or the vertical direction map, and use it as a direction vector.

[0081] The nail direction unit is used to sum and average the direction vectors of all pixel positions to obtain the main nail direction of the nail region;

[0082] An edge point determination unit is used to determine the first edge point and the second edge point based on the pixel position corresponding to the boundary of the length of the projection of the nail in the main direction.

[0083] The key point calculation unit is used to calculate the pixel positions in the other directions by interpolation to obtain the coordinates of the key points on the nail edge. The other directions include arbitrary angle directions divided according to the midpoint between the first edge point and the second edge point.

[0084] Furthermore, the training process of the nail recognition model includes:

[0085] The neural network is trained according to a preset loss function to obtain the nail recognition model; the preset loss function includes a loss that supervises the horizontal direction map and the vertical direction map according to preset direction vector labels.

[0086] Furthermore, the supervision based on preset direction vector labels includes:

[0087] Based on the direction vector label, the position of each pixel in the nail area is calculated with respect to the starting point of the vector, wherein the direction of the direction vector of each pixel position is taken from the starting point of the vector.

[0088] Furthermore, the supervision based on preset direction vector labels includes:

[0089] Based on the direction vector label, the direction vector of each pixel position in the nail region is calculated according to the vector starting point to the vector ending point, wherein the direction of the direction vector of each pixel position is the same.

[0090] Furthermore, the nail segmentation mask includes a binary mask or a multi-class mask; the binary mask includes one nail region and a background region; the multi-class mask includes multiple nail regions and background regions.

[0091] Each unit module of the device 40 can execute the corresponding steps in the above method embodiment, so the details of each unit module will not be elaborated here. Please refer to the description of the corresponding steps above for details.

[0092] This invention also provides a device comprising the CNN-based nail region detection and keypoint estimation apparatus described above, wherein the CNN-based nail region detection and keypoint estimation apparatus can employ... Figure 4 The structure of the embodiment, correspondingly, can be executed Figure 1 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect. For details, please refer to the relevant records in the above embodiments, which will not be repeated here.

[0093] The device includes: a mobile phone, digital camera, or tablet computer with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, processor, input unit, display unit, and power supply.

[0094] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as image playback function), etc.; the data storage area can store data created according to the use of the device. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide access to the memory for the processor and input units.

[0095] The input unit can be used to receive input numerical, character, or image information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, in addition to a camera, the input unit of this embodiment may also include a touch-sensitive surface (e.g., a touch screen) and other input devices.

[0096] The display unit can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of the device. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. The display unit may include a display panel, optionally configured as an LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar display panel. Furthermore, a touch-sensitive surface may cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event.

[0097] This invention also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement... Figure 1 The illustrated method is a CNN-based method for nail region detection and keypoint estimation. The computer-readable storage medium may be a read-only memory, a hard disk, or an optical disk, etc.

[0098] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the device embodiments, equipment embodiments, and storage medium embodiments, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions in the method embodiments.

[0099] Furthermore, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0100] The foregoing description illustrates and describes preferred embodiments of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A CNN-based nail region detection and keypoint estimation method, characterized in that, The method comprises: acquiring a to-be-processed image, and preprocessing the to-be-processed image, wherein the to-be-processed image comprises a hand region; inputting the preprocessed to-be-processed image into a pre-trained nail recognition model, and outputting a nail segmentation mask, a horizontal direction map of each pixel, and a vertical direction map of each pixel; calculating pixels of a nail region according to the nail segmentation mask, and obtaining the nail region; calculating a main direction of a nail and other directions according to the horizontal direction map and the vertical direction map, and obtaining nail edge key point coordinates; the method comprises: obtaining a vector value of a horizontal direction or a vertical direction corresponding to each pixel position as a direction vector according to the horizontal direction map or the vertical direction map; summing and averaging direction vectors of all pixel positions to obtain the main direction of the nail region of the nail region; taking pixel positions corresponding to a boundary of a length of the main direction of the nail as a first edge point and a second edge point; obtaining the nail edge key point coordinates by interpolating pixel positions of the other directions, wherein the other directions comprise any angle direction divided according to a midpoint of the first edge point and the second edge point.

2. The CNN-based nail region detection and keypoint estimation method of claim 1, wherein, The method of calculating pixels of a nail region according to the nail segmentation mask, and obtaining the nail region, comprises: obtaining a probability value corresponding to each pixel position by traversing each pixel position in the nail segmentation mask; comparing the probability value with a preset value, taking pixel positions greater than the preset value as pixels of the nail region, and obtaining the nail region.

3. The CNN-based nail region detection and keypoint estimation method of claim 1, wherein, The training process of the nail recognition model comprises: training a neural network according to a preset loss function to obtain the nail recognition model; the preset loss function comprises a loss of supervision of the horizontal direction map and the vertical direction map according to a preset direction vector label.

4. The CNN-based nail region detection and keypoint estimation method of claim 3, wherein, The supervision according to the preset direction vector label comprises: based on the direction vector label, calculating each pixel position in the nail region with a vector starting point respectively, wherein the direction of the direction vector of each pixel position is taken as the starting point.

5. The CNN-based nail region detection and keypoint estimation method of claim 3, wherein, The supervision according to the preset direction vector label comprises: based on the direction vector label, calculating the direction vector of each pixel position in the nail region from a vector starting point to a vector ending point, wherein the direction of the direction vector of each pixel position is the same.

6. The CNN-based nail region detection and keypoint estimation method of claim 1, wherein, The nail segmentation mask comprises a binary classification mask or a multi-classification mask; the binary classification mask comprises one nail region and a background region; and the multi-classification mask comprises multiple nail regions and background regions.

7. A CNN-based nail region detection and keypoint estimation apparatus, characterized by, The device comprises: an acquisition unit configured to acquire a to-be-processed image, and preprocess the to-be-processed image, wherein the to-be-processed image comprises a hand region; an output unit configured to input the preprocessed to-be-processed image into a pre-trained nail recognition model, and output a nail segmentation mask, a horizontal direction map of each pixel, and a vertical direction map of each pixel; The first calculation unit is configured to calculate pixels of a nail region according to the nail segmentation mask to obtain the nail region. The second calculation unit is configured to calculate a main direction and other directions of the nail according to the horizontal direction map and the vertical direction map to obtain nail edge key point coordinates; the second calculation unit comprises: A vector value acquisition unit is configured to acquire a horizontal direction or a vertical direction vector value corresponding to each pixel position according to the horizontal direction map or the vertical direction map as a direction vector. A nail direction calculation unit is configured to sum and average direction vectors of all pixel positions to obtain the main direction of the nail region. An edge point determination unit is configured to take pixel positions corresponding to boundaries of the length of the main direction projection as first and second edge points. A key point calculation unit is configured to calculate pixel positions of the other directions by interpolation to obtain the nail edge key point coordinates, the other directions including any angle direction divided according to a midpoint of the first and second edge points.

8. An apparatus, comprising: A computer program stored in the memory is executed by the processor to implement the steps of the CNN-based nail region detection and key point estimation method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer readable storage medium and is executed by the processor to implement the steps of the CNN-based nail region detection and key point estimation method according to any one of claims 1 to 6.

Images