A method and system for fingernail identification positioning

Abstract

The application discloses a kind of nail identification positioning method and system, comprising: S1: identification device photograph obtains the first image information of working area, sends to target detection network to generate corresponding second image information, after segmentation, nail mask image is obtained by comparison;S2: the real coordinates of nail point cloud are obtained by original 3D point cloud and nail mask image operation;S3: nail point cloud is converted to calibration coordinate system, and calibration coordinate system is determined by setting the position of the printing device below identification device;S4: each nail is traversed, and nail angle information can be obtained according to the point cloud information of nail;S5: the point cloud coordinates of nail are projected onto two-dimensional plane according to camera external parameter, and the deflection angle of each nail is obtained;The device carries out operation to nail mask image and original 3D point cloud, obtains the real point cloud coordinates of each nail, and the shape of nail can be embodied by multiple point cloud coordinates.

Description

Technical Field

[0001] This invention relates to the field of intelligent nail technology, and in particular to a method and system for nail recognition and positioning. Background Technology

[0002] Currently, most intelligent nail art machines only print on a single nail surface. Before processing the nail art, the machine first needs to apply gel polish to the nail surface. The color of the gel polish must be different from the surrounding environment and the color of the fingers. Then, the machine needs to recognize the outline of the nail. This severely limits the processing time and recognition accuracy. Furthermore, since it only prints on a single nail, it does not need to recognize the rotation angle and tilt of the nail surface.

[0003] Currently, the method involves placing the four fingers (index, middle, ring, and little fingers) and the thumb on different planes. Specifically, based on the structure of the human hand, the left thumb is placed on the right side of the hand, perpendicular to the plane of the four fingers, and the right thumb is placed on the left side of the hand, also perpendicular to the plane of the four fingers. With this method, when fingers are placed in the work area, the nail surface may rotate and tilt. Furthermore, existing technology identifies individual nails, requiring the pre-application of gel polish (usually white, a color different from the surrounding environment and finger color) before identification. This process is cumbersome and increases the time required for manicures.

[0004] It should be noted that the applicant previously applied for a patent with publication number CN113469093A, which is a nail recognition method and system based on deep learning. This method involves pre-acquiring an original image of the hand, obtaining the probability value of each pixel belonging to the nail region, and generating a nail mask image from the original hand image. The outline coordinates of the nails in the nail mask image are extracted, and the deflection angle and tilt information of each nail are obtained. This eliminates the need to pre-apply nail polish and anti-spill gel before nail recognition, and allows for simultaneous recognition of multiple nails with higher accuracy and less overall time, solving the problems of the aforementioned technologies. However, the obtained nail mask image is a two-dimensional image, reflecting the position and region of the nail. The problem with this patent is that the nail is a curved surface, and each finger is not always facing upwards; each finger has a certain tilt angle. The rotation information is contained in the depth (Z-axis), which cannot be processed by the two-dimensional image. Furthermore, the height of each fingernail is completely different, and the lack of Z-axis data in the two-dimensional information makes it impossible to determine the height. The lack of Z-axis data will cause serious errors during the spraying process, affecting the manicure effect.

[0005] The angle of the nail along the X and Y axes is called the rotation angle; the angle of the nail along the Z axis is called the tilt angle. Summary of the Invention

[0006] The purpose of this invention is to address the problem of errors caused by tilt angle and height differences in existing nail art machines during the nail recognition and spraying process, and to propose a nail recognition and positioning method and system.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for nail identification and positioning, comprising:

[0008] S1: The recognition device takes a picture to obtain the first image information of the working area, locates the fingernail position in the first image information through the target detection network, generates the corresponding second image information, transmits the second image information to the segmentation network, segments the second image information into several regions, and compares the first image information with the segmented second image information after a preset probability value to obtain the fingernail mask image of the first image information.

[0009] S2: The true coordinates of the fingernail point cloud are obtained by calculating the original 3D point cloud and the fingernail mask image, and statistical filtering is performed on the fingernail point cloud to eliminate noise in the point cloud;

[0010] S3: Convert the fingernail point cloud to a calibration coordinate system, which is determined by setting the position of the printing device below the recognition device;

[0011] S4: Traverse each fingernail and obtain information such as the highest point, left and right tilt angles, and front and back tilt angles of the fingernail based on the point cloud information of the fingernail.

[0012] S5: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image.

[0013] As a further description of the above technical solution: In step S1, the method further includes:

[0014] S11: The recognition device takes a picture of the finger position in the working area to obtain the first image information, and sends it to the target detection network to locate the fingernail part in the first image information and generate the corresponding second image information. The second image information mainly includes the image information of the fingernail part.

[0015] S12: Send the second image information to the segmentation network, and preset the judgment probability value to obtain the probability value of each pixel in the original hand image belonging to the nail region. Mark the pixels with a probability value greater than the preset probability value as nail regions and the pixels with a probability value less than the preset probability value as non-nail regions to obtain the nail mask image of the first image information.

[0016] As a further description of the above technical solution: In step S2, the following is also included:

[0017] S21: The nail mask image is processed with the original 3D point cloud data to obtain the real point cloud coordinates of the nail. The original 3D point cloud data is the data in the real world.

[0018] S22: Filter each nail mask image to eliminate noise in the point cloud and use the point cloud to provide feedback on the accuracy of image information.

[0019] As a further description of the above technical solution: In step S3, the following is also included:

[0020] S31: Based on the identification device, the initial position of the printing device below the identification device is set as the calibration coordinate system;

[0021] S32: Send the real point cloud of the fingernail to the calibration coordinate system, and the calibration coordinate system is combined with the real point cloud to obtain the result.

[0022] As a further description of the above technical solution: In step S4, the following is also included:

[0023] S41: Based on the point cloud of each fingernail, traverse the fingernails and combine the point cloud data to obtain the basic data of the fingernails;

[0024] S42: After traversing the point cloud data, the point cloud of the fingernail is processed to obtain the highest point of the fingernail point cloud, the left and right tilt angles, and the front and back tilt angles.

[0025] As a further description of the above technical solution: In step S5, it also includes

[0026] S51: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera's extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image.

[0027] S52: Extract the nail mask image of each nail individually in advance, extract the image contour of each nail mask through the image processing function library, and fit it with the minimum rectangle. After the fitting is completed, return the rotation angle of the minimum rectangle with the top left vertex as the origin.

[0028] S53: Obtain the smallest rectangle of the nail outline through the image processing function. Based on the intersection of the rectangle and the nail outline, obtain the left, right and top and bottom endpoints of the nail. The left and right tilt angles of the nail can be calculated based on the point cloud coordinates (x, y, z) of the left and right endpoints. The top and bottom tilt angles can be calculated based on the point cloud coordinates of the top and bottom endpoints.

[0029] As a further description of the above technical solution: the segmentation network includes, but is not limited to, Unet, pspnet, deeplab series, etc., to segment and compare the second image information.

[0030] As a further description of the above technical solution: the recognition device is one of a 3D camera, an RGB binocular camera, a LiDAR (2D / 3D), a 3D structured light camera, or a ToF camera.

[0031] It also includes a nail recognition and positioning system, which is applicable to any of the methods described in the above technical solutions, including...

[0032] The image processing module is used to receive the first image information sent by the recognition device and generate the second image information based on the fingernail portion corresponding to the first image information;

[0033] The communication module can send the second image information to the segmentation network, segment the nail portion of the image, and send the segmented nail mask image back to the image processing module.

[0034] The control module is used to preset probability values, compare the segmented second image information with the first image information, determine the probability value of the corresponding pixel belonging to the nail region, mark the pixels with probability values ​​greater than the preset probability value as nail regions, and mark the pixels with probability values ​​less than the preset probability value as non-nails regions, thereby obtaining the nail mask image of the first image information.

[0035] The 3D point cloud computing module combines the nail mask image with the original 3D point cloud to obtain the real point cloud coordinates of the nail. Then, it transforms the nail point cloud into a calibration coordinate system so that the printing device can print according to the position of the calibration coordinate system.

[0036] The information processing module collects and processes the calculated or converted 3D point cloud coordinates, and obtains the deflection angle based on the 3D point cloud coordinates.

[0037] As a further description of the above technical solution: the calibration coordinate system is the xy plane, which is parallel to the plane of the printing device.

[0038] The above technical solution has the following advantages or beneficial effects:

[0039] 1. By segmenting the image information of the nail and comparing it with the first image information, a nail mask image is obtained. The nail mask image is then processed with the original 3D point cloud to obtain the real point cloud coordinates of each nail. The shape of the nail can be represented by multiple point cloud coordinates.

[0040] 2. After obtaining the actual point cloud of the fingernail, the point cloud is transformed into a calibration coordinate system, so that the point cloud coordinates are converted into a coordinate system applicable to the printing device. This can reduce errors during the printing process and make the printed pattern more accurate.

[0041] 3. The position of the nail can be calibrated by the nail point cloud. The rotation and tilt angle of the nail can be calculated by the point clouds at the top and bottom or the left and right ends, and the nail can be adjusted before printing. Attached Figure Description

[0042] Figure 1 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0043] Figure 2 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0044] Figure 3 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0045] Figure 4 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0046] Figure 5 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0047] Figure 6 This is a flowchart of a nail recognition and positioning method proposed in this invention;

[0048] Figure 7 This is a schematic diagram of the structure of a nail recognition and positioning system proposed in this invention.

[0049] Legend:

[0050] 1. Image processing module; 2. Communication module; 3. Control module; 4. 3D point cloud computing module; 5. Information processing module. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Reference Figure 1 One embodiment of the present invention provides a method for nail identification and positioning, comprising:

[0053] S1: The recognition device takes a picture to obtain the first image information of the working area, locates the fingernail position in the first image information through the target detection network, generates the corresponding second image information, transmits the second image information to the segmentation network, segments the second image information into several regions, and compares the first image information with the segmented second image information after a preset probability value to obtain the fingernail mask image of the first image information.

[0054] S2: Obtain the true coordinates of the fingernail point cloud by calculating the original 3D point cloud and the fingernail mask image, and perform statistical filtering on the fingernail point cloud to eliminate noise in the point cloud;

[0055] S3: Convert the fingernail point cloud to the calibration coordinate system, which is determined by setting the position of the printing device below the recognition device;

[0056] S4: Traverse each fingernail and obtain information such as the highest point, left and right tilt angles, and front and back tilt angles of the fingernail based on the point cloud information of the fingernail.

[0057] S5: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image.

[0058] In this embodiment, the recognition device takes a picture of the finger placed in the working area to generate a first image information, which is the original image of the finger. The fingernail in the generated first image information is located, and the image of the fingernail portion is used to generate a second image information, which is sent to the segmentation network. After the second image information is divided into several regions, it is compared with the first image information to obtain a fingernail mask image. The fingernail mask image is processed with the original 3D point cloud data to obtain the real point cloud coordinates of the fingernail, and noise is eliminated. The point cloud is transformed into a calibration coordinate system to determine the position of the coordinate system. Each fingernail is traversed, and the coordinates of each point cloud are collected to obtain point cloud coordinate information. Further information such as the highest point of the fingernail, the left and right tilt angles, and the front and back tilt angles are determined. The point cloud coordinates of the fingernail are projected onto a two-dimensional plane according to the camera extrinsic parameters. The reference coordinates of a fingernail are calculated using the extrinsic parameters and Gaussian filtering is applied to eliminate possible noise and make the reference position more accurate.

[0059] Point clouds can be generated by automatically measuring information from a large number of points on the surface of an object, and then outputting the point cloud data as a data file. This point cloud data is acquired by scanning equipment, or it can be created using scanned images and the intrinsic parameters of a scanning camera. This is done through camera calibration, using the camera's intrinsic parameters to calculate the relationship between real-world points (x, y, z), 3D points in the world coordinate system, and points (u, v) in the image. This requires the use of both the camera coordinate system and the image coordinate system. The transition from the world coordinate system to the camera coordinate system (extrinsic parameters) is achieved using the following formula:

[0060] PC=[XC*YC*ZC]=[R|T]PW=[R|T][XW*YW*ZW*1]

[0061] The corresponding 3D points can be obtained, where [R|T] represents the camera's extrinsic parameters.

[0062] The process of tracing the coordinates from the camera system to a point in the image (intrinsic parameter) involves converting the three-dimensional points in the camera coordinate system to PC = [XC, YC, ZC]. τ By using matrix transformations to map a two-dimensional point p = (x, y) to the image plane coordinate system, and by mapping the relationship between the three-dimensional coordinate point and the two-dimensional coordinate point in the image coordinate system, it is easier to calculate the position more accurately.

[0063] Reference Figure 2 In step S1, the method further includes:

[0064] S11: The recognition device takes a picture of the finger position in the working area, obtains the first image information, and sends it to the target detection network to locate the fingernail part in the first image information and generate the corresponding second image information. The second image information mainly includes the image information of the fingernail part.

[0065] S12: Send the second image information to the segmentation network, and preset the judgment probability value to obtain the probability value of each pixel in the original hand image belonging to the nail region. Mark the pixels with a probability value greater than the preset probability value as nail regions and the pixels with a probability value less than the preset probability value as non-nail regions to obtain the nail mask image of the first image information.

[0066] In this embodiment, the recognition device can be a 3D camera, and the segmentation network includes, but is not limited to, Unet, PSPnet, DeepLab series, etc., to segment and compare the second image information to obtain the probability value of each pixel in the original image belonging to the nail region, ranging from 0 to 1. The deep learning segmentation method avoids complex feature processing and has good detection accuracy and boundary accuracy. Assuming the preset probability value is 0.5, when comparing, pixels with a probability value greater than the preset probability value are marked as nail regions, and pixels with a probability value less than the preset probability value are marked as non-nail regions. By stitching together the regions marked by several pixels, a nail mask image is formed.

[0067] Reference Figure 3 In step S2, the following is also included:

[0068] S21: Perform calculations between the nail mask image and the original 3D point cloud data to obtain the real point cloud coordinates of the nail. The original 3D point cloud data is the data from the real world as defined.

[0069] S22: Filter each nail mask image to eliminate noise in the point cloud and use the point cloud to provide feedback on the accuracy of image information.

[0070] In this embodiment, the nail mask image and 3D point cloud data are processed to obtain the real point cloud coordinates of the nail. A new nail mask image is obtained by performing median filtering on the nail mask image.

[0071] Reference Figure 4 In step S3, the following is also included:

[0072] S31: Based on the identification device, set the initial position of the printing device below the identification device as the calibration coordinate system;

[0073] S32: Send the real point cloud of the fingernail to the calibration coordinate system, and combine the calibration coordinate system with the real point cloud to obtain the result.

[0074] In this embodiment, the printing device is an inkjet head, and the calibration coordinate system is the xy plane, which is parallel to the printer plane. By identifying the position of the device, the relative position of the printing device below it is determined, and the initial position of the printing device is set as the calibration coordinate system.

[0075] Reference Figure 5 Step S4 also includes:

[0076] S41: Based on the point cloud of each fingernail, traverse the fingernails and combine the point cloud data to obtain the basic data of the fingernails;

[0077] S42: After traversing the point cloud data, organize the point cloud data of the fingernail to obtain the highest point of the fingernail point cloud, the left and right tilt angles, and the front and back tilt angles.

[0078] In this embodiment, by traversing the point cloud coordinates of the fingernail and collecting them using three algorithms—preorder, inorder, or postorder—the identified coordinates are grouped into a set. By analyzing the data within the set, the highest point of the fingernail point cloud and the lowest point located around it can be determined. The left and right tilt angles and front and back tilt angles can be calculated using the corresponding coordinate system, which has high accuracy.

[0079] Reference Figure 6 In step S5, it also includes

[0080] S51: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera's extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image.

[0081] S52: Extract the nail mask image of each nail individually in advance, extract the image contour of each nail mask through the image processing function library, and fit it with the minimum rectangle. After the fitting is completed, return the rotation angle of the minimum rectangle with the top left vertex as the origin.

[0082] S53: Obtain the smallest rectangle of the nail outline through the image processing function. Based on the intersection of the rectangle and the nail outline, obtain the left, right and top and bottom endpoints of the nail. The left and right tilt angles of the nail can be calculated based on the point cloud coordinates (x, y, z) of the left and right endpoints. The top and bottom tilt angles can be calculated based on the point cloud coordinates of the top and bottom endpoints.

[0083] In this embodiment, after obtaining the deflection angle of each nail, a separate nail mask image is extracted for each nail. The length and width of the bounding rectangle of the nail outline are enlarged to an appropriate factor to serve as the length and width of the extracted image; in this embodiment, it can be enlarged by a factor of 2 for easier subsequent outline extraction. The outline of each nail image can be extracted using OpenCV image processing functions, such as findcounter. A minimum rectangle fitting is used to return the rotation angle of the minimum rectangle with its top-left vertex as the origin. This rotation angle is set as the rotation angle of the nail. Based on the intersection of the rectangle and the nail outline, the tilt direction and angle of the nail are determined.

[0084] Specifically, the recognition device is one of the following: a 3D camera, an RGB binocular camera, a LiDAR (2D / 3D), a 3D structured light camera, or a ToF camera.

[0085] Reference Figure 7 The present invention also includes an embodiment of a nail recognition and positioning system, which is applicable to any of the methods described above, including...

[0086] Image processing module 1 is used to receive first image information sent by the recognition device and generate second image information based on the fingernail portion corresponding to the first image information;

[0087] Communication module 2 can send the second image information to the segmentation network through the communication module, segment the nail portion of the image, and send the segmented nail mask image back to the image processing module;

[0088] Control module 3 is used to preset probability values, compare the segmented second image information with the first image information, determine the probability value of the corresponding pixel belonging to the nail region, mark the pixels with probability values ​​greater than the preset probability value as nail regions, and mark the pixels with probability values ​​less than the preset probability value as non-nails regions, thereby obtaining the nail mask image of the first image information;

[0089] The 3D point cloud computing module 4 combines the nail mask image with the original 3D point cloud to obtain the real point cloud coordinates of the nail. Then, it transforms the nail point cloud into the calibration coordinate system so that the printing device can print according to the position of the calibration coordinate system.

[0090] Information processing module 5 collects and processes the calculated or converted 3D point cloud coordinates, and obtains the deflection angle based on the 3D point cloud coordinates.

[0091] Furthermore, the coordinate system is calibrated as the xy plane, which is parallel to the plane of the printing device.

[0092] In this embodiment, the image processing module 1 receives the first image information sent by the recognition device, locates the fingernail in the generated first image information, generates second image information from the image of the fingernail portion, and sends the first image information to the target detection network and the second image information to the segmentation network through the communication module 2 to realize the transmission of image information; the control module 3 is used to preset probability values, compare the segmented second image information with the first image information, mark pixels with probability values ​​greater than the preset probability value as fingernail regions, and mark pixels with probability values ​​less than the preset probability value as non-fingernail regions, to obtain the fingernail mask image of the first image information; the 3D point cloud computing module 4 can measure the point cloud in an automated manner. The system collects a large amount of point information on the object's surface and outputs point cloud data as a data file. By mapping the relationship between 3D coordinate points and 2D coordinate points in the image coordinate system, it facilitates more accurate position calculation. The information processing module 5 collects and processes the calculated or converted 3D point cloud coordinates and obtains the deflection angle based on the 3D point cloud coordinates. It traverses the point cloud coordinates of the fingernail and collects them through three algorithms: preorder, inorder, and postorder. It takes several identified coordinates as a set and analyzes the data within the set to determine the highest point of the fingernail point cloud and the lowest point located around it. Using the corresponding coordinate system, it can calculate the left and right tilt angles and the front and back tilt angles with high accuracy.

[0093] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for nail identification and positioning, characterized in that, include: S1: The recognition device takes a picture to obtain the first image information of the working area, locates the fingernail position in the first image information through the target detection network, generates the corresponding second image information, transmits the second image information to the segmentation network, segments the second image information into several regions, and compares the first image information with the segmented second image information after a preset probability value to obtain the fingernail mask image of the first image information. S2: The true coordinates of the fingernail point cloud are obtained by calculating the original 3D point cloud and the fingernail mask image, and statistical filtering is performed on the fingernail point cloud to eliminate noise in the point cloud; S3: Convert the fingernail point cloud to a calibration coordinate system, which is determined by setting the position of the printing device below the recognition device; S4: Traverse each fingernail and obtain the highest point, left and right tilt angles, and front and back tilt angles of the fingernail based on the point cloud information of the fingernail. S5: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image. Step S4 also includes: S41: Based on the point cloud of each fingernail, traverse the fingernails and combine the point cloud data to obtain the basic data of the fingernails; S42: After traversing the point cloud data, the point cloud of the fingernail is processed to obtain the highest point of the fingernail point cloud, the left and right tilt angles, and the front and back tilt angles.

2. The method for nail identification and positioning according to claim 1, characterized in that: Step S1 further includes: S11: The recognition device takes a picture of the finger position in the working area to obtain the first image information, and sends it to the target detection network to locate the fingernail part in the first image information and generate the corresponding second image information. The second image information mainly includes the image information of the fingernail part. S12: Send the second image information to the segmentation network, and preset the judgment probability value to obtain the probability value of each pixel in the original hand image belonging to the nail region. Mark the pixels with a probability value greater than the preset probability value as nail regions and the pixels with a probability value less than the preset probability value as non-nail regions to obtain the nail mask image of the first image information.

3. The method for nail identification and positioning according to claim 1, characterized in that: Step S2 further includes: S21: The nail mask image is processed with the original 3D point cloud data to obtain the real point cloud coordinates of the nail. The original 3D point cloud data is the data in the real world. S22: Filter each nail mask image to eliminate noise in the point cloud and use the point cloud to provide feedback on the accuracy of image information.

4. The method for nail identification and positioning according to claim 1, characterized in that: Step S3 further includes: S31: Based on the identification device, the initial position of the printing device below the identification device is set as the calibration coordinate system; S32: Send the real point cloud of the fingernail to the calibration coordinate system, and the calibration coordinate system is combined with the real point cloud to obtain the result.

5. The method for nail identification and positioning according to claim 1, characterized in that: In step S5, it also includes S51: Project the point cloud coordinates of the fingernails onto a two-dimensional plane according to the camera's extrinsic parameters, perform Gaussian filtering to eliminate possible noise, and obtain the deflection angle of each fingernail based on the two-dimensional image. S52: Extract the nail mask image of each nail individually in advance, extract the image contour of each nail mask through the image processing function library, and fit it with the minimum rectangle. After the fitting is completed, return the rotation angle of the minimum rectangle with the top left vertex as the origin. S53: Obtain the smallest rectangle of the nail outline through the image processing function. Based on the intersection of the rectangle and the nail outline, obtain the left, right and top and bottom endpoints of the nail. Based on the point cloud coordinates (x, y, z) of the left and right endpoints, the left and right tilt angles of the nail can be calculated. Based on the point cloud coordinates of the top and bottom endpoints, the top and bottom tilt angles can be calculated.

6. The method for nail identification and positioning according to claim 1, characterized in that: The segmentation network includes the Unet, pspnet, and deeplab series, which are used to segment and compare the second image information.

7. The method for nail identification and positioning according to claim 1, characterized in that: The identification device is one of a 3D camera, an RGB binocular camera, a LiDAR, or a ToF camera.

8. A nail recognition and positioning system, characterized in that, The system is applicable to the method described in any one of claims 1-7, including... The image processing module (1) is used to receive the first image information sent by the recognition device and generate the second image information according to the fingernail part corresponding to the first image information; The communication module (2) can send the second image information to the segmentation network through the communication module, segment the nail part image therein, and send the segmented nail mask image back to the image processing module; The control module (3) is used to preset the probability value, compare the segmented second image information with the first image information, determine the probability value of the corresponding pixel belonging to the nail area, mark the pixel with the probability value greater than the preset probability value as the nail area, and mark the pixel with the probability value less than the preset probability value as the non-nails area, so as to obtain the nail mask image of the first image information. The 3D point cloud computing module (4) combines the nail mask image with the original 3D point cloud to obtain the real point cloud coordinates of the nail. Then, it converts the nail point cloud into the calibration coordinate system so that the printing device can print according to the position of the calibration coordinate system. The information processing module (5) collects and processes the calculated or converted 3D point cloud coordinates and obtains the deflection angle based on the 3D point cloud coordinates.

9. A nail recognition and positioning system according to claim 8, characterized in that: The calibration coordinate system is the xy plane, which is parallel to the plane of the printing device.

Images