Full-automatic multi-process nail grinding system

Abstract

The invention provides a full-automatic multi-process nail grinding system, and belongs to the technical field of intellectualization. The system comprises an image acquisition module, a control module, a finger support assembly, a mechanical arm and a tool box, wherein grinding heads of different specifications are arranged in the tool box; the finger support assembly generates a first signal after supporting and clamping a finger; the control module generates a second signal according to the first signal; the image acquisition module performs image acquisition on the finger according to the second signal to obtain an RGB image and a depth image; the control module firstly performs contour extraction on the RGB image to obtain a nail contour image, then performs fusion processing on the nail contour image and the depth image to generate a nail surface model, performs feature extraction analysis on the nail contour image and the depth image to determine nail shape information, and then performs feature extraction analysis on the nail contour image and the depth image according to the nail surface model and nail shape selection information. And the mechanical arm is controlled to select a target polishing head corresponding to the nail shape information from the tool box to polish the fingernails. The nail grinding operation efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligence, in particular to a full-automatic multi-process nail grinding system. BACKGROUND

[0002] With the rapid development of the nail industry, the traditional manual nail grinding operation has problems of low efficiency, strong technical dependence, high labor cost, etc. Therefore, although technicians have designed electric grinding equipment and put it into market application, most of such electric grinding equipment are single-function handheld tools, lacking the ability of automatic identification of nail state and intelligent control of nail grinding. SUMMARY

[0003] The main purpose of the present application is to provide a full-automatic multi-process nail grinding system, aiming to improve the operation efficiency and operation convenience of the entire nail grinding process.

[0004] To achieve the above purpose, one aspect of the present application provides a full-automatic multi-process nail grinding system, which comprises an image acquisition module, a control module, a finger holder assembly, a mechanical arm and a tool box, and different specifications of several grinding heads are placed in the tool box. The finger holder assembly is used to generate a first signal after supporting and clamping the finger and send it to the control module; the control module is used to generate a second signal in response to the first signal and send it to the image acquisition module; the image acquisition module is used to acquire images of the finger in response to the second signal, obtain RGB images and depth images and send them to the control module; the control module is also used to first extract the contour of the RGB image to obtain a nail contour image, then fuse the nail contour image and the depth image to generate a nail surface model, and analyze the feature extraction of the nail contour image and the depth image to determine the nail shape information, and then control the mechanical arm to select the target grinding head corresponding to the nail shape information from the tool box to grind the finger nail according to the nail surface model and the pre-acquired nail type selection information.

[0005] Further, when the control module performs the operation of fusing the nail contour image and the depth image to generate the nail surface model, it is specifically configured to: preprocess the depth image, and then perform three-dimensional coordinate conversion according to the nail contour image and the preprocessed depth image to obtain a point cloud set; preprocess the point cloud set, and then perform Poisson surface reconstruction and model optimization on the preprocessed point cloud set to obtain the nail surface model.

[0006] Further, the control module, when performing the operation of respectively performing feature extraction analysis on the nail contour image and the depth image to determine the nail morphology information, is specifically configured to: perform color feature extraction on the nail contour image to obtain nail color feature information, and then input the nail color feature information after normalization and dimension reduction into a pre-trained first classifier model for processing to obtain a first classification result about the nail morphology; perform texture feature extraction on the nail contour image to obtain nail texture feature information, and then input the nail texture feature information after normalization and dimension reduction into a pre-trained second classifier model for processing to obtain a second classification result about the nail morphology; obtain nail depth feature information from the depth image, and then input the nail depth feature information after normalization and dimension reduction into a pre-trained third classifier model for processing to obtain a third classification result about the nail morphology; fuse the first classification result, the second classification result, and the third classification result to obtain the nail morphology information.

[0007] Further, the plurality of polishing heads placed in the tool box include a flexible sponge strip and a plurality of rigid abrasive strips of different mesh numbers; and the control module, when performing the operation of controlling the mechanical arm to select a target polishing head corresponding to the nail morphology information from the tool box to polish the finger nail according to the nail surface model and pre-acquired nail type selection information, is specifically configured to: when the nail morphology information indicates that the finger nail is coated with nail polish or externally connected with a false nail piece, sequentially perform a first sub-operation and a second sub-operation; when the nail morphology information indicates that the finger nail is a natural nail, perform the second sub-operation; the first sub-operation includes: performing trajectory planning according to the nail surface model to obtain a polishing path; and controlling the mechanical arm to first select a first rigid abrasive strip from the tool box, and then performing preliminary polishing on the finger nail according to the polishing path, the preliminary polishing being in the form of removing a nail polish cover layer or removing a false nail; the second sub-operation includes: performing trajectory planning according to the nail surface model and the nail type selection information to obtain a shaping path and a polishing path; and controlling the mechanical arm to first select a second rigid abrasive strip from the tool box, and then performing shaping on the finger nail according to the shaping path; and when the finger nail shaping is completed, controlling the mechanical arm to first select the flexible sponge strip from the tool box, and then performing polishing on the finger nail according to the polishing path; wherein the mesh number of the first rigid abrasive strip is smaller than that of the second rigid abrasive strip.

[0008] Further, the control module is configured to control the robot arm to first select the flexible sponge strip from the tool box before performing the operation of polishing the finger nail according to the polishing path. control the robot arm to first select a third rigid sand strip from the tool box before performing the operation of preliminarily polishing the finger nail according to the polishing path. wherein the third rigid sand strip has a higher mesh number than the second rigid sand strip.

[0009] Further, the system further comprises a touch display screen; the touch display screen is used to generate the type selection information and send it to the control module in response to user operation.

[0010] Further, the end of each polishing head is provided with a magnetic guide, and the end of the robot arm is provided with an electromagnet; the electromagnet is attracted to the magnetic guide after being powered on, so that the end of the robot arm 140 is connected to the end of the polishing head.

[0011] Further, the system further comprises a disinfection box; the control module is further used to control the robot arm to move the target polishing head to the disinfection box for disinfection after the finger nail polishing is completed and then place it back to the tool box.

[0012] Further, the system further comprises an emergency stop module; the emergency stop module is used to generate an emergency stop instruction and send it to the control module in response to user operation; the control module is further used to control the robot arm to perform a reset operation in response to the emergency stop instruction.

[0013] Further, the system further comprises a dust collection module; the control module is further used to generate a third signal and send it to the dust collection module while controlling the robot arm to select the target polishing head corresponding to the nail shape information from the tool box to polish the finger nail; the dust collection module is used to adsorb the nail powder generated in the process of polishing the finger nail in response to the third signal.

[0014] The present application at least has the following beneficial effects: after the finger support assembly supports and clamps the finger, the first signal is sent to the control module, so that the control module sends the second signal to the image acquisition module, and then the image acquisition module acquires the RGB image and the depth image by image acquisition, then the control module extracts the contour of the RGB image to obtain the nail contour image, and then the nail contour image and the depth image are fused to generate the nail surface model, and the feature extraction analysis is performed on the nail contour image and the depth image to determine the nail shape information, finally, according to the nail surface model and the pre-acquired nail type selection information, the mechanical arm selects the target polishing head corresponding to the nail shape information from the tool box to polish the finger nail, so that the cooperation among the finger support assembly, the image acquisition module and the control module can improve the operation efficiency and the operation convenience of the whole nail polishing process. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a schematic diagram of a full-automatic multi-process nail polishing system provided by an embodiment of the present application; Figure 2 is a schematic diagram of the internal structure of the equipment box provided by an embodiment of the present application; Figure 3 is a schematic diagram of the front cover plate of the equipment box provided by an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementation described in the following exemplary embodiments does not represent all the implementations consistent with the embodiments of the present application, but is only an example of the system consistent with some aspects of the embodiments of the present application as described in the appended claims.

[0017] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various concepts, but unless specifically stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0018] As used herein, the terms "at least one", "multiple", "each", "any", and the like, include one, two, or more, multiple includes two or more, each refers to each of the corresponding plurality, and any refers to any of the plurality.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to be limiting of this application.

[0020] With the rapid development of the nail industry, the traditional manual nail grinding operation has problems of low efficiency, strong technical dependence, high labor cost, etc. Therefore, although technicians have designed electric grinding equipment and put it into market application, most of such electric grinding equipment is a single-function handheld tool, which lacks the ability to automatically identify the nail state and intelligently control the grinding of the nail. Understandably, such electric grinding equipment is usually difficult to achieve full-automatic switching of different processes such as removing the nail polish cover layer, removing the nail, shaping, polishing, etc. Either a single specification grinding head is fixedly configured to complete a specific process, or different specification grinding heads are allowed to be manually replaced to complete the corresponding process, which not only wastes time and effort, but also cannot achieve continuous and efficient grinding work.

[0021] Therefore, embodiments of the present application provide a full-automatic multi-process nail grinding system. After the finger support assembly supports and clamps the finger, it sends a first signal to the control module, so that the control module sends a second signal to the image acquisition module, and then the image acquisition module acquires images of the finger to obtain RGB images and depth images. Subsequently, the control module extracts the contour of the RGB image to obtain a nail contour image, and then fuses the nail contour image and the depth image to generate a nail surface model, and analyzes the features of the nail contour image and the depth image to determine the nail shape information. Finally, according to the nail surface model and the pre-acquired nail type selection information, the control module controls the mechanical arm to select the target grinding head corresponding to the nail shape information from the tool box to grind the finger nail. Thus, by means of the mutual cooperation between the finger support assembly, the image acquisition module and the control module, the operation efficiency and the operation convenience of the entire nail grinding process can be improved without wasting labor operation cost.

[0022] Please refer to Figure 1 , Figure 1is an optional component schematic diagram of a full-automatic multi-process nail grinding system provided by the embodiment of the application. The full-automatic multi-process nail grinding system at least comprises an image acquisition module 110, a control module 120, a finger rest assembly 130, a mechanical arm 140 and a tool box 150, and the image acquisition module 110, the finger rest assembly 130 and the mechanical arm 140 are all connected with the control module 120, wherein the tool box 150 is placed with several polishing heads of different specifications, and the mechanical arm 140 preferably adopts a three-degree-of-freedom mechanical arm.

[0023] In actual application, the finger rest assembly 130 first supports and clamps the finger, then generates a first signal and sends it to the control module 120. The first signal can be understood as a control start signal. The control module 120 generates a second signal in response to the received first signal and sends it to the image acquisition module 110. The second signal can be understood as an image acquisition start signal. The image acquisition module 110 performs image acquisition on the finger in response to the received second signal, obtains an RGB image and a depth image and sends them to the control module 120. The control module 120 first performs contour extraction on the received RGB image to obtain a nail contour image, then performs fusion processing on the nail contour image and the received depth image to generate a nail surface model, and performs feature extraction analysis on the nail contour image and the received depth image to determine nail shape information. Subsequently, according to the nail surface model and the pre-acquired nail type selection information, the mechanical arm 140 selects a target polishing head corresponding to the nail shape information from the tool box 150 to polish the finger nail.

[0024] In some embodiments, the finger rest assembly 130 comprises a U-shaped semi-enclosed guide bracket, a controller and flexible elastic clamping structures arranged on the left and right sides of the guide bracket. A microswitch connected with the controller can be arranged at the bottom of the guide bracket. When the finger is placed on the guide bracket, the microswitch is triggered to conduct and generate an electric signal sent to the controller. Subsequently, the controller controls the flexible elastic clamping structure to tighten in response to the electric signal, so as to non-rigidly clamp and fix the finger, so that the finger nail has stable posture and high repeatability in space, prevents positioning deviation caused by finger movement, and finally the controller generates the first signal and sends it to the control module 120 after determining that the flexible elastic clamping structure is tightened.

[0025] In some embodiments, the image acquisition module 110 comprises a structured light depth camera, which is internally provided with an RGB camera and a depth camera. After receiving the second signal, the structured light depth camera acquires an image of the finger, that is, controls the RGB camera inside to acquire an RGB image of the finger, and controls the depth camera inside to acquire a depth image of the finger, and then sends the acquired RGB image and depth image to the control module 120. It should be noted that the structured light depth camera has performed optical path alignment on the RGB camera and the depth camera in the hardware design stage, so that the images output by the RGB camera and the depth camera are actually images after coordinate system alignment, avoiding acquisition deviation caused by position difference of different cameras.

[0026] In some embodiments, when the control module 120 performs the operation of extracting the contour of the RGB image to obtain the nail contour image, it is specifically configured to perform the following steps S101 to S103.

[0027] Step S101, Gaussian smoothing is performed on the RGB image to suppress high-frequency noise while retaining edge features, thereby obtaining a first RGB image.

[0028] Step S102, the Canny operator is used to detect gradient mutation points on the first RGB image, and continuous edges are formed by double-threshold connection, thereby obtaining an initial contour image.

[0029] Step S103, the Ramer-Douglas-Peucker algorithm is used to perform polygon approximation on the initial contour image to reduce redundant points on the premise of retaining geometric features, thereby obtaining a final nail contour image.

[0030] The two-dimensional contour formed in the nail contour image will serve as a limiting boundary for the subsequent nail polishing area, which can prevent polishing to parts outside the finger nail and enhance polishing safety.

[0031] In some embodiments, when the control module 120 performs the operation of fusing the nail contour image and the depth image to generate a nail surface model, it is specifically configured to perform the following steps S201 to S202.

[0032] Step S201, the depth image is preprocessed, and then three-dimensional coordinate conversion is performed according to the nail contour image and the preprocessed depth image to obtain a point cloud set.

[0033] In this step, regarding the content of pre-processing the depth image, the corresponding implementation can but not limited to include: according to the intrinsic matrix of the depth camera, the depth image is corrected for distortion to restore the real spatial position relationship of the pixels, which is helpful to solve the image distortion problem caused by the lens optical characteristics in the depth camera or the installation error of the depth camera, wherein the intrinsic matrix of the depth camera can contain focal length, principal point position, distortion coefficient and other parameter values; then the median filtering algorithm is used to suppress the noise of the depth image after distortion correction, so as to retain the key features such as depth edge, which is helpful to smooth the abnormal fluctuations in the depth image caused by the measurement noise of the depth camera or environmental interference.

[0034] In this step, regarding the content of performing three-dimensional coordinate conversion on the nail contour image and the pre-processed depth image to obtain a point cloud set, the corresponding implementation can but not limited to include: the nail contour image contains a plurality of effective pixel points, each effective pixel point falls inside the two-dimensional contour formed in the nail contour image, a plurality of target pixel points corresponding to the positions of the plurality of effective pixel points are extracted from the pre-processed depth image, and then according to the intrinsic matrix of the RGB camera, the first expression is used to perform three-dimensional coordinate conversion on the coordinate information of the plurality of effective pixel points and the depth values carried by the corresponding extracted plurality of target pixel points, to obtain a plurality of point cloud data and form a point cloud set.

[0035] Wherein, the first expression is constructed based on the inverse process of pinhole imaging, which is specifically represented as: ; In the formula, for a certain effective pixel point contained in the nail contour image and a target pixel point contained in the pre-processed depth image and corresponding to the position of the effective pixel point, is the coordinate information of the effective pixel point, is the depth value carried by the target pixel point, is the three-dimensional coordinate information of the point cloud determined based on the effective pixel point and the target pixel point, the intrinsic matrix of the RGB camera contains image focal length and image principal point coordinates, the image focal length contains a first focal length and a second focal length, the first focal length refers to the scaling ratio of the image in the x-axis (width direction), the second focal length refers to the scaling ratio of the image in the y-axis (height direction), and the image principal point coordinates refer to the pixel coordinates of the intersection between the optical axis and the image plane, is the first focal length, and , is the physical focal length, is the width of a single pixel, is the second focal length, and , is the height of a single pixel, is the image principal point coordinates.

[0036] In step S202, the point cloud set is preprocessed, and then Poisson surface reconstruction and model optimization are performed on the preprocessed point cloud set to obtain the nail surface model.

[0037] In this step, regarding the preprocessing of the point cloud set, the corresponding embodiments can but are not limited to include: Gaussian filtering of the point cloud set, that is, by assigning a Gaussian weight to the point cloud in the neighborhood of each point cloud and calculating the weighted average value, the surface overall morphology and key detail features of the point cloud can be maximally retained while the dense noise area is smoothed, which is beneficial to eliminate the random noise problem existing in the point cloud set; voxel grid downsampling of the Gaussian filtered point cloud set, that is, by uniformly dividing the three-dimensional space formed by the Gaussian filtered point cloud set into a plurality of voxel grids, the centroid coordinates of all point clouds contained in each voxel grid are calculated, and finally only the centroid of each voxel grid is retained as a sampling point, which can effectively maintain the topological structure and surface distribution features of the point cloud while reducing the amount of point cloud data, which is beneficial to improve the efficiency of subsequent modeling.

[0038] In this step, regarding the Poisson surface reconstruction and model optimization of the preprocessed point cloud set to obtain the nail surface model, the corresponding embodiments can but are not limited to include: Poisson surface reconstruction of the preprocessed point cloud set, that is, first calculating the normal vector of each point cloud and adjusting the direction consistency, then constructing an octree structure to adaptively divide the three-dimensional space formed by the preprocessed point cloud set, dynamically adjusting the voxel resolution according to the point cloud density, then constructing an implicit surface by solving the Poisson equation, taking the normal vector of the point cloud as a constraint condition in the Poisson equation, so that the generated surface is as close as possible to the original point cloud distribution, then setting appropriate reconstruction depth parameters to control the detail accuracy of the surface, and finally extracting the isosurface to generate an initial triangular mesh surface model; Optimization of the initial triangular mesh surface model obtained by reconstruction, that is, first simplifying the initial triangular mesh surface model by using edge collapse algorithm to reduce the number of triangular patches, reducing the calculation complexity while maintaining the model topological structure, then smoothing the initial triangular mesh surface model after grid optimization by using Laplace algorithm, adjusting the vertex position through smoothing iteration, which can eliminate the local noise and jagged edges generated in the model reconstruction process, and finally performing grid quality detection on the initial triangular mesh surface model after grid smoothing, removing the degenerate triangles and optimizing the vertex connection relationship, ensuring that the angles and areas of the model patches are within a reasonable range, and then obtaining the final nail surface model.

[0039] In some embodiments, the control module 120, when performing the operations of respectively performing feature extraction analysis on the nail contour image and the depth image to determine the nail morphology information, is specifically configured to perform the following steps S301 to S304.

[0040] Step S301, color feature extraction is performed on the nail contour image to obtain nail color feature information, which is then normalized and reduced in dimension before being input to a pre-trained first classifier model for processing to obtain a first classification result about the nail morphology.

[0041] In this step, the color distribution information of the nail contour image can be obtained by calculating the color histogram of the nail contour image, that is, the nail contour image is converted from the RGB color space to the HSV color space for representation, and then the hue feature data, saturation feature data and brightness feature data of the converted nail contour image are extracted to form the nail color feature information.

[0042] From the hue characteristic, the hue of natural nails is usually concentrated in the light pink or light white interval, with a narrow distribution range, while the hue of nails coated with nail polish or externally connected artificial nail pieces varies greatly, covering multiple color intervals; from the saturation characteristic, the nails coated with nail polish present a medium-high saturation characteristic, with bright colors, while the saturation of natural nails is low, with relatively dull colors; from the brightness characteristic, the surface of the externally connected artificial nail piece is generally high in brightness and has obvious highlight reflection areas due to the material characteristics, while natural nails have fewer highlight areas and the brightness distribution is more uniform. In this application, the nail color characteristics can be analyzed to determine whether the nail is a natural nail or a nail coated with nail polish or an externally connected artificial nail piece, which is specifically distinguished as follows: natural nails exhibit concentrated hue, low saturation and few high-brightness areas, nails coated with nail polish exhibit varied hue, medium-high saturation and uniform surface color, and externally connected artificial nail pieces exhibit high brightness, local highlight and hue that is either white or blue or transparent.

[0043] In this step, the nail color feature information is first normalized to eliminate dimensional differences, then principal component analysis algorithm is used to reduce the dimension of the normalized nail color feature information to reduce color feature redundancy and extract key color features, and finally the first classifier model is used to analyze the reduced nail color feature information to output the first classification result about the nail morphology; wherein the first classifier model can be constructed based on the random forest algorithm, which effectively handles the complex relationship between color features by using the randomness and ensemble characteristics of the random forest, and enhances the generalization ability and robustness of the first classifier model.

[0044] In step S302, texture feature extraction is performed on the nail contour image to obtain nail texture feature information, and after normalization and dimension reduction of the nail texture feature information, the nail texture feature information is input into a second classifier model trained in advance for processing to obtain a second classification result about the nail shape.

[0045] In this step, texture feature extraction can be performed on the nail contour image by using a local binary pattern (LBP) and a gray level co-occurrence matrix (GLCM). Subsequently, by analyzing the nail texture characteristics, it can be determined whether the nail is a natural nail, a nail coated with nail polish, or a nail with an externally attached artificial nail. The specific distinctions are as follows: the texture of a natural nail is natural and delicate, showing the unique fine structure of human tissue and having the characteristics of biological tissue texture; the nail coated with nail polish has a high smooth surface due to the coating characteristics, and the texture characteristics are weak, with a relatively smooth overall visual effect; although the nail with an externally attached artificial nail has a high smoothness, small bubbles or glue marks may be generated due to the manufacturing process or the pasting process, forming a special texture distribution pattern that is different from other types.

[0046] In this step, the nail texture feature information is first normalized to eliminate dimensional differences, and then principal component analysis algorithm is used to reduce the dimension of the normalized nail texture feature information to reduce texture feature redundancy and extract key texture features. Finally, the second classifier model is used to analyze the reduced nail texture feature information to output the second classification result about the nail shape. The second classifier model can be constructed based on a random forest algorithm, which effectively handles the complex relationships between texture features and enhances the generalization ability and robustness of the second classifier model.

[0047] In step S303, nail depth feature information is obtained from the depth image, and after normalization and dimension reduction of the nail depth feature information, the nail depth feature information is input into a third classifier model trained in advance for processing to obtain a third classification result about the nail shape.

[0048] In this step, the nail depth feature information mainly includes the depth data carried by each pixel point in the depth image. The nail depth feature information is first normalized to eliminate dimensional differences, and then principal component analysis algorithm is used to reduce the dimension of the normalized nail depth feature information to reduce depth feature redundancy and extract key depth features. Finally, the third classifier model is used to analyze the reduced nail depth feature information to output the third classification result about the nail shape. The third classifier model can be constructed based on a random forest algorithm, which effectively handles the complex relationships between depth features and enhances the generalization ability and robustness of the third classifier model.

[0049] Step S304, fusing the first classification result, the second classification result and the third classification result about the nail shape to obtain the nail shape information.

[0050] In this step, the first classification result, the second classification result and the third classification result can be comprehensively analyzed by a voting mechanism or a weighted average mechanism to obtain the nail shape information, and the decision advantage of different features can improve the reliability of classification.

[0051] Among them, the first classification result includes the probability A1 that the nail is a natural nail, the probability A2 that the nail is coated with nail polish, and the probability A3 that the nail is circumscribed by a false nail piece, the second classification result also includes the probability B1 that the nail is a natural nail, the probability B2 that the nail is coated with nail polish, and the probability B3 that the nail is circumscribed by a false nail piece, and the third classification result also includes the probability C1 that the nail is a natural nail, the probability C2 that the nail is coated with nail polish, and the probability C3 that the nail is circumscribed by a false nail piece.

[0052] Specifically, the implementation of the voting mechanism can include but is not limited to the following: first, the maximum value is selected from the probability A1, the probability A2 and the probability A3, and the corresponding classification category is recorded, the maximum value is selected from the probability B1, the probability B2 and the probability B3, and the corresponding classification category is recorded, and the maximum value is selected from the probability C1, the probability C2 and the probability C3, and the corresponding classification category is recorded, and the classification category that appears repeatedly in the three classification categories is taken as the nail shape information.

[0053] Specifically, the implementation of the weighted average mechanism can include but is not limited to the following: three weight values are determined in advance, the first weight value represents the contribution of the color feature to the classification result, the second weight value represents the contribution of the texture feature to the classification result, and the third weight value represents the contribution of the depth feature to the classification result, the probability A1, the probability B1 and the probability C1 are weighted and summed according to the three weight values to obtain a first probability, the probability A2, the probability B2 and the probability C2 are weighted and summed according to the three weight values to obtain a second probability, and the probability A3, the probability B3 and the probability C3 are weighted and summed according to the three weight values to obtain a third probability, and the maximum value is selected from the first probability, the second probability and the third probability, and the corresponding classification category is taken as the nail shape information.

[0054] It should be noted that for the above step S301 to the above step S303, the control module 120 can execute the three steps at the same time, or execute the three steps in other orders, and the present application does not make any limitation in this regard.

[0055] In some embodiments, the several polishing heads placed in the tool box 150 include a flexible sponge strip and a plurality of rigid abrasive strips of different mesh numbers, which can include a 180-mesh rigid abrasive strip, a 240-mesh rigid abrasive strip, a 400-mesh rigid abrasive strip, a 600-mesh rigid abrasive strip, and a 1200-mesh rigid abrasive strip; when the control module 120 performs the operation of controlling the robot arm 140 to select the target polishing head corresponding to the nail shape information from the tool box 150 to polish the finger nail according to the nail surface model and the pre-acquired nail type selection information, it is specifically configured to perform the following contents: When the nail shape information represents that the finger nail is coated with nail polish or externally connected with a false nail piece, the first sub-operation and the second sub-operation need to be sequentially performed; when the nail shape information represents that the finger nail is a natural nail, only the second sub-operation needs to be performed.

[0056] Specifically, the first sub-operation can but is not limited to include: trajectory planning according to the nail surface model to obtain a polishing path; and controlling the robot arm 140 to first select a first rigid abrasive strip from the tool box 150, and then performing preliminary polishing on the finger nail according to the polishing path. It should be noted that when the nail shape information represents that the finger nail is coated with nail polish, the corresponding preliminary polishing method is to remove the nail polish cover layer; when the nail shape information represents that the finger nail is externally connected with a false nail piece, the corresponding preliminary polishing method is to remove the false nail piece.

[0057] Among them, as for the content of trajectory planning according to the nail surface model to obtain a polishing path, the corresponding implementation manner can but is not limited to include: using the equidistant line method to perform trajectory planning according to the nail surface model, taking the nail boundary as the initial path, generating the next path according to the set row offset, and repeating the execution until covering the entire nail polishing area, thereby forming a complete polishing path, and subsequently the control module 120 performs pose planning on the robot arm 140 according to the polishing path, so that the robot arm 140 can polish every position in the area.

[0058] Specifically, the second sub-operation can but is not limited to include: trajectory planning according to the nail surface model and the nail type selection information to obtain a shaping path and a polishing path; controlling the robot arm 140 to first select a second rigid abrasive strip from the tool box 150, and then performing shaping on the finger nail according to the shaping path; when the finger nail shaping is completed, controlling the robot arm 140 to first select a flexible sponge strip from the tool box 150, and then performing polishing on the finger nail according to the polishing path; wherein the mesh number of the second rigid abrasive strip is greater than that of the first rigid abrasive strip, and the second rigid abrasive strip preferably adopts a 240-mesh rigid abrasive strip, and the first rigid abrasive strip preferably adopts a 180-mesh rigid abrasive strip.

[0059] The type of nail information represents that the user selects to trim the finger nail into an oval shape, a square shape, a pointed shape, or other shapes. The content of trajectory planning according to the nail surface model and the type of nail information to obtain the shaping path and the polishing path can include, but is not limited to, the following: selecting a matching nail standard model from a preset model library according to the type of nail information, and aligning and registering the selected nail standard model with the nail surface model in a three-dimensional space; subtracting the nail surface model from the aligned and registered nail standard model to obtain a residual model, and then using the isometric line method to plan a trajectory to obtain the shaping path according to the residual model; and using the isometric line method to plan a trajectory to obtain the polishing path according to the aligned and registered nail standard model.

[0060] In some embodiments, before the control module 120 controls the robot arm 140 to first select a flexible sponge strip from the tool box 150 and then polish the finger nail according to the polishing path, the control module 120 is configured to perform the following content: The robot arm 140 first selects a third rigid sand strip from the tool box 150, and then performs preliminary polishing on the finger nail according to the polishing path to enhance the fine polishing effect of the finger nail; wherein the third rigid sand strip has a higher number of meshes than the second rigid sand strip, and the third rigid sand strip preferably uses a rigid sand strip with 600 meshes.

[0061] In some embodiments, during the operation of the control module 120 controlling the robot arm 140 to remove the nail polish cover layer from the finger nail using the first rigid sand strip, the structured light depth camera can be controlled to perform real-time image acquisition, and then the real-time image acquisition is used to detect and determine whether the finger nail is left with nail polish. If not, the next process operation is performed. During the operation of the control module 120 controlling the robot arm 140 to remove the nail from the finger nail using the first rigid sand strip, the structured light depth camera can be controlled to perform real-time image acquisition, and then the real-time image acquisition is used to detect and determine whether the finger nail is left with a nail piece. If not, the next process operation is performed. During the operation of the control module 120 controlling the robot arm 140 to shape the finger nail using the second rigid sand strip, the structured light depth camera can be controlled to perform real-time image acquisition, and then the real-time image acquisition is used to detect and determine whether the finger nail meets the expected nail type. If so, the next process operation is performed. During the operation of the control module 120 controlling the robot arm 140 to polish the finger nail using the flexible sponge strip, the structured light depth camera can be controlled to perform real-time image acquisition, and then the real-time image acquisition is used to detect and determine whether the finger nail is evenly polished and smooth. If so, the nail polishing work is ended.

[0062] In some embodiments, a micro pressure sensor can be integrated in each polishing head, and when the control module 120 controls the mechanical arm 140 to operate the finger nail with the polishing head, the micro pressure sensor can feed back the contact force data to the control module 120 in real time, so that the control module 120 can dynamically adjust the propulsion amplitude and movement speed of the mechanical arm 140. Through this closed-loop control mechanism, the safety and processing accuracy of the nail polishing operation can be improved.

[0063] In some embodiments, the end of each polishing head is provided with a magnetic guide, and the end of the mechanical arm 140 is provided with an electromagnet. After the electromagnet is powered on, it is attracted to the magnetic guide, so that the end of the mechanical arm 140 is connected to the end of the polishing head. On this basis, since a plurality of receiving spaces for placing a plurality of corresponding polishing heads are opened at different height positions in the tool box 150, when the control module 120 controls the mechanical arm 140 to select the target polishing head corresponding to the nail shape information from the tool box 150, the control module 120 is specifically configured to perform the following operations: According to the position of the receiving space for placing the target polishing head in the tool box 150, the end of the mechanical arm 140 is moved to the top of the end of the target polishing head, then the end of the mechanical arm 140 is vertically lowered to align with the end of the target polishing head, and then the electromagnet provided at the end of the mechanical arm 140 is powered on to be magnetically connected to the end of the target polishing head.

[0064] In some embodiments, the full-automatic multi-process nail polishing system further comprises a disinfection box 160, and the disinfection box 160 is provided with an alcohol disinfection cabin and an ultraviolet disinfection cabin. When the control module 120 controls the mechanical arm 140 to complete the polishing operation on the finger nail with the target polishing head, the target polishing head is moved to the disinfection box 160 for disinfection by the control module 120, and then placed back in the tool box 150, which is specifically as follows: The target polishing head is first moved to the alcohol disinfection cabin for primary disinfection by the control module 120 controlling the mechanical arm 140, and the surface of the target polishing head is subjected to primary disinfection by immersion to effectively kill common bacteria and viruses. Then, the target polishing head after primary disinfection is moved to the ultraviolet disinfection cabin for secondary disinfection, and the target polishing head after primary disinfection is irradiated by ultraviolet rays of a specific wavelength for comprehensive sterilization. Finally, according to the position of the receiving space for placing the target polishing head in the tool box 150, the target polishing head after secondary disinfection is moved to the receiving space for placement, and the electromagnet provided at the end of the mechanical arm 140 is powered off to release the adsorption force, thereby completing the release of the target polishing head, so as to provide a safe and sanitary nail polishing service environment for the next user and effectively avoid the risk of cross infection.

[0065] In some embodiments, the full-automatic multi-process nail grinding system further comprises a touch display screen 170 connected with the control module 120; through the touch display screen 170, in response to user operation, nail type selection information is generated and sent to the control module 120, so that the control module 120 can control the mechanical arm 140 to select the target grinding head corresponding to the nail shape information from the tool box 150 to grind the finger nails.

[0066] Specifically, the touch display screen 170 provides a man-machine interaction interface, and a nail type selection interface is arranged on the man-machine interaction interface, and a plurality of selection controls are arranged on the nail type selection interface, which are bound to a plurality of nail grinding shapes, and when a user touches any one of the selection controls, the touch display screen 170 responds to the touch operation of the selection control, generates corresponding nail type selection information and sends it to the control module 120. In addition, a nail contour preview interface can also be arranged on the man-machine interaction interface, and the control module 120 displays the nail surface model on the nail contour preview interface after generating it. If the user finds that the nail contour is deviated, the nail surface model can be manually fine-tuned through the nail contour preview interface, or the restart control provided on the nail contour preview interface is touched, so that the control module 120 can control the image acquisition module 110 to re-acquire images and then complete the reconstruction operation of the nail surface model, to improve the accuracy of the nail grinding area.

[0067] In some embodiments, the full-automatic multi-process nail grinding system further comprises an emergency stop module 180 connected with the control module 120, and the emergency stop module 180 comprises an emergency stop button; through the emergency stop module 180, in response to user operation, an emergency stop instruction is generated and sent to the control module 120, and then through the control module 120, in response to the received emergency stop instruction, the mechanical arm 140 is controlled to perform a reset operation.

[0068] Specifically, when the user feels uncomfortable or has an emergency during the nail grinding process, the user presses the emergency stop button to send an emergency stop instruction to the control module 120, so that the control module 120 controls the mechanical arm 140 to immediately interrupt the current nail grinding operation and place the target grinding head connected at the end back to the tool box 150, and then quickly reset to the preset initial state, to avoid accidental injury to the finger nails.

[0069] In some embodiments, the full-automatic multi-process nail grinding system further comprises a dust suction module 190 connected with the control module 120; through the control module 120, a third signal is generated and sent to the dust suction module 190 when the control module 120 controls the mechanical arm 140 to select the target grinding head corresponding to the nail shape information from the tool box 150 to grind the finger nails, and the dust suction module 190 responds to the third signal to adsorb the nail powder generated in the process of grinding the finger nails. Wherein, the dust suction module 190 comprises a dust suction device, which is arranged at the bottom of the U-shaped semi-enclosed guide bracket to better perform dust suction.

[0070] In order to make the full-automatic multi-process nail grinding system better applied to actual scenes, a device box 200 is provided in the present application, the device box 200 has a front cover plate 210, at least the image acquisition module 110, the finger rest assembly 130, the mechanical arm 140, the tool box 150, the disinfection box 160 and the dust suction module 190 are integrated in the interior of the device box 200, the emergency stop module 180 is arranged on the outer wall of the device box 200, the touch display screen 170 is arranged on the outer wall of the front cover plate 210, and an opening 211 is arranged on the front cover plate 210, so that the user can insert the fingers into the interior of the device box 200 through the opening 211 and just place them on the U-shaped semi-enclosed guide bracket, which can be specifically seen from Figure 2 and Figure 3 In addition, the control module 120 can be arranged in the interior of the device box 200, or the control module 120 can be arranged as a remote control module.

[0071] The embodiments described in the present application are used to more clearly illustrate the technical solutions of the present application, and do not constitute a limitation on the technical solutions provided by the present application. Those skilled in the art can know that with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the present application are also applicable to similar technical problems.

[0072] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process or system including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes or systems.

[0073] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents a "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0074] The preferred embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the present application shall be within the scope of the present application.

Claims

1. A fully automated multi-process nail polishing system, characterized in that, The system includes an image acquisition module, a control module, a finger support assembly, a robotic arm, and a toolbox, the toolbox containing several grinding heads of different sizes; The finger support assembly generates a first signal and sends it to the control module after supporting and clamping the finger; the control module generates a second signal and sends it to the image acquisition module in response to the first signal; the image acquisition module acquires an image of the finger in response to the second signal, obtains an RGB image and a depth image, and sends them to the control module; the control module is also used to first extract the contour of the RGB image to obtain a nail contour image, then fuse the nail contour image and the depth image to generate a nail surface model, and perform feature extraction analysis on the nail contour image and the depth image respectively to determine nail shape information, and then, based on the nail surface model and pre-acquired nail shape selection information, controls the robotic arm to select a target polishing head corresponding to the nail shape information from the toolbox to polish the fingernail.

2. The fully automated multi-process armor grinding system according to claim 1, characterized in that, When the control module performs the operation of fusing the nail contour image and the depth image to generate a nail surface model, it is specifically configured as follows: The depth image is preprocessed, and then a three-dimensional coordinate transformation is performed based on the nail outline image and the preprocessed depth image to obtain a point cloud set; The point cloud set is preprocessed, and then Poisson surface reconstruction and model optimization are performed on the preprocessed point cloud set to obtain the nail surface model.

3. The fully automated multi-process armor grinding system according to claim 1, characterized in that, When the control module performs feature extraction and analysis on the nail contour image and the depth image respectively to determine nail morphology information, it is specifically configured as follows: Color features are extracted from the nail outline image to obtain nail color feature information. Then, the nail color feature information is normalized and dimensionality reduced before being input into a pre-trained first classifier model for processing to obtain a first classification result about the nail shape. Texture features are extracted from the nail outline image to obtain nail texture feature information. Then, the nail texture feature information is normalized and dimensionality reduced before being input into a pre-trained second classifier model for processing to obtain a second classification result about nail shape. Nail depth feature information is obtained from the depth image, and then the nail depth feature information is normalized and dimensionality reduced before being input into a pre-trained third classifier model for processing to obtain a third classification result about nail shape. The first classification result, the second classification result, and the third classification result are fused to obtain the nail morphology information.

4. The fully automated multi-process armor grinding system according to claim 1, characterized in that, The toolbox contains several polishing heads, including flexible sponge strips and multiple rigid sanding strips of different grits. When the control module performs the operation of selecting a target polishing head from the toolbox corresponding to the nail shape information based on the nail surface model and pre-acquired nail shape selection information, it is specifically configured as follows: When the nail morphology information indicates that the fingernail is coated with nail polish or has an external artificial nail tip, the first sub-operation and the second sub-operation are executed sequentially. When the nail morphology information indicates that the fingernail is a natural nail, the second sub-operation is executed; The first sub-operation includes: performing trajectory planning based on the nail surface model to obtain a polishing path; controlling the robotic arm to first select a first rigid sanding strip from the toolbox, and then performing preliminary polishing on the fingernail according to the polishing path, wherein the preliminary polishing method is to remove the nail polish coating or remove the nail polish. The second sub-operation includes: performing trajectory planning based on the nail surface model and the nail shape selection information to obtain a shaping path and a polishing path; controlling the robotic arm to first select a second rigid sanding strip from the toolbox, and then shaping the fingernail according to the shaping path; when the fingernail is shaped, controlling the robotic arm to first select the flexible sponge strip from the toolbox, and then polishing the fingernail according to the polishing path; The mesh size of the first rigid sandpaper is smaller than that of the second rigid sandpaper.

5. The fully automated multi-process armor grinding system according to claim 4, characterized in that, Before executing the operation of controlling the robotic arm to first select the flexible sponge strip from the toolbox and then polish the fingernail according to the polishing path, the control module is configured as follows: The robotic arm is controlled to first select a third rigid abrasive strip from the toolbox, and then perform preliminary polishing of the fingernails according to the polishing path; The mesh size of the third rigid abrasive strip is greater than that of the second rigid abrasive strip.

6. The fully automated multi-process armor grinding system according to claim 1, characterized in that, The system also includes a touch screen; the touch screen is used to generate the type A selection information in response to user operation and send it to the control module.

7. The fully automated multi-process armor grinding system according to claim 1, characterized in that, Each of the grinding heads is provided with a magnetic conductor at its end, and the end of the robotic arm is provided with an electromagnet. When the electromagnet is energized, it attracts the magnetic conductor with opposite polarities, thereby connecting the end of the robotic arm with the end of the grinding head.

8. The fully automated multi-process armor grinding system according to claim 1, characterized in that, The system also includes a disinfection box; the control module is also used to control the robotic arm to move the target polishing head to the disinfection box for disinfection and then place it back into the tool box when the fingernail polishing is finished.

9. The fully automated multi-process armor grinding system according to claim 1, characterized in that, The system also includes an emergency stop module; the emergency stop module is used to generate an emergency stop command in response to user operation and send it to the control module; the control module is also used to control the robotic arm to perform a reset operation in response to the emergency stop command.

10. The fully automated multi-process armor grinding system according to claim 1, characterized in that, The system also includes a dust collection module; the control module is further configured to generate a third signal and send it to the dust collection module while controlling the robotic arm to select a target grinding head corresponding to the nail shape information from the toolbox to grind the fingernail; the dust collection module is configured to absorb nail powder generated during the fingernail grinding process in response to the third signal.

Images