Method and system for nail curing based on deep learning and spatial positioning

Abstract

The application relates to the field of nail art equipment and computer vision technology and discloses a nail curing method and system based on deep learning and spatial positioning, which comprises the following steps: when a UV-LED module irradiates a nail, an RGB-UV dual-spectrum camera is used to synchronously collect an RGB image and a UV image of the nail; the 3D contour point cloud of the nail is determined by using the RGB image and the UV image, and the 3D contour point cloud is marked with the RGB image to obtain a to-be-cured vertex set belonging to a to-be-cured area; the vector included angle between the normal vector of each to-be-cured vertex in the to-be-cured vertex set and the light source vector is calculated; after the angle-intensity control of a UV light source array is performed by using a DMD chip, the nail is subjected to adjacent LED light supplementing treatment by using the UV light source array, so that the final nail curing result of the nail is determined. The application can solve the defects of the prior art, such as the poor pertinence of the ultraviolet radiation range, the lack of a real-time virtual try-on function and the incomplete curing of the curved surface edge.

Description

Technical Field

[0001] This invention relates to a nail polish solidification method and system based on deep learning and spatial positioning, belonging to the fields of nail art equipment and computer vision technology. Background Technology

[0002] UV curing of nail polish is a key process that triggers the polymerization reaction of photosensitive resin by ultraviolet light source. The current mainstream technology uses a fixed UV lamp array with timer control and relies on 2D image recognition of the gel area. Although deep learning algorithms have improved the accuracy of area recognition, the geometric characteristics of nail as a complex curved surface have not been fully considered. Fixed light source is effective in irradiating flat areas, but the actual irradiance is significantly reduced at the edges of curved surfaces due to the increased incident angle.

[0003] Traditional nail polish devices use ultraviolet light to cure gel polish on the entire back of the hand. This ultraviolet radiation covers the skin of the palm, and long-term use may lead to skin aging, pigmentation, and even the risk of cancer. Traditional nail polish devices lack real-time virtual try-on functionality, preventing users from previewing the effect and increasing decision-making costs. Existing technologies suffer from core defects such as insufficient positioning accuracy and incomplete curing at curved edges. Current nail detection algorithms rely on color segmentation or contour extraction, such as the YCrCb color space algorithm and palm masking algorithm, resulting in a high error rate in recognizing colored nails after the treatment. Key point detection is mostly based on 2D planes, such as the DeepLabV3 algorithm and the YOLOv8 key point detection algorithm, which cannot calculate the normal vector of the nail surface, leading to deviations in the irradiation angle. Furthermore, existing solutions that separately store information about the nail root or tip area do not address the deviation in the ultraviolet irradiation angle for curved nails. Fixed-angle light sources cannot match the normal vectors of each point on the nail surface, resulting in insufficient effective light intensity in edge areas and incomplete gel curing. Relying on timers rather than real-time light intensity feedback, they cannot dynamically compensate for light intensity based on surface thickness and distance, causing over-curing in planar areas.

[0004] Therefore, existing technologies have shortcomings such as insufficient targeting of ultraviolet radiation range, lack of real-time virtual try-on function, and incomplete curing of curved edges. Summary of the Invention

[0005] This invention provides a nail polish curing method and system based on deep learning and spatial positioning. Its main purpose is to solve the defects of existing technologies, such as weak targeting of ultraviolet radiation range, lack of real-time virtual try-on function, and incomplete curing of curved edges.

[0006] To achieve the above objectives, this invention provides a nail polish curing method based on deep learning and spatial localization, comprising: A UV light source array and a DMD chip are deployed inside the nail curing chamber. The UV light source array consists of multiple UV-LED modules, and each UV-LED module includes a base, a UV-LED chip, and heat dissipation fins. When the UV-LED module irradiates the nail, an RGB-UV dual-spectrum camera is used to simultaneously acquire RGB and UV images of the nail; The 3D contour point cloud of the nail is determined using the RGB image and the UV image, and the set of vertices to be solidified in the 3D contour point cloud belonging to the region to be solidified is marked using the RGB image; Calculate the angle between the normal vector and the light source vector of each vertex in the set of vertices to be solidified; Based on the vector angle, the UV light source array is controlled by the DMD chip in terms of angle and light intensity. Then, the nail is treated with adjacent LED supplemental lighting by the UV light source array to determine the final nail polish curing result.

[0007] Optionally, the deployment of a UV light source array and a DMD chip within the nail curing chamber includes: A matrix grid mounting position is set at the top inside the nail curing chamber, and the base is deployed at equal intervals on the matrix grid mounting position; After reserving a positioning pin on the back of the base, the UV-LED chip is encapsulated on the side end face of the base facing the center of the nail curing chamber, with the light-emitting surface of the UV-LED chip facing the center of the nail curing chamber, and the UV-LED chip is controlled to be on the same horizontal plane, so as to deploy the UV-LED chip inside the nail curing chamber. After opening heat dissipation holes in the wall of the nail curing chamber, the heat dissipation fins are positioned so that they face the outside of the wall of the nail curing chamber and are deployed on the wall. After the heat dissipation fins are deployed, the base, the UV-LED chip, and the heat dissipation fins constitute a UV-LED module; The UV-LED modules constitute a UV light source array; Deploy a DMD chip at the outlet of the UV-LED module.

[0008] Optionally, the simultaneous acquisition of RGB and UV images of the nail using an RGB-UV dual-spectrum camera includes: In the RGB-UV dual-spectrum camera, the light beam reflected from the fingernail is separated into a visible light component and an ultraviolet light component by a beam-splitting filter; The visible light component and the ultraviolet light component are focused onto different photosensitive areas of the RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the fingernail.

[0009] Optionally, determining the 3D contour point cloud of the nail using the RGB image and the UV image includes: The RGB image and the UV image are concatenated into a channel-dimensional tensor; Perform YOLO detection on the channel tensor to obtain a 2D mask of the fingernail; ConvNeXt feature extraction is performed on the nail 2D mask to obtain nail surface texture features; HRNet-W48 keypoint detection is performed on the RGB image to obtain 3D keypoints of the hand; Cross-modal attention fusion is performed on the 2D mask of the nail, the nail surface texture features, and the 3D key points of the hand to obtain the 3D contour point cloud of the nail.

[0010] Optionally, the step of using the RGB image to mark the set of vertices in the 3D contour point cloud that belong to the region to be solidified includes: The 3D contour point cloud is reconstructed using Bézier surfaces to obtain a nail surface model; The coating regions in the RGB image are encoded into binary masks using a U-Net network; The binary mask is mapped onto the nail surface model to mark the set of vertices in the nail surface model that belong to the region to be cured.

[0011] Optionally, before calculating the vector angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified, the method further includes: When reconstructing a nail surface model from a 3D contour point cloud using a Bezier surface, the surface normal vector of each vertex in the nail surface model is calculated. Traverse the surface normal vectors to filter the normal vectors belonging to the vertex to be solidified; Obtain UV-LED chips; Calculate the light source vector between the center of the emitting surface of each UV-LED chip and each vertex to be cured.

[0012] Optionally, calculating the angle between the normal vector of each vertex in the vertex set to be solidified and the light source vector includes: The angle between the vectors is obtained by performing an inverse cosine operation on the dot product of the normal vector and the light source vector using the following formula: ; in, Represents the angle between vectors. Represents the normal vector. Represents the light source vector. Represents the inverse cosine operation. This represents the dot product of vectors.

[0013] Optionally, the step of using the DMD chip to perform angle-intensity control on the UV light source array based on the vector angle includes: Find the target UV-LED chip corresponding to the smallest angle among the vector angles; The DMD chip is used to adjust the beam angle output by the target UV-LED chip to the minimum included angle, so as to control the angle of the UV light source array; Obtain the binary mask and nail surface model corresponding to the vertex to be solidified; The gel thickness at the vertex to be cured is determined by using the difference between the pixel depth value corresponding to the binary mask and the nail surface depth on the nail surface model. Extract the coordinate distance between the vertex to be cured and the target UV-LED chip; Based on the gel thickness and the coordinate distance, the target light intensity of the target UV-LED chip is calculated using the following formula: ; in, Indicates the target light intensity. Indicates the reference light intensity. Indicates the reference distance. Indicates coordinate distance. Indicates gel thickness, Indicates the empirical attenuation coefficient; The light intensity of the target UV-LED chip is adjusted to the target light intensity in order to control the light intensity of the UV light source array.

[0014] Optionally, the step of using the UV light source array to perform adjacent LED supplemental lighting on the nail includes: The UV image is multiplied by the 2D mask of the nail to obtain the gray value of the nail. When the grayscale value is lower than the preset grayscale value, the nail is subjected to adjacent LED supplementary lighting treatment by the target UV-LED chip corresponding to the vertex to be cured.

[0015] To address the aforementioned problems, this invention also provides a nail polish curing system based on deep learning and spatial positioning, the system comprising: The device deployment module is used to deploy a UV light source array and a DMD chip in a nail curing chamber. The UV light source array consists of multiple UV-LED modules, and each UV-LED module includes a base, a UV-LED chip, and heat dissipation fins. An image acquisition module is used to simultaneously acquire RGB and UV images of the nail using an RGB-UV dual-spectrum camera when the UV-LED module irradiates the nail; A vertex labeling module is used to determine the 3D contour point cloud of the nail using the RGB image and the UV image, and to label the set of vertices in the 3D contour point cloud that belong to the region to be cured using the RGB image; Angle calculation module, used to calculate the vector angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified; The nail fill light module is used to perform angle-intensity control of the UV light source array using the DMD chip based on the included vector angle, and then use the UV light source array to perform adjacent LED fill light treatment on the nail to determine the final nail curing result.

[0016] Compared to the problems described in the background technology, the embodiments of the present invention employ an RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the nail, enabling the simultaneous acquisition of RGB texture and UV reflection features in a single exposure. This eliminates the need for time-division switching or additional sensors, reducing system complexity and cost. RGB-UV images are aligned along the same optical axis and at the same pixel level, avoiding subsequent registration errors and meeting high-precision illumination requirements. The UV channel directly enhances the reflection signal of the nail polish gel, improving the robustness of the ConvNeXt texture branch in recognizing painted and damaged nails, reducing false detection rates. Simultaneous acquisition reduces the time the user's hand is held, minimizing frame time before curing. The embodiments of the present invention address the problem of existing nail detection algorithms relying on color segmentation or contour extraction, which leads to a high error rate in recognizing colored nails after manicures, by using YOLO detection, ConvNeXt feature extraction, and HRNet-W48 keypoint detection. The invention addresses the shortcomings of traditional methods, such as adding UV images to enhance the recognition of nail polish features on the nail surface. Furthermore, by reconstructing curvature details using Bézier surfaces, it solves the problem of irradiation angle deviation caused by the inability to calculate nail surface normals due to key point detection being primarily based on 2D planes. This also prepares for subsequent quantification of the optimal angle between the normals of each point and the light source, and resolves the issue of missing geometric adaptation. The invention further addresses this by controlling the UV light source array's angle-intensity based on the calculated vector angle, adjusting the steerable LEDs according to the calculated minimum angle. This ensures that the light source is perpendicularly incident on the nail's curing vertex, thereby enhancing edge light intensity. Additionally, the invention utilizes the UV light source array to provide supplemental lighting to the nail using adjacent LEDs, ensuring a total surface light intensity of no less than 120. This eliminates localized under-curing and addresses the shortcomings of existing technologies, such as insufficient targeting of ultraviolet radiation range. Furthermore, this invention addresses the lack of real-time virtual try-on functionality by determining the final nail polish curing result. Therefore, this invention solves the problems of existing technologies, including insufficient targeting of ultraviolet radiation range, lack of real-time virtual try-on functionality, and incomplete curing of curved edges. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a nail polish curing method based on deep learning and spatial positioning provided in an embodiment of the present invention. Figure 2 This is a UV-LED module diagram of a nail polish curing method based on deep learning and spatial positioning provided in an embodiment of the present invention; Figure 3 This is a system architecture diagram of a nail polish curing method based on deep learning and spatial localization provided in an embodiment of the present invention; Figure 4This is a schematic diagram of a module for implementing the nail polish curing system based on deep learning and spatial positioning, provided in an embodiment of the present invention.

[0018] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This application provides a nail polish curing method based on deep learning and spatial positioning. The execution entity of the nail polish curing method based on deep learning and spatial positioning includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application embodiment: a server, a terminal, etc. In other words, the nail polish curing method based on deep learning and spatial positioning can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0021] Reference Figure 1 The diagram shown is a flowchart illustrating a nail polish curing method based on deep learning and spatial localization according to an embodiment of the present invention. In this embodiment, the nail polish curing method based on deep learning and spatial localization includes: S1. Deploy a UV light source array and a DMD chip in the nail curing chamber. The UV light source array consists of multiple UV-LED modules, and each UV-LED module includes a base, a UV-LED chip, and heat sink fins.

[0022] In one embodiment of the present invention, the deployment of a UV light source array and a DMD chip within a nail curing chamber includes: setting a matrix grid mounting position at the top interior of the nail curing chamber and equidistantly deploying a base on the matrix grid mounting position; after pre-reserving positioning pins on the back of the base, encapsulating a UV-LED chip on the side of the base facing the center of the nail curing chamber, with the emitting surface of the UV-LED chip facing the center of the nail curing chamber, and controlling the UV-LED chip to be on the same horizontal plane, thereby deploying the UV-LED chip inside the nail curing chamber; after opening heat dissipation holes on the wall of the nail curing chamber, facing the outside of the wall of the nail curing chamber, thereby deploying the heat dissipation fins on the wall; after deploying the heat dissipation fins, the base, the UV-LED chip, and the heat dissipation fins constitute a UV-LED module; the UV-LED module constitutes a UV light source array; and a DMD chip is deployed at the outlet of the UV-LED module.

[0023] The nail curing chamber refers to a closed or semi-closed space specifically designed for nail curing, with a controllable internal environment. It houses nail equipment and facilitates the curing process. The curing chamber provides a stable curing environment, preventing external interference. The matrix grid mounting positions refer to components arranged in rows and columns to form a two-dimensional grid layout. This ensures the orderly spatial distribution of UV-LED modules, sensors, etc., allowing for uniform coverage of light and detection points within the curing chamber, avoiding localized overcrowding or gaps. Each grid corresponds to an independent mounting position. The base is a support structure used to fix the UV-LED chip and heat sink, providing mechanical support and mounting interfaces to ensure the stability and precise positioning of each component. The positioning pins are used for precise positioning. The mechanical components are typically used for fixing and alignment, ensuring the precise position of the base on the matrix grid mounting position, preventing installation deviations, and guaranteeing uniform distribution of the light source. The UV-LED chip refers to a light-emitting diode chip capable of emitting ultraviolet (UV) light, with a wavelength typically in the range of 365-405nm. As a light source, the UV-LED chip provides ultraviolet light for nail polish curing. Its emitting surface faces the center of the nail polish curing chamber, ensuring that the light can evenly illuminate the nail surface. The emitting surface refers to the emitting surface of the UV-LED chip, which is the source of light emission. The emitting surface ensures that the light can illuminate the target area, i.e., the nail surface, with a specific direction and intensity. The heat dissipation fins refer to metal plates used for heat dissipation. It typically has a large surface area for rapid heat dissipation. The heat dissipation fins prevent the UV-LED chip from being damaged by overheating during operation, ensuring stable operation of the equipment. The UV-LED module refers to an independent module composed of a base, UV-LED chip, and heat dissipation fins, capable of independently controlling the light intensity. As the basic unit of the UV light source array, the UV-LED module achieves uniform curing of the nail surface by adjusting the light intensity and prioritizing which UV-LED module is lit. The UV light source array refers to a light source system composed of UV-LED modules, which can provide all-around ultraviolet irradiation, ensuring that all parts of the nail surface receive sufficient light for uniform curing. The chamber wall refers to the outer wall of the nail curing chamber. The shell wall serves to enclose and support the internal structure, providing a mounting base for various components within the curing chamber. It also isolates the curing process from external environmental interference, ensuring a stable internal environment. The heat dissipation holes are openings in the chamber wall for air circulation and heat dissipation, providing channels for the heat dissipation fins. This allows heat generated inside the curing chamber to escape to the external environment, preventing equipment malfunctions or performance degradation due to overheating. The outer side of the chamber wall refers to the surface facing outwards from the curing chamber. The heat dissipation fins face this outer side to allow heat to dissipate more directly to the external environment, improving heat dissipation efficiency and ensuring the normal operation of the equipment inside the curing chamber. The DMD chip refers to a digital micromirror array.Digital micromirror arrays (DMIs) can cut the UV-LED beam output from the module into a sub-beam array that matches the normal of the nail surface in real time. Each micromirror acts as a small, switchable reflector. For each vertex to be cured, only the micromirror with the smallest angle between its normal and the lamp axis is turned on, while the others are turned off. Furthermore, the DMI can also cut the UV beam into the shape of a nail mask in real time.

[0024] It should be noted that the purpose of controlling the UV-LED chips to be on the same horizontal plane is to ensure that the light emitted by each UV-LED chip can evenly illuminate the nail surface, avoiding light focusing or dispersion due to the tilt of the chip's light-emitting surface, thereby ensuring the uniformity of nail polish curing.

[0025] See Figure 2 The diagram shown is a UV-LED module diagram of a nail polish curing method based on deep learning and spatial positioning provided in an embodiment of the present invention. Figure 2 In this process, the heat dissipation fins do not participate in the nail curing process; their main function is to dissipate heat during the nail curing process. The UV-LED chip is used to irradiate light onto the nails.

[0026] S2. When the UV-LED module irradiates the nail, an RGB-UV dual-spectrum camera is used to simultaneously acquire the RGB and UV images of the nail.

[0027] This invention employs an RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the nail, enabling the simultaneous acquisition of RGB texture and UV reflection features in a single exposure. This eliminates the need for time-division switching or additional sensors, reducing system complexity and cost. RGB-UV images are aligned along the same optical axis and at the same pixel level, avoiding subsequent registration errors and meeting high-precision irradiation requirements. The UV channel directly enhances the reflection signal of the nail polish gel, improving the robustness of the ConvNeXt texture branch in recognizing painted and damaged nails, reducing false detection rates. Simultaneous acquisition reduces the time the user's hand is held, minimizing frame-wide heat loss before curing.

[0028] In one embodiment of the present invention, the method of simultaneously acquiring RGB and UV images of the nail using an RGB-UV dual-spectrum camera includes: separating the reflected light beam from the nail into a visible light component and an ultraviolet light component using a spectrophotometer in the RGB-UV dual-spectrum camera; and focusing the visible light component and the ultraviolet light component onto different photosensitive areas of the RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the nail.

[0029] The RGB-UV dual-spectrum camera refers to a photosensitive unit that integrates visible light (RGB) and near-ultraviolet (UV-A, typically 365-405nm) bands on the same imaging chip. It can simultaneously output RGB and UV images in a single exposure. The RGB channel has a fixed response of 400-700nm wavelength, and the UV channel has a fixed response of 365-405nm wavelength. Both are acquired in parallel. The different photosensitive areas refer to two sets of pixel arrays that are physically non-overlapping on the imaging target surface of the RGB-UV dual-spectrum camera. The front end of the first set of pixel arrays is covered with a visible light bandpass filter, which only receives the 400-700 nm visible light component and is used to generate RGB images. The front end of the second set of pixel arrays is covered with an ultraviolet bandpass filter, which only receives the 365-405nm ultraviolet component and is used to generate UV images. The two sets of arrays are photosensitive in a single exposure, realizing the synchronous output of two spectral images in a single frame. The UV image is actually a grayscale image.

[0030] S3. Use the RGB image and the UV image to determine the 3D contour point cloud of the nail, and use the RGB image to mark the set of vertices in the 3D contour point cloud that belong to the region to be solidified.

[0031] This invention addresses the shortcomings of existing nail detection algorithms that rely on color segmentation or contour extraction, resulting in a high error rate in recognizing colored nails after manicures, by using YOLO detection, ConvNeXt feature extraction, and HRNet-W48 key point detection. Furthermore, adding UV images can enhance the recognition ability of nail polish features on the nail surface.

[0032] In one embodiment of the present invention, determining the 3D contour point cloud of the nail using the RGB image and the UV image includes: concatenating the RGB image and the UV image into a channel tensor; performing YOLO detection on the channel tensor to obtain a 2D nail mask; performing ConvNeXt feature extraction on the 2D nail mask to obtain nail surface texture features; performing HRNet-W48 keypoint detection on the RGB image to obtain 3D keypoints of the hand; and performing cross-modal attention fusion on the 2D nail mask, the nail surface texture features, and the 3D keypoints of the hand to obtain the 3D contour point cloud of the nail.

[0033] Here, the channel tensor refers to a 4-channel tensor formed by directly concatenating an RGB image (3 channels) and a UV image (1 channel) in the channel dimension. YOLO detection refers to using the YOLO series of single-stage object detection networks to perform forward inference on the above 4-channel tensor and output a pixel-level mask of the nail region. The nail 2D mask refers to a binary image output by YOLO detection with the same size as the input image, where 1 represents a pixel belonging to the nail and 0 represents the background. ConvNeXt feature extraction refers to inputting the image region corresponding to the nail 2D mask into the ConvNeXt convolutional network to extract high-dimensional texture feature vectors. Nail surface texture features refer to the high-dimensional feature tensor output by ConvNeXt used to characterize nail surface grooves and nail polish texture. HRNet-W48 keypoint detection refers to the process of converting RGB images into a single-channel object detection network. The image is input to the HRNet-W48 high-resolution network, which outputs the 3D coordinates of 21 key points of the hand. These 3D key points describe the hand's pose in 3D space. Cross-modal attention fusion involves taking nail texture features, a 2D nail mask, and the 3D hand key points as input, and weighting and fusing them through an attention mechanism to output a unified representation of the nail's 3D contour point cloud. It's important to note that YOLO, ConvNeXt, HRNet-W48, and the cross-modal attention fusion network share the same training data: RGB-UV paired samples generated by StyleGAN. The 2D nail mask output by YOLO is for segmentation tasks, using Dice Loss + Edge-aware. The model is trained and optimized using the loss function (edge-aware loss). The 3D keypoints of the hand output by HRNet-W48 belong to the keypoint task, and the model is trained and optimized using the normal vector orthogonal constraint loss. The normal vector orthogonal constraint loss is calculated as follows: for the untrained HRNet-W48 output 3D keypoints of the hand, i.e., the keypoints at the fingertips and bases, the unit vector of this keypoint is calculated. Then, the corresponding true normal vector is found in the real Bézier surface model built based on historical training data. The purpose of the normal vector orthogonal constraint loss is to keep the fingertip-base unit vector perpendicular to the true normal vector. Therefore, the dot product between the keypoint's unit vector and the true normal vector needs to be zero. The normal vector orthogonal constraint loss is precisely the loss function formed by the dot product between the keypoint's unit vector and the true normal vector. Additionally, StyleGAN is a generative adversarial network (GAN) open-sourced by NVIDIA, specifically designed to synthesize high-resolution, realistic images of faces, objects, or textures.

[0034] Furthermore, by reconstructing curvature details using Bezier surfaces, this embodiment of the invention can solve the problem of illumination angle deviation caused by the inability to calculate the normal vector of the nail surface due to the fact that key point detection is mostly based on 2D planes. It also prepares for subsequent quantization of the optimal angle between the normal vector of each point and the light source, and can solve the problem of missing geometric adaptation.

[0035] In one embodiment of the present invention, the step of using the RGB image to mark the set of vertices to be solidified in the 3D contour point cloud that belong to the region to be solidified includes: reconstructing the 3D contour point cloud using a Bezier surface to obtain a nail surface model; encoding the coating region in the RGB image into a binary mask using a U-Net network; and mapping the binary mask onto the nail surface model to mark the set of vertices to be solidified in the nail surface model that belong to the region to be solidified.

[0036] Among them, Bézier surface fitting refers to a computational method that uses the Bézier surface mathematical model to fit discrete three-dimensional data points (3D nail contour point cloud) into a continuous and smooth parametric surface. The U-Net network refers to a deep learning architecture based on convolutional neural networks (CNN), which is widely used in image segmentation tasks. It can segment and label specific regions (such as gel coatings) in the input image. The coating region refers to the area where the nail gel is applied, which usually needs to be cured. The binary mask refers to a binary image, where each pixel has a value of 0 or 1, used to represent a specific region in the image. That is, the coating region is all 1, and the uncoated region is all 0. It should be noted that the nail 2D mask is the information of the entire nail plate, while the binary mask is the information of the area where nail polish gel is applied and needs to be cured by ultraviolet light. The vertex set to be cured is the vertex in the region where the pixel value is 1.

[0037] S4. Calculate the angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified.

[0038] In one embodiment of the present invention, before calculating the vector angle between the normal vector and the light source vector of each vertex in the vertex set to be cured, the method further includes: when reconstructing the 3D contour point cloud into a nail surface model using a Bezier surface, calculating the surface normal vector of each vertex in the nail surface model; traversing the surface normal vectors to filter the normal vectors belonging to the vertex to be cured; obtaining the UV-LED chip; and calculating the light source vector between the center of the emitting surface of each UV-LED chip and each vertex to be cured.

[0039] It should be noted that each vertex to be cured will generate a light source vector between itself and the UV-LED chip. The light source vector is the vector pointing from the center of the light-emitting surface of the UV-LED chip to the vertex to be cured.

[0040] In one embodiment of the present invention, calculating the vector angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified includes: performing an inverse cosine operation on the dot product of the normal vector and the light source vector to obtain the vector angle.

[0041] As another embodiment, the step of performing an inverse cosine operation on the dot product of the normal vector and the light source vector to obtain the vector angle includes: performing an inverse cosine operation on the dot product of the normal vector and the light source vector using the following formula to obtain the vector angle: ; in, Represents the angle between vectors. Represents the normal vector. Represents the light source vector. Represents the inverse cosine operation. This represents the dot product of vectors.

[0042] S5. Based on the vector angle, after controlling the angle and light intensity of the UV light source array using the DMD chip, the UV light source array is used to perform adjacent LED supplementary lighting treatment on the nail to determine the final nail polish curing result.

[0043] In this embodiment of the invention, the UV light source array is angle-intensity controlled according to the included angle of the vector, so that the steerable LED is oriented according to the calculated minimum included angle. This ensures that the light source is perpendicularly incident on the vertex to be cured on the fingernail, thereby achieving the purpose of improving the edge light intensity.

[0044] In one embodiment of the present invention, the step of using the DMD chip to perform angle-intensity control on the UV light source array based on the vector angle includes: querying the target UV-LED chip corresponding to the smallest angle among the vector angles; adjusting the beam angle output by the target UV-LED chip to the smallest angle using the DMD chip to control the angle of the UV light source array; obtaining a binary mask and a nail surface model corresponding to the vertex to be cured; determining the gel thickness at the vertex to be cured using the difference between the pixel depth value corresponding to the binary mask and the nail surface depth on the nail surface model; extracting the coordinate distance between the vertex to be cured and the target UV-LED chip; calculating the target light intensity of the target UV-LED chip based on the gel thickness and the coordinate distance; and adjusting the emitted light intensity of the target UV-LED chip to the target light intensity to control the light intensity of the UV light source array.

[0045] As another embodiment, calculating the target light intensity of the target UV-LED chip based on the gel thickness and the coordinate distance includes: calculating the target light intensity of the target UV-LED chip using the following formula based on the gel thickness and the coordinate distance: ; in, Indicates the target light intensity. Indicates the reference light intensity. Indicates the reference distance. Indicates coordinate distance. Indicates gel thickness, This represents the empirical attenuation coefficient.

[0046] The pixel depth value refers to the depth information of each pixel in the depth image, usually a continuous numerical value representing the distance from the pixel to the camera. Pixel depth values ​​can be acquired using an infrared structured light depth camera. The depth camera can capture the depth information of each pixel to generate a depth image, meaning it can capture the depth information corresponding to each pixel in a binary mask. The nail surface depth refers to the depth value of each vertex in the nail surface model (scanning before applying the gel requires the nail substrate to determine the depth of the nail substrate under the coating), representing the distance from that vertex to the camera. The nail surface model is generated from the point cloud data generated by the infrared structured light depth camera through Bézier surface reconstruction. The depth value of each vertex is directly extracted from the point cloud data. The gel thickness refers to the thickness of the coating area (which is actually the area to be cured), i.e., the actual distance from the coating surface to the nail surface. For each vertex to be cured, the difference between its corresponding coating surface depth and nail surface depth is calculated, which is the gel thickness at that point. The empirical attenuation coefficient describes the exponential attenuation of light intensity by the gel thickness; for every 1mm increase in thickness, the light intensity is multiplied by a factor of 1. , The specific source is as follows: Prepare 50 sets of commercial nail gels of different thicknesses (0.1–2.0 mm), and use a fixed UV light source (365 nm, 150 mW / cm²). 2 Irradiate the gel, measure the transmitted light intensity, and fit the relationship between transmitted light intensity and thickness as follows: = Where β is the attenuation coefficient, The gel thickness (unit: mm) is obtained through nonlinear regression analysis. The reference light intensity It refers to the initial light intensity emitted by the UV-LED chip when there is no attenuation, the distance d is a certain reference distance, and the gel thickness t is zero.

[0047] Furthermore, in this embodiment of the invention, the nail is illuminated by adjacent LEDs using the UV light source array, ensuring that the total surface light intensity is not less than 120 lux. This eliminates localized under-curing and addresses the shortcomings of existing technologies in terms of the limited targeting of ultraviolet radiation range.

[0048] In one embodiment of the present invention, the process of using the UV light source array to perform adjacent LED supplementary lighting on the nail includes: performing a dot product between the UV image and the nail 2D mask to obtain the grayscale value of the nail; when the grayscale value is lower than a preset grayscale value, performing adjacent LED supplementary lighting on the nail through the target UV-LED chip corresponding to the vertex adjacent to the vertex to be solidified.

[0049] The preset grayscale value is obtained through Calculations show that a standard ultraviolet radiometer is used to measure ultraviolet irradiance in the camera's shooting area. For example, if the measured ultraviolet irradiance is 150... The preset grayscale value is 120 / 150*G, where G is the average grayscale value of the nail. Furthermore, since the RGB-UV multispectral camera in the nail curing chamber captures a global UV image of the skin and nail areas, this global UV image needs to be multiplied by the nail 2D mask obtained in step S3 above to obtain a UV image of only the nail area. Optionally, the process of performing neighboring LED supplementary lighting on the nail using the target UV-LED chip corresponding to the neighboring vertex of the vertex to be cured is as follows: For the vertex to be supplemented, first determine the LEDs that are adjacent to it in space. These LEDs are usually the few LEDs in the array that are closest to the vertex. For these neighboring LEDs, calculate the angle between the light source vector of each LED and the normal vector of the vertex to be supplemented. Among these neighboring LEDs, select the LED with the smallest angle as the light source for supplementary lighting. After adjusting the angle, supplement the light intensity according to the same principle as the above steps of angle-intensity control of the UV light source array based on the vector angle. It should be noted that when selecting adjacent LEDs in space, idle neighboring LEDs are preferred. Idle neighboring LEDs refer to LEDs that are not currently assigned a task (i.e., there is no target vertex being illuminated).

[0050] It should be noted that when the UV light source array irradiates the nail, firstly, the DMD chip (Digital Micromirror Array) can cut the UV beam into the shape of a nail mask in real time, so that the light spot falls only within the nail area. Secondly, if the UV beam direction is found to deviate from the nail mask area, the main control board immediately turns off the LED power. The process of detecting UV deviation from the nail mask area is as follows: acquire the collected UV image containing the skin and nail areas, acquire the nail 2D mask obtained in step S3 above, invert the logic of the nail 2D mask (1 becomes 0, 0 becomes 1) to obtain the skin area mask, compare the UV image with the skin area mask pixel by pixel, and if any pixel in the skin area in the UV image has a gray value higher than 0, it is determined that the UV has deviated.

[0051] Furthermore, embodiments of the present invention address the lack of a real-time virtual try-on function by determining the final nail polish curing result of the nail.

[0052] In one embodiment of the present invention, determining the final nail polish curing result of the nail includes: rendering the curing result between the nail polish pattern pre-stored in the nail polish curing chamber and the nail onto the display screen of the nail polish curing chamber in real time to obtain the final nail polish curing result.

[0053] The curing result refers to the result of the nail art pattern being applied to the nail, and the final nail art curing result refers to the result of the nails being displayed on the screen after the nail art has been applied.

[0054] It's important to note that during the real-time rendering of the nail art design pre-stored in the curing chamber onto the display screen, real-time texture mapping technology can be used to parametrically map the nail art design onto a Bézier surface. This means directly applying the 2D nail art design to the reconstructed nail surface model using Bézier surface texture coordinates, similar to Photoshop's texture mapping to 3D objects. Furthermore, finger movement can trigger dynamic rotation of the design, meaning the design is driven by 3D keypoints on the hand. The screen will display the corresponding nail-facing design effect based on how the finger rotates within the curing chamber. This can be achieved using AR SDK technology. This addresses the shortcomings of traditional nail art devices, such as lack of real-time virtual try-on functionality, inability to preview effects, and high decision-making costs. The real-time rendering of the patterned nail surface model is superimposed onto the camera video stream, which can be achieved using OpenGL ES real-time rendering technology. OpenGL ES real-time rendering technology is a commonly used 3D graphics rendering API that can draw 3D models onto the screen in real-time and superimpose them onto the camera image, enabling AR functionality. SDK technology is an augmented reality development kit. It uses a camera to calculate the position and angle of the hand in real time and accurately pastes virtual nail art patterns onto the fingernails. When the user moves their finger, the pattern on the screen rotates synchronously.

[0055] Furthermore, to better understand the selection criteria for the preset threshold in the above-described method for nail polish curing based on deep learning and spatial positioning, please refer to Table 1 below, which shows the implementation of the nail polish curing method based on deep learning and spatial positioning provided in an embodiment of the present invention. The selection criteria table.

[0056] Table 1 Selection Criteria ; Table 1 above clearly shows that in the nail polish curing method based on deep learning and spatial localization, Achieving the optimal balance between curing completion and safety.

[0057] Furthermore, to better understand the comparison of edge curing uniformity in the above-mentioned nail curing method based on deep learning and spatial positioning, please refer to Table 2 below, which is a comparison table of edge curing uniformity in the nail curing method based on deep learning and spatial positioning provided by an embodiment of the present invention.

[0058] Table 2 Comparison of Edge Curing Uniformity ; As can be clearly seen from Table 2 above, in the nail polish curing method based on deep learning and spatial positioning, this solution has high edge curing uniformity, which can reduce the defect of local under-curing.

[0059] Furthermore, to better understand the distribution of light intensity compensation effectiveness in the above-mentioned nail polish curing method based on deep learning and spatial positioning, please refer to Table 3 below, which is a distribution table of light intensity compensation effectiveness in the nail polish curing method based on deep learning and spatial positioning provided by an embodiment of the present invention.

[0060] Table 3 Distribution of Light Intensity Compensation Effectiveness ; Table 3 above clearly shows that the nail polish curing method based on deep learning and spatial positioning can ensure that the edge areas achieve [the desired effect]. The light intensity is increased, thereby solving the problems of insufficient positioning accuracy and incomplete curing of curved surface edges.

[0061] See Figure 3 The diagram shown is a system architecture diagram of a nail polish curing method based on deep learning and spatial localization provided in an embodiment of the present invention. Figure 3In this process, the nail point cloud is collected and the nail surface model is reconstructed. The normal vector is calculated, and then the light source is matched and the angle is adjusted to optimize the curing effect. The light intensity during the curing process is monitored and fed back in real time. The curing quality is ensured by supplementing light with nearby LEDs and adjusting the light intensity.

[0062] Furthermore, to better understand the effectiveness of the above-mentioned nail polish curing method based on deep learning and spatial positioning, please refer to Table 4 below, which shows the effectiveness of the nail polish curing method based on deep learning and spatial positioning provided in an embodiment of the present invention.

[0063] Table 4 Results Table ; As can be clearly seen from Table 4 above, the nail curing method based on deep learning and spatial positioning can reduce nail positioning error and normal vector calculation error, can selectively irradiate only the nail surface with ultraviolet light and not the skin surface, and can render the nail curing effect on the nail and display it on the display screen outside the nail curing chamber.

[0064] Compared to the problems described in the background technology, the embodiments of the present invention employ an RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the nail, enabling the simultaneous acquisition of RGB texture and UV reflection features in a single exposure. This eliminates the need for time-division switching or additional sensors, reducing system complexity and cost. RGB-UV images are aligned along the same optical axis and at the same pixel level, avoiding subsequent registration errors and meeting high-precision illumination requirements. The UV channel directly enhances the reflection signal of the nail polish gel, improving the robustness of the ConvNeXt texture branch in recognizing painted and damaged nails, reducing false detection rates. Simultaneous acquisition reduces the time the user's hand is held, minimizing frame time before curing. The embodiments of the present invention address the problem of existing nail detection algorithms relying on color segmentation or contour extraction, which leads to a high error rate in recognizing colored nails after manicures, by using YOLO detection, ConvNeXt feature extraction, and HRNet-W48 keypoint detection. The invention addresses the shortcomings of traditional methods, such as adding UV images to enhance the recognition of nail polish features on the nail surface. Furthermore, by reconstructing curvature details using Bézier surfaces, it solves the problem of irradiation angle deviation caused by the inability to calculate nail surface normals due to key point detection being primarily based on 2D planes. This also prepares for subsequent quantification of the optimal angle between the normals of each point and the light source, and resolves the issue of missing geometric adaptation. The invention further addresses this by controlling the UV light source array's angle-intensity based on the calculated vector angle, adjusting the steerable LEDs according to the calculated minimum angle. This ensures that the light source is perpendicularly incident on the nail's curing vertex, thereby enhancing edge light intensity. Additionally, the invention utilizes the UV light source array to provide supplemental lighting to the nail using adjacent LEDs, ensuring a total surface light intensity of no less than 120. This eliminates localized under-curing and addresses the shortcomings of existing technologies, such as insufficient targeting of ultraviolet radiation range. Furthermore, this invention addresses the lack of real-time virtual try-on functionality by determining the final nail polish curing result. Therefore, this invention solves the problems of existing technologies, including insufficient targeting of ultraviolet radiation range, lack of real-time virtual try-on functionality, and incomplete curing of curved edges.

[0065] like Figure 4 The diagram shown is a functional module diagram of a nail polish curing system based on deep learning and spatial positioning according to the present invention.

[0066] The nail polish curing system 400 based on deep learning and spatial positioning described in this invention can be installed in an electronic device. Depending on the functions implemented, the nail polish curing system based on deep learning and spatial positioning may include a device deployment module 401, an image acquisition module 402, a vertex marking module 403, an angle calculation module 404, and a nail lighting module 405. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0067] In this embodiment of the invention, the functions of each module / unit are as follows: The device deployment module 401 is used to deploy a UV light source array and a DMD chip in the nail curing chamber. The UV light source array is composed of multiple UV-LED modules, and the UV-LED module includes a base, a UV-LED chip, and heat dissipation fins. The image acquisition module 402 is used to simultaneously acquire RGB and UV images of the nail using an RGB-UV dual-spectrum camera when the UV-LED module irradiates the nail. The vertex marking module 403 is used to determine the 3D contour point cloud of the nail using the RGB image and the UV image, and to mark the set of vertices in the 3D contour point cloud that belong to the region to be solidified using the RGB image; The included angle calculation module 404 is used to calculate the included angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified; The nail fill light module 405 is used to perform angle-intensity control of the UV light source array using the DMD chip according to the vector angle, and then use the UV light source array to perform adjacent LED fill light treatment on the nail to determine the final nail curing result of the nail.

[0068] In detail, the modules in the deep learning and spatial positioning-based nail polish curing system 400 described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The nail polish curing method based on deep learning and spatial positioning described in the article uses the same technical means and can produce the same technical effect, so it will not be repeated here.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0070] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A nail polish curing method based on deep learning and spatial localization, characterized in that, The method includes: A UV light source array and a DMD chip are deployed inside the nail curing chamber. The UV light source array consists of multiple UV-LED modules, and each UV-LED module includes a base, a UV-LED chip, and heat dissipation fins. When the UV-LED module irradiates the nail, an RGB-UV dual-spectrum camera is used to simultaneously acquire RGB and UV images of the nail; The 3D contour point cloud of the nail is determined using the RGB image and the UV image, and the set of vertices to be solidified in the 3D contour point cloud belonging to the region to be solidified is marked using the RGB image; Calculate the angle between the normal vector and the light source vector of each vertex in the set of vertices to be solidified; Based on the vector angle, the UV light source array is controlled by the DMD chip in terms of angle and light intensity. Then, the nail is treated with adjacent LED supplemental lighting by the UV light source array to determine the final nail polish curing result.

2. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The deployment of a UV light source array and DMD chip within the nail polish curing chamber includes: A matrix grid mounting position is set at the top inside the nail curing chamber, and the base is deployed at equal intervals on the matrix grid mounting position; After reserving a positioning pin on the back of the base, the UV-LED chip is encapsulated on the side end face of the base facing the center of the nail curing chamber, with the light-emitting surface of the UV-LED chip facing the center of the nail curing chamber, and the UV-LED chip is controlled to be on the same horizontal plane, so as to deploy the UV-LED chip inside the nail curing chamber. After opening heat dissipation holes in the wall of the nail curing chamber, the heat dissipation fins are positioned so that they face the outside of the wall of the nail curing chamber and are deployed on the wall. After the heat dissipation fins are deployed, the base, the UV-LED chip, and the heat dissipation fins constitute a UV-LED module; The UV-LED modules constitute a UV light source array; Deploy a DMD chip at the outlet of the UV-LED module.

3. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The method of simultaneously acquiring RGB and UV images of the fingernail using an RGB-UV dual-spectrum camera includes: In the RGB-UV dual-spectrum camera, the light beam reflected from the fingernail is separated into a visible light component and an ultraviolet light component by a beam-splitting filter; The visible light component and the ultraviolet light component are focused onto different photosensitive areas of the RGB-UV dual-spectrum camera to simultaneously acquire RGB and UV images of the fingernail.

4. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The step of determining the 3D contour point cloud of the nail using the RGB image and the UV image includes: The RGB image and the UV image are concatenated into a channel-dimensional tensor; Perform YOLO detection on the channel tensor to obtain a 2D mask of the fingernail; ConvNeXt feature extraction is performed on the nail 2D mask to obtain nail surface texture features; HRNet-W48 keypoint detection is performed on the RGB image to obtain 3D keypoints of the hand; Cross-modal attention fusion is performed on the 2D mask of the nail, the nail surface texture features, and the 3D key points of the hand to obtain the 3D contour point cloud of the nail.

5. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The step of using the RGB image to mark the set of vertices belonging to the region to be solidified in the 3D contour point cloud includes: The 3D contour point cloud is reconstructed using Bézier surfaces to obtain a nail surface model; The coating regions in the RGB image are encoded into binary masks using a U-Net network; The binary mask is mapped onto the nail surface model to mark the set of vertices in the nail surface model that belong to the region to be cured.

6. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, Before calculating the angle between the normal vector of each vertex in the set of vertices to be solidified and the light source vector, the method further includes: When reconstructing a nail surface model from a 3D contour point cloud using a Bezier surface, the surface normal vector of each vertex in the nail surface model is calculated. Traverse the surface normal vectors to filter the normal vectors belonging to the vertex to be solidified; Obtain UV-LED chips; Calculate the light source vector between the center of the emitting surface of each UV-LED chip and each vertex to be cured.

7. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The calculation of the angle between the normal vector of each vertex in the vertex set to be solidified and the light source vector includes: The angle between the vectors is obtained by performing an inverse cosine operation on the dot product of the normal vector and the light source vector using the following formula: ； in, Represents the angle between vectors. Represents the normal vector. Represents the light source vector. Represents the inverse cosine operation. This represents the dot product of vectors.

8. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The step of controlling the UV light source array by the DMD chip based on the vector angle includes: Find the target UV-LED chip corresponding to the smallest angle among the vector angles; The DMD chip is used to adjust the beam angle output by the target UV-LED chip to the minimum included angle, so as to control the angle of the UV light source array; Obtain the binary mask and nail surface model corresponding to the vertex to be solidified; The gel thickness at the vertex to be cured is determined by using the difference between the pixel depth value corresponding to the binary mask and the nail surface depth on the nail surface model. Extract the coordinate distance between the vertex to be cured and the target UV-LED chip; Based on the gel thickness and the coordinate distance, the target light intensity of the target UV-LED chip is calculated using the following formula: ； in, Indicates the target light intensity. Indicates the reference light intensity. Indicates the reference distance. Indicates coordinate distance. Indicates gel thickness, Indicates the empirical attenuation coefficient; The light intensity of the target UV-LED chip is adjusted to the target light intensity in order to control the light intensity of the UV light source array.

9. The nail polish curing method based on deep learning and spatial positioning as described in claim 1, characterized in that, The process of using the UV light source array to provide adjacent LED supplemental lighting for the nail includes: The UV image is multiplied by the 2D mask of the nail to obtain the gray value of the nail. When the grayscale value is lower than the preset grayscale value, the nail is subjected to adjacent LED supplementary lighting treatment by the target UV-LED chip corresponding to the vertex to be cured.

10. A nail polish curing system based on deep learning and spatial positioning, characterized in that, The system includes: The device deployment module is used to deploy a UV light source array and a DMD chip in a nail curing chamber. The UV light source array consists of multiple UV-LED modules, and each UV-LED module includes a base, a UV-LED chip, and heat dissipation fins. An image acquisition module is used to simultaneously acquire RGB and UV images of the nail using an RGB-UV dual-spectrum camera when the UV-LED module irradiates the nail; A vertex labeling module is used to determine the 3D contour point cloud of the nail using the RGB image and the UV image, and to label the set of vertices in the 3D contour point cloud that belong to the region to be cured using the RGB image; Angle calculation module, used to calculate the vector angle between the normal vector and the light source vector of each vertex in the vertex set to be solidified; The nail fill light module is used to perform angle-intensity control of the UV light source array using the DMD chip based on the included vector angle, and then use the UV light source array to perform adjacent LED fill light treatment on the nail to determine the final nail curing result.

Images