A laser micro-injection identification and positioning method

Abstract

The invention proposes a recognition and positioning method for laser micro-spray injection, which belongs to the field of medical injection technology; the method comprises the following steps: S1: configuring an RGB camera and a depth camera, and calibrating the RGB camera and the depth camera; S2: making the injection object lie on his back and keep the posture unchanged, and the RGB camera and the depth camera obtain the RGB image and the depth image of the face of the injection object; S3: aligning the RGB image to the depth image to obtain an RGB-depth fusion image; S4: performing feature point detection on the RGB-depth fusion image, and further determining each facial area, and obtaining the skin properties of each facial area; S5: performing grid segmentation on each facial area; S6: performing normal micro-spray injection on the segmented facial area grids in turn, and the depth of the micro-spray injection of each grid and the spacing between adjacent micro-spray injection sites are not exactly the same. The skin conditions of different injection objects are analyzed, and the depth and spacing of the micro-spray injection are adjusted.

Description

Technical Field

[0001] The invention relates to the technical field of medical injection, and in particular to an identification and positioning method for laser micro-injection. Background Art

[0002] There are two main types of beauty and skin care: surface application and intradermal administration. Common surface application is to apply creams and other products on the surface of the skin and massage. This method has low absorption efficiency and limited effect. Intradermal administration is to directly use injections to deliver specific drugs or nutrients through the stratum corneum and epidermis on the surface of the skin, directly to the dermis, where the effective ingredients diffuse and absorb to achieve the desired effect. The common method of intradermal administration is to use a slender needle to puncture the skin surface for injection. It is not only very dependent on the operator's experience, but also difficult to control the injection depth and the amount of the drug. In addition, the injection will cause a tingling sensation to the injected object. Repeated use of the needle can easily lead to skin infection. The skin conditions of different injection objects are different, resulting in very low efficiency of manual injection and poor user experience.

[0003] The Chinese invention patent with the patent publication number CN118001511A discloses an injection system provided by the applicant that uses a power system to eject drugs into subcutaneous tissue at high speed. The laser emitter drives the kinetic energy chamber to generate pressure, and drives the drug storage chamber to spray on the skin surface of the injection object. However, this method has some problems. The eyelids, eye sockets, nose and lips are not suitable for needle-free injection because of thin skin. The spacing of the injection ports of the injection assembly is fixed. The diffusion of drugs under the skin is not taken into account, and the resistance of skin of different properties to drugs is not taken into account. Therefore, it is necessary to provide a method for identifying and positioning laser micro-injection, identify and distinguish the facial areas of laser micro-injection needle-free injection objects, provide targeted injection strategies for different areas, and improve the injection effect. Summary of the invention

[0004] In view of this, the present invention proposes a laser micro-injection identification and positioning method that fully considers the skin conditions of people of different ages and different living habits, divides different areas of the face by feature points, considers the resistance of different skin conditions, and changes the depth of micro-injection and the injection position interval according to the skin properties.

[0005] The present invention provides a method for identifying and positioning laser micro-injection, comprising the following steps:

[0006] S1: configure the RGB camera and depth camera, and calibrate the RGB camera and depth camera;

[0007] S2: The injection subject is made to lie on his back and keep the posture unchanged, and the RGB camera and the depth camera obtain the RGB image and the depth image of the injection subject's face;

[0008] S3: Align the RGB image to the depth image to obtain an RGB-depth fused image;

[0009] S4: Detect feature points on the RGB-depth fusion image, and further determine each facial area to obtain skin properties of each facial area;

[0010] S5: Grid segmentation of each facial region;

[0011] S6: The segmented facial area grids are subjected to normal microinjection injections in sequence. The depth of microinjection injections of each grid and the spacing between adjacent microinjection injection sites are not exactly the same.

[0012] On the basis of the above technical scheme, preferably, the RGB camera and the depth camera described in step S1 are configured with the fixed end of the robotic arm, and the movable end of the robotic arm is also configured with a laser emitter, a kinetic energy chamber, a drug storage chamber and an injection assembly, the laser emitter is connected to the kinetic energy chamber, a pressure liquid is arranged in the kinetic energy chamber, the laser emitter is used to generate energy by the reaction of laser and pressure liquid; the drug storage chamber is arranged on the side of the kinetic energy chamber away from the laser emitter, and is used to store drugs; a power transmission device is arranged between the drug storage chamber and the kinetic energy chamber, the power transmission device transfers the energy generated by the reaction into the drug storage chamber, applies pulse pressure to the drug in the drug storage chamber, and injects the drug into the dermis of the skin through the epidermis in a non-contact manner.

[0013] On the basis of the above technical scheme, preferably, the content of step S3 is: according to the calibration relationship between the RGB camera and the depth camera in step S1, each 2D pixel point in the depth image is converted into the world coordinate system to obtain the 3D point corresponding to each pixel of the depth image; each 3D point converted to the world coordinate system is projected onto the RGB image to obtain the coordinates of the 3D point on the RGB image, the RGB three-channel values ​​corresponding to the projection coordinates of the 3D point are extracted, and the RGB three-channel values ​​are assigned to each 3D point in the world coordinate system to obtain the registered RGB-depth fusion image.

[0014] Preferably, the feature point detection of the RGB-depth fusion image described in step S4 is to input the RGB-depth fusion image into a pre-trained deep learning network CNN model, and the deep learning network CNN model uses a public facial recognition data set to standardize the image so that the input image has a consistent size. The deep learning network CNN model includes at least one convolutional layer, at least one pooling layer and a fully connected layer, at least one convolutional layer is used to extract local features in the input RGB-depth fusion image; at least one pooling layer is used for downsampling; the feature maps of the fully connected layer, at least one convolutional layer and at least one pooling layer are mapped to the final feature point classification result, and Triplet is used. LOSS is used as the loss function; the model of the deep learning network CNN outputs the feature points of the eyes, eyebrows, nose and lips in the RGB-depth fusion image; among them, the feature points of the eyes are the pixels at the corners of both eyes, and several pixels on the eyelid contour; the feature points of the eyebrows are the pixels corresponding to the brow head, brow peak and eyebrow tail of the two eyebrows; the feature points of the nose are the pixels corresponding to the nose tip, nose wing and nose root; the feature points of the lips are the pixels corresponding to the corners of the upper and lower lips, the midpoint of the lower lip, the lip bow and lip valley of the upper lip; the feature points of the cheeks are the pixels corresponding to the maximum depth value and the minimum depth value in the enclosed area of ​​the chin, temples, outer lips, outer nose and below the eyes.

[0015] More preferably, the further determining of each facial region in step S4 is to divide the boundaries of each facial region by feature points:

[0016] 1) Forehead area: The curve at the upper end of the eyebrows formed by the feature points of the two eyebrows is used as the lower boundary, the hairline is used as the upper boundary, and the area between the upper boundary and the lower boundary is the forehead area;

[0017] 2) Lip area: The area inside the closed area where the feature points of the lips are fitted is the lip area;

[0018] 3) Cheek area: The cheek area is the area enclosed by the jaw, temples, outer lips, outer nose and the contours below the eyes;

[0019] 4) Chin area: The area just below the corners of the lips is the chin area;

[0020] 5) Eye socket area: the area inside the closed area enclosed by the feature points of the eye is the eye socket area;

[0021] 6) The closed area formed by the feature points of the nose is the nose area; the inner area of ​​the eye socket area, the inner area of ​​the eyebrow, and the area on the side of the nose close to the lips are discarded.

[0022] More preferably, the step of obtaining the skin properties of each facial area in step S4 is to obtain the water content M of the corresponding facial area after determining each facial area. 1 Value, oil secretion level M 2 The value and number of pores M 3 The value of predicts the type of facial region according to the following model: P(i) is the probability that the current facial region is skin type i, i∈j, j=[1, 2, ..., n], n represents the number of skin types, n=3, i=1 represents oily skin, i=2 represents neutral skin, and i=3 represents dry skin; β 0i , β 1i , β 2i and β 3i With β 0j , β 1j , β 2j and β 3j is the predicted regression coefficient; according to the maximum value of P(i), it is confirmed that the skin type of the current facial area is one of dry, oily or neutral; wherein, the water content M of the facial area 1 It is obtained by measuring the relative dielectric constant ε of the current facial area, according to the formula Find, ε 1 =81,ε 2 =7; Oil secretion level M 2 The average sebum secretion of the current facial area is obtained at least 20 minutes after cleansing, in mg / cm 2 ; Number of pores M 3 It is the average number of pores with a diameter of more than 50 μm per unit area in the facial area, in units of pores / cm 2 .

[0023] Further preferably, the mesh segmentation of each facial region in step S5 is performed by triangular mesh segmentation according to the following rules:

[0024] 1) Obtain feature points corresponding to each facial region, including pixel points corresponding to the boundaries of each facial region, and pixel points corresponding to maximum depth values ​​and minimum depth values ​​inside the facial region; construct a convex polygon with pixel points at the boundaries of the facial region;

[0025] 2) constructing a triangular mesh at the boundary of the convex polygon, selecting at least one discrete pixel point from the boundary, and selecting at least one pixel point from the interior of the facial region, so that the sum of the differences between any two internal angles of the constructed triangular mesh is minimized, and the circumscribed virtual circle of the triangular mesh does not contain any vertex of an adjacent triangular mesh;

[0026] 3) After the triangular mesh is formed, the edge of the current triangular mesh is used as a reference to further select another pixel point inside the polygon to construct an adjacent triangular mesh; if the triangular mesh contains a feature point, a triangular mesh is further constructed inside the triangular mesh with the feature point as a vertex;

[0027] 4) Clipping the border of the triangular mesh constructed from the border of the convex polygon so that the edges of the triangular mesh are all located inside the convex polygon;

[0028] 5) Smooth the vertex positions of all triangular meshes so that the vertex of each triangular mesh is located at the geometric center of the adjacent vertices.

[0029] More preferably, the depth of the micro-spray injection of each grid in step S6 is calculated according to the following formula: Where P is the pressure of micro-injection, in Pa; α is the distance coefficient, h 1 is the actual distance from the injection assembly to the skin, h 0 is the reference distance from the spray assembly to the skin, in centimeters; ρ is the density of the drug, in kilograms per meter 3 ; η is the dynamic viscosity coefficient of the drug, dimensionless; R is the skin resistance coefficient, R = [R T +R A +R E +(R L0 / R L )]k, k is the skin thickness coefficient, R T is the skin type parameter, R T =i; R A is the age parameter, A 0 is the current age of the injected subject, c 1 and c 2 is the fitting coefficient; R E is the environmental impact parameter, E l is the average daily duration of exposure of the injected subjects to the skin damage environment, ω l is the weight of the skin damage environment, l = 1, 2, ..., N; R L0 / R L is the parameter of unhealthy lifestyle, R L0 is the average number of unhealthy lifestyle behaviors per week, R L The recommended number of unhealthy lifestyle behaviors decreases with age. A is the reference age, A = 40

[0030] In a further preferred embodiment, the distance between adjacent micro-spray injection sites in step S6 is calculated by first calculating the diffusion radius r of the micro-spray injection using the following formula: Then, the spacing between adjacent micro-spray injection sites is obtained by the following formula: DIS=min{2r, 2cm}, where min{·} is the minimum value operation.

[0031] In a further preferred embodiment, the volume of the drug injected in a single microinjection is 0.3 μL, and the initial position of the microinjection is the center or vertex position of any triangular mesh in the facial area.

[0032] The laser micro-injection identification and positioning method provided by the present invention has the following beneficial effects compared with the prior art:

[0033] (1) The present invention obtains the minimum measurement unit for micro-spray injection, i.e., a grid, through reasonable modeling, feature point output, facial area partitioning, and gridding of different facial areas; then micro-spray injection is performed one by one along the grid to achieve non-contact skin care. Since the influence of environmental factors, age factors, living habits, etc. is fully considered, the depth of micro-spray injection is reasonably adjusted, which can maximize the effect of injection and reduce the discomfort of the injected subject;

[0034] (2) According to the age, skin condition and other information of different injection subjects, the diffusion range based on the depth of microspray injection was reasonably planned, and the spacing between adjacent microspray injection sites was further designed, so that precise injection of drugs can be achieved, and the effect of microspray injection can be further improved without increasing the amount of drug used. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 The present invention is a flowchart of the steps of a method for identifying and positioning laser micro-injection. DETAILED DESCRIPTION

[0037] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In the existing laser micro-spray beauty process, the spacing of the spray ports of the spray assembly is fixed, and the diffusion of the drug under the skin is not taken into account, nor is the resistance of the skin of different properties to the drug. In view of this, the present invention provides a laser micro-spray injection identification and positioning method, comprising the following steps:

[0039] S1: Configure the RGB camera and depth camera, and calibrate the RGB camera and depth camera.

[0040] The RGB camera and the depth camera described in step S1 are configured with the fixed end of the robotic arm. The movable end of the robotic arm is also configured with a laser emitter, a kinetic energy chamber, a drug storage chamber and an injection assembly. The laser emitter is connected to the kinetic energy chamber. The kinetic energy chamber is provided with a pressure liquid. The laser emitter is used to generate energy by reacting the laser and the pressure liquid. The drug storage chamber is arranged on the side of the kinetic energy chamber away from the laser emitter, and is used to store drugs. A power transmission device is arranged between the drug storage chamber and the kinetic energy chamber. The power transmission device transmits the energy generated by the reaction into the drug storage chamber, applies pulse pressure to the drug in the drug storage chamber, and injects the drug into the dermis of the skin through the epidermis in a non-contact manner.

[0041] The combined structure of the laser emitter, kinetic energy chamber, drug storage chamber and injection assembly can refer to the applicant's previously disclosed invention patent application CN118001511A, the difference being that the injection assembly of the present application is either only one; if multiple injection assemblies need to be set at the same time, the spacing between adjacent injection assemblies is adjustable to meet the precise adjustment of multiple injection sites.

[0042] The purpose of calibrating the RGB camera and the depth camera is to obtain the rotation matrix and translation matrix between the two cameras' respective planar image coordinate systems, so as to subsequently obtain the three-dimensional coordinates of each pixel in the world coordinate system.

[0043] S2: The injection subject is made to lie on his back and keep the posture unchanged, and the RGB camera and the depth camera obtain the RGB image and the depth image of the injection subject's face.

[0044] S3: Align the RGB image to the depth image to obtain an RGB-depth fused image.

[0045] The specific content is as follows: according to the calibration relationship between the RGB camera and the depth camera in step S1, each 2D pixel point in the depth image is converted to the world coordinate system to obtain the 3D point corresponding to each pixel of the depth image; each 3D point converted to the world coordinate system is projected onto the RGB image to obtain the coordinates of the 3D point on the RGB image, the RGB three-channel values ​​corresponding to the projection coordinates of the 3D point are extracted, and the RGB three-channel values ​​are assigned to each 3D point in the world coordinate system to obtain the registered RGB-depth fusion image. The registered depth image includes not only the coordinates in the image coordinate system converted to the real position in the world coordinate system, but also records the distance from the current 3D point to the depth camera. Through distance conversion, the actual distance from the 3D point corresponding to the face to the injection assembly can be further obtained. Usually, the pressure of micro-spray injection is constant, and the actual distance from the 3D point corresponding to the face to the injection assembly will affect the depth of the micro-spray injection drug entering the skin. Therefore, obtaining the real three-dimensional coordinates of the facial skin is very important for the subsequent calculation of the injection depth.

[0046] S4: Perform feature point detection on the RGB-depth fusion image, and further determine each facial area to obtain skin properties of each facial area.

[0047] The feature point detection is to input the RGB-depth fusion image into a pre-trained deep learning network CNN model. The deep learning network CNN model uses a public facial recognition data set to standardize the image so that the input image has a consistent size. The deep learning network CNN model includes at least one convolutional layer, at least one pooling layer and a fully connected layer. At least one convolutional layer is used to extract local features in the input RGB-depth fusion image; at least one pooling layer is used for downsampling; the features of the fully connected layer, at least one convolutional layer and at least one pooling layer are mapped to the final feature point classification result, using Triplet LOSS is used as the loss function; the feature points of the eyes, eyebrows, nose and lips in the RGB-depth fusion image are output through the model of the deep learning network CNN; among them, the feature points of the eyes are the pixels at the corners of the eyes and several pixels on the eyelid contour; the feature points of the eyebrows are the pixels corresponding to the eyebrows, brow peaks and eyebrow tails of the two eyebrows; the feature points of the nose are the pixels corresponding to the nose tip, nose wings and nose roots; the feature points of the lips are the pixels corresponding to the corners of the upper and lower lips, the midpoint of the lower lip, the lip bow and lip valley of the upper lip; the feature points of the cheeks are the pixels corresponding to the maximum depth value and the minimum depth value in the enclosed area of ​​the jaw, temples, the outer side of the lips, the outer side of the nose and the bottom of the eyes. The feature points are selected for accurate regional separation as the boundaries of different facial regions. The model of the deep learning network CNN adopts a conventional model structure. The network structure diagram and implementation code are open source and easy to obtain, so I will not repeat them here.

[0048] After obtaining the feature points of the RGB-depth fusion image output by the deep learning network CNN model, each facial area is further determined based on the feature points, and the boundaries of each facial area are divided by the feature points:

[0049] 1) Forehead area: The curve at the upper end of the eyebrows formed by the feature points of the two eyebrows is used as the lower boundary, the hairline is used as the upper boundary, and the area between the upper boundary and the lower boundary is the forehead area; the disconnected area between the curves at the upper end of the eyebrows can be directly connected with a line segment.

[0050] 2) Lip area: The area inside the closed area where the feature points of the lips are fitted is the lip area; the lip area is an area formed by connecting multiple curves in sequence.

[0051] 3) Cheek area: The area enclosed by the jaw, temples, outer side of the lips, outer side of the nose and the contour below the eyes is taken as the cheek area; the cheek area is relatively large, and usually after determining other areas, the remaining area is obtained according to Boolean operations as the cheek area.

[0052] 4) Chin area: The area just below the corners of the lips is the chin area.

[0053] 5) Eye socket area: The inside of the closed area enclosed by the feature points of the eye is the eye socket area.

[0054] 6) The closed area formed by the characteristic points of the nose is the nose area; the inner area of ​​the eye socket area, the inner area of ​​the eyebrows, and the area near the lips of the nose are discarded. The discarded areas are because the skin is thin or the important organs such as the internal eyes are not suitable for micro-spray injection, so the area is excluded.

[0055] Skin properties usually include oily, neutral and dry. Oily skin is characterized by more sebum secretion and thicker surface oil. Oily skin is usually softer, and more sebum makes the skin more elastic.

[0056] Neutral skin has a balanced amount of water and oil and is in a relatively stable condition, neither too soft nor too hard. It reacts normally to external pressure or injections.

[0057] Dry skin has less sebum secretion, a thicker stratum corneum, and the epidermis is usually dry and easily flaky, with poor elasticity.

[0058] After determining each facial area, the skin properties of each facial area are obtained, specifically, the water content M of the corresponding facial area is obtained respectively. 1 Value, oil secretion level M 2 The value and number of pores M 3 The value of predicts the type of facial region according to the following model: P(i) is the probability that the current facial region is skin type i, i∈j, j=[1, 2, ..., n], n represents the number of skin types, n=3, i=1 represents oily skin, i=2 represents neutral skin, and i=3 represents dry skin; β 0i , β 1i , β 2i and β 3i With β 0j , β 1j , β 2j and β 3j is the predicted regression coefficient; according to the maximum value of P(i), it is confirmed that the skin type of the current facial area is one of dry, oily or neutral; wherein, the water content M of the facial area 1 It is obtained by measuring the relative dielectric constant ε of the current facial area, according to the formula Find, ε 1 =81,ε 2 =7; Oil secretion level M 2 The average sebum secretion of the current facial area is obtained at least 20 minutes after cleansing, in mg / cm 2 ; Number of pores M 3 It is the average number of pores with a diameter of more than 50 μm per unit area in the facial area, in units of pores / cm 2 The values ​​of the above prediction regression coefficients are set by default or empirical values ​​and are obtained through multiple actual calculations covering all skin types as much as possible.

[0059] The skin water content, sebum secretion and number of large pores are used to evaluate the possible skin types of different facial areas of the current injection target. 1 In the calculation formula of 1 =81 corresponds to pure water, relative dielectric constant ε 2 =7 corresponds to the non-water components in the skin, and the actual water content of the skin can be obtained by calculation; the relative dielectric constant ε of the current facial area is measured using CM825, measured by contact with the skin.

[0060] S5: Perform grid segmentation on each facial region.

[0061] Further preferably, the mesh segmentation of each facial region in step S5 is performed by triangular mesh segmentation according to the following rules:

[0062] 1) Obtain feature points corresponding to each facial region, including pixel points corresponding to the boundaries of each facial region, and pixel points corresponding to maximum depth values ​​and minimum depth values ​​inside the facial region; construct a convex polygon with pixel points at the boundaries of the facial region;

[0063] 2) constructing a triangular mesh at the boundary of the convex polygon, selecting at least one discrete pixel point from the boundary, and selecting at least one pixel point from the interior of the facial region, so that the sum of the differences between any two internal angles of the constructed triangular mesh is minimized, and the circumscribed virtual circle of the triangular mesh does not contain any vertex of an adjacent triangular mesh;

[0064] 3) After the triangular mesh is formed, the edge of the current triangular mesh is used as a reference to further select another pixel point inside the polygon to construct an adjacent triangular mesh; if the triangular mesh contains a feature point, a triangular mesh is further constructed inside the triangular mesh with the feature point as a vertex;

[0065] 4) Clipping the border of the triangular mesh constructed from the border of the convex polygon so that the edges of the triangular mesh are all located inside the convex polygon;

[0066] 5) Smooth the vertex positions of all triangular meshes so that the vertex of each triangular mesh is located at the geometric center of the adjacent vertices.

[0067] The rule of the above-mentioned component triangular mesh vertices is to make the triangular mesh at non-feature points as similar to uniformly distributed regular triangles as possible, and to appropriately densely arrange the triangular mesh at the feature points in the non-boundary area to improve the accuracy of the local contour.

[0068] S6: The segmented facial area grids are subjected to normal microinjection injections in sequence. The depth of microinjection injections of each grid and the spacing between adjacent microinjection injection sites are not exactly the same.

[0069] The depth of micro-injection of each grid is calculated according to the following formula: Where P is the pressure of micro-injection, in Pa; α is the distance coefficient, h 1 is the actual distance from the injection assembly to the skin, h 0 is the reference distance from the spray assembly to the skin, in centimeters; ρ is the density of the drug, in kilograms per meter 3 ; η is the dynamic viscosity coefficient of the drug, dimensionless; R is the skin resistance coefficient, R = [R T +R A +R E +(R L0 / R L )]k, k is the skin thickness coefficient, R T is the skin type parameter, R T =i; R A is the age parameter, A 0 is the current age of the injected subject, c1 and c 2 is the fitting coefficient; R E is the environmental impact parameter, E l is the average daily duration of exposure of the injected subjects to the skin damage environment, ω l is the weight of the skin damage environment, l = 1, 2, ..., N; R L0 / R L is the parameter of unhealthy lifestyle, R L0 is the average number of unhealthy lifestyle behaviors per week, R L The recommended number of unhealthy lifestyle behaviors decreases with age. A is the reference age, A=40.

[0070] When the diameter of the jet outlet of the jet assembly remains unchanged, skin resistance is a variable factor that is affected by many factors. Skin type, age, skin damage environment and bad living habits can cause repairable or irreparable damage to the skin. The effect of age on skin resistance is nonlinear, and the age parameter R A The formula is fitted; working in an outdoor skin-damaging environment for a long time, such as strong ultraviolet rays, high temperature, high humidity and air pollution, will cause the skin of the injected object to suffer certain damage during exposure; the recommended number of unhealthy lifestyles that decays with age is the maximum number of unhealthy lifestyles recommended at the corresponding age, which decays rapidly after the age of 40. Unhealthy lifestyles include the accumulation of unhealthy lifestyles such as smoking, drinking, and staying up late.

[0071] The spacing between adjacent micro-injection sites is calculated by first calculating the diffusion radius r of the micro-injection using the following formula: Then the spacing between adjacent micro-spray injection sites is obtained by the following formula: DIS = min{2r, 2cm}, min{·} is the minimum value operation. The above formula means that the smaller of 2 times the diffusion radius and 2cm is selected as the spacing between adjacent micro-spray injection sites. Since the diffusion capacity of different skin types for drugs is taken into account, a reasonable spacing is set to help take into account both the effect of injection and the effect of injection.

[0072] Since the pressure of microspray injection is fixed, in this embodiment, the volume of the drug injected by a single microspray injection is preferably set to 0.3 μL, and the initial position of the microspray injection is the center or vertex position of any triangular mesh in the facial area.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying and positioning laser microinjection, characterized in that: The steps include: S1: configure the RGB camera and depth camera, and calibrate the RGB camera and depth camera; S2: The injection subject is made to lie on his back and keep the posture unchanged, and the RGB camera and the depth camera obtain the RGB image and the depth image of the injection subject's face; S3: Align the RGB image to the depth image to obtain an RGB-depth fused image; S4: Detect feature points on the RGB-depth fusion image, and further determine each facial area to obtain skin properties of each facial area; S5: Grid segmentation of each facial region; S6: The segmented facial area grids are sequentially micro-injected in a normal direction, and the depth of micro-injection of each grid and the spacing between adjacent micro-injection sites are not exactly the same; The depth of the micro-spray injection of each grid in step S6 is calculated according to the following formula: Where P is the pressure of micro-injection, in Pa; α is the distance coefficient, h1 is the actual distance from the injection assembly to the skin, h0 is the reference distance from the injection assembly to the skin, in centimeters; ρ is the density of the drug, in kilograms / meter 3 ; η is the dynamic viscosity coefficient of the drug, dimensionless; R is the skin resistance coefficient, R = [R T +R A +R E +(R L0 / R L )]k, k is the skin thickness coefficient, i=1 indicates oily skin, i=2 indicates normal skin, and i=3 indicates dry skin; R T is the skin type parameter, R T =i; R A is the age parameter, A0 is the current age of the injected subject, c1 and c2 are fitting coefficients; R E is the environmental impact parameter, E l is the average daily duration of exposure of the injected subjects to the skin damage environment, ω l is the weight of the skin damage environment, l = 1, 2, ..., N; R L0 / R L is the parameter of unhealthy lifestyle, R L0 is the average number of unhealthy lifestyle behaviors per week, R L The recommended number of unhealthy lifestyle behaviors decreases with age. A is the reference age, A = 40; The spacing between adjacent micro-spray injection sites in step S6 is calculated by first calculating the diffusion radius r of the micro-spray injection using the following formula: Then, the spacing between adjacent micro-spray injection sites is obtained by the following formula: DIS=min{2r, 2cm}, where min{·} is the minimum value operation.

2. The identification and positioning method of laser microjet injection according to claim 1, characterized in that: The RGB camera and the depth camera described in step S1 are arranged at the fixed end of the robotic arm. The movable end of the robotic arm is also arranged with a laser emitter, a kinetic energy chamber, a drug storage chamber and an injection assembly. The laser emitter is connected to the kinetic energy chamber. A pressure liquid is arranged in the kinetic energy chamber. The laser emitter is used to generate energy by reacting the laser and the pressure liquid. The drug storage chamber is arranged on the side of the kinetic energy chamber away from the laser emitter, and is used to store drugs. A power transmission device is arranged between the drug storage chamber and the kinetic energy chamber. The power transmission device transmits the energy generated by the reaction into the drug storage chamber, applies pulse pressure to the drug in the drug storage chamber, and injects the drug into the dermis of the skin through the epidermis in a non-contact manner.

3. The identification and positioning method of laser microjet injection according to claim 1, characterized in that: The content of step S3 is: according to the calibration relationship between the RGB camera and the depth camera in step S1, each 2D pixel point in the depth image is converted to the world coordinate system to obtain the 3D point corresponding to each pixel of the depth image; each 3D point converted to the world coordinate system is projected onto the RGB image to obtain the coordinates of the 3D point on the RGB image, the RGB three-channel values ​​corresponding to the projection coordinates of the 3D point are extracted, and the RGB three-channel values ​​are assigned to each 3D point in the world coordinate system to obtain the registered RGB-depth fusion image.

4. The identification and positioning method of laser microjet injection according to claim 3, characterized in that: The feature point detection of the RGB-depth fusion image described in step S4 is to input the RGB-depth fusion image into a pre-trained deep learning network CNN model. The deep learning network CNN model uses a public facial recognition dataset to standardize the image so that the input image has a consistent size. The deep learning network CNN model includes at least one convolutional layer, at least one pooling layer and a fully connected layer. At least one convolutional layer is used to extract local features in the input RGB-depth fusion image; at least one pooling layer is used for downsampling; the features of the fully connected layer, at least one convolutional layer and at least one pooling layer are mapped to the final feature point classification result, and Triplet is used. LOSS is used as the loss function; the model of the deep learning network CNN outputs the feature points of the eyes, eyebrows, nose and lips in the RGB-depth fusion image; among them, the feature points of the eyes are the pixels at the corners of both eyes, and several pixels on the eyelid contour; the feature points of the eyebrows are the pixels corresponding to the brow head, brow peak and eyebrow tail of the two eyebrows; the feature points of the nose are the pixels corresponding to the nose tip, nose wing and nose root; the feature points of the lips are the pixels corresponding to the corners of the upper and lower lips, the midpoint of the lower lip, the lip bow and lip valley of the upper lip; the feature points of the cheeks are the pixels corresponding to the maximum depth value and the minimum depth value in the enclosed area of ​​the chin, temples, outer lips, outer nose and below the eyes.

5. The identification and positioning method of laser microjet injection according to claim 4, characterized in that: The further determination of each facial region in step S4 is to divide the boundaries of each facial region by feature points: 1) Forehead area: The curve at the upper end of the eyebrows formed by the feature points of the two eyebrows is used as the lower boundary, the hairline is used as the upper boundary, and the area between the upper boundary and the lower boundary is the forehead area; 2) Lip area: The area inside the closed area where the feature points of the lips are fitted is the lip area; 3) Cheek area: The cheek area is the area enclosed by the jaw, temples, outer lips, outer nose and the contours below the eyes; 4) Chin area: The area just below the corners of the lips is the chin area; 5) Eye socket area: the area inside the closed area enclosed by the feature points of the eye is the eye socket area; 6) The closed area formed by the feature points of the nose is the nose area; the inner area of ​​the eye socket area, the inner area of ​​the eyebrow, and the area on the side of the nose close to the lips are discarded.

6. The identification and positioning method of laser microjet injection according to claim 5, characterized in that: The step S4 of obtaining the skin properties of each facial area is to obtain the values ​​of the water content M1, the oil secretion level M2 and the number of pores M3 of the corresponding facial area after determining each facial area, and predict the type of the facial area according to the following model: P(i) is the probability that the current facial region is of skin type i, i∈j, j=[1, 2, ..., n], n represents the number of skin types, n=3, i=1 represents oily skin, i=2 represents neutral skin, and i=3 represents dry skin; β 0i , β 1i , β 2i and β 3i With β 0j , β 1j , β 2j and β 3j is the predicted regression coefficient; according to the maximum value of P(i), it is confirmed that the skin type of the current facial area is one of dry, oily or neutral; wherein, the water content M1 of the facial area is obtained by measuring the relative dielectric constant ε of the current facial area, according to the formula It is found that ε1 = 81, ε2 = 7; the oil secretion level M2 is the average sebum secretion of the current facial area obtained after at least 20 minutes of cleansing, in mg / cm 2 The number of pores M3 is the average number of pores with a diameter of more than 50 μm per unit area in the facial area, in units of pores / cm 2 .

7. The identification and positioning method of laser microjet injection according to claim 6, characterized in that: The mesh segmentation of each facial region in step S5 is performed by dividing the triangular mesh according to the following rules: 1) Obtain feature points corresponding to each facial region, including pixel points corresponding to the boundaries of each facial region, and pixel points corresponding to maximum depth values ​​and minimum depth values ​​inside the facial region; construct a convex polygon with pixel points at the boundaries of the facial region; 2) constructing a triangular mesh at the boundary of the convex polygon, selecting at least one discrete pixel point from the boundary, and selecting at least one pixel point from the interior of the facial region, so that the sum of the differences between any two internal angles of the constructed triangular mesh is minimized, and the circumscribed virtual circle of the triangular mesh does not contain any vertex of an adjacent triangular mesh; 3) After the triangular mesh is formed, the edge of the current triangular mesh is used as a reference to further select another pixel point inside the polygon to construct an adjacent triangular mesh; if the triangular mesh contains a feature point, a triangular mesh is further constructed inside the triangular mesh with the feature point as a vertex; 4) Clipping the border of the triangular mesh constructed from the border of the convex polygon so that the edges of the triangular mesh are all located inside the convex polygon; 5) Smooth the vertex positions of all triangular meshes so that the vertex of each triangular mesh is located at the geometric center of the adjacent vertices.

8. The identification and positioning method of laser micro-injection according to claim 7, characterized in that: The volume of a single microinjection of drug is 0.3 μL, and the initial position of the microinjection is the center or vertex position of any triangular mesh in the facial area.

Images