A method and system for identifying hair information, and a storage medium

Abstract

The application provides a hair information recognition method, a hair information recognition system and a computer readable storage medium. The method comprises the following steps: obtaining a plurality of hair images comprising two-dimensional end point coordinates of a plurality of hairs; according to a preset constraint condition, traversing the two-dimensional end point coordinates of each hair in the first hair image and the second hair image respectively to determine a candidate hair list of each first hair in the first hair image; extracting feature descriptors of the root points and the tip points of each first hair and each second hair in the candidate hair list respectively to perform feature matching; and in response to a matching result that any second hair in the candidate hair list and the first hair belong to one hair entity, determining three-dimensional spatial information of the hair entity according to a first pixel set of the hair entity in the first hair image, a second pixel set of the hair entity in the second hair image, and relative angle information of the first hair image and the second hair image.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more particularly to a method for recognizing hair information, a system for recognizing hair information, and a computer-readable storage medium. Background Technology

[0002] As people experience increasing life pressures, hair loss is becoming more prevalent and significantly impacts their appearance. Hair transplantation is currently the mainstream solution for treating hair loss. It involves transplanting permanent hair follicles from the back of the head to the forehead, where hair loss is most common, restoring the hairline to a level that does not affect aesthetics.

[0003] In current hair transplant surgeries, hair transplant robots are typically used to assist doctors in the procedure. By automating the hair follicle extraction process, they effectively reduce labor costs and surgical time, and minimize damage to the hair follicles. Existing hair transplant robots typically utilize traditional image processing algorithms such as threshold segmentation and corner detection to locate hairs. However, because hair conditions vary from person to person, current technologies generally require manual adjustment of parameters such as thresholds to adapt to different hair conditions. Therefore, they are highly dependent on the operator's theoretical knowledge and practical experience, and suffer from low detection accuracy. Furthermore, the feature matching algorithms used by existing hair transplant robots can only match single feature points, failing to perform global matching of multiple feature points across the entire hair follicle. Consequently, they cannot accurately reconstruct the three-dimensional spatial model of the hair to provide data reference for the extraction process.

[0004] In order to overcome the above-mentioned defects of the existing technology, there is an urgent need in the field for a hair information recognition technology to accurately identify the three-dimensional spatial information of the hair entity and reconstruct the pose of the entire hair, so as to improve the compatibility and accuracy of hair recognition and provide data reference for subsequent hair follicle extraction. Summary of the Invention

[0005] The following provides a brief overview of one or more aspects to offer a basic understanding of them. This overview is not an exhaustive summary of all conceived aspects, nor is it intended to identify key or decisive elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed descriptions that follow.

[0006] To overcome the aforementioned deficiencies in the existing technology, the present invention provides a hair information recognition method, a hair information recognition system, and a computer-readable storage medium. This system can not only determine the three-dimensional spatial information of each hair entity by performing overall matching based on the features of the hair root point and the hair tip point of multiple hair images, but also reconstruct the pose of the entire hair, thereby improving the compatibility and accuracy of hair recognition and providing data reference for subsequent hair follicle extraction.

[0007] Specifically, the hair information recognition method provided by the first aspect of the present invention includes the following steps: acquiring multiple hair images including the two-dimensional endpoint coordinates of multiple hairs; according to preset constraints, traversing the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image respectively to determine a candidate hair list for each first hair in the first hair image, wherein the candidate hair list records the identification information of at least one second hair whose second endpoint coordinates in the second hair image and the first endpoint coordinates of the first hair meet the constraints, and extracting feature descriptors of the root point and tip point of each first hair and each second hair in the candidate hair list respectively for feature matching; in response to the matching result that any second hair in the candidate hair list belongs to the same hair entity as the first hair, determining the three-dimensional spatial information of the hair entity according to the first pixel set of the hair entity in the first hair image, the second pixel set of the hair entity in the second hair image, and the relative perspective information of the first hair image and the second hair image.

[0008] Preferably, in one embodiment of the present invention, the step of acquiring multiple hair images including the two-dimensional endpoint coordinates of multiple hairs includes: acquiring a left-eye image and a right-eye image including multiple hairs respectively via a binocular camera; performing key point detection on the left-eye image via a pre-trained first neural network model to determine the first endpoint coordinates of each first hair in the left-eye image; and performing key point detection on the right-eye image via the first neural network model to determine the second endpoint coordinates of each second hair in the right-eye image.

[0009] Preferably, in one embodiment of the present invention, the step of acquiring a left-eye image and a right-eye image including multiple hairs via a binocular camera includes: acquiring a left-eye original image and a right-eye original image via the binocular camera; performing object detection on the left-eye original image via a pre-trained second neural network model to determine a left-eye image including multiple first hairs; and performing object detection on the right-eye original image via the second neural network model to determine a right-eye image including multiple second hairs.

[0010] Preferably, in one embodiment of the present invention, the step of performing key point detection via the first neural network model includes: using the OpenCV library to traverse the left eye image and the right eye image to determine the first pixel set involved in each first hair and the second pixel set involved in each second hair respectively; and performing key point detection on the first pixel set and the second pixel set respectively via the first neural network model to determine the first endpoint coordinates of each first hair and the second endpoint coordinates of each second hair.

[0011] Preferably, in one embodiment of the present invention, after determining the coordinates of the first endpoints of each first hair in the left eye image and the coordinates of the second endpoints of each second hair in the right eye image, the recognition method further includes the following steps: performing distortion correction and / or binocular correction remapping on the left eye image and the right eye image according to the camera intrinsic and extrinsic parameters of the binocular camera to obtain the first hair image and the second hair image, and correcting the coordinates of the first endpoints and the second endpoints.

[0012] Preferably, in one embodiment of the present invention, the constraints include epipolar constraints and depth constraints. The step of traversing the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image according to the preset constraints to determine the candidate hair list for each first hair in the first hair image includes: selecting a first hair that has not been traversed from the first hair image; and determining a deviation threshold y according to the epipolar constraints. error And by combining the Y-value of the hair root point coordinates of the first hair, the maximum value y of the hair root point coordinates of the second hair in the second hair image in the Y direction is determined. max and minimum value y min Based on the depth constraints, the permissible depth range is determined, and combined with the Y-value of the hair root coordinates of the first hair, the maximum value x of the hair root coordinates of the second hair in the second hair image in the X direction is determined. max and minimum value x min ; Traverse the root coordinates of each second hair in the second hair image, and add the identification information of the second hair that meets the epipolar constraint and the depth constraint to the candidate hair list of the first hair; and traverse each first hair in the first hair image to determine the candidate hair list of each first hair in the first hair image.

[0013] Preferably, in one embodiment of the present invention, the feature descriptor includes a brief descriptor and / or a SIFT descriptor, and the step of performing the feature matching includes: performing feature matching between the second feature descriptor of the second root point and the second tip point of each second hair in the candidate hair list and the first feature descriptor of the first root point and the first tip point of the first hair; and in response to the fact that the root point and the tip point of any second hair in the candidate hair list match the first hair, determining that the first hair and the second hair belong to the same hair entity, and deleting the identification information of the second hair from the candidate hair lists of the remaining first hairs.

[0014] Preferably, in one embodiment of the present invention, the step of determining the three-dimensional spatial information of the hair entity based on the first pixel set of the hair entity in the first hair image, the second pixel set of the hair entity in the second hair image, and the relative viewpoint information between the first hair image and the second hair image includes: determining a first matrix of the first hair image based on the first pixel set of the hair entity in the first hair image and the relative viewpoint information between the first hair image and the second hair image; determining a second matrix of the second hair image based on the second pixel set of the hair entity in the second hair image and the relative viewpoint information between the first hair image and the second hair image; determining a corresponding overdetermined equation based on the first matrix and the second matrix; and solving the overdetermined equation using the least squares method to determine the three-dimensional spatial information of the hair entity.

[0015] Furthermore, the hair information recognition system provided according to the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the hair information recognition method provided in any of the above embodiments.

[0016] Furthermore, the computer-readable storage medium provided according to the third aspect of the present invention stores computer instructions. When the computer instructions are executed by a processor, the hair information recognition method provided in any of the above embodiments is implemented. Attached Figure Description

[0017] The above-described features and advantages of the present invention will be better understood after reading the following detailed description of embodiments of the present disclosure in conjunction with the accompanying drawings. In the drawings, components are not necessarily drawn to scale, and components having similar related characteristics or features may have the same or similar reference numerals.

[0018] Figure 1A schematic diagram of the architecture of a hair information recognition system provided according to some embodiments of the present invention is shown.

[0019] Figure 2 A flowchart of endpoint detection provided according to some embodiments of the present invention is shown.

[0020] Figure 3 A schematic diagram of a left eye image provided according to some embodiments of the present invention is shown.

[0021] Figure 4 A schematic diagram of a first set of pixels provided according to some embodiments of the present invention is shown.

[0022] Figure 5 A flowchart of a hair information recognition method provided according to some embodiments of the present invention is shown.

[0023] Figure 6 A flowchart illustrating feature matching provided according to some embodiments of the present invention is shown.

[0024] Figure 7 A schematic diagram of feature matching results provided according to some embodiments of the present invention is shown. Detailed Implementation

[0025] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Although the description of the present invention is presented in conjunction with preferred embodiments, this does not mean that the features of the invention are limited to these embodiments. On the contrary, the purpose of describing the invention in conjunction with embodiments is to cover other options or modifications that may be derived based on the claims of the present invention. To provide a thorough understanding of the invention, many specific details will be included in the following description. The invention may also be implemented without using these details. Furthermore, to avoid confusion or obscuring the focus of the invention, some specific details will be omitted in the description.

[0026] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0027] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood as the orientations shown in the relevant paragraphs and accompanying drawings. These relative terms are for illustrative purposes only and do not imply that the described apparatus must be manufactured or operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0028] It is understood that although terms such as "first," "second," and "third" may be used herein to describe various components, regions, layers, and / or parts, these components, regions, layers, and / or parts should not be limited by these terms, and these terms are only used to distinguish different components, regions, layers, and / or parts. Therefore, the first components, regions, layers, and / or parts discussed below may be referred to as second components, regions, layers, and / or parts without departing from some embodiments of the present invention.

[0029] As mentioned above, in existing hair transplant surgeries, hair transplant assistive robots are typically used to assist doctors in the procedure. By automating the hair retrieval process, they effectively reduce labor costs and surgical time, and minimize damage to hair follicles. Existing hair transplant assistive robots typically utilize traditional image processing algorithms such as threshold segmentation and corner detection to locate hairs. However, because hair conditions vary from person to person, existing technologies generally require manual adjustment of parameters such as thresholds to adapt to different hair conditions. Therefore, they are highly dependent on the operator's theoretical knowledge and practical experience, and suffer from low detection accuracy. Furthermore, the feature matching algorithms used by existing hair transplant assistive robots can only match single feature points, and cannot perform global matching of multiple feature points across the entire hair follicle. Consequently, they cannot accurately reconstruct the three-dimensional spatial model of the hair to provide data reference for the hair retrieval process.

[0030] To overcome the aforementioned deficiencies in the existing technology, the present invention provides a hair information recognition method, a hair information recognition system, and a computer-readable storage medium. This system can not only determine the three-dimensional spatial information of each hair entity by performing overall matching based on the features of the hair root point and the hair tip point of multiple hair images, but also reconstruct the pose of the entire hair, thereby improving the compatibility and accuracy of hair recognition and providing data reference for subsequent hair follicle extraction.

[0031] In some non-limiting embodiments, the hair information recognition method provided in the first aspect of the present invention can be implemented via the hair information recognition system provided in the second aspect of the present invention. Specifically, the hair information recognition system may be configured with a memory and a processor. The memory includes, but is not limited to, the computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the hair information recognition method provided in the first aspect of the present invention.

[0032] The working principle of the hair information recognition system described above will be described below with reference to some embodiments of hair information recognition methods. Those skilled in the art will understand that these embodiments of recognition methods are merely non-limiting implementations provided by the present invention, intended to clearly demonstrate the main concepts of the invention and provide specific solutions convenient for public implementation, rather than limiting all functions or all working methods of the recognition system. Similarly, this recognition system is also only one non-limiting implementation provided by the present invention, and does not limit the executing entities or execution order of the steps in these recognition methods.

[0033] Please refer to the following first. Figure 1 , Figure 1 A schematic diagram of the architecture of a hair information recognition system provided according to some embodiments of the present invention is shown.

[0034] exist Figure 1 In the illustrated embodiment, the hair information recognition system provided by the second aspect of the present invention includes an endpoint detection module 11 and a hair entity matching module 12. The endpoint detection module 11 is configured with a hair target detection unit 111 and a hair endpoint extraction unit 112. The hair entity matching module 12 is configured with a feature descriptor extraction unit 121 and a feature matching unit 122.

[0035] In the process of recognizing hair information, the recognition system can first acquire the left and right original images of the user's hair through multiple cameras (e.g., binocular cameras), and input them into the hair target detection unit 111 for target detection to determine the left image containing multiple first hairs and the right image containing multiple second hairs. Then, the hair target detection unit 111 can input the detected left and right images into the backend hair endpoint extraction unit 112, which performs keypoint detection to determine the coordinates of the first endpoint of each first hair in the left image and the coordinates of the second endpoints of each second hair in the right image.

[0036] Please refer to the reference. Figures 1 to 4 . Figure 2A flowchart of endpoint detection provided according to some embodiments of the present invention is shown. Figure 3 A schematic diagram of a left eye image provided according to some embodiments of the present invention is shown. Figure 4 A schematic diagram of a first set of pixels provided according to some embodiments of the present invention is shown.

[0037] like Figure 1 and Figure 2 As shown, the hair target detection unit 111 can be configured with a pre-trained hair detection network model (i.e., a second neural network model). This hair detection network model includes, but is not limited to, single-stage recognition models such as YOLO (You Only LookOnce) series and SSD (Single Shot MultiBox Detector), or deep learning models for two-stage recognition such as Faster-RCNN (Faster-Regions with Convolutional Neural Network features).

[0038] During target detection, the hair target detection unit 111 can first determine the target detection and recognition regions containing multiple hairs from the acquired left and right original images based on the positioning device set in the hair collection area or a predefined region of interest (ROI). Then, the hair detection network model performs target detection on the target detection and recognition regions of the left original image to determine the hairs. Figure 3 The image shown includes a left-eye image containing multiple first hairs, and the target detection region of the right-eye original image is used by a hair detection network model to determine the right-eye image containing multiple second hairs.

[0039] Subsequently, the hair target detection unit 111 can input the detected left and right eye images into the hair endpoint extraction unit 112 at the back end. The hair endpoint extraction unit 112 performs key point detection to determine the coordinates of the first endpoint of each first hair in the left eye image and the coordinates of the second endpoint of each second hair in the right eye image.

[0040] Specifically, the hair endpoint extraction unit 112 may be configured with a pre-trained keypoint extraction network model (i.e., a first neural network model). This keypoint extraction network model includes, but is not limited to, deep learning models of the HRNet (High-Resolution Net) series, and is not limited to the size of the input and output.

[0041] During keypoint detection, the hair endpoint extraction unit 112 can first use the OpenCV library to traverse the obtained left-eye image to determine the first pixel set involved in each first hair (e.g., ...). Figure 4 (As shown). Then, the hair endpoint extraction unit 112 can sequentially perform keypoint detection on each first pixel set via the keypoint extraction network model to determine the coordinates of the first endpoint of each first hair.

[0042] Furthermore, in some embodiments, the aforementioned keypoint extraction network model can perform keypoint detection based on the principle of heatmaps. Specifically, in response to the input first pixel set, the keypoint extraction network model can output a root point heatmap and a tip point heatmap, respectively, to characterize the probability that each pixel coordinate in the first pixel set is a root point or a tip point. Here, the probability value ranges from 0 to 1; the higher the value, the greater the probability that the corresponding pixel is a root point or a tip point. The hair tip extraction unit 112 can determine whether there are root points and tip points in the first pixel set based on whether the root point heatmap and the tip point heatmap contain pixels exceeding a preset probability threshold.

[0043] Furthermore, the aforementioned first endpoint coordinates can preferably include the coordinates of the hair root and the hair tip. In response to extracting two endpoints from a single first hair, the keypoint extraction network model can determine the corresponding first endpoint coordinates of the first hair based on the extracted coordinates of the two endpoints. Conversely, if only one endpoint is extracted from a single first hair, the keypoint extraction network model can treat this endpoint as noise data and ignore its detection result, thereby reducing the amount of useless keypoint detection data and lowering the data processing load of the recognition system.

[0044] In addition, the hair endpoint extraction unit 112 can also use the OpenCV library to traverse the right eye image to determine the second pixel set involved in each second hair, and then perform key point detection on each second pixel set in sequence through the key point extraction network model to determine the second endpoint coordinates of the hair root point and hair tip point of each second hair. The specific principle is similar to that of the above embodiment, and will not be repeated here.

[0045] Thus, the endpoint detection module 11 of the recognition system provided in the second aspect of the present invention can acquire multiple hair images including the two-dimensional endpoint coordinates of multiple hairs, and transmit them to the back-end hair entity matching module 12 to perform overall matching of hair entities involved in each hair image based on the hair root point feature and the hair tip point feature, thereby determining the three-dimensional spatial information of each hair entity.

[0046] Those skilled in the art will understand that the above-described scheme for obtaining multiple hair images including the two-dimensional endpoint coordinates of multiple hairs is merely a non-limiting implementation provided by the present invention, intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are easy for the public to implement, rather than being used to limit the scope of protection of the present invention.

[0047] Alternatively, in other embodiments, those skilled in the art can also identify the two-dimensional endpoint coordinates of the hair root point and / or hair tip point in each hair image based on other neural network model architectures, and input them into the subsequent hair entity matching module 12 to obtain the corresponding feature matching effect, which will not be elaborated here.

[0048] Please refer to further information. Figure 5 , Figure 5 A flowchart of a hair information recognition method provided according to some embodiments of the present invention is shown.

[0049] like Figure 5 As shown, in response to obtaining multiple hair images including the two-dimensional endpoint coordinates of multiple hairs from the endpoint detection module 11, the hair entity matching module 12 can first traverse the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image according to preset constraints to determine the candidate hair list of each first hair in the first hair image.

[0050] Specifically, the hair entity matching module 12 may preferably store the camera intrinsic and extrinsic parameter matrices of the aforementioned multiple cameras. In response to acquiring multiple hair images including the two-dimensional endpoint coordinates of multiple hairs from the endpoint detection module 11, the hair entity matching module 12 may first perform distortion correction and / or binocular correction remapping on the acquired left and right eye images based on the camera intrinsic and extrinsic parameter matrices of the binocular cameras to obtain a corrected first hair image and a second hair image, and obtain the corrected first endpoint coordinates and second endpoint coordinates, thereby realizing binocular correction and binocular reconstruction of the original image.

[0051] Subsequently, the hair entity matching module 12 can traverse the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image according to the preset constraints, so as to add the identification information of at least one second hair whose second endpoint coordinates in the second hair image meet the constraints with the first endpoint coordinates of the first hair to the candidate hair list of each first hair, and extract the first feature descriptor of each first hair and the second feature descriptor of each second hair in its candidate hair list through the feature descriptor extraction unit 121 for feature matching.

[0052] In some embodiments, the above constraints include, but are not limited to, epipolar constraints and depth constraints. Theoretically, the endpoint of the second hair in the second hair image should have the same Y-coordinate as the hair root point of the first hair in the first hair image. However, due to practical reasons such as the accuracy of endpoint detection, the same endpoint in the first and second hair images may deviate to some extent in the Y-direction. Therefore, this invention can artificially assign a deviation threshold y based on the accuracy requirements of feature matching. error Then, combining the Y-value of the hair root point coordinates of the first hair, determine the maximum value y of the hair root point coordinates of the second hair in the Y direction in the second hair image. max and minimum value y min .

[0053] Specifically, in the process of constructing a candidate hair list for a hair, the hair entity matching module 12 can first select an untraversed first hair from the first hair image, and determine the deviation threshold y based on the epipolar constraint. error By combining the Y-value of the hair root point coordinates of the first hair, the maximum value y of the hair root point coordinates of the second hair in the Y direction in the second hair image is determined. max and minimum value y min Here, y error Ideally, the width should be within 1 / 3 of the length of a single hair, for example, 6 to 9 pixels.

[0054] Subsequently, the hair entity matching module 12 can determine the permissible depth range (e.g., depth ∈ [130, 140]) based on the depth constraints, and combine this with the Y value of the hair root point coordinates of the first hair to determine the maximum value x of the hair root point coordinates of the second hair in the X direction in the second hair image. max and minimum value x min .

[0055] Next, the hair entity matching module 12 can traverse the coordinates of the hair root points of each second hair in the second hair image, and add the identification information of the second hairs that meet the above-mentioned epipolar constraints and depth constraints to the candidate hair list of the first hair, so as to determine the candidate hair list of the corresponding first hair. Similarly, the hair entity matching module 12 can also traverse each first hair in the first hair image as described above to determine the candidate hair list of each first hair in the first hair image.

[0056] Furthermore, in some embodiments of the present invention, the feature descriptor extraction unit 121 includes, but is not limited to, the brief descriptor extraction unit and / or the SIFT descriptor extraction unit, for extracting the first feature descriptor of the first hair root point and the first hair tip point of each first hair, and the second feature descriptor of the second hair root point and the second hair tip point of each second hair in the candidate hair list of each first hair, and inputting them into the backend feature matching unit 122 for feature matching.

[0057] Please refer to the reference. Figure 6 and Figure 7 . Figure 6 A flowchart illustrating feature matching provided according to some embodiments of the present invention is shown. Figure 7 A schematic diagram of feature matching results provided according to some embodiments of the present invention is shown.

[0058] like Figure 6 As shown, during the feature matching process, the feature matching unit 122 can perform feature matching between the second feature descriptors of the second root point and the second tip point of each second hair in the candidate hair list and the first feature descriptors of the first root point and the first tip point of the corresponding first hair. Here, this feature matching can be implemented based on the principles of brute-force matching or FLANN (Fast Library for Approximate Nearest Neighbors) matching.

[0059] In response to any second hair in the candidate hair list whose root and tip points match those of a first hair, the feature matching unit 122 can determine that the first hair and the second hair belong to the same hair entity, and delete the identifier information of the second hair from the candidate hair lists of the remaining first hairs. Thus, during the feature matching process for the remaining first hairs, the feature matching unit 122 can automatically skip the second hair that has already been successfully matched, thereby improving the efficiency of feature matching.

[0060] Furthermore, in response to the determination that any first hair and a second hair belong to the same hair entity, the feature matching unit 122 can also delete the successfully matched first hair from the matching list, so as to skip the successfully matched first hair in the next matching cycle, and repeat this cycle for a preset number of rounds (e.g., 3 times) to efficiently and reliably complete the feature matching of each first hair and obtain... Figure 6 The matching results are shown. Figure 7 In the matching results shown, the feature matching unit 122 connects the successfully matched hairs with lines of the same color to indicate that the two belong to the same hair entity.

[0061] Please continue to refer to this. Figure 6In response to the matching result that any second hair in the candidate hair list belongs to the same hair entity as the first hair, the hair entity matching module 12 can determine the three-dimensional spatial information of the hair entity based on the first pixel set of the corresponding hair entity in the first hair image, the second pixel set in the second hair image, and the relative viewpoint information between the first hair image and the second hair image.

[0062] Specifically, the first pixel set may include the coordinates of the root point of the first hair. The second pixel set may include the coordinates of the root point of the second hair. The relative viewpoint information can be determined based on the relative spatial relationship between the first camera and the second camera.

[0063] In determining the three-dimensional spatial information of the hair root point of a hair entity, the recognition system can first determine the first matrix of the first hair image based on the coordinates of the first hair root point of the hair entity in the first hair image, and the relative viewpoint information between the first hair image and the second hair image:

[0064]

[0065] Then, based on the coordinates of the second hair root point of the hair entity in the second hair image, and the relative viewpoint information between the first and second hair images, the second matrix of the second hair image is determined:

[0066]

[0067] Based on the first and second matrices mentioned above, determine the corresponding overdetermined equations:

[0068]

[0069]

[0070]

[0071]

[0072] The above overdetermined equations are solved using the least squares method to determine the three-dimensional spatial information of the hair root point of the hair entity.

[0073] Furthermore, the steps to solve the above overdetermined equations can be implemented in OpenCV using the solve(A,B,XYZ,DECOMP_SVD) method. Here, A is the coefficient matrix on the left side of the overdetermined equation (matrix size 4*3), B is the matrix on the right side of the equation (matrix size 4*1), and XYZ is the column vector of 3D coordinates (i.e., [x...]). w ,y w ,z wDECOMP_SVD is a parameter of this function. The identification system can solve the least-squares solution of a given system of equations using the (cv::DECOMP_QR) instruction or the singular value decomposition instruction (cv::DECOMP_SVD).

[0074] Those skilled in the art will understand that the above-described embodiment for solving the three-dimensional spatial coordinates of the hair root point of a hair entity is merely a non-limiting implementation method provided by the present invention, intended to clearly demonstrate the main concept of the present invention and provide a specific solution that is easy for the public to implement, rather than being used to limit the scope of protection of the present invention.

[0075] Alternatively, in other embodiments, those skilled in the art can also, based on the same principle, solve for and determine the three-dimensional spatial coordinates of the hair tip of the hair entity according to the coordinates of the hair tip of the first hair in the first pixel set, the coordinates of the hair tip of the second hair in the second pixel set, and the relative perspective information, or further determine the three-dimensional spatial angle from the hair root to the hair tip and / or the three-dimensional spatial length from the hair root to the hair tip, so as to reconstruct the three-dimensional spatial model of the hair more accurately.

[0076] In summary, the hair information recognition method, recognition system, and computer-readable storage medium provided by the present invention can not only determine the three-dimensional spatial information of each hair entity by performing overall matching based on the features of the hair root point and the hair tip point of multiple hair images, but also reconstruct the pose of the entire hair, thereby improving the compatibility and accuracy of hair recognition and providing data reference for subsequent hair follicle extraction.

[0077] Although the methods described above are illustrated and depicted as a series of actions for the sake of simplicity, it should be understood and appreciated that these methods are not limited by the order of the actions, as some actions may occur in a different order and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but which may be understood by those skilled in the art, according to one or more embodiments.

[0078] Those skilled in the art will understand that information, signals, and data can be represented using any of a variety of different techniques and arts. For example, the data, instructions, commands, information, signals, bits, symbols, and chips described throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0079] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described above in a generalized manner in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the invention.

[0080] The prior description of this disclosure is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to this disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not intended to be limited to the examples and designs described herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for recognizing hair information, characterized in that, Includes the following steps: Obtain multiple hair images, including the two-dimensional endpoint coordinates of multiple hairs; According to the preset constraints, the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image are traversed to determine the candidate hair list of each first hair in the first hair image. The candidate hair list records the identification information of at least one second hair in the second hair image whose second endpoint coordinates meet the constraints. The feature descriptors of the root point and the tip point of each first hair and each second hair in the candidate hair list are extracted respectively, and the second feature descriptors of the second root point and the second tip point of each second hair in the candidate hair list are matched with the first feature descriptors of the first root point and the first tip point of the first hair. In response to the fact that the root and tip of any second hair in the candidate hair list match those of the first hair, it is determined that the first hair and the second hair belong to the same hair entity; and In response to the matching result that any second hair in the candidate hair list belongs to the same hair entity as the first hair, the three-dimensional spatial information of the hair entity is determined based on the first pixel set of the hair entity in the first hair image, the second pixel set of the hair entity in the second hair image, and the relative viewpoint information between the first hair image and the second hair image.

2. The identification method as described in claim 1, characterized in that, The step of acquiring multiple hair images, including the two-dimensional endpoint coordinates of multiple hairs, includes: Using a binocular camera, left and right eye images, including multiple hairs, are acquired respectively; Keypoint detection is performed on the left eye image using a pre-trained first neural network model to determine the coordinates of the first endpoints of each of the first hairs in the left eye image; and The right eye image is subjected to key point detection via the first neural network model to determine the coordinates of the second endpoint of each second hair in the right eye image.

3. The identification method as described in claim 2, characterized in that, The step of acquiring left and right eye images, including multiple hairs, respectively via binocular cameras includes: The left and right original images are acquired respectively via the binocular camera; The original left-eye image is subjected to object detection via a pre-trained second neural network model to determine a left-eye image that includes multiple strands of the first hair; and The original right eye image is subjected to target detection via the second neural network model to determine the right eye image that includes multiple strands of the second hair.

4. The identification method as described in claim 2, characterized in that, The steps for keypoint detection via the first neural network model include: The OpenCV library is used to traverse the left and right eye images to determine the first pixel set involved in each first hair and the second pixel set involved in each second hair; and The keypoint detection is performed on the first pixel set and the second pixel set respectively via the first neural network model to determine the first endpoint coordinates of each first hair and the second endpoint coordinates of each second hair.

5. The identification method as described in claim 2, characterized in that, After determining the coordinates of the first endpoint of each first hair in the left eye image and the coordinates of the second endpoint of each second hair in the right eye image, the recognition method further includes the following steps: Based on the camera intrinsic and extrinsic parameters of the binocular camera, distortion correction and / or binocular correction remapping are performed on the left and right eye images to obtain the first hair image and the second hair image, and the first endpoint coordinates and the second endpoint coordinates are corrected.

6. The identification method as described in claim 1, characterized in that, The constraints include epipolar constraints and depth constraints. The step of determining the candidate hair list for each first hair in the first hair image by traversing the two-dimensional endpoint coordinates of each hair in the first hair image and the second hair image according to the preset constraints includes: Select a first hair that has not been traversed from the first hair image; Determine the deviation threshold based on the polar constraint. y error By combining the Y-value of the hair root point coordinates of the first hair, the maximum value of the hair root point coordinates of the second hair in the second hair image in the Y direction is determined. y max and minimum value y min ; Based on the depth constraints, the permissible depth range is determined, and combined with the Y-value of the hair root coordinates of the first hair, the maximum value of the hair root coordinates of the second hair in the second hair image in the X-direction is determined. x max and minimum value x min ; Iterate through the root coordinates of each second hair in the second hair image, and add the identification information of the second hairs that meet the epipolar constraint and the depth constraint to the candidate hair list of the first hair; and Iterate through each of the first hairs in the first hair image to determine a candidate hair list for each of the first hairs in the first hair image.

7. The identification method as described in claim 1, characterized in that, The feature descriptor includes a brief descriptor and / or a SIFT descriptor, and the feature matching step further includes: In response to the fact that the first hair and the second hair belong to the same hair entity, the identification information of the second hair is removed from the candidate hair list of the remaining first hairs.

8. The identification method as described in claim 1, characterized in that, The step of determining the three-dimensional spatial information of the hair entity based on the first pixel set of the hair entity in the first hair image, the second pixel set of the hair entity in the second hair image, and the relative viewpoint information between the first hair image and the second hair image includes: Based on the first pixel set of the hair entity in the first hair image and the relative viewpoint information between the first hair image and the second hair image, a first matrix of the first hair image is determined. Based on the second pixel set of the hair entity in the second hair image and the relative viewpoint information between the first hair image and the second hair image, a second matrix of the second hair image is determined; Based on the first matrix and the second matrix, determine the corresponding overdetermined equations; and The overdetermined equations are solved using the least squares method to determine the three-dimensional spatial information of the hair entity.

9. A hair information recognition system, characterized in that, include: Memory, on which computer instructions are stored; as well as A processor, connected to the memory, and configured to execute computer instructions stored in the memory to implement the hair information recognition method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, the hair information recognition method as described in any one of claims 1 to 8 is implemented.

Images