Interbrow vertical stripe detection method, system, equipment and medium

Abstract

The invention relates to the technical field of image processing, and discloses a method, a system, equipment and a medium for detecting vertical lines between eyebrows, According to the method, an MTCNN face detector is used for carrying out face coarse positioning, a Dlib key point detector is used for carrying out cascade architecture of face key point extraction, according to face key point information, an eyebrow region bounding box is determined, and the vertical lines between eyebrows are detected. The method comprises the following steps: preprocessing a facial image, intercepting from the preprocessed facial image by using an interbrow region bounding box to obtain an interbrow region image, inputting the interbrow region image into a trained TransUnet model, outputting a wrinkle segmentation mask image, performing convolution difference edge detection on the interbrow region image to obtain an edge feature image of interbrow vertical lines, and outputting the edge feature image of the interbrow vertical lines. And then the wrinkle segmentation mask image and the edge feature image of the inter-brow vertical stripes are superposed to obtain an inter-brow vertical stripe edge image, so that the accuracy and robustness of the inter-brow vertical stripe detection method are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a glabellar vertical line detection method, system, device and medium. BACKGROUND

[0002] Traditional glabellar vertical line detection methods mainly rely on image processing techniques, such as using Frangi filter and Gabor filter for wrinkle edge detection, and commonly used deep learning wrinkle detection methods include ENet algorithm and RCF edge detection algorithm. However, the existing glabellar vertical line detection methods have poor accuracy, poor robustness in the face of changes in illumination, different skin types or individual differences, and are difficult to ensure stable and reliable results in different scenarios, which is difficult to meet the needs of medical diagnosis analysis for quantifying and classifying the depth or depression of a fixed wrinkle in a region. SUMMARY

[0003] Therefore, the present application provides a glabellar vertical line detection method, system, device and medium, which solves the technical problem that the existing glabellar vertical line detection methods have poor accuracy, poor robustness in the face of changes in illumination, different skin types or individual differences, and are difficult to ensure stable and reliable results in different scenarios, which is difficult to meet the needs of medical diagnosis analysis for quantifying and classifying the depth or depression of a fixed wrinkle in a region.

[0004] The first aspect of the present application provides a glabellar vertical line detection method, comprising:

[0005] Obtaining a face image of a target patient and pre-processing the face image;

[0006] Using an MTCNN face detector to perform face positioning on the pre-processed face image, and based on the face positioning information, using a Dlib key point detector to perform face key point extraction on the pre-processed face image to obtain face key point information;

[0007] According to the face key point information, determining a glabellar region bounding box, and using the glabellar region bounding box to crop the pre-processed face image to obtain a glabellar region image;

[0008] Inputting the glabellar region image into a trained TransUnet model to output a wrinkle segmentation mask image; wherein the trained TransUnet model is obtained by training an initial TransUnet model based on a training data set, wherein the training data set includes a plurality of face wrinkle images and wrinkle segmentation mask images labeled with gray scale labels;

[0009] performing convolutional difference edge detection on the glabella region image to obtain an edge feature image of the glabellar vertical wrinkles;

[0010] superimposing the wrinkle segmentation mask image and the edge feature image of the glabellar vertical wrinkles to obtain a glabellar vertical wrinkle edge image.

[0011] Preferably, the face image is preprocessed, including:

[0012] converting the face image into a face RGB image and a face grayscale image, respectively;

[0013] performing face positioning on the preprocessed face image using the MTCNN face detector, and based on the face positioning information, performing face key point extraction on the preprocessed face image using a Dlib key point detector to obtain face key point information, including:

[0014] performing face detection on the face RGB image using the MTCNN face detector;

[0015] if a face is detected, performing face positioning on the face RGB image using the MTCNN face detector to obtain face positioning information; wherein the face positioning information includes a face bounding box;

[0016] converting the image at the position corresponding to the face bounding box in the face RGB image into a face bounding box grayscale image, and performing face key point extraction on the face bounding box grayscale image using the Dlib key point detector to obtain the face key point information;

[0017] if a face is not detected, performing face key point extraction on the face grayscale image using the Dlib key point detector to obtain the face key point information.

[0018] Preferably, the glabella region bounding box is determined according to the face key point information, and the glabella region image is obtained by cutting the preprocessed face image using the glabella region bounding box, including:

[0019] determining a glabellar midpoint according to the center points of the left eyebrow key point and the right eyebrow key point, and determining an eye midpoint according to the center points of the left eye key point and the right eye key point;

[0020] performing weighted averaging on the glabellar midpoint and the eye midpoint to obtain a glabella region center point;

[0021] determining an x-axis direction reference distance and a y-axis direction reference distance according to the distance between the glabellar midpoint and the eye midpoint, and the distance between the center points of the left and right eye key points;

[0022] determine an inter-brow region width and an inter-brow region height according to the x-axis direction reference distance and the y-axis direction reference distance respectively;

[0023] determine coordinates of the inter-brow region bounding box according to the inter-brow region width, the inter-brow region height and the inter-brow region center point;

[0024] determine an initial inter-brow region bounding box according to the coordinates of the inter-brow region bounding box;

[0025] correct the initial inter-brow region bounding box according to an image size constraint of the face image to obtain the inter-brow region bounding box;

[0026] crop the pre-processed face image according to the inter-brow region bounding box to obtain the inter-brow region image.

[0027] Preferably, the process of training the TransUnet model comprises:

[0028] obtain a plurality of face wrinkle images labeled with a grayscale label; wherein the grayscale label is used to mark the grayscale value of the face wrinkle image;

[0029] perform binarization processing on each of the face wrinkle images to obtain a binarization label image;

[0030] for each iteration process of each binarization label image, perform an erosion operation on the binarization label image at the current iteration to obtain a wrinkle skeleton image, then perform an inflation operation on the wrinkle skeleton image to obtain a label enhanced image at the current iteration, and update the label enhanced image as the binarization label image for the next iteration for iteration operation, until all foreground pixels of the binarization label image are eroded, and output the final label enhanced image corresponding to the binarization label image as a wrinkle segmentation mask image;

[0031] use a plurality of the face wrinkle images and the wrinkle segmentation mask image as mapping samples to construct a training data set;

[0032] use the training data set to train the initial TransUnet model to obtain a trained TransUnet model.

[0033] Preferably, the convolutional differential edge detection on the inter-brow region image to obtain the edge feature image of the inter-brow vertical wrinkle further comprises:

[0034] perform denoising processing on the inter-brow region image using Gaussian blur;

[0035] The step of performing convolutional differential edge detection on the image of the glabella region to obtain the edge feature image of the vertical lines between the eyebrows includes:

[0036] The Sobel operator is used to perform first-order convolution difference processing on the image of the eyebrow region to obtain the vertical edge image;

[0037] The Laplacian operator is used to perform second-order convolution difference processing on the image of the glabella region to obtain a gradient edge image;

[0038] The vertical edge image and the gradient edge image are weighted and fused to obtain a fused edge intensity image, which is used as the edge feature image of the vertical lines between the eyebrows.

[0039] Preferably, the step of superimposing the wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows to obtain the edge image of the vertical lines between the eyebrows includes:

[0040] The wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows are resized to obtain the intermediate image of the wrinkle segmentation mask and the intermediate image of the edge features.

[0041] Extract all position coordinates of the background region in the middle image of the wrinkle segmentation mask, and match a partial region in the middle image of the edge features that corresponds to the position coordinates of the background region. Then, adjust the gray value of the matched partial region to zero to obtain the eyebrow edge image.

[0042] Determine the total gray value of each connected region of wrinkles in the eyebrow edge image, and remove connected regions of wrinkles whose total gray value is less than a preset total gray value threshold to obtain an effective image of the eyebrow edge.

[0043] The wrinkle connected regions in the effective image of the eyebrow edge are classified to obtain multiple vertical wrinkle regions;

[0044] A verticality check is performed on each of the vertical wrinkle regions to obtain the vertical span and horizontal span of the vertical wrinkle regions;

[0045] The vertical wrinkle regions with a vertical span greater than the horizontal span are selected, and the minimum bounding rectangle is fitted to the selected vertical wrinkle regions. The area covered by the fitted minimum bounding rectangle is determined as the vertical wrinkle region between the eyebrows, and the edge image of the vertical wrinkle between the eyebrows is obtained.

[0046] Preferably, the method further includes:

[0047] Based on the edge image of the vertical lines between the eyebrows, the wrinkle index of the vertical lines between the eyebrows is determined; wherein, the wrinkle index includes at least one of the wrinkle area, wrinkle angle, total wrinkle gray value and average wrinkle gray value.

[0048] The severity of the vertical lines between the eyebrows in the target patients was assessed based on the wrinkle index of the vertical lines between the eyebrows.

[0049] Secondly, the present invention also provides a system for detecting vertical lines between the eyebrows, comprising:

[0050] An image processing module is used to acquire facial images of the target patient and preprocess the facial images;

[0051] The key point extraction module is used to perform face localization on the preprocessed facial image using the MTCNN face detector, and based on the face localization information, to extract facial key points from the preprocessed facial image using the Dlib key point detector to obtain facial key point information.

[0052] The eyebrow region cropping module is used to determine the eyebrow region bounding box based on the facial key point information, and to crop the eyebrow region image from the preprocessed facial image using the eyebrow region bounding box.

[0053] The wrinkle mask determination module is used to input the image of the glabella region into the trained TransUnet model and output a wrinkle segmentation mask image; wherein, the trained TransUnet model is obtained by training an initial TransUnet model based on a training dataset, wherein the training dataset includes multiple facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images;

[0054] The differential edge detection module is used to perform convolutional differential edge detection on the image of the eyebrow region to obtain the edge feature image of the vertical lines between the eyebrows;

[0055] The vertical line edge determination module is used to superimpose the wrinkle segmentation mask image and the edge feature image of the vertical line between the eyebrows to obtain the edge image of the vertical line between the eyebrows.

[0056] Thirdly, the present invention also provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the eyebrow vertical line detection method as described in the first aspect.

[0057] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the eyebrow vertical line detection method as described in the first aspect.

[0058] As can be seen from the above technical solutions, this invention utilizes a cascaded architecture of MTCNN face detector for coarse face localization and Dlib keypoint detector for facial keypoint extraction. This reduces background interference and avoids process errors caused by the failure of a single detection method. Furthermore, based on facial keypoint information, it determines the bounding box of the glabella region and uses this bounding box to extract the glabella region image from the preprocessed facial image. This glabella region image is then input into the trained TransUnet model, which outputs a wrinkle segmentation mask image. Convolutional differential edge detection is then performed on the glabella region image to obtain the edge feature image of the vertical glabella lines. This differential method focuses on the relative changes in pixel values, reducing dependence on the absolute value of light intensity and avoiding detection errors caused by changes in illumination. Finally, the wrinkle segmentation mask image and the edge feature image of the vertical glabella lines are superimposed to obtain the edge image of the vertical glabella lines. This improves the accuracy and robustness of the vertical glabella line detection method, ensuring stable and reliable results in different scenarios. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is an application environment diagram of a method for detecting vertical lines between the eyebrows provided in an embodiment of the present invention;

[0061] Figure 2 A flowchart of a method for detecting vertical lines between the eyebrows provided in an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of a facial wrinkle image provided in an embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of a wrinkle segmentation mask image provided in an embodiment of the present invention;

[0064] Figure 5 A schematic diagram illustrating the training process of the loss value of the loss function provided in an embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the glabella region image after performing Gaussian convolution, provided in an embodiment of the present invention.

[0066] Figure 7 This is a schematic diagram of a grayscale image after first-order difference provided in an embodiment of the present invention;

[0067] Figure 8 This is a schematic diagram of the image after second-order difference provided in an embodiment of the present invention;

[0068] Figure 9 This is a schematic diagram of the fused edge intensity image provided in an embodiment of the present invention;

[0069] Figure 10 This is a schematic diagram of the edge image of vertical lines between the eyebrows provided in an embodiment of the present invention;

[0070] Figure 11 This is a schematic diagram of the structure of a vertical line detection system between the eyebrows provided in an embodiment of the present invention;

[0071] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0072] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] The method for detecting vertical lines between the eyebrows provided in this application can be applied to, for example... Figure 1 The application environment shown is illustrated. Terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be integrated onto server 102, or it can be located in the cloud or on another network server. Terminal 101 or server 102 acquires facial images of the target patient and preprocesses them. The MTCNN face detector is used to locate faces in the preprocessed facial images, and based on the face location information, the Dlib keypoint detector is used to extract facial keypoints from the preprocessed facial images to obtain facial keypoint information. Based on the facial keypoint information, the bounding box of the glabella region is determined, and the glabella region image is obtained by cropping from the preprocessed facial images using the glabella region bounding box. The glabella region image is input into a trained TransUnet model, which outputs a wrinkle segmentation mask image. The trained TransUnet model is obtained by training an initial TransUnet model based on a training dataset, which includes multiple facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images. Convolutional differential edge detection is performed on the glabella region image to obtain the edge feature image of the vertical wrinkles between the eyebrows. The wrinkle segmentation mask image and the edge feature image of the vertical wrinkles between the eyebrows are superimposed to obtain the edge image of the vertical wrinkles between the eyebrows.

[0074] Terminal 101 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets.

[0075] Server 102 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0076] like Figure 2 As shown in the embodiment of this application, a method for detecting vertical lines between the eyebrows is provided, which is then applied to... Figure 1 Taking terminal 101 or server 102 as an example, the explanation includes the following steps S1 to S6. Wherein:

[0077] Step S1: Obtain facial images of the target patient and preprocess the facial images.

[0078] In this process, doctors collect facial images of target patients during visual examination and preprocess the facial images to convert them into images that meet the requirements of the subsequent MTCNN face detector and Dlib keypoint detector.

[0079] Step S2: Use the MTCNN face detector to locate faces in the preprocessed facial images, and use the Dlib keypoint detector to extract facial keypoints from the preprocessed facial images based on the face location information to obtain facial keypoint information.

[0080] Among them, the MTCNN (Multi-task Cascaded Convolutional Networks) face detector uses a three-level cascaded convolutional neural network structure to gradually filter out regions that may contain faces and accurately locate the key points of the face.

[0081] In practical applications, the MTCNN face detector first uses a shallow convolutional neural network to quickly generate a large number of candidate windows that may contain faces; then, a deeper convolutional neural network further filters these candidate windows, eliminating most non-face regions; finally, the deepest convolutional neural network is used to finely adjust the remaining face candidate regions and locate the five key points of the face, including the center of the left eye, the center of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth.

[0082] Based on the face region located by the MTCNN face detector, the Dlib (Dlib facial landmark detector) key point detector is then used to perform more detailed facial key point extraction on the preprocessed facial image. The Dlib key point detector is an algorithm based on regression trees that can extract more key point information of the face, such as the contour points of the eyebrows, eyes, nose, mouth, etc., thus obtaining more comprehensive facial key point information and providing an accurate basis for determining the boundary box of the glabella region later.

[0083] Step S3: According to the facial key point information, determine the boundary box of the glabella region, and use the boundary box of the glabella region to intercept from the preprocessed facial image to obtain the glabella region image.

[0084] Among them, the glabellar vertical wrinkle is located between the two eyebrows and extends vertically downward, usually from the center of the eyebrows to near the root of the nose. Other wrinkles such as forehead wrinkles, crow's feet, and nasolabial folds are located in different regions such as the forehead, the corners of the eyes, and both sides of the nose wings, respectively. The glabellar vertical wrinkle usually appears as longitudinal lines, and there may be one, two, or three of them, shaped like the Chinese character "Chuan", so it is also called the "Chuan character wrinkle". Dynamic glabellar wrinkles are obvious when frowning and may disappear when relaxed; static glabellar wrinkles are clearly visible even when the face has no expression. The glabellar vertical wrinkle is mainly caused by the long-term contraction of the corrugator supercilii muscle, along with age growth, skin aging, and long-term frowning habits. Other wrinkles such as crow's feet are mainly caused by the contraction of the orbicularis oculi muscle or skin aging.

[0085] The boundary box of the glabella region is a rectangular area, and its position and size are determined according to the positions of the eyebrows and eyes in the facial key point information to ensure that the area where the glabellar vertical wrinkle may appear can be completely covered. Specifically, the four vertex coordinates of the boundary box can be set by calculating the midpoint between the left and right eyebrows and the vertical distance between the eyebrows and eyes, so as to accurately intercept the glabella region image. By accurately intercepting the glabella region, the interference of other regions of the face can be excluded, making the detection process more focused on the feature extraction and analysis of the glabellar vertical wrinkle.

[0086] Step S4: Input the glabella region image into the trained TransUnet model to output the wrinkle segmentation mask image; among them, the trained TransUnet model is obtained by training the initial TransUnet model based on the training dataset, where the training dataset includes multiple facial wrinkle images marked with grayscale image labels and wrinkle segmentation mask images.

[0087] The TransUnet model is a deep learning model that combines the Transformer and U-Net structures. It utilizes the Transformer's self-attention mechanism to capture long-range dependencies in images, while incorporating the U-Net's skip connection structure to retain more spatial information, thus achieving excellent performance in image segmentation tasks. In this application, by training an initial TransUnet model with a large number of facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images, it learns the ability to accurately segment wrinkle regions from facial images. The trained TransUnet model can receive an image of the glabella region as input and output a corresponding wrinkle segmentation mask image, in which the wrinkle regions are clearly marked, providing an important foundation for subsequent edge feature extraction and vertical wrinkle edge determination.

[0088] Step S5: Perform convolutional differential edge detection on the image of the area between the eyebrows to obtain the edge feature image of the vertical lines between the eyebrows.

[0089] Convolutional differential edge detection is a commonly used image processing technique that detects edges by calculating the gray-level difference between adjacent pixels in an image. In this application, convolutional differential edge detection is performed on the image of the eyebrow region to effectively extract the edge features of the vertical lines between the eyebrows, generating an edge feature image of the vertical lines. This edge feature image highlights the contour and direction of the vertical lines between the eyebrows, providing crucial information for subsequent edge determination. Specifically, convolutional differential edge detection can be implemented using various edge detection operators, such as the Sobel operator and the Canny operator. These operators can calculate the gradient magnitude and direction of pixels in the image, thereby detecting edges. By selecting appropriate operators and parameters, the edge detection effect can be optimized, making the extracted edges of the vertical lines between the eyebrows clearer and more accurate.

[0090] Step S6: Overlay the wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows to obtain the edge image of the vertical lines between the eyebrows.

[0091] When overlaying the wrinkle segmentation mask image with the edge feature image of the vertical lines between the eyebrows, it is necessary to ensure that the boundary information of the wrinkle region accurately matches the spatial positioning of the segmentation mask. The final output image of the edge of the vertical lines between the eyebrows is presented as a visual image in binary form, where white pixels represent the detected vertical line edges, and the black background highlights the non-wrinkle areas, providing a clear analyzable object for subsequent wrinkle index calculation.

[0092] It should be noted that the embodiments of this application utilize a cascaded architecture of MTCNN face detector for coarse face localization and Dlib keypoint detector for facial keypoint extraction. This reduces background interference and avoids process errors caused by the failure of a single detection method. Furthermore, based on facial keypoint information, the bounding box of the glabella region is determined, and the glabella region image is extracted from the preprocessed facial image using this bounding box. This glabella region image is then input into the trained TransUnet model, which outputs a wrinkle segmentation mask image. Convolutional differential edge detection is performed on the glabella region image to obtain the edge feature image of the vertical glabella lines. This differential method focuses on the relative changes in pixel values, with less dependence on the absolute value of light intensity, avoiding detection errors caused by changes in illumination. Finally, the wrinkle segmentation mask image and the edge feature image of the vertical glabella lines are superimposed to obtain the edge image of the vertical glabella lines. This improves the accuracy and robustness of the vertical glabella line detection method, ensuring stable and reliable results in different scenarios.

[0093] In some embodiments, preprocessing of the facial image includes:

[0094] Convert the facial image into an RGB image and a grayscale image, respectively.

[0095] Specifically, the facial image is initially in BGR format (height H × width W × 3 channels), and the following transformation is performed:

[0096] BGR format image Through conversion function The face image is converted to RGB format and used as input for the subsequent MTCNN face detector.

[0097] BGR format image Through conversion function The face grayscale image is converted to grayscale format and used as input for the subsequent Dlib keypoint detector.

[0098] In this case, the MTCNN face detector is used to locate faces in the preprocessed facial images, and based on the face localization information, the Dlib keypoint detector is used to extract facial keypoints from the preprocessed facial images, obtaining facial keypoint information, including:

[0099] Step S201: Use the MTCNN face detector to perform face detection on the RGB facial image.

[0100] Step S202: If a face is detected, the MTCNN face detector is used to locate the face in the RGB image of the face to obtain face location information; wherein, the face location information includes the face bounding box.

[0101] In this embodiment, the MTCNN face detector is used to detect faces. The returned data includes the detected face bounding box and the five basic key points of the MTCNN face detector: left eye, right eye, nose, left corner of mouth, and right corner of mouth.

[0102] Step S203: Convert the image corresponding to the face bounding box in the RGB image of the face into a grayscale image of the face bounding box, and use the Dlib key point detector to extract facial key points from the grayscale image of the face bounding box to obtain facial key point information.

[0103] The process involves transforming the bounding box data obtained by the MTCNN face detector into rectangular bounding boxes in the format (left, top, right, bottom) for the Dlib keypoint detector. This step is equivalent to using the MTCNN face detector to perform image cropping and correction for the Dlib keypoint detector, removing interference from lighting, shadows, and background in the original image. The bounding boxes provide the Dlib keypoint detector with accurate face location, while the facial feature points provide partial keypoint location references, increasing the accuracy of Dlib keypoint detector localization. Since it's not possible to directly input the facial feature keypoints extracted by the MTCNN face detector into the 68-point model of the Dlib keypoint detector to improve its accuracy—the 68-point model of the Dlib keypoint detector operates independently based on its internal algorithm and training data, requiring it to start working from the face regions detected in the original image—a fusion of keypoints from the MTCNN face detector and some keypoints from the Dlib keypoint detector is used to make the extracted target region more accurate, thereby enhancing the code's face localization function.

[0104] Step S204: If no face is detected, the Dlib keypoint detector is used to extract facial keypoints from the grayscale image of the face to obtain facial keypoint information.

[0105] If the MTCNN face detector does not detect a face, i.e., the returned data is zero, then the Dlib keypoint detector is used directly to extract facial keypoints from the grayscale image of the face.

[0106] In some embodiments, facial key point information includes left eyebrow key points (L[18:22], i.e., points 18 to 22), right eyebrow key points (L[23:27]], i.e., points 23 to 27), left eye key points (L[37:42], i.e., points 37 to 42) and right eye key points (L[43:48], i.e., points 43 to 48).

[0107] In this case, based on facial key point information, the bounding box of the glabella region is determined, and the glabella region image is obtained by cropping from the preprocessed facial image using the glabella region bounding box, including:

[0108] Step S301: Determine the midpoint of the eyebrow based on the center points of the left and right eyebrow key points; determine the midpoint of the eye based on the center points of the left and right eye key points.

[0109] The average of the sums of the center points M(L[18:22]) of the left eyebrow key point and M(L[23:27]) of the right eyebrow key point is the midpoint of the eyebrow, and the average of the sums of the center points M(L[37:42]) of the left eye key point and M(L[37:42]) of the right eye key point is the midpoint of the eye.

[0110] Step S302: Calculate the weighted average of the midpoint of the eyebrow and the midpoint of the eye to obtain the center point of the area between the eyebrows.

[0111] Among them, the center point of the glabella area That is:

[0112]

[0113] In the formula, 'a' represents the weight of the eyebrow midpoint, and 'b' represents the weight of the eye midpoint. The sum of the values ​​of 'a' and 'b' is 1, and these values ​​are chosen based on experience, such as 0.4 and 0.6 respectively.

[0114] Step S303: Determine the reference distance in the x-axis direction and the reference distance in the y-axis direction based on the distance between the midpoint of the eyebrow and the midpoint of the eye, as well as the distance between the center points of the key points of the left and right eyes.

[0115] The reference distance D(x,y) is calculated as the distance between the midpoint of the eyebrow and the midpoint of the eye, and also the distance between the centers of the left and right eyes. The reference distance is the average of these two distances, denoted as D(x) (distance along the x-axis) and D(y) (distance along the y-axis), respectively. Specifically, the reference distance D(x,y) is:

[0116]

[0117] The x-axis reference distance is the projection of the reference distance D(x,y) onto the x-axis, and the y-axis reference distance is the projection of the reference distance D(x,y) onto the y-axis. By calculating in this way, we can obtain the reference distances on the x-axis and y-axis that are suitable for determining the size of the bounding box of the glabella region.

[0118] Step S304: Determine the width and height of the glabella region based on the reference distances in the x-axis and y-axis directions, respectively.

[0119] The width and height of the glabella are as follows:

[0120]

[0121]

[0122] In the formula, W and H are the width and height of the glabella, respectively, and c and d are the weights of the x-axis and y-axis, respectively.

[0123] Step S305: Determine the coordinates of the boundary box of the eyebrow region based on the width, height, and center point of the eyebrow region.

[0124] The coordinates of the boundary box of the glabella area include the coordinates of the top left corner, bottom left corner, top right corner, and bottom right corner.

[0125] The coordinates of the top-left corner are obtained by subtracting half the width and height from the center point of the area between the eyebrows, i.e.:

[0126]

[0127] The coordinates of the bottom right corner are obtained by adding the width and height to the coordinates of the top left corner, i.e.:

[0128]

[0129] After obtaining the coordinates of the top-left and bottom-right corners, the coordinates of the bottom-left and top-right corners can be determined. The x-value of the bottom-left corner is the same as that of the top-left corner, while the y-value is the sum of the y-value of the top-left corner and the height of the glabella region, i.e., (x, y + H). The x-value of the top-right corner is the sum of the x-value of the top-left corner and the width of the glabella region, while the y-value is the same as that of the top-left corner, i.e., (x + W, y). Using these four coordinates, the position and size of the bounding box of the glabella region in the facial image can be accurately determined.

[0130] Step S306: Determine the initial bounding box of the glabella region based on the coordinates of the bounding box of the glabella region.

[0131] The initial bounding box of the glabella can be obtained by connecting the coordinates of the glabella bounding box sequentially.

[0132] Step S307: Correct the initial eyebrow region bounding box according to the image size constraints of the facial image to obtain the eyebrow region bounding box.

[0133] To ensure the bounding box of the glabella region remains within the image area, it needs to be adjusted based on the image dimensions of the facial image. Specifically, it checks whether each coordinate point of the initial glabella region bounding box exceeds the boundaries of the facial image. If a coordinate point exceeds the boundary, it is adjusted to the corresponding boundary value. The glabella region bounding box is as follows:

[0134] ( )

[0135] Here, W and H represent the width and height of the facial image, respectively. This correction ensures that the bounding box of the glabella region is completely within the facial image, preventing errors in subsequent processing due to it exceeding the image's boundaries. Furthermore, this correction method offers flexibility, adapting to facial images of different sizes and proportions, thus improving its versatility and robustness.

[0136] Step S308: Crop the preprocessed facial image according to the bounding box of the eyebrow region to obtain the eyebrow region image.

[0137] In this process, based on the determined coordinates of the bounding box of the glabella region, image cropping techniques are used to precisely extract the glabella region from the preprocessed facial image. This step is implemented through programming; the input is the preprocessed facial image and the coordinate information of the bounding box of the glabella region, and the output is an image containing only the glabella region.

[0138] During the cropping process, the corresponding rectangular region is located and extracted on the facial image based on the coordinates of the four vertices of the bounding box, namely the top left, bottom left, top right, and bottom right corners. This region is the glabella region image. The detection is based on this precisely cropped glabella region image, thus effectively avoiding interference from other facial regions on the detection results.

[0139] In some embodiments, the process of training a TransUnet model includes:

[0140] Step S41: Obtain multiple facial wrinkle images labeled with grayscale tags; wherein, the grayscale tags are used to mark the grayscale values ​​of the facial wrinkle images.

[0141] For example, a facial wrinkle image containing 300 labeled images (each image contains a grayscale label) is used. The grayscale value of 0 in the black area represents the background, and the area with a grayscale value greater than 0, that is, the area that is not completely black in the image, is the foreground area. Therefore, grayscale labels are used to mark the grayscale value of the facial wrinkle image. For example, the grayscale value corresponding to label 0 is 0 (background image), the grayscale value corresponding to label 1 is 100, and the grayscale value corresponding to label 2 is 200.

[0142] Although these facial wrinkle images labeled with grayscale images can be directly used for TransUnet model training, without processing, directly using these labels to train the model results in the deep learning model extracting the entire wrinkle including the edges, which is not ideal. Therefore, this application also refines the labels to enhance the accuracy of the training set.

[0143] Step S42: Binarize each facial wrinkle image to obtain a binarized label image.

[0144] The binarization process for facial wrinkle images labeled with grayscale images involves converting the grayscale image into a binary image with only two grayscale values ​​(usually 0 and 255). Specifically, a threshold is set, and pixels with grayscale values ​​greater than the threshold are set to 255 (white, representing the foreground area), while pixels with grayscale values ​​less than or equal to the threshold are set to 0 (black, representing the background area). Binarization allows for clearer separation of wrinkle areas from the background, reducing the interference of irrelevant information on model training.

[0145] Step S43: For each iteration of each binarized label image, perform an erosion operation on the binarized label image under the current iteration to obtain a wrinkle skeleton image, then perform a dilation operation on the wrinkle skeleton image to obtain a label enhancement image under the current iteration, and update the label enhancement image with the binarized label image under the next iteration for iterative operation until all foreground pixels of the binarized label image are eroded, and output the final label enhancement image corresponding to the binarized label image as a wrinkle segmentation mask image.

[0146] The erosion operation compares foreground pixels in an image with their neighboring pixels. If a neighboring pixel does not meet a specific condition (usually all are foreground pixels), the foreground pixel is removed. When processing binarized label images, the erosion operation can gradually eliminate small protrusions and noise at the edges of wrinkles, making the wrinkle region narrower and highlighting its main structure, thus obtaining a wrinkle skeleton image. The specific process is as follows: Input a TransUnet label image, which is a binary image with a background pixel value of 0 (black) and arbitrary foreground (label) pixel values. Use a 3*3 cross-shaped structuring element. This structuring element is used in the erosion and dilation operations to gradually refine the foreground region. Then, the current image is eroded, gradually shrinking the foreground region. This process is repeated by continuously performing erosion operations on the current image to gradually reduce the foreground region. Specifically, the erosion operation is as follows:

[0147]

[0148] In the formula, Given the pixel values ​​of the input binary image, E is a 3x3 cross-shaped structural element. Therefore Centered on E, the set of neighborhood coordinates with a value of 1.

[0149] The subsequent dilation operation is the inverse of the erosion operation, converting background pixels in the neighborhood of foreground pixels into foreground pixels. Performing dilation on the wrinkle skeleton image gradually recovers the eroded wrinkle regions. However, since the previous erosion operation has removed noise and small protrusions, a 3x3 elliptical dilation operation is applied to connect these label region pixels. This operation refines the skeleton, making the correspondence between labels and wrinkle regions in the image more accurate, increasing training accuracy. Therefore, the recovered wrinkle regions are smoother and more accurate, resulting in the label-enhanced image for the current iteration.

[0150] Specifically, a 3x3 elliptical dilation operation is performed on the wrinkle skeleton image. In OpenCV, the elliptical kernel is smoother, covers a more uniform neighborhood, and approximates a circular range, based on x... 2 +y 2 <r 2 The data is discretized into a 3x3 grid. The skeleton lines are uniformly thickened in all directions, suitable for curved, branching wrinkles such as facial wrinkles. Compared to the cross-shaped operation, the elliptical operation produces smoother, more natural skeleton lines. In contrast to directly using a 3x3 cross-shaped structure similar to erosion for dilation, the 3x3 elliptical skeleton structure has smoother lines and better connectivity in curves and branching regions, making it suitable for facial wrinkles with many curves in the training dataset. Specifically, the dilation operation is as follows:

[0151]

[0152] in,

[0153]

[0154] Through multiple iterations of erosion and dilation operations, the labeled image can be continuously optimized to more closely approximate the real shape and location of wrinkles. The iteration process ends when all foreground pixels of the binarized labeled image have been eroded, and the final labeled image output is the wrinkle segmentation mask image. This mask image can accurately mark the location and shape of facial wrinkles, providing high-quality labeled data for subsequent TransUNet model training. Figure 3 and 4 The images shown are the facial wrinkle image before processing and the wrinkle segmentation mask image, respectively.

[0155] Step S44: Use multiple facial wrinkle images and wrinkle segmentation mask images as mapping samples to construct a training dataset.

[0156] Step S45: Train the initial TransUnet model using the training dataset to obtain the trained TransUnet model.

[0157] The initial TransUnet model can be the R50-ViT-B_16 model from the TransUnet model library. Training configuration parameters: Use R50-ViT-B_16 as the backbone network and load its pre-trained model. Adjust the image size to 512×512, a size suitable for medical image segmentation tasks while ensuring the model captures sufficient detail. The training epochs are 300, with a maximum of 30,000 iterations. The initial learning rate is 0.0001, and the minimum learning rate is 0.01 of the initial learning rate to ensure the learning rate doesn't become too low during training. A cosine decay learning rate scheduler is used. Dice Loss is evaluated as the loss function. Dice Loss is suitable for segmentation tasks, especially when the target object has a small pixel ratio, as it better balances positive and negative samples. The training process for the loss function's loss value is as follows... Figure 5 As shown. After training, the training model was tested for its performance in facial wrinkle detection. The results were good, and relatively clear and accurate facial wrinkle data was obtained, which is the wrinkle segmentation mask image.

[0158] In this case, since the facial wrinkle image includes the glabella region image, the process of inputting the glabella region image into the trained TransUnet model and outputting the wrinkle segmentation mask image is as follows:

[0159] First, the cropped image of the eyebrow region is preprocessed to ensure it meets the input requirements of the TransUnet model. This includes resizing the image to the required 512×512 pixels and normalizing the image so that its pixel values ​​fall within the range used during model training.

[0160] Next, the preprocessed image of the glabella region is input into the trained TransUnet model. The model extracts features from the image through its encoder, capturing information at different levels, from low-level edges and textures to high-level semantic information. Subsequently, the decoder uses these features for upsampling and segmentation prediction, gradually reconstructing a segmentation result with the same size as the input image.

[0161] During the decoding process, the model references the features of the wrinkle segmentation mask image obtained during previous training to more accurately locate and segment wrinkles in the glabella region. This process utilizes the prior knowledge learned by the model during training, namely the correspondence between different wrinkle shapes and locations and image features.

[0162] Finally, the TransUnet model outputs a wrinkle segmentation mask image of the same size as the input glabella region image. In this mask image, each pixel corresponds to a label value, indicating whether the pixel belongs to a wrinkle region. Through thresholding, these label values ​​can be converted into a binary image, where white areas represent wrinkles and black areas represent the background.

[0163] Because the wrinkle area extracted by the TransUnet model is relatively large, but it can be seen from the eyebrow area that the patient's wrinkles are not deep, directly using the wrinkle area to judge the severity of wrinkles would lead to misjudgment. Therefore, the difference method is also used to process the eyebrow area.

[0164] In some embodiments, convolutional difference edge detection is performed on the image of the glabella region to obtain the edge feature image of the vertical lines between the eyebrows, which further includes:

[0165] Gaussian blur is used to denoise the image of the area between the eyebrows.

[0166] Specifically, Gaussian blur is used to denoise the image of the area between the eyebrows. This blurring effect is achieved by weighted averaging of each pixel and its surrounding pixels. In the code implementation, a Gaussian convolution filter function is used to perform the following processing on the image:

[0167] Create a Gaussian convolution kernel, generating a two-dimensional Gaussian convolution kernel with a specified size of 5×5 and a standard deviation of 0.

[0168] To perform a Gaussian convolution operation, for each pixel in the image, the product of its surrounding 5×5 region and the Gaussian convolution kernel is summed. The mathematical expression is:

[0169]

[0170] Where I(x,y) is the pixel value at position (x,y) in the original image, I'(x,y) is the pixel value at position (x,y) in the blurred image, and K is the Gaussian convolution kernel, k=(n-1) / 2 (in a 5×5 convolution kernel, k=2). The image of the glabella region after Gaussian convolution is shown below. Figure 6 As shown.

[0171] In essence, the difference operation is a high-pass filter that amplifies high-frequency components in an image. Noise typically manifests as high-frequency random variations, and the difference operation amplifies this noise. Gaussian blur, acting as a low-pass filter, suppresses this high-frequency noise. It also reduces false edges. Applying difference directly to an unblurred image produces numerous "false edges"—these are not true structural features but are caused by minute pixel variations or noise. Gaussian blur smooths these subtle changes, ensuring that the difference algorithm primarily captures meaningful structural variations.

[0172] The finite difference method detects edges and other important features in an image by calculating the difference in grayscale values ​​between adjacent pixels. The core idea is that pixel values ​​typically change significantly at image edges, while in smooth regions, the changes in adjacent pixel values ​​are minimal.

[0173] This application uses the convolutional difference method to process images. Convolutional difference is a method used for edge detection in image processing. It calculates the gradient or second derivative of gray values ​​by applying a specific convolutional kernel to the image.

[0174] Mathematical expression of convolution difference:

[0175]

[0176] Where I is the input image in grayscale matrix format, K is the convolution kernel, (i, j) is the position of the pixel being processed in the image, and k is the radius of the convolution kernel. This formula is equivalent to sliding a convolution kernel across the pixels of the image and calculating a weighted sum of the kernel and the corresponding region of the image at each position. The design of the convolution kernel determines the direction of detection and features such as horizontal and vertical edges or second-order transformations.

[0177] Convolutional differencing is performed in the x-axis using an operator. For the vertical wrinkle extraction in this application, this method calculates the horizontal convolutional differencing. The differencing method detects edges in the image by calculating the difference in grayscale values ​​between adjacent pixels. Since the edge direction is perpendicular to the differencing direction, and the edges of vertical wrinkles mainly change along the x-axis, horizontal convolutional differencing is used to process the edge variation regions of vertical wrinkles. The operator structure is as follows:

[0178]

[0179] Where I represents the input image, and * denotes the convolution operation. The positive and negative values ​​in the kernel calculate the horizontal grayscale changes, highlighting vertical edges. For example, if the grayscale of the right-hand pixel is higher than that of the left-hand pixel, the result is a positive value, and the edge is enhanced.

[0180] In some embodiments, convolutional difference edge detection is performed on the image of the glabella region to obtain the edge feature image of the vertical lines between the glabella, including:

[0181] Step S501: Use the Sobel operator to perform first-order convolution difference processing on the eyebrow region image to obtain the vertical edge image.

[0182] Specifically, the convolution difference formula for detecting vertical edges in the Sobel operator is used, and the first-order difference is:

[0183]

[0184] In the formula, This represents the gradient in the x-direction, highlighting the vertical edges of image regions. The calculation result is an approximation of the rate of gray-level change in the horizontal direction of the image, equivalent to... The output is a grayscale image, such as... Figure 7 As shown, absolute value | The larger the value of |, the more prominent the vertical edge at that location.

[0185] Step S502: Use the Laplacian operator to perform second-order convolution difference processing on the eyebrow region image to obtain the gradient edge image.

[0186] When using only horizontal convolutional differencing, the lack of vertical differencing results leads to poor extraction of some transition regions. Therefore, second-order differencing is used to assist in the extraction of detail regions. Thus, the second-order convolutional differencing operation is represented by using the Laplacian operator:

[0187]

[0188] The second-order difference uses the Laplacian kernel:

[0189]

[0190] Second-order differencing essentially measures the rate of change of first-order differencing. It is particularly sensitive to rapid changes in gradients and often highlights subtle details, such as wrinkles. While first-order differencing is sensitive to gradual changes in intensity and can highlight edges, second-order differencing is sensitive to changes in gradients and can highlight details and textures. Compared to first-order differencing, second-order results are more refined with sharper edges, but are more sensitive to noise. Combining the two provides enhanced detection capabilities for features at different scales.

[0191] Understandably, second-order differencing is used to eliminate the low rate of change of central pixels in wrinkle regions. Deep within wrinkles lies a dark shadow; this region has a low rate of pixel change, reflected in the first-order differencing result as an area of ​​low grayscale values. Second-order differencing can then be used to determine the rate of change. Figure 8 As shown.

[0192] Step S503: Perform weighted fusion processing on the vertical edge image and the gradient edge image to obtain the fused edge intensity image, which is used as the edge feature image of the vertical lines between the eyebrows.

[0193] In this process, after determining the first-order and second-order convolutional differences, the vertical edge image and the gradient edge image are combined and superimposed to comprehensively utilize the edge information captured by the first-order and second-order differences. The superposition process involves linearly superimposing the vertical edge image and the gradient edge image with equal weights. The vertical edge image highlights the intensity changes between pixels, making it suitable for edge detection. The gradient edge image further captures the rate of change of the edges, making it suitable for detecting edge details or transitions. Combining the information from both forms a more comprehensive edge image, such as... Figure 9 As shown. Specifically:

[0194]

[0195] In the formula, C(x,y) is the fused edge intensity image, and α and β are weighting coefficients, which can both be set to 0.5. The weighted averaging process can effectively enhance edge features, where the first-order difference can highlight the main edges, while the second-order difference can capture more subtle changes and textures.

[0196] The wrinkle segmentation mask image is used for binary classification. The masked region is the wrinkle region extracted by the TransUnet model, and the non-masked region is the background of the facial segmentation image. The operation ensures that only the edges within the region of interest are processed, while the original edge values ​​remain unchanged, preserving all the details of the difference processing. The dynamically calculated background value creates a clear contrast between the edges and the background, making wrinkle edges easier for the human eye to recognize and analyze. The mask operation also effectively isolates the region of interest, eliminating other facial features that may interfere with the analysis.

[0197] The results of the difference method are presented in the form of grayscale images, with grayscale ranging from 0 (black) to 255 (white). The larger the difference between a pixel and its surrounding area, the higher the grayscale value. The masked area displays the edge detection results (a combination of first-order and second-order differences), while the non-masked area is directly displayed as black (0), representing the default state where the difference remains unchanged.

[0198] In some embodiments, the wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows are superimposed to obtain the edge image of the vertical lines between the eyebrows, including:

[0199] Step S601: Unify the size of the wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows to obtain the intermediate image of the wrinkle segmentation mask and the intermediate image of the edge features.

[0200] Since the input wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows come from different processing steps, their sizes may differ. To ensure correct overlay, they need to be adjusted to the same size and coordinate system first. This step can be achieved through image scaling techniques, such as bilinear interpolation or bicubic interpolation, which can preserve edge and detail information in the image as much as possible while adjusting the image size. After unifying the sizes, the resulting intermediate wrinkle segmentation mask image and intermediate edge feature image will have the same pixel size, providing a basis for subsequent overlay operations.

[0201] Step S602: Extract all position coordinates of the background region in the middle image of the wrinkle segmentation mask, match the part of the region corresponding to the position coordinates of the background region in the middle image of the edge features, and adjust the gray value of the matched part of the region to zero to obtain the eyebrow edge image.

[0202] In the process of extracting the coordinates of the background region, each pixel in the middle image of the wrinkle segmentation mask can be traversed to determine whether it belongs to the background region (usually, the background region is represented by a specific gray value, such as 0, in the binary classification mask image). Once all the coordinates of the background region are determined, the corresponding region can be found in the middle image of the edge features. Adjusting the gray value of these corresponding regions to zero can effectively remove background interference in the edge feature image, making the final eyebrow edge image more clearly show the edge features of the vertical wrinkles between the eyebrows.

[0203] Step S603: Determine the total gray value of each wrinkle connected region in the eyebrow edge image, and remove wrinkle connected regions whose total gray value is less than the preset total gray value threshold to obtain the effective eyebrow edge image.

[0204] Among them, wrinkle connected regions are areas in the eyebrow edge image composed of adjacent pixels with similar gray values. These regions may correspond to actual vertical wrinkles between the eyebrows. Specifically, a connected component analysis algorithm composed of adjacent pixels in the eyebrow edge image can identify each connected region in the image. For each identified wrinkle connected region, the sum of the gray values ​​of all pixels within it is calculated, i.e., the total gray value. This value reflects the overall intensity of gray-level changes within the connected region and is closely related to the depth and prominence of the wrinkles. By setting a total gray value threshold, connected regions with small total gray values ​​can be eliminated; these regions are often caused by noise or image processing errors.

[0205] In practice, connected component analysis algorithms (such as two-pass scanning or run-length encoding-based algorithms) can be used to identify all connected components in the eyebrow edge image and calculate the total grayscale value of each connected component. Then, connected components with a total grayscale value less than a preset threshold are removed, and connected components with a larger total grayscale value are retained as valid features of the eyebrow edge, ultimately yielding a valid image of the eyebrow edge.

[0206] Step S604: Classify the wrinkles in each connected region of the effective image between the eyebrows to obtain multiple vertical wrinkle regions.

[0207] When wrinkles are vertical, pixels exhibit significant variations in the horizontal region. Therefore, horizontal differencing can more effectively capture the features of vertical wrinkles. In practice, horizontal differencing can be performed on each connected region of a wrinkle in the effective image of the eyebrow edge, calculating the difference in grayscale values ​​between adjacent pixels in the horizontal direction. This difference reflects the degree of change in the wrinkle in the horizontal direction, which is usually more significant for vertical wrinkles.

[0208] By analyzing the horizontal difference results, a certain threshold can be set to distinguish vertical wrinkle regions from other non-vertical regions. Specifically, regions in the horizontal difference results whose variation exceeds the threshold can be marked as vertical wrinkle regions, while regions with smaller variation are considered non-vertical regions. In this way, the connected regions of each wrinkle in the effective image of the eyebrow edge can be classified to obtain multiple vertical wrinkle regions.

[0209] Similarly, horizontal lines are processed using vertical difference.

[0210] Step S605: Perform a verticality check on each vertical wrinkle area to obtain the vertical span and horizontal span of the vertical wrinkle area.

[0211] After obtaining the vertical wrinkle regions, a verticality check is needed to further quantify the characteristics of these wrinkles. The purpose of this step is to determine the vertical and horizontal spans of each vertical wrinkle region, thereby more accurately describing the shape and size of the wrinkles.

[0212] In practice, this can be achieved by iterating through each pixel in the vertical wrinkle region and recording its maximum and minimum coordinate values ​​in the vertical direction (i.e., the y-axis direction). The difference between these two values ​​is the vertical span of the wrinkle region. Similarly, the maximum and minimum coordinate values ​​in the horizontal direction (i.e., the x-axis direction) can be recorded, and the difference between these two values ​​is the horizontal span of the wrinkle region.

[0213] Step S606: Select vertical wrinkle regions with a vertical span greater than the horizontal span, and fit the minimum bounding rectangle to the selected vertical wrinkle regions. The area covered by the fitted minimum bounding rectangles is determined as the vertical wrinkle region between the eyebrows, and the edge image of the vertical wrinkle between the eyebrows is obtained.

[0214] When filtering vertical wrinkle areas, adhering to the principle that the vertical span is greater than the horizontal span effectively eliminates non-vertical or nearly horizontal interfering areas, thus more accurately locating the true vertical wrinkles between the eyebrows. After filtering out the vertical wrinkle areas that meet the criteria, the next step is to perform minimum bounding rectangle fitting on these areas.

[0215] Minimum bounding rectangle fitting is a commonly used image processing method that aims to find the smallest rectangle that can completely contain the target region. In this scenario, performing minimum bounding rectangle fitting on each selected vertical wrinkle region ensures that the fitted rectangle closely fits the edge of the wrinkle, thereby accurately defining the range of the vertical wrinkles between the eyebrows.

[0216] By defining the area covered by multiple fitted minimum bounding rectangles as the glabellar line region, a clear and accurate image of the glabellar line edge can be obtained. This image not only removes background interference and other non-vertical areas but also retains the main features of the glabellar lines, such as... Figure 10 As shown.

[0217] In some embodiments, the method further includes:

[0218] Step S701: Determine the wrinkle index of the vertical wrinkle between the eyebrows based on the edge image of the vertical wrinkle between the eyebrows; wherein the wrinkle index includes at least one of the wrinkle area, wrinkle angle, total gray value of wrinkle and average gray value of wrinkle.

[0219] Since the edge image of the vertical wrinkles between the eyebrows is determined by the minimum bounding rectangle, the total number of pixels contained within the minimum bounding rectangle can be calculated as a quantitative indicator of the wrinkle area. The wrinkle angle can be obtained by calculating the tilt angle of the minimum bounding rectangle, which reflects the degree of skewness of the wrinkle relative to the horizontal or vertical direction. The total gray value and average gray value of the wrinkle are obtained by summing and averaging the gray values ​​of all pixels within the minimum bounding rectangle, respectively. These two indicators reflect the gray-level changes in the wrinkle area and are closely related to the depth and prominence of the wrinkles.

[0220] Step S702: Assess the severity of the vertical lines between the eyebrows in the target patient based on the wrinkle index of the vertical lines between the eyebrows.

[0221] For example, the wrinkle index of vertical lines between the eyebrows is shown in Table 1.

[0222] Table 1. Wrinkle Indicators for Vertical Lines Between the Eyebrows

[0223]

[0224] Among them, the vertical lines between the eyebrows 1, 2, and 3 are obtained by ordering them from left to right. The wrinkle area, wrinkle angle, total gray value of the wrinkle, and average gray value of the wrinkle are used as wrinkle indicators for the vertical lines between the eyebrows. Generally speaking, the larger the wrinkle area, wrinkle angle, total gray value of the wrinkle, and average gray value of the wrinkle, the more severe the vertical lines between the eyebrows of the target patient are.

[0225] For example, the average gray value of wrinkles serves as an important indicator for judging wrinkle depth, reflecting the average salience of wrinkle pixels. A higher average gray value means the wrinkles are more pronounced, indicating a more severe wrinkle problem in the patient, while a lower value indicates shallower wrinkles or blurred edges, suggesting a shallower wrinkle depth and a milder pathology.

[0226] This method also involves setting different levels of severity for glabellar lines and comparing the wrinkle index of the glabellar lines with these levels to achieve a quantitative assessment of the severity of the glabellar lines in the target patient. For example, three levels can be set: mild, moderate, and severe, each corresponding to a certain range of wrinkle indexes. When the glabellar line index of the target patient falls within a certain level range, the severity of their glabellar lines can be determined as that level.

[0227] Based on the same inventive concept, this application also provides a vertical line detection system for implementing the above-mentioned vertical line detection method.

[0228] The solution provided by this system is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the vertical line detection system provided below can be found in the limitations of the vertical line detection method above, and will not be repeated here.

[0229] like Figure 11 As shown in the figure, this application provides a system for detecting vertical lines between the eyebrows, including:

[0230] The image processing module 100 is used to acquire facial images of the target patient and preprocess the facial images;

[0231] The key point extraction module 200 is used to perform face localization on the preprocessed facial image using the MTCNN face detector, and based on the face localization information, to extract facial key points from the preprocessed facial image using the Dlib key point detector to obtain facial key point information.

[0232] The eyebrow region cropping module 300 is used to determine the eyebrow region bounding box based on facial key point information, and to crop the eyebrow region image from the preprocessed facial image using the eyebrow region bounding box.

[0233] The wrinkle mask determination module 400 is used to input the image of the glabella region into the trained TransUnet model and output a wrinkle segmentation mask image. The trained TransUnet model is obtained by training an initial TransUnet model based on a training dataset, which includes multiple facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images.

[0234] The differential edge detection module 500 is used to perform convolutional differential edge detection on the image of the eyebrow region to obtain the edge feature image of the vertical lines between the eyebrows.

[0235] The vertical line edge determination module 600 is used to superimpose the wrinkle segmentation mask image and the edge feature image of the vertical line between the eyebrows to obtain the edge image of the vertical line between the eyebrows.

[0236] In some embodiments, the image processing module 100 is configured to:

[0237] Convert the facial image into an RGB image and a grayscale image, respectively.

[0238] Key point extraction module 200, used for:

[0239] Face detection is performed on RGB facial images using the MTCNN face detector;

[0240] If a face is detected, the MTCNN face detector is used to locate the face in the RGB image of the face to obtain face localization information, which includes the face bounding box.

[0241] The image corresponding to the face bounding box in the RGB image of the face is converted into a grayscale image of the face bounding box. The Dlib key point detector is used to extract facial key points from the grayscale image of the face bounding box to obtain facial key point information.

[0242] If no face is detected, the Dlib keypoint detector is used to extract facial keypoints from the grayscale image of the face to obtain facial keypoint information.

[0243] In some embodiments, the eyebrow region cropping module 300 is used for:

[0244] Determine the midpoint of the eyebrow based on the center points of the left and right eyebrow key points; determine the midpoint of the eye based on the center points of the left and right eye key points.

[0245] The center point of the area between the eyebrows is obtained by taking a weighted average of the midpoints of the eyebrows and the midpoints of the eyes.

[0246] Based on the distance between the midpoint of the eyebrow and the midpoint of the eye, and the distance between the center points of the key points of the left and right eyes, determine the reference distance in the x-axis direction and the reference distance in the y-axis direction;

[0247] The width and height of the glabella are determined based on the reference distances along the x-axis and y-axis, respectively.

[0248] Determine the coordinates of the boundary box of the glabella based on the width, height, and center point of the glabella region.

[0249] Determine the initial bounding box of the glabella region based on its coordinates;

[0250] The initial eyebrow region bounding box is corrected based on the image size constraints of the facial image to obtain the eyebrow region bounding box;

[0251] The preprocessed facial image is cropped based on the bounding box of the glabella region to obtain the glabella region image.

[0252] In some embodiments, the process of training a TransUnet model includes:

[0253] Acquire multiple facial wrinkle images labeled with grayscale tags; wherein, grayscale tags are used to label the grayscale values ​​of the facial wrinkle images;

[0254] Each facial wrinkle image is binarized to obtain a binarized label image;

[0255] For each iteration of each binarized label image, an erosion operation is performed on the binarized label image under the current iteration to obtain a wrinkle skeleton image. Then, a dilation operation is performed on the wrinkle skeleton image to obtain a label enhancement image under the current iteration. The label enhancement image is then updated with the binarized label image under the next iteration and the iteration operation is performed until all foreground pixels of the binarized label image are eroded. The final label enhancement image corresponding to the binarized label image is then output as a wrinkle segmentation mask image.

[0256] Multiple facial wrinkle images and wrinkle segmentation mask images were used as mapping samples to construct a training dataset.

[0257] The initial TransUnet model is trained using the training dataset to obtain the trained TransUnet model.

[0258] In some embodiments, the system further includes:

[0259] The noise reduction module is used to denoise the image of the area between the eyebrows using Gaussian blur.

[0260] Differential edge detection module 500 is used for:

[0261] The Sobel operator is used to perform first-order convolution difference processing on the image of the glabella region to obtain the vertical edge image;

[0262] The Laplacian operator is used to perform second-order convolution difference processing on the image of the glabella region to obtain the gradient edge image;

[0263] The vertical edge image and the gradient edge image are weighted and fused to obtain the fused edge intensity image, which is used as the edge feature image of the vertical lines between the eyebrows.

[0264] In some embodiments, the vertical stripe edge determining module 600 is used for:

[0265] The wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows are resized to obtain the intermediate image of the wrinkle segmentation mask and the intermediate image of the edge features.

[0266] Extract all position coordinates of the background region in the middle image of the wrinkle segmentation mask, match the partial region corresponding to the position coordinates of the background region in the middle image of edge features, and adjust the gray value of the matched partial region to zero to obtain the eyebrow edge image;

[0267] Determine the total gray value of each connected region of wrinkles in the eyebrow edge image, and remove connected regions of wrinkles whose total gray value is less than a preset total gray value threshold to obtain the effective image of the eyebrow edge.

[0268] Wrinkle classification is performed on each connected region of wrinkles in the effective image of the glabella edge to obtain multiple vertical wrinkle regions;

[0269] For each vertical wrinkle region, a verticality check is performed to obtain the vertical span and horizontal span of the vertical wrinkle region;

[0270] Vertical wrinkle regions with a vertical span greater than the horizontal span are selected, and the minimum bounding rectangle is fitted to the selected vertical wrinkle regions. The area covered by the fitted minimum bounding rectangles is determined as the vertical wrinkle region between the eyebrows, thus obtaining the edge image of the vertical wrinkle between the eyebrows.

[0271] In some embodiments, the system further includes: a vertical stripe evaluation module, used for:

[0272] Based on the edge image of the vertical lines between the eyebrows, the wrinkle index of the vertical lines between the eyebrows is determined; wherein the wrinkle index includes at least one of the following: wrinkle area, wrinkle angle, total gray value of wrinkle, and average gray value of wrinkle.

[0273] The severity of glabellar lines in target patients is assessed based on wrinkle indexes.

[0274] like Figure 12 As shown, this application provides an electronic device. The electronic device 10 includes a memory 20 and a processor 30. The memory 20 stores a computer program. When the computer program is executed by the processor 30, the processor 30 performs the steps of the eyebrow vertical line detection method as described in the above embodiment.

[0275] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed, implements the steps of the eyebrow vertical line detection method as described in the above embodiments.

[0276] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, electronic devices, and computer storage media described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0277] It should be noted that the user information (including but not limited to user images, user portrait information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0278] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0279] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0280] In the embodiments provided by this invention, it should be understood that the disclosed systems, electronic devices, computer storage media, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0281] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0282] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0283] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present invention through a computer device (which may be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0284] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting vertical lines between the eyebrows, characterized in that, include: Acquire facial images of the target patient and preprocess the facial images; The MTCNN face detector is used to locate faces in the preprocessed facial images, and based on the face location information, the Dlib keypoint detector is used to extract facial keypoints from the preprocessed facial images to obtain facial keypoint information. Based on the facial key point information, the bounding box of the glabella region is determined, and the bounding box of the glabella region is used to extract the glabella region image from the preprocessed facial image. The image of the glabella region is input into the trained TransUnet model, and the wrinkle segmentation mask image is output. The trained TransUnet model is obtained by training an initial TransUnet model based on a training dataset, which includes multiple facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images. Convolutional differential edge detection is performed on the image of the glabella region to obtain the edge feature image of the vertical lines between the glabella; The wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows are superimposed to obtain the edge image of the vertical lines between the eyebrows.

2. The method for detecting vertical lines between the eyebrows according to claim 1, characterized in that, Preprocessing the facial image includes: The facial image is converted into a facial RGB image and a facial grayscale image, respectively. The process involves using the MTCNN face detector to locate faces in the preprocessed facial image, and then using the Dlib keypoint detector to extract facial keypoints from the preprocessed facial image based on the face location information, resulting in facial keypoint information, including: The MTCNN face detector is used to detect faces in the RGB facial image; If a face is detected, the MTCNN face detector is used to locate the face in the RGB image to obtain face location information; wherein, the face location information includes a face bounding box; The image corresponding to the face bounding box in the RGB image of the face is converted into a grayscale image of the face bounding box. The Dlib key point detector is used to extract facial key points from the grayscale image of the face bounding box to obtain the facial key point information. If no face is detected, the Dlib keypoint detector is used to extract facial keypoints from the grayscale image of the face to obtain the facial keypoint information.

3. The method for detecting vertical lines between the eyebrows according to claim 1, characterized in that, The step of determining the boundary box of the glabella region based on the facial key point information, and using the boundary box of the glabella region to extract the glabella region image from the preprocessed facial image, includes: Determine the midpoint of the eyebrow based on the center points of the left and right eyebrow key points; determine the midpoint of the eye based on the center points of the left and right eye key points. The center point of the area between the eyebrows is obtained by taking a weighted average of the midpoints of the eyebrows and the midpoints of the eyes. Based on the distance between the midpoint of the eyebrow and the midpoint of the eye, and the distance between the center points of the key points of the left and right eyes, determine the reference distance in the x-axis direction and the reference distance in the y-axis direction; The width and height of the glabella are determined based on the x-axis reference distance and the y-axis reference distance, respectively. The coordinates of the boundary box of the eyebrow region are determined based on the width of the eyebrow region, the height of the eyebrow region, and the center point of the eyebrow region. Determine the initial boundary box of the glabella region based on the coordinates of the boundary box of the glabella region; The initial eyebrow region bounding box is corrected based on the image size constraints of the facial image to obtain the eyebrow region bounding box; The preprocessed facial image is cropped based on the bounding box of the glabella region to obtain the glabella region image.

4. The method for detecting vertical lines between the eyebrows according to claim 1, characterized in that, The process of training the TransUnet model includes: Acquire multiple facial wrinkle images labeled with grayscale tags; wherein the grayscale tags are used to label the grayscale values ​​of the facial wrinkle images; Each of the facial wrinkle images is binarized to obtain a binarized label image; For each iteration of each binarized label image, an erosion operation is performed on the binarized label image under the current iteration to obtain a wrinkle skeleton image. Then, a dilation operation is performed on the wrinkle skeleton image to obtain a label enhancement image under the current iteration. The label enhancement image is then updated with the binarized label image under the next iteration and the iteration operation is performed until all foreground pixels of the binarized label image are eroded. Finally, the label enhancement image corresponding to the binarized label image is output as a wrinkle segmentation mask image. Multiple facial wrinkle images and wrinkle segmentation mask images are used as mapping samples to construct a training dataset; The initial TransUnet model is trained using the training dataset to obtain a trained TransUnet model.

5. The method for detecting vertical lines between the eyebrows according to claim 1, characterized in that, The step of performing convolutional difference edge detection on the image of the area between the eyebrows to obtain the edge feature image of the vertical lines between the eyebrows also includes: The image of the area between the eyebrows is denoised using Gaussian blur. The step of performing convolutional differential edge detection on the image of the glabella region to obtain the edge feature image of the vertical lines between the eyebrows includes: The Sobel operator is used to perform first-order convolution difference processing on the image of the eyebrow region to obtain the vertical edge image; The Laplacian operator is used to perform second-order convolution difference processing on the image of the glabella region to obtain a gradient edge image; The vertical edge image and the gradient edge image are weighted and fused to obtain a fused edge intensity image, which is used as the edge feature image of the vertical lines between the eyebrows.

6. The method for detecting vertical lines between the eyebrows according to claim 1, characterized in that, The step of superimposing the wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows to obtain the edge image of the vertical lines between the eyebrows includes: The wrinkle segmentation mask image and the edge feature image of the vertical lines between the eyebrows are resized to obtain the intermediate image of the wrinkle segmentation mask and the intermediate image of the edge features. Extract all position coordinates of the background region in the middle image of the wrinkle segmentation mask, and match a partial region in the middle image of the edge features that corresponds to the position coordinates of the background region. Then, adjust the gray value of the matched partial region to zero to obtain the eyebrow edge image. Determine the total gray value of each connected region of wrinkles in the eyebrow edge image, and remove connected regions of wrinkles whose total gray value is less than a preset total gray value threshold to obtain an effective image of the eyebrow edge. The wrinkle connected regions in the effective image of the eyebrow edge are classified to obtain multiple vertical wrinkle regions; A verticality check is performed on each of the vertical wrinkle regions to obtain the vertical span and horizontal span of the vertical wrinkle regions; The vertical wrinkle regions with a vertical span greater than the horizontal span are selected, and the minimum bounding rectangle is fitted to the selected vertical wrinkle regions. The area covered by the fitted minimum bounding rectangle is determined as the vertical wrinkle region between the eyebrows, and the edge image of the vertical wrinkle between the eyebrows is obtained.

7. The method for detecting vertical lines between the eyebrows according to claim 6, characterized in that, Also includes: Based on the edge image of the vertical lines between the eyebrows, the wrinkle index of the vertical lines between the eyebrows is determined; wherein, the wrinkle index includes at least one of the wrinkle area, wrinkle angle, total wrinkle gray value and average wrinkle gray value. The severity of the vertical lines between the eyebrows in the target patients was assessed based on the wrinkle index of the vertical lines between the eyebrows.

8. A system for detecting vertical lines between the eyebrows, characterized in that, include: An image processing module is used to acquire facial images of the target patient and preprocess the facial images; The key point extraction module is used to perform face localization on the preprocessed facial image using the MTCNN face detector, and based on the face localization information, to extract facial key points from the preprocessed facial image using the Dlib key point detector to obtain facial key point information. The eyebrow region cropping module is used to determine the eyebrow region bounding box based on the facial key point information, and to crop the eyebrow region image from the preprocessed facial image using the eyebrow region bounding box. The wrinkle mask determination module is used to input the image of the glabella region into the trained TransUnet model and output a wrinkle segmentation mask image; wherein, the trained TransUnet model is obtained by training an initial TransUnet model based on a training dataset, wherein the training dataset includes multiple facial wrinkle images and wrinkle segmentation mask images labeled with grayscale images; The differential edge detection module is used to perform convolutional differential edge detection on the image of the eyebrow region to obtain the edge feature image of the vertical lines between the eyebrows; The vertical line edge determination module is used to superimpose the wrinkle segmentation mask image and the edge feature image of the vertical line between the eyebrows to obtain the edge image of the vertical line between the eyebrows.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the eyebrow vertical line detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the method for detecting vertical lines between the eyebrows as described in any one of claims 1-7.

Images