Double-u-type network eyelid line curvature measurement method based on multi-task attention mechanism

Through the dual u-type network with multi-task attention mechanism, the automatic measurement of eyelid line curvature is achieved, which solves the problems of time-consuming, labor-intensive and subjective influence in the existing technology, improves the efficiency and accuracy of measurement, and is suitable for multiple Ophthalmic surgery support in the patient's case.

CN118918085APending Publication Date: 2024-11-08NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Patent Information

Application Number
CN202410970172.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing eyelid line curvature measurement methods require professional doctors to operate, which is time-consuming and labor-intensive. Due to the subjective judgment of the operator, it is difficult to achieve fast and accurate dynamic observation of eyelid morphology, especially when there are few doctors and many patients.

Method used

Using a dual U-type network based on multi-task attention mechanism, the segmentation network model is used to achieve accurate segmentation of the palpitation fissure area and corneal area, and the pixel point value is set to depict the palpitation fissure edge line, and the encoder-decoder structure and attention mechanism are used to perform Automatic measurement of eyelid curvature values, optimized feature extraction and image recovery.

Benefits of technology

The rapid and accurate acquisition of eyelid line curvature values ​​is achieved, the subjective impact of manual operation is reduced, and the efficiency and accuracy of measurement is improved. It is suitable for ophthalmic surgery support in multiple patients.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918085A_ABST
    Figure CN118918085A_ABST
Patent Text Reader

Abstract

The invention discloses a double-u-type network eyelid line curvature measurement method based on a multi-task attention mechanism, and belongs to the technical field of medical image processing. Segmenting the palpebral fissure region and the cornea region through the segmentation network model; after the palpebral fissure region and the cornea region are segmented, a palpebral fissure region feature map and a cornea region feature map are obtained, and the curvature value of an eyelid line is measured by setting pixel point values and depicting a palpebral fissure edge line; during feature segmentation, a segmentation network model is constructed based on a double-u-type network, including setting an encoder and a decoder in each layer of the double-u-type network and keeping the number of convolution kernels of the encoder in each layer unchanged so as to identify and segment a target area; based on an encoder and a decoder, flexible adjustment of a decoding path is realized, and automatic measurement of a curvature value of an eyelid curve is realized by taking curvature similarity as guidance; therefore, the eyelid line curvature value of the testee can be conveniently, quickly and accurately obtained, and a user is helped to know the eyelid contour condition in time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a double U-shaped network eyelid line curvature measurement method based on a multi-task attention mechanism. Background Art

[0002] The shape of the eyelid contour not only expresses visual beauty and expression, but is also an important indicator in the evaluation of ocular surface diseases;

[0003] In the prior art, the eyelid line curvature is often measured by professional instruments and equipment. There are also many studies on the measurement of eyelid line curvature. For example, the palpebral fissure images of 29 patients with Graves' orbital disease, 22 patients with congenital ptosis and 50 patients with no medical history were transferred to a computer and processed using NIH Image1.55 software to obtain the upper and lower eyelid curves of the subjects, and the upper and lower eyelid contours were fitted using a second-order polynomial to obtain the corresponding curvature value; this allows doctors to adjust the contours of congenital ptosis without centering on the midline, but rather using a surgical method with a slight lateral shift to adjust the contours;

[0004] After analyzing the existing eyelid line curvature measurement methods, the following problems were found: 1. Existing relatively mature software is required to extract points on the eyelid line to depict the eyelid contour. Although the operation is relatively convenient, the extraction process still requires professional ophthalmologists to perform evaluation operations, which is time-consuming and labor-intensive. In addition, when selecting points on the eyelid contour to achieve curve fitting of the upper and lower eyelid lines, the number and specific positions of the points are easily affected by the subjective judgment of the operator; 2. After surgery on the upper and lower eyelids, the eyelid morphology needs to be dynamically observed to control the progression of the patient's condition. If the doctor is required to measure through a cumbersome process every time, it is not easy to achieve when there are few doctors and many patients. Summary of the invention

[0005] The purpose of the present invention is to provide a double U-shaped network eyelid line curvature measurement method based on a multi-task attention mechanism to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A double U-shaped network eyelid line curvature measurement method based on a multi-task attention mechanism, the method comprises the following steps:

[0008] Accurate segmentation of the palpebral fissure area and cornea area is achieved through the segmentation network model;

[0009] After the palpebral fissure area and the corneal area are segmented, a palpebral fissure area feature map and a corneal area feature map are obtained, and the palpebral fissure edge line is depicted by setting the pixel value to 1 or 0, and the curvature value of the eyelid line is measured;

[0010] Preferably, the segmentation network model is designed based on a double U-shaped network structure, including a backbone network, an encoder-decoder and an output converter;

[0011] Preferably, the backbone network is used for target area feature extraction, and includes a Doubleconv layer, an integrated attention mechanism layer and a downsampling layer connected in sequence, and the output of the previous layer is used as the input of the next layer, the Doubleconv layer is a double convolution layer, and the integrated attention mechanism layer is suitable for the CAMM structure and SAMM structure of the double U-type network;

[0012] Preferably, an encoder-decoder is set in each layer of the segmentation network model, and the encoder-decoder is based on U-shaped left-right symmetry, wherein the number of convolution kernels of the encoder in each layer remains unchanged, and the decoder adjusts the decoding path according to the target area characteristics of each layer during operation; the target area is the palpebral fissure area or the corneal area;

[0013] Preferably, the output converter is used to output the palpebral fissure area and the corneal area;

[0014] Preferably, based on the palpebral fissure area, approximate circular fitting of the corneal area is achieved in the palpebral fissure area in the form of a minimum circumscribed circle, and automatic measurement of the curvature value of the eyelid curve is achieved through the eyelid curve and the approximate circular curve guided by curvature similarity.

[0015] Furthermore, the decoder achieves image restoration of the target area through upsampling and skip-level connections;

[0016] In the process of image restoration of the target area, a decoding path is determined by upsampling and skip-type hierarchical connection. In each layer of the segmentation network model, a decoding path is constructed by serially connecting different layers based on the upsampling method.

[0017] Further, based on the decoding path, the target region features extracted by the segmentation network model at each level are obtained, and average pooling and global pooling are used to perform weighted processing on the target region features at different levels, wherein the weighted processing is performed once in one decoding path;

[0018] In the weighted processing process, different level weights are preset for the target area features extracted at different levels, and different levels have unique level weight attributes.

[0019] Furthermore, in the process of restoring the image of the target area, the target area features are made to conform to the preset sparsity and discrimination parameters through nonlinear mapping, and the original dimension of the target area is obtained; after weighted processing of the target area features at different levels is achieved by using average pooling and global pooling, dimension compression is performed on the basis of the original dimension;

[0020] The dimension compression method is as follows: taking the hierarchical weight as the coefficient of the target area feature at different hierarchical levels, and respectively calculating the average value after weighted processing under average pooling and the average value after weighted processing under global pooling, and selecting the maximum value from the average values ​​obtained under average pooling and global pooling, and presetting a boundary threshold. If the maximum value is greater than or equal to the boundary threshold, dimension compression is performed; otherwise, dimension compression is not performed and a decoding path is output;

[0021] During the dimensionality compression process, the decoding paths when the maximum value is greater than or equal to the boundary threshold are eliminated, and the selected maximum value is updated until the maximum value is less than the boundary threshold. The updating stops and the decoding path is output.

[0022] Furthermore, the average value and the maximum value generated under the output decoding path are sequentially input into the Sigmoid function, and the feature map of the target area is extracted, wherein the feature map includes the feature map of the palpebral fissure area and the feature map of the cornea area.

[0023] Furthermore, the palpebral fissure region feature map obtained by segmentation of the segmentation network model is used to obtain the palpebral fissure edge line through edge detection, wherein the pixel value on the palpebral fissure edge line is 1, and the pixel value outside the palpebral fissure edge line is 0.

[0024] Furthermore, for the corneal region feature map obtained by segmentation of the segmentation network model, an approximate circular fitting of the corneal region is implemented in the palpebral fissure region feature map in a manner of constructing a minimum circumscribed circle, thereby obtaining an approximate circle of the cornea.

[0025] Further, a two-dimensional coordinate system is established with the lower left corner of the palpebral fissure region feature map as the coordinate origin, and the coordinate values ​​of all pixel points in the palpebral fissure region feature map are traversed and marked; based on the two-dimensional coordinate system, the coordinates of the center point of the corneal approximate circle are identified, recorded as (m, n), and the corneal radius, recorded as r, where m and n respectively represent the values ​​of the horizontal coordinate and the vertical coordinate in the two-dimensional coordinate system;

[0026] Based on the center point coordinates and the corneal radius, the value range of the corneal independent variable of the corneal approximate circle on the horizontal coordinate of the two-dimensional coordinate system is obtained, which is recorded as [mr, m+r];

[0027] Tangents perpendicular to the horizontal coordinate are drawn at the corneal independent variable value mr and the corneal independent variable value m+r to obtain tangents on both sides of the cornea, and the eyelid curve in the palpebral fissure region feature map is depicted through the intersection of the tangents on both sides of the cornea and the palpebral fissure edge line. The eyelid curve includes an upper eyelid line and a lower eyelid line, wherein the intersection is a pixel point on the palpebral fissure edge line with a pixel value of 1;

[0028] The curve function corresponding to the eyelid curve is recorded as y=ax 2 +bx+c, a, b and c are curve coefficients, and the settings of a, b and c are different based on the difference between the upper eyelid line and the lower eyelid line. x and y represent the abscissa value and the ordinate value respectively;

[0029] Guided by curvature similarity and based on curve function, the curvature value calculation formula is obtained: and In the formula, K i represents the curvature value at the i-th pixel on the eyelid curve within the range of corneal independent variables [mr, m+r], r i represents the radius of the curvature circle at the i-th pixel point on the eyelid curve, represents the first-order partial derivative of the curve function, Represents the second-order partial derivative of the curve function.

[0030] A computer-readable storage medium is used to carry a computer program, and the computer program can be executed by a processor to implement the above method.

[0031] The double U-shaped network eyelid line curvature measurement mechanism based on the multi-task attention mechanism includes:

[0032] one or more processors;

[0033] A system device, for enabling the double U-shaped network eyelid line curvature measurement mechanism based on a multi-task attention mechanism to implement the above method when the one or more computer programs are executed by the one or more processors;

[0034] The system device includes a palpebral fissure area segmentation module, a corneal area segmentation module and an eyelid curvature measurement module;

[0035] The palpebral fissure region segmentation module is used to segment the palpebral fissure region and obtain a palpebral fissure region feature map;

[0036] The cornea region segmentation module is used to segment the cornea region and obtain a cornea region feature map;

[0037] The eyelid curvature measurement module realizes automatic measurement of the curvature value of the eyelid curve based on the palpebral fissure region feature map and the cornea region feature map.

[0038] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: in the eyelid line curvature measurement method based on the double U-shaped network multi-task attention mechanism provided by the present invention, the segmentation of the palpebral fissure area and the corneal area is realized by a segmentation network model; after the palpebral fissure area and the corneal area are segmented, a feature map of the palpebral fissure area and a feature map of the corneal area are obtained, and the palpebral fissure edge line is depicted by setting pixel point values, and the curvature value of the eyelid line is measured; during feature segmentation, a segmentation network model is constructed based on a double U-shaped network, including setting an encoder and a decoder in each layer of the double U-shaped network, and the number of convolution kernels of the encoder in each layer remains unchanged, so as to identify and segment the target area; based on the encoder and the decoder, flexible adjustment of the decoding path is realized, and automatic measurement of the curvature value of the eyelid curve is realized with curvature similarity as a guide;

[0039] Therefore, the eyelid line curvature value of the subject can be obtained conveniently, quickly and accurately, helping the user to understand the eyelid contour in a timely manner; it can speed up the training speed of the segmentation network without affecting the segmentation accuracy of the palpebral fissure and corneal area, while optimizing the feature extraction during the segmentation of the palpebral fissure and corneal area, thereby improving the accuracy of the measurement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0041] Figure 1 It is a schematic diagram of the structure of the eyelid fissure and corneal region segmentation network of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0042] Figure 2 It is a schematic diagram of the backbone network structure of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0043] Figure 3 It is a schematic diagram of the Doubleconv layer structure of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0044] Figure 4 It is a schematic diagram of the integrated attention mechanism layer structure of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0045] Figure 5 It is a schematic diagram of the downsampling layer structure of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0046] Figure 6It is a CAMM structure schematic diagram of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0047] Figure 7 It is a SAMM structure diagram of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention;

[0048] Figure 8 This is a system structure diagram of the double U-shaped network eyelid line curvature measurement method based on the multi-task attention mechanism of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Example 1, model design:

[0051] like Figure 1 As shown in the figure, a segmentation network is used to simultaneously segment the palpebral fissure area and the corneal area. Based on the encoder-decoder structure, feature extraction is performed through a series of double convolutional layers and pooling layers, and then the spatial resolution of the image is gradually restored through upsampling and jump connections. Therefore, it is possible to simultaneously process a variety of different segmentation tasks and generate a unified segmentation result. At the encoding and decoding connection of each layer, the model reduces the number of convolution kernels, thereby reducing the training parameters, while keeping the number of convolution kernels of the encoding module unchanged to ensure the quality of feature extraction. In addition, the segmentation network shares a feature extractor, so that only one encoder needs to be trained, and the decoding path can be adjusted according to different hierarchical features, which increases the flexibility and adaptability of the model. In order to further enhance the model's ability to capture key features, the model adds an integrated attention mechanism in the encoding module, which helps the model to more accurately identify and segment the target area, and adds CAMM structures at different positions of the encoding module (such as Figure 6 As shown), and SAMM structure (as Figure 7 As shown in the figure), the network can more easily focus on the features of the palpebral fissure and corneal area during the feature extraction of the fundus image, thus improving the segmentation accuracy of the model. The place where the attention mechanism module is added is represented by Block (as shown in the figure). Figure 4 As shown), it can be divided into two parts: CAMM and SAMM;

[0052] In the CAMM part, firstly, convolution and ReLU activation functions are used to reduce the dimension and enhance the nonlinear mapping of the model, improve the sparsity and discrimination of the features, and restore the original dimension; then, different features are weighted by using average pooling and global pooling, and different weights are assigned to different features to extract features, so that the network can focus on certain features, reduce the weights of features that are not very useful for the current task, and increase the weights of features that are useful for the current task; finally, the two discriminant features are fused so that the model considers the average value and maximum value information of the features at the same time, which enhances the model's diverse expression of input data and helps to improve the generalization ability and robustness of the model;

[0053] In the SAMM part, the sigmoid activation function is used to map the input features to the probability space, which enhances the model's attention to and ability to distinguish features. The average and maximum values ​​in each input feature map are then calculated to continue compressing the feature dimensions and extracting important information. Finally, the results are input into a convolutional layer and a sigmoid function in turn, and each element in each feature map is weighted to find the most important parts for processing.

[0054] The entire Block module can better extract the features of the input image and pay more attention to the feature extraction of the target area. The model generates segmentation results through two different output convolutional layers and merges these results to achieve simultaneous segmentation and recognition of the palpebral fissure and corneal area, which not only improves the accuracy of segmentation, but also optimizes the performance and efficiency of the model.

[0055] Embodiment 2, as Figure 8 As shown, the eyelid curvature measurement process:

[0056] S1. For the palpebral fissure area segmented by the segmentation network model, the palpebral fissure edge line is obtained by edge detection. The pixel value on the palpebral fissure edge line is 1, and the pixel value of the background part is 0. A coordinate system is established with the lower left corner of the image as the coordinate origin, and the coordinate values ​​of all pixels on the palpebral fissure edge line are obtained by traversing the entire image after edge detection;

[0057] S2, performing circular fitting on the corneal region obtained by segmentation of the segmentation network model through the minimum circumscribed circle, and calculating the fitted corneal region to obtain the coordinates c (m, n) of the center point of the corneal region and the corneal radius r;

[0058] S3, in S1, the coordinate values ​​of all points on the palpebral fissure margin line are obtained and stored in a list. In S2, the coordinates of the corneal center point c(m,n) and the radius of the cornea r are obtained. The range of the corneal area on the horizontal coordinate is [mr,m+r]. Find the pixel point whose pixel value is 1 when the horizontal coordinate is mr (or m+r), and you can get the coordinate values ​​of the points where the tangents on both sides of the cornea intersect with the upper and lower eyelid lines respectively. Use ax 2 +bx+c performs curve fitting on the selected upper and lower eyelid line parts; the upper eyelid line part and the lower eyelid line part corresponding to the abscissa of the eyelid line in the interval [mr,m+r] are used to obtain the curvature of the upper eyelid line and the curvature of the lower eyelid line respectively;

[0059] S4. Formula for calculating curvature: The curvature of a curve is the rotation rate of the tangent direction angle at a point on the curve relative to the arc length, which is defined by differential as the degree to which the curve deviates from a straight line. The curvature value in mathematics represents the degree of curvature of the curve at a certain point. The greater the curvature, the greater the curvature. The curvature of the curve at a certain point and the corresponding curvature circle at that point have the same curvature. Therefore, in practical problems, the curvature corresponding to a point on the curve is obtained by calculating the curvature radius at that point. and In the formula, K i represents the curvature value at the i-th pixel on the eyelid curve within the range of corneal independent variables [mr, m+r], r i represents the radius of the curvature circle at the i-th pixel point on the eyelid curve, represents the first-order partial derivative of the curve function, Represents the second-order partial derivative of the curve function;

[0060] S5. In S2, the center point coordinates c(m,n) of the corneal area and the corneal radius r are obtained, and the range of the horizontal coordinate corresponding to the middle area of ​​the curvature is calculated to be [mr,m+r]. The coordinate values ​​obtained in S2 and the fitted curve function in S3 are substituted into the curvature solution formula in S4 to obtain the curvature value corresponding to the coordinate point where the curvature needs to be calculated.

[0061] Example 3: Data testing:

[0062] Data selection: 130 ocular surface images with a size of 2976×1984 pixels provided by an ophthalmology hospital in a certain city were used as training and test data sets. Among them, 100 images were used as training sets to train the parameters of the constructed segmentation convolutional neural network; 30 images were used as test sets to evaluate the segmentation effect of the final segmentation network. In order not to change the proportion of the ocular surface images, the image resolution was uniformly scaled to 744×496, and then data was amplified by random cropping, random flipping in the horizontal direction, and random flipping in the vertical direction. The fundus images in the preprocessed training set were sent to the segmentation convolutional neural network for network training;

[0063] Hardware and software settings: The hardware and software equipment and network parameter settings are consistent during the segmentation of the palpebral fissure area and the corneal area. During the network training process: a total of 200 rounds of training, the number of images input in each batch is set to 1, random gradient descent is selected as the optimizer, and the initial learning rate is set to 0.01; Hardware configuration during the experiment: processor, a certain model i7-6700@3.40GHz quad-core; motherboard, a certain model Z170X-UD3-CF; memory, a certain model 16GB DDR42133MHz; graphics card, a certain model NVIDIA GeForce GTX 1080; Software configuration: The construction of the segmentation network and the measurement of the eyelid curvature are both implemented in Python. The compiler and version, virtual environment and version, and Python language version used in the experiment are PyCharm2023.1.1, PyTorch2.0.0, and Python3.9.16, respectively;

[0064] Model structure operation: Figure 2-Figure 5As shown, the size of x is represented as (HW), and the data x is input into the Doubleconv layer. The Doubleconv layer is composed of two convolutions, batch normalization and ReLU in series. The number of channels of its output x1 will change from 3 to basic_c, that is, 64, and the size remains unchanged; then x1 is input into the cb1 layer, that is, the integrated attention mechanism layer, and convolution, ReLU and convolution are used to reduce the dimension, enhance the nonlinear mapping of the model to improve the sparsity and discrimination of the features, and restore the original dimension. The convolution uses a 1×1 convolution kernel, and then uses 1×1 average pooling and global pooling to weight different features. Different features are assigned different weights to extract features, so that the network Focus on certain features, reduce the weight of features that are not very useful for the current task, increase the weight of features that are useful for the current task, pass the input through the Sigmoid activation function, map the features to the probability space, enhance the model's attention to and ability to distinguish features, then perform maximum pooling and average pooling on the feature map, continue to compress the feature dimension and extract important information, and finally input the results into a convolution layer and Sigmoid function in turn, perform weighted processing on each element in each feature map to find the most important part for processing, use a 7×7 convolution kernel for convolution operation, the step size is 1, and the padding is 3. At this time, the number of output channels is still 64, and the size remains unchanged; then pass through the down1 layer , that is, the downsampling layer, which is composed of a 2×2 maximum pooling and two convolutions in series. The convolution uses a 3×3 convolution kernel with a step size of 1. When the data passes through the downsampling layer, the number of channels will double, that is, 128, and the feature map size will become half of the original, that is, (H / 2W / 2); repeat the above steps, and pass the feature map through cb2, down2, cb3, down3, cb4, and down4 layers in turn. The number of channels of the final output x5 will become 512, and the size will become (H / 16W / 16); the encoder-decoder part can be divided into left and right sides, which are symmetrical. On the right, first x5 passes through the bilinear layer to double the size of its feature map, that is, ( H / 8W / 8), the number of channels is still 512, then x5 and x4 are concatenated into x, x is the same size as x4 and x5, but the number of channels is the sum of the two, then the number of channels of x is 1024, the size is still (H / 8W / 8), then pass it through the Doubleconv layer, the number of output channels becomes 1 / 4 of the input, the size remains unchanged, then the number of output channels becomes 256, the size is still (H / 8W / 8), then pass it through the bilinear layer and concatenate it with x3, then pass it through the Doubleconv layer, repeat the above steps until the output is concatenated with x1 and then pass through the Doubleconv layer to output x11, the number of channels is 64, the size is (HW), and the left side is also symmetrical to output x22;In the output transformation part, x11 and x22 are passed through an out_conv layer, which is two 1×1 convolutions in series, and the results are merged to achieve simultaneous segmentation and recognition of the palpebral fissure and corneal area;

[0065] Data results display: Table 1 shows the GA, average intersection-over-union ratio and Dice coefficient of the segmentation of the palpebral fissure area and corneal area;

[0066]

[0067] Table 2 shows the measurement results of the upper and lower eyelid line curvature of the targets in the test set using manual and automatic measurements;

[0068]

[0069]

[0070] Table 3 shows the intraclass correlation coefficients obtained on the test set using manual and automatic measurement methods;

[0071]

[0072] After segmenting the palpebral fissure area and the corneal area, the selected target upper and lower eyelid lines are fitted. After edge detection of the palpebral fissure area using the Canny algorithm, a coordinate axis can be established with the lower left corner vertex of the entire image as the origin. By establishing the coordinate axis and designing the algorithm, a list of coordinate points on the upper and lower eyelid lines can be obtained respectively. In order to avoid a certain horizontal coordinate value corresponding to multiple vertical coordinate values, which makes it difficult to fit the curve, the list is converted into a set, and finally the upper and lower eyelid lines are fitted with curves using the coordinate values ​​of the pixel points.

[0073] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0074] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A double U-shaped network eyelid line curvature measurement method based on a multi-task attention mechanism, the method comprising: Accurate segmentation of the palpebral fissure area and cornea area is achieved through the segmentation network model; After the palpebral fissure area and the corneal area are segmented, a palpebral fissure area feature map and a corneal area feature map are obtained, and the palpebral fissure edge line is depicted by setting the pixel value to 1 or 0, and the curvature value of the eyelid line is measured; Features: The segmentation network model is designed based on a double U-shaped network structure, including a backbone network, an encoder-decoder and an output converter; The backbone network is used for target area feature extraction, including a Doubleconv layer, an integrated attention mechanism layer and a downsampling layer connected in sequence, and the output of the previous layer is used as the input of the next layer, the Doubleconv layer is a double convolution layer, and the integrated attention mechanism layer is suitable for the CAMM structure and SAMM structure of the double U-type network; An encoder-decoder is set in each layer of the segmentation network model, and the encoder-decoder is based on U-shaped bilateral symmetry, wherein the number of convolution kernels of the encoder in each layer remains unchanged, and the decoder adjusts the decoding path according to the characteristics of the target area of ​​each layer during operation; the target area is the palpebral fissure area or the corneal area; The output converter is used to output the palpebral fissure area and the corneal area; Based on the palpebral fissure area, an approximate circular fitting of the corneal area is achieved in the palpebral fissure area in the form of a minimum circumscribed circle, and the curvature value of the eyelid curve is automatically measured through the eyelid curve and the approximate circular curve guided by the curvature similarity.

2. According to the method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism of claim 1, it is characterized in that: The decoder achieves image restoration of the target area through upsampling and skip-layer connection. In the process of image restoration of the target area, a decoding path is determined by upsampling and skip-type hierarchical connection. In each layer of the segmentation network model, a decoding path is constructed by serially connecting different layers based on the upsampling method.

3. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to claim 2 is characterized in that: Based on the decoding path, the target region features extracted by the segmentation network model at each level are obtained, and average pooling and global pooling are used to perform weighted processing on the target region features at different levels, wherein the weighted processing is performed once in one decoding path; In the weighted processing process, different level weights are preset for the target area features extracted at different levels, and different levels have unique level weight attributes.

4. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to claim 3 is characterized in that: In the process of restoring the image of the target area, nonlinear mapping is used to make the target area features conform to the preset sparsity and discrimination parameters, and the original dimension of the target area is obtained; after weighted processing of the target area features at different levels is achieved by using average pooling and global pooling, dimension compression is performed on the basis of the original dimension; The dimension compression method is as follows: taking the hierarchical weight as the coefficient of the target area feature at different hierarchical levels, and respectively calculating the average value after weighted processing under average pooling and the average value after weighted processing under global pooling, and selecting the maximum value from the average values ​​obtained under average pooling and global pooling, and presetting a boundary threshold. If the maximum value is greater than or equal to the boundary threshold, dimension compression is performed; otherwise, dimension compression is not performed and a decoding path is output; During the dimensionality compression process, the decoding paths when the maximum value is greater than or equal to the boundary threshold are eliminated, and the selected maximum value is updated until the maximum value is less than the boundary threshold. The updating stops and the decoding path is output.

5. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to claim 4 is characterized in that: The average value and the maximum value generated under the output decoding path are sequentially input into the Sigmoid function, and the feature map of the target area is extracted, wherein the feature map includes the feature map of the palpebral fissure area and the feature map of the cornea area.

6. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to claim 1 is characterized in that: The palpebral fissure region feature map obtained by segmentation of the segmentation network model is used to obtain the palpebral fissure edge line through edge detection, where the pixel value on the palpebral fissure edge line is 1, and the pixel value outside the palpebral fissure edge line is 0.

7. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to claim 1 is characterized in that: The corneal region feature map obtained by segmentation of the segmentation network model is constructed in a manner of minimum circumscribed circle to achieve approximate circular fitting of the corneal region in the palpebral fissure region feature map, thereby obtaining an approximate corneal circle.

8. The method for measuring the curvature of the double U-shaped network eyelid line based on the multi-task attention mechanism according to any one of claims 6 or 7, characterized in that: Taking the lower left corner of the palpebral fissure region feature map as the coordinate origin, a two-dimensional coordinate system is established, and the coordinate values ​​of all pixel points in the palpebral fissure region feature map are traversed and marked; Based on the two-dimensional coordinate system, the coordinates of the center point of the corneal approximate circle are identified, which are recorded as (m, n), and the corneal radius, which is recorded as r, wherein m and n represent the values ​​of the horizontal coordinate and the vertical coordinate in the two-dimensional coordinate system, respectively; Based on the center point coordinates and the corneal radius, the value range of the corneal independent variable of the corneal approximate circle on the horizontal coordinate of the two-dimensional coordinate system is obtained, which is recorded as [mr, m+r]; Tangents perpendicular to the horizontal coordinate are drawn at the corneal independent variable value mr and the corneal independent variable value m+r to obtain tangents on both sides of the cornea, and the eyelid curve in the palpebral fissure region feature map is depicted through the intersection of the tangents on both sides of the cornea and the palpebral fissure edge line. The eyelid curve includes an upper eyelid line and a lower eyelid line, wherein the intersection is a pixel point on the palpebral fissure edge line with a pixel value of 1; The curve function corresponding to the eyelid curve is recorded as y=ax 2 +bx+c, a, b and c are curve coefficients, and the values ​​of a, b and c are set differently based on the difference between the upper eyelid line and the lower eyelid line. x and y represent the abscissa value and the ordinate value respectively; Guided by curvature similarity and based on curve function, the curvature value calculation formula is obtained: and In the formula, K i represents the curvature value at the i-th pixel on the eyelid curve within the range of corneal independent variables [mr, m+r], r i represents the radius of the curvature circle at the i-th pixel point on the eyelid curve, represents the first-order partial derivative of the curve function, Represents the second-order partial derivative of the curve function.

9. A computer-readable storage medium, characterized in that: The readable storage medium is used to carry a computer program, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. Double U-shaped network eyelid line curvature measurement mechanism based on multi-task attention mechanism, characterized in that: include: one or more processors; A system device, for enabling the double U-shaped network eyelid line curvature measurement mechanism based on a multi-task attention mechanism to implement the steps of the method according to any one of claims 1 to 8 when the one or more computer programs are executed by the one or more processors; The system device includes a palpebral fissure area segmentation module, a corneal area segmentation module and an eyelid curvature measurement module; The palpebral fissure region segmentation module is used to segment the palpebral fissure region and obtain a palpebral fissure region feature map; The cornea region segmentation module is used to segment the cornea region and obtain a cornea region feature map; The eyelid curvature measurement module realizes automatic measurement of the curvature value of the eyelid curve based on the palpebral fissure region feature map and the cornea region feature map.

Citation Information

Patent Citations

  • Image segmentation method based on double-U-shaped network framework

    CN111340816A

  • Multi-scene iris recognition method based on deep learning

    CN113591747A

  • Eyeball motion segmentation positioning method based on cyclic residual convolutional neural network

    CN114694236A

  • Full-automatic eyelid disease postoperative appearance prediction system and method based on deep learning

    CN115909470A

  • Method for segmenting hard exudate of retina fundus image

    CN116485806A

Cited By

  • Eyelid morphological parameter calculation method

    CN120375453A