Training method and segmentation method based on eyelid margin segmentation model

Abstract

The invention relates to the technical field of deep learning, in particular to a training method and a segmentation method of an eyelid edge segmentation model, and the method comprises the steps: S1, constructing a data training set based on a preset eye image data set; s2, training a pre-constructed eyelid edge segmentation model according to the data training set to obtain a trained eyelid edge segmentation model; in the training process, when any first encoder receives input data, a result is output according to a preset formula I, and the formula I is Ei = RSU (Ei-1); wherein when i is equal to 1, Ei-1 is the initial image data of the input eyelid edge segmentation model; when any first decoder receives the input data, a result is output according to a preset formula II, and the formula II is Di = RSU (Di + 1 + Ei); wherein when i = N-1, Di + 1 is Ei + 1. The method has the beneficial effect that the dependence on the data volume is reduced by a relatively simple network architecture.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a training method and a segmentation method based on an eyelid margin segmentation model. Background Art

[0002] The eyelid margin is the boundary where the eyelid skin meets the conjunctiva. A healthy eyelid margin is essential for maintaining a healthy ocular surface. Blepharitis is a verification of the eyelid margin and one of the most common eye diseases. It is estimated that 47% of patients seen by ophthalmologists suffer from blepharitis, which often leads to blepharkeratoconjunctivitis (BKC), meibomian gland dysfunction, and dry eyes. Among them, blepharitis can cause astigmatism, corneal lesions, amblyopia, vision loss, or corneal perforation. Eyelid margin segmentation has a wide range of applications in medical imaging, virtual reality, and augmented reality.

[0003] At present, eyelid margin segmentation mainly relies on manually designed features or pre-trained models, which often do not work well when dealing with complex scenes. In addition, although many salient object detection technologies have improved in performance, their network architectures are complex, computationally expensive, and most of them rely on pre-trained backbone networks for image classification tasks, making them difficult to adapt to different working environments.

[0004] Artificial intelligence algorithms represented by deep learning have achieved outstanding results in image classification, target detection and other fields by relying on ultra-large-scale data sets and powerful computing resources. Therefore, there is an urgent need for an eyelid margin segmentation model based on deep learning, which can reduce the dependence on data volume with a relatively simple network architecture. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a training method and a segmentation method based on an eyelid margin segmentation model, which solve the technical problems that the current methods relying on manually designed features or pre-trained models have poor effects in processing complex scenes, the technical problems that the target detection technology has a complex network structure and high computational cost, and the technical problem that the backbone network relying on image classification tasks is difficult to adapt to different working environments.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In a first aspect, an embodiment of the present invention provides a method for training an eyelid margin segmentation model, comprising:

[0010] S1. constructing a data training set based on a preset eye image data set; the data training set includes at least one eye image data and a mask corresponding to each eye image data;

[0011] S2, training the pre-built eyelid margin segmentation model according to the data training set to obtain a trained eyelid margin segmentation model; the eyelid margin segmentation model is based on U 2 -Net model, including N-level first encoders and N-1-level first decoders; all the first encoders and first decoders are composed of feature extraction structures RSU-L or RSU-4F;

[0012] During the training process, when any first encoder receives input data, it outputs a result according to a preset formula 1, which is:

[0013] E i =RSU(E i-1 );

[0014] Among them, E i is the output of the first encoder at the i-th level. When i=1, E i-1 That is, the initial image data input into the eyelid margin segmentation model;

[0015] When any first decoder receives input data, it outputs a result according to a preset formula 2, which is:

[0016] D i =RSU(D i+1 +E i );

[0017] Among them, when i=N-1, D i+1 For E i+1 , Di is the output of the i-th or first decoder.

[0018] Optionally, the method further comprises:

[0019] When any feature extraction structure RSU-L or RSU-4F receives input data, the convolution layer obtains a first process feature map based on the received input data and the preset formula three;

[0020] The formula three is:

[0021] z p,j,o =(x*W o ) i,j +b o ;

[0022] Among them, p and j are the indexes of the first process feature map, z is the first process feature map output by the convolution layer, * is the convolution operation, x is the input image data, and Wo is the oth convolution kernel, b o is the oth bias, o is the channel number;

[0023] Subsequently, the first process feature map is batch normalized and input value activated based on a preset batch normalization formula and ReLU activation function;

[0024] The batch normalization formula is:

[0025]

[0026] Among them, u o and σ o are the mean and variance of the oth feature map, respectively, and ∈ is a preset constant;

[0027] The ReLU activation function is:

[0028]

[0029] Then, the first process feature map processed by batch normalization and ReLU activation function is downsampled according to the preset maximum pooling formula, and the maximum pooling formula is:

[0030] y p,j,o =max s,f (a p+s,j+f,o );

[0031] Where, (s, f) is the preset pooling window size;

[0032] Finally, the downsampled first process characteristic graph is upsampled according to a preset bilinear difference formula to obtain a second process characteristic graph;

[0033] The bilinear difference formula is:

[0034]

[0035] Where (X, Y) is the position where the difference is performed in the first process characteristic map, I p,j is the pixel value of the four pixels around (X, Y), w pj is the pre-set interpolation weight.

[0036] Optionally, the S2 further includes:

[0037] During the training process, the difference between the model prediction result and the mask is obtained according to the preset cross entropy loss function, and the model is optimized by the preset gradient descent algorithm until a trained eyelid margin segmentation model is obtained;

[0038] The cross entropy loss function is:

[0039]

[0040] in, is the difference between the model prediction result and the mask, M is the number of eye image data in the data training set, and y k is the mask corresponding to the eye image data numbered k, is the prediction result corresponding to the eye image data numbered k.

[0041] Optionally, the gradient descent algorithm is an AdamW optimizer, and the update formula of the AdamW optimizer is:

[0042]

[0043] Among them, m t and v t is the momentum term, η is the learning rate, λ is the weight decay coefficient, θ is the model parameter, β 1 and β 2 is the momentum parameter and t is the number of iterations.

[0044] Optionally, when N is 6, the eyelid margin segmentation model includes a 6-level first encoder and a 5-level first decoder, and a saliency map fusion device;

[0045] Among them, the first encoders of the first four levels are RSU-7, RSU-6, RSU-5 and RSU-4 respectively, and the first decoders of the first four levels are RSU-7, RSU-6, RSU-5 and RSU-4 respectively; the first encoder of the fifth level, the first encoder of the sixth level and the first decoder of the fifth level are all RSU-4F;

[0046] The input of the first encoder of the first stage is the initial input of the eyelid margin segmentation model, and the input of any first encoder except the first encoder of the first stage is the output of the first encoder of the previous stage of the first encoder;

[0047] In the first four levels of first decoders, the input of any first decoder is the result of splicing the output of the first encoder at the same level corresponding to the first decoder and the output of the first decoder at the next level of the first decoder;

[0048] The input of the fifth-stage first decoder is the result of concatenating the output of the fifth-stage first encoder and the output of the sixth-stage first encoder.

[0049] Optionally, each of the RSU-Ls includes L second encoders and L-1 second decoders;

[0050] The RSU-4F has a height of 4 and a symmetrical encoder structure identical to that of the RSU-L.

[0051] L is a non-zero natural number.

[0052] Optionally, the S1 includes:

[0053] Preprocessing all eye image data in a preset eye image data set, and constructing a data training set based on the preprocessed eye image data set;

[0054] The preprocessing includes normalization, resizing and noise removal.

[0055] Optionally, the S2 further includes:

[0056] A pre-set data validation set is used to evaluate the performance of the trained eyelid margin segmentation model, and the hyperparameters of the eyelid margin segmentation model are adjusted according to the performance evaluation results.

[0057] In a second aspect, an embodiment of the present invention provides a method for segmenting an eyelid margin, the method being based on a trained eyelid margin segmentation model and comprising:

[0058] S3. Input the received original feature image into the trained eyelid margin segmentation model for image segmentation processing to obtain a segmented image corresponding to the original feature image; the segmented image is an image with the eyelid margin area extracted.

[0059] Optionally, the S3 includes:

[0060] Input the received original feature image into the trained eyelid margin segmentation model for image segmentation processing to obtain a segmented image corresponding to the original feature image;

[0061] During this period, the outputs of the last level encoder and all decoders are processed through 3×3 convolution and Sigmoid function, and a corresponding saliency probability map is output from the first encoder of the last level and all first decoders respectively;

[0062] All output saliency probability maps are upsampled to the same size as the original feature image and fused through a cascade operation;

[0063] Generate the segmentation image corresponding to the original feature image through a 1×1 convolution layer and a Sigmoid function;

[0064] The eyelid margin segmentation model also includes a saliency map fuser.

[0065] (III) Beneficial effects

[0066] The beneficial effect of the present invention is that the eyelid margin model training method of the present invention adopts an artificial intelligence algorithm represented by deep learning. Compared with the prior art, it can use a relatively simple network architecture to reduce dependence on data volume and adapt to different working environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A flow chart of a training method for an eyelid margin segmentation model;

[0068] Figure 2 is a schematic diagram of the original feature image;

[0069] Figure 3 Schematic diagram of the segmentation result corresponding to the original feature map;

[0070] Figure 4 Schematic diagram of the eyelid margin segmentation model structure. DETAILED DESCRIPTION

[0071] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0072] The eyelid margin model training method proposed in the embodiment of the present invention adopts an artificial intelligence algorithm represented by deep learning. Compared with the existing technology, it can adapt to different working environments with a relatively simple network architecture, reduce dependence on data volume.

[0073] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0074] Example 1

[0075] This embodiment provides a training method for an eyelid margin segmentation model and an eyelid margin segmentation method. The training method, such as Figure 1 As shown, including:

[0076] S1. construct a data training set based on a preset eye image dataset;

[0077] S2. Training a pre-built eyelid margin segmentation model according to the data training set to obtain a trained eyelid margin segmentation model.

[0078] S1 includes: collecting eye image data including eyelid margins, constructing an eye image dataset, the eye image dataset including high-resolution eye image data, and annotating each eye image data in detail (i.e., a mask corresponding to each eye image data) to ensure the accuracy and completeness of the data.

[0079] All eye image data in the eye image dataset are preprocessed, including image normalization, resizing, and noise removal. Normalization helps eliminate brightness and contrast differences between different images, resizing ensures that the image meets the requirements of the model input, and noise removal improves the quality of the image.

[0080] The eye image dataset is divided into a data training set and a data validation set in a ratio of 8:2.

[0081] The S2 includes: using the data training set to train the eyelid margin segmentation model; the eyelid margin segmentation model is based on U 2 -Net model, while U 2 The -Net model is a deep learning network structure designed specifically for image segmentation tasks. It achieves efficient feature extraction and segmentation through a combination of encoders and decoders.

[0082] During the training process, the cross entropy loss function is used to calculate the difference between the model prediction results and the true annotations (that is, the difference between the model prediction results and the mask), and the model parameters are continuously optimized through the gradient descent algorithm to improve the segmentation accuracy of the model.

[0083] The cross entropy loss function is:

[0084]

[0085] in, is the difference between the model prediction result and the mask, M is the number of eye image data in the data training set, and y k is the mask corresponding to the eye image data numbered k, is the prediction result corresponding to the eye image data numbered k.

[0086] The gradient descent algorithm usually uses the AdamW optimizer, and the update formula of the AdamW optimizer is:

[0087]

[0088] Among them, m t and v t is the momentum term, η is the learning rate, λ is the weight decay coefficient, θ is the model parameter, β 1 and β 2 is the momentum parameter and t is the number of iterations.

[0089] The performance of the trained eyelid margin segmentation model is evaluated using the data validation set. The model's hyperparameters are adjusted to ensure the generalization ability of the model on different data sets, thereby improving the robustness and accuracy of the model.

[0090] The eyelid margin segmentation method includes: Figure 2 As shown in FIG. 1 ), the trained eyelid margin segmentation model is input for segmentation processing. The eyelid margin segmentation model generates a segmentation result according to the image features of the original feature image, such as Figure 3 As shown, the eyelid margin region of interest is extracted, and these segmentation results can be used for subsequent analysis and diagnosis of the eyelid margin health status.

[0091] The eyelid margin segmentation model training method and eyelid margin segmentation method provided in this embodiment do not rely on a pre-trained backbone network, are trained from scratch, and are adaptable to different working environments.

[0092] Example 2

[0093] This embodiment provides a method for training an eyelid margin segmentation model, including:

[0094] Based on the pre-trained eye image dataset, build a data training set and a data validation set;

[0095] Training a pre-built eyelid margin segmentation model according to the data training set to obtain a trained eyelid margin segmentation model;

[0096] The eyelid margin segmentation model is based on U 2 -Net model, such as Figure 4 As shown, it includes 6 levels of first encoders and 5 levels of first decoders; wherein, the first 4 levels of first encoders (i.e., En_1, En_2, En_3 and En_4) are RSU-7, RSU-6, RSU-5 and RSU-4 structures respectively, and the first 4 levels of first decoders (i.e., De_1, De_2, De_3 and De_4) are RSU-7, RSU-6, RSU-5 and RSU-4 structures respectively; the 5th level first encoder, the 6th level first encoder and the 5th level first decoder (i.e., En_5, En_6 and De_5) are all RSU-4F structures; each RSU-L includes L second encoders and L-1 second decoders; RSU-4F is a symmetric encoder structure with a height of 4 and the same as RSU-L; L is a non-zero natural number.

[0097] Among them, the input of the first-level first encoder is the initial input of the eyelid margin segmentation model, and the input of any first encoder except the first-level first encoder is the output of the first encoder of the previous level of the first encoder; in the first four levels of first decoders, the input of any first decoder is the concatenation result of the output of the first encoder of the same level corresponding to the first decoder and the output of the first decoder of the next level of the first decoder; the input of the fifth-level first decoder is the concatenation result of the output of the fifth-level first encoder and the output of the sixth-level first encoder.

[0098] That is, the input of En_1 is the original feature image input to the eyelid margin segmentation model, the input of En_2 is the output of En_1, the input of En_3 is the output of En_2, the input of En_4 is the output of En_3, the input of En_5 is the output of En_4, and the input of En_6 is the output of En_5.

[0099] The input of De_1 is the concatenation of the output of En_1 and the output of De_2; the input of De_2 is the concatenation of the output of En_2 and the output of De_3; the input of De_3 is the concatenation of the output of En_3 and the output of De_4; the input of De_4 is the concatenation of the output of En_4 and the output of De_5; the input of De_5 is the concatenation of the output of En_5 and the output of En_6.

[0100] That is, during the training process, when any first encoder receives input data, it outputs a result according to a preset formula 1, which is:

[0101] E i =RSU(E i-1 );

[0102] Among them, E i is the output of the first encoder at the i-th level. When i=1, E i-1 That is, the initial image data input into the eyelid margin segmentation model;

[0103] When any first decoder receives input data, it outputs a result according to a preset formula 2, which is:

[0104] D i =RSU(D i+1 +E i );

[0105] Among them, when i=N-1, D i+1 For E i+1 , Di is the output of the i-th or first decoder.

[0106] When any feature extraction structure RSU-L or RSU-4F receives input data, the convolution layer obtains a first process feature map based on the received input data and the preset formula three;

[0107] The formula three is:

[0108] z p,j,o =(x*W o ) i,j +b o ;

[0109] Among them, p and j are the indexes of the first process feature map, z is the first process feature map output by the convolution layer, * is the convolution operation, x is the input image data, and W o is the oth convolution kernel, b o is the oth bias, o is the channel number;

[0110] Subsequently, the first process feature map is batch normalized and input value activated based on a preset batch normalization formula and ReLU activation function;

[0111] The batch normalization formula is:

[0112]

[0113] Among them, u o and σ o are the mean and variance of the oth feature map, respectively, and ∈ is a preset constant;

[0114] The ReLU activation function is:

[0115]

[0116] Then, the first process feature map processed by batch normalization and ReLU activation function is downsampled according to the preset maximum pooling formula, and the maximum pooling formula is:

[0117] y p,j,o =max s,f (a p+s,j+f,o );

[0118] Where, (s, f) is the preset pooling window size;

[0119] Finally, the downsampled first process characteristic graph is upsampled according to a preset bilinear difference formula to obtain a second process characteristic graph;

[0120] The bilinear difference formula is:

[0121]

[0122] Where (X, Y) is the position where the difference is performed in the first process characteristic map, I p,j is the pixel value of the four pixels around (X, Y), w pj is the pre-set interpolation weight.

[0123] The present embodiment provides a method for training an eyelid margin segmentation model, which adopts a double-layer nested U structure and an RSU block design, and can effectively extract multi-scale features and achieve high-precision eyelid margin segmentation. The RSU block does not significantly increase the computational cost while maintaining high resolution, and is suitable for environments with limited computing resources.

[0124] Example 3

[0125] A method for training an eyelid margin segmentation model, the method is based on multi-stage adaptive training of contrastive learning and few-sample learning, and is based on the pre-constructed eyelid margin segmentation model in the above-mentioned embodiment 1 or embodiment 2, comprising:

[0126] We collect large-scale unlabeled eye image data including eyelid margins to construct a dataset, and use this dataset to perform contrastive learning self-supervised pre-training on the eyelid margin segmentation model. By comparing the multi-view representations of the image, the eyelid margin segmentation model learns to extract the key features of the image without relying on labels.

[0127] A small amount of labeled data is collected to perform few-shot learning on the eyelid margin segmentation model. A meta-learning framework based on contrastive learning is adopted to enable the model to quickly adapt and learn when faced with a small number of new task samples and generate high-quality segmentation results.

[0128] Based on contrastive learning and few-shot learning, we design multiple stages of adaptive training, each of which is optimized for different characters or data sets. These stages can combine contrastive learning and few-shot learning techniques to improve the generalization ability and learning efficiency of the model on small data sets.

[0129] At each stage of training, dynamically generate task-specific data augmentation samples, using a generative adversarial network (GAN) or a variational autoencoder (VAE) to generate augmented samples, especially for sample types where the model performs poorly on the current task (such as complex shapes, low-contrast images, etc.). Specifically, according to specific needs, design a series of data transformation operations, such as image rotation, flipping, scaling, etc., to generate sample sets. Use pre-trained generative models, such as a generative adversarial network or a variational autoencoder, to generate new datasets based on the above sample sets.

[0130] The sampling reinforcement learning mechanism dynamically adjusts the task weights in multi-task learning. The loss function weights of each task (such as edge detection, region segmentation, semantic segmentation, etc.) in each training iteration are determined through environmental feedback to optimize segmentation performance. That is, the best enhancement strategy is directly searched on the data set of interest through a pre-set search algorithm (such as reinforcement learning). These strategies can contain a variety of image processing operations and response hyperparameters.

[0131] The dynamic feature fusion and decoupling mechanism is introduced. In the framework of multi-task learning, the model can automatically choose to fuse the feature representations of different characters, or decouple these features when necessary to prevent information interference. The dynamic feature fusion and decoupling mechanism allows the model to flexibly switch between shared features or independent learning between different tasks, improving the adaptability and segmentation accuracy of the model, especially in complex or changing task environments.

[0132] Subsequently, an adversarial training mechanism is introduced to process the original sample set collected in advance, generate an adversarial sample set, and test and enhance the robustness of the model by perturbing the input data. Adversarial training can effectively improve the model's ability to cope with various input interferences, especially in practical applications, when the input data may have problems such as noise and bias, the model can still maintain a stable segmentation effect. Specifically, an adversarial attack method (such as FGSM, PGD) is used to generate an adversarial sample set, and the adversarial sample set is mixed with the original sample set to form an enhanced data set, and the model is trained with the enhanced data set.

[0133] Subsequently, the uncertainty of the model in each prediction is estimated, and the model's prediction output is adaptively adjusted or post-processed according to the uncertainty level. Bayesian Neural Networks or Monte Carlo-based Dropout methods can be introduced for uncertainty assessment. Through uncertainty estimation, the model can identify high-risk prediction areas, and then further refine or adjust the output in these areas, thereby improving the reliability of the overall segmentation.

[0134] Example 4

[0135] A method for training an eyelid margin segmentation model, the method being based on the pre-constructed eyelid margin segmentation model in the above-mentioned embodiment 1 or embodiment 2, comprising:

[0136] The pre-constructed eyelid margin segmentation model is trained according to the data training set, and the size of the loss function value is calculated. The model parameters are updated through the back propagation algorithm. The number of iterations reaches the maximum, and the regularization parameter B that obtains the smallest loss function value during the iteration process is selected as the optimal regularization parameter; the regularization parameter obtained by Bayesian optimization is used to verify the performance of the model using a pre-set verification data set. After the verification is completed, the model corresponding to the optimal regularization parameter is the final trained eyelid margin segmentation model.

[0137] Specifically, the randomly selected regularization parameter B is used as the input a of F(a), and the corresponding loss value is used as the output R, R = F(a), then the output F(a) is generated by a Gaussian process and is expressed as:

[0138] F(a)~GP(c(a),v(a,a′));

[0139] Where c(a) is the mean function and v(a, a′) is the covariance function;

[0140] For a set of parameters, we get an output R, which is used to update the next Gaussian process. Let D represent all the data, then:

[0141] D={(a 1 , R 1 ),(a 2 , R 2 ),…,(a n , R n )};

[0142] Then we get the mean μ(A) and variance σ of the Gaussian process 2 .

[0143] Use expectation maximization to select a new a * To carry out the next Gaussian process, where a * = arg max a∈A μ(x), add an exploration term based on expectation maximization, and set the exploration term to Gaussian distribution. Then the new a * Expressed as: a * = arg max a∈A μ(x)+Uσ(x);

[0144] Where U is the exploration coefficient, σ(x) is the prediction standard deviation of the Gaussian process at sample point a;

[0145] By continuously calculating and obtaining the maximum expected value point a * The actual value of R(x) corresponding to the actual value is used to update the Gaussian process with the minimum value with the highest prediction probability in the least number of calculation steps as the target;

[0146] Repeatedly select the parameter combination of the next sampling point and perform the Gaussian process and update the Gaussian process until the maximum number of iterations is reached or the termination condition is met. At this time, a * is the optimal regularization parameter, and the trained eyelid margin segmentation model is obtained.

[0147] Use Bayesian optimization to automatically select and adjust the model's hyperparameters to ensure the best performance of the model in different application scenarios. Automated hyperparameter optimization can reduce manual intervention and trial and error costs, improve model tuning efficiency, and ensure high performance of the model before deployment.

[0148] Example 5

[0149] A method for segmenting an eyelid margin, the segmentation method is based on the eyelid margin segmentation model trained in any one of the above embodiments 1 to 3, comprising:

[0150] S3, using edge computing or hybrid cloud computing to achieve real-time adaptive deployment of models;

[0151] The received original feature image is input into the trained eyelid margin segmentation model for image segmentation processing to obtain a segmented image corresponding to the original feature image; the segmented image is an image with the eyelid margin area extracted.

[0152] During this period, the outputs of the last level encoder and all decoders are processed through 3×3 convolution and Sigmoid function, and a corresponding saliency probability map is output from the first encoder of the last level and all first decoders respectively;

[0153] All output saliency probability maps are upsampled to the same size as the original feature image and fused through a cascade operation;

[0154] Generate the segmentation image corresponding to the original feature image through a 1×1 convolution layer and a Sigmoid function;

[0155] The eyelid margin segmentation model also includes a saliency map fuser.

[0156] This embodiment adopts U 2 -Net's eyelid margin segmentation model does not rely on a pre-trained backbone network and can be trained from scratch to adapt to different working environments while reducing dependence on computing resources.

[0157] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0158] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0159] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0160] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0161] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for training an eyelid margin segmentation model, characterized in that: include: S1. construct a data training set based on a preset eye image dataset; The data training set includes at least one eye image data and a mask corresponding to each eye image data; S2, training the pre-built eyelid margin segmentation model according to the data training set to obtain a trained eyelid margin segmentation model; the eyelid margin segmentation model is based on U 2 -Net model, including N-level first encoders and N-1-level first decoders; all the first encoders and first decoders are composed of feature extraction structures RSU-L or RSU-4F; During the training process, when any first encoder receives input data, it outputs a result according to a preset formula 1, which is: AND i =RSU(E i-1 ); Among them, E i is the output of the first encoder at the i-th level. When i=1, E i-1 That is, the initial image data input into the eyelid margin segmentation model; When any first decoder receives input data, it outputs a result according to a preset formula 2, which is: D i =RSU(D i+1 +E i ); Among them, when i=N-1, D i+1 For E i+1 , Di is the output of the first decoder at the i-th level.

2. The eyelid margin segmentation model training method according to claim 1, characterized in that: The method further comprises: When any feature extraction structure RSU-L or RSU-4F receives input data, the convolution layer obtains a first process feature map based on the received input data and the preset formula three; The formula three is: z p,j,o =(x*W o ) i,j +b o ; Among them, p and j are the indexes of the first process feature map, z is the first process feature map output by the convolution layer, * is the convolution operation, x is the input image data, and W o is the oth convolution kernel, b o is the oth bias, o is the channel number; Subsequently, the first process feature map is batch normalized and input value activated based on a preset batch normalization formula and ReLU activation function; The batch normalization formula is: Among them, u o and σ o are the mean and variance of the oth feature map, respectively, and ∈ is a preset constant; The ReLU activation function is: Then, the first process feature map processed by batch normalization and ReLU activation function is downsampled according to the preset maximum pooling formula, and the maximum pooling formula is: and p,j,o =max s,f (to p+s,j+f,o ); Where, (s, f) is the preset pooling window size; Finally, the downsampled first process characteristic graph is upsampled according to a preset bilinear difference formula to obtain a second process characteristic graph; The bilinear difference formula is: Where (X, Y) is the position where the difference is performed in the first process characteristic map, I p,j is the pixel value of the four pixels around (X, Y), w pj is the pre-set interpolation weight.

3. The eyelid margin segmentation model training method according to claim 1, characterized in that: The S2 further includes: During the training process, the difference between the model prediction result and the mask is obtained according to the preset cross entropy loss function, and the model is optimized by the preset gradient descent algorithm until a trained eyelid margin segmentation model is obtained; The cross entropy loss function is: in, is the difference between the model prediction result and the mask, M is the number of eye image data in the data training set, and y k is the mask corresponding to the eye image data numbered k, is the prediction result corresponding to the eye image data numbered k.

4. The eyelid margin segmentation model training method according to claim 3, characterized in that: The gradient descent algorithm is the AdamW optimizer, and the update formula of the AdamW optimizer is: Among them, m t and v t is the momentum term, η is the learning rate, λ is the weight decay coefficient, θ is the model parameter, β1 and β2 are momentum parameters, and t is the number of iterations.

5. The eyelid margin segmentation model training method according to claim 1, characterized in that: When N is 6, the eyelid margin segmentation model includes a 6-level first encoder and a 5-level first decoder, and a saliency map fusion device; Among them, the first encoders of the first four levels are RSU-7, RSU-6, RSU-5 and RSU-4 respectively, and the first decoders of the first four levels are RSU-7, RSU-6, RSU-5 and RSU-4 respectively; the first encoder of the fifth level, the first encoder of the sixth level and the first decoder of the fifth level are all RSU-4F; The input of the first encoder of the first stage is the initial input of the eyelid margin segmentation model, and the input of any first encoder except the first encoder of the first stage is the output of the first encoder of the previous stage of the first encoder; In the first four levels of first decoders, the input of any first decoder is the result of splicing the output of the first encoder at the same level corresponding to the first decoder and the output of the first decoder at the next level of the first decoder; The input of the fifth-stage first decoder is the result of concatenating the output of the fifth-stage first encoder and the output of the sixth-stage first encoder.

6. The eyelid margin segmentation model training method according to claim 1, characterized in that: Each of the RSU-Ls comprises L second encoders and L-1 second decoders; The RSU-4F has a height of 4 and a symmetrical encoder structure identical to that of the RSU-L. L is a non-zero natural number.

7. The eyelid margin segmentation model training method according to claim 1, characterized in that: The S1 includes: Preprocessing all eye image data in a preset eye image data set, and constructing a data training set based on the preprocessed eye image data set; The preprocessing includes normalization, resizing and noise removal.

8. The eyelid margin segmentation model training method according to claim 1, characterized in that: The S2 further includes: A pre-set data validation set is used to evaluate the performance of the trained eyelid margin segmentation model, and the hyperparameters of the eyelid margin segmentation model are adjusted according to the performance evaluation results.

9. A method for segmenting an eyelid margin, the method being based on the eyelid margin segmentation model trained in any one of claims 1 to 8, characterized in that: The segmentation method comprises: S3. Input the received original feature image into the trained eyelid margin segmentation model for image segmentation processing to obtain a segmented image corresponding to the original feature image; the segmented image is an image with the eyelid margin area extracted.

10. The eyelid margin segmentation method according to claim 9, characterized in that: The S3 includes: Input the received original feature image into the trained eyelid margin segmentation model for image segmentation processing to obtain a segmented image corresponding to the original feature image; During this period, the outputs of the last level encoder and all decoders are processed through 3×3 convolution and Sigmoid function, and a corresponding saliency probability map is output from the first encoder of the last level and all first decoders respectively; All output saliency probability maps are upsampled to the same size as the original feature image and fused through a cascade operation; Generate the segmentation image corresponding to the original feature image through a 1×1 convolution layer and a Sigmoid function; The eyelid margin segmentation model also includes a saliency map fuser.

Images