Eyelid edge feature classification system

Abstract

The invention relates to the technical field of image processing, in particular to an eyelid edge feature classification system, which comprises an eyelid edge segmentation module used for inputting a received original feature image into an eyelid edge segmentation model to obtain a segmented image corresponding to the original feature image; wherein the outputs of the last-stage encoder and all decoders of the eyelid edge segmentation model are subjected to 3 * 3 convolution and Sigmoid function processing to generate significant probability graphs, all the significant probability graphs are subjected to upward sampling and then fused, and finally, through 1 * 1 convolution and Sigmoid function processing, a segmentation image corresponding to an original feature graph is generated; and the feature classification module is used for inputting the segmented image into a feature classification model to obtain a feature classification result. The beneficial effects of the method are that through the eyelid edge segmentation model and the feature classification model, feature classification is assisted by utilizing segmentation information, optimization is carried out according to the eyelid subtraction features, and the effect of processing eyelid edge features is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a feature classification system for eyelid margins. Background Art

[0002] The eyelid margin is the junction of the eyelid skin and the conjunctiva. Maintaining a healthy eyelid margin is essential for the health of the overall ocular surface. Blepharitis is one of the most common eye diseases, affecting approximately 47% of ophthalmological patients. Blepharitis often causes meibomian gland dysfunction, dry eyes, and blepharokeratoconjunctivitis (BKC). BKC may cause astigmatism, corneal lesions, amblyopia, vision loss, or corneal perforation. Therefore, early diagnosis and treatment of blepharitis are very important to prevent vision loss. According to the American Academy of Ophthalmology classification, blepharitis is divided into anterior blepharitis and posterior blepharitis according to anatomical location.

[0003] In recent years, deep learning has shown significant potential in medical image segmentation and classification. A study proposed a deep learning-based method to automatically and objectively evaluate the morphological features of the eyelids. The method showed extremely high reliability and repeatability, showing great potential in the automatic diagnosis and remote monitoring of eyelid-related diseases. Another study applied deep learning technology to analyze the eyelid morphology of patients with thyroid-related eye diseases, showing strong consistency between automatic and manual measurements, suggesting the potential application of deep learning in automatic eyelid morphology measurement. In addition, significant progress has been made in the automatic detection and classification of diabetic retinopathy images using deep learning models.

[0004] However, a common shortcoming of these studies is that they pay insufficient attention to the fine-grained features specific to the eyelid margin, and often rely on broader, non-specific morphological features. Therefore, there is an urgent need for a deep learning-based system dedicated to fine-grained eyelid margin image classification to detect subtle features such as eyelid margin redness and meibomian gland occlusion, filling the technical gap in this area. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a feature classification system for the eyelid margin, which solves the technical problem that the prior art regards segmentation and classification as independent tasks, fails to fully utilize segmentation information to assist feature classification, and results in limited accuracy of classification results.

[0007] (II) Technical solution

[0008] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0009] An embodiment of the present invention provides an eyelid margin feature classification system, comprising:

[0010] An eyelid margin segmentation module is used to input the received original feature image into a preset eyelid margin segmentation model to obtain a segmented image corresponding to the original feature image;

[0011] The outputs of the last encoder and all decoders of the eyelid margin segmentation model are processed by a preset 3×3 convolution and Sigmoid function to generate a significant probability map corresponding to each output, and all significant probability maps are upsampled to the same size as the original feature image, and then fused through a cascade operation box and processed by a 1×1 convolution and Sigmoid function to generate a segmentation image corresponding to the original feature map;

[0012] A feature classification module, used for inputting the segmented image into a preset feature classification model to obtain a feature classification result;

[0013] The eyelid margin segmentation model is based on U 2 -Net model, including encoder, decoder and saliency fuser, and the encoder and decoder are connected by skip connection;

[0014] The feature classification result includes eyelid margin opening or eyelid margin congestion, and the corresponding grade of eyelid margin opening or eyelid margin congestion;

[0015] The feature classification model includes: a shallow feature extraction unit for extracting shallow features of an image, a deep feature extraction unit for extracting shallow features into deep features, an output unit for converting deep features into outputs, and a classification head for classifying the outputs.

[0016] Optionally, the feature classification module is used to input the segmented image into a preset feature classification model to obtain a feature classification result, including:

[0017] The shallow feature extraction unit performs image dimension upscaling on the segmented image according to the preset 3×3 convolution layer, performs batch normalization on it, and finally outputs the shallow feature image corresponding to the segmented image according to the preset Swish activation function;

[0018] The deep-level feature extraction unit extracts features from the shallow-level feature image according to a preset moving reverse residual convolution group to obtain a deep-level feature image;

[0019] The output unit is used to reduce the dimension of the deep feature image according to the preset 1×1 convolution layer, and then obtain the feature image corresponding to the segmented image through the preset average pooling layer and full connection layer;

[0020] The classification head is used to input the feature image into a preset fully connected layer and a Softmax layer to obtain a feature classification result.

[0021] Optionally, the moving reverse residual convolution group comprises: MBConv blocks connected in sequence;

[0022] Each MBConv block is used to increase the dimension of the input image through a 1×1 dimensionality increase convolution layer, and then perform batch normalization and Swish activation function activation in sequence to obtain the first process image;

[0023] After the first process image is processed by a k×k depthwise separable convolutional layer, batch normalization and Swish activation function activation are performed again to obtain the second process image, where k is 3 or 5;

[0024] After the second process image is processed by the SE layer, it is sent to the Dropout layer after a 1×1 dimensionality reduction convolution layer and batch normalization;

[0025] The output of the Dropout layer and the input image are added in phase to obtain an output image; the input image includes a shallow feature image or an output image of an MBConv block.

[0026] Optionally, the depthwise separable convolution layer includes depthwise convolution and pointwise convolution;

[0027] Each of the MBConv blocks processes the first process image through a k×k depth-separable convolution layer, and then performs batch normalization and Swish activation function activation again to obtain a second process image, including:

[0028] The depth convolution performs convolution processing on each channel of the first process image according to a preset formula to obtain a third process image;

[0029] Point-by-point convolution performs weighted combination on the third process image in the channel direction according to a preset formula 2 to obtain a second process image;

[0030] The formula 1 is:

[0031]

[0032] in, is the c-th output channel feature map obtained after the deep convolution operation, X c is the feature map of a specific channel of the input, W c is the convolution kernel corresponding to the input channel c, * is the convolution symbol;

[0033] The formula 2 is:

[0034]

[0035] in, is the c-th output channel feature map obtained after the deep convolution operation, W 1×1,c is the 1×1 convolution kernel in the point-by-point convolution, corresponding to the cth channel, Y pointwise It is the final output image after point-by-point convolution.

[0036] Optionally, the MBConv block processes the second process image through the SE layer, and then processes the second process image through a 1×1 dimensionality reduction convolution layer and batch normalization and then sends the second process image to the Dropout layer, including:

[0037] The second process image is subjected to average pooling processing according to a preset global average pooling formula; the average pooling formula is:

[0038]

[0039] Among them, z c is the global average pooling result of the cth channel, H and W are the height and width of the input image, Y c,i,j is the pixel value of the cth channel at the spatial position (i, j) in the input feature map;

[0040] Subsequently, the second process image after the average pooling process is input into two consecutive fully connected layers, and the features of each channel are weighted according to the preset formula three;

[0041] The formula three is:

[0042] s c =σ(W2·ReLU(W1·z c ));

[0043] Among them, s c is the attention weight of the cth channel, W1 is the weight matrix of the first fully connected layer, and W2 is the weight matrix of the second fully connected layer;

[0044] After that, after channel calibration is performed using the preset formula 4, the image is sent to the Dropout layer after a 1×1 dimensionality reduction convolution layer and batch normalization processing; the formula 4 is:

[0045]

[0046] in, is the output image after channel calibration, Y c is the original value of the second process image.

[0047] Optionally, the encoder is used to obtain a third process image according to a preset formula 5 and input data; the formula 5 is:

[0048] y1 = ReLU(BN(W*x1+b));

[0049] Among them, W is the convolution kernel, x1 is the input data, b is the bias term, * is the convolution operation, and BN is batch normalization;

[0050] The third process image is downsampled by a preset maximum pooling formula to obtain a downsampling result; the maximum pooling formula is:

[0051] y2=MaxPool(x2);

[0052] Among them, x2 is the third process image, and y2 is the output result after downsampling;

[0053] The decoder is used to upsample the received data according to a preset bilinear difference formula to obtain an upsampling result; the bilinear difference formula is:

[0054]

[0055] Where (x3, y3) is the position where the difference is performed in the data received by the decoder, I o,p is the pixel value of the four pixels around (x3, y3), W op is the preset difference weight.

[0056] Optionally, the system further comprises: a preprocessing module

[0057] The preprocessing module is used to preprocess all image data in the first data training set and the second data training set, wherein the preprocessing includes normalization, size scaling and noise removal; the first data training set includes at least one eye image data and a mask corresponding to each eye image data; the second data training set includes at least one eyelid margin segmentation image and a classification result corresponding to each eyelid margin segmentation image;

[0058] A model training module, used for training a pre-built eyelid margin segmentation model according to a pre-set first data training set to obtain a trained eyelid margin segmentation model;

[0059] The pre-built feature classification model is trained according to a pre-set second data training set to obtain a trained feature classification model.

[0060] Optionally, the system further comprises a model verification module;

[0061] The model verification module is used to use a preset first data verification set to perform performance evaluation on the trained eyelid margin segmentation model, and adjust the hyperparameters of the eyelid margin segmentation model according to the performance evaluation result;

[0062] A pre-set second data validation set is used to perform performance evaluation on the trained feature classification model, and the hyperparameters of the feature classification model are adjusted according to the performance evaluation results.

[0063] Optionally, the system further comprises a classification model adjustment module;

[0064] The classification model adjustment module is used to simultaneously scale the width, depth and resolution of the feature classification model according to a load factor scaling strategy; the load factor scaling strategy is:

[0065]

[0066] Among them, α, β, γ are scaling coefficients, and φ is the composite coefficient.

[0067] Optionally, the encoder and decoder are both composed of RSU-L or RSU-4F.

[0068] (III) Beneficial effects

[0069] The beneficial effects of the present invention are as follows: an eyelid margin feature classification system of the present invention, by using an eyelid margin segmentation model and a feature classification model, realizes the use of segmentation information to assist feature classification, and optimizes the characteristics of eyelid attrition, thereby improving the effect of processing eyelid margin features. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of the structure of a system for classifying eyelid margin features;

[0071] Figure 2 This is a simplified diagram of the eyelid margin segmentation model structure;

[0072] Figure 3 This is a simplified diagram of the feature classification model structure;

[0073] Figure 4 This is a simplified diagram of the MBConv block structure;

[0074] Figure 5 This is a schematic diagram of the feature classification model structure after quantum optimization. DETAILED DESCRIPTION

[0075] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0076] An eyelid margin feature classification system proposed in an embodiment of the present invention uses an eyelid margin segmentation model and a feature classification model to utilize segmentation information to assist feature classification, and is optimized based on the characteristics of eyelid attrition, thereby improving the effect of processing eyelid margin features.

[0077] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0078] Example 1

[0079] This embodiment provides an eyelid margin feature classification system. Figure 1 As shown, including:

[0080] The preprocessing module collects eye image data including the eyelid margin and constructs a first data training set. The first data training set should include high-resolution eye image data and annotate each image in detail (i.e., the corresponding mask) to ensure the accuracy and completeness of the data.

[0081] The eyelid margin segmentation images are collected to construct a second data training set. The second data training set includes at least one eyelid margin segmentation image and a classification result corresponding to each eyelid margin segmentation image.

[0082] The collected image data is preprocessed, including steps such as image normalization, resizing, and noise removal. Normalization helps eliminate brightness and contrast differences between different images, resizing ensures that the image meets the requirements of the model input, and noise removal improves the quality of the image.

[0083] The model training module is used to train the eyelid margin segmentation model according to the first data training set to obtain a trained eyelid margin segmentation model; during the training process, the cross entropy loss function is used to calculate the difference between the model prediction result and the actual annotation, and the model parameters are continuously optimized through the gradient descent algorithm to improve the segmentation accuracy of the model.

[0084] The feature classification model is trained according to the second data training set to obtain a trained feature classification model. During the training process, a cross entropy loss function is selected to calculate the difference between the model prediction result and the actual label.

[0085] A model validation module, used to use a preset first data validation set to perform performance evaluation on the trained eyelid margin segmentation model, and adjust hyperparameters of the eyelid margin segmentation model according to the performance evaluation results;

[0086] A pre-set second data validation set is used to perform performance evaluation on the trained feature classification model, and the hyperparameters of the feature classification model are adjusted according to the performance evaluation results.

[0087] Classification model adjustment module;

[0088] The classification model adjustment module is used to simultaneously scale the width, depth and resolution of the feature classification model according to a load factor scaling strategy; the load factor scaling strategy is:

[0089]

[0090] Among them, α, β, γ are scaling coefficients, and φ is the composite coefficient.

[0091] An eyelid margin segmentation module is used to input the received original feature map into a preset eyelid margin segmentation model to obtain a segmented image corresponding to the original feature map;

[0092] Eyelid margin segmentation model based on U 2 -Net model is established, including: encoder part, skip connection part, decoder part and saliency fuser part, which is used for feature extraction and information fusion between channels. The encoder part and the decoder part are connected by the skip connection part.

[0093] In general, the eyelid margin segmentation model structure is as follows Figure 2 As shown, it includes 6 levels of first encoders and 5 first decoders; wherein, the first 4 levels of first encoders (i.e., En_1, En_2, En_3 and En_4) are RSU-7, RSU-6, RSU-5 and RSU-4 structures respectively, and the first 4 levels of first decoders (i.e., De_1, De_2, De_3 and De_4) are RSU-7, RSU-6, RSU-5 and RSU-4 structures respectively; the 5th level first encoder, the 6th level first encoder and the 5th level first decoder (i.e., En_5, En_6 and De_5) are all RSU-4F structures; each RSU-L includes L second encoders and L-1 second decoders; RSU-4F is a symmetric encoder structure with a height of 4 and the same as RSU-L; L is a non-zero natural number.

[0094] Each encoder (first encoder or second encoder) or decoder (first decoder or second decoder) is mainly composed of a convolutional layer, a batch normalization layer and a ReLU activation function.

[0095] The encoder is used to obtain a third process image according to a preset formula 5 and input data; the formula 5 is:

[0096] y1 = ReLU(BN(W*x1+b));

[0097] Among them, W is the convolution kernel, x1 is the input data, b is the bias term, * is the convolution operation, and BN is batch normalization;

[0098] The third process image is downsampled by a preset maximum pooling formula to obtain a downsampling result; the maximum pooling formula is:

[0099] y2=MaxPool(x2);

[0100] Among them, x2 is the third process image, and y2 is the output result after downsampling;

[0101] The decoder is used to upsample the received data according to a preset bilinear difference formula to obtain an upsampling result; the bilinear difference formula is:

[0102]

[0103] Where (x3, y3) is the position where the difference is performed in the data received by the decoder, I o,p is the pixel value of the four pixels around (x3, y3), W op is the preset difference weight.

[0104] Among them, the input of the first-level first encoder is the initial input of the eyelid margin segmentation model, and the input of any first encoder except the first-level first encoder is the output of the first encoder of the previous level of the first encoder; in the first four levels of first decoders, the input of any first decoder is the concatenation result of the output of the first encoder of the same level corresponding to the first decoder and the output of the first decoder of the next level of the first decoder; the input of the fifth-level first decoder is the concatenation result of the output of the fifth-level first encoder and the output of the sixth-level first encoder.

[0105] That is, the input of En_1 is the original feature image input to the eyelid margin segmentation model, the input of En_2 is the output of En_1, the input of En_3 is the output of En_2, the input of En_4 is the output of En_3, the input of En_5 is the output of En_4, and the input of En_6 is the output of En_5.

[0106] The input of De_1 is the concatenation of the output of En_1 and the output of De_2; the input of De_2 is the concatenation of the output of En_2 and the output of De_3; the input of De_3 is the concatenation of the output of En_3 and the output of De_4; the input of De_4 is the concatenation of the output of En_4 and the output of De_5; the input of De_5 is the concatenation of the output of En_5 and the output of En_6.

[0107] When the eyelid margin segmentation module receives the original feature image, the saliency fuser part processes the image output by the last level first encoder (En_6) and the images output by all the first decoders (De_1~De_5) through a preset 3×3 convolution and Sigmoid function to generate a saliency probability map corresponding to each output, and upsamples all saliency probability maps to the same size as the original feature image, and then fuses them through a cascade operation box and processes them through a 1×1 convolution and Sigmoid function to generate a segmentation image corresponding to the original feature map.

[0108] A feature classification module, used for inputting the segmented image into a preset feature classification model to obtain a feature classification result;

[0109] The feature classification model is based on EfficientNet, such as Figure 3 As shown in the figure, it includes: stage1 to stage9, 9 stages; each convolution layer is followed by BN and Swish activation function by default. Stage1 (shallow feature extraction unit) is a 3×3 convolution layer, while stage2 to stage8 (deep feature extraction unit) are repeatedly stacked MBConv blocks. Stage9 (output unit) consists of three parts, first a 1×1 convolution, then average pooling, and finally a fully connected layer.

[0110] And each MBConv block, such as Figure 4 As shown in the figure, first there is a 1×1 dimensionality increase convolution (followed by BN and Swish activation function), followed by a k×k depth separable convolution layer (after BN and Swish activation function), k=3 or 5, then a SE layer (using the attention mechanism to adjust the feature matrix), followed by a 1×1 dimensionality reduction convolution (followed by BN), and finally a Droupout layer. The output of Droupout and the input of the MBConv are phase-added as the final output of the MBConv block.

[0111] Depthwise separable convolution is divided into two parts: depthwise convolution and pointwise convolution; depthwise convolution performs convolution on each input channel separately according to formula 1. Pointwise convolution performs 1×1 convolution on the output of depthwise convolution according to formula 2.

[0112] The formula 1 is:

[0113]

[0114] in, is the c-th output channel feature map obtained after the deep convolution operation, X c is the feature map of a specific channel of the input, W c is the convolution kernel corresponding to the input channel c, * is the convolution symbol;

[0115] The formula 2 is:

[0116]

[0117] in, is the c-th output channel feature map obtained after the deep convolution operation, W 1×1,c is the 1×1 convolution kernel in the point-by-point convolution, corresponding to the cth channel, Y pointwise It is the final output image after point-by-point convolution.

[0118] The SE layer is used to recalibrate the channel weights, which includes a global averaging layer and two fully connected layers connected in sequence; the global averaging layer performs average pooling on the input data according to the preset global average pooling formula, and the average pooling results are input into two fully connected layers, and the features of each channel are weighted according to formula three; then, channel calibration is performed using formula four.

[0119] The average pooling formula is:

[0120]

[0121] Among them, z c is the global average pooling result of the cth channel, H and W are the height and width of the input image, Y c,i,j is the pixel value of the cth channel at the spatial position (i, j) in the input feature map; the formula three is:

[0122] s c =σ(W2·ReLU(W1·z c ));

[0123] Among them, s c is the attention weight of the cth channel, W1 is the weight matrix of the first fully connected layer, and W2 is the weight matrix of the second fully connected layer;

[0124] The formula 4 is:

[0125]

[0126] in, is the output image after channel calibration, Y c is the original value of the second process image.

[0127] The classification head, including the fully connected layer and the Softmax layer, converts the input data into feature classification results.

[0128] This embodiment can effectively extract multi-scale features and achieve high-precision eyelid margin segmentation through the double-layer nested U structure and RSU block design, while the RSU block does not significantly increase the computational cost while maintaining high resolution, and is suitable for environments with limited computing resources.

[0129] Moreover, this embodiment does not rely on a pre-trained backbone network, can be trained from scratch, adapt to different working environments, and can perform fine-grained feature classification on the segmented eyelid margin area through a feature classification model, thereby improving classification accuracy and efficiency.

[0130] Example 2

[0131] This embodiment provides an eyelid margin feature classification system, the system is based on embodiment 1, the feature classification model is as follows Figure 5 As shown, it also includes: quantum feature extraction units corresponding to stage2 to stage8 (deep-level feature extraction units), and the quantum feature extraction units corresponding to each stage are located between the MBConv block of that stage and the MBConv block of the next stage. For example, the output of stage2 is used as the input of the quantum feature extraction unit corresponding to stage2, and the output of the quantum feature extraction unit corresponding to stage2 is used as the input of the MBConv block of stage3.

[0132] Each quantum feature extraction unit is used to encode the input data in quantum state according to the classical bits and classical digital information of the received input data (feature map) to obtain the quantum state corresponding to the input data; extract the quantum features in the quantum state according to the pre-set quantum circuit to obtain the quantum feature map; perform quantum measurement on the quantum state according to the quantum feature map, that is, the quantum state corresponding to the input data collapses to the Z basis, and then obtains the quantum measurement result (usually in the form of electrostatic bits), and performs multiple measurements to collect statistical data, extract the process data obtained by quantum calculation, and post-process the process to parse and format the classical data. The post-processing includes statistical analysis, data conversion and feature extraction (usually by converting to numerical values, calculating probability distribution or applying specific decoding algorithms). The classical data is the output image after quantum feature extraction. After the output image after quantum feature extraction is spliced ​​with the input data, it is input into the next-order MBConv block.

[0133] Methods for quantum state encoding include:

[0134] Encoding of classical information into quantum information:

[0135] Qubit mapping: Mapping the state of a classical bit to a qubit. For example, |0> can be used to represent a classical 0, and |1> can be used to represent a classical 1.

[0136] Multi-qubit encoding: For more complex classical information, multiple qubits can be used. For example, the classical binary string "101" can be encoded as a quantum state

[0137] The form of quantum state encoding:

[0138] Superposition state: Classical information can be encoded through superposition state. For example, quantum state Represents the superposition state of the classical bit.

[0139] Entangled state: Quantum entangled state is a special quantum state in which quantum bits have strong correlations. Entangled state such as Bell state Used in quantum communication and quantum computing.

[0140] The encoding schemes include:

[0141] Simple coding:

[0142] Direct mapping of quantum states: For example, a classical bit is directly mapped to a quantum state |0> or |1>.

[0143] Amplitude coding: Encoding classical information into a specific value of the quantum state amplitude. For example, mapping classical digital information α into a quantum state

[0144] Complex encoding:

[0145] Quantum error correction coding: To protect quantum information from noise and errors, quantum error correction codes are used. For example, Shor codes and Steane codes improve the robustness of quantum information through expansion and redundancy techniques.

[0146] Quantum data encoding: In quantum computing, quantum algorithms may need to encode data into specific quantum states to optimize computing performance. For example, quantum Fourier transforms are used to encode quantum states to speed up the computing process.

[0147] Since quantum computing is currently limited by the number of quantum bits and the effects of noise, a hybrid optimization strategy can be adopted, such as combining quantum parameter updates with gradient descent of classical network parameters and using a quantum approximate optimization algorithm (QAOA) to train the quantum circuit part.

[0148] This embodiment adds a quantum feature extraction unit on the basis of the feature classification model, which improves the overall feature extraction capability of the model and is more suitable for tasks with complex features.

[0149] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0150] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0151] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.

[0152] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0153] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An eyelid margin feature classification system, characterized in that: include: An eyelid margin segmentation module is used to input the received original feature image into a preset eyelid margin segmentation model to obtain a segmented image corresponding to the original feature image; The outputs of the last encoder and all decoders of the eyelid margin segmentation model are processed by a preset 3×3 convolution and Sigmoid function to generate a significant probability map corresponding to each output, and all significant probability maps are upsampled to the same size as the original feature image, and then fused through a cascade operation box and processed by a 1×1 convolution and Sigmoid function to generate a segmentation image corresponding to the original feature map; A feature classification module, used for inputting the segmented image into a preset feature classification model to obtain a feature classification result; The eyelid margin segmentation model is based on U 2 -Net model, including encoder, decoder and saliency fuser, and the encoder and decoder are connected by skip connection; The feature classification result includes eyelid margin opening or eyelid margin congestion, and the corresponding grade of eyelid margin opening or eyelid margin congestion; The feature classification model includes: a shallow feature extraction unit for extracting shallow features of an image, a deep feature extraction unit for extracting shallow features into deep features, an output unit for converting deep features into outputs, and a classification head for classifying the outputs.

2. The eyelid margin feature classification system according to claim 1, characterized in that: The feature classification module is used to input the segmented image into a preset feature classification model to obtain a feature classification result, including: The shallow feature extraction unit performs image dimension upscaling on the segmented image according to the preset 3×3 convolution layer, performs batch normalization on it, and finally outputs the shallow feature image corresponding to the segmented image according to the preset Swish activation function; The deep-level feature extraction unit extracts features from the shallow-level feature image according to a preset moving reverse residual convolution group to obtain a deep-level feature image; The output unit is used to reduce the dimension of the deep feature image according to the preset 1×1 convolution layer, and then obtain the feature image corresponding to the segmented image through the preset average pooling layer and full connection layer; The classification head is used to input the feature image into a preset fully connected layer and a Softmax layer to obtain a feature classification result.

3. The eyelid margin feature classification system according to claim 2, characterized in that: The moving reverse residual convolution group includes: MBConv blocks connected in sequence; Each MBConv block is used to increase the dimension of the input image through a 1×1 dimensionality increase convolution layer, and then perform batch normalization and Swish activation function activation in sequence to obtain the first process image; After the first process image is processed by a k×k depthwise separable convolutional layer, batch normalization and Swish activation function activation are performed again to obtain the second process image, where k is 3 or 5; After the second process image is processed by the SE layer, it is sent to the Dropout layer after a 1×1 dimensionality reduction convolution layer and batch normalization; The output of the Dropout layer and the input image are added in phase to obtain an output image; the input image includes a shallow feature image or an output image of an MBConv block.

4. The eyelid margin feature classification system according to claim 3, characterized in that: The depth-wise separable convolution layer includes depth-wise convolution and point-wise convolution; Each of the MBConv blocks processes the first process image through a k×k depth-separable convolution layer, and then performs batch normalization and Swish activation function activation again to obtain a second process image, including: The depth convolution performs convolution processing on each channel of the first process image according to a preset formula to obtain a third process image; Point-by-point convolution performs weighted combination on the third process image in the channel direction according to a preset formula 2 to obtain a second process image; The formula 1 is: in, is the c-th output channel feature map obtained after the deep convolution operation, X c is the feature map of a specific channel of the input, W c is the convolution kernel corresponding to the input channel c, * is the convolution symbol; The formula 2 is: in, is the c-th output channel feature map obtained after the deep convolution operation, W 1×1,c is the 1×1 convolution kernel in the point-by-point convolution, corresponding to the cth channel, Y pointwise It is the final output image after point-by-point convolution.

5. The eyelid margin feature classification system according to claim 3, characterized in that: The MBConv block processes the second process image through the SE layer, and then sends it to the Dropout layer after a 1×1 dimensionality reduction convolution layer and batch normalization processing, including: The second process image is subjected to average pooling processing according to a preset global average pooling formula; the average pooling formula is: Among them, z c is the global average pooling result of the cth channel, H and W are the height and width of the input image, Y c,i,j is the pixel value of the cth channel at the spatial position (i, j) in the input feature map; Subsequently, the second process image after the average pooling process is input into two consecutive fully connected layers, and the features of each channel are weighted according to the preset formula three; The formula three is: s c =σ(W2 ReLU(W1 z c )); Among them, s c is the attention weight of the cth channel, W1 is the weight matrix of the first fully connected layer, and W2 is the weight matrix of the second fully connected layer; After that, after channel calibration is performed using the preset formula 4, the image is sent to the Dropout layer after a 1×1 dimensionality reduction convolution layer and batch normalization processing; the formula 4 is: in, is the output image after channel calibration, Y c is the original value of the second process image.

6. The eyelid margin feature classification system according to claim 1, characterized in that: The encoder is used to obtain a third process image according to a preset formula 5 and input data; the formula 5 is: y1 = ReLU(BN(W*x1+b)); Among them, W is the convolution kernel, x1 is the input data, b is the bias term, * is the convolution operation, and BN is batch normalization; The third process image is downsampled by a preset maximum pooling formula to obtain a downsampling result; the maximum pooling formula is: y2=MaxPool(x2); Among them, x2 is the third process image, and y2 is the output result after downsampling; The decoder is used to upsample the received data according to a preset bilinear difference formula to obtain an upsampling result; the bilinear difference formula is: Where (x3, y3) is the position where the difference is performed in the data received by the decoder, I o,p is the pixel value of the four pixels around (x3, y3), W op is the preset difference weight.

7. The eyelid margin feature classification system according to claim 1, characterized in that: The system also includes: a preprocessing module The preprocessing module is used to preprocess all image data in the first data training set and the second data training set, wherein the preprocessing includes normalization, size scaling and noise removal; the first data training set includes at least one eye image data and a mask corresponding to each eye image data; the second data training set includes at least one eyelid margin segmentation image and a classification result corresponding to each eyelid margin segmentation image; A model training module, used for training a pre-built eyelid margin segmentation model according to a pre-set first data training set to obtain a trained eyelid margin segmentation model; The pre-built feature classification model is trained according to a pre-set second data training set to obtain a trained feature classification model.

8. The eyelid margin feature classification system according to claim 1, characterized in that: The system also includes a model validation module; The model verification module is used to use a preset first data verification set to perform performance evaluation on the trained eyelid margin segmentation model, and adjust the hyperparameters of the eyelid margin segmentation model according to the performance evaluation result; A pre-set second data validation set is used to perform performance evaluation on the trained feature classification model, and the hyperparameters of the feature classification model are adjusted according to the performance evaluation results.

9. The eyelid margin feature classification system according to claim 1, characterized in that: The system also includes a classification model adjustment module; The classification model adjustment module is used to simultaneously scale the width, depth and resolution of the feature classification model according to a load factor scaling strategy; the load factor scaling strategy is: Among them, α, β, γ are scaling coefficients, and φ is the composite coefficient.

10. The eyelid margin feature classification system according to claim 1, characterized in that: The encoder and decoder are both composed of RSU-L or RSU-4F.

Images