Multi-task recognition model training, eye appearance photograph recognition method and device

Abstract

The application discloses a multi-task recognition model training method and an eye appearance photograph recognition method and device. The model training method comprises the following steps: based on an eye appearance photograph sample, a second extraction result is obtained through a feature sharing convolution layer and a sub-task first independent convolution layer; the second extraction result of each sub-task is combined with other second extraction results, and dimension reduction is performed to obtain a new second extraction result; each new second extraction result is input into a sub-task second independent convolution layer to obtain a third image feature; the third image feature of each sub-task is input into a sub-task independent classification layer to obtain a fourth image feature; a model category preset probability is calculated according to the fourth image feature; a cross-entropy loss value of the model category preset probability and an actual category label is calculated; and the cross-entropy loss value is optimized to obtain a multi-task recognition model. The application solves the problem of low efficiency of the existing deep learning technology as a preliminary auxiliary method for judging TAO.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method and apparatus for training a multi-task recognition model and recognizing eye appearance from photographs. Background Technology

[0002] Because the clinical manifestations of thyroid-associated ophthalmopathy (TAO) are complex and varied, and early symptoms are not obvious, it is easy to miss, misdiagnose, or mistreat if the initial consultation is with a physician who is not an orbital specialist. Reports of misdiagnosis and mistreatment of TAO have been frequent both domestically and internationally for many years. With the development of artificial intelligence technology, deep learning technology has begun to be used as an early intervention tool. This primarily involves using deep learning to perform preliminary identification of relevant medical images (such as CT images) to assist doctors in their judgment and effectively reduce the probability of misdiagnosis of TAO. However, the inventors found that existing deep learning technologies used as an early intervention tool still have significant limitations and low efficiency. Summary of the Invention

[0003] The main purpose of this application is to address the problem of low efficiency in existing deep learning techniques as an early aid for determining TAO.

[0004] To achieve the above objectives, according to a first aspect of this application, a method for training a multi-task recognition model is provided. This method utilizes a deep learning network model for training. The method for training the multi-task recognition model according to this application includes: based on eye appearance photograph samples, performing feature extraction sequentially through a feature-sharing convolutional layer and a sub-task first independent convolutional layer to obtain second extraction results corresponding to different sub-tasks. Different sub-task first independent convolutional layers correspond to different sub-tasks, and each sub-task corresponds to a disease sign. Both the feature-sharing convolutional layer and the sub-task first independent convolutional layer are convolutional layers of the deep learning network model; merging the second extraction result corresponding to each sub-task with the second extraction results of all other sub-tasks through dimensionality increase, and then performing dimensionality reduction through convolution to obtain a new second extraction result with the same dimension but different data as the original second extraction result; inputting the new second extraction result corresponding to each sub-task into the sub-task's first independent convolutional layer. Two independent convolutional layers extract features to obtain a third image feature. Different sub-tasks correspond to different second independent convolutional layers, and each sub-task corresponds to a third image feature. The second independent convolutional layers for each sub-task are convolutional layers of a deep learning network model. The third image feature corresponding to each sub-task is input into an independent classification layer for each sub-task, which consists of fully connected layers in the deep learning network model, to obtain a fourth image feature containing disease signs categories. Each sub-task corresponds to a fourth image feature. The preset probability of the multi-task recognition model category is calculated based on the fourth image feature. The second cross-entropy loss value between the preset probability of the multi-task recognition model category and the actual category label is calculated based on the cross-entropy loss function. The model is iteratively trained by optimizing the second cross-entropy loss value using an optimizer to obtain the multi-task recognition model, which is a single model for recognizing multiple disease signs.

[0005] Optionally, the method further includes: training multiple teacher models using multiple deep learning network models based on the eye appearance photo samples, each teacher model corresponding to the recognition of an eye appearance photo of a disease sign; iteratively training the model based on the distance between the output results of the network layers in the teacher models and the output results of the corresponding network layers in the training of the multi-task recognition model, wherein the network layers include convolutional layers and fully connected layers.

[0006] Optionally, the step of training multiple teacher models based on eye appearance photos using multiple deep learning network models includes: extracting first image features from the eye appearance photos using a feature extractor composed of multiple convolutional layers, activation functions, and batch normalization layers in the deep learning network models; inputting the first image features into a classifier composed of fully connected layers in the deep learning network models to obtain second image features, wherein the second image features are features containing disease sign category information; calculating the teacher model category prediction probability based on the softmax activation function and the second image features; calculating a first cross-entropy loss value between the teacher model category prediction probability and the actual category label based on the cross-entropy loss function; optimizing the first cross-entropy loss value based on an optimizer, and determining the model corresponding to the minimum first cross-entropy loss value as the teacher model.

[0007] Optionally, the iterative training of the model based on the distance between the output of the network layer in the teacher model and the output of the corresponding network layer in the training of the multi-task recognition model includes: calculating the first maximum mean difference loss value between the first image feature and the third image feature corresponding to each disease sign; calculating the second maximum mean difference loss value between the output of other convolutional layers before obtaining each third image feature and the output of other convolutional layers before obtaining each corresponding first image feature; calculating the KL divergence loss value between the fourth image feature and the second image feature corresponding to each disease sign; and iteratively training the model by optimizing the first maximum mean difference loss value, the second maximum mean difference loss value, and the KL divergence loss value based on the optimizer.

[0008] Optionally, the step of extracting features from the eye appearance photo sample by sequentially passing through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer to obtain second extraction results corresponding to different subtasks includes: sequentially inputting the eye appearance photo sample into multiple feature-sharing convolutional layers and obtaining a first extraction result after passing through an activation function and a batch normalization layer; and inputting the first extraction result into the subtask-specific first independent convolutional layer corresponding to each subtask to extract image features and obtain the corresponding second extraction result.

[0009] To achieve the above objectives, according to a second aspect of this application, a method for eye appearance photo recognition is provided. The method for eye appearance photo recognition according to this application includes: acquiring an eye appearance photo to be recognized; inputting the eye appearance photo to be recognized into a multi-task recognition model to obtain an eye feature recognition result, wherein the multi-task recognition model is acquired by the multi-task recognition model training method described in any one of the first aspects.

[0010] Optionally, the method further includes: visualizing the eye sign recognition results to determine the image region that contributes the most to the eye sign recognition results.

[0011] To achieve the above objectives, according to a third aspect of this application, a multi-task recognition model training apparatus is provided. The apparatus utilizes a deep learning network model for training. The multi-task recognition model training apparatus according to this application includes: a first extraction unit, used to extract features from eye appearance photos by sequentially passing a feature-sharing convolutional layer and a sub-task first independent convolutional layer to obtain second extraction results corresponding to different sub-tasks. Different sub-task first independent convolutional layers correspond to different sub-tasks, and each sub-task corresponds to a disease sign. The feature-sharing convolutional layer and the sub-task first independent convolutional layer are both convolutional layers of the deep learning network model; a second extraction unit, used to perform dimensionality-upgrading and merging of the second extraction results corresponding to each sub-task with the second extraction results of all other sub-tasks, and then perform dimensionality-downgrading through convolution to obtain a new second extraction result with the same dimension but different data as the original second extraction result; and a third extraction unit, used to input the new second extraction result corresponding to each sub-task into the sub-task second independent convolutional layer. A convolutional layer extracts features to obtain a third image feature. Different sub-tasks have second independent convolutional layers corresponding to different sub-tasks, and each sub-task corresponds to a third image feature. The second independent convolutional layer for each sub-task is a convolutional layer of the deep learning network model. A classification unit is used to input the third image feature corresponding to each sub-task into an independent classification layer for each sub-task, which is composed of fully connected layers in the deep learning network model, to obtain a fourth image feature containing disease signs categories. Each sub-task corresponds to a fourth image feature. A probability calculation unit is used to calculate the preset probability of the multi-task recognition model category based on the fourth image feature. A loss value calculation unit is used to calculate the second cross-entropy loss value between the preset probability of the multi-task recognition model category and the actual category label based on the cross-entropy loss function. An optimization unit is used to iteratively train the model to obtain the multi-task recognition model by optimizing the second cross-entropy loss value based on an optimizer. The multi-task recognition model is a single model for recognizing multiple disease signs.

[0012] Optionally, the device further includes: a training unit, used to train multiple teacher models using multiple deep learning network models based on the eye appearance photo samples, each teacher model corresponding to the recognition of an eye appearance photo of a certain disease sign; the optimization unit is further used to perform iterative training of the model based on the distance between the output results of the network layers in the teacher models and the output results of the corresponding network layers in the training of the multi-task recognition model, wherein the network layers include convolutional layers and fully connected layers.

[0013] Optionally, the training unit includes: a first extraction module, used to extract first image features from the eye appearance photo sample by a feature extractor composed of multiple convolutional layers, activation functions, and batch normalization layers in a deep learning network model; a classification module, used to input the first image features into a classifier composed of fully connected layers in a deep learning network model to obtain second image features, wherein the second image features are features containing disease sign category information; a probability calculation module, used to calculate the teacher model category prediction probability based on the softmax activation function and the second image features; a first loss calculation module, used to calculate the first cross-entropy loss value between the teacher model category prediction probability and the actual category label based on the cross-entropy loss function; and a first optimization module, used to optimize the first cross-entropy loss value based on an optimizer, and determine the model corresponding to the minimum first cross-entropy loss value as the teacher model.

[0014] Optionally, the optimization unit includes: a second loss calculation module, used to calculate the first maximum mean difference loss value between the first image feature and the third image feature corresponding to each disease sign; a third loss calculation module, used to calculate the second maximum mean difference loss value between the output results of other convolutional layers before obtaining each third image feature and the output results of other convolutional layers before obtaining each corresponding first image feature; a fourth loss calculation module, used to calculate the KL divergence loss value between the fourth image feature and the second image feature corresponding to each disease sign; and a second optimization module, used to perform iterative training of the model by optimizing the first maximum mean difference loss value, the second maximum mean difference loss value, and the KL divergence loss value based on the optimizer.

[0015] Optionally, the first extraction unit includes: a second extraction module, used to sequentially input the eye appearance photo sample into multiple feature-shared convolutional layers and obtain a first extraction result after passing through an activation function and a batch normalization layer; and a third extraction module, used to input the first extraction result into the first independent convolutional layer of each subtask to extract image features and obtain the corresponding second extraction result.

[0016] To achieve the above objectives, according to a fourth aspect of this application, an apparatus for recognizing an eye appearance photograph is provided. The apparatus for recognizing an eye appearance photograph according to this application includes: an acquisition unit for acquiring an eye appearance photograph to be recognized; and a recognition unit for inputting the eye appearance photograph to be recognized into a multi-task recognition model to obtain an eye feature recognition result, wherein the multi-task recognition model is acquired by a multi-task recognition model training method as described in any one of the first aspects.

[0017] Optionally, the device further includes a visualization unit for visualizing the eye sign recognition results and determining the image region that contributes the most to the eye sign recognition results.

[0018] To achieve the above objectives, according to a fifth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing the computer to perform the multi-task recognition model training method according to any one of the first aspects, or to perform the eye appearance photo recognition method according to any one of the second aspects.

[0019] To achieve the above objectives, according to a sixth aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the multi-task recognition model training method according to any one of the first aspects, or to perform the eye appearance photo recognition method according to any one of the second aspects.

[0020] In the multi-task recognition model training method and apparatus of this application embodiment, a multi-task recognition model is constructed. During the model training process, in this embodiment, after the feature-sharing convolutional layer, features are extracted by corresponding independent convolutional layers according to different sub-tasks. This allows for the learning of more refined features for different sub-tasks, thereby improving the accuracy of the multi-task recognition model. The subsequent classification layer also performs classification by corresponding independent classification layers according to different sub-tasks, again to learn more refined features for different tasks, thus improving the accuracy of the multi-task recognition model. In addition, before classification, the second extraction results corresponding to each sub-task are subjected to dimensionality merging and dimensionality reduction processing. This ensures that while the independent convolutional layers of each sub-task learn specific refined features, certain effective information can still be shared between them (between different sub-tasks). This allows for better sharing of common semantic information and features among different categories of disease signs across different dimensions in the deep learning network, enabling the model to achieve fast and accurate recognition of multiple categories in a single model. The multi-task recognition model in this embodiment is a single model capable of recognizing multiple disease signs. Compared to existing single recognition models designed for different categories, it has fewer parameters and faster inference speed. Applying the multi-task recognition model to the method and device for eye appearance photo recognition can greatly improve the efficiency of eye image recognition, thereby better assisting in the judgment of TAO (Typical Occult Disease). Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings:

[0022] Figure 1 This is a flowchart of a method for training a multi-task recognition model according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of the structure for training a multi-task recognition model according to an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the network structure of a multi-task recognition model and a teacher model provided according to an embodiment of this application;

[0025] Figure 4 A flowchart of a method for eye appearance recognition according to an embodiment of this application;

[0026] Figure 5 This is a block diagram of a multi-task recognition model training apparatus provided according to an embodiment of this application;

[0027] Figure 6 This is a block diagram of an apparatus for training another multi-task recognition model according to an embodiment of this application;

[0028] Figure 7 This is a block diagram of an eye appearance photo recognition device provided according to an embodiment of this application;

[0029] Figure 8 This is a block diagram of another eye appearance photo recognition device provided according to an embodiment of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] According to an embodiment of this application, a method for training a multi-task recognition model is provided. This embodiment is based on training a deep learning network model. Figure 1 As shown, the method includes the following steps S101-S107: S101. Based on eye appearance photos, feature extraction is performed sequentially through a feature-sharing convolutional layer and a subtask-first independent convolutional layer to obtain second extraction results corresponding to different subtasks. Different subtask-first independent convolutional layers correspond to different subtasks, and each subtask corresponds to a disease sign. Both the feature-sharing convolutional layer and the subtask-first independent convolutional layer belong to the convolutional layers of the deep learning network model; S102. The second extraction result corresponding to each subtask is dimensionality-upgraded and merged with the second extraction results of all other subtasks, and then dimensionality-reduced through convolution to obtain a new second extraction result with the same dimension as the second extraction result but different data; S103. The new second extraction result corresponding to each subtask is input into the subtask-second independent convolutional layer for feature extraction to obtain third image features. Different subtask-first independent convolutional layers correspond to different subtasks. The second independent convolutional layer of each task corresponds to a different subtask, and each subtask corresponds to a third image feature; the second independent convolutional layer of each subtask belongs to the convolutional layer of the deep learning network model; S104. The third image feature corresponding to each subtask is input into the independent classification layer of each subtask, which is composed of fully connected layers in the deep learning network model, to obtain a fourth image feature containing disease signs categories, and each subtask corresponds to a fourth image feature; S105. The preset probability of the multi-task recognition model category is calculated based on the fourth image feature; S106. The second cross-entropy loss value of the preset probability of the multi-task recognition model category and the actual category label is calculated based on the cross-entropy loss function; S107. The model is iteratively trained by optimizing the second cross-entropy loss value based on the optimizer to obtain the multi-task recognition model, which is a single model for recognizing multiple disease signs.

[0034] Before starting model training, sample data needs to be acquired. The eye appearance photograph samples in this application are obtained from eye appearance photographs taken from different angles, after image preprocessing including category labeling (one appearance photograph will be labeled with one or more disease signs), data cleaning, scaling to the same standard, random cropping, flipping, and color jittering. Data cleaning is to remove low-quality images. Random cropping, flipping, and color jittering are used to expand the sample dataset. In this embodiment, eye appearance photographs refer to eye appearance photographs taken from different angles, including frontal and lateral positions, and frontal positions. Frontal and lateral positions include left and right side views, and frontal positions include nine eye positions: frontal, left-turned, upward-turned, downward-turned, right-turned, left-upward-turned, right-upward-turned, left-downward-turned, and right-downward-turned. The eye appearance photograph samples include eye appearance photographs with all disease signs to be identified. Different categories of disease signs in the eye appearance photographs in this embodiment include: eyelid edema, eyelid congestion, conjunctival edema, conjunctival congestion, corneal ulcer, etc.

[0035] In addition, a suitable deep learning network model needs to be selected before training. This embodiment selects a lightweight network model; specifically, MobileNetV3 can be used as the backbone network. In practical applications, other lightweight network structures can also be chosen, such as SqueezeNet and ShuffleNet. This embodiment uses MobileNetV3 as an example to illustrate the subsequent steps. Figure 2 The diagram shown illustrates a schematic representation of the MobileNetV3 network architecture. Figure 2 The deep learning network model structure of the multi-task recognition model trained in the training includes four convolutional layers (feature-sharing convolutional layer 1, feature-sharing convolutional layer 2, subtask first independent convolutional layer, subtask second independent convolutional layer) and one classification layer (subtask independent classification layer).

[0036] In step S101, based on the eye appearance photo sample, feature extraction is performed sequentially through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer to obtain the second extraction result corresponding to different subtasks. Specifically, this includes: inputting the eye appearance photo sample into multiple feature-sharing convolutional layers in sequence and obtaining the first extraction result after passing through an activation function and a batch normalization layer; and inputting the first extraction result into the subtask-specific first independent convolutional layer corresponding to each subtask to extract image features and obtain the corresponding second extraction result.

[0037] Feature-sharing convolutional layers, subtask-specific first independent convolutional layers, and subsequent subtask-specific second independent convolutional layers are all convolutional layers in deep learning network models. The difference between feature-sharing and subtask-specific convolutional layers lies in their approach: subtask-specific convolutional layers extract features independently for each category based on different subtasks, while feature-sharing convolutional layers do not. Typically, existing multi-class recognition models extract features using feature-sharing convolutional layers, without subtask-specific convolutional layers. In this embodiment, after the feature-sharing convolutional layer, feature extraction is performed by subtask-specific convolutional layers for different subtasks. This allows for the learning of more refined features for different subtasks, improving the accuracy of the multi-task recognition model. Subsequent classification is also performed by corresponding subtask-specific independent classification layers, again to learn more refined features for different tasks, further enhancing the accuracy of the multi-task recognition model.

[0038] like Figure 2 As shown, the sample photos of the eye appearance are S={s1,s2,...,s n The input is fed into a convolutional layer that shares multiple features. Figure 2 Specifically, there are two feature-sharing convolutional layers (the number of feature-sharing convolutional layers can be adjusted adaptively depending on the network structure). After passing through an activation function (ReLU function) and a batch normalization layer (BN layer), the first extraction result is obtained. The first extraction result is then input into multiple subtask first independent convolutional layers (the first independent convolutional layer of each subtask is the first independent convolutional layer corresponding to each subtask, and the parameters corresponding to the first independent convolutional layer of different subtasks are different, resulting in multiple second extraction results) to extract image features and obtain multiple second extraction results. Figure 2 The output of each convolutional layer and classification layer is an image feature. The image features output by different layers are different. Therefore, the first extraction result and the second extraction result are also image features.

[0039] In step S102, the second extraction result corresponding to each subtask is dimensionality-upgraded and merged with the second extraction results of all other subtasks, and then dimensionality-reduced using convolution to obtain a new second extraction result with the same dimension but different data as the original second extraction result. Figure 2 Specifically, the data corresponding to each second extraction result is combined with the data corresponding to all other second extraction results by dimension to obtain new feature data. This new feature data is then subjected to dimensionality reduction through a convolutional layer to obtain a new second extraction result with the same dimension as the second extraction result but different data.

[0040] The purpose of dimensionality-increasing, merging, and then reducing the dimensionality of the second extraction results is to ensure that independent subtasks learn specific refined features through their independent convolutional layers, while also guaranteeing that some effective information can still be shared between different subtasks. This makes the multi-task recognition model more accurate and faster. The implementation of "merging and reducing the dimensionality of each second extraction result with all other second extraction results" is illustrated with an example: Assume there are two subtasks, each corresponding to one second extraction result, resulting in a total of two second extraction results, each with a dimension of D. The first and second second extraction results are merged to obtain new feature data with a dimension of 2D. Then, this new 2D feature data is passed through a 1×1 convolutional layer to reduce the dimensionality to a new second extraction result with a dimension of D. Each subtask can obtain a new second extraction result.

[0041] In step S103, the new second extraction result corresponding to each subtask is input into the second independent convolutional layer of the subtask for feature extraction to obtain the third image feature. Different subtasks have different second independent convolutional layers, and each subtask corresponds to a third image feature. Figure 2 Specifically, the new second extraction result corresponding to each subtask is input into the corresponding subtask's second independent convolutional layer to extract image features and obtain multiple third image features F. s ={f s1 ,f s2 ,...,f sn}, where f sn F represents the third image feature corresponding to the nth eye appearance photo. s This represents a set of third image features corresponding to a set of input eye appearance images S. Similarly, the second independent convolutional layer for each subtask is the second independent convolutional layer corresponding to each subtask. The parameters of the second independent convolutional layer for different subtasks are different, resulting in various third image features.

[0042] In step S104, the third image features corresponding to each subtask are input into the independent classification layer for each subtask, which consists of fully connected layers in the deep learning network model, to obtain the fourth image features containing disease sign categories. Figure 2 Specifically, for each third image feature F s The input is fed into a subtask-independent classifier consisting of fully connected layers to obtain a fourth image feature L. s (Logits) = {l s1 ,l s2 ,...,l sn The fourth image feature is a feature that contains category information; where l snL represents the fourth image feature corresponding to the nth eye appearance photo. s This represents a set of fourth image features corresponding to a set of input eye appearance photos S. It should be noted that in the subtask independent classifiers corresponding to each third image feature input, different subtasks correspond to subtask independent classifiers with different parameters, and different subtask independent classifiers yield different fourth image features.

[0043] In step S105, the preset probability of the multi-task recognition model category is calculated based on the fourth image features. Specifically, each fourth image feature is input into the softmax activation function to calculate the category prediction probability. Each eye appearance pair corresponding to each fourth image feature will obtain a category prediction probability. Each category prediction probability belongs to the preset probability of the multi-task recognition model category.

[0044] In step S106, the second cross-entropy loss value (BCELoss) between the preset probability of the multi-task recognition model category and the actual category label is calculated based on the cross-entropy loss function. Specifically, the second cross-entropy loss value between the preset probability of each multi-task recognition model category and the corresponding actual category label is calculated. The formula for calculating the second cross-entropy loss value is as follows:

[0045]

[0046] Where, p i This indicates the actual category label for the i-th eye appearance photograph. Loss1 represents the predicted category probability of the i-th eye appearance photo. i This represents the second cross-entropy loss value corresponding to the i-th eye appearance photo.

[0047] In step S107, the optimizer is an algorithm that optimizes the loss function to obtain the machine learning model that minimizes the loss function value. Commonly used optimizers include stochastic gradient descent (SGD). Specifically, the multi-task recognition model is obtained through iterative training by optimizing the second cross-entropy loss value using the optimizer: the cross-entropy loss function is optimized using the optimizer to minimize the second cross-entropy loss value, and the model corresponding to the minimum second cross-entropy loss value is determined as the final multi-task recognition model.

[0048] As can be seen from the above description, the multi-task recognition model training method in this application embodiment constructs a multi-task recognition model. During the model training process, in this embodiment, after the feature-sharing convolutional layer, features are extracted by corresponding independent convolutional layers for different sub-tasks. This allows for the learning of more refined features for different sub-tasks, thereby improving the accuracy of the multi-task recognition model. The subsequent classification layer also performs classification by corresponding independent classification layers for different sub-tasks, again to learn more refined features for different tasks, thus improving the accuracy of the multi-task recognition model. Furthermore, before classification, the second extraction results for each sub-task are subjected to dimensionality merging and dimensionality reduction processing. This ensures that while independent convolutional layers for each sub-task learn specific refined features, certain effective information can still be shared between them (between different sub-tasks). This allows for better sharing of common semantic information and features among different categories of disease signs across different dimensions in the deep learning network, enabling the model to achieve fast and accurate recognition of multiple categories within a single model. The multi-task recognition model in this embodiment is a single model capable of recognizing multiple disease signs. Compared with existing single recognition models designed for different categories, it has fewer parameters and faster inference speed.

[0049] Furthermore, to improve the model's accuracy, knowledge distillation is performed during the training of the multi-task recognition model, using a teacher model for recognizing eye appearance photos of a single disease sign category as a reference. Knowledge distillation is a training technique used in "teacher-student models," which primarily involves constructing a lightweight smaller model and using the supervision information from a larger, more powerful model to train this smaller model, aiming to achieve better performance and accuracy. It was first proposed and applied to classification tasks by Hinton in 2015. The larger model is called the teacher model, and the smaller model is called the student model. The supervision information output from the teacher model is called knowledge, and the process by which the student learns and transfers the supervision information from the teacher is called distillation. The training method incorporating knowledge distillation is based on the training method in the aforementioned embodiments, and specifically includes the following steps:

[0050] S201. Based on eye appearance photos, multiple teacher models are trained using multiple deep learning network models, and each teacher model corresponds to the recognition of an eye appearance photo of a certain disease sign.

[0051] First, it should be noted that, for better knowledge distillation, the deep learning network model used in training the multi-task recognition model is the same network model used in the teacher model. For example...Figure 3 The teacher model and multi-task recognition model shown both use a MobileNetV3 architecture with four convolutional layers and one classification layer. It should also be noted that the multi-task recognition model is essentially the student model in the teacher-student model. The following section will combine... Figure 3 The network structure diagram illustrates the training process of the teacher model.

[0052] a1. A feature extractor composed of multiple convolutional layers, activation functions, and batch normalization layers in a deep learning network model extracts the first image features of eye appearance photo samples;

[0053] This step involves taking a sample of eye appearance photos S = {s1, s2, ..., s...} n The input is fed into a feature extractor, which extracts the first image feature F through four convolutional layers (convolutional layer 1-convolutional layer 4), an activation function (Rectified Linear Unit, ReLU function), and a batch normalization (BN) layer. t ={f t1 ,f t2 ,...,f tn}. Where s n Let S represent the nth eye appearance photo, and let f represent a set of input eye appearance photos. tn F represents the first image feature corresponding to the nth eye appearance photo. t This represents a set of first image features corresponding to a set of input eye appearance photos S. The activation function is used to increase the non-linear expressive power of the model, while the batch normalization layer is used to pull the irregular distribution to a normal distribution, ensuring that the data is distributed in the region with a large gradient of the activation function, thus improving generalization ability and solving the gradient vanishing problem to some extent.

[0054] a2. Transfer the first image feature F t ={f1,f2,...,f n The input is fed into a classifier consisting of fully connected layers in a deep learning network model (corresponding to...). Figure 3 The classification layer in the middle obtains the second image feature L. t (Logits) = {l t1 ,l t2 ,...,l tn The second image feature is a feature that contains information about the category of disease signs; where l tn L represents the second image feature corresponding to the nth eye appearance photo. t This represents a set of second image features corresponding to a set of input eye appearance photos S.

[0055] a3. Calculate the teacher model category prediction probability based on the softmax activation function and the second image features;

[0056] The second image features are input into the softmax activation function to calculate the class prediction probability. Each eye appearance pair will yield a class prediction probability, and each class prediction probability belongs to the class prediction probability of the teacher model.

[0057] a4. Calculate the first cross-entropy loss value between the teacher model's predicted category probability and the actual category labeling based on the cross-entropy loss function;

[0058] The formula for calculating the first cross-entropy loss value is the same as the formula for calculating the second cross-entropy loss value in step S106 above, and will not be repeated here. You can refer to the above calculation formula for calculation.

[0059] a5. Optimize the first cross-entropy loss value based on the optimizer, and determine the model corresponding to the minimum first cross-entropy loss value as the teacher model.

[0060] An optimizer is an algorithm that optimizes the loss function to obtain a machine learning model that minimizes the loss function value. Common optimizers include stochastic gradient descent (SGD). In this step, the optimizer is used to optimize the cross-entropy loss function to minimize the first cross-entropy loss value, and the model corresponding to the minimum first cross-entropy loss value is determined as the final teacher model.

[0061] Multiple teacher models can be obtained by training the eye appearance photos samples using multiple deep learning network models (the structures of the multiple deep network models are the same, but they are trained for different types of disease signs, so the parameters of each deep network model are different) according to steps a1-a5 above. Figure 3 The diagram shows the network structure of two teacher models, which correspond to the recognition of two categories of disease signs. In practical applications, a teacher model with the same number of categories is trained. Figure 3 The number of teacher models given is illustrative and is not limited by the diagram in actual applications.

[0062] S202. Iterative training of the model is performed based on the distance between the output of the network layer in the teacher model and the output of the corresponding network layer in the multi-task recognition model training. The network layers include convolutional layers and fully connected layers (classification layers).

[0063] First, calculate the third image feature F corresponding to each disease sign. s With the corresponding first image feature F tThe first maximum mean difference loss value (the maximum mean difference loss value is the MMD distance); calculate the other convolutional layers before each third image feature ( Figure 3 The second maximum mean difference loss value is the output of the first independent convolutional layer (i.e., the second extraction result) of the subtask (i.e., the first independent convolutional layer) and the output of the other convolutional layers (i.e., convolutional layer 3) before obtaining the first image features. The calculation of the first maximum mean difference loss value and the second maximum mean difference loss value reflects the distance between the convolutional layers corresponding to each teacher model and the multi-task recognition model. Figure 3 The output of each convolutional layer and classification layer is an image feature. The image features output by different layers are different. The output of convolutional layer 3 is also an image feature. In order to distinguish it from the output of other layers, the output of convolutional layer 3 is denoted as the fifth image feature. In addition, in order to reflect the consistency of the type of the object calculated by the second maximum mean difference loss, the second extraction result is denoted as the sixth image feature.

[0064] Figure 3 The diagram illustrates two teacher models. The distance between the convolutional layers corresponding to each teacher model and the multi-task recognition model needs to be calculated, i.e., the maximum mean difference loss (MMD) value. The selection of the convolutional layers for each teacher model and the multi-task recognition model is based on the same category and the same layer. A concrete example illustrates this: Consider two teacher models, A and B. Model A corresponds to the recognition of disease symptom A, and model B corresponds to the recognition of disease symptom B. The multi-task recognition model, in its independent convolutional layer, is divided into two sub-tasks: sub-task 1 for disease symptom A recognition and sub-task 2 for disease symptom B recognition. When calculating the MMD distance, teacher model A corresponds to sub-task 1, and teacher model B corresponds to sub-task 2. This selection is based on the same category. The same layer selection refers to being located in the same convolutional layer. For example, assuming both the teacher model and the multi-task recognition model use a 4-layer convolutional layer structure, the same convolutional layer means being located in the same layer. Figure 3 The diagram shows the calculation of MMD distance between the third and fourth layers of the teacher model and the third and fourth layers of the multi-task recognition model.

[0065] The calculation principle for the first maximum mean difference loss value and the second maximum mean difference loss value is the same, to calculate F s and F t Taking the first maximum mean difference loss value as an example, the calculation principle of the maximum mean difference loss value is explained. The specific calculation formula is as follows:

[0066] First, a Gaussian kernel function G is needed to transform F... s and F t The two features are mapped to Hilbert space to obtain F's and F' t :

[0067] F' s =G(F s ), F' t =G(F t )

[0068] Then calculate F' s and F' t The Euclidean distance, i.e., F, is obtained. s and F t MMD distance Loss2

[0069]

[0070] Secondly, calculate the fourth image feature L corresponding to each disease sign. s With the corresponding second image feature L t The KL divergence loss value Loss3;

[0071] This step is to calculate the distance between the classification layers corresponding to each teacher model and the multi-task recognition model. Figure 3 The diagram shows two teacher models. We need to calculate the distance between each teacher model and the classification layer corresponding to the multi-task recognition model, i.e., the KL divergence loss value. The selection of the classification layer corresponding to each teacher model and the multi-task recognition model is also based on the same category and the same layer. This will not be elaborated further here.

[0072] Specifically, calculate the KL divergence loss value Loss3 according to the following formula:

[0073] Loss3 = KL(Softmax(L s / T),Softmax(L t / T))

[0074] Where T is a hyperparameter.

[0075] Finally, the model is iteratively trained by optimizing the first maximum mean difference loss value, the second maximum mean difference loss value, and the KL divergence loss value based on the optimizer.

[0076] In practical applications, this step is performed simultaneously with the optimization of the second cross-entropy loss value in the aforementioned step S107. In practical applications, this embodiment is equivalent to optimizing multiple loss functions based on the optimizer to obtain the minimum values ​​of multiple loss functions (second cross-entropy loss value, first maximum mean difference loss value, second maximum mean difference loss value, KL divergence loss value), and finally obtaining a multi-task recognition model.

[0077] From the above description, it can be seen that Figure 4 A corresponding implementation uses knowledge distillation to guide the multi-task recognition model's learning in a more favorable direction early in its training, thereby improving the model's accuracy. The advantage of knowledge distillation is that it utilizes pre-trained teacher model parameters to constrain the newly trained model, better assisting the training and learning of the student model (multi-task recognition model).

[0078] Furthermore, embodiments of this application also provide a method for eye appearance image recognition, such as... Figures 1-3 As shown, the method for eye appearance photo recognition according to this application includes the following steps S301-S302: S301. Obtain the eye appearance photo to be recognized; S302. Input the eye appearance photo to be recognized into a multi-task recognition model to obtain the eye feature recognition result.

[0079] The eye appearance photograph to be identified in step S301 is one or more eye appearance photographs in a frontal-lateral position and / or a frontal position.

[0080] In step S302, the multi-task recognition model is obtained by training a multi-category image of the eye appearance using the aforementioned multi-task recognition model training method. After inputting the image of the eye appearance to be recognized into the multi-task recognition model, the resulting feature recognition result can be the probability that the image of the eye appearance to be recognized belongs to each category.

[0081] The purpose of obtaining recognition results through a multi-task recognition model is to assist doctors in judging TAO (Transient Auricular Ocular Disorder). To further assist, this application also proposes to visualize the ocular sign recognition results and identify the image regions that contribute the most to the ocular sign recognition results. Specifically, this is achieved through visualization tools for the network model, such as the Grad-CAM visualization tool, which displays on the ocular appearance image to be identified which region the probability result of each category is mainly based on, thereby assisting doctors in locating the signs.

[0082] Additionally, it should be noted that since the multi-task recognition model in this application embodiment is a lightweight model, the method for eye appearance photo recognition can be deployed in lightweight terminals, such as mobile phones, tablets, and other lightweight devices, making it more convenient to use.

[0083] As can be seen from the above description, the eye appearance photo recognition method in the embodiments of this application applies a multi-task recognition model to eye appearance photo recognition, which can greatly improve the efficiency of eye image recognition and thus better assist in TAO judgment.

[0084] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0085] According to embodiments of this application, a method for implementing the above is also provided. Figure 5 The apparatus 400 for training the multi-task recognition model of this application utilizes a deep learning network model for training, such as... Figure 6 As shown, the multi-task recognition model training device 400 of this application includes: a first extraction unit 41, used to extract features from eye appearance photos by sequentially passing a feature-sharing convolutional layer and a sub-task first independent convolutional layer to obtain second extraction results corresponding to different sub-tasks. Different sub-task first independent convolutional layers correspond to different sub-tasks, and each sub-task corresponds to a disease sign. The feature-sharing convolutional layer and the sub-task first independent convolutional layer are both convolutional layers of a deep learning network model; a second extraction unit 42, used to perform dimensionality-upgrading and merging of the second extraction results corresponding to each sub-task with the second extraction results of all other sub-tasks, and then perform dimensionality-downgrading by convolution to obtain a new second extraction result with the same dimension as the second extraction result but different data; and a third extraction unit 43, used to input the new second extraction result corresponding to each sub-task into the sub-task second independent convolutional layer for feature extraction to obtain third image features. Different sub-task second independent convolutional layers correspond to different sub-tasks. The convolutional layers correspond to different sub-tasks, and each sub-task corresponds to a third image feature. The second independent convolutional layer of each sub-task belongs to the convolutional layer of the deep learning network model. The classification unit 44 is used to input the third image feature corresponding to each sub-task into the independent classification layer of each sub-task, which is composed of fully connected layers in the deep learning network model, to classify and obtain a fourth image feature containing disease signs. Each sub-task corresponds to a fourth image feature. The probability calculation unit 45 is used to calculate the preset probability of the multi-task recognition model category based on the fourth image feature. The loss value calculation unit 46 is used to calculate the second cross-entropy loss value between the preset probability of the multi-task recognition model category and the actual category label based on the cross-entropy loss function. The optimization unit 47 is used to iteratively train the model to obtain the multi-task recognition model by optimizing the second cross-entropy loss value based on the optimizer. The multi-task recognition model is a single model for recognizing multiple disease signs.

[0086] Furthermore, such as Figure 6As shown, the device further includes: a training unit 48, used to train multiple teacher models using multiple deep learning network models based on the eye appearance photo samples, each teacher model corresponding to the recognition of an eye appearance photo of a disease sign; the optimization unit 47 is further used to perform iterative training of the model based on the distance between the output results of the network layers in the teacher models and the output results of the corresponding network layers in the training of the multi-task recognition model, wherein the network layers include convolutional layers and fully connected layers.

[0087] Furthermore, such as Figure 6 As shown, the training unit 48 includes: a first extraction module 481, used to extract first image features from the eye appearance photo sample by a feature extractor composed of multiple convolutional layers, activation functions, and batch normalization layers in a deep learning network model; a classification module 482, used to input the first image features into a classifier composed of fully connected layers in a deep learning network model to obtain second image features, wherein the second image features are features containing disease sign category information; a probability calculation module 483, used to calculate the teacher model category prediction probability based on the softmax activation function and the second image features; a first loss calculation module 484, used to calculate the first cross-entropy loss value between the teacher model category prediction probability and the actual category label based on the cross-entropy loss function; and a first optimization module 485, used to optimize the first cross-entropy loss value based on the optimizer, and determine the model corresponding to the minimum first cross-entropy loss value as the teacher model.

[0088] Furthermore, such as Figure 6 As shown, the optimization unit 47 further includes: a second loss calculation module 471, used to calculate the first maximum mean difference loss value between the first image feature and the third image feature corresponding to each disease sign; a third loss calculation module 472, used to calculate the second maximum mean difference loss value between the output results of other convolutional layers before obtaining each third image feature and the output results of other convolutional layers before obtaining each corresponding first image feature; a fourth loss calculation module 473, used to calculate the KL divergence loss value between the fourth image feature and the second image feature corresponding to each disease sign; and a second optimization module 474, used to perform iterative training of the model by optimizing the first maximum mean difference loss value, the second maximum mean difference loss value, and the KL divergence loss value based on the optimizer.

[0089] Furthermore, such as Figure 4As shown, the first extraction unit 41 includes: a second extraction module 411, used to sequentially input the eye appearance photo sample into multiple feature-sharing convolutional layers and obtain a first extraction result after passing through an activation function and a batch normalization layer; and a third extraction module 412, used to input the first extraction result into the first independent convolutional layer of each subtask to extract image features and obtain the corresponding second extraction result.

[0090] Specifically, the detailed process by which each unit and module in the device of this application implements its function can be found in the relevant description in the method embodiment, and will not be repeated here.

[0091] As can be seen from the above description, the multi-task recognition model training device in this application embodiment constructs a multi-task recognition model. During the model training process, in this embodiment, after the feature-sharing convolutional layer, features are extracted by corresponding independent convolutional layers for different sub-tasks. This allows for the learning of more refined features for different sub-tasks, thereby improving the accuracy of the multi-task recognition model. The subsequent classification layer also performs classification by corresponding independent classification layers for different sub-tasks, again to learn more refined features for different tasks, thus improving the accuracy of the multi-task recognition model. In addition, before classification, the second extraction results corresponding to each sub-task are subjected to dimensionality merging and dimensionality reduction processing. This ensures that while the independent convolutional layers for each sub-task learn specific refined features, certain effective information can still be shared between them (between different sub-tasks). This allows for better sharing of common semantic information and features among different categories of disease signs across different dimensions in the deep learning network, enabling the model to achieve fast and accurate recognition of multiple categories within a single model. The multi-task recognition model in this embodiment is a single model capable of recognizing multiple disease signs. Compared to existing single recognition models designed for different categories, it has fewer parameters and faster inference speed. Furthermore, knowledge distillation is achieved by using the knowledge from the teacher model for recognizing eye appearance photos of a single disease sign category as the training knowledge for the multi-task recognition model. This allows the model to learn in a more favorable direction early in the training process, improving its accuracy.

[0092] According to embodiments of this application, a method for implementing the above is also provided. Figure 7 The method of eye appearance photo recognition device 500, such as Figures 1-3 As shown, the eye appearance photo recognition device of this application includes: an acquisition unit 51, used to acquire an eye appearance photo to be recognized; and a recognition unit 52, used to input the eye appearance photo to be recognized into a multi-task recognition model to obtain an eye feature recognition result, wherein the multi-task recognition model uses the aforementioned... Figure 8The method for training a multi-task recognition model in the dataset is obtained.

[0093] Furthermore, such as ​ As shown, the device further includes a visualization unit 53, used to visualize the eye sign recognition results and determine the image region that contributes the most to the eye sign recognition results.

[0094] Specifically, the detailed process by which each unit and module in the device of this application implements its function can be found in the relevant description in the method embodiment, and will not be repeated here.

[0095] As can be seen from the above description, the eye appearance photo recognition device of this application applies a multi-task recognition model to eye appearance photo recognition, which can greatly improve the efficiency of eye image recognition and thus better assist in TAO judgment.

[0096] According to an embodiment of this application, a computer-readable storage medium is also provided, characterized in that the computer-readable storage medium stores computer instructions, which are used to cause the computer to execute the multi-task recognition model training method or the eye appearance photo recognition method in the above method embodiments.

[0097] According to an embodiment of this application, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the multi-task recognition model training method or the eye appearance photo recognition method in the above method embodiments.

[0098] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0099] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a multi-task recognition model, wherein the method utilizes a deep learning network model for training, characterized in that, The method includes: Based on eye appearance photos, feature extraction is performed sequentially through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer to obtain second extraction results corresponding to different subtasks. Different subtask-specific first independent convolutional layers correspond to different subtasks, and each subtask corresponds to a specific disease sign. Both the feature-sharing convolutional layer and the subtask-specific first independent convolutional layer are convolutional layers in a deep learning network model. The process of obtaining second extraction results for different subtasks by sequentially performing feature extraction through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer based on eye appearance photos includes: first, inputting the eye appearance photos into the feature-sharing convolutional layer to obtain a first extraction result; then, inputting the first extraction result into the subtask-specific first independent convolutional layer corresponding to each subtask to obtain the corresponding second extraction result. The second extraction result corresponding to each subtask is dimensionality-upgraded and merged with the second extraction results of all other subtasks, and then dimensionality-downgraded by convolution to obtain a new second extraction result with the same dimension as the second extraction result but different data. The new second extraction result corresponding to each subtask is input into the second independent convolutional layer of the subtask to extract features and obtain the third image features. Different subtasks have different second independent convolutional layers corresponding to different subtasks. Each subtask corresponds to a third image feature. The second independent convolutional layer of the subtask belongs to the convolutional layer of the deep learning network model. The third image feature corresponding to each subtask is input into the independent classification layer of each subtask, which is composed of fully connected layers in the deep learning network model, to obtain the fourth image feature containing the disease signs category. Each subtask corresponds to one fourth image feature. Calculate the preset probability of the multi-task recognition model category based on the fourth image feature; The second cross-entropy loss value between the preset probability of the category and the actual category label of the multi-task recognition model is calculated based on the cross-entropy loss function. The multi-task recognition model is obtained by iteratively training the model by optimizing the second cross-entropy loss value based on an optimizer. The multi-task recognition model is a single model that identifies multiple disease signs.

2. The method for training a multi-task recognition model according to claim 1, characterized in that, The method further includes: Based on the aforementioned eye appearance photos, multiple teacher models were trained using multiple deep learning network models, with each teacher model corresponding to the recognition of an eye appearance photo for a specific disease sign. The model is iteratively trained based on the distance between the output of the network layer in the teacher model and the output of the corresponding network layer in the training of the multi-task recognition model. The network layer includes convolutional layers and fully connected layers.

3. The method for training a multi-task recognition model according to claim 2, characterized in that, The multiple teacher models, trained using multiple deep learning network models based on eye appearance photograph samples, include: A feature extractor, consisting of multiple convolutional layers, activation functions, and batch normalization layers in a deep learning network model, extracts the first image features of the eye appearance photo sample. The first image feature is input into a classifier composed of fully connected layers in a deep learning network model to obtain the second image feature, which is a feature containing disease sign category information. The teacher model category prediction probability is calculated based on the softmax activation function and the second image features. The first cross-entropy loss value between the predicted class probability of the teacher model and the actual class label is calculated based on the cross-entropy loss function. The optimizer optimizes the first cross-entropy loss value, and the model corresponding to the minimum first cross-entropy loss value is determined as the teacher model.

4. The method for training a multi-task recognition model according to claim 3, characterized in that, The iterative training of the model based on the distance between the output of the network layer in the teacher model and the output of the corresponding network layer in the training of the multi-task recognition model includes: Calculate the first maximum mean difference loss value between the first image feature and the third image feature corresponding to each disease sign; The second maximum mean difference loss value is calculated between the output of other convolutional layers before obtaining each third image feature and the output of other convolutional layers before obtaining each corresponding first image feature; Calculate the KL divergence loss value between the fourth image feature and the second image feature corresponding to each disease feature; The model is iteratively trained by optimizing the first maximum mean difference loss value, the second maximum mean difference loss value, and the KL divergence loss value based on the optimizer.

5. The method for training a multi-task recognition model according to claim 1, characterized in that, Based on the aforementioned eye appearance photograph samples, feature extraction is performed sequentially through a feature-sharing convolutional layer and a first independent convolutional layer for each subtask to obtain second extraction results corresponding to different subtasks, including: The eye appearance photo samples are sequentially input into multiple feature-sharing convolutional layers and then processed through activation functions and batch normalization layers to obtain the first extraction result. The first extraction result is input into the first independent convolutional layer of each subtask to extract image features and obtain the corresponding second extraction result.

6. A method for eye appearance recognition, characterized in that, The method includes: Obtain an image of the eye to be identified; The eye appearance photo to be identified is input into a multi-task recognition model to obtain the eye feature recognition result. The multi-task recognition model is obtained by the method of training a multi-task recognition model as described in any one of claims 1 to 5.

7. An apparatus for training a multi-task recognition model, the apparatus utilizing a deep learning network model for training, characterized in that, The device includes: The first extraction unit is used to extract features from an eye appearance photo sample by sequentially passing through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer to obtain second extraction results corresponding to different subtasks. Different subtask-specific first independent convolutional layers correspond to different subtasks, and each subtask corresponds to a disease sign. Both the feature-sharing convolutional layer and the subtask-specific first independent convolutional layer are convolutional layers of a deep learning network model. The process of extracting features from an eye appearance photo sample by sequentially passing through a feature-sharing convolutional layer and a subtask-specific first independent convolutional layer to obtain second extraction results for different subtasks includes: first inputting the eye appearance photo sample into the feature-sharing convolutional layer to obtain a first extraction result, and then inputting the first extraction result into the subtask-specific first independent convolutional layer corresponding to each subtask to obtain the corresponding second extraction result. The second extraction unit is used to merge the second extraction results corresponding to each subtask with the second extraction results of all other subtasks in terms of dimensionality, and then reduce the dimensionality by convolution to obtain a new second extraction result with the same dimension as the second extraction result but different data. The third extraction unit is used to input the new second extraction result corresponding to each subtask into the second independent convolutional layer of the subtask to extract features and obtain the third image features. Different subtasks have different second independent convolutional layers corresponding to different subtasks, and each subtask corresponds to a third image feature. The second independent convolutional layer of the subtask belongs to the convolutional layer of the deep learning network model. The classification unit is used to input the third image features corresponding to each subtask into the independent classification layer of each subtask, which is composed of fully connected layers in the deep learning network model, to classify and obtain the fourth image features containing disease signs categories. Each subtask corresponds to one fourth image feature. The probability calculation unit is used to calculate the preset probability of the multi-task recognition model category based on the fourth image features; The loss value calculation unit is used to calculate the second cross-entropy loss value between the preset probability of the category of the multi-task recognition model and the actual category label based on the cross-entropy loss function. The optimization unit is used to iteratively train the model to obtain the multi-task recognition model by optimizing the second cross-entropy loss value based on the optimizer. The multi-task recognition model is a single model that recognizes multiple disease signs.

8. A device for eye appearance photo recognition, characterized in that, The device includes: The acquisition unit is used to acquire an image of the eye appearance to be identified; The recognition unit is used to input the eye appearance photo to be recognized into the multi-task recognition model to obtain the eye feature recognition result. The multi-task recognition model is obtained by the method of training the multi-task recognition model as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method for training a multi-task recognition model as described in any one of claims 1 to 5, or the method for recognizing an eye appearance photo as described in claim 6.

10. An electronic device, characterized in that, include: At least one processor; The at least one processor is also connected in communication with a memory, wherein the memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the multi-task recognition model training method according to any one of claims 1 to 5, or to perform the eye appearance photo recognition method according to claim 6.

Images