A method and system for eyelid tumor classification

Abstract

The application provides an eyelid tumor classification method and system, relates to the technical field of medical image analysis, and comprises the following steps: feature map blocking; main similarity matrix construction; dense feature similarity matrix construction; cosine angle calculation between the main similarity matrix and the corresponding row vectors of the dense feature similarity matrix in units of behaviors; definition of a partial cosine adjustment formula according to the cosine angle to globally fine-tune each element in each row vector of the dense feature similarity matrix; after adjustment of each element in units of behaviors, an adjusted dense feature similarity matrix is obtained; similarity calculation of each category feature of the query set and the support set according to the adjusted dense feature similarity matrix to obtain a classification result, and output of the probability of eyelid tumor belonging to different categories through a Softmax function, so that the query set image is determined as the category with the maximum probability. The accuracy and reliability of eyelid tumor benignity and malignancy classification under small sample conditions are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image analysis, and in particular, to an eyelid tumor classification method and system. BACKGROUND

[0002] In the field of medical imaging, early diagnosis of ocular tumors is of vital importance to improve cure rate and survival rate. Ocular tumors often manifest as fundus abnormalities, with early symptoms not obvious, posing a challenge to doctors' diagnosis. Therefore, ocular tumor classification technology based on computer vision and deep learning has been widely studied and applied. By utilizing medical image data and advanced algorithms, it can effectively assist doctors in classifying benign and malignant ocular tumors. In recent years, with the rapid development of deep learning, convolutional neural networks (CNN) and transfer learning have shown excellent results in medical image analysis. Specifically in ocular tumor detection, many studies have applied CNN to image classification tasks to distinguish between benign and malignant tumors. However, due to the scarcity and complexity of tumor samples, traditional deep learning methods do not perform well in the few-shot learning (FSL) scenario. The challenge of few-shot learning is that there are usually only a limited number of samples available for learning during the training process, which puts higher requirements on the generalization ability and accuracy of the model. To address these issues, few-shot learning (FSL) technology has emerged, which enables machine learning models to have good learning ability even with only a few labeled samples. FSL transfers existing knowledge to new tasks to make up for the lack of data. Therefore, how to accurately classify ocular tumors under the condition of small samples has become a key challenge in medical image analysis.

[0003] In the field of medical image analysis, especially in the task of eyelid tumor classification, few-shot learning faces many challenges. Traditional deep learning models highly rely on large-scale labeled data, while medical image labeling requires the involvement of professional doctors, which is costly and time-consuming, resulting in a scarcity of labeled data. This directly causes traditional models to perform poorly in data-scarce scenarios, making it difficult to accurately classify eyelid tumors.

[0004] In the process of implementing the present application, the applicant found that in the existing Few-shot learning method, the method based on similarity measurement has serious defects. Due to the scarcity of samples, the intra-class difference is large, so that the image embedding of the same class in the metric space is easy to deviate. The existing method often only depends on the similarity matrix formed by the dense features to represent the relationship between the support set and the query set, completely ignoring the important factor that the backbone features can also represent the similarity relationship. For example, the common practice is to extract features by using a backbone network such as ResNet, and then further extract dense features by means of pyramid convolution and construct a similarity matrix, but only the backbone features are used to further standardize the dense features, without truly mining the similarity value of the backbone features, which leads to the inability to comprehensively and accurately depict the similarity relationship between images, greatly affecting the classification accuracy. In the aspect of dense feature extraction, the existing method also has obvious defects. The traditional method based on full convolution network (FCN) has limited feature expression ability due to the fixed receptive field, which is difficult to effectively capture multi-scale information and cannot meet the high requirements of feature extraction for the diversity of eyelid tumor morphology. Although the method based on pyramid structure can fuse features of different resolutions through multi-scale pooling operation, the feature fusion method is mostly simple splicing or summation, which fails to fully utilize the interaction information between features, and when processing medical images, the dynamic relationship between local details and global context is generally ignored, resulting in insufficient discriminative ability of feature representation and difficulty in accurately extracting key features of eyelid tumor images.

[0005] Therefore, how to accurately and efficiently distinguish the class (benign or malignant) of eye tumors becomes a technical problem to be solved. SUMMARY

[0006] The present application aims to at least solve one of the technical problems existing in the prior art or related art, and discloses an eyelid tumor classification method and system, which greatly improves the accuracy and reliability of benign and malignant classification of eye tumors under small sample conditions, and effectively solves the problems of feature complexity, similarity calculation and inter-class difference faced by existing methods.

[0007] The first aspect of the present application discloses an eyelid tumor classification method, comprising: feature map blocking: respectively blocking the backbone feature maps of the support set and the query set to obtain a plurality of backbone feature image blocks of the same size, and respectively blocking the dense feature maps of the support set and the query set to obtain a plurality of dense feature image blocks of the same size, wherein the support set and the query set comprise a plurality of eyelid tumor images;

[0008] Constructing a main similarity matrix: calculating the cosine similarity between the backbone feature image blocks of the query set and the backbone feature image blocks of the support set to construct a main similarity matrix with the query set as the row and the support set as the column;

[0009] Constructing dense feature similarity matrix: the cosine similarity between the dense feature image blocks of the query set and the dense feature image blocks of the support set is calculated to construct a dense feature similarity matrix with the query set as rows and the support set as columns;

[0010] Adjusting dense feature similarity matrix: the cosine angle between the corresponding row vectors of the main similarity matrix and the dense feature similarity matrix is calculated in units of rows, and a partial cosine adjustment formula is defined according to the cosine angle to globally fine-tune each element in the row vector of the dense feature similarity matrix. After adjusting each element in units of rows, the adjusted dense feature similarity matrix is obtained.

[0011] Classification decision: according to the adjusted dense feature similarity matrix, the similarity of each class feature of the query set and the support set is calculated to obtain the classification result, and the Softmax function is used to output the probability of the eyelid tumor belonging to different categories, and the query set image is determined as the category with the maximum probability.

[0012] According to the eyelid tumor classification method disclosed by the application, preferably, the step of adjusting the dense feature similarity matrix specifically comprises:

[0013] Let each row of the main similarity matrix be alpha, and each row of the dense feature similarity matrix be beta.

[0014] The cosine angle theta between the two row vectors alpha and beta is calculated, and the calculation formula is:

[0015] theta= ;

[0016] A partial cosine adjustment formula is defined based on the cosine angle theta to globally fine-tune the row vector of the dense feature similarity matrix, and each element in the adjusted row vector is:

[0017]

[0018] Wherein, represents the jth element in each row of the adjusted dense feature similarity matrix, represents the jth element in each row of the unadjusted dense feature similarity matrix, and gamma is an adjustable parameter. When alpha*beta>0, sgn(alpha*beta)=1; when alpha*beta=0, sgn(alpha*beta)=0; when alpha*beta<0, sgn(alpha*beta)=-1.

[0019] In the technical solution, the core principle of the cosine adjustment formula is that when the angle theta between the row vector of the main similarity matrix and the dense feature similarity matrix is small, cos(theta) is close to 1, and (1-cos(theta)) tends to 0, at this time, the adjustment amplitude of the elements of the corresponding row vector of the dense feature similarity matrix is small, because the similarity of the two is high itself; when the angle is large, cos(theta) is small, (1-cos(theta)) is large, and the adjustment amplitude is correspondingly increased. Gamma is used to flexibly control the strength of the overall adjustment, and by multiplying gamma, the adjustment degree can be flexibly adjusted according to different application scenarios and data characteristics, to ensure that good optimization effect can be achieved in various cases. At the same time, sgn(alpha*beta) is used to determine the direction of adjustment, so that the row vector of the adjusted dense feature similarity matrix and the main similarity matrix are more consistent in direction, thereby effectively reducing the angle between the two and greatly improving the accuracy of the feature similarity measure.

[0020] According to the eyelid tumor classification method disclosed in the application, preferably, before the feature map blocking step, the method further comprises: backbone feature extraction: extracting the backbone features of the eyelid tumor images in the support set and the query set based on a convolutional neural network; dense feature enhancement: inputting the backbone features into a dense feature extractor to perform feature enhancement on the backbone features and obtain dense features.

[0021] According to the eyelid tumor classification method disclosed in the application, preferably, the step of dense feature enhancement specifically comprises:

[0022] The backbone features are processed by an AGP (Attention-Guided Pyramid) module: the input feature map is divided into a plurality of sub-feature maps of different scales; attention weights are calculated for each scale of the sub-feature maps; each scale of the sub-feature maps is multiplied by the corresponding attention weight map to highlight important feature regions, and then the weighted sub-feature maps of different scales are fused to form a dense feature map.

[0023] Alternatively, the backbone features are processed by a DFEG (Dynamic Feature Enhancement Grid) module: the input feature map is divided into a plurality of grid regions, each grid region representing a local part of the image; for each grid region, a local convolutional neural network module is used for processing, and the local convolutional neural network module will perform weighted fusion or feature transformation on the features of the grid region according to the similarity and difference of the features of adjacent grid regions; the features of the grid regions after dynamic adjustment are recombined to form a dense feature map.

[0024] Preferably, the step of making a classification decision specifically comprises: EMD measurement: taking the adjusted dense feature similarity matrix as a probability distribution, calculating the bulldozer distance, and obtaining a classification result based on the nearest support set category of the query set; MEL measurement: constructing a transition graph based on the same matrix, calculating the Katz centrality, and obtaining a classification result; and simultaneously performing the EMD measurement and the MEL measurement to evaluate the input features from multiple angles.

[0025] Preferably, the step of constructing the main similarity matrix specifically comprises: for a backbone feature image block corresponding to an image in the query set, traversing the backbone feature image blocks in the support set, and calculating the cosine similarity between the query set image block and each support set image block; repeating the operation for each image block in the query set to construct a main similarity matrix with the query set image blocks as rows and the support set image blocks as columns; and the elements in the matrix present the similarity degree between the query set image blocks and the support set image blocks based on the backbone features.

[0026] In this technical solution, adaptive similarity calculation is adopted, and the similarity calculation between the support set and the query set is dynamically adjusted by introducing cosine similarity. As an alternative, traditional similarity calculation methods such as Euclidean distance and Manhattan distance can be considered. Although these methods do not have the dynamic learning ability in the present application, they still have certain applicability in some scenarios with lower requirements for computing resources. In addition, a graph neural network (GNN) based method can also be used as an alternative to calculate the similarity between the support set and the query set by processing the relationship between them through a graph structure. GNN has strong ability in processing graph structure data and adjacency relationship, and is suitable for processing complex data relationships.

[0027] Preferably, the step of constructing the dense feature similarity matrix specifically comprises: using adaptive pooling to keep the dense feature similarity matrix and the main similarity matrix consistent in dimension, and then obtaining the dense feature similarity matrix according to the cosine similarity formula; for a dense feature image block corresponding to an image in the query set, traversing the dense feature image blocks in the support set, and calculating the cosine similarity between the query set image block and each support set image block; repeating the operation for each image block in the query set to construct a dense feature similarity matrix with the query set image blocks as rows and the support set image blocks as columns; and the elements in the matrix present the similarity degree between the query set image blocks and the support set image blocks based on the dense features.

[0028] Preferably, the dense feature extractor specifically comprises an AGP module, a DFEG module, or a Transformer module.

[0029] Preferably, the convolutional neural network specifically comprises ResNet, Conv4, VGG, Inception, EfficientNet or MobileNet.

[0030] A second aspect of the present application discloses an eyelid tumor classification system, comprising: a memory for storing program instructions; and a processor for invoking the program instructions stored in the memory to implement the eyelid tumor classification method according to any of the above technical solutions.

[0031] The beneficial effects of the present application at least include: the prior art generally relies on a single feature extraction method (such as ResNet), which is difficult to capture the details of global and local features at the same time. The present application introduces attention-guided pyramid (AGP) and dynamic feature enhancement grid (DFEG), realizes multi-scale feature division and dynamic feature adjustment, significantly improves the discriminability of feature representation, and can more accurately capture the complex features of eyelid tumor images. The prior art generally relies on a single feature space in the construction of the similarity matrix, which is difficult to fully reflect the complex relationship between images. The present application constructs a main similarity matrix and a dense feature similarity matrix, and combines a bias cosine angle adjustment strategy, realizes double similarity collaborative optimization, significantly improves the accuracy of similarity measurement, and performs excellently in the few-shot learning scene. The present application innovatively regards each row of the main similarity matrix and the dense feature similarity matrix as an independent feature similarity unit, calculates the cosine angle row by row and applies the bias cosine adjustment strategy, and realizes the fine adjustment of the dense feature similarity matrix. This method not only optimizes the feature similarity from multiple dimensions, but also enhances the robustness and generalization ability of the classification model, and is particularly suitable for eyelid tumor classification tasks in the few-shot learning scene. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 A flowchart of an eyelid tumor classification method according to an embodiment of the present application is shown.

[0033] Figure 2 A schematic block diagram of an eyelid tumor classification system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. In the following description, a large number of specific details are set forth in order to facilitate a full understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the present application is not limited to the specific embodiments disclosed below.

[0035] As Figure 1As shown, according to one embodiment of the present application, a eyelid tumor classification method is disclosed, which is based on an n-way k-shot classification framework, and specifically includes the following steps:

[0036] Step S1, backbone feature extraction: based on a convolutional neural network, the backbone features of the eyelid tumor images in the support set and the query set are extracted;

[0037] Step S2, dense feature enhancement: the backbone features are input into a dense feature extractor to enhance the features, and the dense features are obtained;

[0038] Step S3, feature map blocking: the backbone feature maps of the support set and the query set are blocked respectively to obtain a plurality of backbone feature image blocks of the same size, and the dense feature maps of the support set and the query set are blocked respectively to obtain a plurality of dense feature image blocks of the same size, wherein the support set and the query set include a plurality of eyelid tumor images;

[0039] Step S4, constructing a main similarity matrix: the cosine similarity between the backbone feature image blocks of the query set and the backbone feature image blocks of the support set is calculated to construct a main similarity matrix with the query set as the row and the support set as the column;

[0040] Step S5, constructing a dense feature similarity matrix: the cosine similarity between the dense feature image blocks of the query set and the dense feature image blocks of the support set is calculated to construct a dense feature similarity matrix with the query set as the row and the support set as the column;

[0041] Step S6, adjusting the dense feature similarity matrix: the cosine angle between the corresponding row vectors of the main similarity matrix and the dense feature similarity matrix is calculated, and a bias cosine adjustment formula is defined according to the cosine angle to fine-tune each element in each row vector of the dense feature similarity matrix, and after adjusting each element in a row, an adjusted dense feature similarity matrix is obtained;

[0042] Step S7, classification decision: according to the adjusted dense feature similarity matrix, the similarity of each class feature of the query set and the support set is calculated to obtain a classification result, and the Softmax function is used to output the probability of the eyelid tumor belonging to different categories, and the query set image is determined as the category with the highest probability.

[0043] According to the above embodiment, preferably, the step of adjusting the dense feature similarity matrix specifically includes:

[0044] Let each row of the main similarity matrix be α, and each row of the dense feature similarity matrix be β;

[0045] The cosine angle θ between the two row vectors α and β is calculated, and the calculation formula is:

[0046] θ= ;

[0047] A bias cosine adjustment formula is defined based on the cosine angle θ to globally fine-tune the row vectors of the dense feature similarity matrix, and each element in the adjusted row vector is:

[0048]

[0049] wherein, represents the jth element in each row of the adjusted dense feature similarity matrix, represents the jth element in each row of the unadjusted dense feature similarity matrix, γ is an adjustable parameter, and sgn() is a sign function, sgn(α·β)=1 when α·β>0, sgn(α·β)=0 when α·β=0, and sgn(α·β)=-1 when α·β<0.

[0050] According to another embodiment of the application, a specific implementation process and working principle of the eyelid tumor classification method provided by the above-mentioned embodiments are also disclosed:

[0051] Data preparation stage: Collect eyelid tumor image data from various medical institutions, which covers a wide variety of tumor types and development stages. Then, according to the international authoritative medical annotation standard, senior ophthalmic experts carefully label the key information in the image, such as the specific type of tumor, the position in the image, the boundary contour, etc. After completing the labeling, a series of preprocessing operations are carried out on the original image, which is uniformly adjusted to a specific size (512x512 pixels), and the pixel value is normalized to the interval [0,1], and data augmentation means such as random rotation (±15 degrees), horizontal flip, vertical flip, etc. are used to expand the data set size.

[0052] Feature extraction stage: Select the pre-trained network ResNet with outstanding performance in the field of deep learning and Conv4 to carry out efficient feature extraction of the input tumor image. Taking ResNet as an example, the image will go through a series of convolutional layers, pooling layers and activation functions. In the convolutional layer, different sizes of convolutional kernels are responsible for extracting local features of the image; the pooling layer reduces the feature dimension through downsampling operation; the activation function is used to enhance the nonlinear expression ability of the model, and finally the feature map with strong representation ability is output. After that, the extracted features will enter the specially designed dense feature extractor, namely Attention-Guided Pyramid (AGP) and Dynamic Feature Enhanced Grid (DFEG), to accept further feature enhancement. The AGP layer (AGP module) guides the pyramid structure by means of attention mechanism, which can significantly highlight important feature regions; the DFEG layer (DFEG module) constructs a dynamic feature enhancement grid, fully considers the feature information of adjacent regions, and accurately distinguishes the features of the support set and the query set.

[0053] Dual similarity coordination and optimization: The main similarity matrix and the dense feature similarity matrix are calculated respectively. The main similarity matrix is constructed based on image blocks, and the similarity between each image block and all image blocks in other images is calculated by cosine similarity, which reflects the similarity relationship of the backbone features. The dense feature similarity matrix is constructed based on the image block dense feature vector processed by the AGP and DFEG layers. What is particularly key is that the invention uses a unique fine-tuning strategy based on the cosine angle to optimize the dense feature similarity matrix. Considering factors such as the directionality and importance of the features. According to the backbone feature similarity information contained in the main similarity matrix, the cosine angle of each feature vector pair corresponding to the elements in the dense feature similarity matrix is calculated, and the value in the dense feature similarity matrix is finely adjusted according to the difference between the cosine angle and the original cosine angle. This process realizes the accurate balance of the role of backbone features and dense features in the classification process, greatly improving the accuracy of feature similarity measurement.

[0054] The classification decision stage: based on the efficient query method of EMD (Earth Mover's Distance) and MEL (Matching Equilibrium Learning), an end-to-end classification measurement method is adopted. EMD measures the similarity between two feature distributions by calculating the minimum cost, in the invention, it compares the feature distribution of the query set with the feature distribution of each category in the support set, and gives the distance value reflecting the similarity. MEL is a method of constructing a transition matrix based on dense features and using Katz centrality to determine the similarity of categories. MEL uses transition matrix and Katz centrality to mine category similarity from the structure and relationship of features. These query methods work in parallel to evaluate the input features from multiple angles. The system inputs the feature information after double similarity coordination and optimization into the classification decision module, assigns appropriate weights to each query result through the pre-trained weight distribution mechanism, analyzes and integrates these results comprehensively, and finally outputs the probability of eyelid tumor belonging to different categories through the Softmax function. The image is judged as the category with the highest probability, realizing accurate classification of benign and malignant eyelid tumors.

[0055] According to the above embodiment, the support set (Support Set) refers to a small amount of labeled samples provided to the model for support classification or reasoning in the few-shot learning task. Specifically, the support set contains a certain number of samples, which have known labels and are used to guide the model to classify or regress in the query set (Query Set). The key role of the support set is to provide a reference for the model in the case of sample scarcity, helping the model to learn enough knowledge from limited labeled data, so as to effectively infer the class of unknown samples in the query set. The support set usually constitutes a training process together with the query set. Through the 2-way 1-shot or similar few-shot learning framework, each sample in the support set will provide the model with prior knowledge of the class. The samples in the support set do not need to be evenly distributed in the entire data set, but can represent the typical features of different categories. In a specific task, such as the benign and malignant classification task of eyelid tumors, the samples in the support set will include different types of tumor images, through which the model can learn the potential features to distinguish benign tumors from malignant tumors.

[0056] Similarly, each sample in the support set will extract preliminary features through a pre-trained model such as a deep convolutional neural network (ResNet or Conv4). These pre-trained models have been trained on large-scale datasets, so they can effectively capture low-level and mid-level features in the input image, such as edge, texture, and other information. However, these preliminary features only represent the basic structure and visual information of the image, and cannot fully reflect the high-level relationship and feature distribution between samples. In order to further enhance the feature expression ability and improve the generalization ability of the model in the few-shot learning scenario, the preliminary extracted features will be passed to a specially designed dense feature extractor. The dense feature extractor further processes the preliminary features through a more complex network structure to capture fine-grained differences and high-level semantic information between samples. Through this process, the sample features in the support set will be more accurately represented, effectively reflecting the distinguishing information between different categories. This multi-stage feature extraction method not only ensures the quality of the extracted features, but also ensures that the support set samples can provide more accurate and highly discriminative feature vectors when calculating the similarity with the query set samples. This lays a solid foundation for subsequent similarity calculation and category prediction, thereby improving the classification effect and overall performance of the model in the few-shot learning task.

[0057] The query set refers to the sample set used for classification prediction in the few-shot learning task. The role of the query set is to compare and calculate the similarity with the support set (Support Set) to help the model determine its category. The task of the query set (Query Set) is to judge the benign and malignant of eyelid tumors. Each sample in the query set is a tumor image to be classified, and the goal is to compare the similarity with the samples in the support set to infer the category of the query set sample, i.e., to determine whether the tumor is benign or malignant. The query set plays a crucial role because it carries the samples to be classified, and in the model training process, the classification labels of the query set samples are usually unknown. Therefore, the samples in the query set need to be inferred based on the known support set samples when classifying.

[0058] In this invention, the role of the query set mainly reflects in the following aspects:

[0059] Determine the benign and malignant of tumors: Each tumor image in the query set represents a sample to be classified. By comparing with the tumor images in the support set with known categories, the features of the query set samples can help the model identify their benign and malignant. Query set samples may come from different patients, and their features may have some differences, but through high-quality feature extraction and accurate similarity calculation, the model can accurately classify the tumors.

[0060] Similarity calculation: the similarity calculation between the query set and the support set samples is the core of the classification process. The features of the query set samples are matched with the features of the support set samples, and the relationship between the query samples and the support set samples is obtained through similarity calculation. The query set samples obtain the prediction result of their categories through this process. In order to improve the accuracy of this process, each sample in the query set will undergo deep feature extraction, ensuring that the unique features of the query sample can be fully expressed when compared with the support set samples.

[0061] Application of few-shot learning: the classification task of eyelid tumor often has the problem of sample scarcity, especially for some special types of tumors. The operation of query set and support set under the framework of few-shot learning is particularly important. The query set is compared with the limited samples in the support set, and the model learns the class information of the support set to accurately classify the query set samples. This few-shot learning method is particularly suitable for the medical field, especially in the early screening of eyelid tumors, which can significantly improve the classification efficiency and accuracy.

[0062] According to another embodiment of the application, the implementation principle of the dense feature extractor of the above-mentioned embodiment is also disclosed: in order to ensure that the output of the dense feature representation layer has a uniform spatial dimension, the scheme adopts a multi-module parallel feature extraction architecture. Among them, the existing PyramidFCN architecture naturally meets the spatial consistency requirement through its inherent multi-scale pooling structure, providing basic global feature representation for the system. On this basis, the invention innovatively introduces two new modules parallel to it: attention guided pyramid (AGP) strengthens the feature response of the key area of the tumor through spatial attention mechanism, and dynamic feature enhancement grid (DFEG) models the spatial correlation of local features through adaptive grid topology.

[0063] AGP Attention-Guided Pyramid: In the dense feature representation layer, in order to ensure the consistency of the spatial dimensions of the feature maps output by different backbone networks, facilitate the refinement and alignment of the subsequent similarity matrix, a dense feature transformation T is adopted. At the same time, an Attention-Guided Pyramid (AGP) layer is introduced, which can enhance the discriminative ability of feature representation while maintaining spatial consistency. Specifically, the AGP layer uses an attention mechanism to guide the pyramid structure. It first divides the input feature map into multiple sub-feature maps of different scales, for example, by using different size pooling operations. Attention weights are calculated for each scale of sub-feature map. Specifically, a small convolutional neural network is used to take the sub-feature map as input and output an attention weight map of the same size as the sub-feature map. Each element in the weight map represents the importance of the feature at the corresponding position. Multiply each scale of sub-feature map with the corresponding attention weight map to highlight important feature regions, and then fuse the weighted sub-feature maps of different scales, for example, by concatenation, to obtain the feature map output by the AGP layer, thereby enhancing the discriminative ability of feature representation.

[0064] DFEG Dynamic Feature Enhancement Grid: It works by constructing a dynamic feature enhancement grid. It first divides the input feature map into multiple grid regions, each of which represents a local part of the image. For each grid region, a local convolutional neural network module is used for processing. This module can fully consider the feature information of adjacent grid regions and dynamically adjust the feature representation of the current grid region. Specifically, the module will weight the fusion or feature transformation of the feature of the current grid region according to the similarity and difference of the features of adjacent grid regions. For example, if the adjacent grid regions have similar texture features, the expression of the texture features is enhanced through weighted fusion; if the features of adjacent grid regions are quite different, feature transformation is performed to better highlight the unique features of the current grid region. The features of each grid region after dynamic adjustment are recombined to form the feature map output by the DFEG layer. This method further improves the expression ability of the features, and at the same time, since the spatial dimension consistency principle is strictly followed in the processing process, the compatibility of the output feature map with other modules is ensured, providing a more discriminative and consistent feature basis for subsequent similarity-based classification tasks.

[0065] According to another embodiment of the present application, the implementation principle of the above-mentioned embodiment of the partial cosine adjustment dense similarity matrix is also disclosed:

[0066] The specific process of constructing the main similarity matrix includes: firstly, using convolutional neural network and the like to extract basic features for the eyelid tumor images. Then, the images in the query set and the support set are processed, and each image is divided into image blocks of a fixed size (512*512 pixels). Taking the feature vector of an image block corresponding to an image in the query set as an example, the feature vectors corresponding to the image blocks in the support set are traversed, and the similarity between the image block in the query set and each image block in the support set is calculated according to the cosine similarity formula. In this way, the operation is repeated for each image block in the query set, and the main similarity matrix with the image blocks in the query set as rows and the image blocks in the support set as columns is constructed. The elements in the matrix clearly present the similarity between the image blocks in the query set and the image blocks in the support set based on the backbone features, laying a foundation for subsequent bias cosine adjustment.

[0067] The specific process of constructing the dense feature similarity matrix includes: using advanced dense feature extractors (for example, AGP and DFEG) containing attention mechanisms to further process the image blocks in the query set and the support set, and obtain the corresponding dense feature vectors. Then, the calculation logic used when constructing the main similarity matrix is followed, and the similarity between the image blocks in the query set and the support set is calculated based on the newly obtained dense feature vectors, and then the dense feature similarity matrix is constructed. The adaptive pooling method is used to ensure that the main feature similarity matrix maintains the same dimension:

[0068] F out = AdaptivePool(F in ,h,w) ⊕ Conv 1×1 (F in )

[0069] wherein, F out represents the output feature map, AdaptivePool(F in ,h,w) represents the input feature map F in adjusted to the target spatial dimension h x w, Conv 1×1 (F in ) represents applying a 1*1 convolution to adjust the channel dimension of the input feature map F in , and ⊕ represents element-wise addition. This transformation ensures that the spatial dimension of the output feature map is consistent.

[0070] After the above adaptive pooling process, the dense feature similarity matrix is obtained according to the cosine similarity formula.

[0071] The partial cosine angle adjustment (optimization) process comprises: in order to optimize the dense feature similarity matrix and make it more accurately reflect the complex internal relationship between the eyelid tumor image features, the application designs a fine-tuning mechanism based on the main similarity matrix. The mechanism takes the unit of behavior, calculates the cosine angle between the main similarity matrix and the corresponding row vector of the dense feature similarity matrix, and adjusts the row vector of the dense feature similarity matrix by means of the parameter γ, and then narrows the difference between the two. The specific process is as follows:

[0072] Let each row of the main similarity matrix be α, and each row of the dense feature similarity matrix be β.

[0073] The cosine angle θ between the two row vectors α and β is calculated, and the calculation formula is:

[0074] θ= ;

[0075] Based on the cosine angle θ, the partial cosine adjustment formula is defined to fine-tune the row vector of the dense feature similarity matrix as a whole. The elements in the adjusted row vector are:

[0076]

[0077] Wherein, represents the jth element in each row of the adjusted dense feature similarity matrix, represents the jth element in each row of the unadjusted dense feature similarity matrix, and γ is an adjustable parameter. In a series of experiments for eyelid tumor image classification, when γ=0.2, the performance of the classification model reaches a relatively ideal state; sgn() is a sign function, when α·β>0, sgn(α·β)=1; when α·β=0, sgn(α·β)=0; when α·β<0, sgn(α·β)=-1. The core principle of the partial cosine adjustment formula is that when the row vectors of the main similarity matrix and the dense feature similarity matrix have a small angle θ, cos(θ) is close to 1, and (1-cos(θ)) tends to 0, so the adjustment amplitude of the corresponding row vector elements of the dense feature similarity matrix is small, because the similarity between the two is high. When the angle is large, cos(θ) is small, (1-cos(θ)) is large, and the adjustment amplitude increases accordingly. γ is used to flexibly control the strength of the overall adjustment. By multiplying γ, the adjustment degree can be flexibly adjusted according to different application scenarios and data characteristics, to ensure good optimization effect in various situations. At the same time, sgn(α·β) is used to determine the direction of adjustment, so that the row vector of the adjusted dense feature similarity matrix is more consistent with the main similarity matrix in direction, thereby effectively reducing the angle between the two and greatly improving the accuracy of feature similarity measurement.

[0078] In this embodiment, each row of the primary similarity matrix and each row of the dense feature similarity matrix is regarded as an independent "Patch", that is, a feature similarity unit; this innovative design enables us to optimize feature similarity from a more fine-grained perspective, thereby significantly improving classification performance. Each row of the primary similarity matrix represents the similarity distribution of the query image and the support image in the primary feature space, while each row of the dense feature similarity matrix reflects the similarity distribution of the query image and the support image in the dense feature space. By regarding each row as a "Patch", the similarity relationship between the two can be analyzed and adjusted row by row to ensure that the dense feature similarity matrix can more accurately capture the complex relationship between images. During the adjustment process, the cosine angle between each row of the primary similarity matrix and the corresponding row of the dense feature similarity matrix is first calculated to measure the difference in direction between the two. Based on this angle, a bias cosine angle adjustment strategy is further introduced to fine-tune each row of the dense feature similarity matrix. This adjustment not only considers the consistency between the primary features and the dense features, but also flexibly controls the optimization intensity by introducing an adjustment parameter, thereby achieving the best classification effect in different scenarios.

[0079] As Figure 2 shown, according to another embodiment of the application, an eyelid tumor classification system 200 is also disclosed, comprising: a memory 201 for storing program instructions; a processor 202 for calling the program instructions stored in the memory to realize the eyelid tumor classification method of the above-mentioned embodiment.

[0080] In summary, the application designs a dual similarity collaborative optimization mechanism: by introducing the dual collaborative optimization mechanism of the primary similarity matrix (primary similarity matrix) and the dense feature similarity matrix, the application can more comprehensively mine the complex relationship between the query set and the support set images. The primary similarity matrix captures the global feature similarity, while the dense feature similarity matrix focuses on the fine association of local features, and the combination of the two significantly improves the classification accuracy. The application proposes a bias cosine angle adjustment mechanism: by calculating the cosine angle between the primary similarity matrix and the dense feature similarity matrix, a bias cosine angle adjustment strategy is proposed. This strategy can dynamically adjust the row vector of the dense feature similarity matrix, so that it is more consistent with the primary similarity matrix in direction, thereby reducing the difference between the two and improving the accuracy of feature similarity measurement.

[0081] All or part of the steps of various methods of the above embodiments can be completed by a program controlling relevant hardware, and the program can be stored in a readable storage medium, including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other readable medium capable of carrying or storing data.

[0082] The above description is merely preferred embodiments of the present application, but not for limiting the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for classifying eyelid tumors, characterized in that, include: Feature map segmentation: The backbone feature maps of the support set and the query set are segmented into blocks to obtain multiple backbone feature image blocks of the same size. The dense feature maps of the support set and the query set are also segmented into blocks to obtain multiple dense feature image blocks of the same size. The support set and the query set include multiple eyelid tumor images. Construct the main similarity matrix: Calculate the cosine similarity between the backbone feature image patches of the query set and the backbone feature image patches of the support set to construct the main similarity matrix with the query set as the row and the support set as the column; Construct a dense feature similarity matrix: Calculate the cosine similarity between dense feature image patches in the query set and dense feature image patches in the support set to construct a dense feature similarity matrix with the query set as the rows and the support set as the columns; Dense feature similarity matrix adjustment: Calculate the cosine angle between the corresponding row vectors of the main similarity matrix and the dense feature similarity matrix on a row-by-row basis. Define a partial cosine adjustment formula based on the cosine angle to fine-tune each element in each row vector of the dense feature similarity matrix. After adjusting each element on a row-by-row basis, the adjusted dense feature similarity matrix is ​​obtained. Classification decision: Based on the adjusted dense feature similarity matrix, the similarity of each category feature between the query set and the support set is calculated to obtain the classification result. The probability of eyelid tumors belonging to different categories is output through the Softmax function, and the query set image is determined to be the category with the highest probability.

2. The method for classifying eyelid tumors according to claim 1, characterized in that, The steps for adjusting the dense feature similarity matrix specifically include: Let each row of the primary similarity matrix be α, and each row of the dense feature similarity matrix be β; The cosine angle θ between the two row vectors α and β is calculated using the following formula: θ= ; Based on the cosine angle θ, a partial cosine adjustment formula is defined to fine-tune the row vectors of the dense feature similarity matrix. The elements of the adjusted row vectors are: ； in, This represents the j-th element in each row of the adjusted dense feature similarity matrix. denoted as the j-th element in each row of the unadjusted dense feature similarity matrix, γ is an adjustable parameter, and sgn() is the sign function. When α·β>0, sgn(α·β)=1; when α·β=0, sgn(α·β)=0; when α·β<0, sgn(α·β)=-1.

3. The method for classifying eyelid tumors according to claim 1, characterized in that, Before the feature map segmentation step, the following is also included: Bone feature extraction: Bone features of eyelid tumor images in the support set and query set are extracted based on convolutional neural networks; Dense Feature Enhancement: The backbone features are input into a dense feature extractor to enhance the backbone features and obtain dense features.

4. The method for classifying eyelid tumors according to claim 3, characterized in that, The dense feature enhancement steps specifically include: The backbone features are processed by the AGP module: the input feature map is divided into multiple sub-feature maps of different scales; attention weights are calculated for each sub-feature map of different scales; each sub-feature map of different scales is multiplied with the corresponding attention weight map to highlight important feature regions; and then the weighted sub-feature maps of different scales are fused to form a dense feature map. Alternatively, the backbone features can be processed through the DFEG module: the input feature map is divided into multiple grid regions, each representing a local part of the image; for each grid region, a local convolutional neural network module is used for processing, and the local convolutional neural network module performs weighted fusion or feature transformation on the features of the grid region based on the similarity and difference of the features of adjacent grid regions; the features of each grid region after dynamic adjustment are recombined to form a dense feature map.

5. The method for classifying eyelid tumors according to claim 1, characterized in that, The classification decision steps specifically include: EMD metric: Treat the adjusted dense feature similarity matrix as a probability distribution, calculate the bulldozer distance, and obtain the classification result based on the nearest support set category of the query set; MEL metric: Construct a transition graph based on the same matrix, calculate Katz centrality, and obtain classification results; Simultaneously execute the EMD metric and the MEL metric to evaluate the input features from multiple perspectives.

6. The method for classifying eyelid tumors according to claim 1, characterized in that, The steps for constructing the main similarity matrix specifically include: For a backbone feature image block corresponding to an image in the query set, traverse the backbone feature image blocks in the support set and calculate the cosine similarity between the query set image block and each support set image block. Repeat the operation for each image patch in the query set to construct a main similarity matrix with query set image patches as rows and support set image patches as columns; the elements in the matrix represent the degree of similarity between query set image patches and support set image patches based on backbone features.

7. The method for classifying eyelid tumors according to claim 1, characterized in that, The step of constructing the dense feature similarity matrix specifically includes: An adaptive pooling method is used to maintain the dimensionality consistency between the dense feature similarity matrix and the main similarity matrix, and then the dense feature similarity matrix is ​​obtained according to the cosine similarity formula: For a dense feature image patch corresponding to an image in the query set, traverse the dense feature image patches in the support set and calculate the cosine similarity between the query set image patch and each support set image patch. Repeat the operation for each image patch in the query set to construct a dense feature similarity matrix with query set image patches as rows and support set image patches as columns; the elements in the matrix represent the degree of similarity between query set image patches and support set image patches based on dense features.

8. The method for classifying eyelid tumors according to claim 3, characterized in that, The dense feature extractor specifically includes: an AGP module, a DFEG module, or a Transformer module.

9. The method for classifying eyelid tumors according to claim 3, characterized in that, The convolutional neural networks specifically include: ResNet, Conv4, VGG, Inception, EfficientNet, or MobileNet.

10. An eyelid tumor classification system, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke the program instructions stored in the memory to implement the eyelid tumor classification method as described in any one of claims 1 to 9.

Images