Cell image multi-task classification method and system based on location awareness and feature modulation

Abstract

The application discloses a kind of based on position perception and feature modulation cell image multi-task classification method and system, belong to medical image analysis field.The method first utilizes the global visual feature of cervical cell image extracted by depth backbone network, and predicts its hierarchical position (surface layer / middle layer / base layer).On this basis, introduce position perception hybrid expert module, utilize position information by two paths: one will position label be mapped into semantic embedding feature;Another generates channel level modulation parameter according to position label, carries out dynamic affine transformation to global feature, and the feature after modulation is sent into multiple expert networks.At the same time, based on the routing network of image content generates dynamic weight, weights the fusion of expert feature, obtains enhanced feature.Finally, enhanced feature and position semantic embedding feature are fused, and lesion type classification and position classification tasks are collaboratively trained.The application guides feature modulation and expert fusion by position information, effectively improves the accuracy and explainability of cervical cell image classification, and can be used for computer-aided screening of cervical cancer.

Description

Technical Field

[0001] This invention relates to the field of medical image analysis technology, specifically to a multi-task classification method and system for cell images based on position awareness and feature modulation, which is particularly suitable for cytopathological image analysis in cervical cancer screening. Background Technology

[0002] Cervical cancer is one of the most common malignant tumors in women, with over 600,000 new cases each year. 88% of these deaths occur in areas with limited medical resources, making early screening crucial for significantly improving the cure rate. Pap smears and the Bethesda classification system (e.g., no lesions, atypical squamous cells, atypical high- or low-grade lesions) are currently the mainstream pathological diagnostic criteria. However, this process requires manual interpretation of a large number of samples, which is time-consuming and susceptible to subjective factors, leading to missed diagnoses and misdiagnoses.

[0003] To improve diagnostic efficiency and accuracy, computer-aided diagnostic (CAD) systems have been extensively studied. Early systems relied on cell segmentation and manual feature extraction (such as morphology and texture), combined with traditional classifiers such as SVM and random forest for identification. However, due to the complexity of cell aggregation, large background interference, inaccurate segmentation, and the time-consuming and labor-intensive feature design, their application scope was limited.

[0004] In recent years, deep learning has made significant progress in cervical cell image classification. The DeepPap model, using an end-to-end convolutional neural network, achieved an accuracy of 98% and an AUC of 0.99 on the Herlev dataset without cell segmentation. Subsequent research has focused on improving feature representation capabilities and model efficiency. For example, the Cluster-GAT model innovatively employs a graph attention network to deeply fuse local texture features extracted by CNNs and global morphological context features captured by Vision Transformers, achieving an accuracy of 93.2% on the more challenging CRIC seven-class dataset, surpassing traditional CNN methods. Meanwhile, lightweight designs such as ICA-Res2Net, through improved coordinate attention mechanisms and multi-scale feature extraction, achieved a five-class accuracy of 98.65% on the SIPaKMeD dataset, demonstrating the potential for high precision and high efficiency.

[0005] Despite the breakthroughs in classification accuracy achieved by the aforementioned methods, they still have some limitations: Locational semantic information is not fully utilized: Existing methods treat cells as isolated image entities, failing to establish a strong correlation between hierarchical location (surface / middle / basal layer) and pathological state, thus limiting the model's interpretability and diagnostic capabilities. Static feature fusion bottleneck: Current fusion strategies are essentially fixed-weight feature combinations, unable to dynamically adjust the processing mechanism according to lesion type. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a cell image multi-task classification method and system based on position awareness and feature modulation. This method, based on position awareness feature modulation and dynamic fusion, solves the problems of insufficient utilization of position semantics and static fusion defects, and improves classification accuracy and model interpretability.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] The cell image multi-task classification method based on position awareness and feature modulation provided by this invention includes the following steps:

[0009] S1. Standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ;

[0010] S2. Based on the global visual features The data is input into a location classifier to predict the hierarchical location category of the cell.

[0011] S3. Based on the hierarchical location categories obtained in S2, two types of location-related features are generated simultaneously. One type is location semantic embedding features obtained through a learnable embedding structure. Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features; a routing network, using the global visual features as input, generates weight coefficients for expert fusion, and weights the expert features accordingly to obtain expert-enhanced features. ;

[0012] S4. The enhanced features are... and the location semantic embedding features The fusion is performed to obtain fusion features. The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

[0013] Furthermore, the modulation structure generates an intermediate feature vector based on hierarchical location categories through two layers of perceptrons. And based on this vector, channel-level modulation parameters γ and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows:

[0014]

[0015] Among them, the channel-level modulation parameters, scaling factor γ, and offset factor The calculation formulas are as follows:

[0016]

[0017]

[0018] in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

[0019] Furthermore, the routing network utilizes global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows:

[0020]

[0021]

[0022]

[0023] in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks, This is an expert feature sequence.

[0024] Furthermore, the fusion obtained in S4 include:

[0025] The enhanced features and location embedding features The features are concatenated along the channel dimension and then compressed and integrated through a fusion layer to generate fused features. ,

[0026] The lesion classifier is then input to output the final prediction result, which is a multilayer perceptron structure.

[0027] Furthermore, it also includes the following steps:

[0028] The joint loss function of location classification loss and lesion classification loss is used as the optimization objective, and the formula is as follows:

[0029]

[0030] in, For lesion classification loss, For location classification loss, α is a hyperparameter used to balance task weights, and .

[0031] The cell image multi-task classification system based on position awareness and feature modulation provided by the present invention includes a preprocessing and feature extraction module, a position prediction module, a position awareness feature enhancement module, and a classification output module;

[0032] The preprocessing and feature extraction module is used to standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ;

[0033] The location prediction module is used to predict based on the global visual features. The data is input into a location classifier to predict the hierarchical location category of the cell.

[0034] The location-aware feature enhancement module is used to simultaneously generate two types of location-related features based on the hierarchical location categories, one of which is a location semantic embedding feature obtained through a learnable embedding structure. Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features; a routing network, using the global visual features as input, generates weight coefficients for expert fusion, and weights the expert features accordingly to obtain expert-enhanced features. ;

[0035] The classification output module is used to classify the enhanced features. and the location semantic embedding features By fusion The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

[0036] Furthermore, the embedding structure in the location-aware feature enhancement module inputs cell location labels into a learnable embedding layer, mapping them into low-dimensional continuous semantic vectors, and then generates the final location embedding features through a fully connected layer and a non-linear activation function. .

[0037] Furthermore, the modulation structure generates an intermediate feature vector based on hierarchical location categories through two layers of perceptrons. And based on this vector, channel-level modulation parameters γ and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows:

[0038]

[0039] The channel-level modulation parameters, scaling factor γ, and offset factor The calculation formulas are as follows:

[0040]

[0041]

[0042] in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

[0043] Furthermore, the routing network utilizes global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows:

[0044]

[0045]

[0046]

[0047] in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks, This is an expert feature sequence.

[0048] Furthermore, the classification output module includes a fusion layer and a lesion classifier, wherein the fusion layer is used to classify the enhanced features. and location embedding features The features are concatenated along the channel dimension and then compressed and integrated to generate fused features. The lesion classifier is a multilayer perceptron structure, used to classify lesions based on... Output the predicted lesion type of the cells.

[0049] The beneficial effects of this invention are as follows:

[0050] This invention discloses a multi-task classification method and system for cell images based on location awareness and feature modulation, belonging to the field of medical image analysis. The method first utilizes a deep backbone network to extract global visual features from cellular pathology images and predict their hierarchical location (surface / middle / basal layer). Based on this, a location-aware hybrid expert module is introduced, utilizing location information through two paths: one maps location labels to semantic embedding features; the other generates channel-level modulation parameters based on location labels, performs dynamic affine transformations on the global features, and feeds the modulated features into multiple expert networks. Simultaneously, a routing network based on image content generates dynamic weights, weighted and fused the expert features to obtain enhanced features. Finally, the enhanced features are fused with the location semantic embedding features to collaboratively train lesion type classification and location classification tasks. This invention, by guiding feature modulation and expert fusion with location information, effectively improves the accuracy and interpretability of cervical cell image classification and can be used for computer-aided screening of cervical cancer. It has the following beneficial effects:

[0051] First, it enhances the ability to distinguish cells at different hierarchical levels. This invention utilizes a dual-path location modeling mechanism to transform cell hierarchical location labels into structured information that can directly participate in feature modulation and semantic fusion, enabling the model to explicitly perceive the cell's origin hierarchy. Compared to traditional methods that rely solely on visual features to infer location, this invention significantly improves the ability to differentiate between cells at different hierarchical levels.

[0052] Second, this invention enhances the model's adaptability to complex data sub-distributions. It employs a location-modulated expert network structure, guiding different experts to perform adaptive feature transformations for specific location sub-distributions through channel-level modulation parameters γ and β, and combining this with an image content-based routing network to achieve autonomous weight allocation. This design effectively handles cell samples with significant morphological differences and complex imaging conditions, while avoiding location information dominating expert selection, thus ensuring the model's ability to learn the semantic essence.

[0053] Third, it improves classification accuracy and reliability. This invention employs a multi-source feature fusion strategy to jointly model enhanced features generated by a hybrid expert module with location semantic features, guiding the model to make decisions based on sample hierarchical information. This makes the classification results more consistent with physiological structural patterns, thereby improving the accuracy and reliability of classification.

[0054] Fourth, multi-task collaborative optimization is achieved. This invention constructs a multi-task learning framework for lesion diagnosis and location recognition. Through a joint optimization strategy, positive transfer between the two tasks is achieved, enhancing the main task's focus on key regions and hierarchical differences, thereby improving the overall model performance without increasing additional annotation costs.

[0055] Fifth, it possesses excellent deployment applicability. The system proposed in this invention features high computational efficiency and strong scalability, with a lightweight expert mechanism structure, making it suitable for deployment in auxiliary diagnostic platforms or intelligent screening devices, and showing promising clinical application prospects.

[0056] The above and other objects, advantages, and features of the present invention will be more fully set forth and demonstrated through the following detailed description of specific embodiments in conjunction with the accompanying drawings. Those skilled in the art, upon referring to the following detailed description and the accompanying drawings, will be able to better understand and realize the above advantages of the present invention. Other objects, features, and advantages of the present invention will become clearer after being described in detail in the detailed description section in conjunction with the accompanying drawings. Attached Figure Description

[0057] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following drawings are provided for illustration.

[0058] Figure 1 This is a flowchart of a cell image multi-task classification method based on position awareness and feature modulation.

[0059] Figure 2 This is a schematic diagram of a cell image multi-task classification system based on position awareness and feature modulation.

[0060] Figure 3 This is an architecture diagram of a cell image multi-task classification method based on position awareness and feature modulation.

[0061] Figure 4 This is a processed image of a cervical smear. Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0063] Example 1

[0064] like Figure 1 As shown, Figure 1 The flowchart below illustrates a multi-task classification method for cell images based on position awareness and feature modulation. This embodiment provides a multi-task classification method for cell images based on position awareness and feature modulation, which specifically includes the following steps:

[0065] S1. Standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ;

[0066] S2. Based on the global visual features The data is input into a location classifier to predict the hierarchical location category of the cell.

[0067] S3. Based on the hierarchical location categories obtained in S2, two types of location-related features are generated simultaneously.

[0068] One type obtains location semantic embedding features through learnable embedding structures. ,

[0069] Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features;

[0070] The routing network, taking the global visual features as input, generates weight coefficients for expert fusion, and performs weighted fusion of the expert features accordingly to obtain expert-enhanced features. ;

[0071] S4. The enhanced features are... and the location semantic embedding features The fusion is performed to obtain fusion features. The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

[0072] In this embodiment, the embedding structure in S3 inputs cell location labels into a learnable embedding layer, mapping them to low-dimensional continuous semantic vectors, and then generates the final location embedding features through a fully connected layer and a non-linear activation function. .

[0073] The modulation structure described in this embodiment is based on hierarchical position categories and generates intermediate feature vectors through two layers of perceptrons. And based on this vector, channel-level modulation parameters are generated. and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows:

[0074]

[0075] The channel-level modulation parameters, scaling factor γ, and offset factor described in this embodiment are... The calculation formulas are as follows:

[0076]

[0077]

[0078] in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

[0079] The multiple expert networks described in this embodiment are feedforward neural networks with independent parameters, and each expert modulates the input features. Extract features from different semantic subspaces to form expert feature sequences. .

[0080] The routing network described in this embodiment uses global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows:

[0081]

[0082]

[0083]

[0084] in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks.

[0085] The fusion obtained in S4 of this embodiment Includes: the enhanced features and location embedding features The features are concatenated along the channel dimension and then compressed and integrated through a fusion layer to generate fused features. The result is then input into a lesion classifier to output the final prediction result. The lesion classifier is a multilayer perceptron structure.

[0086] The method described in this embodiment uses a joint loss function of location classification loss and lesion classification loss as the optimization objective, as shown in the following formula:

[0087]

[0088] in, For lesion classification loss, For location classification loss, α is a hyperparameter used to balance task weights, and .

[0089] like Figure 2As shown, the cell image multi-task classification system based on position awareness and feature modulation provided in this embodiment includes a preprocessing and feature extraction module, a position prediction module, a position awareness feature enhancement module, and a classification output module.

[0090] The preprocessing and feature extraction module is used to standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ;

[0091] The location prediction module is used to predict based on the global visual features. The data is input into a location classifier to predict the hierarchical location category of the cell.

[0092] The location-aware feature enhancement module is used to simultaneously generate two types of location-related features based on the hierarchical location categories, one of which is a location semantic embedding feature obtained through a learnable embedding structure. Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features. Furthermore, a routing network, using the global visual features as input, generates weight coefficients for expert fusion, and weights the expert features accordingly to obtain expert-enhanced features. ;

[0093] The classification output module is used to classify the enhanced features. and the location semantic embedding features By fusion The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

[0094] In this embodiment, the embedding structure in the location-aware feature enhancement module inputs cell location labels into a learnable embedding layer, mapping them into low-dimensional continuous semantic vectors. The final location embedding features are then generated through a fully connected layer and a non-linear activation function. .

[0095] The modulation structure described in this embodiment is based on hierarchical position categories and generates intermediate feature vectors through two layers of perceptrons. And based on this vector, channel-level modulation parameters γ and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows:

[0096]

[0097] The channel-level modulation parameters, scaling factor γ, and offset factor described in this embodiment are... The calculation formulas are as follows:

[0098]

[0099]

[0100] in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

[0101] The multiple expert networks described in this embodiment are feedforward neural networks with independent parameters, and each expert modulates the input features. Extract features from different semantic subspaces to form expert feature sequences. .

[0102] The routing network described in this embodiment uses global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows:

[0103]

[0104]

[0105]

[0106] in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks.

[0107] The classification output module described in this embodiment includes a fusion layer and a lesion classifier. The fusion layer is used to process the enhanced features. and location embedding features The features are concatenated along the channel dimension and then compressed and integrated to generate fused features. The lesion classifier is a multilayer perceptron structure, used to classify lesions based on... Output the predicted lesion type of the cells.

[0108] The system described in this embodiment uses a joint loss function of location classification loss and lesion classification loss as the optimization objective, as shown in the following formula:

[0109]

[0110] in, For lesion classification loss, For location classification loss, α is a hyperparameter used to balance task weights, and .

[0111] Example 2

[0112] This embodiment provides a multi-task classification method for cell images based on position awareness and feature modulation. The cell images in this embodiment use cervical smear images as cytopathological images for analysis and processing. Specifically, the method includes the following sequentially executed steps:

[0113] S1. Image Preprocessing and Global Feature Extraction

[0114] The input cytopathological images are standardized, including image size normalization, color channel normalization, and noise suppression, to ensure consistency and stability of the images within the deep network. Image size normalization adjusts the input images to a fixed size of 224×224 pixels. Color channel normalization uses the mean and standard deviation from the ImageNet dataset to normalize pixel values. Specifically, it performs the following operations on the RGB channels: R channel: reduce the mean by 0.485 and divide by the standard deviation of 0.229; B channel: reduce the mean by 0.456 and divide by the standard deviation of 0.224; B channel: reduce the mean by 0.406 and divide by the standard deviation of 0.225. Noise suppression filters out high-frequency noise using Gaussian blur (kernel size 5, standard deviation range 0.1~2.0), while random erasure (erasure probability 0.3, erased region scale 0.02~0.1, aspect ratio 0.3~3.3) enhances the model's robustness to local noise. The preprocessed image is input into a deep backbone network to extract global visual features (Fglobal). The backbone network preferentially adopts the Swing Transformer, which efficiently captures global semantic information and local detail features of the image through window attention mechanism and hierarchical feature fusion, providing high-dimensional and strong representational basic features for subsequent location prediction and lesion classification.

[0115] S2. Hierarchical Location Category Prediction

[0116] Based on the global visual features The data is input to a location classifier to predict the hierarchical location category of cells. The location classifier consists of a feedforward network composed of linear layers and nonlinear activation functions. The nonlinear activation function uses Gaussian Error Linear Unit (GELU), which combines the sparsity of ReLU with the smoothness of Gaussian distribution, effectively alleviating the gradient vanishing problem and improving the expressive power of feature mapping. The specific structure of the location classifier is as follows: first, layer normalization is performed on the global visual features to stabilize the feature distribution; then, the feature dimension is mapped to 3D through a fully connected linear layer, corresponding to three types of location labels: surface, middle, and basal layers. Finally, the mapping from global visual features to discrete location labels is realized, and one location label from the surface, middle, or basal layer is output.

[0117] S3. Location-Aware Feature Generation and Expert Enhancement

[0118] Based on the hierarchical location categories predicted by S2, two types of location-related features are constructed simultaneously:

[0119] (1) Location semantic embedding features Cell location labels are input into a learnable embedding layer, mapped to low-dimensional continuous semantic vectors, and then the final location embedding features are generated through a fully connected layer and a non-linear activation function. It is used to capture histological differences between cells at different levels.

[0120] (2) Modulation characteristics Enhanced features with experts The hierarchical position category is represented as a one-hot vector, and an intermediate feature vector is generated through two layers of perceptrons. The two-layer perceptron has the following structure: the input layer has a dimension of 3, the hidden layer has a dimension of 512, and the output layer has a dimension equal to that of the global visual features. The number of channels is consistent; both perceptron layers use GELU as the activation function, and a Dropout layer is added after the hidden layer to prevent overfitting. Channel-level modulation parameters are generated based on this vector. and The formula is as follows:

[0121]

[0122]

[0123] in, , This is the weight matrix for the corresponding linear layer; , Both the input dimension and the output dimension are 1024, which is consistent with global visual features. The number of channels remains the same. The intermediate vector output by the two-layer perceptron has a dimension of 1024.

[0124] Using the above parameters to analyze global visual features Channel-by-channel scaling and offset are performed to form modulation features. The formula is as follows:

[0125]

[0126] Here, the symbol ⊙ represents the element-wise multiplication operation of each channel, representing global visual features. The dimension is batch size × 1024 × 7 × 7, and the dimensions of modulation parameters γ and β are both batch size × 1024 × 1 × 1. Channel-by-channel operation is achieved through a broadcast mechanism.

[0127] Modulation characteristics The input is fed into multiple expert networks, with N being 3 in number. Each expert network is a feedforward neural network with independent parameters. The specific structure is as follows: the input layer has a dimension of 1024, followed by a normalization layer, a Gaussian error linear unit activation function, a first linear layer, another Gaussian error linear unit activation function, a random deactivation layer with a dropout probability of 0.1, and a second linear layer. The first linear layer has a dimension of 2048, and the second linear layer has a dimension of 1024. Each expert network extracts different semantic subspace features, forming an expert feature sequence. The sequence has dimensions of N × batch size × 1024 × 7 × 7.

[0128] With global visual features For the input routing network, expert weight vectors are generated through linear mapping and nonlinear transformation. The final weight coefficients for expert fusion are obtained through a normalization function. The formula is as follows:

[0129]

[0130]

[0131]

[0132] in, , For learnable routing parameters, Wr has an input dimension of 1024 and an output dimension of 4. For a bias term with dimension 4; ( () represents the Gaussian error linear unit activation function. The expert enhancement features are obtained through the above calculations. Its dimensions are batch size × 1024 × 7 × 7, and global visual features. The dimensions remain consistent.

[0133] S4. Feature fusion and lesion classification enhance expert features. Location semantic embedding features The features are stitched together along the channel dimension, and then compressed and integrated through a fusion layer to generate fused features. The fusion layer is a single-layer 1×1 convolutional layer, with the following structural details: the convolutional kernel size is 1×1, the stride is 1, the padding value is 0, and the number of input channels is 1280. Channel number 1024 and The total dimension after concatenation is 256 channels, and the output channel count is 512. After convolution, the GELU activation function and batch normalization layer are connected in sequence to achieve feature dimension compression and distribution stability.

[0134] The fused features are then compressed into one-dimensional features using global average pooling and input into the lesion type classifier. The lesion classifier is a three-layer multilayer perceptron structure with the following parameters: 512 nodes in the input layer (dimension after pooling of fused features), 1024 nodes in the first hidden layer, 512 nodes in the second hidden layer, and 5 nodes in the output layer, corresponding to five lesion types: NILM, ASC-US, LSIL, ASC-H, and HSIL. The activation function is uniformly GELU, and a random deactivation layer with a dropout probability of 0.1 is configured after each hidden layer to prevent overfitting. The output layer has no activation function and directly outputs the predicted probability distribution of each type of lesion, ultimately outputting the predicted lesion type of the cell.

[0135] The joint loss optimization method uses the joint loss function of location classification loss and lesion classification loss as the optimization objective, as shown in the following formula:

[0136]

[0137] in, For lesion classification loss, the cross-entropy loss function is adopted to optimize the probability distribution of multi-classification tasks for five types of lesions. For location classification loss, the cross-entropy loss function is also adopted, adapting to classification tasks for surface, middle, and basal layers. α is a hyperparameter used to balance task weights, and α∈[0,1], with a value of 0.7 in this invention. Joint optimization improves the efficiency of collaborative information utilization among multiple tasks, enhancing the accuracy and generalization ability of lesion recognition.

[0138] Example 3

[0139] This embodiment further illustrates the method with specific illustrations and implementation details.

[0140] like Figure 3 As shown, Figure 3This paper presents an architecture diagram of a multi-task cell image classification method based on location awareness and feature modulation. The core idea of ​​this architecture is: after extracting cell image features, first predict the cell's hierarchical location (surface / middle / bottom layer), then use this location information to enhance the original features through two different paths, and finally combine the enhanced features with location semantics for the final lesion classification. Details are as follows:

[0141] 1. Input and Feature Extraction

[0142] Input image: raw cell pathology image.

[0143] Backbone feature extraction: Use deep neural networks (such as Swing Transformer) to extract global features of the image.

[0144] 2. Location hierarchy classification

[0145] Location-level classification: Global features are input into a classifier (composed of LayerNorm and Linear layers) to predict the level to which the cell belongs.

[0146] Hierarchical label: The output result is a specific category from "Top", "Middle", and "Bottom".

[0147] 3. Utilization of dual-path location information

[0148] After obtaining the hierarchical labels, the architecture transforms them into two forms of information, which are then fed into subsequent networks in parallel:

[0149] Path 1: Semantic Embedding

[0150] Embedding: Maps discrete hierarchical labels (such as "middle layer") into a dense, continuous vector through a learnable embedding layer.

[0151] Linear -> Gelu -> Dropout: This process performs further nonlinear transformations and regularization on the embedding vectors to generate the final embedding features. This path primarily provides high-level semantic information about the location.

[0152] Path 2: Feature Modulation and Expert Fusion

[0153] One-hot vector: Converts hierarchical labels into one-hot encoded form.

[0154] Parameter generation: Based on the one-hot vector, specific modulation parameters are generated through a small network.

[0155] Feature modulation: The generated parameters are used to perform a channel-level affine transformation on the original global features to obtain the modulated features Fmod. Therefore, the original features carry positional information.

[0156] Expert Networks: The modulation features Fmod are input in parallel into multiple expert networks (experts 1, 2, and 3) with the same structure but independent parameters, allowing them to extract features from different perspectives.

[0157] Routing: Simultaneously, the original global features are input into the routing network. Based on the image content itself, the routing network calculates how to best fuse the outputs of the three experts (i.e., generate weighted weights).

[0158] Enhanced features: Based on the weights calculated by the routing network, the outputs of the three experts are weighted and summed to obtain the final enhanced features.

[0159] 4. Feature Fusion and Classification

[0160] Concatenation and fusion: The embedded features from path one and the enhanced features from path two are concatenated along the channel dimension to achieve the fusion of high-level semantics and enhanced visual features.

[0161] LayerNorm -> Linear: Normalizes and linearly transforms the fused features for the final classification task.

[0162] Lesion type classification: The final lesion type prediction result of the output cells (such as normal, ASC-US, etc.).

[0163] like Figure 4 As shown, Figure 4 The image processing effect diagram of cervical smear provided in the embodiment of the present invention is shown in this embodiment. The visualization results of three sets of representative cell samples intuitively demonstrate the gradual evolution of the method in the process of feature extraction, position awareness and dynamic modulation. They are the original cell pathological image, the global visual feature extraction result, the position awareness prediction result and the modulation effect of the position awareness hybrid expert module on the features.

[0164] Figure 4 (a) shows the original cytopathological image after standardization, with the image size uniformly adjusted to 224×224 pixels, which was used as the input to the deep backbone network.

[0165] Figure 4 (b) in the image represents the global visual features extracted by the Swing Transformer backbone network. The heatmap visualization shows that brighter areas represent visual semantic regions with stronger network responses, reflecting the backbone network's initial attention to the cell nucleus, cytoplasm, and background. Differences in heatmap distribution among different samples indicate that the backbone network can effectively capture the diversity of cell morphology.

[0166] Figure 4In the figure, (c) represents the hierarchical location prediction result output by the location classifier. The percentage values ​​marked in the figure (such as 35.91%, 26%, 18%, etc.) are the confidence scores of the model in that the current cell sample belongs to a specific hierarchical location (such as the surface, middle, or basal layer). This prediction result corresponds to step S2 of the method and provides key hierarchical location prior information for subsequent feature modulation.

[0167] Figure 4 (d) in the figure is a heatmap showing the effect of the location-aware hybrid expert module modulating global visual features. The red areas represent the enhanced features, and the blue areas represent the suppressed features. This visualization corresponds to the feature modulation in step S3 of the method. Enhanced features with experts The generation process.

[0168] By comparing the modulation patterns among different samples, it can be seen that the model depends on the predicted location category. Figure 4 (c) applies differentiated channel-level modulation strategies to different cells, achieving dynamic adjustment of features according to location.

[0169] For example, the nucleus region was significantly enhanced in some samples (red), while background noise was effectively suppressed in others (blue). This diversity fully demonstrates the effectiveness of the proposed position-aware modulation mechanism, that is, the model can adaptively adjust the feature expression according to hierarchical position information, thereby providing more discriminative feature input for subsequent lesion classification.

[0170] Figure 4 In the diagram, (e) represents the enhanced feature map generated after fusing the location embedding features and the modulated visual features. This corresponds to step S4 in the method where the enhanced features are... Location semantic embedding features The fusion process is described. Visualization results show that the fused feature map, while retaining key morphological information, further strengthens discriminative regions related to hierarchical location, making the feature representation more semantically rich. Differences in the fused feature maps among different samples indicate that positional semantic information successfully guides visual features towards a more diagnostically valuable focus.

[0171] Figure 4 In the figure, (f) represents the final prediction result output by the lesion classifier. The figure shows the model's confidence score for each lesion type of the current cell sample in the form of a probability distribution. This corresponds to the feature fusion step in method S4. The lesion type prediction is output after inputting into the lesion classifier. Combining the complete processing flow of the first three sets of samples, it can be seen that the model, based on cell morphology characteristics and hierarchical location information, ultimately provides a classification decision that conforms to pathological principles.

[0172] In summary, Figure 4 The core working mechanism of this method is clearly revealed through visualization: from the original image to global feature extraction, then to location-aware prediction, and finally to location-based dynamic feature modulation and enhancement. This not only verifies the feasibility and effectiveness of the method but also significantly improves the interpretability of the model, enabling medical experts to intuitively understand the model's decision-making basis and providing more reliable technical support for clinical auxiliary diagnosis.

[0173] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A cell image multi-task classification method based on position awareness and feature modulation, characterized in that, Includes the following steps: S1. Standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ; S2. Based on the global visual features The data is input into a location classifier to predict the hierarchical location category of the cell. S3. Based on the hierarchical location categories obtained in S2, two types of location-related features are generated simultaneously. One type is location semantic embedding features obtained through a learnable embedding structure. Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features; a routing network, using the global visual features as input, generates weight coefficients for expert fusion, and weights the expert features accordingly to obtain expert-enhanced features. ; S4. The enhanced features are... and the location semantic embedding features The fusion is performed to obtain fusion features. The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

2. The cell image multi-task classification method based on position awareness and feature modulation as described in claim 1, characterized in that, The modulation structure is based on hierarchical location categories and generates intermediate feature vectors through two layers of perceptrons. And based on this vector, channel-level modulation parameters γ and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows: Among them, the channel-level modulation parameters, scaling factor γ, and offset factor The calculation formulas are as follows: in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

3. The cell image multi-task classification method based on position awareness and feature modulation as described in claim 1, characterized in that, The routing network has global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows: in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks, This is an expert feature sequence.

4. The cell image multi-task classification method based on position awareness and feature modulation as described in claim 1, characterized in that, The fusion obtained in S4 include: The enhanced features and location embedding features The features are concatenated along the channel dimension and then compressed and integrated through a fusion layer to generate fused features. , The lesion classifier is then input to output the final prediction result, which is a multilayer perceptron structure.

5. The cell image multi-task classification method based on position awareness and feature modulation as described in claim 1, characterized in that, It also includes the following steps: The joint loss function of location classification loss and lesion classification loss is used as the optimization objective, and the formula is as follows: in, For lesion classification loss, For location classification loss, α is a hyperparameter used to balance task weights, and .

6. A cell image multi-task classification system based on position awareness and feature modulation, characterized in that, It includes a preprocessing and feature extraction module, a location prediction module, a location-aware feature enhancement module, and a classification output module; The preprocessing and feature extraction module is used to standardize the input cytopathological images and extract global visual features of the images using a deep backbone network. ; The location prediction module is used to predict based on the global visual features. The data is input into a location classifier to predict the hierarchical location category of the cell. The location-aware feature enhancement module is used to simultaneously generate two types of location-related features based on the hierarchical location categories, one of which is a location semantic embedding feature obtained through a learnable embedding structure. Another type generates modulated features by performing channel-level modulation of global visual features through modulation structures. The modulation features are then input into multiple expert networks to obtain expert features; a routing network, using the global visual features as input, generates weight coefficients for expert fusion, and weights the expert features accordingly to obtain expert-enhanced features. ; The classification output module is used to classify the enhanced features. and the location semantic embedding features By fusion The data is then input into the lesion type classifier, which outputs the predicted lesion type of the cell.

7. The cell image multi-task classification system based on position awareness and feature modulation as described in claim 6, characterized in that, The embedding structure in the location-aware feature enhancement module inputs cell location labels into a learnable embedding layer, mapping them into low-dimensional continuous semantic vectors. The final location embedding features are then generated through a fully connected layer and a non-linear activation function. .

8. The cell image multi-task classification system based on position awareness and feature modulation as described in claim 6, characterized in that, The modulation structure is based on hierarchical location categories and generates intermediate feature vectors through two layers of perceptrons. And based on this vector, channel-level modulation parameters γ and Used for the global visual features Channel-by-channel scaling and offset are performed to generate modulation features. The formula is as follows: The channel-level modulation parameters, scaling factor γ, and offset factor The calculation formulas are as follows: in, , This is the weight matrix for the corresponding linear layer. This is the intermediate vector output by the two layers of perceptrons.

9. The cell image multi-task classification system based on position awareness and feature modulation as described in claim 6, characterized in that, The routing network has global visual features. Perform linear mapping and nonlinear transformation to generate expert weight vectors. The final weight coefficients for expert fusion are obtained through a normalization function. This allows for a weighted combination of outputs from different experts, ultimately forming enhanced features. The formula is as follows: in, , For learnable routing parameters, ( ) represents a non-linear activation function. For the number of expert networks, This is an expert feature sequence.

10. The cell image multi-task classification system based on position awareness and feature modulation as described in claim 6, characterized in that, The classification output module includes a fusion layer and a lesion classifier. The fusion layer is used to process the enhanced features. and location embedding features The features are concatenated along the channel dimension and then compressed and integrated to generate fused features. The lesion classifier is a multilayer perceptron structure, used to classify lesions based on... Output the predicted lesion type of the cells.

Images