Parotid gland segmentation and classification method and device based on heterogeneous perception mapping network and medium

Abstract

The application discloses a parotid gland segmentation and classification method based on a heterogeneous perception mapping network, a device and a medium. The method acquires a parotid gland ultrasound image after preprocessing and inputs the image into a heterogeneous perception mapping network model. The model is a multi-task model composed of a shared encoder, a segmentation decoder and a classification module, and simultaneously performs segmentation and classification tasks. The shared encoder cascades two types of encoding stages, cooperates with a symmetrical topological segmentation decoder to realize progressive upsampling and feature fusion, and achieves sub-pixel level accurate segmentation. The classification module is directly connected to the output end of the encoder, performs spatial dissection constraint based on deep semantic features to focus on lesions, a third type of encoding stage is added at the end of the encoder to provide high-quality classification features, and a parallel double-branch heterogeneous multi-scale context fusion module is built-in the classification module to enhance the classification robustness of morphologically similar lesions. The application takes into account the segmentation and classification accuracy, realizes model lightweight, and adapts to the real-time inference requirements of primary terminals.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and artificial intelligence technology, specifically to a parotid gland segmentation and classification method, device, and medium based on heterogeneous perceptual mapping networks. Background Technology

[0002] As the largest salivary gland in the human body, the parotid gland requires precise preoperative diagnosis and anatomical localization of tumors for oral and maxillofacial surgery. This is crucial for developing surgical plans, assessing the extent of tumor invasion, and protecting facial nerve function. The parotid gland itself has a highly irregular morphology, and its deep lobe region exhibits low contrast with adjacent soft tissues such as the masseter muscle and parapharyngeal space in medical imaging such as ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI), resulting in significant ambiguity in anatomical boundaries. Furthermore, common clinical conditions such as pleomorphic adenoma (PA) and warthin tumor (WT) share significant "same image, different disease" characteristics on imaging, meaning high textural overlap, making differentiation based solely on imaging features extremely challenging. Therefore, achieving high-precision pixel-level segmentation and simultaneous multi-category intelligent triage of parotid lesions has become a core technical challenge in the field of computer-aided diagnosis.

[0003] Furthermore, compared to examination methods such as CT and MRI, parotid gland ultrasound examination presents both unique challenges and advantages in clinical application. From a technical perspective, ultrasound imaging has a relatively limited field of view and inconsistent scanning sections, leading to variable nodule morphology and boundaries. Simultaneously, ultrasound images are easily affected by probe handling techniques, patient positioning, and tissue obstruction, significantly increasing the difficulty of lesion feature extraction, accurate classification, and contour segmentation. From a clinical application perspective, ultrasound examination poses no risk of ionizing radiation, the equipment is portable and flexible, adaptable to the needs of multiple follow-up visits and large-scale clinical screening, and can clearly capture detailed information such as nodule echo, morphology, and blood flow, effectively determining the nature of the nodules. Therefore, it possesses extremely high clinical practical value in the early screening and preliminary diagnosis of parotid gland nodules.

[0004] In recent years, medical image analysis technology, represented by deep learning, has made significant progress. In particular, convolutional neural networks, represented by UNet and its variants, have been widely used in medical image segmentation and classification tasks. Existing technologies typically employ two technical approaches: one is to improve the encoding and decoding structure of UNet by introducing attention mechanisms or residual connections to enhance feature extraction capabilities and improve general segmentation accuracy; the other is to design multi-task learning frameworks that attempt to output segmentation masks and classification results simultaneously through simple parallel branches at the network ends.

[0005] However, in practical applications targeting the parotid gland, a specific anatomical structure, the aforementioned existing technical solutions still have the following significant technical shortcomings, making it difficult to meet the clinical needs for high-precision and high-efficiency diagnosis and treatment:

[0006] First, existing feature extraction mechanisms lack the ability to adaptively model complex geometric shapes, resulting in insufficient segmentation accuracy. Existing UNet variants and their attention mechanisms, when dealing with the non-convex, multi-branched geometry of the parotid gland, primarily rely on local receptive fields or global context compression for feature capture, lacking the ability to adaptively remap the spatial structure of the target boundary. Especially in regions with extremely low contrast, such as the deep lobes of the gland, existing general-purpose models struggle to effectively distinguish the glandular parenchyma from adjacent muscle or adipose tissue, often resulting in boundary overflow or undersegmentation in the output segmentation results. This pixel-level error prevents the generated segmentation mask from meeting the spatial accuracy requirements for surgical planning, limiting its clinical value in accurately defining the extent of tumor invasion.

[0007] Second, existing architectures handle segmentation and classification tasks in isolation, making classification accuracy highly susceptible to interference from background noise. Most existing studies treat "lesion segmentation" and "attribute classification" as two weakly correlated independent processes, or only perform simple parallel task branches at the network's end. This fragmented design prevents the classification branch from obtaining precise spatial anatomical constraints during the feature learning phase. When faced with complex background noise such as oral metal artifacts, cervical lymph nodes, and surrounding normal tissue, the classification branch easily extracts interfering features from non-lesion areas, severely impacting the accuracy of differentiating between morphologically similar benign and malignant lesions such as PA and WT. In short, existing technologies fail to effectively utilize the fine spatial information provided by segmentation results to guide classification decisions, resulting in poor robustness of the model in complex clinical imaging environments.

[0008] Third, existing global perception modules suffer from excessively high computational complexity, making real-time inference and deployment difficult. To address the context modeling problem, some existing technologies have introduced global perception modules such as Transformers or large-kernel convolutions. However, when processing high-resolution medical image sequences, the computational complexity of these modules often increases exponentially with the input pixel size (e.g., the quadratic complexity of self-attention). This high computational cost not only leads to excessively long model training and inference times, but more importantly, it makes it difficult for models to achieve real-time or near-real-time inference analysis on computationally limited primary healthcare terminals or personal computers, severely hindering the widespread application of advanced AI-assisted diagnostic technologies in clinical scenarios. Summary of the Invention

[0009] The technical problem this invention aims to solve is to provide a method that uses a first type of encoding stage (employing a dual convolutional module to extract local detail features) and a second type of encoding stage (employing a dual self-attention module with an embedded multi-level attention mechanism to extract global contextual features with linear complexity) cascaded in a shared encoder, combined with a symmetrically topologically connected segmentation decoder for progressive upsampling and feature fusion, to achieve adaptive modeling and sub-pixel-level accurate segmentation of the complex geometry of the parotid gland. Furthermore, by directly connecting the input of the classification module to the output of the shared encoder, the classification module applies spatial anatomical constraints based on deep semantic feature maps to focus on the lesion region. Furthermore, a third type of encoding stage (embedded with a dual self-attention module) is added at the end of the encoder to provide high-quality feature sources for the classification task. At the same time, a parallel dual-branch heterogeneous multi-scale context fusion module is set up inside the classification module (using a small-sized convolutional kernel to extract local texture details and a large-sized convolutional kernel to extract global context information, and the two are spliced ​​and fused along the channel) to achieve highly robust classification of morphologically similar lesions. Under the premise of ensuring segmentation accuracy and classification accuracy, the overall computational load is reduced, so that the model can adapt to the real-time inference needs of primary medical terminals. This is a parotid gland segmentation and classification method, equipment and medium based on heterogeneous perception mapping network.

[0010] To address the aforementioned technical problems, the present invention provides a parotid gland segmentation and classification method based on a heterogeneous perceptual mapping network, comprising:

[0011] Acquire the parotid gland ultrasound image data to be processed, and preprocess it to obtain the standard input image;

[0012] The standard input image is input into a heterogeneous perceptual mapping network model for integrated analysis to obtain an output feature map; the integrated analysis includes segmentation and classification tasks executed simultaneously, and the output feature map includes a segmentation mask and classification results;

[0013] The heterogeneous sensing mapping network model is a multi-task model trained based on sample data and corresponding annotations, including:

[0014] A shared encoder is used to extract multi-scale features from the standard input image and generate a deep semantic feature map. The shared encoder includes at least a cascaded first type of encoding stage and a second type of encoding stage. The first type of encoding stage uses a dual convolutional module and a pooling layer to extract local detail features, and the second type of encoding stage uses a dual self-attention module and a pooling layer to extract global context features.

[0015] A segmentation decoder, which is symmetrically connected to the shared encoder, is used to progressively upsample and fuse the deep semantic feature map to restore the spatial resolution of the standard input image and output the segmentation mask.

[0016] A classification module, whose input is connected to the output of the shared encoder, is used to perform spatial anatomical constraints based on the deep semantic feature map to focus on the lesion region for classification and output the classification result.

[0017] In a preferred embodiment, the first type of encoding stage includes a plurality of sequentially connected dual convolutional coding layers, each of the dual convolutional coding layers including a dual convolutional module and a max pooling downsampling module whose input is connected to the output of the dual convolutional module;

[0018] The dual convolution module includes a 3×3 convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 depthwise separable convolutional layer, a layer normalization layer, and a ReLU activation function layer connected in sequence.

[0019] In a preferred embodiment, the segmentation decoder is provided with inverse double convolution decoding layers corresponding one-to-one with the double convolution coding layers. Each of the inverse double convolution decoding layers includes a transposed convolution module and an inverse double convolution module whose input is connected to the output of the transposed convolution module.

[0020] The inverse double convolution module includes a 3×3 depth-separable convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 convolutional layer, a layer normalization layer, and a ReLU activation function layer connected in sequence.

[0021] The output of the dual convolution module in the dual convolution coding layer is connected to the input of the corresponding inverse dual convolution decoding layer.

[0022] In a preferred embodiment, the second type of encoding stage includes several sequentially connected double convolutional self-attention encoding layers, each of which includes a double self-attention module, a double convolution module, and a max pooling downsampling module connected in sequence.

[0023] The segmentation decoder is provided with an inverse double convolutional self-attention decoding layer that corresponds one-to-one with the double convolutional self-attention coding layer. The inverse double convolutional self-attention decoding layer includes a transposed convolutional module, an inverse double convolutional module, and a double self-attention module connected in sequence.

[0024] The output of the dual self-attention module in the dual convolutional self-attention coding layer is connected to the input of the inverse dual convolution module in the corresponding inverse dual convolutional self-attention decoding layer.

[0025] In a preferred embodiment, the dual self-attention module extracts features using the following steps:

[0026] Perform layer normalization on the input feature map;

[0027] The layer-normalized feature map is input into a multi-level spatial attention module to obtain the first intermediate feature;

[0028] The first intermediate feature is concatenated with the input feature map through channels, and after layer normalization, it is input into the global feature gate perception module. The local context information is encoded by depthwise separable convolution and the feature response is adaptively modulated by the gate unit to obtain the second intermediate feature.

[0029] The second intermediate feature and the first intermediate feature are concatenated by channels, and after layer normalization, they are input into a multi-level channel attention module. The channel dimension is modeled based on the transposed attention mechanism to obtain the third intermediate feature.

[0030] The third intermediate feature is concatenated with the second intermediate feature, and after layer normalization, it is input again into the global feature gating perception module for feature modulation to obtain the fourth intermediate feature.

[0031] The fourth intermediate feature and the third intermediate feature are concatenated to output the final dual self-attention feature map.

[0032] In a preferred embodiment, the step of inputting the layer-normalized feature map into a multi-level spatial attention module to obtain the first intermediate feature is implemented through the following steps:

[0033] The layer-normalized feature map is obtained by linear projection. Mapped to query features Key features Value characteristics ;

[0034] The query features Key features Value characteristics Divided into multiple feature groups according to channel dimension ;

[0035] For the key features and value features in each feature group, multi-scale feature extraction is performed using depthwise separable convolutions with different kernel sizes. After batch normalization and ReLU activation function processing, the enhanced key features and enhanced value features are output.

[0036] The query features of each feature group, as well as the enhanced key features and enhanced value features, are concatenated along the channel dimension to obtain the aggregated query features, aggregated key features, and aggregated value features.

[0037] The Softmax function is applied to the aggregated query features and aggregated key features respectively. Attention weights are obtained by calculating attention weights using the dot product attention method. The attention weights are then multiplied by the aggregated value features to obtain the first intermediate feature.

[0038] In a preferred embodiment, the shared encoder further includes a third type of encoding stage, with its input end connected to the output end of the second type of encoding stage and its output end connected to the input end of the segment decoder.

[0039] The third type of encoding stage includes a dual attention module.

[0040] In a preferred embodiment, the classification module includes a heterogeneous multi-scale context fusion module connected to the output of the third type of encoding stage, and a classifier connected to the output of the heterogeneous multi-scale context fusion module.

[0041] The heterogeneous multi-scale context fusion module includes:

[0042] The first convolutional branch, which uses a convolutional kernel of the first size, is used to extract local detail features of the input features;

[0043] The second convolutional branch, which uses a convolutional kernel of a second size larger than the first size, is used to extract global contextual features of the input features.

[0044] The first convolutional branch and the second convolutional branch are set in parallel, and their outputs are spliced ​​and fused along the channel dimension;

[0045] The classifier includes:

[0046] The feature compression unit is used to convert the fused feature map output by the heterogeneous multi-scale context fusion module into a feature vector of fixed dimension.

[0047] The fully connected classification unit, connected to the output of the feature compression unit, consists of multiple alternating linear layers, normalization layers, activation function layers, and Dropout layers. It is used to progressively compress the dimensionality and semantically map the feature vector, and output the corresponding classification result.

[0048] The present invention also provides an apparatus including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods as described in any of the preceding claims.

[0049] The present invention also provides a storage medium including a computer program / instructions that, when executed by a processor, implement the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network as described above.

[0050] The parotid gland segmentation and classification method, device, and medium based on heterogeneous perceptual mapping networks of the present invention have the following advantages compared with the prior art:

[0051] (1) The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention includes acquiring parotid gland ultrasound image data to be processed, preprocessing to obtain a standard input image, and preprocessing the original parotid gland medical image to eliminate differences in image size, grayscale range, etc. caused by different devices and different scanning parameters, thereby obtaining a standard input image of uniform specifications, laying a data foundation for the stable feature extraction of the subsequent deep learning model, and improving the model's generalization ability to image data from different sources. The standard input image is input into the heterogeneous perceptual mapping network model for integrated analysis to obtain an output feature map; the integrated analysis includes segmentation and classification tasks executed synchronously, and the output feature map includes a segmentation mask and classification results. Unlike the existing technology that treats segmentation and classification as two independent processes or simply parallel processes at the end, this technology places the segmentation task and classification task under the same network framework for integrated analysis, enabling the two tasks to achieve deep coupling in the feature learning stage. This synchronous execution design allows the spatial structure information learned by the segmentation task to implicitly assist the classification task, while the semantic discrimination information learned by the classification task can also guide the segmentation task. The two promote each other and significantly improve the overall diagnostic performance compared with the existing isolated processing methods.

[0052] The heterogeneous perceptual mapping network model is a multi-task model trained on sample data and corresponding annotations, comprising a shared encoder, a segmentation decoder, and a classification module. The shared encoder performs multi-scale feature extraction on the standard input image, generating a deep semantic feature map. By setting the shared encoder as the common feature extraction backbone network for both segmentation and classification tasks, it avoids the parameter redundancy and computational waste associated with building independent feature extraction networks for different tasks. This sharing mechanism allows the features learned by the encoder to serve both segmentation and classification tasks simultaneously, improving feature utilization efficiency while reducing the overall model's computational overhead, thus creating conditions for subsequent deployment in computationally constrained environments.

[0053] The shared encoder includes at least a cascaded first-type encoding stage and a second-type encoding stage. The first-type encoding stage uses a combination of dual convolutional modules and pooling layers to extract local detail features. By using a combination of dual convolutional modules and pooling layers in the first-type encoding stage, the local perception characteristics of convolutional operations are utilized to efficiently extract shallow local detail features such as edges and textures of the parotid gland image. The stacked design of dual convolutions enhances the network's ability to express local structures, providing a rich foundation of detailed information for the construction of subsequent deep semantic features, which helps to improve the localization accuracy of gland boundaries. The second type of encoding stage employs a dual self-attention module and pooling layers to extract global contextual features. By introducing a dual self-attention module to replace the traditional convolution or Transformer structure in the second type of encoding stage, this module embeds multi-level spatial attention modules, multi-level channel attention modules, and a global feature gating perception module. It can capture the complex geometric features of the non-convex, multi-branched parotid gland with linear computational complexity. Compared with the existing technology based on convolution operations or standard self-attention mechanisms with quadratic complexity, it achieves long-range spatial dependency modeling for low-contrast regions such as the deep lobe of the gland, significantly enhances the ability to distinguish between the glandular parenchyma and adjacent muscle and adipose tissue, effectively avoids overflow or under-segmentation of the segmentation boundary, and ensures computational efficiency.

[0054] The segmentation decoder is symmetrically topologically connected to the shared encoder. It performs progressive upsampling and feature fusion on the deep semantic feature map, restoring it to the spatial resolution of the standard input image and outputting a segmentation mask. By setting up a segmentation decoder symmetrically topologically connected to the shared encoder, and employing a progressive upsampling and feature fusion strategy, the low-resolution deep semantic feature map output by the encoder is gradually restored to the spatial resolution of the original input image. The symmetrical topology, combined with skip connections (hidden features), allows the decoding process to fully utilize the shallow spatial details retained at each stage of the encoder, compensating for the loss of spatial localization accuracy in deep features. This results in a high-precision segmentation mask with clear boundaries and accurate localization, achieving the pixel-level accuracy required for surgical planning.

[0055] The input of the classification module is connected to the output of the shared encoder. This connection allows for spatial anatomical constraints based on deep semantic feature maps, focusing on the lesion region for classification and outputting the classification result. Unlike existing technologies that only perform simple parallel classification branches at the network's end, this approach directly connects the input of the classification module to the output of the shared encoder. This enables the classification module to directly acquire deep semantic feature maps rich in spatial location and geometric morphology information. By applying spatial anatomical constraints to these feature maps, the classification module can autonomously focus on the core lesion region, suppressing background noise from oral metal artifacts, cervical lymph nodes, and surrounding normal tissue. This structural design ensures that the features used for classification decisions primarily originate from semantically validated lesion target areas, rather than the mixed information from the entire image. This significantly improves the classification accuracy for morphologically similar lesions such as pleomorphic adenomas, adenolymphomas, and malignant tumors, enhancing the model's robustness in complex clinical imaging environments.

[0056] By organically coordinating a shared encoder, segmentation decoder, and classification module, this invention achieves end-to-end integrated analysis from input parotid gland medical images to simultaneous output of segmentation masks and classification results. Compared to existing technologies that require separate training of segmentation and classification models or complex post-processing fusion, this invention simplifies the classification and diagnosis process, reduces computational resource consumption, and achieves superior performance in both segmentation accuracy and classification robustness compared to existing single-task models or simple multi-task parallel models through implicit feature sharing and constraints between tasks.

[0057] (2) The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention includes several sequentially connected double convolutional coding layers in the first type of encoding stage. Each double convolutional coding layer includes a double convolution module and a max pooling downsampling module connected to the input end and the output end of the double convolution module. By setting several sequentially connected double convolutional coding layers in the first type of encoding stage, each layer adopts the structure of "double convolution + max pooling downsampling", multi-level local feature extraction of the input image is realized. The max pooling downsampling module expands the receptive field while reducing the spatial resolution of the feature map, enabling the network to capture multi-scale local texture and edge information with fewer parameters. The cascaded design of several layers allows the shallow layer to focus on subtle edge structures and the deep layer to focus on semantic component features, providing a rich and hierarchical local feature foundation for subsequent global context modeling, which helps to improve the localization accuracy of the parotid gland boundary.

[0058] The second type of encoding stage includes several sequentially connected double convolutional self-attention encoding layers. Each double convolutional self-attention encoding layer consists of a double self-attention module, a double convolution module, and a max-pooling downsampling module connected in sequence. By setting up a cascaded structure of "double self-attention + double convolution + pooling" in the second type of encoding stage, the network can have both global receptive field and local detail modeling capabilities when extracting deep semantic features. The double self-attention module captures the long-range spatial dependencies of the non-convex, multi-branch complex morphology of the parotid gland with linear complexity, solving the problem of limited receptive field in traditional convolution operations. The following double convolution module further refines the attention-enhanced features locally, making up for the shortcomings of the attention mechanism in local detail modeling. The sequential cascaded design of the two realizes the feature optimization strategy of "global perception first, then local fine-tuning", which significantly enhances the ability to distinguish tissues in low-contrast regions such as deep lobes of the gland and effectively avoids overflow or under-segmentation of the segmentation boundary.

[0059] The segmentation decoder is equipped with inverse double convolutional decoding layers that correspond one-to-one with the double convolutional coding layers. Each inverse double convolutional decoding layer includes a transposed convolutional module and an inverse double convolutional module whose input is connected to the output of the transposed convolutional module. By setting inverse double convolutional decoding layers in the segmentation decoder that correspond one-to-one with the double convolutional coding layers in the first type of encoding stage, a complete symmetrical encoding and decoding topology is constructed. The transposed convolutional module progressively enlarges the spatial size of the deep low-resolution feature map, gradually restoring it to the resolution of the original input image; the inverse double convolutional module takes the upsampled feature map and performs feature decoupling and reconstruction optimization. This one-to-one corresponding hierarchical symmetrical design allows the decoding path to fully utilize the spatial position information retained at each level of the encoding path, effectively compensating for the loss of pixel-level localization accuracy in deep features.

[0060] The segmentation decoder includes inverse double convolutional self-attention decoding layers, each corresponding to a double convolutional self-attention encoding layer. These inverse double convolutional self-attention decoding layers consist of a transposed convolutional module, an inverse double convolutional module, and a double self-attention module connected sequentially. By incorporating these inverse double convolutional self-attention decoding layers within the segmentation decoder, and employing a structure of "transposed convolutional upsampling + inverse double convolution + double self-attention," progressive spatial resolution restoration and feature optimization of deep semantic feature maps are achieved. The transposed convolutional module doubles the feature map size; the inverse double convolutional module (structurally symmetrical to the double convolutional module) decouples and reconstructs the fused features; and the double self-attention module reintroduces global context modeling during the decoding stage, further optimizing pixel-level predictions of boundary regions. This symmetrical decoding design ensures the spatial localization accuracy of the segmentation mask, generating high-quality segmentation results with clear boundaries.

[0061] Compared to existing technologies that only introduce attention mechanisms at the end of the encoder or do not introduce attention at all, the technical solution of this invention introduces dual self-attention modules in both the deep layers of the encoding stage and the corresponding layers of the decoding stage, and realizes the cross-stage transfer of attention-enhanced features through skip connections. This "dual-stage attention + skip enhancement" design allows the network to benefit from global context modeling capabilities in both the key stages of feature extraction and spatial reconstruction. Especially for regions with extremely low contrast, such as the deep lobe of the parotid gland, it can realize spatial structure constraints throughout the entire process from feature extraction to pixel prediction, significantly improving the boundary accuracy and internal consistency of the segmentation mask.

[0062] This invention employs a multi-level, progressive feature extraction strategy, starting with local features and then moving to global ones. The shallow, first-class encoding stage uses a dual convolutional module to meticulously capture local details such as the boundary texture and edge direction of the parotid gland. The deep, second-class encoding stage uses a dual self-attention module to establish global spatial dependencies based on local features. These two stages complement each other: local features provide fine-grained spatial anchors for the attention mechanism, while the attention mechanism, in turn, guides local features to focus on semantically relevant regions, jointly enhancing the representation ability of non-convex, multi-branch complex geometric shapes.

[0063] This invention achieves efficient feature extraction during the encoding stage by using depthwise separable convolutions in the dual convolution module and linear complexity attention computation in the dual self-attention module. Simultaneously, the symmetrical decoder design avoids introducing additional heavy modules, maintaining a lightweight overall computational load. This collaborative design of "high-efficiency encoding + lightweight decoding" enables the model to meet the requirements of real-time inference on computationally limited primary healthcare terminals while ensuring segmentation accuracy.

[0064] (3) The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention adds a third type of encoding stage in the shared encoder. This stage inherits the output of the second type of encoding stage and embeds a dual self-attention module. It is a special design for the unique anatomical structure of the parotid gland. As the largest salivary gland in the human body, the parotid gland has the following special characteristics: extremely irregular shape (non-convex, multi-branched geometric shape, with deep lobes and shallow lobes intersecting each other), extremely low contrast between the deep lobe region and soft tissues such as the masseter muscle and parapharyngeal space, highly heterogeneous internal acini, ducts, fat and lymph nodes, and typical "same image, different disease" characteristics of benign and malignant lesions. These attributes make the segmentation and classification task of the parotid gland significantly different from the medical image processing of other organs such as the liver, lungs and heart. The latter have relatively regular shapes, clearer boundaries and more distinct classification features. Conventional UNet variants can achieve good results by adding a simple attention module at the end of the encoder. This invention sets up a third type of encoding stage at the very end of the encoder (i.e., the position with the highest level of semantic abstraction and the lowest feature map spatial resolution, such as 7×7 or 4×4). This stage can achieve global feature modeling with the largest receptive field at extremely low computational cost, enabling the model to gain a "global understanding" of the overall anatomical structure of the parotid gland. At the same time, the deep semantic feature map output by this stage not only retains the geometric morphological information required for the segmentation task, but also undergoes global enhancement through dual self-attention, and is rich in semantic features that are highly correlated with tumor category discrimination. This effectively solves the problem of insufficient segmentation accuracy caused by the lack of adaptive modeling ability for complex geometric morphology in existing feature extraction mechanisms in the background technology, and provides a high-quality feature source for classification tasks.Building upon this foundation, the input of the classification module is directly connected to the output of the third-class encoding stage. This enables the classification task to acquire a deep semantic feature map enhanced by dual self-attention, unlike existing technologies where the classification branch is simply connected to the encoder end or features are extracted directly from shallow layers. This ensures that the classification task obtains the deepest and most abstract semantic features to facilitate the differentiation of morphologically similar lesions such as pleomorphic adenoma, adenolymphoma, and malignant tumors. The heterogeneous multi-scale context fusion module within the classification module employs a parallel dual-branch structure. The first convolutional branch uses small-sized convolutional kernels (e.g., 3×3) to extract local texture details, thus keenly capturing microstructures such as microcalcifications and cystic changes within the tumor. The second convolutional branch uses large-sized convolutional kernels (e.g., 5×5 or 7×7) to extract global contextual information, thereby grasping the overall morphology and boundary features of the tumor, as well as other macrostructures. The outputs of both are spliced ​​and fused along the channel dimension, achieving a complementary description of the local details and global structure of the lesion area. This effectively suppresses the interference features of background noise such as oral metal artifacts and cervical lymph nodes, significantly improving the four-class classification accuracy of morphologically similar lesions. At the same time, it overcomes the technical difficulty that single-scale feature extraction is often sufficient in other organ classification tasks, while the parotid gland must rely on multi-scale comprehensive judgment. The classifier further transforms the fused feature map into a fixed-dimensional feature vector through a feature compression unit to eliminate the influence of spatial dimension on the classification results. The fully connected classification unit, which consists of multiple linear layers, normalization layers, activation function layers and dropout layers, performs progressive compression of feature dimensions and semantic mapping. This design ensures the robustness of the model on small samples or imbalanced datasets, and is especially suitable for clinical scenarios with a small number of parotid gland malignant lesion samples. In summary, this invention, through a third-class encoding stage and classification module designed for the unique anatomical features of the parotid gland, such as "extremely blurred boundaries, extremely irregular shape, and different diseases in the same image," simultaneously achieves sub-pixel-level segmentation accuracy and highly robust four-class classification output while ensuring linear computational complexity. Its core innovations include: using dual self-attention to achieve adaptive modeling of non-convex geometric shapes; using heterogeneous multi-scale context fusion to address the texture overlap problem; using the classification module to directly connect to the encoder end to achieve spatial anatomical constraints; and using a linear complexity attention mechanism to adapt to the computing power of the grassroots terminal.

[0065] Furthermore, it is particularly important to emphasize that the initial intention behind proposing the aforementioned technical solution stems from a profound understanding of actual clinical needs and a deep concern for patient well-being. Currently, while accurate diagnosis of parotid gland lesions can rely on examination methods with excellent soft tissue contrast, such as MRI, MRI equipment is expensive, examination costs are high, scanning time is long, and patient cooperation is required to a high degree. Moreover, there is a shortage of equipment resources in some grassroots areas. In contrast, ultrasound examination has significant advantages such as no ionizing radiation, portable equipment, low cost, high availability, and strong repeatability, making it particularly suitable for large-scale early screening and grassroots medical institutions. However, due to the limited field of view, inconsistent cross-sections, variable boundaries, and susceptibility to manipulation and tissue obstruction in ultrasound images, it has long been considered a technological disadvantage unsuitable for refined parotid gland segmentation and classification. This invention, through a systematic architectural innovation of heterogeneous perceptual mapping networks—including cascaded local and global feature extraction, spatially anatomically constrained classification guidance, and a dual self-attention mechanism with linear complexity—successfully overcomes the inherent limitations of ultrasound images. It achieves fine segmentation and highly robust classification results on low-contrast, high-noise ultrasound images, previously only achievable with high-cost MRI or CT scans. If applied, this invention will bring significant social value: firstly, ultrasound examinations are significantly cheaper than MRI and CT, greatly reducing the economic burden on patients while ensuring diagnostic accuracy, thus enabling precision medicine to benefit a wider population; secondly, ultrasound equipment is far more prevalent in primary healthcare institutions than MRI and CT, and this technology can enable these institutions to possess refined parotid gland diagnostic capabilities, promoting the effective implementation of hierarchical medical services. This invention concept, driven by technological innovation for universal healthcare, and the systematic architectural design to solve specific clinical imaging challenges, vividly embodies the inventiveness of this application, enabling a qualitative leap in technical effectiveness—breaking down the technical barrier that ultrasound images are unsuitable for refined parotid gland segmentation, and opening up new possibilities for low-cost, high-efficiency precise diagnosis of the parotid gland. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the overall technical process of an embodiment of the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention;

[0067] Figure 2 This is a schematic diagram of the heterogeneous perceptual mapping network model, which is an embodiment of the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention.

[0068] Figure 3 This is a schematic diagram of the principle of the dual convolution module in an embodiment of the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention.

[0069] Figure 4This is a schematic diagram of the principle of the inverse double convolution module in an embodiment of the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention.

[0070] Figure 5 This is a schematic diagram illustrating the principle of the dual self-attention module in an embodiment of the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network of the present invention. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0072] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0073] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, an integral connection, or a detachable connection; they can refer to the internal connection of two components; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0074] Example 1

[0075] This embodiment provides a parotid gland segmentation and classification method based on a heterogeneous perceptual mapping network. This method aims to simultaneously achieve high-precision anatomical structure segmentation and multi-class classification using a single deep learning model, solving the problems of blurred segmentation boundaries and susceptibility to noise interference in existing technologies. Figure 1 As shown, it includes the following steps:

[0076] Step S100: Acquire the parotid gland ultrasound image data to be processed and preprocess it to obtain a standard input image. Since raw medical images often suffer from inconsistent resolution, noise interference, or insufficient contrast, the preprocessing step is particularly important. Preprocessing operations include, but are not limited to, normalization, image cropping, denoising filtering, and contrast enhancement. For example, normalization can map image pixel values ​​to the range of 0 to 1, eliminating brightness differences caused by different acquisition devices; image cropping is used to remove irrelevant background areas, focusing on the anatomical region where the parotid gland is located, thereby reducing computational overhead and increasing model focus. After preprocessing, a standard input image with uniform size and clear features is obtained, providing a high-quality data foundation for subsequent model analysis. It should be noted that this embodiment focuses on segmenting and classifying parotid gland ultrasound image data. Parotid gland ultrasound image data differs significantly from CT / MRI image data, as shown in Table 1:

[0077] Table 1. Differences between parotid gland ultrasound images and CT / MRI images

[0078]

[0079] Step S200: The standard input image is input into the heterogeneous perceptual mapping network model for integrated analysis to obtain the output feature map. The integrated analysis includes synchronously executed segmentation and classification tasks, and the output feature map includes a segmentation mask and classification results. Specifically, in this embodiment, "integrated analysis" refers to the model simultaneously outputting two different dimensions of diagnostic information during a single inference process through a shared feature extraction mechanism. This is drastically different from the traditional approach of treating segmentation and classification as two independent, sequential processes. In traditional approaches, the classification task often relies on the segmented region image as input. If the segmentation result has deviations (such as boundary overflow), the classification task will inevitably be affected by the erroneous input, leading to error accumulation. In this embodiment, through the "synchronous execution" design, the segmentation and classification tasks share the same deep semantic features extracted by the same encoder. They are processed in parallel at the feature level, mutually constraining and complementing each other. The spatial mask generated by the segmentation task provides "spatial anatomical constraints" for the classification task, forcing the classification branch to focus on the true lesion area rather than being misled by surrounding lymph nodes or metal artifacts; conversely, the semantic features extracted by the classification task can also feed back into the segmentation task, strengthening the ability to identify lesion boundaries. This deep coupling mechanism effectively solves the identification problem caused by overlapping features of "same image, different disease", and improves the robustness of the overall diagnostic system.

[0080] Among them, the heterogeneous perception mapping network model is a multi-task model trained based on sample data and corresponding annotations. Its overall topology is a goblet-shaped asymmetric encoding and decoding dual-drive architecture, including a shared encoder, a segmentation decoder and a classification module.

[0081] A shared encoder is used to extract features at multiple scales from a standard input image, generating a deep semantic feature map. Specifically, the shared encoder is the core feature extraction unit of the model, and its input receives the preprocessed standard input image. For example... Figure 2 As shown, the shared encoder employs a cascaded multi-stage structure, gradually expanding the receptive field through layer-by-layer downsampling, transitioning from shallow texture details to deep semantic abstractions. The shared encoder includes at least two cascaded encoding stages. The first encoding stage uses a dual convolutional module and pooling layers to extract local detail features, focusing on capturing low-frequency geometric information such as texture and shape of the parotid gland edge, providing a fine pixel-level reference for subsequent boundary regression. The second encoding stage uses a dual self-attention module and pooling layers to extract global contextual features, focusing on establishing long-range dependencies between pixels and understanding the overall spatial layout of the lesion and surrounding tissues, thereby achieving accurate tissue differentiation in low-contrast regions (such as the deep lobe of the parotid gland). It should be understood that although this embodiment describes the cascaded order of the first and second encoding stages, in other embodiments, the number and order of encoding stages can be adaptively adjusted according to the morphological complexity of the specific lesion. For example, a third encoding stage can be added to further refine deep semantics, as long as multi-scale feature extraction can be achieved.

[0082] The segmentation decoder, symmetrically connected to the shared encoder, performs progressive upsampling and feature fusion on the deep semantic feature map, restoring the spatial resolution to the standard input image and outputting a segmentation mask. Specifically, the segmentation decoder is responsible for restoring deep abstract features to concrete pixel locations. Its symmetrical topology with the shared encoder means that the number of decoding stages corresponds one-to-one with the number of encoding stages. Through progressive upsampling (such as transposed convolution), the spatial size of the feature map is gradually enlarged, and the resolution is gradually restored. During this process, the segmentation decoder utilizes a skip connection mechanism to fuse the shallow detail features output from each stage of the encoder with the deep semantic features from the corresponding stage of the decoder. This "deep-shallow fusion" design effectively compensates for the loss of detail information during simple upsampling, enabling the final segmentation mask to accurately delineate the non-convex, multi-branched geometric boundaries of the parotid gland, achieving sub-pixel-level segmentation accuracy. The output segmentation mask is a binarized or probabilistic map with the same size as the input image, where the pixel values ​​of the lesion region are prominently marked, visually demonstrating the tumor's infiltration range and spatial morphology.

[0083] The classification module, whose input is connected to the output of the shared encoder, performs spatial anatomical constraints based on deep semantic feature maps to focus on lesion regions for classification and outputs the classification results. Specifically, unlike existing technologies that only perform simple parallel classification branches at the network's end, this module directly connects its input to the shared encoder's output, enabling it to directly acquire deep semantic feature maps rich in spatial location and geometric morphology information. By applying spatial anatomical constraints to these feature maps, the classification module can autonomously focus on the core lesion region, suppressing interference from background noise such as oral metal artifacts, cervical lymph nodes, and surrounding normal tissue. This structural design ensures that the features used for classification decisions primarily originate from semantically validated lesion target areas, rather than the mixed information from the entire image. This significantly improves the classification accuracy for morphologically similar lesions such as pleomorphic adenomas, adenolymphomas, and malignant tumors, enhancing the model's robustness in complex clinical imaging environments.

[0084] Through the above scheme, this embodiment constructs an end-to-end intelligent parotid gland image analysis system. The heterogeneous feature extraction mechanism of the shared encoder solves the segmentation problem of low-contrast regions, the symmetrical fusion structure of the segmentation decoder ensures accurate boundary restoration, and the spatial anatomical constraint design of the classification module achieves high-accuracy lesion characterization. These three components work collaboratively, simultaneously outputting segmentation masks and classification results in a single inference process, providing clinicians with comprehensive and multi-dimensional decision support and greatly improving the efficiency of classification diagnosis.

[0085] Example 2

[0086] Based on Example 1, this embodiment elaborates in detail on the specific structure of the first type of encoding stage in the shared encoder and the symmetrical topological connection relationship of the segment decoder.

[0087] like Figure 2 and Figure 3 As shown, the first type of encoding stage comprises several sequentially connected double convolutional coding layers. Specifically, as the shallow part of the shared encoder, the first type of encoding stage is mainly responsible for extracting high-resolution local detail features from the standard input image, such as the texture of the parotid gland edge, microcalcifications, and subtle contrast differences between tissues. This stage typically contains two or more double convolutional coding layers, which are connected in a cascaded manner, with the output of the previous layer serving as the input of the next layer, thereby progressively deepening the expressive power of the features. After completing feature extraction, each double convolutional coding layer reduces the feature map size through a downsampling operation, expanding the receptive field to prepare for subsequent deeper semantic extraction.

[0088] Any double convolutional coding layer includes a double convolutional module and a max-pooling downsampling module whose input is connected to the output of the double convolutional module. The double convolutional module is the core unit for feature extraction, designed to balance the effectiveness and computational efficiency of feature extraction. The max-pooling downsampling module is used to reduce the dimensionality of the feature map, preserving the most salient feature responses while reducing computational cost.

[0089] Specifically, the dual convolution module comprises a 3×3 convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 depthwise separable convolutional layer, a layer normalization layer, and a ReLU activation function layer connected sequentially. This structure employs a hybrid orchestration strategy of "standard convolution followed by depthwise separable convolution." First, the 3×3 convolutional layer, as a standard two-dimensional convolutional operation, performs a preliminary global feature scan of the input feature map, capturing basic edge and texture information. Subsequently, the layer normalization layer standardizes the feature distribution, accelerating model convergence and improving training stability. The ReLU activation function layer introduces a non-linear transformation, enhancing the model's ability to fit complex features. Next, the 3×3 depthwise separable convolutional layer is introduced. Unlike standard convolution, depthwise separable convolution decouples spatial convolution from channel convolution, performing spatial convolution independently on each channel first, and then integrating channel information through a 1×1 convolution. This design significantly reduces the number of parameters and computational overhead while maintaining similar feature extraction performance. At the end of the module, a layer normalization layer and a ReLU activation function layer are added again to further optimize the feature distribution and activate the nonlinear response. Through this dual convolutional structure, the model can efficiently extract the fine geometric features of the parotid gland region in the shallow encoding stage, laying a solid foundation for subsequent processing. It should be noted that the parotid gland boundary in ultrasound images often presents a gradual grayscale transition rather than a sharp edge, and traditional single convolution is difficult to capture this "soft boundary". This module adopts a cascaded design of dual convolution (standard convolution + depthwise separable convolution). The first 3×3 standard convolution performs preliminary localization of the coarse-grained echo region in the ultrasound image and extracts the approximate outline of the nodules; the second 3×3 depthwise separable convolution, based on the preliminary localization, captures the weak gradient changes at the boundary with a finer-grained receptive field, achieving sub-pixel-level fine characterization of the blurred boundary.

[0090] After feature extraction, the max pooling downsampling module reduces the size of the feature map output by the dual convolution module. In this embodiment, a 2×2 pooling window and a stride of 2 are used to halve both the length and width of the feature map while retaining the feature values ​​with the strongest responses in each local region. This operation not only reduces the computational burden of subsequent layers but also increases the translation invariance of the features to some extent.

[0091] Accordingly, to restore the spatial resolution of the image and achieve accurate segmentation, the segmentation decoder is equipped with inverse double convolutional decoding layers that correspond one-to-one with the double convolutional coding layers. This one-to-one correspondence constitutes the "symmetric topological connection" described in Example 1. Specifically, if the first type of encoding stage of the shared encoder contains two double convolutional coding layers, then the segmentation decoder is equipped with two corresponding inverse double convolutional decoding layers. This symmetrical design ensures that during the decoding process, each layer can obtain feature information at the same scale as the encoding stage, thereby achieving pixel-level accurate localization.

[0092] Any inverse double convolutional decoding layer includes a transposed convolutional module and an inverse double convolutional module whose input is connected to the output of the transposed convolutional module. The transposed convolutional module performs the upsampling task, which learns reversible convolutional parameters to enlarge low-resolution deep feature maps to the same size as the corresponding coding layer feature maps. In this embodiment, a 2×2 transposed convolutional kernel is used with a stride of 2, doubling the length and width of the feature map. This process is not a simple interpolation operation, but rather intelligent padding using learnable parameters, so that the upsampled feature map contains richer semantic information.

[0093] The inverse double convolution module consists of a 3×3 depthwise separable convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 convolutional layer, a layer normalization layer, and another ReLU activation function layer connected in sequence. It presents a "mirror image" or "reverse order" relationship with the double convolution module in the encoder. In the double convolution module, the order is "standard convolution -> depthwise separable convolution," while in the inverse double convolution module, the order is reversed to "depthwise separable convolution -> standard convolution." This design has profound complementary significance: in the decoding stage, the depthwise separable convolution is first used to perform lightweight and refined processing on the upsampled features, removing noise or checkerboard effects that may be introduced by upsampling, and efficiently restoring local details; subsequently, the standard 3×3 convolution is used to perform comprehensive channel integration and semantic reconstruction on the processed features, ensuring that the final output feature map contains both fine boundary information and accurate semantic discrimination capability. The alternating setting of layer normalization and ReLU activation functions also plays a role in stabilizing feature distribution and enhancing nonlinear expression.

[0094] To further enhance the accuracy of the segmentation boundaries, such as Figure 2As shown, the output of the double convolution module in the double convolutional coding layer is connected to the input of the corresponding inverse double convolutional decoding layer. This connection method is known in the art as "skip connection". Specifically, during feature extraction, as the number of downsampling times increases, shallow details (such as the precise location of the parotid gland edge and subtle texture changes) are easily lost. Although the decoder attempts to recover these details through upsampling, simple upsampling often fails to fully restore them. Through skip connections, the shallow high-resolution feature map output by the encoder's double convolution module is directly passed to the corresponding decoding layer and fused with the deep feature map upsampled by the transposed convolution module (e.g., channel concatenation). This fusion operation allows the decoding layer to simultaneously possess "deep semantic guidance" and "shallow detail supplementation", effectively solving the problem of detail loss in deep networks. For the parotid gland, an anatomical structure with irregular shape and blurred boundaries, the introduction of skip connections is crucial. It enables the segmentation decoder to accurately delineate the non-convex boundaries of the gland, avoiding overflow or undersegmentation in the segmentation results, and significantly improving the quality of the segmentation mask.

[0095] Through the symmetrical cooperation of the aforementioned double convolutional coding layer and inverse double convolutional decoding layer, as well as the feature complementarity mechanism of skip connections, this embodiment constructs an efficient and accurate feature extraction and reconstruction closed loop. The encoding end focuses on "compression and refinement," using the double convolutional structure to efficiently capture local details; the decoding end focuses on "restoration and reconstruction," using the inverse double convolutional structure to intelligently restore spatial resolution. The two work together to ensure that the final output segmentation mask can achieve sub-pixel accuracy requirements.

[0096] Example 3

[0097] During ultrasound scanning, the angle, depth, and probe pressure are not fixed, resulting in the same nodule exhibiting drastically different two-dimensional morphologies under different sections. The model must infer the three-dimensional spatial structure of the nodule within a single section, which relies on a deep understanding of the global context.

[0098] This embodiment, based on Embodiments 1 and 2, further elaborates on the specific structure of the second type of encoding stage in the shared encoder and the detailed algorithm flow of the core dual self-attention module. It should be understood that the first type of encoding stage described in Embodiment 2 focuses on capturing local details, while the second type of encoding stage in this embodiment focuses on modeling global contextual information. The two complement each other in the feature extraction dimension and together constitute the "heterogeneous" core of the heterogeneous perceptual mapping network model.

[0099] like Figure 2 and Figure 4As shown, the second type of encoding stage includes several sequentially connected double convolutional self-attention encoding layers. Specifically, the second type of encoding stage is located in the deep part of the shared encoder, inheriting the local feature maps output by the first type of encoding stage. As the network depth increases, the spatial resolution of the feature maps gradually decreases, while the semantic abstraction level gradually increases. At this point, it is difficult to effectively capture the long-range dependencies between pixels by simply relying on convolution operations (such as the spatial association between the deep and shallow lobes of the parotid gland). Therefore, this embodiment introduces double convolutional self-attention encoding layers, using the self-attention mechanism to break the limitations of the local receptive field and achieve the aggregation of global information. This stage typically contains two or more double convolutional self-attention encoding layers, which are connected in a cascaded manner to deepen the understanding of global semantics layer by layer.

[0100] Any dual-convolutional self-attention encoding layer comprises a dual self-attention module, a dual convolutional module, and a max-pooling downsampling module connected in sequence. In this structure, the dual self-attention module serves as the primary processing unit, responsible for global context modeling of the input feature map and establishing long-range associations between pixels. Subsequently, the dual convolutional module (whose specific structure is described in Example 2) further refines the globally enhanced features locally, remapping abstract semantic information onto specific feature channels. Finally, the max-pooling downsampling module reduces the dimensionality of the feature map, further expanding the receptive field and reducing computational cost. This "global-first, local-later" arrangement strategy effectively solves the problem of the separation between global and local features in traditional networks, enabling the model to both "see far" (understand the overall layout) and "see in detail" (preserve boundary textures).

[0101] Accordingly, to recover global information during decoding, the segmentation decoder is equipped with inverse double convolutional self-attention decoding layers that correspond one-to-one with the double convolutional self-attention coding layers. This symmetrical design ensures that deep global features can be completely transmitted to the decoding end. Any inverse double convolutional self-attention decoding layer includes a transposed convolutional module, an inverse double convolutional module, and a double self-attention module connected in sequence. The transposed convolutional module first upsamples the deep feature map to restore the spatial dimensions; the inverse double convolutional module (whose specific structure is described in Example 2) performs detailed reconstruction of the upsampled features; finally, the double self-attention module intervenes again to perform global semantic calibration on the restored features, ensuring the rationality of the spatial distribution of the decoded features.

[0102] In this design, the output of the dual self-attention module in the dual convolutional self-attention coding layer is connected to the input of the inverse dual convolution module in the corresponding inverse dual convolutional self-attention decoding layer. This skip connection mechanism is consistent with the connection method described in Example 2, aiming to directly transmit the global semantic features extracted from the deep layers of the encoder to the decoder, making up for the high-level semantic information that may be lost during the upsampling process, thereby improving the accuracy of the segmentation mask in low-contrast regions.

[0103] Next, we will focus on explaining the core innovation of this embodiment, specifically the internal algorithm flow of the dual self-attention module. For example... Figure 5 As shown, the dual self-attention module extracts features using the following steps:

[0104] Step S301: Perform layer normalization on the input feature map. Layer normalization can stabilize the feature distribution, accelerate model convergence, and provide standardized input for subsequent attention calculation.

[0105] Step S302 involves inputting the layer-normalized feature map into a multi-level spatial attention module to obtain the first intermediate feature. This step aims to capture long-range dependencies within the feature map from a spatial perspective.

[0106] Step S303: The first intermediate feature is concatenated with the input feature map through channels, and after layer normalization, it is input into the global feature gated perception module. The local context information is encoded by depthwise separable convolution and the feature response is adaptively modulated by a gated unit to obtain the second intermediate feature. The specific calculation method is as follows:

[0107] ,

[0108] in, Features output by the global feature-gated perception module. The features are the input features of the global feature-gated perception module. For linear projection, For layer normalization processing, Activation processing for the GELU activation function. The system employs a 3×3 depthwise separable convolution. Specifically, the channel concatenation operation implements the concept of residual connections, fusing the original input features with features enhanced by spatial attention, preserving the original information while introducing new global information. The global feature gating perception module efficiently extracts local context using depthwise separable convolution and adaptively modulates the feature response through gating units, suppressing irrelevant noise and enhancing the expression of key features. In this embodiment, the gating unit uses a combination of the GELU activation function and linear transformation to adaptively modulate the feature response.

[0109] Step S304 involves concatenating the second intermediate feature with the first intermediate feature via channels, normalizing the result, and then inputting it into a multi-level channel attention module. Based on the transposed attention mechanism, the channel dimension is modeled to obtain the third intermediate feature. This step models the feature along the channel dimension. The transposed attention mechanism differs from traditional spatial attention; it constructs attention weights by calculating the similarity between channels, effectively capturing the correlation between different semantic channels and further refining the expressive power of the features.

[0110] In step S305, the third intermediate feature and the second intermediate feature are concatenated through channels, and after layer normalization, they are input again into the global feature gating perception module for feature modulation to obtain the fourth intermediate feature. This step is a secondary modulation of the features, which aims to deeply fuse the channel attention-enhanced features with the previous local context features to further optimize the feature distribution.

[0111] Step S306: The fourth intermediate feature and the third intermediate feature are concatenated by channels to output the final dual self-attention feature map. Through the above multi-level and multi-dimensional progressive processing, the dual self-attention module realizes comprehensive modeling of the spatial, channel and local contextual information of the features. The output feature map contains both accurate spatial location information and rich semantic channel information, providing strong feature support for subsequent segmentation and classification tasks.

[0112] The multi-level spatial attention module, with its specific algorithm flow, is key to achieving linear complexity global perception. The first intermediate feature is obtained by inputting the layer-normalized feature map into the multi-level spatial attention module, specifically through the following steps:

[0113] Step S3021: Apply linear projection to the normalized feature map of the layer. Mapped to query features Key features Value characteristics This process linearly transforms the input features using a learnable weight matrix to generate the three elements used for attention computation.

[0114] Step S3022, query features Key features Value characteristics Divided into multiple feature groups according to channel dimension This partitioning strategy is similar to a multi-head attention mechanism, aiming to allow different feature groups to focus on different semantic subspaces, thereby enhancing the diversity of feature representation. In this embodiment, the number of channels is... Feature maps are divided into There are 1 group, and the number of channels in each group is 1. .

[0115] Step S3023: For the key features and value features in each feature group, multi-scale feature extraction is performed using depthwise separable convolutions with different kernel sizes. After batch normalization and ReLU activation function processing, the enhanced key features and enhanced value features are output. The specific calculation method is as follows:

[0116] ,

[0117] ,

[0118] in, The first Key features and value features of each feature group; For the first Multi-scale depthwise separable convolutions corresponding to each feature group; For batch normalization processing, This involves processing with the ReLU activation function. This is the core step in achieving linear complexity in this embodiment. Traditional self-attention mechanisms typically perform global dot product operations on the entire feature map, with computational complexity increasing quadratically with the number of pixels. This embodiment, however, introduces multi-scale depthwise separable convolution to extract local features from key and value features of different feature groups. For the first... Each feature group is processed using a depthwise separable convolutional kernel of size . This operation restricts the computation of global attention to the region covered by the local convolutional kernel, thereby reducing the computational complexity from quadratic to linear, greatly improving the model's inference efficiency and making it adaptable to medical terminal devices with limited computing power. Simultaneously, convolutional kernels of different sizes can capture spatial features at different scales, enhancing the model's ability to perceive multimorphic lesions.

[0119] Step S3024 involves concatenating the query features, enhanced key features, and enhanced value features of each feature group along the channel dimension to obtain the aggregated query features, aggregated key features, and aggregated value features. The specific calculation method is as follows:

[0120] ,

[0121] ,

[0122] ,

[0123] in, These are the aggregated query features, key features, and value features. This is for channel splicing operations. For the first Query features of feature groups For the first Enhanced key features are obtained by multi-scale extraction of feature groups. For the first The enhanced features are obtained by multi-scale extraction of each feature group. The concatenation operation integrates the processed results of each feature group, restoring them to the original channel dimension to form a complete feature representation.

[0124] Step S3025: Apply the Softmax function to the aggregated query features and aggregated key features respectively, obtain the attention weights through dot product attention calculation, and multiply the attention weights by the aggregated value features to obtain the first intermediate feature. The specific calculation method is as follows:

[0125] ,

[0126] in, An attention-weighted output function; These are the aggregated query features, key features, and value features; The process involves activation using the Softmax activation function. The Softmax function normalizes the dot product result into a probability distribution, representing the level of attention at different locations. Finally, the attention weights are multiplied by the value features using a weighted summation method to generate output features that contain global contextual information. Since the key features and value features have already undergone multi-scale convolution enhancement in step S3023, the attention calculation at this point is actually performed in the multi-scale feature space, which preserves the ability to perceive the world while avoiding high computational costs.

[0127] Through the design of the aforementioned dual self-attention module, this embodiment successfully introduces an efficient global perception mechanism into the deep stages of the shared encoder. The multi-level spatial attention module employs depthwise separable convolutions with different kernel sizes (e.g., 3×3, 5×5, 7×7) to extract key and value features at multiple scales, enabling the model to simultaneously perceive the local edge texture (small convolution kernel) and overall contour direction of the nodule (large convolution kernel). Regardless of changes in the cross-section, it can stably capture the core structure of the nodule. Furthermore, multi-scale convolution achieves linear complexity feature extraction, solving the computational bottleneck in high-resolution medical image processing. The progressive cooperation between the global feature-gated perception module and the multi-level channel attention module deeply optimizes features from both local context and channel semantic dimensions, suppressing noise responses caused by tissue occlusion and acoustic artifacts in ultrasound images, and enhancing features from the nodule's parenchymal region. This heterogeneous feature aggregation method allows the model to accurately capture the complex, non-convex, multi-branched geometry of the parotid gland region, enhancing the distinction between the glandular parenchyma and adjacent tissues, laying a solid foundation for final high-precision segmentation and classification.

[0128] Example 4

[0129] This embodiment, based on embodiments 1 to 3, elaborates in detail on the end structure of the shared encoder and the specific implementation of the classification module. It should be understood that the first and second type of encoding stages in the aforementioned embodiments mainly focus on feature extraction and downsampling, while the third type of encoding stage and classification module introduced in this embodiment undertake the key tasks of deep semantic refinement and final qualitative judgment.

[0130] like Figure 2 As shown, the shared encoder also includes a third type of encoding stage, where the input is connected to the output of the second type of encoding stage, and the output is connected to the input of the segmentation decoder. Specifically, the third type of encoding stage is located at the deepest layer of the shared encoder, receiving the global context features output by the second type of encoding stage. At this point, the spatial resolution of the feature map has been reduced to the lowest level, but the semantic abstraction level is the highest. The introduction of the third type of encoding stage aims to further refine this deep feature, remove redundant information, and extract the most discriminative semantic features. The third type of encoding stage includes a dual attention module. The specific structure of this dual attention module can reuse the dual self-attention module described in Embodiment 3, or adopt other forms of attention mechanism combinations. Its core function is to perform a final global calibration of the deep features, strengthen the feature response of the core lesion region, suppress the residual background noise, and provide high-quality "clean" semantic input for the subsequent segmentation decoder and classification module. It should be understood that although this embodiment places the third type of encoding stage after the second type of encoding stage, in other embodiments, depending on the design requirements of the network depth, the third type of encoding stage can also be set in parallel or fused with the second type of encoding stage, as long as the function of extracting deep semantic features can be achieved.

[0131] The classification module includes a heterogeneous multi-scale context fusion module connected to the output of the third-class encoding stage, and a classifier connected to the output of the heterogeneous multi-scale context fusion module. In ultrasound images, nodules are often surrounded by interfering structures with similar echogenicity, such as lymph nodes, vascular cross-sections, and adipose tissue. If the classification branch extracts features independently, these background areas are easily misclassified as lesion features. The feature map of the shared encoder has acquired spatial localization capabilities through implicit training by the segmentation decoder: the feature response of the lesion area is significantly higher than that of the background. The classification module is directly connected to this, which is equivalent to forcing the classification branch to extract features from "verified lesion areas" rather than from mixed information in the whole image. In parotid gland ultrasound images, pleomorphic adenoma (PA) and adenolymphoma (WT) often exhibit similar echogenicity and boundaries, showing "same image, different lesion" characteristics. Single-scale feature extraction is difficult to distinguish between the two. The heterogeneous multi-scale context fusion module includes a first convolutional branch and a second convolutional branch. These two branches are set in parallel to capture feature information of the lesion area from different scales. Specifically, the first convolutional branch uses a convolutional kernel of a first size to extract local detail features from the input features. The first size is typically small, using 1×1 or 3×3. Smaller kernels can accurately capture texture details, microcalcifications, or edge morphological changes within lesions, which is crucial for differentiating morphologically similar lesions (such as pleomorphic adenoma and adenolymphoma). In this embodiment, in the first convolutional branch, a 3×3 kernel is used for point-by-point convolution, combined with batch normalization and the ReLU activation function, to efficiently extract pixel-level local features.

[0132] Accordingly, the second convolutional branch uses a convolutional kernel of a second size, which is larger than the first size, to extract global contextual features of the input features. The second size is typically large, using 5×5, 7×7, or even global average pooling. A larger convolutional kernel can cover a larger receptive field, capturing the overall contour of the lesion, its spatial relationship with surrounding tissues, and global statistical distribution features. This is significant for determining the benign or malignant nature of lesions (e.g., malignant tumors often exhibit unclear boundaries and invasive growth). In this embodiment, a 5×5 depthwise separable convolution is used in the second convolutional branch to expand the receptive field while maintaining computational efficiency and extracting global contextual information.

[0133] The first and second convolutional branches are set up in parallel, and their outputs are concatenated and fused along the channel dimension. This parallel concatenation design allows the fused feature map to simultaneously contain both "microscopic details" and "macroscopic morphology" information, achieving comprehensive capture of multi-scale information of the lesion region. This heterogeneous multi-scale fusion strategy effectively solves the problem of insufficient single-scale feature representation and enhances the model's ability to distinguish between different types of lesions.

[0134] The classifier includes a feature compression unit and a fully connected classification unit. The feature compression unit transforms the fused feature map output by the heterogeneous multi-scale context fusion module into a fixed-dimensional feature vector. Specifically, the fused feature map retains its spatial dimension (e.g., H×W×C), while the classification task ultimately requires a one-dimensional vector for discrimination. The feature compression unit typically employs global average pooling or adaptive pooling to compress the feature map in the spatial dimension, calculating the average value for each channel, thereby transforming the three-dimensional feature map into a C-dimensional one-dimensional feature vector. This operation not only eliminates the influence of spatial dimension on the classification result but also prevents overfitting.

[0135] The fully connected classification unit, connected to the output of the feature compression unit, consists of alternating layers of linear layers, normalization layers, activation function layers, and dropout layers. It progressively compresses the dimensionality of the feature vector and maps it semantically, outputting the corresponding classification result. Specifically, the fully connected classification unit is a multilayer perceptron structure. First, the linear layers reduce the dimensionality of the input high-dimensional feature vector, progressively compressing it to a lower dimension (e.g., 512->256->128). After each linear transformation, a normalization layer (e.g., layer normalization) and an activation function layer (e.g., ReLU) are typically added to stabilize the training process and introduce non-linearity. Simultaneously, the introduction of dropout layers randomly discards some neurons, further preventing the model from overfitting on small sample data. Finally, the last linear layer maps the feature dimension to the number of categories in the classification task (e.g., 4 categories: malignant, pleomorphic adenoma, adenolymphoma, and other benign), and outputs the probability distribution of each category through the Softmax function, taking the category with the highest probability as the final classification result.

[0136] It's important to note that the input of the classification module is connected to the output of the shared encoder. This seemingly simple connection actually contains profound constraint logic. The deep semantic feature map output by the shared encoder (especially in the third encoding stage) has already undergone implicit training for the segmentation task. During model training, the segmentation decoder requires that the features extracted by the shared encoder accurately locate the spatial boundaries of the lesion. Therefore, in the feature map output by the shared encoder, the feature response values ​​of the lesion region are significantly higher than those of the background region. When the classification module is directly connected to the output of the shared encoder, it actually receives a feature map with "spatial location markers." When processing this feature map, the heterogeneous multi-scale context fusion module in the classification module naturally focuses on the region with high feature response (i.e., the core target area of ​​the lesion), while ignoring the background region with low feature response. This mechanism forces the classification branch to focus on the lesion, rather than being misled by surrounding lymph nodes, metal artifacts, and other interfering information. This is the specific meaning of "spatial anatomical constraint": utilizing the segmentation task's sensitivity to spatial location to guide the classification task to focus on the correct anatomical region. Through this design, this embodiment effectively solves the problems of isolated operation of classification tasks and low accuracy due to noise interference in the prior art, and significantly improves the diagnostic robustness of the model in complex clinical scenarios.

[0137] Example 5

[0138] This embodiment also provides a device including a memory and a processor. Specifically, the memory is the data storage center of a computer device, which stores not only the computer program code (i.e., software instructions) used to execute the method of the present invention, but also the pre-trained weight parameters of the heterogeneous perceptual mapping network model, the model architecture configuration file, the sample dataset (for continuous optimization or validation of the model), and intermediate feature data, segmentation masks, and classification result logs generated during operation. The memory may include various types of storage media, such as random access memory (RAM) for providing high-speed operating space for the processor and caching the medical image data being processed and the intermediate layer feature maps of the model; read-only memory (ROM) or flash memory for permanently storing the basic input / output system (BIOS) of the computer program and the core parameters of the model; and high-capacity hard disks or solid-state drives (SSDs) for persistently storing massive historical case image databases and trained model files. In this embodiment, the memory stores a computer program configured to be executed by the processor, and the computer program includes instructions for performing the method as described in any of the embodiments 1-4 above. These instructions exist in the form of binary code or machine code compiled from a high-level language, precisely describing every logical step from image preprocessing, feature extraction, attention calculation to the final output.

[0139] The processor is configured to execute computer programs stored in memory. The processor is the logical operation core of a computer device, responsible for parsing and executing instruction sequences in memory. In this embodiment, when executing the program, the processor will progressively call various modules of the heterogeneous perceptual mapping network model according to the processes described in Embodiments 1 to 4. Specifically, the processor first reads the parotid ultrasound image data to be processed, executes preprocessing instructions such as normalization and cropping, and generates a standard input image. Subsequently, the processor calls the instructions of the shared encoder to control the computing unit (such as the CPU core or the GPU stream processor) to perform a large number of convolution operations, pooling operations, and complex self-attention matrix operations. Especially for the dual self-attention module involved in Embodiment 3, the processor needs to efficiently complete the linear projection, multi-scale depth-separable convolution, and dot product attention calculations in the multi-level spatial attention module, which often requires the processor to have powerful parallel computing capabilities or be equipped with a dedicated graphics processing unit (GPU) accelerator card. In the segmentation and decoding stage, the processor executes transposed convolution and feature fusion instructions to progressively restore deep features to a high-resolution segmentation mask. Simultaneously, the processor executes the instructions of the classification module in parallel, extracts lesion features using the heterogeneous multi-scale context fusion module, and outputs a classification probability vector through a fully connected layer. Finally, the processor writes the segmentation mask and classification results to a designated area of ​​memory or sends them to a display device via an output interface for clinicians to view.

[0140] This embodiment also provides a storage medium, which is one of the carrier forms for implementing the parotid gland segmentation and classification method based on heterogeneous perceptual mapping networks described in embodiments 1 to 4 above. It should be understood that although embodiment 5 described a computer device as a hardware entity, the storage medium described in this embodiment focuses on the software carrier that carries the core algorithm logic. It can exist independently of a specific hardware device or be embedded inside a hardware device, forming an indispensable part of the technical solution of this invention.

[0141] Storage media, including computer programs / instructions, specifically refers to the physical medium used to store program code and data. Computer programs / instructions are sequences of machine instructions encoded in binary or compiled from a high-level language and capable of being recognized and executed by a computer processor. These instruction sequences precisely define the topological parameters of the heterogeneous perceptual mapping network model (such as kernel size, number of channels, and hierarchical connections), model weight data (such as the trained neural network parameter matrix), and specific operational logic (such as the forward inference process and attention calculation formula). In this embodiment, the computer programs / instructions stored in the storage medium, when executed by the processor, implement the parotid gland segmentation and classification method based on heterogeneous perceptual mapping networks as described in any of Embodiments 1-4.

[0142] Specifically, when the processor reads and executes the instructions in the storage medium, it triggers a series of specific hardware operations, thereby logically reproducing the method flow of the aforementioned embodiments. For example, when the instructions are executed, the processor is first controlled to read the parotid ultrasound image data to be processed, perform preprocessing operations such as normalization and cropping, and generate a standard input image. Subsequently, the instructions control the processor to call the parameters of the shared encoder, perform double convolution operations and double self-attention operations, and extract multi-scale features. In particular, the instructions guide the processor to calculate multi-level spatial attention weights and perform global feature gating modulation according to the algorithm logic described in Embodiment 3, thereby achieving accurate perception of the complex geometric shape of the parotid gland. Next, the instructions control the segmentation decoder to perform transposed convolution upsampling and feature fusion, and output a segmentation mask; at the same time, the parallel control classification module performs heterogeneous multi-scale context fusion and fully connected classification, and outputs the classification result. Through this "instruction-driven hardware" approach, the storage medium transforms the abstract algorithmic ideas into concrete physical operation processes, realizing intelligent analysis of parotid gland images.

[0143] It should be understood that the storage medium described in this embodiment has a wide variety of physical forms, aiming to cover various possible software distribution and application scenarios. As one implementation, the storage medium can be a magnetic storage medium, such as a hard disk, magnetic tape, or floppy disk. These media have large capacity and low cost, making them suitable for storing massive training datasets and model parameter libraries, facilitating offline training and large-scale deployment of models. As another implementation, the storage medium can be an optical storage medium, such as an optical disc (CD-ROM, DVD-ROM) or Blu-ray disc. These media have good portability and long-term preservation, making them suitable as a distribution carrier for software products, delivering the algorithm program of this invention to medical institutions or research units. As yet another implementation, the storage medium can be a semiconductor storage medium, such as a solid-state drive (SSD), USB flash drive, SD card, or embedded flash memory. These media have fast read / write speeds and small size, making them suitable for integration into portable medical imaging equipment or embedded diagnostic terminals to achieve real-time on-site inference. Furthermore, the storage medium can also be a virtual storage space in a cloud storage server, transmitting instructions over a network to achieve remote cloud-based diagnostic services. This embodiment does not limit the specific physical form of the storage medium; as long as it can stably store computer programs / instructions and allow the processor to read and execute them, it falls within the protection scope of this embodiment.

[0144] Through the design of the aforementioned storage medium, the barriers to dissemination and application of the technical solution of this invention are significantly reduced. Medical institutions do not need to develop complex algorithms themselves; they only need to acquire the storage medium and install it in their existing computer equipment to quickly deploy a high-precision parotid gland segmentation and classification system. This "software as a service" model greatly promotes the popularization of advanced medical AI technology in primary hospitals, solving the problems of difficult implementation and high promotion costs of existing technologies. At the same time, the replicability of the storage medium also makes iterative upgrades of the technical solution convenient. Only by updating the program instructions in the medium can the performance of the model be optimized (such as improving segmentation accuracy or adding new lesion classification categories), without replacing hardware equipment, and it has good scalability and maintainability.

[0145] In summary, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A parotid gland segmentation and classification method based on heterogeneous perceptual mapping networks, characterized in that, include: Acquire the parotid gland ultrasound image data to be processed, and preprocess it to obtain the standard input image; The standard input image is input into a heterogeneous perceptual mapping network model for integrated analysis to obtain an output feature map; the integrated analysis includes segmentation and classification tasks executed simultaneously, and the output feature map includes a segmentation mask and classification results; The heterogeneous sensing mapping network model is a multi-task model trained based on sample data and corresponding annotations, including: A shared encoder is used to extract multi-scale features from the standard input image and generate a deep semantic feature map. The shared encoder includes at least a cascaded first type of encoding stage and a second type of encoding stage. The first type of encoding stage uses a dual convolutional module and a pooling layer to extract local detail features, and the second type of encoding stage uses a dual self-attention module and a pooling layer to extract global context features. A segmentation decoder, which is symmetrically connected to the shared encoder, is used to progressively upsample and fuse the deep semantic feature map to restore the spatial resolution of the standard input image and output the segmentation mask. A classification module, the input of which is connected to the output of the shared encoder, is used to perform spatial anatomical constraints based on the deep semantic feature map to focus on the lesion region for classification and output the classification result.

2. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to claim 1, characterized in that: The first type of encoding stage includes several sequentially connected dual convolutional coding layers, each of which includes a dual convolutional module and a max pooling downsampling module whose input is connected to the output of the dual convolutional module; The dual convolution module includes a 3×3 convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 depthwise separable convolutional layer, a layer normalization layer, and a ReLU activation function layer connected in sequence.

3. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to claim 2, characterized in that: The segmentation decoder is provided with an inverse double convolution decoding layer that corresponds one-to-one with the double convolution coding layer. Each of the inverse double convolution decoding layers includes a transposed convolution module and an inverse double convolution module whose input is connected to the output of the transposed convolution module. The inverse double convolution module includes a 3×3 depth-separable convolutional layer, a layer normalization layer, a ReLU activation function layer, a 3×3 convolutional layer, a layer normalization layer, and a ReLU activation function layer connected in sequence. The output of the dual convolution module in the dual convolution coding layer is connected to the input of the corresponding inverse dual convolution decoding layer.

4. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to any one of claims 1-3, characterized in that: The second type of encoding stage includes several sequentially connected double convolutional self-attention encoding layers, each of which includes a double self-attention module, a double convolution module, and a max pooling downsampling module connected in sequence. The segmentation decoder is provided with an inverse double convolutional self-attention decoding layer that corresponds one-to-one with the double convolutional self-attention coding layer. The inverse double convolutional self-attention decoding layer includes a transposed convolutional module, an inverse double convolutional module, and a double self-attention module connected in sequence. The output of the dual self-attention module in the dual convolutional self-attention coding layer is connected to the input of the inverse dual convolution module in the corresponding inverse dual convolutional self-attention decoding layer.

5. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to claim 4, characterized in that: The dual self-attention module extracts features using the following steps: Perform layer normalization on the input feature map; The layer-normalized feature map is input into a multi-level spatial attention module to obtain the first intermediate feature; The first intermediate feature is concatenated with the input feature map through channels, and after layer normalization, it is input into the global feature gate perception module. The local context information is encoded by depthwise separable convolution and the feature response is adaptively modulated by the gate unit to obtain the second intermediate feature. The second intermediate feature and the first intermediate feature are concatenated by channels, and after layer normalization, they are input into a multi-level channel attention module. The channel dimension is modeled based on the transposed attention mechanism to obtain the third intermediate feature. The third intermediate feature is concatenated with the second intermediate feature, and after layer normalization, it is input again into the global feature gating perception module for feature modulation to obtain the fourth intermediate feature. The fourth intermediate feature and the third intermediate feature are concatenated to output the final dual self-attention feature map.

6. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to claim 5, characterized in that: The step of inputting the layer-normalized feature map into a multi-level spatial attention module to obtain the first intermediate feature is implemented through the following steps: The layer-normalized feature map is then projected using linear projection. Mapped to query features Key features Value characteristics ; The query features Key features Value characteristics Divided into multiple feature groups according to channel dimension ; For the key features and value features in each feature group, multi-scale feature extraction is performed using depthwise separable convolutions with different kernel sizes. After batch normalization and ReLU activation function processing, the enhanced key features and enhanced value features are output. The query features of each feature group, as well as the enhanced key features and enhanced value features, are concatenated along the channel dimension to obtain the aggregated query features, aggregated key features, and aggregated value features. The Softmax function is applied to the aggregated query features and aggregated key features respectively. Attention weights are obtained by calculating attention weights using the dot product attention method. The attention weights are then multiplied by the aggregated value features to obtain the first intermediate feature.

7. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to any one of claims 1-3 or 5, characterized in that: The shared encoder also includes a third type of encoding stage, whose input is connected to the output of the second type of encoding stage and whose output is connected to the input of the segment decoder. The third type of encoding stage includes a dual attention module.

8. The parotid gland segmentation and classification method based on heterogeneous perceptual mapping network according to claim 7, characterized in that: The classification module includes a heterogeneous multi-scale context fusion module connected to the output of the third type of encoding stage, and a classifier connected to the output of the heterogeneous multi-scale context fusion module. The heterogeneous multi-scale context fusion module includes: The first convolutional branch, which uses a convolutional kernel of the first size, is used to extract local detail features of the input features; The second convolutional branch, which uses a convolutional kernel of a second size larger than the first size, is used to extract global contextual features of the input features. The first convolutional branch and the second convolutional branch are set in parallel, and their outputs are spliced ​​and fused along the channel dimension; The classifier includes: The feature compression unit is used to convert the fused feature map output by the heterogeneous multi-scale context fusion module into a feature vector of fixed dimension. The fully connected classification unit, connected to the output of the feature compression unit, consists of multiple alternating linear layers, normalization layers, activation function layers, and Dropout layers. It is used to progressively compress the dimensionality and semantically map the feature vector, and output the corresponding classification result.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1-8.

10. A storage medium comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the parotid gland segmentation and classification method based on heterogeneous perceptual mapping network as described in any one of claims 1-8.

Images