Nasal endoscopic secretion segmentation method based on anatomical structure perception

Abstract

The invention discloses a nasal endoscopic secretion segmentation method based on anatomical structure perception. The method comprises the following steps: acquiring a plurality of training samples; the method comprises the following steps of: generating anatomical sub-region representation and a pseudo tag with spatial and semantic consistency through an anatomical structure modeling mechanism according to a nasal endoscope image and a secretion tag; dissecting the sub-region representation and the pseudo tag, and generating multi-scale image features through a multi-head self-attention and position enhancement adaptation mechanism; carrying out cross-region contrast learning based on the nasal endoscope image so as to extract a multi-scale prediction mask from the multi-scale image features; and fusing the multi-scale prediction masks, and determining secretions in the nasal endoscope image. Therefore, through a progressive technical path of structured modeling (anatomical prior), adaptive perception (regional dynamics), relation learning (cross-regional comparison) and information fusion (multi-scale), the core problem in nasal endoscopic secretion segmentation is systematically solved, and the accuracy of secretion identification in a nasal endoscopic image is improved.

Description

Technical Field

[0001] This invention belongs to the field of nasal cavity image processing, and particularly relates to a method for endoscopic secretion segmentation based on anatomical structure perception. Background Technology

[0002] With the continuous development of medical imaging technology, nasal endoscopy has become an indispensable tool in otolaryngological clinical diagnosis. During the examination of rhinitis and sinusitis, the distribution, color, and morphological changes of nasal secretions (such as clear nasal discharge (CND) and purulent discharge (PUS)) are important imaging indicators reflecting the degree of inflammation and disease progression. However, secretions in nasal endoscopic images are typically transparent, reflective, have blurred boundaries, and exhibit varied morphologies. Furthermore, they are attached to complex nasal anatomical structures (including the central cavity, nasolabial groove, and lateral turbinate walls), posing significant challenges to visual observation and manual annotation by physicians.

[0003] To improve the accuracy of nasal secretion identification, it is usually necessary to directly detect the collected nasal secretions. For example, patent CN118888124B discloses a deep learning-based nasal secretion smear diagnostic method and system. This method uses a sliding window approach to read the nasal secretion smear in blocks. The read window images are preprocessed and input into a trained cell detection model. A series of feature extractions are performed on the input images, and the pyramid feature map is enhanced through bottom-up and top-down feature fusion. Finally, the coordinates and scores of the bounding boxes of detected targets are obtained based on the feature maps at each scale. Bounding boxes below a set score threshold are filtered out, resulting in a bounding box set R. Based on the coordinates of the bounding boxes in R, corresponding images are cropped from the original window images, preprocessed, and input into a trained cell classification model. The model outputs the cell categories of the corresponding images, which are then post-processed to obtain the cell categories of the original nasal secretion smear to be diagnosed. This allows for efficient and accurate diagnosis of nasal secretion smears.

[0004] However, this method still requires sampling and analysis of the secretions themselves, and the process remains quite complex. Summary of the Invention

[0005] This invention provides a method for segmenting nasal secretions based on anatomical structure perception, which improves the accuracy of nasal secretion identification.

[0006] According to a first aspect of the present invention, embodiments of the present invention provide a method for endoscopic nasal secretion segmentation based on anatomical structure perception, which may include: Multiple training samples were acquired. Each training sample included a nasal endoscopy image and a secretion label corresponding to each nasal endoscopy image. The secretion label included the secretion type and the secretion region. Nasal endoscopy images and secretion labels are used to generate anatomical sub-region representations and pseudo-labels with spatial and semantic consistency through an anatomical structure modeling mechanism. Anatomical sub-region representations and pseudo-labels are used to generate multi-scale image features through multi-head self-attention and location-enhanced adaptation mechanisms. Cross-regional contrastive learning based on nasal endoscopy images is used to extract multi-scale prediction masks from multi-scale image features. Multi-scale prediction masks are fused to identify secretions in nasal endoscopy images.

[0007] Optionally, multiple training samples can be obtained, including: Multiple clinical diagnostic images were acquired, each with a corresponding secretion label; Based on the clarity of secretions in clinical diagnostic images, multiple nasal endoscopic images with clarity meeting preset requirements are extracted from multiple clinical diagnostic images; According to a preset ratio, multiple training samples are extracted from multiple nasal endoscopy images based on the type of secretion.

[0008] Optionally, based on nasal endoscopic images and secretion labels, anatomical sub-region representations and pseudo-labels with spatial and semantic consistency are generated through an anatomical structure modeling mechanism, including: Based on the nasal cavity anatomy, the nasal endoscopy images were divided into multiple initial sub-regions; Spatial features are extracted from each initial sub-region using a spatial encoder; Semantic features are extracted from each initial sub-region using a semantic encoder based on secretion labels. The product of spatial features and spatial feature weights, plus the product of semantic features and semantic feature weights, is used to represent the anatomical sub-region. The sum of the spatial feature weights and semantic feature weights is 1. Based on the anatomical sub-region representation, pseudo-labels are generated.

[0009] Optionally, pseudo-labels are generated based on the anatomical sub-region representation, including: By introducing a clustering mechanism containing N topic prototypes, the soft assignment number between each feature representation and each topic prototype in each initial sub-region is calculated to determine the semantic clusters corresponding to different nasal anatomical structures. Based on the anatomical sub-region representation, a probability analysis is performed on each initial sub-region to obtain pseudo-labels.

[0010] Optionally, a probabilistic analysis is performed on each initial sub-region based on the anatomical sub-region representation to obtain pseudo-labels, including: By performing probability calculations on the anatomical sub-region representation using a multi-scale decoder, a probability map of secretions belonging to different secretion types in the initial sub-region is obtained. The probability maps of each secretion are fused to obtain pseudo-labels.

[0011] Optionally, the multi-scale image features include: semantic multi-scale image features and location multi-scale image features; Anatomical sub-region representations and pseudo-labels are used to generate multi-scale image features through multi-head self-attention and location-enhanced adaptation mechanisms, including: By using a multi-head self-attention submodule to perform correlation analysis on multiple initial sub-regions within the same nasal cavity anatomical structure, an enhanced semantic feature region is obtained. By adapting multi-layer coded features using a linear adapter and performing residual fusion, an enhanced location feature region is obtained.

[0012] Optionally, cross-regional contrastive learning is performed based on nasal endoscopy images to extract a multi-scale prediction mask from multi-scale image features, including: Image transformation and enhancement are performed on nasal endoscopy images to obtain enhanced images; The consistency loss is determined based on the cosine distance between the enhanced image and the nasal endoscopy image; The top-level features of the nasal endoscopy image are decoupled to obtain foreground features, background features, and uncertainty features; The constrained loss is calculated based on the foreground characteristics, background characteristics, and uncertainty characteristics. Contrast loss is obtained by comparing foreground and background features; When the constraint loss and contrastive loss satisfy the training stopping condition, a multi-scale prediction mask is obtained.

[0013] Optionally, a consistency loss is determined based on the cosine distance between the enhanced image and the nasal endoscopy image, including:

[0014] in, Features of nasal endoscopy images To enhance the image with Corresponding characteristics.

[0015] Optionally, the constraint loss is calculated based on foreground features, background features, and uncertainty features, including: The constraint loss is calculated using the following formula;

[0016] in, Foreground features, For background features, Uncertainty characteristics To limit losses, These are the weighting coefficients.

[0017] Optionally, the multi-scale prediction mask includes small-scale masks, medium-scale masks, and large-scale masks; Multi-scale prediction masks are fused to identify secretions in nasal endoscopic images, including: The small-scale mask, medium-scale mask, and large-scale mask are weighted and fused to obtain the secretion segmentation mask; The secretion is determined based on the pixel values ​​of the secretion segmentation mask.

[0018] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: This invention provides a method for nasal secretion segmentation based on anatomical structure awareness, which involves acquiring multiple training samples; generating anatomical sub-region representations and pseudo-labels with spatial and semantic consistency through an anatomical structure modeling mechanism using nasal endoscopy images and secretion labels; generating multi-scale image features from the anatomical sub-region representations and pseudo-labels through a multi-head self-attention and position enhancement adaptation mechanism; performing cross-regional contrastive learning based on nasal endoscopy images to extract multi-scale prediction masks from the multi-scale image features; and fusing the multi-scale prediction masks to determine the secretions in the nasal endoscopy images. Based on this, the generated spatially and semantically consistent anatomical sub-region representations and pseudo-labels provide the model with strong semantic guidance at the region level, helping to overcome the problems of blurred secretion boundaries and varied morphologies. Employing a multi-head self-attention and positional enhancement adaptation mechanism, the model can adaptively capture the morphological and textural differences of secretions in different regions and enhance positional information, thereby effectively suppressing the influence of interference factors such as reflection. The cross-regional contrastive learning strategy models the relationship between secretions and surrounding tissues and applies feature similarity constraints, enabling the model to learn common features across regions, ensuring consistency and coordination in segmentation results across different anatomical regions. By generating and fusing multi-scale prediction masks, the model can comprehensively utilize the details preserved by high-resolution features (small scale) and the rich semantic information contained in deep features (large scale), achieving comprehensive perception from local details to global context. Thus, through a progressive technical path of structured modeling (anatomical prior) → adaptive perception (regional dynamics) → relationship learning (cross-regional contrast) → information fusion (multi-scale), the core challenges in nasal endoscopic secretion segmentation are systematically solved, improving the accuracy of secretion identification in nasal endoscopic images.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention, but do not constitute an undue limitation of the invention.

[0021] Figure 1 This is a flowchart illustrating an anatomically aware endoscopic secretion segmentation method according to an exemplary embodiment; Figure 2 This is a schematic diagram of a nasal endoscope secretion segmentation device based on anatomical structure perception, according to an exemplary embodiment. Figure 3 This is a structural block diagram of a nasal endoscopic secretion segmentation device based on anatomical structure perception, according to an exemplary embodiment. Detailed Implementation

[0022] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0023] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0024] As described in the background section, secretions in nasal endoscopy images are typically transparent, reflective, have blurred boundaries, and vary in shape. They are also attached to complex nasal anatomical structures (including the central cavity, nasal groove, and turbinate lateral wall), posing a significant challenge to doctors' visual observation and manual annotation.

[0025] Based on this, the present invention provides a nasal endoscopic secretion segmentation method based on anatomical structure perception. The following is a description of the nasal endoscopic secretion segmentation method based on anatomical structure perception provided by an embodiment of the present invention.

[0026] like Figure 1 As shown; Example 1; It may include the following steps: S101, acquire multiple training samples, each training sample includes a nasal endoscopy image and a secretion label corresponding to each nasal endoscopy image, the secretion label includes secretion type and secretion region; S102, nasal endoscopy images and secretion labels, generate anatomical sub-region representations and pseudo-labels with spatial and semantic consistency through an anatomical structure modeling mechanism; S103, anatomical sub-region representation and pseudo-labels, generates multi-scale image features through multi-head self-attention and location enhancement adaptation mechanism; S104, perform cross-regional contrast learning based on the nasal endoscopy image to extract a multi-scale prediction mask from multi-scale image features; S105, the multi-scale prediction mask is fused to determine the secretions in the nasal endoscopy image.

[0027] Based on the above embodiments, multiple training samples are obtained; nasal endoscopy images and secretion labels are used to generate anatomical sub-region representations and pseudo-labels with spatial and semantic consistency through an anatomical structure modeling mechanism; the anatomical sub-region representations and pseudo-labels are used to generate multi-scale image features through multi-head self-attention and position enhancement adaptation mechanisms; cross-regional contrastive learning is performed based on nasal endoscopy images to extract multi-scale prediction masks from multi-scale image features; the multi-scale prediction masks are fused to determine the secretions in the nasal endoscopy images. Based on this, the generated spatially and semantically consistent anatomical sub-region representations and pseudo-labels provide the model with strong semantic guidance at the region level, helping to overcome the problems of blurred secretion boundaries and varied morphologies. Employing a multi-head self-attention and positional enhancement adaptation mechanism, the model can adaptively capture the morphological and textural differences of secretions in different regions and enhance positional information, thereby effectively suppressing the influence of interference factors such as reflection. The cross-regional contrastive learning strategy models the relationship between secretions and surrounding tissues and applies feature similarity constraints, enabling the model to learn common features across regions, ensuring consistency and coordination in segmentation results across different anatomical regions. By generating and fusing multi-scale prediction masks, the model can comprehensively utilize the details preserved by high-resolution features (small scale) and the rich semantic information contained in deep features (large scale), achieving comprehensive perception from local details to global context. Thus, through a progressive technical path of structured modeling (anatomical prior) → adaptive perception (regional dynamics) → relationship learning (cross-regional contrast) → information fusion (multi-scale), the core challenges in nasal endoscopic secretion segmentation are systematically solved, improving the accuracy of secretion identification in nasal endoscopic images.

[0028] In S101 above, multiple training samples can be extracted from the medical diagnosis report, and each training sample has its corresponding secretion label.

[0029] Specifically, the medical diagnostic report includes nasal endoscopy images obtained during a nasal examination, as well as diagnostic information made by the doctor based on the nasal endoscopy images. The doctor's diagnostic information is the secretion label corresponding to the nasal endoscopy image. The secretion label includes the type of secretion and the area where the secretion is located in the nasal cavity.

[0030] The types of discharge can include clear discharge and purulent discharge; the areas of discharge can include the central cavity, the nasolabial groove, and the lateral wall of the nail.

[0031] In S102 above, similar to the criteria for dividing the secretion region, the nasal endoscopy image can be divided into multiple initial sub-regions based on the nasal cavity anatomical features. This allows the corresponding initial sub-regions to be found based on the secretion region in the secretion label. Each initial sub-region has a corresponding secretion label. The initial sub-region is spatial, and the secretion label is semantic. Thus, the model performs reinforcement learning on the initial sub-regions and their corresponding secretion labels to generate spatially and semantically consistent anatomical sub-region representations and pseudo-labels.

[0032] The anatomical structure modeling mechanism achieves region-level structural representation by constructing an anatomical structure-aware encoder. This encoder is built through explicit region location embedding and channel attention mechanisms, enabling the model to recognize the boundary constraints and spatial layout relationships of different anatomical regions.

[0033] In S103 above, the anatomical sub-region representation and pseudo-label related to the initial sub-region in the nasal endoscopy image are enhanced by capturing information of different dimensions through multi-head self-attention, and then adaptive modeling is performed, so that the model can generate image features of different scales, enabling the model to better identify the morphological changes and texture differences of secretions in different nasal anatomical features, thereby enhancing the robustness of the model to complex visual features such as reflection and blur.

[0034] Specifically, adaptive modeling can be achieved by introducing a Regional Dynamic Perception (RDP) module into the model.

[0035] Among them, multi-head self-attention refers to a deep learning mechanism that captures the internal correlation of a sequence by processing multiple attention heads in parallel and dynamically assigning weights.

[0036] In S104 above, a visual relationship guidance network is constructed, and a cross-regional contrastive learning strategy is used for modeling, enabling the model to analyze the relationship between secretions and surrounding tissues from multi-scale image features. Specifically, through feature similarity constraints and a contrastive loss function, the model can distinguish the secretion boundary from background tissue, strengthening cross-regional consistent expression.

[0037] A mask is essentially a pixel-level "indicator image" that uses the simplest "yes / no" logic to precisely separate foreground objects from the background in an image. In medical image segmentation, obtaining a high-quality mask is equivalent to accurately identifying and delineating lesions or targets.

[0038] Multi-scale masks refer to masks of different sizes and precision.

[0039] In step S105 above, multi-layer anatomical perception features and visual relationship features are integrated, and joint optimization is performed using structural consistency constraints to output accurate segmentation results for the secretion region. This process achieves structured segmentation prediction from local to global perspectives, taking into account both fine-grained boundaries and global consistency.

[0040] Alternatively, in one example, S101 may include: S1011, acquire multiple clinical diagnostic images, each of which has a corresponding secretion label; S1012, Based on the clarity of secretions in clinical diagnostic images, extract multiple nasal endoscopy images with clarity meeting preset requirements from multiple clinical diagnostic images; S1013, according to a preset ratio, extracts multiple training samples from multiple nasal endoscopy images based on the type of secretion.

[0041] In S1011 above, clinical diagnostic images from the ENT department of a tertiary-level Class A hospital were used as research data. All clinical diagnostic images were acquired using a high-definition electronic nasal endoscope under natural lighting and constant exposure conditions, with a uniform image resolution of 1920×1080 pixels. Based on each clinical diagnostic image, the doctor would make a corresponding medical diagnosis, which could include the type and area of ​​the secretion, i.e., a secretion label.

[0042] In S1012 above, in order to ensure the quality and structural integrity of the training samples, high-resolution clinical diagnostic images must be selected from the clinical diagnostic images before they can be used as nasal endoscopy images for training analysis.

[0043] Specifically, meeting the preset requirements for clarity means that the image must contain a clearly identifiable area of ​​secretions, without obvious motion blur or overexposure, occlusion, and ensure that the nasal cavity structure is intact and key areas such as the nasal passages, turbinates, and nasopharynx can be clearly identified.

[0044] In S1013 above, different types of secretions are extracted from multiple nasal endoscopy images according to a preset ratio to obtain multiple training samples.

[0045] For example, among 1000 training samples, there were 650 training samples with clear discharge and 350 training samples with purulent discharge.

[0046] Alternatively, in one example, the above S102 may include: S1021, based on the nasal cavity anatomy, the nasal endoscopy image is divided into multiple initial sub-regions; S1022, spatial features are extracted from each initial sub-region using a spatial encoder; S1023, semantic features are extracted from each initial sub-region by a semantic encoder based on the secretion label; S1024, determine the product of spatial features and spatial feature weights, add the product of semantic features and semantic feature weights, to obtain the anatomical sub-region representation, and the sum of spatial feature weights and semantic feature weights is 1; S1025, Generate pseudo-labels based on the anatomical sub-region representation.

[0047] In the above S1021, the nasal cavity can be structurally divided into the central cavity region, the nasal groove region, and the turbinate lateral wall region. The nasal endoscope image is divided according to these regions to form multiple initial sub-regions.

[0048] In S1022 above, the spatial encoder mainly captures geometric and positional information, focusing on the relative positions between pixels, the boundaries of regions, and the absolute and relative coordinates of sub-regions within the entire image. Spatial features can be extracted from each initial sub-region using the spatial encoder.

[0049] In S1023 above, the semantic encoder mainly captures visual content and semantic information, using deeper network layers or channel attention mechanisms. These layers can aggregate the semantics represented by the secretion labels to form higher-level semantic concepts. It can recognize the semantics of the initial sub-region, such as the semantics of the initial sub-region being "mucosa", "purulent secretion", or "clear secretion".

[0050] In S1024 above, the anatomical sub-region representation is obtained by calculation using the following formula.

[0051]

[0052] in, This represents the anatomical sub-region. For spatial features, For semantic features, This is a balance coefficient between spatial and semantic features, which can be adaptively adjusted according to task complexity.

[0053] The anatomical subregion representation is a feature vector that integrates spatial location and visual semantic information. It is a digital description of the initial subregion in a nasal endoscopy image and is the cornerstone of the model's "anatomical structure perception" because it allows the model not only to "see" the image content but also to "understand" the specific location and semantics of that content within the anatomical structure.

[0054] In S1025 above, based on the representation of each sub-region, it is determined whether the initial sub-region contains secretions, and thus the pseudo-label is determined according to the probability of whether secretions exist.

[0055] The pseudo-label is a preliminary, pixel-level segmentation map of secretions generated by the model based on the anatomical structure perception module. It acts as a bridge between "structural understanding" and "fine segmentation," driving the entire model to perform more accurate and robust bootstrapping learning by providing internal supervision signals.

[0056] Alternatively, in one example, the above S1025 may include: S10251, by introducing a clustering mechanism containing N topic prototypes, calculates the soft allocation number between each feature representation and each topic prototype in each initial sub-region, so as to determine the semantic clusters corresponding to different nasal anatomical structures; S10252, based on the anatomical sub-region representation, performs probability analysis on each initial sub-region to obtain pseudo-labels.

[0057] In S10251 above, to enhance semantic consistency within the region and capture the topic semantic clustering structure, this invention introduces a Pull-Topic clustering mechanism. Given a set of topic prototypes... The initial subregion p is relative to the prototype The soft allocation is defined as:

[0058] This mechanism can form separable clusters of regional themes in the feature space, improving the model's ability to semantically recognize different functional areas of the nasal cavity.

[0059] The Pull-Topic clustering mechanism refers to Pulsar's Topic clustering mechanism, which is mainly implemented through Partitioned Topic. Its core is to split a single Topic into multiple partitions, which are then processed in parallel by multiple Broker nodes to improve throughput and scalability.

[0060] Topic prototypes are a set of feature vectors learned autonomously by the model. Each vector represents a typical visual theme of a nasal cavity image, such as purulent secretions in the central cavity or clear secretions in the nasal groove. These are different topic prototypes. Through the Pull-Topic clustering mechanism, they act as "magnetic centers," attracting sub-regions with similar visual features together, thus forming semantically meaningful cluster structures in the feature space. This greatly enhances the model's feature representation ability and semantic recognition ability.

[0061] In S10252 above, the probability of different subject prototype secretions in each initial sub-region is analyzed, thereby further improving the accuracy of secretion identification.

[0062] Optionally, in one example, S10252 above further includes: By performing probability calculations on the anatomical sub-region representation using a multi-scale decoder, a probability map of secretions belonging to different secretion types in the initial sub-region is obtained. The probability maps of each secretion are fused to obtain pseudo-labels.

[0063] based on The Patch classification head outputs probability vectors for each sub-region regarding secretions and the three types of secretion regions (central cavity, nasal groove, and lateral turbinate wall):

[0064] Based on the above sub-region probabilities and topic assignments, this invention employs a multi-scale decoder to predict small / medium / large-scale secretion probability maps. , as well as Then they are merged into the final pseudo-tag: Z = σ( + + ) Where σ is a pixel-wise activation function (such as sigmoid).

[0065] Small-scale secretion probability map Focusing on local details and boundary information, mesoscale secretion probability maps Balancing local and contextual information, large-scale secretion probability maps Pay attention to the overall structure and regional distribution.

[0066] Optionally, in one example, the multi-scale image features include: semantic multi-scale image features and location multi-scale image features; The above S103 may include: S1031, using a multi-head self-attention sub-module to perform correlation analysis on multiple initial sub-regions within the same nasal cavity anatomical structure, thereby obtaining an enhanced semantic feature region; S1032, by adapting the multi-layer coding features through a linear adapter and performing residual fusion, an enhanced location feature region is obtained.

[0067] In S1031 above, the anatomical sub-region is represented. The system constructs a sequence of regions as input and captures the visual relationships and contextual dependencies of the initial sub-regions through a multi-head attention (MHA) module, thereby obtaining the enhanced semantic feature regions.

[0068] in, This represents the output of multi-head attention to the p-th sub-region (including residual connections). To enhance semantic feature regions.

[0069] In S1032 above, to enhance position sensitivity, this invention introduces a position-enhanced (pos) linear adapter, which applies lightweight adaptation and residual fusion to the multi-layer coding features layer by layer, formally as follows:

[0070] in For the features of the encoder's l-th layer, For the linear adapter of this layer, To adapt to the intensity, the area dynamic sensing module uses With adapted multi-layer features As input, generate region-aware features for each functional area for use in subsequent stages.

[0071] Alternatively, in one example, the above S104 may include: S1041, Perform image conversion and enhancement on the nasal endoscopy image to obtain an enhanced image; S1042, determine the consistency loss based on the cosine distance between the enhanced image and the nasal endoscopy image; S1043, decouple the top-level features of the nasal endoscopy image to obtain foreground features, background features and uncertainty features; S1044, the constraint loss is calculated based on the foreground features, background features, and uncertainty features; S1045, contrastive loss is obtained by performing contrastive learning on foreground and background features; S1046, under the condition that the contrastive loss satisfies the training stopping condition, obtain the multi-scale prediction mask.

[0072] In S1041 above, the acquired nasal endoscope image is subjected to enhancement operations such as image rotation, flipping, and color perturbation, so that the nasal endoscope image can have multiple enhanced images with the same secretion, but different sizes, shapes, and colors, thereby better training the model and enhancing the model's ability to accurately identify secretions from different images.

[0073] In S1042 above, the similarity between the enhanced image and the nasal endoscopy image is measured by measuring the cosine similarity between them.

[0074] Specifically, the cosine distance can be calculated using the following formula:

[0075] in, Features of nasal endoscopy images To enhance the image with Corresponding characteristics.

[0076] In S1043 above, the top-level features of the nasal endoscopy image are output by the encoder network and then decoupled by a decoupling network to obtain foreground features, background features and uncertainty features.

[0077] Among them, foreground features refer to features related to secretions.

[0078] Background features refer to features associated with normal nasal tissues (mucosa, blood vessels, etc.).

[0079] Uncertain features refer to ambiguous or difficult features that cannot be clearly classified as either foreground or background.

[0080] In S1044 above, the constraint loss can be calculated based on the following formula:

[0081] in, Foreground features, For background features, Uncertainty characteristics To limit losses, These are the weighting coefficients.

[0082] In S1045 above, based on the initial separation of foreground and background features through decoupling, the cosine distance (contrast loss) between the two is calculated through a contrastive learning strategy. The magnitude of the contrast loss can reflect the degree of model's ability to distinguish between the foreground and background in the nasal endoscopy image in the feature space, thereby making the model's segmentation boundary more accurate.

[0083] In S1046 above, the training stopping condition includes the contrast loss being less than a threshold. If the contrast loss meets the training stopping condition, it means that the model can accurately identify the difference between secretions and other background features in the nasal endoscopy image. The model's decoder can identify features related to secretions (multi-scale prediction mask) from the nasal endoscopy image and know that the feature is secretions, not other background features, thus achieving a dual understanding of features and semantics.

[0084] Optionally, in one example, the multi-scale prediction mask includes a small-scale mask, a medium-scale mask, and a large-scale mask; The above S105 may include: S1051, weighted fusion of small-scale mask, medium-scale mask and large-scale mask to obtain secretion segmentation mask; S1052, determine the secretion based on the pixel values ​​of the secretion segmentation mask.

[0085] In S1051 above, the small-scale prediction mask is... The mesoscale prediction mask is Large-scale prediction mask is The secretion segmentation mask is obtained by weighted fusion according to the following formula.

[0086]

[0087] During weighted fusion, pixel-level supervision can be employed using a combination of BCE and Dice:

[0088] Will , Comparative loss and pixel segmentation loss Together, we can form the ultimate training goal:

[0089] In S1052 above, the secretion segmentation mask is a two-dimensional image (or probability map) of the same size as the original image, wherein: A value of 1 for each pixel (or high probability) indicates that the pixel is identified as secretion by the model.

[0090] A value of 0 for each pixel (or low probability) indicates that the pixel is identified by the model as background (non-secretions, such as mucous membranes, tissues, etc.).

[0091] To objectively evaluate model performance, this experiment uses six standard metrics, including Dice, mIoU, Precision, Recall, F1-score, and Accuracy.

[0092] The Dice coefficient (Dice) is commonly used to measure the similarity or overlap between the model's prediction (P) and the ground truth label (G). It is more sensitive to small segmentation targets and is often used in medical image segmentation.

[0093] Where P and G represent the segmentation region predicted by the model and the true label region, respectively. The size of the intersection between the predicted and true labels, i.e. The number of (true positives), The total size of the prediction region, i.e. , The total size of the actual tags, i.e. .

[0094] Mean Intersection over Union (mIoU) is one of the most commonly used metrics in segmentation tasks. It calculates the ratio of the intersection and union of the predicted set and the true set, and then averages the ratios over all categories.

[0095]

[0096] Accuracy (ACC) measures the proportion of pixels that are correctly classified out of the total number of pixels.

[0097]

[0098] In pixel-level calculations, It represents the total number of correctly classified pixels, and the denominator is the total number of pixels in the image.

[0099] Precision measures the proportion of pixels that the model predicts as positive (the segmentation target) that are actually positive. It focuses on the "purity" of the model's predictions.

[0100]

[0101] Recall, also known as recall or sensitivity, measures the proportion of true positive pixels (segmentation targets) that the model successfully predicts. It focuses on the model's ability to cover positive targets.

[0102]

[0103] The F1 score is the harmonic mean of precision and recall. It balances precision and recall and is a commonly used metric for evaluating the overall performance of a model.

[0104]

[0105] Among them, TP, FP, TN, and FN represent the number of true positives, false positives, true negatives, and false negatives at the pixel level, respectively.

[0106] Experimental results.

[0107] After introducing the anatomical structure perception and visual relationship guidance framework proposed in this invention, the overall performance of the nasal endoscopic secretion segmentation task is significantly improved. Specifically, based on the CLZ subset of clear nasal discharge (CND) images, in terms of region segmentation metrics, the mIOU coefficient is improved by 2.4% compared to ConDSeg, mDSC by 2.56%, recall by 0.4%, and precision by 1.56%. However, due to the limited number of images on the TSZ subset, these evaluation metrics are slightly lower than the TransNet method, but the precision is improved by 0.7%.

[0108] In the ablation experiment, the model performance decreased to varying degrees after removing the pos and RDP modules, with mIOU decreasing by 9.07% and 4.44% respectively, verifying the key role of each module in improving structure awareness and location.

[0109] Experimental results show that the framework proposed in this invention can maintain stable segmentation performance under complex lighting, reflection and blurred boundary conditions, significantly improve the localization accuracy and boundary consistency of nasal endoscopic secretions, and verify the effectiveness and clinical applicability of the method.

[0110] Example 2: like Figure 2 As shown, optionally, in one example, the present invention also provides a nasal endoscopic secretion segmentation device 200 based on anatomical structure perception, which includes: The acquisition module 210 is used to acquire multiple training samples. Each training sample includes a nasal endoscopy image and a secretion label corresponding to each nasal endoscopy image. The secretion label includes the secretion type and the secretion region. The first generation module 220 is used for nasal endoscopy images and secretion labels. Through an anatomical structure modeling mechanism, it generates anatomical sub-region representations and pseudo-labels with spatial and semantic consistency. The second generation module 230 is used for anatomical sub-region representation and pseudo-labels, and generates multi-scale image features through multi-head self-attention and position enhancement adaptation mechanisms. Extraction module 240 is used for cross-regional contrast learning based on nasal endoscopy images to extract multi-scale prediction masks from multi-scale image features; The fusion module 250 is used to fuse multi-scale prediction masks to determine secretions in nasal endoscopy images.

[0111] Based on the above embodiments, the acquisition module 210 acquires multiple training samples; the first generation module 220 generates anatomical sub-region representations and pseudo-labels with spatial and semantic consistency based on nasal endoscopy images and secretion labels through an anatomical structure modeling mechanism; the second generation module 230 generates multi-scale image features from the anatomical sub-region representations and pseudo-labels through a multi-head self-attention and position enhancement adaptation mechanism; the extraction module 240 performs cross-regional contrastive learning based on nasal endoscopy images to extract multi-scale prediction masks from the multi-scale image features; and the fusion module 250 fuses the multi-scale prediction masks to determine the secretions in the nasal endoscopy images. Based on this, the generated spatially and semantically consistent anatomical sub-region representations and pseudo-labels provide the model with strong semantic guidance at the region level, helping to overcome the problems of blurred secretion boundaries and varied morphologies. Employing a multi-head self-attention and positional enhancement adaptation mechanism, the model can adaptively capture the morphological and textural differences of secretions in different regions and enhance positional information, thereby effectively suppressing the influence of interference factors such as reflection. The cross-regional contrastive learning strategy models the relationship between secretions and surrounding tissues and applies feature similarity constraints, enabling the model to learn common features across regions, ensuring consistency and coordination in segmentation results across different anatomical regions. By generating and fusing multi-scale prediction masks, the model can comprehensively utilize the details preserved by high-resolution features (small scale) and the rich semantic information contained in deep features (large scale), achieving comprehensive perception from local details to global context. Thus, through a progressive technical path of structured modeling (anatomical prior) → adaptive perception (regional dynamics) → relationship learning (cross-regional contrast) → information fusion (multi-scale), the core challenges in nasal endoscopic secretion segmentation are systematically solved, improving the accuracy of secretion identification in nasal endoscopic images.

[0112] Optionally, the acquisition module 210 may include: The acquisition submodule is used to acquire multiple clinical diagnostic images, each of which has a corresponding secretion label. The first extraction submodule is used to extract multiple nasal endoscopy images with a resolution that meets preset requirements from multiple clinical diagnostic images based on the clarity of secretions in the clinical diagnostic images. The second extraction submodule is used to extract multiple training samples from multiple nasal endoscopy images according to a preset ratio and the type of secretion.

[0113] Optionally, the first generation module 220 may include: The sub-module is used to divide the nasal endoscopy image into multiple initial sub-regions based on the nasal cavity anatomy. The third extraction submodule is used to extract spatial features from each initial sub-region through a spatial encoder; The fourth extraction submodule is used to extract semantic features from each initial sub-region based on the secretion labels using a semantic encoder; The determination submodule is used to determine the product of spatial features and spatial feature weights, plus the product of semantic features and semantic feature weights, to represent the dissected sub-region. The sum of the spatial feature weights and semantic feature weights is 1. The generation submodule is used to generate pseudo-labels based on the anatomical subregion representation.

[0114] Optionally, generating submodules may include: The computational unit is used to calculate the soft assignment number between each feature representation and each topic prototype in each initial sub-region by introducing a clustering mechanism containing N topic prototypes, so as to determine the semantic clusters corresponding to different nasal anatomical structures. The first analysis unit is used to perform probabilistic analysis on each initial sub-region based on the anatomical sub-region representation to obtain pseudo-labels.

[0115] Optionally, the analysis unit may include: The computational subunit is used to perform probability calculations on the anatomical sub-region representation through a multi-scale decoder to obtain a probability map of secretions belonging to different secretion types in the initial sub-region. The fusion subunit is used to fuse the probability maps of various secretions to obtain pseudo-labels.

[0116] Optionally, the multi-scale image features include: semantic multi-scale image features and location multi-scale image features; The second generation module 230 may include: The second analysis unit is used to perform correlation analysis on multiple initial sub-regions within the same nasal cavity anatomical structure through the multi-head self-attention sub-module to obtain enhanced semantic feature regions. The first fusion unit is used to adapt multi-layer coded features through a linear adapter and perform residual fusion to obtain an enhanced location feature region.

[0117] Optionally, the extraction module 240 may include: The enhancement unit is used to perform image conversion and enhancement on nasal endoscopy images to obtain enhanced images; The first determining unit is used to determine the consistency loss based on the cosine distance between the enhanced image and the nasal endoscopy image; The decoupling unit is used to decouple the top-level features of the nasal endoscopy image to obtain foreground features, background features, and uncertainty features; The computational unit is used to calculate the constraint loss based on foreground features, background features, and uncertainty features; The contrastive learning unit is used to perform contrastive learning using foreground and background features to obtain contrastive loss. The obtained unit is used to obtain a multi-scale prediction mask when the constraint loss and contrastive loss meet the training stopping condition.

[0118] Optionally, a consistency loss is determined based on the cosine distance between the enhanced image and the nasal endoscopy image, including:

[0119] in, Features of nasal endoscopy images To enhance the image with Corresponding characteristics.

[0120] Optionally, the constraint loss is calculated based on foreground features, background features, and uncertainty features, including: The constraint loss is calculated using the following formula;

[0121] in, Foreground features, For background features, Uncertainty characteristics To limit losses, These are the weighting coefficients.

[0122] Optionally, the multi-scale prediction mask includes small-scale masks, medium-scale masks, and large-scale masks; The fusion module 250 may include: The second fusion unit is used to perform weighted fusion of small-scale masks, medium-scale masks and large-scale masks to obtain a secretion segmentation mask; The second determining unit is used to determine the secretion based on the pixel values ​​of the secretion segmentation mask.

[0123] In the above embodiment 2, each method described in any one of embodiments 1-2 is included, and each process in any one of embodiments 1-2 can be implemented, achieving the same technical effect. To avoid repetition, it will not be described again here.

[0124] Example 3: Figure 3 The diagram shows a hardware structure schematic of a nasal endoscopic secretion segmentation device based on anatomical structure perception provided by an embodiment of the present invention.

[0125] The anatomically-based nasal endoscopic secretion segmentation device may include a processor 301 and a memory 302 storing computer program instructions.

[0126] Specifically, the processor 301 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0127] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 32 is non-volatile solid-state memory.

[0128] In certain embodiments, memory 302 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Thus, typically, memory 302 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors 301), it is operable to perform the operations described with reference to one aspect of the invention.

[0129] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any of the anatomically-based nasal secretion segmentation methods in the above embodiments.

[0130] In one example, the anatomically-based nasal endoscopic secretion segmentation device may also include a communication interface 303 and a bus 304. As shown in the figure, the processor 301, memory 302, and communication interface 303 are connected via the bus 304 and communicate with each other.

[0131] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0132] Bus 304 may include hardware, software, or both. For example, and not as a limitation, bus 304 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, a Wireless Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Control Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 304 may include one or more buses 304. Although specific bus 304s are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus 304 or interconnect.

[0133] This anatomically-aware nasal secretion segmentation device can be based on a current anatomically-aware nasal secretion segmentation method, thereby achieving a combination of... Figure 1 , 2 This paper describes a method for nasal secretion segmentation based on anatomical structure perception and a nasal secretion segmentation device 200 based on anatomical structure perception.

[0134] In addition, this embodiment of the invention also provides a computer program product, including computer program instructions, which, when executed by processor 301, can implement the steps and corresponding content of the aforementioned method embodiments.

[0135] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0136] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0137] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0138] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, or other programmable processor of an anatomically aware nasal endoscopic secretion segmentation method, apparatus, and device to produce a machine such that these instructions, executable via a computer or other programmable processor of an anatomically aware nasal endoscopic secretion segmentation method, apparatus, and device, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, an application-specific processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0139] In the above embodiment 3, each method described in any one of embodiments 1-3 is included, and each process in any one of embodiments 1-3 can be implemented, achieving the same technical effect. To avoid repetition, it will not be described again here.

[0140] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for endoscopic nasal secretion segmentation based on anatomical structure perception, characterized in that, The steps of the method are as follows: Multiple training samples are acquired, each training sample including a nasal endoscopy image and a secretion label corresponding to each nasal endoscopy image, the secretion label including secretion type and secretion region; The nasal endoscopy images and the secretion labels are used to generate anatomical sub-region representations and pseudo-labels with spatial and semantic consistency through an anatomical structure modeling mechanism. The anatomical sub-region representation and the pseudo-label are used to generate multi-scale image features through a multi-head self-attention and position enhancement adaptation mechanism. Cross-regional contrastive learning is performed based on the nasal endoscopy images to extract multi-scale prediction masks from multi-scale image features; The multi-scale prediction masks are fused to determine the secretions in the nasal endoscopy image.

2. The endoscopic secretion segmentation method based on anatomical structure perception as described in claim 1, characterized in that, The acquisition of multiple training samples includes: Multiple clinical diagnostic images are acquired, and each of the clinical diagnostic images has a corresponding secretion label; Based on the clarity of the secretions in the clinical diagnostic images, multiple nasal endoscopic images with clarity meeting preset requirements are extracted from multiple clinical diagnostic images; According to a preset ratio, multiple training samples are extracted from multiple nasal endoscopy images based on the type of secretion.

3. The method as described in claim 1, characterized in that, Based on the nasal endoscopy images and the secretion labels, an anatomical sub-region representation and pseudo-labels with spatial and semantic consistency are generated through an anatomical structure modeling mechanism, including: Based on the nasal cavity anatomy, the nasal endoscopy image is divided into multiple initial sub-regions; Spatial features are extracted from each of the initial sub-regions using a spatial encoder; Semantic features are extracted from each of the initial sub-regions using a semantic encoder based on the secretion labels. The product of the spatial features and the spatial feature weights, plus the product of the semantic features and the semantic feature weights, is used to represent the anatomical sub-region, where the sum of the spatial feature weights and the semantic feature weights is 1. The pseudo-label is generated based on the anatomical sub-region representation.

4. The endoscopic secretion segmentation method based on anatomical structure perception as described in claim 3, characterized in that, The step of generating the pseudo-label based on the anatomical sub-region representation includes: By introducing a clustering mechanism containing N topic prototypes, the soft allocation number between each feature representation and each topic prototype in each initial sub-region is calculated to determine the semantic clusters corresponding to different nasal anatomical structures; Based on the anatomical sub-region representation, a probability analysis is performed on each of the initial sub-regions to obtain pseudo-labels.

5. The method as described in claim 4, characterized in that, The step of performing probability analysis on each initial sub-region based on the anatomical sub-region representation to obtain pseudo-labels includes: By performing probability calculations on the anatomical sub-region representation using a multi-scale decoder, a probability map of secretions belonging to different secretion types in the initial sub-region is obtained. The probability maps of the various secretions are fused to obtain pseudo-labels.

6. The method as described in claim 1, characterized in that, The multi-scale image features include: semantic multi-scale image features and location multi-scale image features; The anatomical sub-region representation and the pseudo-label generate multi-scale image features through a multi-head self-attention and position enhancement adaptation mechanism, including: By performing correlation analysis on multiple initial sub-regions within the same nasal cavity anatomical structure using a multi-head self-attention sub-module, an enhanced semantic feature region is obtained. By adapting multi-layer coded features using a linear adapter and performing residual fusion, an enhanced location feature region is obtained.

7. The method as described in claim 1, characterized in that, The cross-regional contrast learning based on the nasal endoscopy image to extract a multi-scale prediction mask from multi-scale image features includes: The nasal endoscope image is transformed and enhanced to obtain an enhanced image; The consistency loss is determined based on the cosine distance between the enhanced image and the nasal endoscopy image; The top-level features of the nasal endoscopy image are decoupled to obtain foreground features, background features, and uncertainty features; The constraint loss is calculated based on the foreground features, the background features, and the uncertainty features; A contrastive loss is obtained by comparing and learning the foreground features and the background features. When the constraint loss and the contrast loss satisfy the training stopping condition, a multi-scale prediction mask is obtained.

8. The method as described in claim 7, characterized in that, The step of determining the consistency loss based on the cosine distance between the enhanced image and the nasal endoscopy image includes: in, Features of nasal endoscopy images To enhance the image with Corresponding characteristics.

9. The method as described in claim 7, characterized in that, The step of calculating the constraint loss based on the foreground features, the background features, and the uncertainty features includes: The constraint loss is calculated using the following formula; in, Foreground features, For background features, Uncertainty characteristics To limit losses, These are the weighting coefficients.

10. The method according to any one of claims 1-9, characterized in that, The multi-scale prediction mask includes small-scale masks, medium-scale masks, and large-scale masks; The step of fusing the multi-scale prediction masks to determine the secretions in the nasal endoscopy image includes: The small-scale mask, the medium-scale mask, and the large-scale mask are weighted and fused to obtain a secretion segmentation mask; The secretion is determined based on the pixel values ​​of the secretion segmentation mask.

Images