A nasal image segmentation method based on global information guidance and implicit expression reformulation

Abstract

The present invention proposes a nasal image segmentation method based on global information guidance and implicit expression reorganization. The specific steps of the method are as follows: Step 1, using a large amount of unlabeled head CT image data, the encoder part of the multi-level structure segmentation network is pre-trained based on the autoencoder self-supervision; Step 2, constructing a global encoding branch based on implicit neural expression, and pre-training it using resampled unlabeled data; Step 3, using the excellent initialization of the pre-trained encoder parameters and the assistance of the global encoding branch, constructing a multi-level nasal segmentation network based on implicit neural expression and completing the segmentation of the nasal image. The present invention successfully solves the problems faced by the existing technology in terms of data and methods for nasal image segmentation, realizes accurate nasal segmentation, and has broad application prospects in the field of rhinology clinical practice.

Description

Technical Field

[0001] The present invention relates to a medical image segmentation method based on implicit representation reorganization and global information guidance, belonging to the fields of digital image processing, pattern recognition, and computer vision. The present invention primarily relates to self-supervised learning, implicit neural representation, and medical image segmentation. It has broad application prospects in clinical medical image analysis and research related to rhinology. Background Art

[0002] Nasal image segmentation plays a crucial role in medical image analysis. The nasal cavity is a vital respiratory organ, and the normality of its structure and function directly impacts respiratory function. Clinically, problems such as morphological analysis of nasal structure and the study of respiratory disorders rely on accurate nasal image segmentation.

[0003] First, nasal image segmentation can provide accurate nasal structural information, which is of great reference value for clinicians to understand and analyze the morphological structure of the nasal cavity, as well as to analyze nasal diseases such as sinusitis and nasal polyps. Secondly, nasal image segmentation is also an important step before nasal surgery, such as endoscopic nasal surgery and nasal tumor resection surgery, which all rely on accurate nasal image segmentation results to plan the surgical path and scope. In addition, for respiratory disorders such as nasal stenosis and nasal septum deviation, it is also necessary to rely on accurate nasal image segmentation for disease assessment and analysis. However, the design and implementation of automated nasal segmentation methods face the following problems:

[0004] 1. There is a lack of sufficient expert-annotated nasal datasets in the field of rhinology

[0005] While the collection and use of medical images are subject to strict constraints and restrictions, there are still some publicly available medical image datasets. However, current research on medical image segmentation primarily focuses on the segmentation of organs, tissues, and lesions, and data with accurate nasal annotations is relatively scarce. The performance and generalization of supervised deep learning methods rely heavily on the availability of high-quality annotated data, and this lack of available annotated data hinders the performance of segmentation methods.

[0006] 2. The small number of voxels belonging to the nasal cavity makes it difficult to describe complex shape structures

[0007] In a complete head CT scan that includes the nasal cavity, the nasal cavity occupies a relatively small volume. Further segmentation of the nasal cavity into the nasal vestibule, nasal conchae, and sinuses would further reduce the number of voxels associated with each structure. The nasal cavity's structure is complex and varied, and fewer voxels have limited ability to describe these complex shapes.

[0008] 3. The segmentation process of 3D images lacks global information of head CT

[0009] Unlike the open nature of semantic segmentation data, the overall structure of medical images is relatively fixed. Besides their inherent texture and grayscale characteristics, various tissues and organs in the human body exhibit a high degree of regularity in their mutually constrained positions and shapes. For example, the shape of the skull constrains the shape and orientation of the nasal cavity. However, due to the limited memory capacity of current computer hardware, most deep learning-based 3D image segmentation methods use small patches of the original 3D image as training samples, which prevents the training process from utilizing the global information of the 3D image.

[0010] 4. The segmentation accuracy of the boundary surfaces of the nasal structure is poor

[0011] In the nasal cavity, the grayscale values ​​of the cavity and the cavity wall tissues differ significantly. However, the cavity entrance and exit end surfaces are composed of air, so there is no difference in their grayscale values. This results in poor segmentation of the end surfaces of the cavity structures in the nasal cavity image segmentation task, even when the overall evaluation index is high.

[0012] Based on the above analysis of the nasal cavity segmentation problem, the present invention proposes a nasal cavity image segmentation method based on global information guidance and implicit expression reorganization. In the method, a global encoding branch based on implicit expression is designed to provide global information guidance for the segmentation network. An implicit expression reorganization module is designed in the segmentation network to improve the network's ability to extract complex nasal cavity shape features. A method for automatically extracting difficult-to-segment end face voxels from nasal cavity annotations is designed to enable the network to enhance the segmentation effect of the boundary end face. At the same time, the concept of autoencoder is used to design an unsupervised pre-training method based on nasal cavity reconstruction to make full use of possible unlabeled image data to improve the network parameter initialization effect. Summary of the Invention

[0013] To address the above issues, the present invention aims to provide a nasal image segmentation method based on global information guidance and implicit expression reorganization. This method utilizes a self-supervised pre-training process to maximize the feature extraction capability of the segmentation network in the absence of labeled data. This method introduces global head image information by adding an implicit neural expression autoencoder branch to the segmentation network. This method also incorporates an implicit neural expression module into the hierarchical structure of the segmentation network to enhance the segmentation network's performance in segmenting the nasal shape, particularly the end face of the nasal structure. The overall technical solution of the present invention is primarily embodied in the following four aspects:

[0014] 1) Establish a self-supervised pre-training method based on autoencoders, using large quantities of unlabeled head CT image data to pre-train the encoder part of the segmentation network, so that it has a good initialization state before the formal supervised training begins, improving its feature extraction ability;

[0015] 2) Using a global implicit neural expression branch outside the segmentation network backbone, the global feature information of the complete head CT image to which the sample belongs is added to the network to enhance the overall effect of the segmentation network;

[0016] 3) Design a 3D implicit neural representation module and introduce it into the segmentation network to enhance the segmentation network’s ability to express object shapes and improve the network’s cross-resolution segmentation capabilities;

[0017] 4) Refine the original segmentation labels, subdivide the labels of the nasal structure into sidewall labels and end face labels, give the end face labels a higher weight, and improve the network's segmentation effect on the end face.

[0018] The present invention relates to a nasal cavity image segmentation method based on global information guidance and implicit expression reformation. The specific steps of the method are as follows:

[0019] Step 1: Using a large amount of unlabeled head CT image data, the encoder part of the multi-level structure segmentation network is pre-trained based on the autoencoder self-supervision;

[0020] A multi-layer encoder structure is constructed for the hierarchical segmentation network. A large number of unlabeled head CT image datasets are used. The concept of the autoencoder is used to supplement the encoder with a decoder that is only used for the pre-training process. In the autoencoder learning framework, image restoration is used as a training task in the pre-training process. The encoder parameters after training can be used as excellent initialization parameters for the encoder during the formal supervised training process. In addition to image restoration, rotation angle prediction, color prediction, noise removal, and super-resolution can all be set as pre-training task types. Taking image restoration as an example, the specific process of step one is as follows:

[0021] S11. Build an autoencoder learning framework. This framework consists of an encoder and a decoder. The input is an image block, which is first processed by the encoder to obtain a latent variable representing the image block information. The decoder then uses this latent variable as input to recover the reconstructed image block. The framework's output is the autoencoder loss function between the original image block and the reconstructed image block, which is primarily composed of a reconstruction loss and a regularization loss.

[0022] S12. Randomly crop the unlabeled image used for self-supervised pre-training to obtain an image block, which is then randomly mirrored and rotated for augmentation and used as the input sample. The segmentation network encoder is then pre-trained according to the autoencoder learning framework described in S11. The specific training process is as follows: the original image block passes through the encoder-decoder structure to obtain a reconstructed image block; the original image block and the reconstructed image block are used to calculate the autoencoder loss function value; and based on this loss function value, backpropagation and parameter optimization are performed on the segmentation network encoder in the autoencoder learning framework.

[0023] Step 2: Construct a global encoding branch based on implicit neural representation and pre-train it using resampled unlabeled data;

[0024] Based on the concept of implicit neural representation, a deep convolutional network is constructed to extract global information from a complete 3D image. The network consists of an encoder and a decoder. In this network, the encoder receives the original image and encodes it into an implicit representation. The decoder receives this implicit representation and a spatial coordinate and ultimately determines the category to which the spatial coordinate belongs. This network will serve as the global branch network of the hierarchical segmentation network, providing global information of the image during the segmentation process. The specific process is as follows:

[0025] S21. Resample and binarize the unlabeled image data. First, resample all unlabeled head CT image data to unify the resolution of all images. This approach ensures the consistency and accuracy of subsequent processing and analysis. The resampled images are then binarized. Binarization is the process of converting the original grayscale image into a binary image. In this process, all voxels are divided into two categories: air and human tissue. Specifically, we select a threshold and then mark voxels with grayscale values ​​below the threshold as air, and voxels with grayscale values ​​above the threshold as human tissue.

[0026] S22. Build an implicit neural representation learning framework consisting of a deep convolutional encoder and a deep convolutional decoder. The encoder encodes a 3D image input into an implicit representation that contains global information about the input image. The decoder receives the implicit representation and the voxel coordinates of any 3D image as input and outputs the probability that the voxel at that coordinate belongs to human tissue. The encoder and decoder utilize deep convolutional neural networks to efficiently process 3D data.

[0027] S23. Pre-training of implicit neural expression learning framework. The binarized three-dimensional image is input into the encoder, which encodes the three-dimensional image into an implicit representation, and then uses the decoder for self-supervised training. Specifically: the implicit representation and the coordinates of each voxel in the three-dimensional image are input into the decoder, and the decoder outputs the occupancy probability at the voxel coordinate. The loss between the output of the decoder and the value of the original binarized three-dimensional image at the voxel coordinate is calculated, and the encoder and decoder are optimized with this loss. After the pre-training of the framework is completed, its encoder part can be used as the global encoding branch of the hierarchical segmentation network, connected to the segmentation network, and the complete resampled three-dimensional image is encoded to obtain an implicit expression of global information.

[0028] Step 3: Leveraging the excellent initialization of pre-trained encoder parameters and the assistance of the global encoding branch, a multi-level nasal cavity segmentation network based on implicit neural representation is constructed to complete the segmentation of nasal cavity images;

[0029] Based on steps 1 and 2, a complete multi-level nasal cavity segmentation network based on implicit neural representation is constructed. During training, both image patches and resampled full images are used as inputs. In the loss function, a greater influence factor is assigned to the segmentation loss of nasal end face voxels to encourage the network to focus on improving the segmentation ability of the nasal end face.

[0030] S31, additionally mark the end face voxels in the segmentation label. The specific method is: first, extract the outer surface voxel set S of the nasal cavity ext The head CT is divided into two categories, head tissue and background air, using threshold binarization. The voxel value of head tissue is 1 and the voxel value of background air is 0. The binary image is represented by I Bin Indicates. In the label diagram corresponding to CT I label In the P n Represents all voxels corresponding to the nasal cavity. n Each voxel p contained in the binary image I is judged if the eight neighborhoods of p are in the binary image I Bin If there is a voxel with a value of 0 in the voxel, then the voxel p is considered to belong to the outer surface S ext Then for S ext The eight neighborhoods of each voxel p are judged, if its eight neighborhoods are in the binary image I Bin If a voxel with a value of 1 exists in the nasal cavity, it is considered to belong to the lateral wall of the nasal cavity. Otherwise, it is considered to belong to the boundary end face, and the voxel position is additionally labeled. If the data label is a more detailed division of the nasal cavity, such as the nasal vestibule, turbinate, sinus, etc., the end face of each structure can also be labeled separately according to this operation.

[0031] S32. Construct a three-dimensional implicit expression reorganization module. The three-dimensional implicit expression reorganization module in the present invention is a tool for processing and optimizing three-dimensional implicit expressions. In the encoder of the segmentation network, the number of channels and sizes of the feature maps output by the convolution blocks at each level are different, and the abstract levels of the features they express are also different. These feature maps are regarded as implicit expressions of the image blocks. The main function of the three-dimensional implicit expression reorganization module is to reorganize the implicit expressions of different scales into the same scale, so as to better aggregate and process the implicit expressions at different receptive field scales, thereby improving the network's ability to represent shape features. The specific working process of this module is as follows: First, the coordinates of each point in the input feature map and the coordinates of each point in the target size are normalized, and a sampling grid is created according to the target size. Then, the input feature map is sampled by the nearest neighbor according to the coordinates of each point in the sampling grid, and the relative distance between the sampling grid coordinates and the nearest neighbor sampled point in the feature map is recorded. The relative distance of each point is encoded and appended to the corresponding sampling value, which is the reorganized implicit expression output.

[0032] S33. Construct a multi-level nasal cavity segmentation network based on the concept of implicit neural representation. The encoder portion of the segmentation network adopts a hierarchical architecture, with multi-layer convolutional encoders used to maximize the mining and extraction of effective feature information contained in the image at different levels of abstraction and receptive field ranges. As the encoder layer of the segmentation network deepens, the number of channels of the feature map it outputs increases and the shape becomes smaller. The receptive field of the convolutional layer increases accordingly, and the level of abstraction of the features it represents increases. This design enables different layers of the encoder to extract image information at different scales. Each level of the encoder consists of two serially connected convolutional blocks, each of which contains a convolutional layer, a normalization layer, and an activation function layer. The output feature maps of each level of the encoder are processed by a three-dimensional implicit representation reorganization module and unified into tensors with the same number of channels and shape. This tensor is then connected in parallel with the tensor representing the entire image information output by the global encoding branch. The connected tensors are then processed by the segmentation head to output the segmentation prediction.

[0033] The process of nasal image segmentation based on implicit neural expression is as follows: Figure 1 As shown in the figure, the self-supervised pre-training process is designed using the concept of autoencoders to fully utilize the potential of unlabeled data to improve the feature extraction ability of the segmentation network encoder, a global encoding branch is designed to introduce global information and structural constraints of head images to improve the network segmentation accuracy, and a three-dimensional implicit expression reorganization module is used to effectively fuse the implicit features of various scales in the hierarchical encoder, ultimately achieving accurate segmentation of the nasal cavity structure.

[0034] Advantages and effects of the present invention: The present invention proposes a nasal image segmentation method based on global information guidance and implicit expression reorganization. A self-supervised pre-training process based on an autoencoder is designed to fully tap the potential of unlabeled head CT image data, provide excellent initialization for the segmentation network encoder, and thus enhance its feature extraction capability. A global implicit neural expression branch is designed for the segmentation network, so that the global feature information of the complete head CT image is utilized in each network forward transmission process, further improving the segmentation effect. A method for automatically extracting boundary end face voxels in nasal labels is proposed, and applied to the loss function of the segmentation network to improve the segmentation effect of the segmentation network on difficult-to-segment end faces. A three-dimensional implicit expression reorganization module is designed and used in the hierarchical segmentation network to enhance the network's ability to express the shape of the nasal cavity and improve its cross-resolution segmentation capability. In general, the present invention successfully solves the problems faced by the existing technology in terms of data and methods for nasal image segmentation, realizes accurate nasal segmentation, and has broad application prospects in the fields of rhinology image analysis and clinical research. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the overall flow chart of the nasal image segmentation method based on implicit neural expression.

[0036] Figure 2 Diagram of the autoencoder pre-training framework.

[0037] Figure 3 Schematic diagram of the global encoding branch structure.

[0038] Figure 4 Schematic diagram of the nasal vestibule segmentation effect. DETAILED DESCRIPTION

[0039] The present invention can be applied to the automatic segmentation of the nasal cavity and its various subdivided structures. In order to better illustrate the technical solution of the present invention, the segmentation task of the nasal vestibule structure is taken as an example and the embodiment of the present invention is further described in conjunction with the accompanying drawings. The overall process of the present invention is as follows Figure 1 As shown, the specific implementation details of each part are as follows:

[0040] Step 1: Self-supervised pre-training of the implicit neural representation network on a large-scale unlabeled image dataset;

[0041] S11. Construct a self-supervised learning framework based on an autoencoder. This framework consists of an encoder and a decoder. The encoder is identical to the hierarchical segmentation network encoder used in formal supervised training. The decoder consists of a symmetrical hierarchical structure, with transposed convolutions reducing the number of feature map channels and increasing the feature map size. The encoder encodes the input nasal vestibule image into an implicit vector, while the decoder decodes this implicit vector based on the task settings. In this example, taking the reconstruction task as an example, the expected output of the decoder is the image block input to the encoder. The output of the framework is the autoencoder loss function value between the original image block and the reconstructed image block. The autoencoder loss function consists of a reconstruction loss and a regularization loss, where the reconstruction loss uses a cross-entropy loss and the regularization loss uses an L2 loss.

[0042] The reconstruction loss is calculated as:

[0043] L reconstruction =-∑(y true *log(y pred )+(1-y true )*log(1-y pred ))

[0044] Among them, y true is the original image block, y pred is the reconstructed image block.

[0045] The calculation formula of regularization loss is:

[0046] L regularization =∑(w 2 )

[0047] Here, w is the weight of the encoder.

[0048] S12. Use unlabeled nasal vestibular image data to perform self-supervised pre-training on the implicit neural expression network. The goal of pre-training is to enable the network to achieve good segmentation results by learning the intrinsic structure and features of the image without label information. The unlabeled image used for self-supervised pre-training is randomly cropped to obtain an image block, which is randomly mirrored and rotated for enhancement as an input sample. Then, the segmentation network encoder is pre-trained according to the autoencoder learning framework described in S11. The specific training process is: the original image block passes through the encoder-decoder structure to obtain a reconstructed image block; the original image block and the reconstructed image block are used to calculate the autoencoder loss function value; based on the loss function value, the segmentation network encoder in the autoencoder learning framework is back-propagated and parameter optimized. This step uses the Adam optimizer for parameter optimization, and the update formula of the Adam optimizer is:

[0049] m=β1*m+(1-β1)*dx

[0050] v=β2*v+(1-β2)*(dx 2 )

[0051]

[0052] Among them, m and v are the estimates of the first and second moments of the gradient, dx is the gradient, x is the parameter to be updated, learning rate is the learning rate, β1 and β2 are hyperparameters, usually set to 0.9 and 0.999, and ε is a small number, usually set to 1e-8 to prevent division by zero.

[0053] The goal of this pre-training method is to enable the autoencoder to restore the input image block as much as possible, that is, to make the reconstructed image block as close to the input image block as possible. In this way, the autoencoder can learn the inherent structure and pattern of the image data during the pre-training process. This process does not require any labeled data, but only requires a large amount of unlabeled image data. It can make full use of the existing large-scale unlabeled image data to improve the efficiency and effect of pre-training. Its specific framework is as follows Figure 2 shown.

[0054] Step 2: Construct a global encoding branch based on implicit neural representation and pre-train it using resampled unlabeled data;

[0055] S21. Resample and binarize the unlabeled image data. First, resample all unlabeled head CT image data so that the resolution of all images is unified to (256, 256, 192). This approach can ensure the consistency and accuracy of subsequent processing and analysis. Then binarize the resampled images. Binarization is the process of converting the original grayscale image into a binary image. In this process, we divide all voxels into two categories: air and human tissue. Specifically, we select an absorption intensity threshold of -850, and then mark voxels with grayscale values ​​below the threshold as air, and voxels with grayscale values ​​above the threshold as human tissue.

[0056] S22. Construct an implicit neural expression learning framework, which includes a deep convolutional encoder and a deep convolutional decoder. The encoder's task is to encode the input three-dimensional image into an implicit representation. Ideally, the implicit representation contains the global information of the input image. The encoder consists of a convolutional layer, a normalization layer, and a nonlinear layer. The decoder's task is to receive the implicit representation and the voxel coordinates of any three-dimensional image as input, and output the probability that the coordinate voxel belongs to human tissue. The structure of the encoder and decoder adopts a deep convolutional neural network to effectively process three-dimensional data. The structure of the network framework is as follows Figure 3 shown.

[0057] S23. Pre-training of the implicit neural expression learning framework. The binary three-dimensional image is input into the encoder, which encodes the three-dimensional image into an implicit representation, and then uses the decoder for self-supervised training. Specifically, the implicit representation and the coordinates of each voxel in the three-dimensional image are input into the decoder, and the decoder outputs the probability that the voxel belongs to human tissue. The loss between the output of the decoder and the value of the original binary three-dimensional image at the voxel coordinate is calculated, and the encoder and decoder are optimized with this loss. In this process, the mean square error loss function is used, and an optimization algorithm such as Adam is used to update the parameters.

[0058] The calculation formula of the mean square error loss function is:

[0059] L MSE =1 / n*∑(y true -y pred ) 2

[0060] Among them, y true is the value of the original binary 3D image at the voxel coordinate, y pred is the occupancy probability output by the decoder, and n is the total number of voxels.

[0061] After pre-training, the encoder part of the learning framework is used as the global encoding branch of the hierarchical segmentation network for formal supervised training.

[0062] Step 3: Construct a multi-level nasal vestibule segmentation network based on implicit neural expression and complete the segmentation of the nasal vestibule image;

[0063] S31. Additional annotation of the end face voxels in the segmentation label. The specific method is: first, extract the outer surface voxel set S of the nasal vestibule ext The head CT is divided into two categories, head tissue and background air, using threshold binarization. The voxel value of head tissue is 1 and the voxel value of background air is 0. The binary image is represented by I Bin Indicates. In the label diagram corresponding to CT I label In the P n Represents all voxels corresponding to the nasal vestibule. n Each voxel p contained in the binary image I is judged if the eight neighborhoods of p are in the binary image I Bi If there is a voxel with a value of 0 in the voxel, then the voxel p is considered to belong to the outer surface S ext Then for S ext The eight neighborhoods of each voxel p are judged, if its eight neighborhoods are in the binary image I Bin If there is a voxel with a value of 1 in the image, it is considered that the voxel belongs to the lateral wall of the nasal vestibule; otherwise, the voxel is considered to belong to the boundary end surface, and the position of the voxel is additionally marked.

[0064] S32. Construct a 3D implicit expression reorganization module. This method designs a 3D implicit expression reorganization module to reorganize implicit expressions of different sizes into any same size, so as to aggregate implicit expressions under different receptive field scales and improve the network's ability to represent shape features. The specific working process of this module is as follows: First, the coordinates of all points in the input feature map and the target feature map are normalized to C in and C out , and all the target coordinates form a sampling grid. Then, according to the coordinates c of each point in the sampling grid out Perform nearest neighbor sampling on the input feature map to obtain the nearest neighbor feature f sample . At the same time, record the sampling coordinates c out and the coordinates c of its nearest neighbor sampling point in the input feature map in The normalized relative distance d norm Finally, for the relative distance d norm Encode to get d decode , and d decode In the channel direction, parallel to f sample After that, we get f out After executing this process at each point in the sampling grid, the rearranged implicit expression F is obtained. out .

[0065] S33. Construction and training of a multi-level segmentation network based on implicit neural representations. The encoder portion of the segmentation network adopts a hierarchical architecture with an additional global encoding branch. The multi-layer convolutional encoder is used to maximize the extraction of effective image feature information at different levels of abstraction and receptive fields. The encoder designed in this method comprises six layers. Each layer consists of two concatenated convolutional blocks, with the first convolutional block adjusting the number of channels and size of the output feature map. The output feature map of the first layer has 32 channels and its size is maintained at the same size as the input image block. The number of channels of the output feature maps of each layer is increased by 2, and the size is increased by 2. The convolutional blocks consist of convolutional layers, instance normalization layers, and leaky relu layers. During training, the network receives an image block and the corresponding resampled full image as input. The encoder receives an image block, and the output feature maps of each level are processed by a 3D implicit representation reshape module. The output feature maps are uniformly distributed to 259 channels and the same size as the output feature map of the first encoder level. This tensor is then concatenated with the output of the global encoding branch. The concatenated tensor is then processed by the segmentation head to output the segmentation prediction. The loss function is the cross entropy of the segmentation prediction and the label, with an additional weighting for the cross entropy loss of the end-face voxels. The calculation process is as follows:

[0066] l=L base +λL interface

[0067]

[0068] Where L is the total loss. base L is the base loss, calculated in the form of cross entropy loss, which is used to measure the general difference between the model prediction and the true label. interface It is a loss term specifically designed to focus on the segmentation of the end face, and λ is a hyperparameter used to adjust the weight between the two loss terms so that more attention can be paid to the segmentation of the cavity end face during training. interface is the end face voxel set, which contains all voxels marked as end faces. Then, for each voxel p i , calculate the model prediction value and the true value of the label The cross entropy loss between them is added to L interface middle.

[0069] Finally, based on the loss function value, the hierarchical segmentation network is back-propagated and parameter optimized. The Adam optimizer is used for parameter optimization in this process.

[0070] In this experiment, 116 head CT images with nasal vestibule annotation data were used, of which 92 images were used as training sets and 24 images were used as test sets. Figure 4 In which a~c are the segmentation results of the present invention, and d~f are the labeled true values. It can be seen that the segmentation results of the nasal vestibule are highly consistent with the label true values. The present invention proposes a nasal image segmentation method based on global information guidance and implicit expression reorganization. It fully utilizes large-scale unlabeled data through self-supervised pre-training of autoencoders, designs global encoding branches to introduce full-image structural information, and designs and implements implicit expression reorganization modules in the hierarchical segmentation network, integrating multi-scale influencing features, effectively improving the network's ability to express complex shapes of the nasal cavity, and finally realizing accurate and rapid segmentation of the nasal cavity and various subdivided structures in the nasal cavity. The present invention can be widely used in the fields of ophthalmology medical image analysis, disease analysis, surgical planning, etc., and has important clinical value and market potential.

Claims

1. A nasal image segmentation method based on global information guidance and implicit expression reformation, characterized in that: The specific steps of this method are as follows: Step 1: Using a large amount of unlabeled head CT image data, the encoder part of the multi-level structure segmentation network is pre-trained based on the autoencoder self-supervision; Step 2: Construct a global encoding branch based on implicit neural representation and pre-train it using resampled unlabeled data. In step 2, construct an implicit neural representation learning framework. This framework includes a deep convolutional encoder and a deep convolutional decoder. The encoder's task is to encode the input 3D image into an implicit representation that contains the global information of the input image. The decoder's task is to receive the implicit representation and the voxel coordinates of any 3D image as input and output the probability that the voxel at that coordinate belongs to human tissue. The encoder and decoder structures use deep convolutional neural networks to process 3D data. Step 3: Utilize the excellent initialization of pre-trained encoder parameters and the assistance of the global encoding branch to construct a multi-level nasal segmentation network based on implicit neural expression and complete the segmentation of nasal images; in step 3, a multi-level nasal segmentation network is constructed based on the concept of implicit neural expression; the encoder part of the segmentation network adopts a hierarchical architecture, and the multi-layer convolution encoder is used to mine and extract the effective feature information contained in the image as much as possible under different abstraction levels and receptive field ranges; as the encoder level of the segmentation network deepens, the number of channels of its output feature map increases and the shape becomes smaller, the receptive field of the convolution layer increases accordingly, and the abstract level of the represented features increases; each level of the encoder is composed of two serially connected convolution blocks, and each convolution block contains a convolution layer, a normalization layer and an activation function layer in sequence; the output feature maps of each level of the encoder are processed by the three-dimensional implicit expression reorganization module respectively, and unified into tensors with the same number of channels and shape, and this part of the tensor is connected in parallel with the tensor representing the full image information output by the global encoding branch. The parallel tensor is processed by the segmentation head to output the segmentation prediction.

2. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 1, characterized in that: In step one, a multi-layer encoder structure is constructed for the hierarchical segmentation network. Using a large batch of unlabeled head CT image datasets, the concept of autoencoder is used to supplement the encoder with a decoder that is only used for the pre-training process. Under the autoencoder learning framework, image restoration is used as the training task of the pre-training process, and the encoder parameters after training are used as the initialization parameters of the encoder in the formal supervised training process. In addition to image restoration, rotation angle prediction, color prediction, noise removal and super-resolution are all set as pre-training task types.

3. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 1 or 2, characterized in that: In step 1, an autoencoder learning framework is constructed; the framework includes an encoder and a decoder; the input is an image block, which is first processed by the encoder to obtain the latent variable of the image block information, and then the decoder uses the latent variable as input to restore the reconstructed image block; the output of the framework is the autoencoder loss function value between the original image block and the reconstructed image block, which consists of reconstruction loss and regularization loss.

4. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 3, characterized in that: In step one, the unlabeled image used for self-supervised pre-training is randomly cropped to obtain an image block, which is then randomly mirrored and rotated for enhancement as the input sample; then, the segmentation network encoder is pre-trained according to the autoencoder learning framework; the specific training process is: the original image block passes through the encoder-decoder structure to obtain a reconstructed image block; the original image block and the reconstructed image block are used to calculate the autoencoder loss function value; based on the loss function value, the segmentation network encoder in the autoencoder learning framework is back-propagated and parameter optimized.

5. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 1, characterized in that: In step two, the unlabeled image data is resampled and binarized. First, all unlabeled head CT image data is resampled to unify the resolution of all images. Then, the resampled images are binarized. Binarization is the process of converting the original grayscale image into a binary image. In this process, all voxels are divided into two categories: air and human tissue. Specifically, a threshold is selected, and then voxels with grayscale values ​​below the threshold are marked as air, and voxels with grayscale values ​​above the threshold are marked as human tissue.

6. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 1, characterized in that: In step 2, the implicit neural expression learning framework is pre-trained; the binarized three-dimensional image is input into the encoder, the encoder encodes the three-dimensional image into an implicit expression, and then the decoder is used for self-supervised training; specifically: the implicit expression and the coordinates of each voxel in the three-dimensional image are input into the decoder, and the decoder outputs the occupancy probability at the voxel coordinate; the loss between the output of the decoder and the value of the original binarized three-dimensional image at the voxel coordinate is calculated, and the encoder and decoder are optimized with this loss; after the pre-training of the framework is completed, its encoder part is used as the global encoding branch of the hierarchical segmentation network, connected to the segmentation network, and the complete resampled three-dimensional image is encoded to obtain an implicit expression of global information.

7. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 1, characterized in that: In step 3, the end face voxels in the segmentation label are additionally labeled; the specific method is: first, extract the outer surface voxel set S of the nasal cavity ext ; Use threshold binarization to divide the head CT into two categories: head tissue and background air. The voxel value of head tissue is 1 and the voxel value of background air is 0. The binary image is I Bin Indicates; in the label map I corresponding to CT label In the P n Represents all voxels corresponding to the nasal cavity; n Each voxel p contained in the binary image I is judged if the eight neighborhoods of p are in the binary image I Bin If there is a voxel with a value of 0 in the voxel, then the voxel p is considered to belong to the outer surface S ext ; Then S ext The eight neighborhoods of each voxel p are judged, if its eight neighborhoods are in the binary image I Bin If there is a voxel with a value of 1 in , it is considered that the voxel belongs to the nasal side wall, otherwise it is considered that the voxel belongs to the boundary end surface, and the position of the voxel is additionally marked.

8. The nasal cavity image segmentation method based on global information guidance and implicit expression reformation according to claim 7, characterized in that: In step three, a three-dimensional implicit expression reorganization module is constructed; these feature maps are regarded as implicit expressions of image blocks. The specific working process of this module is: first, the coordinates of each point in the input feature map and the coordinates of each point in the target size are normalized, and a sampling grid is created according to the target size; then, the input feature map is sampled by the nearest neighbor according to the coordinates of each point in the sampling grid, and the relative distance between the sampling grid coordinates and the nearest neighbor sampled point in the feature map is recorded; the relative distance of each point is encoded and appended to the corresponding sampling value, which is the reorganized implicit expression output.

Images