A pancreatic lesion CT image positioning method and system based on multi-tissue segmentation

Abstract

The present application relates to the technical field of medical image processing, in particular to a pancreatic lesion CT image positioning method and system based on multi-tissue segmentation, comprising the following steps: acquiring a CT image containing a pancreatic tumor; using a pre-established feature extraction network to perform feature extraction on the CT image to obtain image features; using a region proposal network to generate candidate regions for the image features to obtain pancreatic tumor segmentation candidate regions. The present application uses a global feature extraction network structure and a local feature extraction network structure as a teacher model to guide the training of a lightweight MobileNet network to construct a feature extraction network for pancreatic tumor positioning, which can achieve high-precision positioning performance by aggregating global and local features, that is, it can extract tumor lesions of multiple sizes and improve the lightweight structure to reduce the amount of calculation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a pancreatic lesion CT image positioning method and system based on multi-tissue segmentation. Background Art

[0002] Pancreatic ductal adenocarcinoma (PDAC) is one of the most devastating malignancies, and computed tomography (CT) is the preferred imaging modality for the diagnosis and evaluation of PDAC. In recent years, radiomics, a technique that analyzes imaging features of regions of interest, has demonstrated excellent results in predicting tumor biological complexity and intratumor heterogeneity. Radiomics has also been used to predict the biological behavior of PDAC tumors through high-throughput analysis of intratumoral CT images. While most studies have focused on the primary tumor, recent studies have also highlighted the important auxiliary role of the tumor periphery in assessing cancer heterogeneity and its importance in prognostic prediction.

[0003] At present, when extracting pancreatic tumor imaging features, it includes two aspects: intra-tumor imaging features and peri-tumor imaging features. Before segmenting the intra-tumor imaging features and peri-tumor imaging features, locating the tumor lesion area first can often help improve the accuracy of segmentation of intra-tumor imaging features and peri-tumor imaging features. The existing pancreatic tumor localization method aggregates global and local features for positioning to achieve high-precision performance, that is, it can extract tumor lesions of various sizes, but it has the defects of complex structure and large computational cost. The use of global features or local features alone can reduce the computational cost, but there is a defect that small tumor lesions cannot be detected. Therefore, it is difficult to achieve a high-precision and lightweight feature extraction structure. Summary of the Invention

[0004] The purpose of the present invention is to provide a pancreatic lesion CT image localization method and system based on multi-tissue segmentation to solve the technical problem in the prior art that it is difficult to achieve a high-precision and lightweight feature extraction structure.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0006] A pancreatic lesion CT image localization method based on multi-tissue segmentation includes the following steps:

[0007] Obtain CT images that include the pancreatic tumor;

[0008] Extracting features from the CT image using a pre-established feature extraction network to obtain image features;

[0009] The region proposal network will be used to generate candidate regions for the image features and locate candidate regions for pancreatic tumor segmentation.

[0010] As a preferred solution of the present invention, the method for constructing the feature extraction network includes:

[0011] Performing global feature extraction on the CT image using a global feature extraction network structure to obtain global features of the image;

[0012] Performing local feature extraction on the CT image using a local feature extraction network structure to obtain local features of the image;

[0013] Both the global feature extraction network structure and the local feature extraction network structure are used as teacher models to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the feature extraction network.

[0014] As a preferred solution of the present invention, the global feature extraction method of the global feature extraction network structure includes:

[0015] The CT image is sequentially passed through a 3×3 convolution layer, a global average pooling layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain the global feature extraction method;

[0016] The structural formula of the global feature extraction network structure is:

[0017] ;

[0018] Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function.

[0019] As a preferred solution of the present invention, the local feature extraction method of the local feature extraction network structure includes:

[0020] The CT image is sequentially passed through a 3×3 convolution layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain the local feature extraction method;

[0021] The structural formula of the local feature extraction network structure is:

[0022] ;

[0023] Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

[0024] As a preferred solution of the present invention, a method for guiding lightweight MobileNet network training by using both the global feature extraction network structure and the local feature extraction network structure as teacher models includes:

[0025] The global feature extraction network structure is used as the first teacher model to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the first training model;

[0026] Using the local feature extraction network structure as the second teacher model to guide the training of the first training model, and using the trained first training model as the feature extraction network;

[0027] The loss function that guides MobileNet network training is:

[0028] ;

[0029] The loss function that guides the training of the first training model is:

[0030] ;

[0031] Where, To guide the loss of MobileNet network training, To guide the loss of the first training model training, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the MobileNet network. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the first training model. is the true value of the candidate region for pancreatic tumor segmentation, 、 are all hyperparameters, is a 2-norm formula.

[0032] As a preferred solution of the present invention, the loss function for training the global feature extraction network structure and the local feature extraction network structure includes:

[0033] ;

[0034] Where, The loss value for training the global feature extraction network structure and the local feature extraction network structure, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. is the KL divergence formula.

[0035] As a preferred embodiment of the present invention, the present invention provides a pancreatic lesion CT image positioning system based on multi-tissue segmentation, which is applied to a pancreatic lesion CT image positioning method based on multi-tissue segmentation. The system includes:

[0036] a data acquisition unit, configured to acquire a CT image containing a pancreatic tumor;

[0037] A feature extraction unit, configured to extract features from the CT image using a pre-established feature extraction network to obtain image features;

[0038] The region positioning unit is used to generate candidate regions for the image features using a region proposal network, and locate candidate regions for pancreatic tumor segmentation.

[0039] As a preferred solution of the present invention, the structure of the feature extraction network in the feature extraction unit is:

[0040] ;

[0041] Where, is the image feature, For the MobileNet network, For CT images.

[0042] As a preferred solution of the present invention, both the global feature extraction network structure and the local feature extraction network structure are used as teacher models for constructing the feature extraction network, wherein the structural formula of the global feature extraction network structure is:

[0043] ;

[0044] Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function;

[0045] The structural formula of the local feature extraction network structure is:

[0046] ;

[0047] Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

[0048] As a preferred solution of the present invention, the structure of the region proposal network in the region positioning unit is:

[0049] ;

[0050] Where, For pancreatic tumor segmentation candidate regions, is the image feature, Propose a network for the region.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention uses both the global feature extraction network structure and the local feature extraction network structure as teacher models to guide the training of a lightweight MobileNet network to construct a feature extraction network for pancreatic tumor localization. It can achieve high-precision positioning performance by aggregating global and local features, that is, it can extract tumor lesions of various sizes, improve the lightweight structure and reduce the amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0054] Figure 1 A flow chart of a pancreatic lesion CT image localization method based on multi-tissue segmentation provided in an embodiment of the present invention;

[0055] Figure 2 A block diagram of a pancreatic lesion CT image localization system based on multi-tissue segmentation provided by an embodiment of the present invention;

[0056] Figure 3 A flowchart of MobileNet network guided training provided by an embodiment of the present invention;

[0057] Figure 4 The candidate region for pancreatic tumor segmentation in the CT image provided by the embodiment of the present invention;

[0058] Figure 5 This is the true value of the candidate region for pancreatic tumor segmentation provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] like Figure 1 As shown, the present invention provides a pancreatic lesion CT image localization method based on multi-tissue segmentation, comprising the following steps:

[0061] Obtain CT images that include the pancreatic tumor;

[0062] Use the pre-established feature extraction network to extract features from CT images and obtain image features;

[0063] The region proposal network will be used to generate candidate regions based on image features and locate candidate regions for pancreatic tumor segmentation.

[0064] In order to improve the accuracy of segmenting the tumor region of interest and the region of interest around the tumor on CT images, the present invention pre-locates the pancreatic tumor area on the CT image, marks the pancreatic tumor area on the CT image through the region proposal network RPN, and then uses the marked pancreatic tumor area on the CT image to perform intratumoral-peritumoral region segmentation. This can provide precise position information of the target through positioning, help to more accurately identify and separate the target during image segmentation, ensure that the segmentation algorithm focuses on the correct area, reduce noise interference, and thus improve the accuracy and efficiency of segmentation.

[0065] To improve the accuracy of pancreatic tumor region localization on CT images using the region proposal network (RPN), the present invention sets up two feature extraction networks as teacher models to guide the training of the lightweight MobileNet network, thereby obtaining a feature extraction network structure that combines high-precision positioning performance with a lightweight structure, reducing the computational complexity of the feature extraction process, and at the same time, the extracted features can achieve high-precision pancreatic tumor localization performance.

[0066] Among them, one of the two feature extraction networks is the global feature extraction network, which guides the lightweight MobileNet network training through the global feature extraction network structure and uses is the loss function

[0067] , the image features extracted by the trained MobileNet network can obtain pancreatic tumor localization results similar to those obtained through global features, that is, satisfying Minimize, and at the same time, in the teacher model guided training process (knowledge distillation), it is also necessary to make the pancreatic tumor localization results obtained by the image features extracted by the trained MobileNet network similar to the actual tumor localization, that is, to meet minimize.

[0068] Global features are used to locate pancreatic tumor regions. They are generally superior in detecting large targets but have limited accuracy in detecting small targets. Image features extracted by the trained MobileNet network achieve pancreatic tumor localization results similar to those obtained using global features, enabling the MobileNet network to learn the feature extraction process for detecting large targets.

[0069] The other is a local feature extraction network, which guides the lightweight MobileNet network training through the local feature extraction network structure. As the loss function, the image features extracted by the trained MobileNet network can obtain pancreatic tumor localization results similar to those obtained by local features, that is, satisfying Minimize, and at the same time, in the teacher model guided training process (knowledge distillation), it is also necessary to make the pancreatic tumor localization results obtained by the image features extracted by the trained MobileNet network similar to the actual tumor localization, that is, to meet minimize.

[0070] Local features are used to locate pancreatic tumor regions and are generally superior in detecting small targets. Image features extracted by the trained MobileNet network yield pancreatic tumor localization results similar to those obtained using local features, enabling the MobileNet network to learn the feature extraction process for detecting small targets.

[0071] After being guided and trained by the global feature extraction network and the local feature extraction network in sequence, the MobileNet network is able to extract image features that are advantageous for detecting both large and small targets. This is equivalent to aggregating the image features to achieve the detection performance of both large and small targets. It can locate tumor lesions of various sizes in CT images containing multi-lesion tissues with good positioning accuracy. At the same time, the MobileNet network has a lightweight network structure with the advantages of reducing computational complexity, improving operational efficiency, reducing memory usage, and enhancing cross-platform compatibility.

[0072] Furthermore, in the present invention, the global feature extraction network and the local feature extraction network each have their own strengths, one is good at large target detection performance, and the other is good at small target detection performance. When training the global feature extraction network and the local feature extraction network, the present invention expects that the two can learn from each other and learn from each other's strengths, so as to achieve mutual learning and communication between the two teacher models, so that the two teacher models can master some of each other's skills, so that when guiding the training of the MobileNet network separately, they will not be separated from each other, that is, when the global feature extraction network guides the training of the MobileNet network, while imparting the feature extraction skills of the MobileNet network for large target detection, it will also impart some small target detection skills. The feature extraction skills of large target detection are constrained and balanced by each other, avoiding that the MobileNet network only learns the feature extraction skills of large target detection in a fragmented manner, and avoids the overfitting phenomenon of large target detection. When the local feature extraction network guides the training of the MobileNet network, while imparting the feature extraction skills of small target detection to the MobileNet network, it also imparts some feature extraction skills of large target detection. The feature extraction skills of small target detection are constrained and balanced by each other, avoiding that the MobileNet network only learns the feature extraction skills of small target detection in a fragmented manner, and avoids the overfitting phenomenon of small target detection, so that the model is forced to learn more general representations rather than over-adapting to specific tasks.

[0073] To this end, the present invention sets the loss function of the global feature extraction network structure and the local feature extraction network structure as ,in, The corresponding global feature extraction network structure learns from the local feature extraction network structure, that is, the tumor region positioning result obtained by the global features extracted by the global feature extraction network structure is fitted with the tumor region positioning result obtained by the local features extracted by the local feature extraction network structure. The corresponding local feature extraction network structure learns from the global feature extraction network structure, that is, the tumor region localization result obtained by the local features extracted by the local feature extraction network structure fits the tumor region localization result obtained by the global features extracted by the global feature extraction network structure. Therefore, the tumor region localization result obtained by the global feature extraction network structure is constrained by the tumor region localization result extracted by the local feature extraction network structure, and the tumor region localization result obtained by the local feature extraction network structure is constrained by the tumor region localization result extracted by the global feature extraction network structure.

[0074] Correspondingly, when the global feature extraction network guides the training of the MobileNet network, when imparting the feature extraction skills of the MobileNet network for large target detection, it is constrained by the small target detection feature skills of the local feature extraction network structure, so as to avoid the trained MobileNet network extracting features only in the direction of large target detection, that is, the features extracted by the trained MobileNet network overfit to the features used for large target detection, resulting in invalid feature extraction. When the local feature extraction network guides the training of the MobileNet network, when imparting the feature extraction skills of the MobileNet network for small target detection, it is constrained by the large target detection feature skills of the global feature extraction network structure, so as to avoid the trained MobileNet network extracting features only in the direction of small target detection, that is, the features extracted by the trained MobileNet network overfit to the features used for small target detection, resulting in invalid feature extraction. In this way, the mutual constraint balance of the feature extraction guidance of the two teacher models is achieved in the training process, so that the MobileNet network can extract image features that are dominant for both large and small target detection, and there is no overfitting defect.

[0075] The construction method of the feature extraction network includes:

[0076] The global feature extraction network structure is used to extract global features of CT images to obtain global features of the images;

[0077] The local feature extraction network structure is used to extract local features of CT images to obtain local features of the images;

[0078] Both the global feature extraction network structure and the local feature extraction network structure are used as teacher models to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the feature extraction network.

[0079] The global feature extraction methods of the global feature extraction network structure include:

[0080] The CT image is sequentially passed through a 3×3 convolution layer, a global average pooling layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain a global feature extraction method;

[0081] The structural formula of the global feature extraction network structure is:

[0082] ;

[0083] Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function.

[0084] In the present invention, the global feature extraction network structure undergoes a global average pooling (GAP) operation to extract global context information, then undergoes dimensionality reduction through a 1×1 convolution layer to reduce the amount of computation, and then uses a batch normalization (BN) layer and a ReLU activation function to enhance the nonlinear expression capability of the network. Finally, the original number of channels is restored again through a 1×1 convolution, and a BN layer is used to prevent overfitting to obtain global features.

[0085] The local feature extraction methods of the local feature extraction network structure include:

[0086] The CT image is sequentially passed through a 3×3 convolution layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain a local feature extraction method;

[0087] The structural formula of the local feature extraction network structure is:

[0088] ;

[0089] Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

[0090] In the present invention, the local feature extraction network structure is reduced in dimension by a 1×1 convolution layer to reduce the amount of calculation, and then passes through a batch normalization layer (BN) and a ReLU activation function that can improve the nonlinear expression ability of the network. Finally, the original number of channels is restored again through a 1×1 convolution, and a BN layer is passed to prevent overfitting to obtain local features.

[0091] The method of using both the global feature extraction network structure and the local feature extraction network structure as teacher models to guide the training of the lightweight MobileNet network includes:

[0092] The global feature extraction network structure is used as the first teacher model to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the first training model;

[0093] The local feature extraction network structure is used as the second teacher model to guide the training of the first training model, and the trained first training model is used as the feature extraction network;

[0094] The loss function that guides MobileNet network training is:

[0095] ;

[0096] The loss function that guides the training of the first training model is:

[0097] ;

[0098] Where, To guide the loss of MobileNet network training, To guide the loss of the first training model training, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the MobileNet network. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the first training model. The ground truth values ​​of the candidate regions for pancreatic tumor segmentation were obtained by experienced doctors from the 3Dslicer software (e.g. Figure 5 shown), 、 These are all hyperparameters, usually set to 1. is a 2-norm formula.

[0099] The present invention guides lightweight MobileNet network training through global feature extraction network structure, and uses As the loss function, the image features extracted by the trained MobileNet network can obtain pancreatic tumor localization results similar to those obtained by global features, that is, satisfying Minimize, and at the same time, in the teacher model guided training process (knowledge distillation), it is also necessary to make the pancreatic tumor localization results obtained by the image features extracted by the trained MobileNet network similar to the actual tumor localization, that is, to meet minimize.

[0100] Global features are used to locate pancreatic tumor regions. They are generally superior in detecting large targets but have limited accuracy in detecting small targets. Image features extracted by the trained MobileNet network achieve pancreatic tumor localization results similar to those obtained using global features, enabling the MobileNet network to learn the feature extraction process for detecting large targets.

[0101] The present invention guides lightweight MobileNet network training through local feature extraction network structure, and uses As the loss function, the image features extracted by the trained MobileNet network can obtain pancreatic tumor localization results similar to those obtained by local features, that is, satisfying Minimize, and at the same time, in the teacher model guided training process (knowledge distillation), it is also necessary to make the pancreatic tumor localization results obtained by the image features extracted by the trained MobileNet network similar to the actual tumor localization, that is, to meet minimize.

[0102] Local features are used to locate pancreatic tumor regions and are generally superior in detecting small targets. Image features extracted by the trained MobileNet network yield pancreatic tumor localization results similar to those obtained using local features, enabling the MobileNet network to learn the feature extraction process for detecting small targets.

[0103] The loss functions for training the global feature extraction network structure and the local feature extraction network structure include:

[0104] ;

[0105] Where, The loss value for training the global feature extraction network structure and the local feature extraction network structure, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. is the KL divergence formula.

[0106] The present invention sets the loss function of the global feature extraction network structure and the local feature extraction network structure as ,in, The corresponding global feature extraction network structure learns from the local feature extraction network structure, that is, the tumor region positioning result obtained by the global features extracted by the global feature extraction network structure is fitted with the tumor region positioning result obtained by the local features extracted by the local feature extraction network structure. The corresponding local feature extraction network structure learns from the global feature extraction network structure, that is, the tumor region localization result obtained by the local features extracted by the local feature extraction network structure fits the tumor region localization result obtained by the global features extracted by the global feature extraction network structure. Therefore, the tumor region localization result obtained by the global feature extraction network structure is constrained by the tumor region localization result extracted by the local feature extraction network structure, and the tumor region localization result obtained by the local feature extraction network structure is constrained by the tumor region localization result extracted by the global feature extraction network structure.

[0107] Correspondingly, when the global feature extraction network guides the training of the MobileNet network, when imparting the feature extraction skills of the MobileNet network for large target detection, it is constrained by the small target detection feature skills of the local feature extraction network structure, so as to avoid the trained MobileNet network extracting features only in the direction of large target detection, that is, the features extracted by the trained MobileNet network overfit to the features used for large target detection, resulting in invalid feature extraction. When the local feature extraction network guides the training of the MobileNet network, when imparting the feature extraction skills of the MobileNet network for small target detection, it is constrained by the large target detection feature skills of the global feature extraction network structure, so as to avoid the trained MobileNet network extracting features only in the direction of small target detection, that is, the features extracted by the trained MobileNet network overfit to the features used for small target detection, resulting in invalid feature extraction. In this way, the mutual constraint balance of the feature extraction guidance of the two teacher models is achieved in the training process, so that the MobileNet network can extract image features that are dominant for both large and small target detection, and there is no overfitting defect.

[0108] like Figure 2 As shown, the present invention provides a pancreatic lesion CT image positioning system based on multi-tissue segmentation, which is applied to a pancreatic lesion CT image positioning method based on multi-tissue segmentation. The system includes:

[0109] a data acquisition unit, configured to acquire a CT image containing a pancreatic tumor;

[0110] A feature extraction unit, configured to extract features from CT images using a pre-established feature extraction network to obtain image features;

[0111] The region localization unit is used to generate candidate regions for image features using a region proposal network and locate candidate regions for pancreatic tumor segmentation.

[0112] The structure of the feature extraction network in the feature extraction unit is:

[0113] ;

[0114] Where, is the image feature output by the feature extraction network, For the MobileNet network, For CT images.

[0115] The global feature extraction network structure and the local feature extraction network structure are both used as teacher models for constructing the feature extraction network. The structural formula of the global feature extraction network structure is:

[0116] ;

[0117] Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function;

[0118] The structural formula of the local feature extraction network structure is:

[0119] ;

[0120] Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

[0121] The structure of the region proposal network in the region positioning unit is:

[0122] ;

[0123] Where, Segment candidate regions for pancreatic tumors, such as Figure 4 As shown, is the image feature output by the feature extraction network, Propose a network for the region.

[0124] The present invention uses both the global feature extraction network structure and the local feature extraction network structure as teacher models to guide the training of a lightweight MobileNet network to construct a feature extraction network for pancreatic tumor localization. It can achieve high-precision positioning performance by aggregating global and local features, that is, it can extract tumor lesions of various sizes, improve the lightweight structure and reduce the amount of computation.

[0125] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A pancreatic lesion CT image localization method based on multi-tissue segmentation, characterized in that: The following steps are involved: Obtain CT images that include the pancreatic tumor; Extracting features from the CT image using a pre-established feature extraction network to obtain image features; A region proposal network is used to generate candidate regions for the image features and locate candidate regions for pancreatic tumor segmentation; The method for constructing the feature extraction network includes: Performing global feature extraction on the CT image using a global feature extraction network structure to obtain global features of the image; Performing local feature extraction on the CT image using a local feature extraction network structure to obtain local features of the image; The global feature extraction network structure and the local feature extraction network structure are used as teacher models to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the feature extraction network; The method of using both the global feature extraction network structure and the local feature extraction network structure as teacher models to guide the training of the lightweight MobileNet network includes: The global feature extraction network structure is used as the first teacher model to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the first training model; Using the local feature extraction network structure as the second teacher model to guide the training of the first training model, and using the trained first training model as the feature extraction network; The loss function that guides MobileNet network training is: ； The loss function that guides the training of the first training model is: ； Where, To guide the loss of MobileNet network training, To guide the loss of the first training model training, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the MobileNet network. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the first training model. is the true value of the candidate region for pancreatic tumor segmentation, 、 are all hyperparameters, is a 2-norm formula.

2. The method for pancreatic lesion CT image localization based on multi-tissue segmentation according to claim 1, characterized in that: The global feature extraction method of the global feature extraction network structure includes: The CT image is sequentially passed through a 3×3 convolution layer, a global average pooling layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain the global feature extraction method; The structural formula of the global feature extraction network structure is: ； Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function.

3. The method for pancreatic lesion CT image localization based on multi-tissue segmentation according to claim 1, characterized in that: The local feature extraction method of the local feature extraction network structure includes: The CT image is sequentially passed through a 3×3 convolution layer, a 1×1 convolution layer, a normalization layer, a ReLU activation function, a 1×1 convolution layer, and a normalization layer to obtain the local feature extraction method; The structural formula of the local feature extraction network structure is: ； Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

4. The method for pancreatic lesion CT image localization based on multi-tissue segmentation according to claim 1, characterized in that: The loss functions for training the global feature extraction network structure and the local feature extraction network structure include: ； Where, The loss value for training the global feature extraction network structure and the local feature extraction network structure, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. is the KL divergence formula.

5. A pancreatic lesion CT image localization system based on multi-tissue segmentation, characterized in that: A method for CT image localization of pancreatic lesions based on multi-tissue segmentation as described in any one of claims 1 to 4, the system comprising: a data acquisition unit, configured to acquire a CT image containing a pancreatic tumor; A feature extraction unit, configured to extract features from the CT image using a pre-established feature extraction network to obtain image features; a region positioning unit, configured to generate candidate regions for the image features using a region proposal network, and locate candidate regions for pancreatic tumor segmentation; The construction method of the feature extraction network includes: Performing global feature extraction on the CT image using a global feature extraction network structure to obtain global features of the image; Performing local feature extraction on the CT image using a local feature extraction network structure to obtain local features of the image; The global feature extraction network structure and the local feature extraction network structure are used as teacher models to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the feature extraction network; The method of using both the global feature extraction network structure and the local feature extraction network structure as teacher models to guide the training of the lightweight MobileNet network includes: The global feature extraction network structure is used as the first teacher model to guide the training of the lightweight MobileNet network, and the trained MobileNet network is used as the first training model; Using the local feature extraction network structure as the second teacher model to guide the training of the first training model, and using the trained first training model as the feature extraction network; The loss function that guides MobileNet network training is: ； The loss function that guides the training of the first training model is: ； Where, To guide the loss of MobileNet network training, To guide the loss of the first training model training, The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the global features output by the global feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the MobileNet network. The candidate region for pancreatic tumor segmentation is obtained by the region proposal network based on the local features output by the local feature extraction network structure. The region proposal network obtains the candidate region for pancreatic tumor segmentation based on the image features output by the first training model. is the true value of the candidate region for pancreatic tumor segmentation, 、 are all hyperparameters, is a 2-norm formula.

6. The pancreatic lesion CT image localization system based on multi-tissue segmentation according to claim 5, characterized in that: The structure of the feature extraction network in the feature extraction unit is: ； Where, is the image feature, For the MobileNet network, For CT images.

7. The pancreatic lesion CT image localization system based on multi-tissue segmentation according to claim 6, characterized in that: The global feature extraction network structure and the local feature extraction network structure are both used as teacher models for constructing the feature extraction network. The structural formula of the global feature extraction network structure is: ； Where, is a global feature extraction method, For CT images, is a 3×3 convolutional layer, is a 1×1 convolution layer, GAP is a global average pooling layer, and BN is a normalization layer. is the ReLU activation function; The structural formula of the local feature extraction network structure is: ； Where, is a local feature extraction method. For CT images, is a 3×3 convolutional layer, is a 1×1 convolution, BN is a normalization layer, is the ReLU activation function.

8. The pancreatic lesion CT image localization system based on multi-tissue segmentation according to claim 7, characterized in that: The structure of the region proposal network in the region positioning unit is: ； Where, For pancreatic tumor segmentation candidate regions, is the image feature, Propose a network for the region.

Images