Modeling method of canceration region recognition model based on microscopic spectral feature synthetic image

Abstract

The invention provides a modeling method of a cancerous region recognition model based on a microscopic spectrum feature synthetic image, and is used for solving the technical problem that the existing cancerous region recognition model based on a microscopic spectrum technology is difficult to accurately position and further improve the segmentation precision due to lack of spatial information. The modeling method of the canceration region recognition model based on the microscopic spectral feature synthesis image is not limited to spectral dimension difference, more attention is paid to learning of canceration map migration features, and the microscopic spectral feature synthesis image is obtained through synthesis by taking every three of the first five principal components with large information content proportion as a group. The spectrum migration model between the digital pathology and the microscopic spectrum is constructed by utilizing more related data information and the characteristics of an existing efficient model, a powerful bridge is established between multi-source data, effective fusion of spectrum dimension migration characteristics and image dimension migration characteristics is realized, and comprehensive and rich detail characteristics of cancerous tissues can be better extracted.

Description

Technical Field

[0001] The invention relates to a modeling method for a recognition model, and in particular to a modeling method for a cancerous region recognition model based on a microscopic spectral feature synthetic image. Background Art

[0002] Digital pathology is based on analysis and modeling of a large number of labeled pathology data sets. Compared with traditional pathology diagnosis, it has the advantages of low difficulty in quantitative analysis, high information utilization and repetition rate. However, the development of digital pathology technology has also encountered some bottlenecks. For example, it is difficult to improve the performance of existing models, and the cumulative effect of more data sets is getting weaker and weaker. The relevant models are effective in primary tumor classification, but are difficult to apply to more complex deep classification, which makes it difficult to carry out large-scale clinical applications.

[0003] With the influx of more sample data and the establishment of multi-dimensional and efficient models, medical hyperspectral technology is expected to become a powerful supplement to existing tissue pathology tools. The cancerous region recognition model based on hyperspectral technology can automatically analyze and detect the region of interest, and quantitatively evaluate the degree of variation in the lesion area, so it solves the above problems very well. For example, using microscopic hyperspectral imaging to image skin tissue, the epithelial cells of skin tissue are segmented based on information such as spectral angle and spectral distance, and its performance is better than the simple pixel-by-pixel segmentation method. However, due to the lack of spatial information, the existing cancerous region recognition model based on hyperspectral technology is difficult to accurately locate and further improve the segmentation accuracy. Summary of the invention

[0004] The purpose of the present invention is to solve the technical problem that the existing cancerous region recognition model based on hyperspectral technology is difficult to accurately locate and further improve the segmentation accuracy due to the lack of spatial information, and to provide a modeling method for the cancerous region recognition model based on microscopic spectral feature synthetic images.

[0005] In order to achieve the above object, the technical solution provided by the present invention is as follows:

[0006] A modeling method for a cancerous region recognition model based on a microscopic spectral feature synthetic image is characterized in that it comprises the following steps:

[0007] Step 1, construct pathological color data set D1;

[0008] Step 2, collect microscopic spectral images of multiple cancer patients' pathological tissue sections, extract image samples from the collected microscopic spectral images to obtain pseudo-color composite images; mark the pseudo-color composite images to form a marked document; extract atlas samples from the collected microscopic spectral images according to the marked document to obtain microscopic spectral samples; perform principal component analysis on the extracted microscopic spectral samples, synthesize the first five principal components with the largest information content in the analysis results into groups of three to obtain multiple groups of different microscopic spectral feature composite images corresponding to the current microscopic spectral image; construct atlas dataset D2 according to the multiple groups of microscopic spectral feature composite images corresponding to each microscopic spectral image;

[0009] Step 3, constructing a graph migration model;

[0010] Step 3.1, build a VGG model, input the pathological color dataset D1 into the VGG model for pre-training and performance evaluation, and learn the features and patterns in the pathological color dataset D1; after the pre-training and performance evaluation are completed, set a fully connected layer after the last convolutional layer of the VGG model to form a Trans-CNN model;

[0011] Step 3.2, input the atlas dataset D2 into the Trans-CNN model, and through transfer learning, transfer the features and patterns learned by the Trans-CNN model in the pathological color dataset D1 to the training task of the atlas dataset D2 for training, so as to obtain the atlas transfer model;

[0012] In step 3.3, the last convolutional layer and the fully connected layer of the VGG model in the atlas migration model are further trained. The trained atlas migration model is the cancerous region recognition model.

[0013] Furthermore, in step 3.2, when the features and patterns learned by the Trans-CNN model in the pathological color dataset D1 are transferred to the training task of the atlas dataset D2 for training, training is performed through a low learning rate and multiple iterations.

[0014] Furthermore, in step 3.3, the SGD optimization method is used to further train the last convolutional layer and the fully connected layer of the VGG model in the graph transfer model.

[0015] Furthermore, in step 3.1, the pathological color dataset D1 is input into the VGG model for pre-training as follows: the pathological color dataset D1 is divided into a training set and a test set and input into the VGG model, the VGG model is pre-trained by the training set, and the performance of the pre-trained VGG model is evaluated by the test set.

[0016] Furthermore, in step 1, an existing pathological color data set is used as the pathological color data set D1, or a high-definition camera is used to collect pathological color data to construct the pathological color data set D1.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. Different from the previous microscopic spectral pathology research, the modeling method of the cancerous area recognition model based on the microscopic spectral feature synthetic image provided by the present invention is not limited to the spectral dimension difference, but pays more attention to the learning of the migration characteristics of the cancerous spectrum. The first five principal components with relatively large information content are synthesized in groups of three to obtain the microscopic spectral feature synthetic image. By utilizing more relevant data information and the characteristics of the existing efficient models, a spectrum migration model between digital pathology and microscopic spectroscopy is constructed, which builds a strong bridge between multi-source data, realizes the effective fusion of spectral dimension and image dimension migration characteristics, and can better extract the comprehensive and rich detail characteristics of cancerous tissues.

[0019] 2. The present invention introduces microscopic spectral feature synthetic images and combines the migration features of conventional pathological color data sets, making maximum use of the existing pathological color data set information to achieve rapid and accurate identification of cancerous areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of sample extraction from the collected microscopic spectrum image in steps 2.1-2.2 of an embodiment of the present invention, wherein (a) is an extracted pseudo-color composite image, and (b) is an extracted microscopic spectrum sample;

[0021] Figure 2 is the grayscale image of the first five principal components in step 2.3 of the embodiment of the present invention;

[0022] Figure 3 The microscopic spectral feature composite image and pseudo-color composite image of PCA134, PCA154, PCA135, and PCA354 in step 1.4 of the embodiment of the present invention are shown in FIG.

[0023] Figure 4 This is a schematic diagram of the process of constructing a graph migration model in step 2.3 of an embodiment of the present invention;

[0024] Figure 5 1 is a comparison diagram of the classification and recognition results of the cancerous tissue region according to an embodiment of the present invention, wherein (a) is the real region marked by the doctor, and (b) is a schematic diagram of the classification and recognition results of the gastric cancer tissue region predicted by the atlas migration model SST-CNN-2 of the present invention;

[0025] Figure 6The schematic diagram of the classification and recognition results of cancerous tissue regions when the atlas migration model SST-CNN-2 of the present invention is integrated into the acquisition software, wherein (a) is a schematic diagram of the classification and recognition results of gastric cancer tissue regions predicted by the atlas migration model SST-CNN-2 of the present invention for the tissue slice to be tested, (b) is the cancerous region marked by the doctor, and (c) is a software visualization effect diagram;

[0026] Figure 7 This is a picture showing the software interface for cancerous area identification results. DETAILED DESCRIPTION

[0027] In order to make the advantages and features of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] A modeling method for a cancerous region recognition model based on a microscopic spectral feature synthetic image specifically comprises the following steps:

[0029] Step 1: Construct a pathological color dataset D1.

[0030] The existing pathology color dataset is used as the pathology color dataset D1, or a high-definition camera is used to collect pathology color data to construct the pathology color dataset D1. The pathology color dataset D1 includes multiple types of mixed adenocarcinoma samples (gastric cancer, thyroid cancer, etc.), and also contains detailed labeling documents such as the range of tumor tissue and normal tissue, as well as the specific locations of some cancer cells and normal cells.

[0031] Step 2: construct the graph dataset D2.

[0032] Step 2.1, using a microscopic hyperspectral imaging system to collect microscopic spectral images of multiple cancer patients' pathological tissue sections, and extracting image samples to obtain pseudo-color composite images, such as Figure 1 As shown in a in the figure, the pseudo-color composite image is marked, mainly marking the range of tumor tissue and normal tissue, as well as the specific locations of some cancer cells and normal cells, and forming a marking document.

[0033] Step 2.2, extract the spectrum sample of the collected micro-spectral image according to the marked document in step 2.1 according to the size of 250×250 (number of spatial dimension rows × number of spatial dimension columns) to obtain the micro-spectral sample, such as Figure 1 As shown in b. When extracting the atlas samples, for normal samples in the microscopic spectral image, the whole area is traversed for sample extraction, and samples with too many blank areas are removed. For cancerous samples in the microscopic spectral image, only the marked cancerous areas are extracted.

[0034] Step 2.3, perform principal component analysis (PCA) on the extracted microscopic spectrum samples, and obtain grayscale images of different principal components in turn. In this embodiment, the first 10 principal components contain more than 98.2% of the spectral information, of which the first principal component PC1 contains 76.2% of the information. The principal components after the fifth principal component contain very little information. Therefore, this embodiment mainly uses the first five principal components, such as Figure 2 As shown, from left to right are the first principal component PC1, the second principal component PC2, the third principal component PC3, the fourth principal component PC4, and the fifth principal component PC5. Since the second principal component PC2 mainly contains noise information rather than useful spectral information, it can be excluded, and the remaining four principal components are mainly used for synthesis. The microscopic spectral feature composite image composed of three channels using PC1, PC3, and PC4 is denoted as PCA134, and the microscopic spectral feature composite image composed of three channels using PC1, PC5, and PC4 is denoted as PCA1545. Similarly, PCA13 and PCA354 are obtained. Figure 3 As shown, from left to right are PCA134, PCA154, PCA135, PCA354 and the micro-spectral feature synthetic image obtained in step 1.2. The different micro-spectral feature synthetic images synthesized by the above principal components constitute the atlas dataset D2.

[0035] Step 3: Build a graph migration model.

[0036] In order to make full use of more samples of cancer datasets, it is necessary to extract more generalized cancer tissue structure features, and then train our specific spectral cancer data on this basis. Therefore, the present invention introduces the idea of ​​transfer learning, constructs a graph migration model between digital pathology and microscopic spectrum, extracts comprehensive and rich detailed features of cancer tissue, and maximizes the use of existing pathology dataset information.

[0037] Step 3.1, such as Figure 4 As shown in the figure, a traditional VGG model is built, which includes multiple convolutional layers. The weights are loaded based on the traditional VGG model, and the pathological color dataset D1 is divided into a training set and a test set to train and evaluate the performance of the VGG model to learn the features and patterns in the pathological color dataset D1. After the training and performance evaluation are completed, a fully connected layer is set after the last convolutional layer of the VGG model to form a Trans-CNN model.

[0038] In step 3.2, based on the Trans-CNN model, the weights of each convolutional layer are used as initialization parameters, and the atlas dataset D2 is input into the Trans-CNN model. Through transfer learning, the features and patterns learned by the Trans-CNN model in the pathological color dataset D1 are transferred to the training task of the atlas dataset D2. Training is performed at a low learning rate and 50 iterations to extract atlas features and obtain the atlas migration model SST-CNN-1.

[0039] Step 3.3, fine-tune the atlas transfer model SST-CNN-1 to further improve the model performance. Specifically, further train the last convolutional layer and fully connected layer of the VGG model in the atlas transfer model SST-CNN-1 to obtain the trained atlas transfer model SST-CNN-2, which is the cancerous region recognition model. The SGD (stochastic gradient descent) optimization method is used for training, the learning rate is set to 0.0008, the batch_size of the number of data (samples) passed to the program for training at a time is 50, and the number of epochs is 20.

[0040] In addition, according to the training results of the atlas migration model SST-CNN-2, the best microscopic spectral feature synthetic image is obtained, and the three principal components corresponding to the best microscopic spectral feature synthetic image are analyzed for subsequent use. In the final training results of this embodiment, PCA134 achieved the best result, followed by PCA514 and PCA145. Therefore, the three principal components corresponding to the best microscopic spectral feature synthetic image are PC1, PC3, and PC4. The pre-training of the pathological color data set D1 has played a positive role in improving the classification performance of the model, and moderate fine-tuning is also conducive to improving the performance of the model. When using three principal components, the accuracy is from high to low: PCA134 (95.46%)>PCA145 (94.39%)>PCA514 (94.25%)>PCA123 (80.78%). PCA134 can provide the best performance, while PCA123 provides poor performance. The results of PCA145 and PCA514 are similar, indicating that the order of synthesizing the images of the microscopic spectral features of each principal component has almost no effect on the results. When only a single principal component is considered, PC1 has the best effect, PC2 has the worst effect, and the accuracy of the principal components after PC4 is not much different. This is also consistent with the information volume data obtained after the principal component analysis in step 1.4, indicating that the classification effect is positively correlated with the data packet content, and the generalized cancerous tissue structure characteristics obtained by pre-training have played a positive role in the overall model classification performance.

[0041] The effect of the graph migration model constructed in the present invention is further illustrated by specific experiments below.

[0042] First, a 20× microscopic spectral image of the tissue slice to be tested is collected through the microscopic hyperspectral imaging system. According to the imaging size and sample radius of the existing equipment, in order to cover all sample tissues on the tissue slice to be tested, each tissue slice to be tested needs to be collected in different areas on average, and a total of about 2-4 microscopic spectral images need to be collected.

[0043] The collected microscopic spectral images are affected by factors such as the system light source, optical devices, and experimental environment, resulting in some random errors or systematic errors in the spectral dimension and the spatial dimension. Therefore, after obtaining the microscopic spectral images of the tissue slices to be tested, they need to be standardized and preprocessed to eliminate the influence of the system light source, optical components, and sensor performance. The specific standardized preprocessing methods are as follows:

[0044] Keep the light source intensity, focal length and magnification of the micro-spectral imaging system unchanged, collect and save the micro-spectral image B1 of the blank sample on the tissue section to be tested, and calculate the spectral dimension correction coefficient according to the following formula:

[0045]

[0046] Among them, (i, j, λ) is the coordinate of any pixel on the blank sample micro-spectral image, i, j, λ are the coordinates of the spatial two-dimensional and spectral dimensions of the blank sample micro-spectral image, h1(i, j, λ) is the spectral dimension correction coefficient corresponding to the pixel at the coordinate (i, j, λ), and N is the number of spectral segments;

[0047] Then the spatial dimension correction coefficient is calculated according to the following formula:

[0048]

[0049] Among them, P and Q are the horizontal and vertical numbers of pixels in the blank sample microspectral image, respectively, and h2(i,j,λ) is the spatial dimension correction coefficient corresponding to the pixel at the coordinate (i,j,λ).

[0050] The spatial dimension and spectral dimension of the micro-spectral image collected in step a are jointly corrected according to the following formula to obtain a standard micro-spectral image S′, and the preprocessing is completed;

[0051]

[0052] Where S is the collected microscopic spectrum image, and S′ is the standard microscopic spectrum image obtained after S is corrected.

[0053] After the preprocessing is completed, the sample feature extraction is performed. In this embodiment, the entire area is traversed for extraction according to the size of 250×250 (number of spatial dimension rows × number of spatial dimension columns), and too many samples in the blank area are removed. Since the size of each standard microscopic spectrum image is 1000×1000×256 (number of spatial dimension rows × number of spatial dimension columns × spectral dimension), 16 small sample microscopic spectrum images can be obtained for each standard microscopic spectrum image.

[0054] Perform principal component analysis on all small sample micro-spectral images. Since the three principal components corresponding to the micro-spectral feature composite image with the best result analyzed in step 2.3 are PC1, PC3, and PC4, each small sample micro-spectral image uses the three channels of PC1, PC3, and PC4 to form the corresponding micro-spectral feature composite image PCA134, which is used as the small spectrum sample to be tested for each small sample micro-spectral image.

[0055] All the small atlas samples to be tested are input into the trained atlas migration model SST-CNN-2. In order to have a clearer understanding of the diagnosis process, we calculate the probability of each pixel being classified as cancerous tissue based on the classification results.

[0056] like Figure 5 (b) shows the classification and recognition results of a gastric cancer tissue area. The closer the color is to red, the higher the probability that the area is a cancerous area, and the closer it is to blue, the higher the probability that it is a normal area. At the same time, for the small atlas samples diagnosed as cancerous tissue or normal tissue, their number information is recorded and compared. Figure 5 From the real area marked by the doctor in (a), it can be seen that the classification results of the atlas transfer model SST-CNN-2 of the present invention have a high accuracy rate.

[0057] In order to better apply it to clinical practice, we integrated the atlas migration model SST-CNN-2 into the acquisition software. After the tissue slices to be tested are collected, the corresponding tumor cancer type can be selected to perform regional recognition analysis, and the final recognition results, probabilities, and visualization images are instantly presented on the software interface. Figure 6 As shown, Figure 6 (a) is the result of the tissue slice to be tested after model classification and identification. Figure 6 (b) is the cancerous area marked by the doctor. In order to make the display in the software faster and more intuitive, we further improved the result visualization process. The pseudo-color image of the tissue slice to be tested is used as the base map, and the high-probability cancerous areas predicted by the model are marked differently. Specifically, the normal area (model output probability <0.1) is marked in green, the cancerous area (model output probability >0.9) is marked in blue, and the suspected area near the threshold is marked in yellow. The visualization effect is as follows: Figure 6 (c) shows that Figure 6 The cancerous areas marked by the doctor in (b) have a high degree of consistency. The software result display interface is as follows Figure 7 shown.

[0058] The above description is only used to illustrate the technical solution of the present invention rather than to limit it. For ordinary professional and technical personnel in the field, the specific technical solution recorded in the above embodiment can be modified, or some of the technical features therein can be replaced by equivalents, and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution protected by the present invention.

Claims

1. A modeling method for cancerous region recognition model based on microscopic spectral feature synthetic image, characterized in that: The following steps are involved: Step 1, construct pathological color data set D1; Step 2, collecting microscopic spectral images of multiple cancer patients' pathological tissue sections, extracting image samples from the collected microscopic spectral images to obtain pseudo-color composite images; marking the pseudo-color composite images to form a marked document; extracting atlas samples from the collected microscopic spectral images according to the marked document to obtain microscopic spectral samples; performing principal component analysis on the extracted microscopic spectral samples, synthesizing the first five principal components with the largest information content in the analysis results into groups of three, and obtaining multiple groups of different microscopic spectral feature composite images corresponding to the current microscopic spectral image; Construct the atlas dataset D2 based on multiple groups of micro-spectral feature synthetic images corresponding to each micro-spectral image; Step 3, constructing a graph migration model; Step 3.1, build a VGG model, input the pathological color dataset D1 into the VGG model for pre-training and performance evaluation, and learn the features and patterns in the pathological color dataset D1; after the pre-training and performance evaluation are completed, set a fully connected layer after the last convolutional layer of the VGG model to form a Trans-CNN model; Step 3.2, input the atlas dataset D2 into the Trans-CNN model, and through transfer learning, transfer the features and patterns learned by the Trans-CNN model in the pathological color dataset D1 to the training task of the atlas dataset D2 for training, so as to obtain the atlas transfer model; In step 3.3, the last convolutional layer and the fully connected layer of the VGG model in the atlas migration model are further trained. The trained atlas migration model is the cancerous region recognition model.

2. The modeling method of the cancerous region recognition model based on microscopic spectral feature synthetic image according to claim 1 is characterized in that: In step 3.2, when the features and patterns learned by the Trans-CNN model in the pathological color dataset D1 are transferred to the training task of the atlas dataset D2 for training, training is performed using a low learning rate and multiple iterations.

3. The modeling method of the cancerous region recognition model based on microscopic spectral feature synthetic image according to claim 2 is characterized in that: In step 3.3, the SGD optimization method is used to further train the last convolutional layer and fully connected layer of the VGG model in the graph transfer model.

4. The modeling method of the cancerous region recognition model based on the microscopic spectral feature synthetic image according to claim 3 is characterized in that: In step 3.1, the pathological color dataset D1 is input into the VGG model for pre-training. Specifically, the pathological color dataset D1 is divided into a training set and a test set and input into the VGG model. The VGG model is pre-trained with the training set, and the performance of the pre-trained VGG model is evaluated with the test set.

5. The modeling method of the cancerous region recognition model based on microscopic spectral feature synthetic image according to claim 4 is characterized in that: In step 1, an existing pathological color data set is used as the pathological color data set D1, or a high-definition camera is used to collect pathological color data to construct the pathological color data set D1.

Images