Full-slice feature extraction method and device in combination with multi-proportion tiles

Abstract

The invention relates to a medical image processing technology, and discloses a full-slice feature extraction method and device combined with multi-proportion tiles, and the method comprises the steps: determining the multi-proportion aspect ratio of a physical distance, extracting tiles of different proportions and sizes, reasoning multi-structure tiles features, and carrying out the cross fusion of tiles level and full-slice level features. The effectiveness of the full-slice features is effectively improved, the execution of subsequent downstream classification, report writing and other tasks is assisted, the labor cost is reduced, the working efficiency is greatly improved, the possibility of misjudgment and missing detection is reduced, and the diagnosis accuracy is ensured.

Description

Technical Field

[0001] The present invention relates to medical image processing technology, and in particular to a full-slice feature extraction method and device combining multi-scale tiles. Background Art

[0002] Different from conventional natural images or other types of small-sized images, the whole slice of tissue pathology is huge in size and has obvious effective and invalid areas. However, due to the large demand for auxiliary diagnosis tasks of tissue pathology and the complex background, relevant practitioners have invested a lot of energy in studying how to algorithmically process digital slices of tissue pathology with a size of tens of thousands by tens of thousands, and how to use advanced methods of deep learning to provide auxiliary diagnostic tools for tissue pathology images. For a long time, whether in the era of convolutional neural networks or TRANSFORMER, most people directly extracted effective TILES (local small-sized images on the whole slice, with a size of hundreds or thousands) from a whole slice image, extracted features from each TILES and performed downstream tasks separately, and finally spliced ​​the TILES results into the processing results of the whole slice according to the needs of classification, segmentation or detection tasks.

[0003] In the past two years, with the continuous development of technology and hardware, the field of tissue pathology research has seen the emergence of single-channel extraction of full-slice features and full-slice downstream tasks. One representative work is: PAIGEAI uses the PERCEIVE network to embed and integrate the features of thousands of TILES extracted by VIRCHOW, obtains the features of the full slice, and then performs downstream tasks. Another representative work is: Microsoft uses the LONGNET structure to process the features of all TILES extracted by TRANSFORMER, obtains the embedded features of the full slice, and then continues the downstream tasks. These two solutions are the most logical and show the best experimental results. However, in principle, both solutions have a process of slicing TILES at the beginning, that is, slicing the full slice. Although this is inevitable from the perspective of digital slice generation, considering the particularity of pathological tissue slices, how to ensure the tissue structure morphology in the tissue slice is also an indispensable factor in improving the effect of downstream tasks.

[0004] Hospitals, medical examination centers and other institutions have a large demand for reading tissue sections from different parts of the body every day. However, manual reading is inefficient. Pathologists will be extremely tired after reading for a long time, which can easily lead to misjudgments and missed detections. At this time, highly accurate auxiliary tools are urgently needed to assist in reading.

[0005] This is a full-slice assisted diagnosis task. Slices from different tissue parts show different tissue structural morphology. It is precisely because of the different tissue structural morphology that doctors can tell at a glance where most slices belong. Often different doctors are responsible for interpreting slices from different tissue parts. For example, cervical lesions generally refer to cervical squamous intraepithelial lesions. Digital slices of cervical tissue generally have very obvious strip-shaped epithelial areas, and the position and direction of this epithelial area are also variable. If the digital slices are only cut into square TILES, the epithelial structure will be very incomplete. Even if the TILES features are spliced ​​in subsequent steps, such structural information cannot be fully combined, and the effect achieved is limited. Therefore, it is necessary to propose a method to better integrate the TILES features of tissue parts with different structures.

[0006] At the same time, if there is such a software that can accurately extract slice features and greatly improve the effect of downstream tasks according to the needs of users, it can be installed on the computer to automatically, quickly and accurately assist in making pre-diagnosis, which can effectively improve the current status of pathology departments. Summary of the invention

[0007] Aiming at the problems of slow speed and low accuracy of slicing technology in the prior art, the present invention provides a full-slice feature extraction method and device combined with multi-scale TILES.

[0008] In order to solve the above technical problems, the present invention is solved by the following technical solutions: A full-slice feature extraction method combining multi-scale tiles, the method comprising: The whole-slice digital image and the acquisition of the whole-slice digital image, inputting the whole-slice digital image, and processing the inputted whole-slice digital image to obtain the MPP value of the whole-slice digital image corresponding to the whole-slice digital image; The aspect ratio of the cropped full-slice digital image is determined based on the pathological structural characteristics of the full-slice digital image.

[0009] Extracting tiles of the full-slice digital image: extracting tiles of the full-slice digital image according to the aspect ratio of the full-slice digital image and the center position of the effective tile area of ​​the full-slice digital image; obtaining n full-slice digital image tiles at each tile aspect ratio; Extracting features of full-slice digital image tiles: extracting features of the extracted full-slice digital image tiles; obtaining n 1*m-dimensional full-slice digital image tile features at each aspect ratio; The features of tiles are cross-fused. The features of the extracted full-slice digital image tiles are cross-fused to obtain the updated n*m-dimensional full-slice digital image tile features under different tile aspect ratios. Slide feature extraction: Slide feature extraction is performed on the tiles features of the full-slice digital image after cross-fusion; Cross-fusion of slices: cross-fusion of slices is performed on the extracted Slide features; 1*l-dimensional full-slice digital image Slide features are obtained; and the cross-fused slice features are output.

[0010] Preferably, the feature extraction of the full-slice digital image tiles includes: Input the full-slice digital image tiles image block, and then update one of the input layers through Dinov2 to extract the features of the full-slice digital image tiles; For full-slice digital image tiles with different aspect ratios, convolution layers with different convolution kernels are selected as embedding layers of the full-slice digital image tiles, thereby outputting features of the full-slice digital image tiles with different aspect ratios.

[0011] Preferably, the feature cross-fusion of tiles includes: Determine the main features and secondary features under all aspect ratios. For the n*m ​​dimensional tiles with multiple aspect ratios, when the feature of one aspect ratio tile is the main feature, the others are secondary features. After adding the secondary feature tiles, they are input into the fully connected layer. The output features will be concatenated with the main features and enter the subsequent operation steps. For the tiles with multiple aspect ratios, they are rotated as the main features. The number of main features plus secondary features is equal to the number of aspect ratios. The splicing of tiles features, the splicing of tile features for the main feature tiles and the secondary feature tiles after the fully connected layer; thus obtaining n*2m dimensions; Fully connected layer dimensionality reduction: Perform dimensionality reduction on the input concatenated tiles to obtain n*m dimensional features and input them into the subsequent fully connected layer; The output of the fusion feature, for the fully connected layer after dimensionality reduction, output its fusion feature.

[0012] Preferably, the center position of the full-slice digital image is statistically analyzed for sample observations of tissue pathology slices, and the difference in pixel grayscale values ​​in the area corresponding to the area containing tissue and the area without tissue is statistically calculated. When the difference in pixel grayscale values ​​is greater than a threshold, the area is the center position.

[0013] Preferably, the feature cross-fusion of tiles includes: For L full-slice digital image features of 1*k dimensions, Route 1 obtains 1*KL-dimensional features through vertical Concat splicing, then inputs the fully connected layer to reduce the dimension to obtain 1*k-dimensional features, and then inputs the fully connected layer to obtain 1*k-dimensional features. The L features are directly summed horizontally to obtain 1*k dimensional features, and then input into the fully connected layer to obtain 1*k dimensional features. Then the 1*k dimensional features of the previous route are vertically spliced ​​and the dimensionality is reduced through the fully connected layer and the fully connected layer to obtain the final 1*k dimensional fused features; the features of the fused full-slice digital image are obtained.

[0014] In order to solve the above technical problems, the present invention also provides a full-slice digital image feature extraction device combined with multi-scale tiles, which comprises: A full-slice digital image and a full-slice digital image acquisition module, an input of a full-slice digital image, an input of a full-slice digital image, and processing the input full-slice digital image to obtain an MPP value of the full-slice digital image corresponding to the full-slice digital image; The module for determining the aspect ratio of the cropped full-slice digital image determines the aspect ratio of the cropped full-slice digital image according to the pathological characteristics of the full-slice digital image.

[0015] The full-slice digital image tiles extraction module extracts the full-slice digital image tiles according to the aspect ratio of the full-slice digital image and the center position of the full-slice digital image; and obtains n full-slice digital image tiles at each aspect ratio; A feature extraction module for full-slice digital image tiles, which extracts features of the extracted full-slice digital image tiles; The tiles feature cross-fusion module cross-fuses the features of the extracted full-slice digital image tiles to obtain n*m-dimensional full-slice digital image tile features; Slide feature extraction module, which extracts Slide features from the cross-fused full-slice digital image tiles; The slice cross-fusion module performs slice cross-fusion on the extracted Slide features; obtains 1*1 dimensional full-slice digital image Slide features; and outputs the cross-fused slice features.

[0016] The present invention has significant technical effects due to the adoption of the above technical solution: The present invention determines the multi-scale aspect ratio of physical distance, extracts tiles of different scales and sizes, infers multi-structure tile features, and cross-fuses tile-level and full-slice-level features, thereby effectively improving the effectiveness of full-slice features, facilitating the execution of subsequent downstream classification, report writing and other tasks, reducing labor costs, greatly improving work efficiency, reducing the possibility of misjudgment and missed detection, and ensuring the accuracy of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a gear dynamics model that takes into account the influence of meshing position changes; Figure 2 The present invention is a comparison diagram of the meshing stiffness of gears considering the influence of meshing position changes and the meshing stiffness of traditional gears; Figure 3 The present invention is a comparison diagram of the gear radial deformation response under the influence of meshing position change and the gear radial deformation response without considering the influence of meshing position change; Figure 4 The present invention is a comparison diagram of the gear axial deformation response under the influence of meshing position change and the gear axial deformation response under the influence of meshing position change not considered. Figure 5 This is a flow chart of feature extraction of a single tile of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0019] Example 1

[0020] A full-slice feature extraction method combining multi-scale tiles, the method comprising: The whole-slice digital image and the acquisition of the whole-slice digital image, inputting the whole-slice digital image, and processing the inputted whole-slice digital image to obtain the MPP value of the whole-slice digital image corresponding to the whole-slice digital image; The aspect ratio of the cropped full-slice digital image is determined based on the pathological structural characteristics of the full-slice digital image.

[0021] Extracting tiles of the full-slice digital image: extracting tiles of the full-slice digital image according to the aspect ratio of the full-slice digital image and the center position of the effective tile area of ​​the full-slice digital image; obtaining n full-slice digital image tiles at each tile aspect ratio; Feature extraction of full-slice digital image tiles: extracting features of the extracted full-slice digital image tiles; obtaining n 1*m-dimensional full-slice digital image tile features at each aspect ratio; The features of tiles are cross-fused. The features of the extracted full-slice digital image tiles are cross-fused to obtain the updated n*m-dimensional full-slice digital image tile features under different tile aspect ratios. Slide feature extraction: Slide feature extraction is performed on the tiles features of the full-slice digital image after cross-fusion; Cross-fusion of slices: cross-fusion of slices is performed on the extracted Slide features; 1*l-dimensional full-slice digital image Slide features are obtained; and the cross-fused slice features are output.

[0022] Feature extraction of full-slice digital image tiles includes: Input the full-slice digital image tiles image block, and then update one of the input layers through Dinov2 to extract the features of the full-slice digital image tiles; For full-slice digital image tiles with different aspect ratios, convolution layers with different convolution kernels are selected as embedding layers of the full-slice digital image tiles, thereby outputting features of the full-slice digital image tiles with different aspect ratios.

[0023] The feature cross-fusion of tiles includes: Determine the main features and secondary features under all aspect ratios. For the n*m ​​dimensional tiles with multiple aspect ratios, when the feature of one aspect ratio tile is the main feature, the others are secondary features. Add the secondary feature tiles and then input them into the fully connected layer. The output features will be concatenated with the main features and enter the subsequent operation steps. For the tiles with multiple aspect ratios, rotate them as the main features. The number of main features plus secondary features is equal to the number of aspect ratios. The splicing of tiles features, the splicing of tile features for the main feature tiles and the secondary feature tiles after the fully connected layer; thus obtaining n*2m dimensions; Fully connected layer dimensionality reduction: Perform dimensionality reduction on the input concatenated tiles to obtain n*m dimensional features and input them into the subsequent fully connected layer; The output of the fusion feature, for the fully connected layer after dimensionality reduction, output its fusion feature.

[0024] The center position of the full-slice digital image, the statistics of sample observations of tissue pathology slices, and the difference in pixel grayscale values ​​in the corresponding areas of the areas containing tissue and the areas without tissue are counted. When the difference in pixel grayscale values ​​is greater than the threshold, the area is the center position.

[0025] The feature cross-fusion of tiles includes: For L full-slice digital image features of 1*k dimensions, Route 1 obtains 1*KL-dimensional features through vertical Concat splicing, then inputs the fully connected layer to reduce the dimension to obtain 1*k-dimensional features, and then inputs the fully connected layer to obtain 1*k-dimensional features. The L features are directly summed horizontally to obtain 1*k dimensional features, and then input into the fully connected layer to obtain 1*k dimensional features. Then the 1*k dimensional features of the previous route are vertically spliced ​​and the dimensionality is reduced through the fully connected layer and the fully connected layer to obtain the final 1*k dimensional fused features; the features of the fused full-slice digital image are obtained.

[0026] Example 2

[0027] Based on Example 1, in this example, after taking tissue from the patient, the doctor will make it into a physical slide that can be observed under a microscope. The physical slide can be stored for about 10 years and will fade over time. Due to the emergence of scanners, many laboratories will scan physical slides into digital slides for permanent storage, and they do not take up space in the hospital. Correspondingly, assuming a slide with a physical distance of 75mm*25mm, after scanning, it may be a digital slice of tens of thousands*tens of thousands of pixels. At the same time, due to the differences in scanners from different manufacturers, the images after scanning are also very different, but the digital slices will retain an MPP parameter, which means how many microns of physical distance each pixel on the digital slice corresponds to. If we want to observe the image within the physical distance range of 100*100 microns at a certain position on the slide, according to the MPP value of the digital slice, we can infer how many pixels of the image on the digital slice need to be cropped, so as to obtain the same area range on different scanners.

[0028] like Figure 1 As shown in the figure, a full-slice image of any part is input, and the features of the full-slice are directly output. For a full-slice digital image, its size and micrometer per pixel (MPP) values ​​vary greatly, and scanners from different manufacturers also have a large gap. Therefore, relevant practitioners generally determine the physical distance of the slice tiles that need to be cropped (unit: micrometer, the size of the corresponding tissue area on the actual glass slide), and then determine the pixel value size of the current slice to be cropped based on the physical distance divided by the MPP of the current slice, so that the final cropped tiles have a consistent physical distance. Therefore, considering the adaptability to slices of any type and format, this article directly simplifies it to the ratio of different physical distances.

[0029] First, set several different physical distance aspect ratios based on experience (many target forms have a length-to-width ratio of 3:1, so set an aspect ratio of 3:1. You can set multiple aspects, depending on the actual target form). Suppose you set three different aspect ratios of 1:1, 1:3, and 3:1. Figure 4The center point is used as the center of the conventional cropping with an aspect ratio of 1:1. Around the center point, three tiles are cropped with three (or more) set ratios. Therefore, the same tissue pathology section can be cut into exactly the same number of tiles with different aspect ratios.

[0030] To ensure that the same number of tiles are cropped from the same slice, the effective area of ​​1:1 tiles (that is, valid tiles) can also get the center position of these 1:1 tiles. The cropping of tiles of different ratios needs to rely on the center position of the 1:1 tiles. The following describes how to extract the effective organization tile area.

[0031] Based on the statistics of sample observations of tissue pathology sections, the difference in pixel grayscale values ​​in the corresponding areas of areas containing tissue and areas without tissue is calculated. Assuming the threshold is k, when the mean grayscale value in the area exceeds this value, the area is a valid area containing tissue areas, otherwise it is a background area, and these background areas do not need to be considered when extracting tiles. Therefore, the number of valid areas is n, the number of center positions is n, and the number of tiles with different aspect ratios is also n.

[0032] After extracting tiles with different aspect ratios, we obtained tiles of tissue sections cut into square, 1:3 rectangular and 3:1 rectangular shapes (depending on the aspect ratio setting, tiles of various shapes can be obtained), and then entered the subsequent tiles feature extraction stage.

[0033] Due to the three tiles of different proportions, in order to maintain the structural relationship of tiles of different proportions, tiles use different convolution kernels to perform linear projection operations of Patch Embedding during feature extraction to obtain input embedding. Therefore, this paper updates one of the input layers (convolution layer of Patch Embedding) as a tile feature extractor based on Dinov2. For the three aspect ratios of 1:1, 1:3 and 3:1, convolution layers with convolution kernel size ratios of 1:1, 1:3 and 3:1 (the actual convolution kernel size can be (16, 16), (16, 48), (48, 16)) are selected as tiles embedding layers, so that the dimensions of the final tile embedding are consistent.

[0034] Finally, the transformer layer of Dinov2 outputs the n*m ​​dimensional tiles features, where n is the total number of tiles in this slice, and m refers to the feature dimension of a single tile. The feature extraction process of a single tile is as follows: Figure 5 shown.

[0035] After obtaining the features of tiles of different proportions at the same position on a single slice, the next step is to perform feature cross-fusion module A. Figure 2 First, add two secondary features (features of three proportions, one of which is the main feature and the other two are secondary features. The original features of each proportion are used as the main features in turn, and the dimensions are all n*m) directly, and then enter the fully connected layer (output n*m dimensions), and then splice with the main features to obtain n*2m-dimensional features, then reduce the dimension through the fully connected layer to obtain n*m-dimensional features, and finally pass through a fully connected layer to output the updated n*m-dimensional main features after fusion.

[0036] Figure 1 The structure of the three feature cross fusion A modules in the above example is exactly the same. The difference is that each time different proportions of features are selected as the main features, and the feature cross fusion A module after the tile feature extraction module is based on tile feature 1 as the main feature. Due to the three different proportions set by default, tile features 1, 2, and 3 are rotated as the main features in turn. For example, when tile feature 1 is the main feature, tile features 2 and 3 are secondary features. Figure 2 Through the process, we obtain the updated tiles fusion feature 1. Similarly, we also obtain tiles fusion feature 2 and tiles fusion feature 3. The feature dimension after fusion is still n*m.

[0037] The fused features of the three scales are each fed into the [slide feature extraction] module, which specifically refers to the Perceiver network, and each outputs a 1*k-dimensional full-slice feature embedding, that is, the 1*k-dimensional full-slice feature 1 of scale 1, the 1*k-dimensional full-slice feature 2 of scale 2, and the 1*k-dimensional full-slice feature 3 of scale 3. Finally, it enters the feature cross-fusion B module. The specific process is as follows Figure 3 shown.

[0038] For the 1*k-dimensional full-slice features 1, 2, and 3, one route is to obtain 1*3k-dimensional features through vertical Concat splicing, then input the fully connected layer for dimensionality reduction to obtain 1*k-dimensional features, and then input the fully connected layer to obtain 1*k-dimensional features. Another route is to directly sum the three features horizontally to obtain 1*k-dimensional features, then input the fully connected layer to obtain 1*k-dimensional features, and then vertically splice the 1*k-dimensional features of the two routes and pass them through two fully connected layers to obtain the final 1*k-dimensional fused features. In this way, the fused full-slice features are obtained.

[0039] In order to be consistent with clinical needs, based on the fused 1*k-dimensional full-slice features, we can perform tasks such as full-slice classification, full-slice report generation, and key area extraction. For different tasks, we collect the required training and test data accordingly, and conduct full-process model training. After the training is completed, we can obtain the inference model of the subtask.

[0040] Assume that the number of valid regions is n, the number of center positions is n, and the number of tiles with different aspect ratios is n. Assume the aspect ratios are 1:3, 1:1, and 3: Extraction of full-slice digital image tiles: Each aspect ratio outputs n r*s dimensional tiles. For example, if the ratio is 1:1, then r and s can be 256. Tile feature extraction: Each aspect ratio is extracted separately. Input n tiles of r*s dimension and output n*m (also called n 1*m) dimensional tile features, where n is the total number of tiles in this slice and m is the feature dimension of a single tile. So three n*m ​​dimensional tile features are generated.

[0041] Feature cross fusion A: When the n*m ​​dimensional feature output by tiles feature extraction 1 is the main feature, the n*m ​​dimensional feature output by tiles feature extraction 2 and tiles feature extraction 3 is the secondary feature. After feature cross fusion A processing, the fused updated n*m dimensional main feature of branch 1 is obtained.

[0042] When the n*m ​​dimensional feature output by tiles feature extraction 2 is the main feature, the n*m ​​dimensional feature output by tiles feature extraction 1 and tiles feature extraction 3 is the secondary feature. After feature cross fusion A processing, the fused n*m dimensional main feature of the updated branch 2 is obtained.

[0043] When the n*m-dimensional features output by tiles feature extraction 3 are the main features, the n*m-dimensional features output by tiles feature extraction 1 and tiles feature extraction 2 are the secondary features. After feature cross fusion A, the updated n*m-dimensional main features of branch 3 are obtained. Slide feature extraction module: This module specifically refers to the Perceiver network. Each branch inputs different n*m-dimensional features and outputs 1*k-dimensional full-slice feature embedding. Feature cross fusion B: Input the 1*k-dimensional features on the three branches and output the updated 1*k features.

[0044] Example 3

[0045] Based on the above embodiment, this embodiment is a full-slice digital image feature extraction device combined with multi-scale tiles, which includes: A full-slice digital image and a full-slice digital image acquisition module, an input of a full-slice digital image, an input of a full-slice digital image, and processing the input full-slice digital image to obtain an MPP value of the full-slice digital image corresponding to the full-slice digital image; The module for determining the aspect ratio of the cropped full-slice digital image determines the aspect ratio of the cropped full-slice digital image according to the pathological characteristics of the full-slice digital image.

[0046] The full-slice digital image tiles extraction module extracts the full-slice digital image tiles according to the aspect ratio of the full-slice digital image and the center position of the full-slice digital image; and obtains n full-slice digital image tiles at each aspect ratio; A full-slice digital image tile feature extraction module performs tile feature extraction on the extracted full-slice digital image tiles; The tiles feature cross-fusion module cross-fuses the features of the extracted full-slice digital image tiles to obtain n*m-dimensional full-slice digital image tiles; Slide feature extraction module, which extracts Slide features from the cross-fused full-slice digital image tiles; The slice cross-fusion module performs slice cross-fusion for extracting Slide features; obtains 1*1 dimensional full-slice digital image Slide features; and outputs the slice features after cross-fusion.

Claims

1. A full-slice feature extraction method combining multi-scale tiles, the method comprising: The whole-slice digital image and the acquisition of the whole-slice digital image, inputting the whole-slice digital image, and processing the inputted whole-slice digital image to obtain the MPP value of the whole-slice digital image corresponding to the whole-slice digital image; Determining the aspect ratio of the cropped full-slice digital image, and determining the aspect ratio of the cropped full-slice digital image according to the pathological structural characteristics of the full-slice digital image; Extracting tiles of the full-slice digital image: extracting tiles of the full-slice digital image according to the aspect ratio of the full-slice digital image and the center position of the effective tile area of ​​the full-slice digital image; obtaining n full-slice digital image tiles at each tile aspect ratio; Feature extraction of full-slice digital image tiles: extracting features of the extracted full-slice digital image tiles; obtaining n 1*m-dimensional full-slice digital image tile features at each aspect ratio; The features of tiles are cross-fused. The features of the extracted full-slice digital image tiles are cross-fused to obtain the updated n*m-dimensional full-slice digital image tile features under different tile aspect ratios. Slide feature extraction: Slide feature extraction is performed on the tiles features of the full-slice digital image after cross-fusion; Cross-fusion of slices: cross-fusion of slices is performed on the extracted Slide features; 1*l-dimensional full-slice digital image Slide features are obtained; and the cross-fused slice features are output.

2. A method for extracting features from a full-slice digital image combining multi-scale tiles according to claim 1, characterized in that: Feature extraction of full-slice digital image tiles includes: Input the full-slice digital image tiles image block, and then update one of the input layers through Dinov2 to extract the features of the full-slice digital image tiles; For full-slice digital image tiles with different aspect ratios, convolution layers with different convolution kernels are selected as embedding layers of the full-slice digital image tiles, thereby outputting features of the full-slice digital image tiles with different aspect ratios.

3. The method for extracting features of a full-slice digital image combining multi-scale tiles according to claim 1, characterized in that: The feature cross-fusion of tiles includes: Determine the main features and secondary features under all aspect ratios. For the n*m ​​dimensional tiles with multiple aspect ratios, when the feature of one aspect ratio tile is the main feature, the others are secondary features. After adding the secondary feature tiles, they are input into the fully connected layer. The output features will be concatenated with the main features and enter the subsequent operation steps. For the tiles with multiple aspect ratios, they are rotated as the main features. The number of main features plus secondary features is equal to the number of aspect ratios. The splicing of tiles features, the splicing of tile features for the main feature tiles and the secondary feature tiles after the fully connected layer; thus obtaining n*2m dimensions; Fully connected layer dimensionality reduction: Perform dimensionality reduction on the input concatenated tiles to obtain n*m dimensional features and input them into the subsequent fully connected layer; The output of the fusion feature, for the fully connected layer after dimensionality reduction, output its fusion feature.

4. The full-slice feature extraction method combining multi-scale tiles according to claim 1, characterized in that: The center position of the full-slice digital image, the statistics of sample observations of tissue pathology slices, and the difference in pixel grayscale values ​​in the corresponding areas of the areas containing tissue and the areas without tissue are counted. When the difference in pixel grayscale values ​​is greater than the threshold, the area is the center position.

5. The method for extracting features of a full-slice digital image combining multi-scale tiles according to claim 1, characterized in that: The cross-fusion of full-slice digital image features includes: For L full-slice digital image features of 1*k dimensions, Route 1 obtains 1*KL-dimensional features through vertical Concat splicing, then inputs the fully connected layer to reduce the dimension to obtain 1*k-dimensional features, and then inputs the fully connected layer to obtain 1*k-dimensional features. The L features are directly summed horizontally to obtain 1*k dimensional features, and then input into the fully connected layer to obtain 1*k dimensional features. Then the 1*k dimensional features of the previous route are vertically spliced ​​and the dimensionality is reduced through the fully connected layer and the fully connected layer to obtain the final 1*k dimensional fused features; the features of the fused full-slice digital image are obtained.

6. A full-slice digital image feature extraction device combining multi-scale tiles, comprising: A full-slice digital image and a full-slice digital image acquisition module, an input of a full-slice digital image, an input of a full-slice digital image, and processing the input full-slice digital image to obtain an MPP value of the full-slice digital image corresponding to the full-slice digital image; A module for determining the aspect ratio of the cropped full-slice digital image, which determines the aspect ratio of the cropped full-slice digital image according to the pathological characteristics of the full-slice digital image; The full-slice digital image tiles extraction module extracts the full-slice digital image tiles according to the aspect ratio of the full-slice digital image and the center position of the full-slice digital image; and obtains n full-slice digital image tiles at each aspect ratio; A feature extraction module for full-slice digital image tiles, which extracts features of the extracted full-slice digital image tiles; The tiles feature cross-fusion module cross-fuses the features of the extracted full-slice digital image tiles to obtain n*m-dimensional full-slice digital image tiles; Slide feature extraction module, which extracts Slide features from the cross-fused full-slice digital image tiles; The slice cross-fusion module performs slice cross-fusion for extracting Slide features; obtains 1*1 dimensional full-slice digital image Slide features; and outputs the slice features after cross-fusion.

Images