Histopathology full-slice image classification method and system based on position coding converter

Abstract

The invention belongs to the technical field of histopathology full-slice image analysis, and particularly relates to a histopathology full-slice image classification method and system based on a position coding converter, and the method comprises the steps: obtaining a pathological full-slice data set with an image and a class label thereof, and carrying out the segmentation preprocessing; using a feature encoder to extract block image features, and establishing a converter multi-instance learning network model based on position coding guidance; training a network model by using the pathological full-slice data set containing the weak label; according to the method, the problem that detailed annotations of pathological images are difficult to obtain can be solved through a multi-instance learning weak supervision method, and the importance of spatial and semantic relationships among different sub-regions in the pathological sections can be effectively considered; and completing a classification task of pathological full slices and carrying out visual interpretive analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of histopathology whole-slice image analysis, and in particular relates to a histopathology whole-slice image classification method and system based on a position encoding converter. Background Art

[0002] Traditional pathology full-slice diagnosis relies on experienced pathologists to observe the morphological characteristics of suspicious areas under a microscope to draw a diagnosis. However, with the increasing demand for pathology diagnosis, the workload of pathologists has gradually increased, resulting in a negative impact on the efficiency of the entire pathology diagnosis process. In order to improve diagnostic efficiency and accuracy, artificial intelligence technology has been introduced into pathology image analysis. In particular, with the popularization of digital scanners, pathology full slices can be digitally stored and observed on the screen, which makes it possible to use artificial intelligence technology for pathology image analysis.

[0003] Due to the high resolution of pathological whole sections, they cannot be directly input into neural networks for learning like natural images. Each pathological whole section needs to be cut into image blocks of appropriate size for analysis. However, there is usually only slice-level annotation information, and there is a lack of fine-grained annotation, which makes it impossible to use strong supervised learning for training. To solve this problem, weakly supervised learning methods, especially multi-instance learning methods, are widely used in pathological image analysis. However, existing multi-instance learning methods often ignore the location information within the tissue region, which is closely related to the tumor microenvironment and the growth and development process of the tumor, and may provide additional useful information for diagnosis. Therefore, existing methods have certain limitations when processing pathological whole sections. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for classifying whole-slice images of tissue pathology based on a position coding converter, so as to solve the problems of lack of detailed annotation of pathological images and neglect of position information within tissue areas proposed in the background technology. By introducing a position coding module and constructing a multi-instance learning network model of a converter guided by position coding, accurate recognition and classification of tumor areas in whole-slice pathological sections are achieved.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] In a first aspect, the present invention proposes a method for classifying whole-slice images of histopathology based on a position encoding converter, characterized in that the method comprises:

[0007] S1. Obtain a pathological full-slice dataset with images and their category labels and perform block preprocessing to obtain a block image set X i , in represents the jth block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice;

[0008] S2, using a feature encoder to extract the block image set X i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features

[0009] S3, normalizing the coordinate information of the block image to form a block image with characteristics E i and the data set to be processed with normalized coordinate information;

[0010] S4, input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ;

[0011] S5, predicting the fusion feature F through mask and self-attention mechanism to obtain the reconstructed coordinates;

[0012] S6. Predict P based on the probability i , calculating the cross entropy loss and the position reconstruction loss based on the mean square error using the normalized coordinate information and the reconstructed coordinates, and training the position encoding-guided converter multi-instance learning network model based on the position reconstruction loss;

[0013] S7. Use the trained position-encoding-guided transformer multi-instance learning network model to classify the target digital pathology full slice image and obtain the classification result.

[0014] Further, the S1 includes:

[0015] S101, obtaining a pathological image dataset D with image category labels = {(X1, Y1), (X2, Y2), ..., (X N , Y N )}, where X i represents the i-th pathological full slice in the pathological image dataset D, Y i represents the label of the i-th image in the pathological image dataset D, and N represents the number of pathological full slices contained in the pathological image dataset D;

[0016] S102, using the Otsu threshold algorithm to generate a tissue mask image of the entire pathological slice, and then using the window sliding method to X i Cut into fixed-size block images, filter out the background block images through the tissue mask image, retain the block images of the tissue area, and obtain the block image set X after segmentation i ,

[0017] Further, the S3 includes:

[0018] S301, decomposing the position information into two lists L representing row and column information row ={r1, r2, ..., r n} and L col ={c1, c2, ..., c n}, where r i and c i Respectively represent the row and column values ​​in the position information of the i-th block image; calculate the minimum and maximum values ​​of the rows and columns in the block image set, and the formula is:

[0019] r min , r max ←min(L row ), max(L row )

[0020] C min , c max ←min(L col ), max(L col )

[0021] Among them, min(·) and max(·) represent functions for obtaining the minimum and maximum element values ​​in a list, respectively;

[0022] S302, the maximum interval of rows and columns is calculated by the minimum and maximum values ​​of rows and columns, the maximum value is defined as the scale scaling factor λ, and then the minimum values ​​of rows and columns are subtracted from all coordinate information in the block image set to achieve overall translation, and all position information is normalized using the scale scaling factor λ, and the formula is:

[0023] λ←max(r max -r min , c max -c min )

[0024]

[0025] Among them, r′ i and c′ iRespectively represent the normalized row and column information of the position information of the ith block image. All the block images The normalized position information is expressed as O = {O1, O2, ..., O n}={(r′1,c′1), (r′2,c′2),…, (r′ n , c′ n )}.

[0026] Furthermore, the pre-built position-encoding-guided transformer multi-instance learning network model includes: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein,

[0027] The position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function; the position encoding module is used to receive and process the normalized position information O from S302 to obtain the spatial embedding features O′ of all the block images;

[0028] The position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function; the position decoding module is used to receive the fusion feature F after random masking according to the set mask ratio. mask , and process it to get the predicted coordinates O pred ={(r″1,c″1), (r″2,c″2),…,(r″ n , c″ n )}.

[0029] Further, the S4 includes:

[0030] S401, the block image feature E obtained in S2 i Fuse with the spatial embedding feature O′ to obtain a fused feature F;

[0031] S402, inputting the fusion feature F into the multi-head self-attention module of the backbone network to exchange information between the divided image blocks, and then pooling the output features to obtain a global feature h that characterizes the entire digital pathology slice;

[0032] S403: Input the global feature h into the classifier for processing to obtain the probability prediction P i .

[0033] Further, the S6 includes:

[0034] S601, predicting P based on the label Y of the whole pathological section and the probability i Calculate the cross entropy loss L slide :

[0035]

[0036] Where C represents the number of categories, is the probability of the jth category in the prediction result of the i-th tissue pathology section, Y i is the one-hot encoded label of the ith full slice;

[0037] S602: According to the normalized position information O and the predicted coordinates O pred Calculate the position reconstruction loss L based on the mean square error pos :

[0038]

[0039] Among them, τ represents a temperature value;

[0040] S603, jointly optimizing and training the position encoding-guided converter multi-instance learning network model by using the cross entropy loss and the mean square error-based position reconstruction loss.

[0041] In a second aspect, the present invention proposes a histopathology full-slice image classification system based on a position encoding converter, which is applied to perform any of the above-mentioned histopathology full-slice image classification methods, and the system comprises:

[0042] The data preprocessing module is used to obtain the pathological full-slice dataset with images and their category labels and perform block preprocessing to obtain the block image set X i , in represents the jth block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice;

[0043] A feature extraction module is used to extract the block image set X using a feature encoder i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features

[0044] A coordinate normalization module is used to normalize the coordinate information of the block image to form a block image with block image features E i and the data set to be processed with normalized coordinate information;

[0045] The model processing module is used to input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ;

[0046] A coordinate reconstruction module, used to predict the fusion feature F through a mask and self-attention mechanism to obtain a reconstructed coordinate;

[0047] Model training module, used to predict P according to the probability i , calculating the cross entropy loss and the position reconstruction loss based on the mean square error using the normalized coordinate information and the reconstructed coordinates, and training the position encoding-guided converter multi-instance learning network model based on the position reconstruction loss;

[0048] The image classification module is used to classify the target digital pathology full slice image by using the trained position encoding-guided transformer multi-instance learning network model to obtain a classification result.

[0049] Furthermore, the pre-built position-encoding-guided transformer multi-instance learning network model includes: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein,

[0050] The position encoding module is configured to receive normalized position information provided by the coordinate normalization module; the position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function, which is used to process the normalized position information and output the spatial embedding features of all the block images;

[0051] The position decoding module is configured to receive masked fused features, where the fused features are obtained by randomly masking the features processed by the multi-head self-attention module of the backbone network according to a set mask ratio; the position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function, which are used to process the masked fused features and output predicted coordinates.

[0052] The beneficial effects of the present invention are:

[0053] 1. The full-slice image classification method proposed in the present invention can effectively solve the problem of lack of detailed annotation of pathological images. By introducing a position encoding module, the present invention can encode detailed position information, thereby learning the relationship between different tissue regions in the full pathological slice, and providing more information for the classification of the model. This method not only improves the efficiency and accuracy of diagnosis, but also helps to reveal the potential connection between pathological tissues, providing a more comprehensive basis for the diagnosis and treatment of diseases.

[0054] 2. The multi-instance learning network model of the converter guided by position coding constructed by the present invention makes full use of the advantages of the multi-head self-attention mechanism and position coding technology to achieve effective information exchange between features and accurate learning of spatial semantics. During the model training process, the normalized position information is processed by the position coding module to obtain the spatial embedding features of all block images. These features not only contain the semantic information of the image, but also incorporate the position information, so that the model can better understand the spatial relationship in the image. At the same time, the position decoding module predicts the reconstruction coordinates through masks and self-attention mechanisms, further strengthening the model's understanding and utilization of position information. This design not only improves the classification performance of the model, but also helps the model maintain the consistency of feature space semantics during training, thereby avoiding problems such as overfitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of a process of the histopathology whole-slice image classification method of the present invention;

[0056] Figure 2 A technical roadmap for the histopathology whole-slice image classification method of the present invention;

[0057] Figure 3 A model architecture diagram of a position-encoding-guided converter multi-instance learning network model in the present invention;

[0058] Figure 4 This is a system architecture diagram of the tissue pathology whole-slice image classification system in the present invention. DETAILED DESCRIPTION

[0059] The present application is further described in detail below in conjunction with the accompanying drawings. It is necessary to point out here that the following specific implementation methods are only used to further illustrate the present application and cannot be understood as limiting the scope of protection of the present application. Technical personnel in this field can make some non-essential improvements and adjustments to the present application based on the above application content.

[0060] Example 1

[0061] like Figure 1-3As shown, this embodiment proposes a method for classifying whole-slice images of histopathology based on a position encoding converter, which uses a position encoding module to introduce detailed position information and the relationship between different tissue regions in the whole-slice pathology. In addition, the position information of the feature is predicted by the position decoding module to maintain the consistency of spatial semantics during the feature learning process; in specific implementation, the method includes the following steps:

[0062] S1. Obtain a pathological full-slice dataset with images and their category labels and perform block preprocessing to obtain a block image set X i , in represents the jth block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice;

[0063] Further preferably, the S1 comprises:

[0064] S101, obtaining a pathological image dataset D with image category labels = {(X1, Y1), (X2, Y2), ..., (X N , Y N )}, where X i represents the i-th pathological full slice in the pathological image dataset D, Y i represents the label of the i-th image in the pathological image dataset D, and N represents the number of pathological full slices contained in the pathological image dataset D;

[0065] S102, using the Otsu threshold algorithm to generate a tissue mask image of the entire pathological slice, and then using the window sliding method to X i Cut into fixed-size block images, filter out the background block images through the tissue mask image, retain the block images of the tissue area, and obtain the block image set X after segmentation i ,

[0066] In some optional embodiments, the data used includes two breast cancer subtypes: 794 cases of invasive ductal carcinoma (IDC) and 204 cases of invasive lobular carcinoma (ILC). The information of each case is verified and confirmed by a professional pathologist. All data are divided into training set and test set according to 7:3. For each pathological section, it is divided into 256×256 block images without duplication at 20 times.

[0067] S2, using a feature encoder to extract the block image set X i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features

[0068] In some optional embodiments, the pre-trained feature encoder uses a convolutional neural network ResNet50 loaded with weights trained on the natural image dataset ImageNet, and the dimension of extracted features is 1024 dimensions.

[0069] S3, normalizing the coordinate information of the block image to form a block image with characteristics E i and the data set to be processed with normalized coordinate information;

[0070] Further preferably, the S3 includes:

[0071] S301, decomposing the position information into two lists L representing row and column information row ={r1, r2, ..., r n} and L col ={c1, c2, ..., c n}, where r i and c i Respectively represent the row and column values ​​in the position information of the i-th block image; calculate the minimum and maximum values ​​of the rows and columns in the block image set, and the formula is:

[0072] r min , r max ←min(L row ), max(L row )

[0073] c min , c max ←min(L col ), max(L col )

[0074] Among them, min(·) and max(·) represent functions for obtaining the minimum and maximum element values ​​in a list, respectively;

[0075] S302, the maximum interval of rows and columns is calculated by the minimum and maximum values ​​of rows and columns, the maximum value is defined as the scale scaling factor λ, and then the minimum values ​​of rows and columns are subtracted from all coordinate information in the block image set to achieve overall translation, and all position information is normalized using the scale scaling factor λ, and the formula is:

[0076] λ←max(r max -r min , c max -c min )

[0077]

[0078] Among them, r′ i and c′ i Respectively represent the normalized row and column information of the position information of the ith block image. All the block images The normalized position information is expressed as O = {O1, O2, ..., O n}={(r′1,c′1), (r′2,c′2),…, (r′ n , c′ n )}.

[0079] S4, input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ;

[0080] Further preferably, in step S4, the pre-constructed position-encoding-guided converter multi-instance learning network model comprises: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein,

[0081] The position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function; the position encoding module is used to receive and process the normalized position information O from S302 to obtain the spatial embedding features O′ of all the block images;

[0082] In an embodiment of the present invention, the calculation process of the multi-head attention (converter) module is shown in the following formula:

[0083] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0084]

[0085] Q=OW Q , K=OW K , V=OW V

[0086] Among them, Concat(·) is the concatenation function, W O , W Q , W K and W V is a learnable network parameter, d is the feature dimension (1024);

[0087] The position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function; the position decoding module is used to receive the fusion feature F after random masking according to the set mask ratio. mask , and process it to get the predicted coordinates O pred ={(r″1,c″1), (r″2,c″2),…,(r″ n , c″ n )}.

[0088] Further preferably, the S4 includes:

[0089] S401, the block image feature E obtained in S2 i Fuse with the spatial embedding feature O′ to obtain a fused feature F;

[0090] S402, inputting the fusion feature F into the multi-head self-attention module of the backbone network to exchange information between the divided image blocks, and then pooling the output features to obtain a global feature h that characterizes the entire digital pathology slice;

[0091] S403: Input the global feature h into the classifier for processing to obtain the probability prediction P i .

[0092] S5, predicting the fusion feature F through mask and self-attention mechanism to obtain the reconstructed coordinates;

[0093] It can be understood that in step S5 of this embodiment, the position decoding module is used to predict the position information of all the block images. The specific process first pre-sets a mask ratio m, and then randomly masks the fusion feature F according to the mask ratio m to obtain the masked feature F mask Then, the feature F mask Input the position decoding module to obtain a set of predicted coordinates O pred ={(r″1,c″1), (r″2,c″2),…,(r″ n , c″ n )}. In some optional embodiments, the mask ratio m is set to 0.25.

[0094] S6. Predict P based on the probability i, the normalized coordinate information and the reconstructed coordinates calculate the cross entropy loss and the position reconstruction loss based on the mean square error, and train the position encoding guided converter multi-instance learning network model based on the position reconstruction loss; specifically, the cross entropy loss and the position reconstruction loss based on the mean square error are calculated respectively. The two losses are combined together to optimize the neural network model;

[0095] Further preferably, the S6 comprises:

[0096] S601, predicting P based on the label Y of the whole pathological section and the probability i Calculate the cross entropy loss L slide :

[0097]

[0098] Where C represents the number of categories, is the probability of the jth category in the prediction result of the i-th tissue pathology section, Y i is the one-hot encoded label of the ith full slice;

[0099] S602: According to the normalized position information O and the predicted coordinates O pred Calculate the position reconstruction loss L based on the mean square error pos :

[0100]

[0101] Among them, τ represents a temperature value;

[0102] S603, jointly optimizing and training the position encoding-guided converter multi-instance learning network model by using the cross entropy loss and the mean square error-based position reconstruction loss.

[0103] S7. Use the trained position-encoding-guided transformer multi-instance learning network model to classify the target digital pathology full slice image and obtain the classification result.

[0104] It should be noted that, in the embodiment of the present invention, the testing process of step S7 includes data processing of the whole pathological section (steps S2 and S3) and neural network model reasoning (S4).

[0105] According to the above-mentioned embodiments of the present invention, the present invention can complete the training of the network using only slice-level labels without the need for detailed annotation of cell regions, so that the present invention has a broader application scenario; the scheme can encode detailed position information by introducing a position encoding module, so as to learn the relationship between different tissue regions in the whole pathological slice and provide more information for the classification of the model; the self-attention module is used to exchange information between features, including position information encoding, feature semantic learning and position information prediction. In this way, the network model can learn richer spatial semantic information and improve the accuracy of classification; and a unique mask-based position decoding module is designed, which alleviates the overfitting problem caused by the limited number of samples through the mask mechanism. By predicting the position information of the fused features, the network model can always maintain the consistency of the feature space semantics during the training process.

[0106] like Figure 4 As shown, based on the same inventive concept, this embodiment also proposes a histopathology whole-slice image classification system based on a position encoding converter, which is applied to execute the above-mentioned histopathology whole-slice image classification method, and the system includes:

[0107] The data preprocessing module 10 is used to obtain a pathological full-slice data set with images and their category labels and perform block preprocessing to obtain a block image set X i , in represents the i-th block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice;

[0108] The feature extraction module 11 is used to extract the block image set X using a feature encoder. i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features

[0109] The coordinate normalization module 12 is used to normalize the coordinate information of the block image to form a block image with the block image feature E i and the data set to be processed with normalized coordinate information;

[0110] The model processing module 13 is used to input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ;

[0111] A coordinate reconstruction module 14, used to predict the fusion feature F through a mask and a self-attention mechanism to obtain a reconstructed coordinate;

[0112] Model training module 15, used to predict P according to the probability i , calculating the cross entropy loss and the position reconstruction loss based on the mean square error using the normalized coordinate information and the reconstructed coordinates, and training the position encoding-guided converter multi-instance learning network model based on the position reconstruction loss;

[0113] The image classification module 16 is used to classify the target digital pathology full slice image using the trained position coding-guided transformer multi-instance learning network model to obtain a classification result.

[0114] Further preferably, the pre-built position encoding-guided converter multi-instance learning network model includes: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein,

[0115] The position encoding module is configured to receive normalized position information provided by the coordinate normalization module; the position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function, which is used to process the normalized position information and output the spatial embedding features of all the block images;

[0116] The position decoding module is configured to receive masked fused features, where the fused features are obtained by randomly masking the features processed by the multi-head self-attention module of the backbone network according to a set mask ratio; the position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function, which are used to process the masked fused features and output predicted coordinates.

[0117] It should be noted here that each module in the above classification system corresponds to steps S1 to S7 in implementing the above tissue pathology whole-slice image classification method, and the instances and application scenarios implemented by multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1.

[0118] In another embodiment provided by the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned histopathology whole-slice image classification methods are implemented.

[0119] In another embodiment provided by the present invention, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any of the histopathology whole-slice image classification methods in the above embodiments.

[0120] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for classifying whole-slice images of histopathology based on a positional encoder converter, characterized in that: The method comprises: S1. Obtain a pathological full-slice dataset with images and their category labels and perform block preprocessing to obtain a block image set X i , in represents the jth block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice; S2, using a feature encoder to extract the block image set X i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features S3, normalizing the coordinate information of the block image to form a block image with characteristics E i and the data set to be processed with normalized coordinate information; S4, input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ; S5, predicting the fusion feature F through mask and self-attention mechanism to obtain the reconstructed coordinates; S6. Predict P based on the probability i , calculating the cross entropy loss and the position reconstruction loss based on the mean square error using the normalized coordinate information and the reconstructed coordinates, and training the position encoding guided converter multi-instance learning network model based on the position reconstruction loss; S7. Use the trained position-encoding-guided transformer multi-instance learning network model to classify the target digital pathology full slice image and obtain the classification result.

2. The method for classifying whole-slice images of histopathology based on positional encoder converter according to claim 1, characterized in that: The S1 includes: S101, obtain a pathological image dataset D with image category labels = {(X1, Y1), (X2, Y2), ..., (X N ,Y N )}, where X i represents the i-th pathological full slice in the pathological image dataset D, Y i represents the label of the i-th image in the pathological image dataset D, and N represents the number of pathological full slices contained in the pathological image dataset D; S102, using the Otsu threshold algorithm to generate a tissue mask image of the entire pathological slice, and then using the window sliding method to X i Cut into fixed-size block images, filter out the background block images through the tissue mask image, retain the block images of the tissue area, and obtain the block image set X after segmentation i , 3. The method for classifying whole-slice images of histopathology based on positional encoder converter according to claim 1, characterized in that: The S3 includes: S301, decomposing the position information into two lists L representing row and column information row ={r1,r2,…,r n } and L col ={c1,c2,…,c n }, where r i and c i Respectively represent the row and column values ​​in the position information of the i-th block image; calculate the minimum and maximum values ​​of the rows and columns in the block image set, and the formula is: r min ,r max ←min(L row ),max(L row ) c min ,c max ←min(L col ),max(L col ) Among them, min(·) and max(·) represent functions for obtaining the minimum and maximum element values ​​in a list, respectively; S302, the maximum interval of rows and columns is calculated by the minimum and maximum values ​​of rows and columns, the maximum value is defined as the scale scaling factor λ, and then the minimum values ​​of rows and columns are subtracted from all coordinate information in the block image set to achieve overall translation, and all position information is normalized using the scale scaling factor λ, and the formula is: λ←max(r max -r min ,c max -c min ) Among them, r′ i and c′ i Respectively represent the normalized row and column information of the position information of the ith block image. All the block images The normalized position information is expressed as O = {O1, O2, ..., O n }={(r′1,c′1),(r′2,c′2),…,(r′ n ,c′ n )}.

4. The method for classifying whole-slice images of histopathology based on positional encoder converter according to claim 3, characterized in that: The pre-built position-encoding-guided transformer multi-instance learning network model includes: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein, The position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function; the position encoding module is used to receive and process the normalized position information O from S302 to obtain the spatial embedding features O′ of all the block images; The position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function; the position decoding module is used to receive the fusion feature F after random masking according to the set mask ratio mask , and process it to get the predicted coordinates O pred ={(r″1,c″1),(r″2,c″2),…,(r″ n ,c″ n )}.

5. The method for classifying whole-slice images of histopathology based on positional encoder converter according to claim 4, characterized in that: The S4 includes: S401, the block image feature E obtained in S2 i Fuse with the spatial embedding feature O′ to obtain a fused feature F; S402, inputting the fusion feature F into the multi-head self-attention module of the backbone network to exchange information between the divided image blocks, and then pooling the output features to obtain a global feature h that characterizes the entire digital pathology slice; S403: Input the global feature h into the classifier for processing to obtain the probability prediction P i .

6. The method for classifying whole-slice images of histopathology based on positional encoder converter according to claim 5, characterized in that: The S6 includes: S601, predicting P based on the label Y of the whole pathological section and the probability i Calculate the cross entropy loss L slide : Where C represents the number of categories, is the probability of the jth category in the prediction result of the i-th tissue pathology section, Y i is the one-hot encoded label of the ith full slice; S602: According to the normalized position information O and the predicted coordinates O pred Calculate the position reconstruction loss L based on the mean square error pos : Among them, τ represents a temperature value; S603, jointly optimizing and training the position encoding-guided converter multi-instance learning network model by using the cross entropy loss and the mean square error-based position reconstruction loss.

7. A histopathology whole-slice image classification system based on positional encoding converter, characterized in that: Applied to executing the histopathological whole-slice image classification method according to any one of claims 1 to 6, the system comprises: The data preprocessing module is used to obtain the pathological full-slice dataset with images and their category labels and perform block preprocessing to obtain the block image set X i , in represents the jth block image in the i-th full slice, represents the coordinate information of the jth block image in the i-th full slice, including its real row and column position information in the pathological image, and n represents the number of block images in the i-th full slice; A feature extraction module is used to extract the block image set X using a feature encoder i The block image features in are used for each block image Generate a feature vector Update tile image set Get several block image features A coordinate normalization module is used to normalize the coordinate information of the block image to form a block image with block image features E i and the data set to be processed with normalized coordinate information; The model processing module is used to input the data set to be processed into a pre-built transformer multi-instance learning network model based on position coding guidance for processing, and obtain the fusion features F of the block image and the position coordinates, the global features characterizing the digital pathology full slice, and the probability prediction P i ; A coordinate reconstruction module, used to predict the fusion feature F through a mask and self-attention mechanism to obtain a reconstructed coordinate; Model training module, used to predict P according to the probability i , calculating the cross entropy loss and the position reconstruction loss based on the mean square error using the normalized coordinate information and the reconstructed coordinates, and training the position encoding guided converter multi-instance learning network model based on the position reconstruction loss; The image classification module is used to classify the target digital pathology full slice image by using the trained position encoding-guided transformer multi-instance learning network model to obtain a classification result.

8. The histopathology whole-slice image classification system based on positional encoding converter according to claim 7, characterized in that: The pre-built position-encoding-guided transformer multi-instance learning network model includes: at least one backbone network multi-head self-attention module, at least one position encoding module and at least one position decoding module; wherein, The position encoding module is configured to receive normalized position information provided by the coordinate normalization module; the position encoding module comprises a linear network layer, a multi-head self-attention module and a layer normalization function, which is used to process the normalized position information and output the spatial embedding features of all the block images; The position decoding module is configured to receive masked fused features, which are obtained by randomly masking the features processed by the multi-head self-attention module of the backbone network according to a set mask ratio; the position decoding module includes a multi-head self-attention module, a layer normalization function, a linear network layer and a sigmoid function, which are used to process the masked fused features and output predicted coordinates.

Images