Discrete cosine transform-based pathological full-slide end-to-end learning method, device and equipment

Abstract

The invention discloses a pathological full-slide end-to-end learning method, device and equipment based on discrete cosine transform. The method comprises the following steps: acquiring pathological full-slide training data and carrying out slicing processing; frequency domain processing based on discrete cosine transform is carried out on slices obtained after segmentation; performing dynamic random multi-scale sampling on the compressed data, and constructing a training sample; inputting the sample into an end-to-end network for training; and utilizing the trained model to carry out classification prediction on the pathological all-glass slide. According to the method, discrete cosine transform is innovatively introduced to realize slice data compression, so that the occupation of the slice data on a video memory is effectively reduced, the calculation demand of efficient end-to-end learning on large-scale slice parallel processing is supported, a feature encoder can extract more discriminative features for specific downstream tasks, and the accuracy of the extracted features is improved. The classification performance is remarkably improved in cooperation with an attention-enhancing aggregation network; and meanwhile, the data loading speed can be further increased through the compressed smaller data volume, and the calculation efficiency is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning and biomedical science, and more particularly, to a pathological whole slide end-to-end learning method, device and equipment based on discrete cosine transform. BACKGROUND

[0002] A pathological whole slide (WSI) is a core carrier for realizing the digitization of pathological images in the field of computational pathology, and is also a key bridge connecting traditional optical pathology and modern computer analysis. The technical principle is to scan and collect the whole pathological section by a professional digital section scanner at a very high resolution (usually up to 0.25-0.5 microns / pixel), and then generate a gigapixel-level (GB-level) digital image file covering all tissue regions of the section through image stitching and fusion technology.

[0003] Compared with traditional glass sections, pathological whole slides have three obvious advantages: first, information integrity, pathological whole slides can completely retain all pathological details on the section, including small cell morphology, tissue structure abnormalities, etc.; second, digital virtualization, digital images can be stored on the cloud or local server for a long time, supporting repeated calling, magnification and sharing without loss, solving the problems of easy wear and tear, easy pollution and large storage space of physical glass sections; third, cross-time and space collaboration, pathologists can access pathological whole slides remotely through the network, realize multi-center consultation, teaching training and research data sharing, and break the geographical limitations of traditional pathological diagnosis. Relying on these advantages, pathological whole slides have become an indispensable part of the clinical pathological diagnosis process. However, the interpretation and diagnosis of pathological whole slides require experienced pathologists, which may be difficult to meet the requirements in some areas, and due to the large data volume of pathological whole slides, pathologists need to spend a lot of time to analyze and diagnose a pathological whole slide. In addition, due to human factors, the interpretation of pathological whole slides by pathologists may have some subjectivity.

[0004] Computational pathology is a cross-disciplinary field that combines knowledge from pathology, gigapixel image analysis, and computer science to develop computational methods for automatically analyzing and interpreting pathological whole slides. This field uses advanced algorithms and artificial intelligence techniques to help automate diagnosis and provide expert analysis to assist in the diagnosis of pathological sections, which can greatly reduce labor costs and improve diagnostic efficiency. Due to the large data volume of a single pathological whole slide, traditional deep learning methods cannot directly support the training and learning of the whole pathological whole slide. The current mainstream solution is to divide the complete pathological whole slide image into multiple small sections, and then use a two-stage paradigm of "pre-training encoding network to extract features + multi-instance learning to aggregate section features" to carry out downstream tasks.

[0005] However, although this two-stage paradigm effectively alleviates the computational pressure by relying on a pre-trained encoding network, it also has obvious defects and deficiencies. The pre-trained encoding network causes the extracted slice features to be unable to train according to specific tasks, so it cannot make targeted adjustments, thereby causing a bottleneck to the network performance. Although using joint encoding network and multi-instance learning at the whole slide label level (hereinafter referred to as end-to-end learning) is an intuitive solution to this problem, end-to-end learning faces two major challenges of excessive computational pressure and collaborative optimization between the encoding network and the multi-instance aggregator. Existing end-to-end learning works mostly do not consider the collaborative optimization between the encoding network and the multi-instance aggregator, ignore the structural design of the multi-instance aggregator, and usually rely on complex sampling strategies, without starting from the amount of data carried by the slice as a starting point, and often have the problem of high computational cost. SUMMARY

[0006] In view of the above technical problems, the present application provides a pathological whole slide end-to-end learning method based on discrete cosine transform, which can effectively cope with the two major challenges of excessive computational pressure and collaborative optimization between the encoding network and the multi-instance aggregator raised by end-to-end learning, thereby supporting efficient development of end-to-end learning, breaking through the performance bottleneck brought by the traditional two-stage paradigm, and achieving better classification effect under the premise of ensuring computational efficiency.

[0007] In order to achieve the above technical effects, the present application provides a pathological whole slide end-to-end learning method based on discrete cosine transform in the first aspect, comprising: Obtain an original pathological whole slide image from a training set, and cut it into non-overlapping slices after background filtering at different magnifications; Perform frequency domain processing based on discrete cosine transform on the slices obtained after cutting to realize data compression and obtain compressed frequency domain data of each slice; For the compressed slice frequency domain data set of each pathological whole slide in the training set obtained by frequency domain processing, respectively perform dynamic random multi-scale sampling, and take the sampled instances as training samples; Construct an end-to-end network framework, including a frequency domain feature encoder, an enhanced attention multi-instance aggregator, and a classifier, wherein the frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature encoding, the enhanced attention multi-instance aggregator is used to receive the feature encoding output by the frequency domain feature encoder and generate a whole slide level feature representation, and the classifier is used to obtain the classification result of the pathological whole slide according to the whole slide level feature representation; Use the constructed end-to-end network framework to learn the training samples; The pathological whole slide to be diagnosed is pretreated to generate frequency domain slice features matching the model input, and is input to the trained end-to-end network for classification.

[0008] In an embodiment, the slices obtained after segmentation are subjected to frequency domain processing based on discrete cosine transform to realize data compression, and compressed frequency domain data of each slice is obtained, including: The slices obtained after segmentation are subjected to frequency domain transformation and channel reconstruction to generate a multi-channel frequency domain representation. Key frequency component channels are selected from the generated multi-channel frequency domain representation.

[0009] In an embodiment, the slices obtained after segmentation are subjected to frequency domain transformation and channel reconstruction to generate a multi-channel frequency domain representation, including: The slices are converted from the spatial domain to the frequency domain through two-dimensional discrete cosine transform to obtain a frequency domain representation containing different frequency components. The frequency domain representation containing different frequency components is subjected to channel reconstruction to separate and map different frequency components into independent channels to generate a multi-channel frequency domain representation.

[0010] In an embodiment, key frequency component channels are selected from the generated multi-channel frequency domain representation, including: Based on statistical analysis of the training data set, key frequency component channels are preselected and retained; Or through a learnable dynamic gating module, key frequency component channels are adaptively selected for specific tasks.

[0011] In an embodiment, the compressed slice frequency domain data sets of each pathological whole slide in the training set obtained through frequency domain processing are respectively subjected to dynamic random multi-scale sampling, and the sampled instances are used as training samples, including: Under multiple preset magnification scales, a specified number of compressed slice frequency domain data representative of the whole pathological slide are dynamically and randomly sampled from each pathological whole slide in the training set as training samples.

[0012] In an embodiment, the frequency domain feature encoder adopts an improved ResNet network, including a convolution module, four residual network layers, one average pooling, and one tensor flattening operation, wherein the residual network layers are stacked by a plurality of residual blocks, the first residual block controls the change of the number of channels, and the remaining residual blocks keep the number of input and output channels consistent.

[0013] In an embodiment, the processing process of the enhanced attention multi-instance aggregator includes: Add the frequency domain feature code and the PPEG position code to obtain a feature code carrying position information, wherein the PPEG position code is obtained by a PPEG position code module, and the PPEG position code module captures position information at different levels through a multi-scale convolution structure to provide context guidance for the encoded features; The feature code carrying position information is divided into multiple attention heads, and a group of sparse attention scores are calculated as first attention to guide the aggregator to focus on key slices; The feature code carrying position information is interacted by using the first attention, and another group of sparse attention scores are calculated based on the interacted feature code as second attention, and the second attention is optimized through the interaction between the slices; The two groups of sparse attention scores are added by residual and then processed by SoftMax, and then the features are aggregated to output full slide level feature representation.

[0014] Based on the same inventive concept, the second aspect of the present application provides a pathological whole slide end-to-end learning device based on discrete cosine transform, comprising: A background filtering and slicing module is used to obtain an original pathological whole slide image from a training set, and the original pathological whole slide image is filtered and sliced into non-overlapping slices under different magnifications; A frequency domain processing module based on discrete cosine transform is used to perform frequency domain processing based on discrete cosine transform on the sliced slices to realize data compression and obtain compressed frequency domain data of each slice; A dynamic random multi-scale sampling module is used to perform dynamic random multi-scale sampling on the compressed slice frequency domain data set of each pathological whole slide in the training set obtained by frequency domain processing, and the sampled instances are used as training samples; A model construction module is used to construct an end-to-end network framework, including a frequency domain feature encoder, an enhanced attention multi-instance aggregator, and a classifier, wherein the frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature codes, the enhanced attention multi-instance aggregator is used to receive the feature codes output by the frequency domain feature encoder and generate full slide level feature representation, and the classifier is used to obtain the classification result of the pathological whole slide according to the full slide level feature representation; An end-to-end learning module is used to learn the training samples using the constructed end-to-end network framework; A classification module is used to preprocess the pathological whole slide to be diagnosed to generate frequency domain slice features matching the model input, and input the frequency domain slice features to the trained end-to-end network for classification.

[0015] Based on the same inventive concept, the third aspect of the present application provides a non-transitory computer readable storage medium, which stores a computer program, the program being executed by a processor to implement the pathological whole slide end-to-end learning method based on discrete cosine transform of the first aspect.

[0016] Based on the same inventive concept, the fourth aspect of the present application provides an electronic device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the pathological whole slide end-to-end learning method based on discrete cosine transform of the first aspect.

[0017] In the prior art, discrete cosine transform is applied to natural image classification tasks. Compared with natural image classification tasks, the present application applies discrete cosine transform to pathological whole slide classification tasks, and the main differences are as follows: The first is the characteristics of pathological whole slide large-size images. The application of frequency domain transform in pathological whole slides often means that the core training mode is changed from a two-stage paradigm to end-to-end learning, which needs to face two technical difficulties: limited computing resources (overcome by the present application through the design of data compression and sampling strategy based on discrete cosine transform) and the challenge of collaborative optimization of the encoding network and the multi-instance aggregator (overcome by the present application through the design of attention-enhanced multi-instance aggregator); the previous researchers usually start from the mainstream two-stage paradigm, and under this premise, their positioning of frequency domain transform is usually limited to the level of "using frequency domain information to assist spatial domain information to help classification", which makes the role of frequency domain transform limited and the mode transition cost high in the eyes of most researchers; the research mode of the present application is end-to-end learning from the beginning, in order to play the unique advantages of end-to-end learning over the two-stage paradigm, the positioning of frequency domain transform is also changed from "providing auxiliary information" to "realizing data compression", and this change in positioning makes the role of frequency domain transform change from "bringing technical difficulties" to "solving technical difficulties"; this is the reason why frequency domain transform is rarely used in previous pathological whole slide classification, the core being that the starting point cannot be separated from the two-stage paradigm.

[0018] Second, the difference between the image properties of pathological images and natural images, pathological images are different from natural images, and natural image classification relies more on macro information, while for pathological images, in addition to macro information, more detailed information such as tissue structure and cell nucleus morphology is also important; This makes it impossible to treat pathological images as natural images, so when applying discrete cosine transform to pathological whole slides, frequency domain processing needs to be designed for the characteristics of pathological images, for example, discrete cosine transform can be performed on small blocks, which makes some image components that are detailed in the macro level also represented by most low-frequency components in small blocks, and performing discrete cosine transform on small blocks can also bring lower computational complexity, which meets the need to reduce computational pressure.

[0019] Compared with the prior art, the advantages and beneficial technical effects of the present application are as follows: (1) Significantly reduce the computational pressure faced by end-to-end training: the present application innovatively introduces discrete cosine transform into the pathological whole slide classification task, and designs a series of frequency domain processing methods in combination with the characteristics of pathological whole slides to realize data compression, effectively reducing the memory occupation of slice data, thereby supporting efficient end-to-end learning on large-scale slices on the memory resource, overcoming the problem of insufficient computing resources and excessive computing data faced by end-to-end learning; On the other hand, the smaller data amount after compression can further speed up the data loading speed and ensure the computing efficiency.

[0020] (2) Overcome the collaborative optimization problem raised by end-to-end learning: the present application builds an enhanced attention multi-instance aggregator, so that the network can not only focus on key instances and reduce the interference of redundant instances, but also use global information to optimize the focus object, prevent false focus on redundant instances, and guide attention to better focus on key slices. Good attention mode can guide the encoding network to learn more discriminative features, and better features can in turn help optimize attention, forming a positive feedback, thereby overcoming the collaborative optimization problem raised by end-to-end learning.

[0021] (3) Better classification effect: the present application realizes efficient end-to-end learning, which overcomes the limitation of the traditional two-stage paradigm that the encoder cannot be trained for specific downstream tasks, breaks through the performance bottleneck brought by the traditional two-stage paradigm, significantly improves the classification effect while ensuring the computing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0023] Figure 1 Flow chart of the pathological whole slide end-to-end learning method based on discrete cosine transform in the embodiment of the present application; Figure 2 Example diagram of performing discrete cosine transform on 8x8 sub-blocks in the embodiment of the present application; Figure 3 Flow chart of data frequency domain processing in the embodiment of the present application; Figure 4 Channel screening heat map statistically obtained in the embodiment of the present application; Figure 5 Enhanced attention multi-instance aggregation network structure diagram in the embodiment of the present application; Figure 6 PPEG position encoding module structure diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0024] EMBODIMENT The present application provides a pathological whole slide end-to-end learning method based on discrete cosine transform, the workflow of which is shown in Figure 1 , including the following steps: S1. Obtain original pathological whole slide images from a training set, and cut them into non-overlapping sections after background filtering at different magnifications; The training set is a pathological whole slide dataset with whole slide level labels; in this embodiment, the pathological whole slide dataset uses the Camelyon16 dataset; the sectioning operation is performed at 10x10 and 20x20 magnifications, and the size of each section is 256x256; in the subsequent steps, the section set obtained by cutting a pathological whole slide is regarded as a bag, and a section represents an instance in the bag; an instance bag can be represented as:

[0025] wherein X i represents the instance bag obtained after cutting the i-th pathological whole slide in the dataset (i=1, 2, …, m, where m is the number of pathological whole slides), a i p represents the p-th instance in the instance bag X i , and n represents the number of sections in the instance bag.

[0026] S2. Perform frequency domain processing based on discrete cosine transform on the sections obtained after cutting to realize data compression and obtain compressed frequency domain data of each section.

[0027] Specifically, in view of the feature distribution characteristics of pathological images, perform customized frequency domain processing based on discrete cosine transform on the sections. Referring to Figure 3, frequency domain processing based on discrete cosine transform, including two key steps of frequency domain transform and channel reconstruction, and frequency domain component channel screening.

[0028] Among them, the frequency domain transform and channel reconstruction are: This part first converts all the slices in each instance package from RGB format to YCrCb format, where Y channel represents brightness (light and dark), Cr and Cb channels represent chrominance (red and blue differences of color) respectively; This conversion effectively separates the brightness information and chrominance information of the image data, avoiding the interference caused by the high coupling of brightness and color features in RGB space, which is beneficial to extract frequency domain features, and can more targetedly screen important information in subsequent data compression, resulting in higher compression rate; In pathological images, brightness information (Y channel) usually contains more structural and detailed features, and more frequency domain components need to be retained, while chrominance information (Cr and Cb channels) usually changes smoothly, and only a few frequency domain components are usually needed to represent. In contrast, if discrete cosine transform is based on RGB format, brightness information is distributed in three channels, and more frequency domain components need to be retained in three channels, resulting in lower compression rate; Then, discrete cosine transform is carried out on Y, Cr and Cb channels respectively in 8x8 small blocks, and the formula of two-dimensional discrete cosine transform is:

[0029] Among them, α is the normalization coefficient, M and N represent the length and width of the input two-dimensional signal, f(x, y) represents the pixel value at the spatial domain coordinate (x, y), and F(u, v) represents the component coefficient at the frequency domain coordinate (u, v); input any two-dimensional signal with length and width of M and N, MxN frequency domain base component coefficients can be obtained; due to the fixed characteristics of discrete cosine transform frequency domain components, no matter how the input two-dimensional signal changes, as long as the length and width are M and N, MxN frequency domain base components are always fixed and unchanged, and only the coefficients of the base components change; As shown in Figure 2 In this embodiment, discrete cosine transform is carried out on 8x8 small blocks, so each small block will output 64 fixed frequency domain base component coefficients with the shape of 8x8, and each base component represents a different frequency wave; The advantage of performing discrete cosine transform on 8x8 small blocks is that: unlike natural images, in natural image classification tasks, high-frequency information usually represents noise, redundancy, and classification mainly relies on low-frequency information in the image; while in pathological images, high-frequency information and low-frequency information are equally important, such as tissue texture, cell nucleus structure, and other medium-high frequency information, which are often the key features for identifying lesions; if a two-dimensional discrete cosine transform is directly performed on a 256x256 size slice, both high and low frequency information contain key features, which is not convenient for data compression; while performing discrete cosine transform on 8x8 small blocks, many high-frequency and detailed information on the 256x256 macro level will appear low-frequency and smooth in the 8x8 small blocks, which means that discrete cosine transform can use fewer low-frequency components to represent high-frequency information in the slice in the 8x8 small block, thereby achieving higher compression ratio. On the other hand, since the computational complexity of two-dimensional discrete cosine transform is O(N^2), performing discrete cosine transform on 8x8 small blocks is beneficial to reduce the computational complexity; As shown in Figure 3 After obtaining the frequency domain representation of each slice, in order to facilitate subsequent frequency component screening and data compression, the obtained frequency domain representation is channel reconstructed to separate different frequency components; Since each 8x8 small block performs discrete cosine transform, the 64 frequency domain basis components used to represent the frequency domain data are fixed, so the 64 different frequency domain basis components are mapped to different channels and spliced again in 8x8 small blocks to obtain a multi-channel frequency domain representation; After channel reconstruction, each frequency domain channel only contains the coefficients of one frequency domain basis component, and subsequent frequency component screening can be directly realized by screening the frequency domain channel. The slice instances in all instance packages are subjected to frequency domain transformation and channel reconstruction processing:

[0030] Among them, DCT 2D represents the two-dimensional discrete cosine transform performed on 8x8 small blocks, Reshape represents the channel reconstruction operation, b i p represents the i-th instance package p-th slice instance a i p The new instance representation obtained after frequency domain transformation and channel reconstruction; accordingly, all instance packages in the data set are updated, and defined:

[0031] That is, all slices in any instance package X i are replaced by the corresponding frequency domain instance; The shape of the YCrCb corresponding to the frequency domain data before channel reconstruction is 256x256x3. After channel reconstruction, because 64 base components on the 8x8 small block are divided into different channels, the length and width of the data become 256 / 8=32, and the number of channels becomes 3x64=192. That is, the shape of any frequency domain instance b i p

[0032] Frequency domain component screening: as shown in the last step of operation in Figure 3 According to the importance of each frequency domain channel, the frequency domain component screening is carried out to realize data compression; wherein, the frequency domain component channel screening step includes static screening as the core strategy and dynamic screening as the auxiliary strategy; Static screening: based on the statistical information of the difference significance between the mean value of each frequency domain component, the normal slice and the abnormal slice, the channel screening is carried out; Firstly, the difference significance of each component between the normal slice and the cancerous slice is analyzed by t-test, the significant level is 0.05, and the significant frequency domain components are reserved; For the part of components that do not pass the t-test, the overall mean value of each component is further calculated, and the part of components with the largest mean value is reserved until the number of frequency domain components reserved reaches the expected number; In this embodiment, the number of frequency domain components expected to be reserved is 32; the static screening can select and reserve the key frequency domain component channels based on the statistical analysis of the training data set, so as to realize data compression before model training; Dynamic screening: including a dynamic gating structure, which is used to adaptively screen the key component channels according to the importance of different frequency domain channels; The dynamic gating structure is modified from the SE module (a classic channel attention module) and is composed of a global average pooling layer, two-level fully connected layers and a Sigmoid activation function; Different from the traditional channel attention mechanism outputting continuous weighted scores, the output of this module in the present application is a binary value describing the channel state, realizing the "on-off control" rather than the "weight adjustment" at the channel level; In the training stage, the binary gating score corresponding to each channel is obtained by back propagation optimization, and after Gumbel-Softmax normalization processing, the on-off state of the channel is determined by threshold binaryzation. After the shape remodeling of S3, different frequency components are mapped into different frequency domain channels, so that one frequency domain channel represents one frequency component. If the channel is turned on, it means that the component data of the corresponding channel continues to participate in the classification task, and if the channel is turned off, it means that the component data ​No longer participate in classification; the network will adaptively learn to retain important channels and close useless channels. This process can retain useful components and filter out redundant components.

[0033] For any instance in the instance package Where b i pj Represents instance b i p Data of the jth channel, c represents the total number of channels, c = 192 in this embodiment, and the retained channel set S is:

[0034] Where represents the element product, G represents the dynamic gate module, and if the binary output is 1, the channel j is retained. This module can adaptively filter important frequency domain components according to the specific task characteristics and suppress redundant channel interference. After the model training is completed, the channel gating scores output by the dynamic gating module can be statistically analyzed to calculate the average activation rate of each frequency domain channel in the training process. According to the statistical results, high contribution channels can be selected to provide a reference for static component filtering.

[0035] Document 2 CN115239622B discloses a capsule endoscope image lesion recognition method based on frequency domain combination transformation attention, which involves discrete cosine transformation. The differences between the method of document 2 and the present invention are as follows: Firstly, the purpose of the model, document 2 mainly provides auxiliary function for spatial domain data, and provides more information from the frequency domain. The core purpose of introducing discrete cosine transformation in the present invention is to realize data compression to support the development of end-to-end learning. Secondly, in terms of technical details, document 2 carries out discrete cosine transformation on RGB format, while the present invention converts the image from RGB format to YCRCB format, decoupling the brightness information and color difference information. Although document 2 selects the relatively potential G channel after statistical analysis, in the RGB format, the correlation between the three channels is strong, and the data distribution is often relatively chaotic. In the YCRCB format, the brightness information and color difference information are independent of each other, which makes the data distribution more regular, and when processing different channels, the algorithm can more flexibly optimize the compression effect of each channel. In addition, document 2 adopts the method of directly processing the whole image, inputting an image of 256*256 size, and outputting 256*256 frequency domain components. The present invention adopts the method of carrying out discrete cosine transformation on 8*8 small blocks of 256*256 size slices, which is more suitable for pathological images, realizes higher compression rate and brings lower computational complexity.

[0036] Benefiting from the advantage of high energy concentration of discrete cosine transform (being able to concentrate information in a small number of frequency domain components), and the YCrCb format conversion and 8x8 small block processing carried out by the present application when obtaining the frequency domain representation, channel screening can achieve a relatively high compression rate, such as Figure 4 As shown in the channel screening situation based on dynamic screening module statistics, it can be seen that the frequency domain components that mainly play a role are concentrated in the frequency domain data generated by the Y channel, and most of the frequency domain components are almost rarely reserved, which means that even if the first 32 components in the Y channel frequency domain data are selected as the static component screening, most of the frequency domain channels that play a role can be covered; At the same time, unlike static screening, the execution of dynamic screening is carried out after the complete data is input, and itself does not realize the effect of data compression, so that in most cases dynamic screening does not need to be used. Only in some special tasks, especially in new application scenarios, such as on new downstream tasks, or on new pathological whole slide types, if the researchers believe that due to the change of application scenarios, the distribution of key frequency domain components has changed, and a more fine-grained component screening strategy is pursued, dynamic screening strategy can be considered. In Figure 4 , after accumulating certain statistical information, when similar tasks are executed again in the future, the statistical information can be used as a reference, and the static screening strategy can still be used.

[0037] The frequency domain instances b i p in all frequency domain packages are screened by channel components to obtain compressed instance data c i p :

[0038] Among them, S represents the reserved channel set; At the same time, the instance package is updated again:

[0039] At this point, the frequency domain processing based on discrete cosine transform is completed. In order to facilitate understanding of the data compression performance that this part of work can bring, the relevant data overhead is shown in Table 1: Table 1

[0040] As can be seen from Table 1, after the frequency domain processing based on the discrete cosine transform is designed, the compressed data only accounts for about 1 / 6 of the data amount of the original slice; the significant reduction in data amount improves the maximum number of parallel instances that the network can withstand in the end-to-end learning process. Under the limitation of 24G of display memory, in the case of ResNet18 as the encoder, the original slice number that the display memory allows to process in parallel is between 900 and 1000, while the compressed frequency domain data allows to process 4000 to 4100 instances in parallel; and if ResNet50 is used as the encoder, the number of original slices that the end-to-end network allows to process in parallel is less than 100, which almost completely limits the development of end-to-end learning, while the compressed frequency domain data allows to process 1100 to 1200 instances in parallel, which still can sample sufficient instance data to support the development of end-to-end learning; on the other hand, since the I / O operation of data is frequently performed in the process of end-to-end learning, the loading speed of data also becomes a major time cost in the process of end-to-end learning, and the compression of data directly affects the loading speed of data. Under 16 CPU cores, loading 512 original slice instances takes 0.8s, while loading the compressed instance data only takes 0.3s, which improves the loading speed by more than half.

[0041] S3, for the compressed slice frequency domain data set of each pathological whole slide in the training set obtained by the frequency domain processing, dynamic random multi-scale sampling is performed respectively, and the sampled instances are used as training samples.

[0042] Among them, the compressed slice frequency domain data set of each pathological whole slide in the training set is obtained by the frequency domain processing based on the discrete cosine transform in step S2, that is, the instance bag, and dynamic random multi-scale sampling is performed on these instance bags respectively. A specified number of frequency domain data instances are sampled from the instance bag of each pathological whole slide, and the sampled instances represent the pathological whole slide and are used as training samples. As Figure 1 shown, dynamic random multi-scale sampling: Dynamic means that the sampling process of the instance needs to be re-executed in each Epoch of the training process, which can effectively avoid the occurrence of overfitting while improving the data coverage; Random means that a simple and effective random sampling strategy is adopted in the sampling process of the instance, which avoids introducing additional calculation cost by an overly troublesome sampling strategy; Multi-scale means that the instance data of the pathological whole slide obtained under the magnification of 10x10 and 20x20 is sampled respectively, so as to introduce image features under different scales and help improve the classification effect; This process is expressed as:

[0043] X i e represents the i-th instance bag after instance sampling in the e-th epoch, after dynamic random multi-scale sampling is carried out on all instance bags, the sampled instance bag is converted into tensor data and normalized to be used as a training sample; N represents that the total number of instances contained in the instance bag to be sampled is N, represents that k instances are randomly sampled from N instances, c i p represents the sampled instance in X i In this embodiment, k is 512.

[0044] S4, an end-to-end network framework is constructed for carrying out a classification task of a pathological whole slide; wherein the end-to-end network framework comprises a frequency domain feature encoder, an enhanced attention multi-instance aggregator and a classifier; The frequency domain feature encoder does not have a hard structure limit, in this method, it is configured as a ResNet network, and because the characteristics of the frequency domain data are different from those of the spatial domain data and the shape of the frequency domain data is adjusted in the method, it is necessary to adaptively adjust the characteristics of the frequency domain data; under the premise of meeting this requirement, the frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature encoding.

[0045] In this embodiment, the frequency domain feature encoder is modified by taking ResNet18 network as an example, and specific adjustments are made according to the characteristics of the frequency domain data, which specifically include: The convolutional layer and the pooling layer before the residual block in ResNet18 are deleted: because the data processed by the frequency domain is smaller in the length and width dimensions than the feature map of the original slice, and is larger in the channel dimension, and the input frequency domain data is compressed, the information is more condensed; therefore, the standard ResNet18 front end used for performing a large amount of spatial downsampling (few -> many channels, large -> small size) convolution and pooling layer is not suitable; retaining them is not only functionally redundant, but the strong downsampling operation will also destroy the frequency domain structure, resulting in loss of key information; A 1x1 convolutional layer is added: used to extract features between different frequency domain component channels; The basic structure of the residual block is retained; The input and output channel numbers of each residual block are modified: adjusted according to the characteristics of the input frequency domain data, one is to match the channel number of the input data; two is to appropriately reduce the channel number according to the actual needs of the classification task, which is conducive to saving computing resources; The last fully connected layer is removed: because the encoder is only used to extract feature encoding and does not directly carry out a classification task, the last fully connected layer is removed; The modified frequency domain encoder includes a convolutional module, four residual network layers, one average pooling operation, and one tensor flattening operation. The convolution module includes a 1×1 convolution that does not change the number of channels, a BatchNorm2d operation, and a ReLU activation operation. The residual network layer is composed of two stacked residual blocks. The basic structure of the residual blocks remains unchanged. The first residual block controls the change in the number of channels, while the second residual block keeps the number of input and output channels consistent. Compared to the original ResNet18 network, the modified frequency domain encoder adjusts the number of input and output channels of each residual network layer as follows: the input and output of the four residual network layer modules are 32→32, 32→64, 64→128, and 128→256, respectively. The stride of the first two residual connection layers is 1, and the stride of the last two residual connection layers is 2. The frequency domain feature encoder receives the training samples obtained after S3 processing and extracts the feature codes of all instance packets in the training samples:

[0046]

[0047] in, This represents a frequency domain feature encoder obtained by modifying the ResNet18 network. This represents instance c in the i-th instance package output in S3. i p Feature instances obtained after encoder processing; Z ie X represents the frequency domain data instance of the i-th compressed slice in the training samples of the e-th epoch. i e The encoded feature instance package obtained after inputting into the encoder; An enhanced attention multi-instance aggregator is used to receive feature codes from the frequency domain feature encoder and generate full-slide-level feature representations, such as... Figure 5 As shown, it specifically includes: Encode the frequency domain features Z ie With PPEG position encoding Z ie PPEG Adding them together yields the encoded feature Z that incorporates location information. ie Pos ; Z ie Pos Divided into multiple attention heads:

[0048] in, Representative coding feature Z iePos The h-th attention head obtained through multi-head partitioning, in this embodiment, has a total number of attention heads of 8; k represents the number of encoded features, which has the same meaning as the number of previously sampled instances, and has a value of 512; For each attention head, calculate a set of sparse attention scores, which are used as the first attention.

[0049] in, Represents sparse attention computation. The sparse attention score calculated by the h-th attention head in the encoding features is represented as a vector of length k. Sparse attention is obtained by training two linear mapping weight matrices. Figure 5 In the first sparse attention computation, the two linear mapping weight matrices corresponding to the computation are W. A and W B W A and W B This can be viewed as two attention branches, and both are used to calculate a set of attention scores; W A W is obtained by performing a linear transformation and a Tanh activation on the attention head. B It is obtained by performing a linear transformation and a sigmoid activation on the attention head; by applying W... A and W B Perform dot product operations to calculate the sparse attention score; An enhanced attention mechanism based on self-attention is implemented by performing a self-attention calculation for each attention head:

[0050] Z ie SelfAtth Represents attention The feature encoding obtained after performing attention slice interaction and feature aggregation; Q and K represent the query matrix and key-value matrix used for self-attention computation; Z is a feature encoding that applies self-attention, overslicing interaction, and feature aggregation. ie SelfAtth Calculate another set of sparse attention scores as the second attention:

[0051] The method for calculating sparse attention scores here is the same as... The calculation method is consistent, and the two attention branches correspond to Figure 5 W in C and W D ; By performing a residual connection on the two sets of sparse attention scores, we obtain an enhanced sparse attention. :

[0052] Utilizing Enhanced Sparse Attention Attention Perform feature aggregation:

[0053] The obtained Z h S Let Z represent the full-slide-level feature representation obtained by aggregating the h-th attention head. By splicing the feature representations obtained from aggregating each attention head, the complete full-slide-level feature representation Z is obtained. S :

[0054] Here, Concat represents the concatenation operation; The sparse attention pattern is beneficial for multi-instance aggregators to focus attention on a few slices. In the diagnosis of whole slides in pathology, there are often situations where the lesion area only occupies a very small part of the whole slide. When aggregating features, a large number of non-critical slices may interfere with classification. In this case, the characteristic of sparse attention focusing on a few slices can effectively prevent critical areas from being overwhelmed by non-critical instances. However, sparse attention may mistakenly focus on redundant instances. To address this, the present invention introduces a multi-head attention mechanism and self-attention-based enhanced attention in the enhanced attention multi-instance aggregator. Multi-head attention settings allow the aggregator to learn multiple attention patterns. Individual votes from multiple heads on the focus area are more stable than a single attention pattern, reducing the occurrence of misfocusing. Enhanced attention establishes global dependencies between instances through a self-attention mechanism, enabling the model to capture the structural relationships between different instances in pathological tissues. This global information interaction compensates for the limitation of sparse attention, which can only focus on local areas, making attention allocation more consistent with the semantic consistency of the overall pathological structure. The sparse attention score calculated after feature interaction is then applied to the model. With local sparse attention score Performing residual connections can correct sparse attention at the global information level; The structure of PPEG positional encoding is as follows: Figure 6 As shown; First, the coordinate information (x, y) of each instance is obtained from the pathological whole slide and standardized. Then, the coordinate information is processed using sine and cosine encoding. The formula for one-dimensional sine and cosine encoding is as follows:

[0055] in, Represents sine coding. represents cosine coding, pos represents position information, r is used for the index position in the position coding vector, and d is the length of the generated code, which is 128 in this embodiment. Since the image is two-dimensional, the rows and columns are encoded separately before being concatenated:

[0056] The resulting sine and cosine codes PE(x,y) have a length of 256, where sin(x) represents the sine code for coordinate x. The cosine code representing the x-coordinate. The sine code representing the coordinate y. Cosine encoding of coordinate y; The core of PPEG positional encoding is to capture positional information at different levels by using convolutional operations at different scales. To facilitate convolution, the positions of each instance are encoded and arranged in a grid. Specifically, the number of slices is squared and rounded up to obtain a suitable grid size. To accommodate the largest convolution kernel, the size is not less than 7. In this embodiment, N=512, and its square root √512≈22.627, which is rounded up to 23. Therefore, the width and height of the grid are set to 23. Then, using the original spatial distribution of coordinates as a reference, the sine and cosine codes of each instance are arranged in the grid, with each grid containing the sine and cosine codes of one instance. Considering that the slices cannot completely fill the grid, the empty grids are first filled by copying and filling. The filling method is to copy the arranged slice codes from the upper left corner of the grid to the lower right corner of the grid, so that the distribution of coordinate information is more even. If the number of arranged slice codes is less than half the number of grids, the empty grids cannot be completely filled by copying and filling only once. The remaining empty grids are filled by filling blanks and filling with 0. The grid position code shape after filling is 256×23×23. After this operation, convolutional processing is performed on the grid position code using 3×3, 5×5, and 7×7 kernels in sequence. The results of the three convolutions are then summed to output the final PPEG position code Z. PPEG .

[0057] Regarding enhanced attention multi-instance aggregators, reference 1 (CN118196516A) discloses a multi-classification method, system, and device for cervical tissue pathology whole-slide images. Both share the goal of focusing the attention of the aggregation network on key slices. The main differences lie in their model concepts and implementation methods, as detailed below: The core guiding principle and implementation method of the aggregation model in Reference 1 is "training a mask to extract key slices + refining key features through a self-attention mechanism," while the core guiding principle and implementation method of the enhanced attention multi-instance aggregation network of this invention is "sparse attention focusing on key slices + self-attention correcting the focusing situation of sparse attention." Furthermore, the role of self-attention differs in the two inventions. In Reference 1, the self-attention mechanism is mainly used to refine information, focusing on aggregation features; while the self-attention mechanism of this invention is mainly for information interaction, focusing on adjusting sparse attention.

[0058] Reference 1, mentioned in the background section of this invention, belongs to the mainstream two-stage paradigm in current pathological whole slide classification tasks. Therefore, its focus is on optimizing the multi-instance aggregator itself. While the multi-instance aggregator of this invention also achieves a similar effect, its focus is on driving the encoder's learning through optimizing the multi-instance aggregator, realizing collaborative optimization, and ultimately serving the end-to-end learning itself.

[0059] The classifier consists of a simple linear layer that encodes full-slide-level features into diagnostic categories.

[0060] S5. Use the constructed end-to-end network framework to learn from the training samples; In S2~S4, this invention overcomes the two major technical challenges of excessive computational pressure and collaborative optimization of end-to-end networks by relying on frequency domain processing based on discrete cosine transform and enhanced attention multi-instance aggregator; based on this, end-to-end learning is realized in this step. The training process is conducted under weak supervision, guided by full-slide labels and cross-entropy loss function, with the AdamW optimizer as the optimizer and a learning rate of 8×10⁻⁶. -5 There is no weight decay; in this embodiment, an epoch of 70 is preferred. Leveraging the inherent advantages of end-to-end learning, the network can train the encoder to extract more discriminative coding features based on specific downstream labels, thus overcoming the performance bottleneck faced by the two-stage paradigm and achieving better classification performance.

[0061] S6. Preprocess the whole glass slide of the pathology to be diagnosed to generate frequency domain slice features that match the model input, and input them into the trained end-to-end network for classification. The above steps (S1~S6) are used to classify pathological whole slide images.

[0062] Among them, preprocessing the whole slide of pathology to be diagnosed refers to performing the same preprocessing process S1~S3 as the training phase on the whole slide of pathology to be diagnosed (including background filtering, slicing, frequency domain processing based on discrete cosine transform, and dynamic random multi-scale sampling).

[0063] As described above, the end-to-end learning method for pathological whole-slides provided in this disclosure overcomes the problem of excessive computational pressure faced by end-to-end learning by achieving effective data compression through a series of frequency domain processing based on discrete cosine transform. Furthermore, it overcomes the challenge of collaborative optimization proposed by end-to-end learning by constructing an enhanced attention multi-instance aggregator. Based on this, it can effectively leverage the advantages of end-to-end learning in training the encoder while ensuring high efficiency, extracting higher-quality coding features and achieving better classification results compared to the two-stage paradigm.

[0064] To demonstrate the technical effects of this invention, Table 2 provides an experimental comparison between this invention and existing technologies on the Camelyon16 dataset: Table 2

[0065] The experimental results above demonstrate that this invention exhibits superior performance in both the AUC (Average Comparison Value) metric, reflecting classification performance, and the average inference speed per pathological whole slide. Specifically, using a modified ResNet18 encoder for end-to-end learning, the invention achieves an AUC of 0.933; using a modified ResNet50 encoder, the AUC reaches 0.958, significantly higher than most two-stage baseline methods. Although the AUC is slightly lower than the classification performance achieved using the UNI large model as the encoder, this invention is significantly faster in inference than the UNI+TransMIL method, and even faster than the most lightweight ResNet18+AB-MIL structure among several traditional two-stage baseline methods. By comparing with other methods, this invention achieves optimal results in both core metrics (AUC and inference speed), meaning that this method can achieve superior classification performance in a shorter time. Therefore, the end-to-end learning method based on discrete cosine transform demonstrates its application potential in pathological whole slide classification tasks.

[0066] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. The system, device, computer-readable storage medium storing the corresponding computer program of the technical solution of this invention, and computer equipment including the computer program running the corresponding computer program should also be within the protection scope of this invention.

[0067] The following describes the electronic device provided in this disclosure for end-to-end learning of pathological whole slides based on discrete cosine transform. This disclosure provides an end-to-end learning device for pathological whole slides based on discrete cosine transform, comprising: The background filtering and slicing module is used to obtain original pathological whole slide images from the training set, filter the background at different magnifications, and then slice them into non-overlapping slices. The frequency domain processing module based on discrete cosine transform is used to perform frequency domain processing based on discrete cosine transform on the slices obtained after segmentation, so as to achieve data compression and obtain compressed frequency domain data of each slice. The dynamic random multi-scale sampling module is used to perform dynamic random multi-scale sampling on the frequency domain data set of compressed slices of each pathological whole glass slide in the training set obtained by frequency domain processing, and use the sampled instances as training samples. The model building module is used to build an end-to-end network framework, including a frequency domain feature encoder, an enhanced attention multi-instance aggregator, and a classifier. The frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature codes. The enhanced attention multi-instance aggregator is used to receive the feature codes output by the frequency domain feature encoder and generate a full slide-level feature representation. The classifier is used to obtain the classification result of the pathological whole slide based on the full slide-level feature representation. The end-to-end learning module is used to learn from the training samples using the constructed end-to-end network framework; The classification module is used to preprocess the whole slides of pathology to be diagnosed, generate frequency domain slice features that match the model input, and input them into the trained end-to-end network for classification.

[0068] The following describes the electronic device provided in this disclosure for end-to-end learning of pathological whole slides based on discrete cosine transform. The electronic device described below can be referred to in correspondence with the method for end-to-end learning of pathological whole slides based on discrete cosine transform described above.

[0069] The electronic device may include a processor, a communications interface, memory, and a communication bus. The processor, communications interface, and memory communicate with each other via the communication bus. The processor can call logical instructions from the memory to execute an end-to-end learning method for pathological whole-slide slides based on discrete cosine transform, mainly including the software processing portion described above.

[0070] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the software processing portion of the pathological whole slide end-to-end learning method based on discrete cosine transform provided by the above methods.

[0072] In another aspect, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the software processing part of the pathological whole slide end-to-end learning method based on discrete cosine transform provided by the above methods.

[0073] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end learning method for pathological whole slides based on discrete cosine transform, characterized in that, include: Original pathological whole slide images were obtained from the training set, and after background filtering at different magnifications, they were cut into non-overlapping slices. The slices obtained after segmentation are processed in the frequency domain based on discrete cosine transform to achieve data compression and obtain compressed frequency domain data of each slice. For the frequency domain data set of compressed slices of each pathological whole glass slide in the training set obtained by frequency domain processing, dynamic random multi-scale sampling is performed, and the sampled instances are used as training samples. An end-to-end network framework is constructed, including a frequency domain feature encoder, an enhanced attention multi-instance aggregator, and a classifier. The frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature codes. The enhanced attention multi-instance aggregator is used to receive the feature codes output by the frequency domain feature encoder and generate a full slide-level feature representation. The classifier is used to obtain the classification result of the pathological whole slide based on the full slide-level feature representation. The training samples are learned using the constructed end-to-end network framework; The whole glass slides of the pathology to be diagnosed are preprocessed to generate frequency domain slice features that match the model input, and then input into the trained end-to-end network for classification.

2. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 1, characterized in that, The resulting slices are then subjected to frequency domain processing based on discrete cosine transform, including: The slices obtained after segmentation are subjected to frequency domain transformation and channel reconstruction to generate a multi-channel frequency domain representation. The key frequency domain component channels are selected from the generated multi-channel frequency domain representation.

3. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 2, characterized in that, The slices obtained after segmentation undergo frequency domain transformation and channel reconstruction to generate a multi-channel frequency domain representation, including: The slices are transformed from the spatial domain to the frequency domain by using two-dimensional discrete cosine transform to obtain a frequency domain representation containing different frequency components; Channel reconstruction is performed on the frequency domain representation containing different frequency components, and the different frequency components are separated and mapped to independent channels to generate a multi-channel frequency domain representation.

4. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 2, characterized in that, Key frequency domain component channels are selected from the generated multi-channel frequency domain representation, including: Based on statistical analysis of the training dataset, key frequency domain component channels are pre-selected and retained; Alternatively, a learnable dynamic gating module can be used to adaptively select key frequency domain component channels for specific tasks.

5. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 1, characterized in that, For the frequency domain data set of compressed slices of each pathological whole slide in the training set obtained by frequency domain processing, dynamic random multi-scale sampling was performed on each sample, and the sampled instances were used as training samples, including: At multiple preset magnification scales, a specified number of compressed slice frequency domain data are dynamically and randomly sampled from each pathological whole slide in the training set to represent the whole pathological slide as training samples.

6. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 1, characterized in that, The frequency domain feature encoder uses an improved ResNet network, which includes a convolutional module, four residual network layers, one average pooling operation, and one tensor flattening operation. The residual network layers are composed of several stacked residual blocks. The first residual block is used to control the change in the number of channels, and the remaining residual blocks are used to maintain the consistency of the number of input and output channels.

7. The end-to-end learning method for pathological whole slides based on discrete cosine transform as described in claim 1, characterized in that, The processing steps of the enhanced attention multi-instance aggregator include: The frequency domain feature code is added to the PPEG position code to obtain the feature code carrying position information. The PPEG position code is obtained through the PPEG position code module, which captures position information at different levels through a multi-scale convolutional structure to provide contextual guidance for the encoded features. The feature encoding carrying location information is divided into multiple attention heads, and a set of sparse attention scores is calculated as the first attention to guide the aggregator to focus on the key slice. The first attention is used to perform feature interaction on the feature encoding carrying location information, and another set of sparse attention scores is calculated based on the interactive feature encoding as the second attention. The second attention is optimized by the interaction between slices. The residuals of the two sets of sparse attention scores are summed and then processed by SoftMax. The features are then aggregated to output a full-slide-level feature representation.

8. An end-to-end learning device for pathological whole-slides based on discrete cosine transform, characterized in that, include: The background filtering and slicing module is used to obtain original pathological whole slide images from the training set, filter the background at different magnifications, and then slice them into non-overlapping slices. The frequency domain processing module based on discrete cosine transform is used to perform frequency domain processing on the slices obtained after segmentation based on discrete cosine transform. The dynamic random multi-scale sampling module is used to perform dynamic random multi-scale sampling on the frequency domain data set of compressed slices of each pathological whole slide in the training set obtained by frequency domain processing, and use the sampled instances as training samples. The model building module is used to build an end-to-end network framework, including a frequency domain feature encoder, an enhanced attention multi-instance aggregator, and a classifier. The frequency domain feature encoder is used to convert the sampled compressed slice frequency domain data into feature codes. The enhanced attention multi-instance aggregator is used to receive the feature codes output by the frequency domain feature encoder and generate a full slide-level feature representation. The classifier is used to obtain the classification result of the pathological whole slide based on the full slide-level feature representation. The end-to-end learning module is used to learn from the training samples using the constructed end-to-end network framework; The classification module is used to preprocess the whole slides of pathology to be diagnosed, generate frequency domain slice features that match the model input, and input them into the trained end-to-end network for classification.

9. A non-transitory computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the end-to-end learning method for pathological whole slides based on discrete cosine transform as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the end-to-end learning method for pathological whole slides based on discrete cosine transform as described in any one of claims 1 to 7.

Images