Spatial transcriptional atlas generation method based on multi-level pathological image fusion

Abstract

The present invention discloses a method for generating a spatial transcriptional map based on multi-level pathological image fusion, which is used for generating a spatial transcriptional map by relying on digital pathological images in the absence of a spatial transcriptome analysis device. The present invention belongs to the field of electronic information technology, and the key points of the technical solution include: obtaining images of regions of interest of glass slide specimens at different magnifications; sending the images of regions of interest of each level into their respective convolutional embedding layers to obtain serialized features of each level; performing single-scale intra-level feature extraction on the serialized features of each level to obtain a single-scale element mixed feature sequence; performing cross-scale inter-level feature extraction on the single-scale element mixed feature sequences of all levels to obtain a cross-scale channel mixed feature sequence; sending the end element of the cross-scale channel mixed feature sequence into a linear regressor to predict and generate the spatial transcriptome gene count of the region of interest; repeating the above to obtain a spatial transcriptional map of the entire glass slide specimen.

Description

Technical Field

[0001] The present application relates to the field of electronic information technology, and in particular to a method for generating a spatial transcription map based on multi-level pathological image fusion. Background Art

[0002] Traditional gene sequencing technology based on bulk RNA can often only obtain coarse-grained gene expression information of the entire tissue, while the emergence of spatial transcriptome technology makes it possible to obtain the spatial distribution of gene expression on pathological tissue sections. By combining spatial transcriptome sequencing results with digital pathological images, doctors will be able to locate and analyze lesions more accurately and achieve precision treatment.

[0003] However, the current spatial transcriptome technology requires supporting equipment and the detection cost is very expensive, which makes it difficult to be widely promoted and used. Summary of the invention

[0004] In view of this, the purpose of the embodiment of the present application is to provide a method for generating a spatial transcription map based on multi-level pathology image fusion, by inputting the tissue image obtained by full scanning after magnification of the digital pathology image by a microscope at different magnifications into the network at one time, and continuously fusing the visual information of different scales and resolutions therein during the visual feature extraction process, thereby imitating the process of professional pathologists continuously zooming in and out of the digital pathology image when reading the film to combine multi-scale information for the final diagnosis, thereby achieving higher precision downstream task performance.

[0005] According to a first aspect of an embodiment of the present application, a method for generating a spatial transcription map based on multi-level pathological image fusion is provided, comprising:

[0006] S1: Obtain full scan pathological images of the slide specimen at at least two different magnifications;

[0007] S2: sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level;

[0008] S3: Send the images of each level to their respective convolutional embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence;

[0009] S4: Perform single-scale intra-level feature extraction on the serialized features of each level to obtain a single-scale element mixed feature sequence;

[0010] S5: splicing the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, performing cross-scale inter-level feature extraction on the two-dimensional matrix to obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layering to re-obtain a cross-scale channel mixed feature sequence after being split into several levels;

[0011] S6: Take out and concatenate the elements at the end of the cross-scale channel mixed feature sequence after splitting at several levels to obtain a one-dimensional vector, and send the one-dimensional vector to the linear regressor to obtain the spatial transcriptome gene counts of the region of interest;

[0012] S7: Repeat S2-S6 to obtain the spatial transcriptional map of the entire slide specimen.

[0013] Optionally, the sampling process of the region of interest needs to follow the following principles:

[0014] In full-scan pathology images at different magnifications, in addition to ensuring that the center of the sampling area can be uniformly aligned to the same position of the slide specimen, the sampled image size also needs to be enlarged or reduced by a corresponding factor with the image magnification.

[0015] Optionally, each level image is fed into its own convolutional embedding layer to obtain the serialized features of each level, and a learnable element is added to the end of each serialized feature, including:

[0016] The images of each level are sent to the corresponding convolution embedding layer of each layer to obtain the serialized features of each level, where the convolution kernel size and step size of each layer are enlarged or reduced according to the magnification of each level to ensure that the downsampled images of each level can finally be converted into serialized features with consistent length;

[0017] An additional randomly initialized learnable element is added to the back end of each serialized feature, increasing the sequence length by one.

[0018] Optionally, single-scale intra-level feature extraction is performed on the serialized features of each level to obtain a single-scale element mixed feature sequence, including:

[0019] S41: performing layer normalization processing on the serialized features of each layer;

[0020] S42: processing the normalized serialized features, and converting the serialized features of each level into three different channel compression feature matrices through three learnable parameter matrices;

[0021] S43: performing matrix multiplication on the first channel compression feature matrix and the transpose of the second channel compression feature matrix to obtain an autocorrelation matrix of the elements in the sequence, and then performing matrix multiplication on the autocorrelation matrix and the third channel compression feature matrix to obtain a single-scale element mixed feature sequence after channel compression;

[0022] S44: performing channel restoration on the single-scale element mixed feature sequence through another learnable matrix, making the number of channels completely consistent with the number before normalization, and obtaining a final single-scale element mixed feature sequence.

[0023] Optionally, cross-scale inter-level feature extraction is performed on the two-dimensional matrix to obtain a cross-scale channel mixed feature sequence, including:

[0024] S51: performing layer normalization processing on the concatenated two-dimensional matrix;

[0025] S52: Using the first learnable parameter matrix, the normalized two-dimensional matrix is ​​channel compressed to remove redundant information between layers;

[0026] S53: The number of channels of the two-dimensional matrix after channel compression is restored to be consistent with that before the input module through a second learnable parameter matrix, thereby obtaining a cross-scale channel mixed feature sequence.

[0027] Optionally, the one-dimensional vector is fed into a linear regressor to obtain the spatial transcriptome gene counts of the region of interest, including:

[0028] S61: performing layer normalization processing on the one-dimensional vector;

[0029] S62: A learnable linear regression parameter matrix is ​​used to perform matrix multiplication with the normalized one-dimensional vector to obtain the gene count to be predicted.

[0030] According to a second aspect of an embodiment of the present application, a spatial transcription atlas generating device based on multi-level pathological image fusion is provided, comprising:

[0031] An acquisition module, used for acquiring full scan pathological images of the slide specimen at at least two different magnifications;

[0032] A sampling module, used for sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level;

[0033] The convolution processing module is used to send the images of each level into their respective convolution embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence;

[0034] The single-scale feature extraction module is used to extract single-scale intra-level features of the serialized features of each level to obtain a single-scale element mixed feature sequence;

[0035] A cross-scale feature extraction module is used to splice the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, perform cross-scale inter-level feature extraction on the two-dimensional matrix, obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layer to re-obtain a cross-scale channel mixed feature sequence after splitting into several levels;

[0036] The regression processing module is used to extract and concatenate the elements at the end of the cross-scale channel mixed feature sequence after splitting at several levels to obtain a one-dimensional vector, and send the one-dimensional vector to the linear regressor to obtain the spatial transcriptome gene counts of the region of interest;

[0037] The atlas generation module is used to repeat the sampling module to the regression processing module to obtain the spatial transcription atlas of the entire slide specimen.

[0038] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:

[0039] one or more processors;

[0040] A memory for storing one or more programs;

[0041] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in the first aspect.

[0042] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0043] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0044] This application uses deep learning technology based on digital pathology images to predict spatial transcriptional maps, overcoming the problem that conventional spatial transcriptome analysis requires dedicated equipment and has high detection prices. It can provide spatial transcriptional information at an extremely low cost to assist doctors in disease diagnosis, greatly reducing patients' medical expenses and achieving the effect of benefiting the people with medical technology.

[0045] This application adopts a multi-level pathological image feature fusion method in the deep learning network design, which overcomes the deficiency of traditional deep learning schemes that can only capture single-scale features, and further improves the accuracy of spatial transcription map prediction, thereby helping doctors to understand the patient's condition more accurately and provide targeted and accurate diagnosis plans, which is conducive to improving the efficiency of diagnosis and treatment.

[0046] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0048] Figure 1 The present invention is a flowchart of a method for generating a spatial transcription atlas based on multi-level pathological image fusion according to an exemplary embodiment.

[0049] Figure 2 It is an overall structural diagram of a deep learning network used in a method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to an exemplary embodiment.

[0050] Figure 3 It is a structural diagram of a single-scale intra-level feature extraction module in a deep learning network used in a spatial transcription atlas generation method based on multi-level pathological image fusion according to an exemplary embodiment.

[0051] Figure 4 It is a structural diagram of a cross-scale inter-level feature extraction module in a deep learning network used in a spatial transcription atlas generation method based on multi-level pathological image fusion according to an exemplary embodiment.

[0052] Figure 5 It is a structural schematic diagram of a spatial transcription atlas generating device based on multi-level pathological image fusion according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0054] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0055] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0056] Figure 1 is a flow chart of a method for generating a spatial transcription atlas based on multi-level pathological image fusion according to an exemplary embodiment. Figure 2 is a diagram showing the overall structure of a deep learning network used in a method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to an exemplary embodiment. Figure 1 and Figure 2 As shown, the method may include the following steps:

[0057] S1: Obtain full scan pathological images of the slide specimen at at least two different magnifications;

[0058] Specifically, taking the digital pathology image scanner used by most hospitals as an example, digital pathology images at magnifications of 40 times, 20 times, 10 times, and 5 times can be acquired. As an example, digital pathology images at magnifications of 40 times, 20 times, and 10 times are selected for subsequent operations. Obviously, the sizes of the three-level images satisfy the relative relationship of 4 times, 2 times, and 1 time.

[0059] S2: sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level;

[0060] Specifically, the sampling process of the region of interest needs to follow the following principles:

[0061] In full scan pathology images of different magnifications, in addition to ensuring that the center of the sampling area can be uniformly aligned to the same position of the slide specimen, the sampled image size must also be enlarged or reduced with the image magnification. As an example, a 224*224 area of ​​interest is sampled for a 40x magnification image, so the sampling size becomes 112*112 on a 20x magnification image, 56*56 on a 10x magnification image, and so on.

[0062] S3: Send the images of each level to their respective convolutional embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence; this step includes the following sub-steps:

[0063] S31: Send the images of each level to the corresponding convolution embedding layer of each layer respectively to obtain the serialized features of each level, wherein the convolution kernel size and step size of each layer are enlarged or reduced according to the magnification of each level to ensure that the downsampled images of each level can finally be converted into serialized features with the same length;

[0064] Specifically, taking the 224*224 image obtained by upsampling at 40 times magnification with a convolution kernel and convolution step size of 16*16 as an example, the image will be converted into a sequence feature with a length of (224 / 16)*(224 / 16)=196; correspondingly, the 112*112 image at 20 times magnification should use a convolution kernel and convolution step size of 8*8 to ensure that it can also be converted into a sequence feature with a length of 196. The feature channels of each sequence do not need to be the same and can be set as In this manual, it is recommended to increase or decrease the image in proportion to the magnification. For example, if 192 channels are used at a 40x magnification, 96 channels should be used at a 20x magnification.

[0065] S32: In order to facilitate the subsequent integration of the features of the entire sequence into a single element, an additional learnable element is added to the back end of each serialized feature, increasing the sequence length by one bit.

[0066] Specifically, the feature sequence of the i-th level is randomly initialized to a shape of The learnable vector is then concatenated to the end of the original sequence feature of length 196, changing the sequence length to 197.

[0067] S4: Extract single-scale intra-level features from the serialized features of each level to obtain a single-scale element mixed feature sequence; Figure 3 , which includes the following sub-steps:

[0068] S41: performing layer normalization processing on the serialized features of each layer;

[0069] Specifically, the overall mean is calculated on all channels of each sequence feature and standard deviation . These statistics are then used to normalize each channel of this sample:

[0070]

[0071] in, is the ith channel, is the i-th channel of the layer-normalized feature.

[0072] S42: processing the normalized serialized features, and converting the serialized features of each level into three different channel compression feature matrices through three learnable parameter matrices;

[0073] Specifically, each sequence feature is first converted into three new feature matrices Q, K, and V (all with the shape , Refers to the number of channels of the i-th sequence feature, Refers to the number of channels after Q, K, V matrix transformation).

[0074] S43: performing matrix multiplication on the first channel compression feature matrix and the transpose of the second channel compression feature matrix to obtain an autocorrelation matrix of the elements in the sequence, and then performing matrix multiplication on the autocorrelation matrix and the third channel compression feature matrix to obtain a single-scale element mixed feature sequence after channel compression;

[0075] Specifically, under the guidance of the parameter of random mask ratio m, the following operations are performed:

[0076]

[0077] in, is a two-dimensional mask matrix whose elements are replaced by ( The probability of is randomly set to 0, and the probability of 1-m is set to 1. After this operation, each element in the sequence is associated with other elements in the sequence according to the correlation, thereby realizing global information interaction, and the random mask further enhances the robustness of the model.

[0078] S44: performing channel restoration on the single-scale element mixed feature sequence through another learnable matrix, making the number of channels completely consistent with the number before normalization, and obtaining a final single-scale element mixed feature sequence.

[0079] Specifically, after that, you need to use another shape The learnable parameter matrix of The result is restored to the same serialized characteristics as the input shape.

[0080] S5: splicing the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, performing cross-scale inter-level feature extraction on the two-dimensional matrix to obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layering to re-obtain a cross-scale channel mixed feature sequence after being split into several levels;

[0081] Specifically, splicing is performed according to the channel dimension to obtain A two-dimensional matrix of shape.

[0082] Among them, the cross-scale inter-level feature extraction is performed on the two-dimensional matrix to obtain the cross-scale channel mixed feature sequence. Figure 4 , specifically including the following sub-steps:

[0083] S51: performing layer normalization processing on the concatenated two-dimensional matrix;

[0084] Specifically, the layer normalization operation is consistent with S41, which can further help the rapid convergence of the network during training.

[0085] S52: Using the first learnable parameter matrix, the normalized two-dimensional matrix is ​​channel compressed to remove redundant information between layers;

[0086] Specifically, the first parameter matrix will be The feature matrix of is first compressed into characteristics.

[0087] S53: The number of channels of the two-dimensional matrix feature is restored to the same shape as before the input module through the second learnable parameter matrix, and a cross-scale channel mixed feature sequence is obtained.

[0088] Specifically, the second parameter matrix converts the previous The two-dimensional matrix of the shape is restored to The shape of is obtained to obtain the cross-scale channel mixing feature sequence.

[0089] Such a design will help remove redundant information between multiple scales and achieve more effective cross-level feature extraction with less computational effort.

[0090] The cross-scale channel mixed feature sequence is then split again according to the input layering to obtain a cross-scale channel mixed feature sequence after splitting into several levels.

[0091] S6: Take out and concatenate the elements at the end of the cross-scale channel mixed feature sequence after splitting at several levels to obtain a one-dimensional vector, and send the one-dimensional vector to the linear regressor to obtain the spatial transcriptome gene counts of the region of interest; this step includes the following sub-steps:

[0092] S61: performing layer normalization processing on the one-dimensional vector;

[0093] Specifically, the elements at the end of each sequence are taken out and concatenated to obtain dimensional vector. The layer normalization operation on this vector is consistent with S41, which can further help the rapid convergence of network training.

[0094] S62: A learnable linear regression parameter matrix is ​​used to perform matrix multiplication with the normalized one-dimensional vector to obtain the gene count to be predicted.

[0095] Specifically, through a shape The learnable matrix is ​​realized by , where G is the number of genes to be predicted. The output is G numbers, each number represents the number of times the corresponding gene appears in the pathological section sample.

[0096] S7: Repeat S2-S6 to obtain the spatial transcriptional map of the entire slide specimen.

[0097] Specifically, predictions are made one by one in a grid-like form according to fixed intervals and fixed sizes of regions of interest, and ultimately a spatial transcription map of the entire pathological section can be generated.

[0098] It can be seen from the above embodiments that the present application can generate a spatial transcriptional map of the entire pathological tissue section more accurately by performing feature fusion on multi-level pathological images when only digital pathological images are required, thereby greatly reducing the cost of spatial transcriptome analysis in disease diagnosis.

[0099] Three public data sets (HBC, HER2+, cSCC) were selected for testing. The results showed that the method proposed in the present invention can achieve 47.78%, 43.98%, and 50.70% Pearson correlation coefficients (PCC) with the least number of parameters and floating-point operations.

[0100]

[0101] Corresponding to the aforementioned embodiment of the method for generating a spatial transcription atlas based on multi-level pathological image fusion, the present application also provides an embodiment of a device for generating a spatial transcription atlas based on multi-level pathological image fusion.

[0102] Figure 5is a block diagram of a device for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to an exemplary embodiment. Figure 5 , the device comprises:

[0103] An acquisition module 1 is used to acquire full scan pathological images of a slide specimen at at least two different magnifications;

[0104] Sampling module 2, used for sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level;

[0105] Convolution processing module 3 is used to send the images of each level into their respective convolution embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence;

[0106] The single-scale feature extraction module 4 is used to extract single-scale intra-level features of the serialized features of each level to obtain a single-scale element mixed feature sequence;

[0107] The cross-scale feature extraction module 5 is used to splice the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, perform cross-scale inter-level feature extraction on the two-dimensional matrix, obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layer to re-obtain a cross-scale channel mixed feature sequence after splitting into several levels;

[0108] The regression processing module 6 is used to extract and concatenate the elements at the end of the cross-scale channel mixed feature sequence after being split into several levels to obtain a one-dimensional vector, and send the one-dimensional vector to a linear regressor to obtain the spatial transcriptome gene count of the region of interest;

[0109] The atlas generation module 7 is used to repeat the sampling module to the regression processing module to obtain the spatial transcription atlas of the entire slide specimen.

[0110] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0111] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0112] Correspondingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the spatial transcription map generation method based on multi-level pathological image fusion as described above.

[0113] Accordingly, the present application also provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the spatial transcription atlas generation method based on multi-level pathological image fusion as described above is implemented.

[0114] Those skilled in the art will readily appreciate other embodiments of the present application after considering the description and practicing the contents disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The description and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the claims.

[0115] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating a spatial transcriptional atlas based on multi-level pathological image fusion, characterized in that: include: S1: Obtain full scan pathological images of the slide specimen at at least two different magnifications; S2: sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level; S3: Send the images of each level to their respective convolutional embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence; S4: Perform single-scale intra-level feature extraction on the serialized features of each level to obtain a single-scale element mixed feature sequence; S5: splicing the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, performing cross-scale inter-level feature extraction on the two-dimensional matrix to obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layering to re-obtain a cross-scale channel mixed feature sequence after being split into several levels; S6: Take out and concatenate the elements at the end of the cross-scale channel mixed feature sequence after splitting at several levels to obtain a one-dimensional vector, and send the one-dimensional vector to the linear regressor to obtain the spatial transcriptome gene counts of the region of interest; S7: Repeat S2-S6 to obtain the spatial transcriptional map of the entire slide specimen.

2. The method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to claim 1, characterized in that: The sampling process of the region of interest needs to follow the following principles: In full-scan pathology images at different magnifications, in addition to ensuring that the center of the sampling area can be uniformly aligned to the same position of the slide specimen, the sampled image size also needs to be enlarged or reduced by a corresponding factor with the image magnification.

3. The method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to claim 1, characterized in that: The images at each level are sent to their respective convolutional embedding layers to obtain the serialized features of each level, and a learnable element is added to the end of each serialized feature, including: The images of each level are sent to the corresponding convolution embedding layer of each layer to obtain the serialized features of each level, where the convolution kernel size and step size of each layer are enlarged or reduced according to the magnification of each level to ensure that the downsampled images of each level can finally be converted into serialized features with consistent length; An additional randomly initialized learnable element is added to the back end of each serialized feature, increasing the sequence length by one.

4. The method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to claim 1, characterized in that: The serialized features of each level are subjected to single-scale intra-level feature extraction to obtain a single-scale element mixed feature sequence, including: S41: performing layer normalization processing on the serialized features of each layer; S42: processing the normalized serialized features, and converting the serialized features of each level into three different channel compression feature matrices through three learnable parameter matrices; S43: performing matrix multiplication on the first channel compression feature matrix and the transpose of the second channel compression feature matrix to obtain an autocorrelation matrix of the elements in the sequence, and then performing matrix multiplication on the autocorrelation matrix and the third channel compression feature matrix to obtain a single-scale element mixed feature sequence after channel compression; S44: performing channel restoration on the single-scale element mixed feature sequence through another learnable matrix, making the number of channels completely consistent with the number before normalization, and obtaining a final single-scale element mixed feature sequence.

5. The method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to claim 1, characterized in that: The cross-scale inter-level feature extraction is performed on the two-dimensional matrix to obtain a cross-scale channel mixed feature sequence, including: S51: performing layer normalization processing on the concatenated two-dimensional matrix; S52: Using the first learnable parameter matrix, the normalized two-dimensional matrix is ​​channel compressed to remove redundant information between layers; S53: The number of channels of the two-dimensional matrix after channel compression is restored to be consistent with that before the input module through a second learnable parameter matrix, thereby obtaining a cross-scale channel mixed feature sequence.

6. The method for generating a spatial transcriptional atlas based on multi-level pathological image fusion according to claim 1, characterized in that: This one-dimensional vector is fed into a linear regressor to obtain the spatial transcriptome gene counts of the region of interest, including: S61: performing layer normalization processing on the one-dimensional vector; S62: A learnable linear regression parameter matrix is ​​used to perform matrix multiplication with the normalized one-dimensional vector to obtain the gene count to be predicted.

7. A spatial transcription atlas generation device based on multi-level pathological image fusion, characterized in that: include: An acquisition module, used for acquiring full scan pathological images of the slide specimen at at least two different magnifications; A sampling module, used for sampling the region of interest of the pathological image at each level of the full scan pathological image at different magnifications to obtain images at each level; The convolution processing module is used to send the images of each level into their respective convolution embedding layers to obtain the serialized features of each level, and add a learnable element at the end of the serialized features of each level to encode the information of the entire sequence; The single-scale feature extraction module is used to extract single-scale intra-level features of the serialized features of each level to obtain a single-scale element mixed feature sequence; A cross-scale feature extraction module is used to splice the single-scale element mixed feature sequences of all levels into a complete two-dimensional matrix, perform cross-scale inter-level feature extraction on the two-dimensional matrix, obtain a cross-scale channel mixed feature sequence, and then re-split the cross-scale channel mixed feature sequence according to the input layer to re-obtain a cross-scale channel mixed feature sequence after splitting into several levels; The regression processing module is used to extract and concatenate the elements at the end of the cross-scale channel mixed feature sequence after splitting at several levels to obtain a one-dimensional vector, and send the one-dimensional vector to the linear regressor to obtain the spatial transcriptome gene counts of the region of interest; The atlas generation module is used to repeat the sampling module to the regression processing module to obtain the spatial transcription atlas of the entire slide specimen.

8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images