A method, apparatus, equipment and medium for three-dimensional reconstruction of biological tissues

Abstract

This application discloses a method, apparatus, device, and medium for three-dimensional reconstruction of biological tissues, relating to the field of three-dimensional reconstruction technology. The method includes: acquiring the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in a current pair of slices to be aligned; inputting the current first third-order tensor and the current second third-order tensor into a slice alignment network to output target alignment actions; determining whether there are adjacent slices to be aligned after the second slice; if not, aligning the tissue slices according to each target alignment action to perform three-dimensional reconstruction. By fusing gene expression matrix and spatial coordinate information to construct a third-order tensor, the spatial structure is explicitly encoded, solving the problem of geometric cues being obscured by the dominant expression pattern.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a method, apparatus, equipment and medium for three-dimensional reconstruction of biological tissues. Background Technology

[0002] Precise three-dimensional tissue reconstruction of continuous spatial transcriptome slices is key to revealing tissue structure and function.

[0003] In existing methods for three-dimensional reconstruction of biological tissues, gene expression patterns often mask local structural information. This results in weak geometric cues between slices, leading to local errors in the reconstruction results. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for three-dimensional reconstruction of biological tissues, which can construct a third-order tensor by fusing gene expression matrix and spatial coordinate information, thereby explicitly encoding spatial structure and solving the problem of geometric cues being masked by dominant expression patterns. The specific solution is as follows: Firstly, this application provides a method for three-dimensional reconstruction of biological tissues, including: Obtain the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. Input the current first third-order tensor and the current second third-order tensor into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice. Here, the third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned; the target slice alignment network is a feature pyramid network. Determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, update the current second third-order tensor corresponding to the second slice based on the target alignment action, determine the second slice and the adjacent slice to be aligned as a new pair of slices to be aligned, and jump to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned. If there are no adjacent slices to be aligned after the second slice, the corresponding tissue slices are aligned according to the target alignment actions, and the tissue slices are reconstructed in three dimensions according to the alignment results.

[0005] Optional, the generation process of the third-order tensor includes: Spatial transcriptome sequencing is performed on the current tissue slice to obtain spatial transcriptome data corresponding to the current slice; the spatial transcriptome data includes the gene expression matrix and spatial coordinate information corresponding to the current tissue slice. The gene expression matrix is ​​dimensionality reduced to obtain the corresponding dimensionality-reduced data. The dimensionality-reduced data is then concatenated with the corresponding spatial coordinate information to obtain the target feature matrix. Based on the target feature matrix, a third-order tensor corresponding to the current tissue slice is generated.

[0006] Optionally, generating the third-order tensor corresponding to the current tissue slice based on the target feature matrix includes: The gene expression features in the target feature matrix are normalized to obtain the corresponding normalized features, and the normalized features are mapped to a two-dimensional space grid of a preset size to obtain the coordinate positions of each point corresponding to the normalized features in the two-dimensional space grid. The normalized features are aggregated within the preset receptive fields corresponding to each of the coordinate positions to obtain the third-order tensor corresponding to the current tissue slice.

[0007] Optionally, before obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned, the method further includes: Define a target reward function, and iteratively train an initial slice alignment network based on the target reward function to obtain the target slice alignment network; wherein, the target reward function includes a geometric overlap term for measuring the degree of spatial overlap between two tissue slices, a positional deviation term for characterizing the distance between the centers of two tissue slices, an expression similarity term for evaluating the consistency of gene expression distribution between two tissue slices, and a reconstruction error term for measuring the difference between the third-order tensors of two tissue slices.

[0008] Optionally, the iterative training of the initial slice alignment network based on the target reward function includes: Obtain the current first historical tensor corresponding to the first historical slice and the current second historical tensor corresponding to the second historical slice in the historical slice pair to be aligned. Input the current first historical tensor and the current second historical tensor into the initial slice alignment network so that the initial slice alignment network iteratively aligns the historical slice pair to be aligned in several time steps and generates corresponding iterative trajectories. The iterative trajectory includes the reward, alignment action and alignment state corresponding to each time step. Determine whether the number of iteration trajectories is less than a preset threshold. If the number of iteration trajectories is less than the preset threshold, then proceed to the step of inputting the current first history tensor and the current second history tensor into the initial slice alignment network. If the number of iteration steps of the iteration trajectory is less than a preset threshold, the network parameters of the initial slice alignment network are iteratively updated according to each iteration trajectory and the target iteration number.

[0009] Optionally, the process of the target slice alignment network outputting the target alignment action corresponding to the second slice includes: The current first third-order tensor and the current second third-order tensor are concatenated to obtain the current state tensor of the current slice pair to be aligned; Several convolutional layers are used to extract features at different scales corresponding to the current state tensor to obtain corresponding multi-scale features, and the target alignment action corresponding to the second slice is obtained based on the multi-scale features.

[0010] Optionally, the three-dimensional reconstruction method for biological tissue further includes: Define the action space of the target slice alignment network; the action space includes translational degrees of freedom and rotational degrees of freedom. Accordingly, the process of the target slice alignment network outputting the target alignment action corresponding to the second slice includes: The target alignment action corresponding to the second slice is output according to the action space; the target alignment action includes the rotation angle of the second slice, the translation amount in the X-axis direction, and the translation amount in the X-axis direction.

[0011] Secondly, this application provides a three-dimensional reconstruction device for biological tissues, comprising: The tensor acquisition module is used to acquire the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. The current first third-order tensor and the current second third-order tensor are input into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice. The third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned. The target slice alignment network is a feature pyramid network. The step jump module is used to determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, the current second third-order tensor corresponding to the second slice is updated based on the target alignment action. The second slice and the adjacent slice to be aligned are determined as a new pair of slices to be aligned. Then, the jump is made to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned. The slice alignment module is used to align the corresponding tissue slices according to the target alignment actions if there are no adjacent slices to be aligned after the second slice, and to perform three-dimensional reconstruction of each tissue slice according to the corresponding alignment results.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the aforementioned method for three-dimensional reconstruction of biological tissues.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program that, when executed by a processor, implements the aforementioned method for three-dimensional reconstruction of biological tissues.

[0014] This application first obtains the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. The current first third-order tensor and the current second third-order tensor are then input into a target slice alignment network, so that the target slice alignment network outputs the target alignment action corresponding to the second slice. Here, the third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slices to be aligned. The target slice alignment network is a feature pyramid network. Then, it determines whether there are adjacent slices to be aligned after the second slice. If there are adjacent slices to be aligned after the second slice, the current second third-order tensor corresponding to the second slice is updated based on the target alignment action. The second slice and the adjacent slices to be aligned are determined as a new current slice pair to be aligned, and the process jumps to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. Finally, if there are no adjacent slices to be aligned after the second slice, the corresponding tissue slices are aligned according to each target alignment action, and three-dimensional reconstruction is performed on each tissue slice based on the corresponding alignment results. As can be seen, this application constructs a third-order tensor by fusing gene expression matrix and spatial coordinate information, which enables the explicit encoding of spatial structure and solves the problem of geometric cues being obscured by the dominant expression mode. By using a feature pyramid network as the target slice alignment network, it achieves the perception of multi-scale features and can capture global contours and local details at the same time, thereby using global geometric constraints to correct local errors. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a three-dimensional reconstruction method for biological tissue disclosed in this application; Figure 2 This is a schematic diagram of a mathematical modeling method disclosed in this application; Figure 3 This is a schematic diagram of a deep reinforcement learning network based on feature pyramids disclosed in this application; Figure 4 This application discloses a flowchart of a network parameter update process. Figure 5 This is a schematic diagram of a slice alignment result disclosed in this application; Figure 6 This is a schematic diagram of a tensor construction method disclosed in this application; Figure 7 This is a flowchart of a tensor construction method disclosed in this application; Figure 8 This is a schematic diagram of the structure of a three-dimensional reconstruction device for biological tissue disclosed in this application; Figure 9 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for three-dimensional reconstruction of biological tissue, comprising: Step S11: Obtain the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. Input the current first third-order tensor and the current second third-order tensor into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice. The third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned. The target slice alignment network is a feature pyramid network.

[0019] In this embodiment, the process of generating the third-order tensor includes: performing spatial transcriptome sequencing on the current tissue slice to obtain spatial transcriptome data corresponding to the current slice; the spatial transcriptome data includes the gene expression matrix and spatial coordinate information corresponding to the current tissue slice; performing dimensionality reduction on the gene expression matrix to obtain the corresponding dimensionality-reduced data; concatenating the dimensionality-reduced data with the corresponding spatial coordinate information to obtain the target feature matrix; and generating the third-order tensor corresponding to the current tissue slice based on the target feature matrix.

[0020] The process of generating the third-order tensor corresponding to the current tissue slice based on the target feature matrix includes: normalizing the gene expression features in the target feature matrix to obtain the corresponding normalized features, and mapping the normalized features to a two-dimensional spatial grid of a preset size to obtain the coordinate positions of each point corresponding to the normalized features in the two-dimensional spatial grid; and performing feature aggregation on the normalized features within the preset receptive fields corresponding to each coordinate position to obtain the third-order tensor corresponding to the current tissue slice.

[0021] In other words, in this embodiment, the gene expression matrix corresponding to the tissue slice is dimensionality-reduced, and the dimensionality-reduced data is concatenated with the corresponding coordinates. Finally, a third-order tensor corresponding to the current tissue slice is generated based on the concatenated target feature matrix. It can be understood that the generation process of the first and second third-order tensors is the same.

[0022] In addition, in this embodiment, mathematical modeling of the slice 3D reconstruction problem is also required to provide a theoretical basis for the slice 3D reconstruction process. The modeling results are as follows: Figure 2 As shown, specifically: Step 1: Model the slice alignment problem as a sequential decision process. Define the state, action, reward function, and state transition function.

[0023] Step 1.1: State Space. In each step, the model constructs the current state by concatenating the transformed third-order tensors of the two slices to be aligned.

[0024] Step 1.2: Action Space. Actions correspond to geometric transformations applied to the slices, including translations along the x and y axes and rotations about the center. The goal is to maximize structural and molecular consistency between adjacent slices by iteratively adjusting their positions.

[0025] Step 1.3: Reward Function (i.e., Target Reward Function). To quantify alignment quality, a combined reward function was designed that integrates geometric overlap, spatial consistency, and gene expression similarity.

[0026] ; Among them, IoU measures the spatial overlap between slices, Euclidean distance (ED) represents the distance between slice centers, FPCC assesses the consistency of gene expression distribution, and MSE measures the reconstruction error in tensor representation.

[0027] Step 1.3.1: The formula for calculating IoU is as follows: ; in and These represent the areas of the minimum bounding boxes of the two slices, respectively. This represents the area where two slices intersect.

[0028] Step 1.3.2: The formula for calculating ED is as follows: ; Where (x0,y0) and (x1,y1) represent the center points of the two slices, respectively.

[0029] Step 1.3.3: When calculating FPCC, for each target point tk, find the nearest [target point] in the distant slice. Align and match their gene expression characteristics and It measures the degree of matching of gene expression characteristics. The formula for calculating FPCC is as follows: ; Where N represents the number of points in the target slice.

[0030] Step 1.3.4: The formula for calculating MSE is as follows: ; in and This represents a three-dimensional tensor (dimensions C×H×W, C=50) generated based on spatial coordinates and gene expression features, with the input dimensions being consistent.

[0031] Step 1.4: To ensure the stability of the learning process, this embodiment designs a deterministic state transition function to update the slice state: ; in These are the current spatial coordinates of slice i. is the spatial coordinate of slice i after the model output action is executed, at is the current action, c is the centroid coordinate of the slice, R is the rotation matrix of action θ, and tr is the translation vector (x,y).

[0032] Accordingly, before obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned, the method further includes: defining a target reward function and iteratively training the initial slice alignment network based on the target reward function to obtain the target slice alignment network; wherein, the target reward function includes a geometric overlap term for measuring the degree of spatial overlap between two tissue slices, a positional deviation term for characterizing the distance between the centers of two tissue slices, an expression similarity term for evaluating the consistency of gene expression distribution between two tissue slices, and a reconstruction error term for measuring the difference between the third-order tensors of the two tissue slices.

[0033] The process of iteratively training the initial slice alignment network based on the target reward function can specifically include: obtaining the current first historical tensor corresponding to the first historical slice and the current second historical tensor corresponding to the second historical slice in the historical slice pair to be aligned; inputting the current first historical tensor and the current second historical tensor into the initial slice alignment network so that the initial slice alignment network iteratively aligns the historical slice pair to be aligned in several time steps and generates corresponding iterative trajectories; the iterative trajectory includes the reward, alignment action, and alignment state corresponding to each time step; determining whether the number of iterative trajectories is less than a preset number threshold; if the number of iterative trajectories is less than the preset number threshold, then jumping to the step of inputting the current first historical tensor and the current second historical tensor into the initial slice alignment network; if the number of iteration steps is less than the preset number threshold, then iteratively updating the network parameters of the initial slice alignment network according to each iterative trajectory and the target number of iterations.

[0034] The process of defining the objective function can be referred to the previous content, and will not be repeated here. The process of training the initial alignment network is as follows: Step 1: Given a set of states, a deep reinforcement learning action prediction network (i.e., the initial slice alignment network) is constructed to align the slices. This network is a feature pyramid network, taking the transformed and concatenated third-order tensor states of the slices to be aligned as input and outputting actions (x, y, θ). Rewards are generated based on actions, while the loss is calculated based on records. Specifically, the deep reinforcement learning network designed in this embodiment is as follows: Figure 3 As shown.

[0035] Step 1.1: Construct a neural network model, which can be a 4-layer feature pyramid network.

[0036] Step 1.2: Input the 3D tensor representation of the i-th slice pair into the network. In the figure, T represents the total number of time steps, and st represents the state at step t. After inputting the 3D tensor, features are extracted through four downsampling convolutional layers. Finally, three independent convolutional layers with pooling are used, and actions (x, y, θ) are generated using the tanh activation function. ReLU is used as the activation function for all layers. To prevent over-reliance on a single neuron during action generation, Dropout with a dropout rate of 0.05 is applied.

[0037] Step 2: Parameter training process as follows Figure 4 As shown.

[0038] Step 2.1: At each time step, the agent interacts with the environment: based on the current state, it selects an action through the network described above, obtains a reward, and transitions to the next state, forming a transition tuple. The four parameters in the transition tuple represent, in turn, the state of the slice pair to be aligned at the current time step, the alignment action, the reward value, and the next state after performing the action.

[0039] Step 2.2: Execute S4 for T time steps, where T is the number of time steps, and record the reward, action, state, and new state for each step. This forms a trajectory.

[0040] Step 2.3: Record O trajectories, where O can be 2. Estimate the loss using these O trajectories and perform backpropagation. The training objective is defined based on the cumulative discounted reward across multiple trajectories: ; in It is a discount factor. This represents the reward starting from time step t, where O and T represent the number of trajectories and the number of steps per trajectory, respectively. The network parameters are optimized using backpropagation and gradient descent.

[0041] Step 2.4: Iterate E times, where E can be 500, and save the training weights. These training weights can be used to generate the action to align the two slices to be aligned. Unlike traditional methods, this method directly aligns the global slices instead of extracting local structures for alignment.

[0042] Step 3: Perform steps 2.1-2.4 for each slice to be aligned to obtain training weights that can align all slices.

[0043] Step 4: The trained weights are called sequentially, the data is restored to its original size before regression normalization, and the results are visualized. The experimental effects before and after 3D reconstruction are shown below. Figure 5 As shown.

[0044] In addition, the process of the target slice alignment network outputting the target alignment action corresponding to the second slice includes: concatenating the current first third-order tensor and the current second third-order tensor to obtain the current state tensor of the current slice pair to be aligned; using several convolutional layers to extract features of different scales corresponding to the current state tensor to obtain the corresponding multi-scale features, and obtaining the target alignment action corresponding to the second slice based on the multi-scale features.

[0045] That is, the 3D tensor representation of the i-th slice pair is input into the network. After inputting the 3D tensor, features are extracted through four downsampling convolutional layers. Finally, three independent convolutional layers with pooling are used, and the alignment action (x, y, θ) is generated through the tanh activation function.

[0046] Furthermore, the method in this embodiment also includes: defining the action space of the target slice alignment network; the action space includes translational degrees of freedom and rotational degrees of freedom; Correspondingly, the process of the target slice alignment network outputting the target alignment action corresponding to the second slice includes: outputting the target alignment action corresponding to the second slice according to the action space; the target alignment action includes the rotation angle of the second slice, the translation amount in the X-axis direction, and the translation amount in the X-axis direction.

[0047] That is, this embodiment also defines a target action space, the process of which can be referred to the relevant content disclosed in the aforementioned mathematical modeling section, and will not be repeated here. By defining the target action space, the model can output the corresponding slice alignment action.

[0048] Step S12: Determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, update the current second third-order tensor corresponding to the second slice based on the target alignment action, determine the second slice and the adjacent slice to be aligned as a new pair of slices to be aligned, and jump to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned.

[0049] It is understandable that the slices in this embodiment are aligned in pairs. After aligning the current pair of slices to be aligned, it is necessary to determine whether there are any adjacent slices to be aligned. If so, the current second and third order tensors need to be updated, and the current second slice and the adjacent slices to be aligned are determined as the new current pair of slices to be aligned, and the above alignment operation is repeated. For example, in this embodiment, there are sequentially adjacent slices 1, 2, and 3. First, slices 1 and 2 are determined as the current pair of slices to be aligned. After obtaining the alignment action corresponding to slice 2, since slice 3 also exists, the third order tensor of slice 2 needs to be updated according to the alignment action of slice 2, and slices 2 and 3 are determined as the new current pair of slices to be aligned, so as to output the alignment action corresponding to slice 3, thereby aligning the entire slice sequence.

[0050] Step S13: If there are no adjacent slices to be aligned after the second slice, then align the corresponding tissue slices according to each target alignment action, and perform three-dimensional reconstruction of each tissue slice according to the corresponding alignment result.

[0051] It is understandable that if there are no adjacent slices to be aligned after the second slice, it means that all slices except the first slice have obtained the corresponding alignment action. At this time, the slice sequence is aligned according to the alignment action corresponding to each slice to achieve three-dimensional reconstruction of biological tissue.

[0052] In summary, this embodiment converts point clouds into third-order tensors to obtain structured features combining spatial and gene expression data. Sequential decision modeling formalizes the alignment process as a sequential decision problem, with a focus on designing a combined reward function. A pyramid feature extraction network is used to learn multi-scale gene expression features in the spatial data. Furthermore, this embodiment also designs a deep reinforcement learning feature pyramid network to learn multi-scale slice features, achieving global slice alignment. First, the gene expression information of the complete slices to be aligned is obtained and converted into a third-order tensor. Then, sequential decision modeling is performed based on the combined reward function. Finally, the deep reinforcement learning feature pyramid network learns multi-scale slice features to achieve global slice alignment.

[0053] As can be seen, this application constructs a third-order tensor by fusing gene expression matrix and spatial coordinate information, which enables the explicit encoding of spatial structure and solves the problem of geometric cues being obscured by the dominant expression mode. By using a feature pyramid network as the target slice alignment network, it achieves the perception of multi-scale features and can capture global contours and local details at the same time, thereby using global geometric constraints to correct local errors.

[0054] As can be seen from the foregoing embodiments, this application describes the overall process of three-dimensional reconstruction of biological tissue slices. To make the process of generating third-order tensors in this application more complete, the process of generating third-order tensors will be described in detail below. See [link to relevant documentation]. Figure 6 As shown, this embodiment of the invention discloses a process for generating a third-order tensor, including: Step S21: Perform spatial transcriptome sequencing on the current tissue slice to obtain the spatial transcriptome data corresponding to the current slice; the spatial transcriptome data includes the gene expression matrix and spatial coordinate information corresponding to the current tissue slice.

[0055] In this embodiment, the spatial transcriptome data corresponding to the tissue slices must first be obtained in order to obtain the corresponding third-order tensor based on the spatial transcriptome data. The specific process is as follows: Step 1: Obtain the biological tissue sample to be aligned, for example, from a university or affiliated hospital of a medical school. Use a slicing device to prepare multiple tissue slices from the tissue sample. The slices are then fixed and embedded to maintain the integrity of the tissue structure. During this process, the multiple tissue slices may be twisted, rotated, displaced, or damaged. In this embodiment, the slicing step can be performed by a physician or biologist with slicing skills.

[0056] Step 2: Perform spatial transcriptome sequencing on the multiple tissue slices that have undergone distortion, rotation, displacement, or damage, to obtain the gene expression count matrix (i.e., gene expression matrix) and its spatial coordinate information for all slices. In this embodiment, the sequencing step is performed by a biotechnology company with mature spatial transcriptome sequencing technology and analysis capabilities.

[0057] Step S22: Perform dimensionality reduction on the gene expression matrix to obtain the corresponding dimensionality-reduced data, concatenate the dimensionality-reduced data with the corresponding spatial coordinate information to obtain the target feature matrix, and generate the third-order tensor corresponding to the current tissue slice based on the target feature matrix.

[0058] In this embodiment, the above-mentioned generation of the third-order tensor corresponding to the current tissue slice based on the target feature matrix includes: normalizing the gene expression features in the target feature matrix to obtain the corresponding normalized features, and mapping the normalized features to a two-dimensional spatial grid of a preset size to obtain the coordinate positions of each point corresponding to the normalized features in the two-dimensional spatial grid; and performing feature aggregation on the normalized features within the preset receptive fields corresponding to each coordinate position to obtain the third-order tensor corresponding to the current tissue slice.

[0059] The above process for dimensionality reduction of the gene expression matrix to obtain a third-order tensor includes: Step 3: Dimensionality reduction of gene expression data from the spatial transcriptome data of continuous tissue sections. Specifically, for each locus, this embodiment uses principal component analysis to generate 50 new gene expression features through linear transformation, and concatenates them with spatial coordinates to obtain a feature matrix (i.e., the target feature matrix) of shape (s × 52), where s is the number of loci in the section. For all sections, the resulting data is formalized as (n, sm, 52), where n is the number of sections and sm is the maximum number of loci in the section. Sections with fewer than sm loci are padded with zeros accordingly.

[0060] Step 4: The feature matrix of shape (s × 52) is converted into a third-order tensor of shape (W, H, 50) using a tensor transformation module, where W and H represent the spatial width and height, and 50 represents the gene expression features. Specifically, the gene expression features are first normalized between sites, then the normalized features are mapped to a spatial grid using normalized spatial coordinates. Finally, to capture local spatial continuity, this embodiment aggregates features within a square receptive field around each mapped coordinate to construct a tensor representation.

[0061] Step 4.1: Normalize gene expression characteristics: ; Where k represents the index of the k-th site (sample), k=1,2,…,N, and c represents the expression feature of the c-th gene, i.e., the principal component number after PCA (c=1,2,…,50). This represents the original value (principal component score) of site k on the c-th feature. This represents the minimum value of the c-th feature across all sites (index j). Conversely, To represent a very small positive number (10⁻⁸), to prevent the denominator from being 0, This represents the normalized eigenvalues, which fall within the range [0,1].

[0062] Step 4.2: Map the normalized features to a spatial grid.

[0063] ; in The original spatial coordinates of site k (e.g., micrometer coordinates on a tissue slice) are represented by W and H, which represent the width and height of the target mesh. It is a very small positive number (10⁻⁸), to prevent the denominator from being 0. This represents the floating-point coordinates of the point k mapped onto the grid, approximately within the range of [0, W) and [0, H).

[0064] Step 4.3: Aggregate features within a square receptive field around each mapped coordinate to construct a tensor representation.

[0065] Step 4.3.1: For each site k (original normalized coordinates are...), First, calculate its corresponding integer grid coordinates: ; The round(·) function indicates rounding to the nearest integer.

[0066] Step 4.3.2: Define a square receptive field Rk with side length L centered at (yk, xk): ; in, N is the total number of sites in the slice, and floor is the floor function.

[0067] Step 4.3.3: Evenly distribute the 50-dimensional gene expression features of this locus to each grid point within Rk. Finally, for each grid point (y, x), average all its accumulated feature vectors to obtain the tensor element: ; Where Sy,x is the set of all site-offset pairs that fall into the grid point after feature assignment. If a grid point has no features falling into it, its value is set to 0. This yields the third-order tensor. .

[0068] In summary, the third-order tensor construction process in this embodiment is as follows: Figure 7 As shown, the steps include spatial transcriptome sequencing, spatial transcriptome data dimensionality reduction, and transformation of the feature matrix into a third-order tensor.

[0069] See Figure 8 As shown, an embodiment of the present invention discloses a three-dimensional reconstruction device for biological tissue, comprising: Tensor acquisition module 11 is used to acquire the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned, and input the current first third-order tensor and the current second third-order tensor into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice; wherein, the third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned; the target slice alignment network is a feature pyramid network; The step jump module 12 is used to determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, the current second third-order tensor corresponding to the second slice is updated based on the target alignment action. The second slice and the adjacent slice to be aligned are determined as a new pair of slices to be aligned. Then, the jump is made to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned. The slice alignment module 13 is used to align the corresponding tissue slices according to the target alignment actions if there are no adjacent slices to be aligned after the second slice, and to perform three-dimensional reconstruction of each tissue slice according to the corresponding alignment results.

[0070] In some specific embodiments, the tensor acquisition module 11 may specifically include: The transcriptome data acquisition unit is used to perform spatial transcriptome sequencing on the current tissue slice to obtain the spatial transcriptome data corresponding to the current slice; the spatial transcriptome data includes the gene expression matrix and spatial coordinate information corresponding to the current tissue slice. The tensor generation submodule is used to perform dimensionality reduction processing on the gene expression matrix to obtain the corresponding dimensionality-reduced data, concatenate the dimensionality-reduced data with the corresponding spatial coordinate information to obtain the target feature matrix, and generate the third-order tensor corresponding to the current tissue slice based on the target feature matrix.

[0071] In some specific embodiments, the tensor generation submodule may specifically include: The feature mapping unit is used to normalize the gene expression features in the target feature matrix to obtain the corresponding normalized features, and to map the normalized features to a two-dimensional space grid of a preset size to obtain the coordinate positions of each point corresponding to the normalized features in the two-dimensional space grid. The tensor acquisition unit is used to perform feature aggregation on the normalized features within the preset receptive fields corresponding to each of the coordinate positions, so as to obtain the third-order tensor corresponding to the current tissue slice.

[0072] In some specific embodiments, the tensor acquisition module 11 further includes: The model training submodule is used to define a target reward function and iteratively train the initial slice alignment network based on the target reward function to obtain the target slice alignment network; wherein, the target reward function includes a geometric overlap term for measuring the degree of spatial overlap between two tissue slices, a positional deviation term for characterizing the distance between the centers of two tissue slices, an expression similarity term for evaluating the consistency of gene expression distribution between two tissue slices, and a reconstruction error term for measuring the difference between the third-order tensors of two tissue slices.

[0073] In some specific embodiments, the model training submodule may specifically include: The historical data acquisition unit is used to acquire the current first historical tensor corresponding to the first historical slice and the current second historical tensor corresponding to the second historical slice in the historical slice pair to be aligned, and input the current first historical tensor and the current second historical tensor into the initial slice alignment network so that the initial slice alignment network iteratively aligns the historical slice pair to be aligned in several time steps and generates a corresponding iterative trajectory; the iterative trajectory includes the reward, alignment action and alignment state corresponding to each time step; The step jump unit is used to determine whether the number of iteration trajectories is less than a preset number threshold. If the number of iteration trajectories is less than the preset number threshold, the jump is made to the step of inputting the current first history tensor and the current second history tensor into the initial slice alignment network. The parameter update unit is used to iteratively update the network parameters of the initial slice alignment network based on each iterative trajectory and the target iteration number if the number of iteration steps of the iterative trajectory is less than a preset threshold.

[0074] In some specific embodiments, the target slice alignment network may specifically include: The tensor splicing module is used to splice the current first third-order tensor and the current second third-order tensor to obtain the current state tensor of the current slice pair to be aligned; The feature extraction module is used to extract features of different scales corresponding to the current state tensor using several convolutional layers to obtain corresponding multi-scale features, and to obtain the target alignment action corresponding to the second slice based on the multi-scale features.

[0075] In some specific embodiments, the biological tissue three-dimensional reconstruction device further includes: The action space definition module is used to define the action space of the target slice alignment network; the action space includes translational degrees of freedom and rotational degrees of freedom. Accordingly, the target slice alignment network includes: The alignment action output module is used to output the target alignment action corresponding to the second slice according to the action space; the target alignment action includes the rotation angle of the second slice, the translation amount in the X-axis direction, and the translation amount in the X-axis direction.

[0076] Furthermore, embodiments of this application also disclose an electronic device, Figure 9 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0077] Figure 9 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the three-dimensional reconstruction method of biological tissue disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0078] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0079] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0080] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the biological tissue three-dimensional reconstruction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0081] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for three-dimensional reconstruction of biological tissues. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0082] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0083] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0084] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0085] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for three-dimensional reconstruction of biological tissue, characterized in that, include: Obtain the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. Input the current first third-order tensor and the current second third-order tensor into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice. Here, the third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned; the target slice alignment network is a feature pyramid network. Determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, update the current second third-order tensor corresponding to the second slice based on the target alignment action, determine the second slice and the adjacent slice to be aligned as a new pair of slices to be aligned, and jump to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned. If there are no adjacent slices to be aligned after the second slice, the corresponding tissue slices are aligned according to the target alignment actions, and the tissue slices are reconstructed in three dimensions according to the alignment results.

2. The method for three-dimensional reconstruction of biological tissue according to claim 1, characterized in that, The generation process of a third-order tensor includes: Spatial transcriptome sequencing is performed on the current tissue slice to obtain spatial transcriptome data corresponding to the current slice; the spatial transcriptome data includes the gene expression matrix and spatial coordinate information corresponding to the current tissue slice. The gene expression matrix is ​​dimensionality reduced to obtain the corresponding dimensionality-reduced data. The dimensionality-reduced data is then concatenated with the corresponding spatial coordinate information to obtain the target feature matrix. Based on the target feature matrix, a third-order tensor corresponding to the current tissue slice is generated.

3. The method for three-dimensional reconstruction of biological tissues according to claim 2, characterized in that, The step of generating the third-order tensor corresponding to the current tissue slice based on the target feature matrix includes: The gene expression features in the target feature matrix are normalized to obtain the corresponding normalized features, and the normalized features are mapped to a two-dimensional space grid of a preset size to obtain the coordinate positions of each point corresponding to the normalized features in the two-dimensional space grid. The normalized features are aggregated within the preset receptive fields corresponding to each of the coordinate positions to obtain the third-order tensor corresponding to the current tissue slice.

4. The method for three-dimensional reconstruction of biological tissue according to claim 1, characterized in that, Before obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned, the method further includes: Define a target reward function, and iteratively train an initial slice alignment network based on the target reward function to obtain the target slice alignment network; wherein, the target reward function includes a geometric overlap term for measuring the degree of spatial overlap between two tissue slices, a positional deviation term for characterizing the distance between the centers of two tissue slices, an expression similarity term for evaluating the consistency of gene expression distribution between two tissue slices, and a reconstruction error term for measuring the difference between the third-order tensors of two tissue slices.

5. The method for three-dimensional reconstruction of biological tissue according to claim 4, characterized in that, The iterative training of the initial slice alignment network based on the target reward function includes: Obtain the current first historical tensor corresponding to the first historical slice and the current second historical tensor corresponding to the second historical slice in the historical slice pair to be aligned. Input the current first historical tensor and the current second historical tensor into the initial slice alignment network so that the initial slice alignment network iteratively aligns the historical slice pair to be aligned in several time steps and generates corresponding iterative trajectories. The iterative trajectory includes the reward, alignment action and alignment state corresponding to each time step. Determine whether the number of iteration trajectories is less than a preset threshold. If the number of iteration trajectories is less than the preset threshold, then proceed to the step of inputting the current first history tensor and the current second history tensor into the initial slice alignment network. If the number of iteration steps of the iteration trajectory is less than a preset threshold, the network parameters of the initial slice alignment network are iteratively updated according to each iteration trajectory and the target iteration number.

6. The method for three-dimensional reconstruction of biological tissue according to claim 1, characterized in that, The process by which the target slice alignment network outputs the target alignment action corresponding to the second slice includes: The current first third-order tensor and the current second third-order tensor are concatenated to obtain the current state tensor of the current slice pair to be aligned; Several convolutional layers are used to extract features at different scales corresponding to the current state tensor to obtain corresponding multi-scale features, and the target alignment action corresponding to the second slice is obtained based on the multi-scale features.

7. The method for three-dimensional reconstruction of biological tissue according to any one of claims 1 to 6, characterized in that, Also includes: Define the action space of the target slice alignment network; The motion space includes translational degrees of freedom and rotational degrees of freedom; Accordingly, the process of the target slice alignment network outputting the target alignment action corresponding to the second slice includes: Output the target alignment action corresponding to the second slice according to the action space; The target alignment action includes the rotation angle of the second slice, the translation amount in the X-axis direction, and the translation amount in the X-axis direction.

8. A three-dimensional reconstruction device for biological tissue, characterized in that, include: The tensor acquisition module is used to acquire the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the current slice pair to be aligned. The current first third-order tensor and the current second third-order tensor are input into the target slice alignment network so that the target slice alignment network outputs the target alignment action corresponding to the second slice. The third-order tensor is data constructed based on the gene expression matrix and spatial coordinate information of the slice to be aligned. The target slice alignment network is a feature pyramid network. The step jump module is used to determine whether there is an adjacent slice to be aligned after the second slice. If there is an adjacent slice to be aligned after the second slice, the current second third-order tensor corresponding to the second slice is updated based on the target alignment action. The second slice and the adjacent slice to be aligned are determined as a new pair of slices to be aligned. Then, the jump is made to the step of obtaining the current first third-order tensor corresponding to the first slice and the current second third-order tensor corresponding to the second slice in the pair of slices to be aligned. The slice alignment module is used to align the corresponding tissue slices according to the target alignment actions if there are no adjacent slices to be aligned after the second slice, and to perform three-dimensional reconstruction of each tissue slice according to the corresponding alignment results.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the three-dimensional reconstruction method of biological tissue as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the three-dimensional reconstruction method of biological tissue as described in any one of claims 1 to 7.

Images