A virus structure three-dimensional reconstruction method and system

Abstract

The embodiment of the application discloses a virus structure three-dimensional reconstruction method and system, the method comprises the following steps: acquiring stereoscopic data and point cloud data containing a target virus sample; separating virus particles from the point cloud data by using an optimized point cloud segmentation algorithm; separating virus regions from the stereoscopic data by using a stereoscopic data segmentation algorithm; processing the virus stereoscopic data by using a first neural network to obtain density characteristics of the virus structure; processing the virus point cloud by using a second neural network to obtain geometric characteristics of the virus surface; fusing the density characteristics and the geometric characteristics by using a multi-modal fusion model to generate unified feature representation; and reconstructing a three-dimensional model of the virus structure based on the unified feature representation. The application can accurately obtain the three-dimensional structure of the virus in a complex background, and improves the accuracy of virus three-dimensional reconstruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of biomedical imaging and computer vision, and particularly relates to a virus structure three-dimensional reconstruction method and system. BACKGROUND

[0002] Three-dimensional reconstruction of virus structure is an important research direction in virology, drug development and vaccine design. Traditional virus structure analysis methods, such as cryo-electron microscopy stereomicroscopy, can provide high-resolution electron density distribution data, but still face challenges such as complex biological background interference and difficulty in extracting surface geometric features. At the same time, the fine features of virus structure (such as sub-nanometer details of surface proteins) require high-precision segmentation and reconstruction algorithms.

[0003] Therefore, there is a need for a virus structure three-dimensional reconstruction method that combines virus biology priori and realizes sub-nanometer resolution segmentation and multi-modal fusion to overcome the limitations of existing technologies. SUMMARY

[0004] The present application provides a virus structure three-dimensional reconstruction method, which improves the accuracy of virus three-dimensional reconstruction.

[0005] The present application provides the following solutions:

[0006] According to a first aspect, a virus structure three-dimensional reconstruction method is provided, the method comprising: obtaining stereoscopic data containing a target virus sample, the stereoscopic data including electron density distribution of the target virus sample; obtaining point cloud data, the point cloud data including a three-dimensional point set representing surface geometric features of the target virus sample; separating virus particles from the point cloud data using an optimized point cloud segmentation algorithm, the optimized point cloud segmentation algorithm combining biological prior knowledge of virus surface protein distribution and sub-nanometer attention mechanism to remove interference of cell debris and non-virus particles, generating separated virus point cloud; separating virus region from the stereoscopic data using a stereoscopic data segmentation algorithm, wherein the stereoscopic data segmentation algorithm aligns the virus point cloud through cross-modal collaborative segmentation to remove interference of cell debris and non-virus particles, generating separated virus stereoscopic data; processing the virus stereoscopic data using a first neural network to obtain density features of the virus structure; processing the virus point cloud using a second neural network to obtain geometric features of the virus surface; fusing the density features and the geometric features using a multi-modal fusion model to generate unified feature representation; reconstructing a three-dimensional model of the virus structure based on the unified feature representation to obtain a three-dimensional real scene model of the target virus sample.

[0007] According to an implementable manner in the embodiments of the present application, the optimized point cloud segmentation algorithm is implemented by a point cloud neural network based on a sub-nanometer attention mechanism; wherein the sub-nanometer attention mechanism comprises: for the sub-nanometer resolution of the virus point cloud, the local neighborhood range of attention calculation is limited to a radius of 0.5-2 nanometers; biological prior knowledge of virus surface protein distribution is utilized to generate prior weights according to virus symmetry and expected protein positions, and the attention score is adjusted according to the prior weights.

[0008] According to an implementable manner in the embodiments of the present application, the generation of prior weights according to virus symmetry and expected protein positions and the adjustment of the attention score according to the prior weights comprise: calculating the local density of each point in the point cloud data, wherein the local density is determined by the average distance inverse between the k-neighbor points of each point in the point cloud data; generating a density weight based on the local density, enhancing the weight of high-density areas and attenuating the weight of low-density areas through a nonlinear function; generating a prior weight by combining the biological prior of the virus, wherein the prior weight is determined based on the symmetry and surface protein distribution of the virus, and the distance between the point cloud points and the virus center and the protein center is determined through a Gaussian kernel function; fusing the density weight and the prior weight to generate a comprehensive weight; and adjusting the score of the attention module in the point cloud segmentation algorithm according to the comprehensive weight.

[0009] According to an implementable manner in the embodiments of the present application, the first neural network adopts a deep learning model based on a three-dimensional U-Net.

[0010] According to an implementable manner in the embodiments of the present application, the second neural network is implemented by a PointNet model, a PointNet++ model or a PointTransformer model.

[0011] According to an implementable manner in the embodiments of the present application, the multi-modal fusion model aligns the density features and the geometric features by using a cross-modal attention mechanism, and optimizes feature extraction by combining biological prior knowledge of virus symmetry and surface protein distribution; wherein the cross-modal attention mechanism comprises: projecting the density features from the stereoscopic grid to the point cloud coordinate space, and generating a density feature representation aligned with the geometric features through trilinear interpolation; generating a biological prior weight based on the symmetry and surface protein distribution of the virus, wherein the prior weight is obtained by calculating the distance between the point cloud points and the virus center or the protein center in the Gaussian kernel function; constructing a cross-modal attention module, taking the density features as the query, the geometric features as the key and the value, calculating the attention score, and adjusting the score by the prior weight; taking the geometric features as the query, the density features as the key and the value, calculating the reverse attention score, fusing the prior weight to generate enhanced geometric features; and generating fusion features by fusing the enhanced density features and geometric features through a multi-layer perception.

[0012] According to an implementable manner in the embodiments of the present application, the calculating the attention score comprises: obtaining the attention score through parallel calculation of 4 or more attention heads; wherein the attention heads comprise: an attention head for analyzing the correlation degree of the density feature and the geometric feature of the spike protein region, and an attention head for analyzing the correlation degree of the smooth geometry and the low density feature of the virus membrane region.

[0013] According to a second aspect, a virus structure three-dimensional reconstruction system is provided, comprising: a stereoscopic data acquisition unit configured to acquire stereoscopic data containing a target virus sample, the stereoscopic data comprising an electron density distribution of the target virus sample; a point cloud data acquisition unit configured to acquire point cloud data, the point cloud data comprising a three-dimensional point set representing the surface geometric feature of the target virus sample; a point cloud data segmentation unit configured to separate virus particles from the point cloud data using an optimized point cloud segmentation algorithm, the optimized point cloud segmentation algorithm combining biological prior knowledge of virus surface protein distribution and sub-nanometer attention mechanism to remove interference of cell debris and non-virus particles, and generating separated virus point cloud; a stereoscopic data segmentation unit configured to separate virus regions from the stereoscopic data using a stereoscopic data segmentation algorithm, wherein the stereoscopic data segmentation algorithm aligns the virus point cloud through cross-modal collaborative segmentation, removes interference of cell debris and non-virus particles, and generates separated virus stereoscopic data; a density feature extraction unit configured to process the virus stereoscopic data using a first neural network to obtain the density feature of the virus structure; a geometric feature extraction unit configured to process the virus point cloud using a second neural network to obtain the geometric feature of the virus surface; a feature fusion unit configured to fuse the density feature and the geometric feature using a multi-modal fusion model to generate a unified feature representation; and a model reconstruction unit configured to reconstruct a three-dimensional model of the virus structure based on the unified feature representation to obtain a three-dimensional real scene model of the target virus sample.

[0014] According to a third aspect, a computer readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, implements the steps of the method of any one of the above first aspect.

[0015] According to a fourth aspect, an electronic device is provided, comprising: one or more processors; and a memory associated with the one or more processors, the memory being configured to store program instructions, which, when executed by the one or more processors, perform the steps of the method of any one of the above first aspect.

[0016] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0017] The application combines the multi-modal fusion of stereoscopic data and point cloud data, adopts an optimized point cloud segmentation algorithm and stereoscopic data segmentation algorithm, effectively removes the interference of cell debris and non-virus particles, and accurately separates virus particles and virus regions. By introducing biological prior knowledge and sub-nanometer attention mechanism, the segmentation accuracy and processing efficiency are improved. The first neural network and the second neural network are used to extract the density features and geometric features of the virus structure respectively, and then the multi-modal fusion model is used to integrate the two features to generate a unified feature representation, and accurately reconstruct the three-dimensional model of the virus. The method can accurately obtain the three-dimensional structure of the virus in a complex background, and improves the accuracy of virus three-dimensional reconstruction.

[0018] Of course, implementing any product of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 The system architecture diagram applicable to the embodiments of the present application;

[0021] Figure 2 The flowchart of the method for three-dimensional reconstruction of virus structure provided by the embodiments of the present application;

[0022] Figure 3 The structural block diagram of the virus structure three-dimensional reconstruction system provided by the embodiments of the present application;

[0023] Figure 4 The schematic block diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art belong to the scope of protection of the present application.

[0025] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates otherwise.

[0026] It should be understood that the term "and / or" used herein only describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B, which can represent three cases of existence of A alone, existence of A and B together, and existence of B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0027] Depending on the context, the word "if" as used herein can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if it is determined" or "if (a stated condition or event) is detected" can be interpreted as meaning "when it is determined" or "in response to determining" or "when (a stated condition or event) is detected" or "in response to detecting (a stated condition or event)".

[0028] Currently, there are some technologies that can perform three-dimensional reconstruction based on volume data. These technologies do not use point cloud data, but directly convert volume data into a surface mesh. Although these technologies can achieve three-dimensional reconstruction, they still face problems such as complex background interference, loss of surface details, and insufficient sub-nanometer precision.

[0029] In view of this, the present application provides a new approach. In order to facilitate understanding of the present application, the system architecture based on the present application is first described. Figure 1 An exemplary system architecture to which embodiments of the present application can be applied is shown in FIG. 1, which can include a user device and a three-dimensional reconstruction system located on a server side. Figure 1

[0030] The user can input volume data and point cloud data through the user device, and the user device sends them to the three-dimensional reconstruction system on the server side. The three-dimensional reconstruction system can use the method provided in the embodiments of the present application to perform three-dimensional reconstruction on the target virus sample and obtain a three-dimensional real scene model. The server side can send the three-dimensional real scene model to the user terminal, and the user terminal can use the three-dimensional real scene model to perform rendering and obtain a two-dimensional image, a three-dimensional image, a VR (Virtual Reality) scene, or an AR (Augmented Reality) scene, etc.

[0031] ​The user equipment can include, but is not limited to, a smart mobile terminal, a smart home device, a wearable device, a PC (Personal Computer), and the like. The smart mobile terminal can include a mobile phone, a tablet computer, a notebook computer, a PDA (Personal Digital Assistant), an Internet car, and the like. The smart home device can include a smart television, a smart refrigerator, and the like. The wearable device can include a smart watch, smart glasses, a virtual reality device, an augmented reality device, a mixed reality device (i.e., a device that can support virtual reality and augmented reality), and the like.

[0032] The three-dimensional reconstruction system can be set as a stand-alone server, a server group, or a cloud server. The cloud server, also known as a cloud computing server or a cloud host, is a host product in a cloud computing service system, which solves the defects of large management difficulty and weak service scalability in traditional physical hosts and virtual private server (VPS) services. In addition to the architecture shown in the figure, the three-dimensional reconstruction system can also be set in a computer terminal with strong computing power. Figure 1

[0033] It should be understood that Figure 1 The user equipment and the three-dimensional reconstruction system in the figure are only illustrative. According to the implementation needs, there can be any number of user equipment and three-dimensional reconstruction systems.

[0034] Figure 2 A flowchart of a method for three-dimensional reconstruction of a virus structure is provided for the embodiments of the present application. The method can be executed by the virus structure three-dimensional reconstruction system in the system shown in the figure. As shown in the figure, the method can include the following steps: Figure 1 Figure 2

[0035] Step 201: Obtain stereoscopic data containing a target virus sample, the stereoscopic data including an electron density distribution of the target virus sample.

[0036] Step 202: Obtain point cloud data, the point cloud data including a three-dimensional point set representing the surface geometric features of the target virus sample.

[0037] Step 203: Separate virus particles from the point cloud data using an optimized point cloud segmentation algorithm, the optimized point cloud segmentation algorithm combining biological prior knowledge of virus surface protein distribution and sub-nanometer attention mechanism to remove interference of cell debris and non-virus particles, and generating separated virus point cloud.

[0038] ​​​Step 204: separating the virus region from the stereoscopic data using a stereoscopic data segmentation algorithm, wherein the stereoscopic data segmentation algorithm aligns the virus point cloud through cross-modal collaborative segmentation, removes the interference of cell debris and non-virus particles, and generates separated virus stereoscopic data.

[0039] Step 205: processing the virus stereoscopic data using a first neural network to obtain density features of the virus structure.

[0040] Step 206: processing the virus point cloud using a second neural network to obtain geometric features of the virus surface.

[0041] Step 207: fusing the density features and the geometric features using a multi-modal fusion model to generate a unified feature representation.

[0042] Step 208: reconstructing a three-dimensional model of the virus structure using the unified feature representation to obtain a three-dimensional real scene model of the target virus sample.

[0043] As can be seen from the above process, the present application effectively removes the interference of cell debris and non-virus particles by combining the multi-modal fusion of stereoscopic data and point cloud data, using optimized point cloud segmentation algorithms and stereoscopic data segmentation algorithms, and accurately separates virus particles and virus regions. By introducing biological prior knowledge and sub-nanometer attention mechanism, the segmentation accuracy and processing efficiency are improved. The first neural network and the second neural network are used to extract the density features and the geometric features of the virus structure, respectively, and then the multi-modal fusion model is used to integrate the two features to generate a unified feature representation, and accurately reconstruct the three-dimensional model of the virus. This method can accurately obtain the three-dimensional structure of the virus in a complex background, and improves the accuracy of virus three-dimensional reconstruction.

[0044] The steps in the above process and the effects that can be further produced will be described in detail below in conjunction with embodiments. It should be noted that the "first", "second" and the like in the present disclosure do not have the limitations of size, order and quantity, and are only used to distinguish the names, for example, "first neural network" and "second neural network" are used to distinguish two neural networks.

[0045] First, the above step 201, i.e., "obtaining stereoscopic data containing a target virus sample, the stereoscopic data including the electron density distribution of the target virus sample", will be described in detail in conjunction with an embodiment.

[0046] The target virus sample is the target sample for three-dimensional reconstruction in the present application. The tomographic data is a digitized data set stored in the form of a three-dimensional grid, used to represent the electron density distribution of the target virus sample in three-dimensional space. Through the tomographic microscopy technology such as Cryo-EM, the three-dimensional electron density distribution data of the target virus sample is collected to form a data set in the form of a tomographic grid.

[0047] The tomographic data is represented in a three-dimensional array, where each voxel stores a three-dimensional coordinate point and its electron density value, reflecting the internal structure and surface features of the virus, such as high-density protein regions and low-density background regions. Among them, the high-density protein region includes the spike protein, which is a key functional unit for virus infection, and its high charge density and high curvature characteristics make it particularly prominent in the density protein region, and it is also a key position for three-dimensional reconstruction.

[0048] As an implementable way, the process of obtaining tomographic data includes preparing the target virus sample into a thin layer sample suitable for Cryo-EM imaging through cryofixation technology, using Cryo-EM to collect two-dimensional projection images of the sample from multiple angles, and recording electron density information through the interaction of the electron beam and the sample. Apply tomographic reconstruction algorithm to reconstruct two-dimensional projection into three-dimensional electron density map to form tomographic grid data.

[0049] The above step 202, i.e., "obtaining point cloud data, the point cloud data including a three-dimensional point set representing the surface geometric features of the target virus sample", is described in detail below in conjunction with an embodiment.

[0050] Point cloud data is a data structure representing the geometric features of an object's surface in three-dimensional space, composed of a set of discrete three-dimensional coordinate points, each point usually containing spatial position information, possibly with additional attributes such as normal vector, color or intensity. In the context of three-dimensional reconstruction of virus structure, point cloud data specifically refers to a three-dimensional point set representing the surface geometric features of the target virus.

[0051] Point cloud data can be obtained in various ways, for example, using atomic force microscopy, scanning electron microscopy, transmission electron microscopy scanning, auxiliary three-dimensional laser scanning or structured light scanning to collect three-dimensional point cloud data of the virus sample surface. Preferably, point cloud data can be obtained from tomographic data, extracting three-dimensional geometric information of the target virus surface from tomographic data to generate a set of discrete three-dimensional coordinate points, i.e., point cloud data. Specifically, this process analyzes the density distribution of the tomographic data through a surface extraction algorithm, identifies the boundary voxels of the virus surface, and converts them into a data structure in the form of point cloud.

[0052] The step 203, i.e., "separating virus particles from point cloud data using an optimized point cloud segmentation algorithm, which combines biological prior knowledge of virus surface protein distribution and sub-nanometer attention mechanism to remove the interference of cell debris and non-virus particles, and generates separated virus point cloud", will be described in detail below in combination with an embodiment.

[0053] The application separates the point cloud data corresponding to the target virus sample from the point cloud data using an optimized point cloud segmentation algorithm, accurately separates the point set representing the virus particles from the point cloud data, and removes the interference of background such as cell debris and non-virus particles. The core of the algorithm is optimization, which combines the biological characteristics of the virus, such as the distribution of virus surface proteins and sub-nanometer attention mechanism. Biological prior knowledge uses the expected position information of virus surface proteins, such as the distribution of spike proteins obtained by protein structure prediction tools, to help the algorithm identify the characteristic regions of virus particles.

[0054] As an implementable method, the sub-nanometer attention mechanism optimized point cloud segmentation algorithm is implemented by a point cloud neural network based on a sub-nanometer attention mechanism, which enhances the ability to capture the details of the virus surface by limiting the algorithm's attention to local regions within 0.5 to 2 nanometers in the point cloud, thereby improving the segmentation accuracy. At the same time, biological prior knowledge of virus surface protein distribution is used to generate prior weights based on virus symmetry and expected protein positions, and the attention score is adjusted based on the prior weights. Specifically, biological prior knowledge of virus surface protein distribution, such as the expected position of spike proteins obtained by protein structure prediction tools, and the geometric symmetry of the virus, can be used to identify key feature regions of virus particles. Based on this information, the algorithm generates prior weights that reflect the degree of association of each point in the point cloud with the virus center or protein position, with higher weight values indicating that the point is more likely to belong to the surface of the virus particle. Then, the algorithm applies these prior weights to the attention mechanism, adjusting the attention score to enhance attention to key regions of the virus surface while reducing attention to background noise.

[0055] As an implementable way, adjusting the attention score based on the prior weights of virus symmetry and / or expected protein positions includes: calculating the local density of each point in the point cloud based on the spatial distribution of its k-neighbor points, where the local density is determined by the inverse of the average distance between k-neighbor points; generating a density weight based on the local density, enhancing the weight of high-density regions and attenuating the weight of low-density regions through a nonlinear function; generating prior weights based on the biological prior of the virus, which are determined by the distance between the point cloud points and the virus center and the protein center through a Gaussian kernel function based on the symmetry or surface protein distribution of the virus; fusing the density weight and the prior weight to generate a comprehensive weight; and adjusting the score of the attention module in the point cloud segmentation algorithm according to the comprehensive weight.

[0056] Specifically, first, the algorithm analyzes the local density of each point in the point cloud, which is determined by calculating the inverse of the average distance between each point and its k nearest neighbors, reflecting the density of the point cloud, especially adapting to the high-density characteristics of the viral surface protein region. Next, based on the local density, a density weight is generated, using a nonlinear function to enhance the weight of high-density regions while weakening the weight of low-density regions, to effectively suppress noise points in complex biological backgrounds, such as cell debris or non-viral particles. In addition, the algorithm generates prior weights based on the biological prior knowledge of viruses, which are determined according to the symmetry of the virus and the expected distribution of surface proteins, highlighting the importance of the high-density region of the virus by measuring the distance between the point cloud points and the virus center and the protein center. Subsequently, the algorithm fuses the density weight and the prior weight to generate a comprehensive weight, and the fusion process can be realized by weighted combination or other methods. Finally, the comprehensive weight is used to adjust the score of the attention module in the point cloud segmentation algorithm. The attention module calculates the attention score of each point to determine its importance in segmentation, and the comprehensive weight is directly multiplied by the attention score to adjust the contribution of the point.

[0057] Among them, as a preferred implementation method, the generation of prior weights based on the biological prior knowledge of viruses can be realized by the following method:

[0058] First, the input data is point cloud data, which contains a set of three-dimensional points representing the surface of the virus, each point having three-dimensional coordinates. The biological prior information includes two parts: the symmetry information of the virus and the surface protein distribution information. The symmetry information is based on the geometric characteristics of the virus, such as the central symmetry of spherical viruses, where the center point can be obtained by calculating the average coordinates of all points in the point cloud. The surface protein distribution information is derived from protein structure prediction tools such as AlphaFold, which predicts the three-dimensional structure of the viral surface spike protein to obtain a set of protein center coordinates, such as the spatial positions of 10 protein centers. These prior information provides a biological basis for subsequent weight generation.

[0059] Next, the distance weight of each point in the point cloud from the virus center is calculated. The algorithm iterates through each point in the point cloud to calculate its spatial distance from the virus center point. A Gaussian kernel function is used to process these distances to generate a symmetry weight: points closer to the virus center are assigned higher weights because they are more likely to be in the core region of the virus; points farther away have lower weights and may belong to the background or edge region. Specifically, the Gaussian kernel function generates a smoothly decaying weight based on the distance value: when the distance is close to 0, i.e., the point is very close to the virus center, the weight is close to 1; when the distance exceeds 2 nanometers, the weight is close to 0. To adapt to the scale of the virus point cloud, the standard deviation of the Gaussian kernel function is set to 1 nanometer, ensuring that the weight changes smoothly in the range of 0.5 to 2 nanometers.

[0060] Subsequently, the distance weight of each point in the point cloud to the surface protein center is calculated. The algorithm traverses each point in the point cloud again, and calculates the minimum distance of the point to all protein center points, that is, the spatial distance of the point to the nearest protein center. This minimum distance is also converted into a protein distribution weight by a Gaussian kernel function: points closer to the protein center obtain a higher weight, because these points are more likely to be located in high-density regions such as spike proteins; points farther away have a lower weight, because they may correspond to the virus membrane or other non-protein regions.

[0061] Then, the symmetry weight and the protein distribution weight are weighted and fused to generate the final prior weight. The fusion process is realized by a weighted average, and the weight can be assigned according to the actual situation, for example, 30% weight is assigned to the symmetry weight and 70% weight is assigned to the protein distribution weight, reflecting the importance of protein distribution in virus surface segmentation.

[0062] As an implementable way, the present application can also combine the virus surface charge distribution to determine the prior weight by a dynamic modulation method based on charge density. First, the point cloud data and the virus surface charge distribution predicted by bioinformatics tools are obtained to generate the charge weight of the point cloud data. High charge density regions usually correspond to the most functional part of the virus surface, such as the binding site of the spike protein, which plays a key role in the infection mechanism of the virus. The virus surface charge distribution is determined by the charged amino acids of the protein, and high charge density regions are often rich in these amino acids, forming sites with concentrated positive or negative charges. Specifically, point clouds close to high charge density regions obtain higher charge weights, and low charge regions obtain lower charge weights. Then, the algorithm dynamically adjusts the contribution of the symmetry weight and the protein distribution weight with the charge weight as the modulation factor: point cloud points with high charge weight prefer to use the protein distribution weight because the spike protein often has high charge; point cloud points with low charge weight prefer to use the symmetry weight because the virus membrane has more obvious symmetry. In specific implementation, the charge weight acts as a gating mechanism to determine the relative proportion of the two weights, and the gating mechanism determines the fusion proportion of the two weights by a smooth interpolation function based on the value of the charge weight. For example, when the charge weight is greater than 0.7, 90% weight is assigned to the protein distribution weight and 10% weight is assigned to the symmetry weight; when the charge weight is less than 0.7, 20% weight is assigned to the protein distribution weight and 80% weight is assigned to the symmetry weight; when the charge weight is between 0.3 and 0.7, a smooth function is used to dynamically adjust the contribution proportion of the protein distribution weight and the symmetry weight based on the charge weight. Specifically, when the charge weight value is between 0.3 and 0.7, a sigmoid function is used to map the charge weight value to between 0 and 1, outputting a smooth interpolation factor for linear combination of the two weights.

[0063] The step 204, i.e., "separating the virus region from the volume data by using a volume data segmentation algorithm, wherein the volume data segmentation algorithm aligns the virus point cloud through cross-modal collaborative segmentation, removes the interference of cell debris and non-virus particles, and generates separated virus volume data", will be described in detail below in combination with an embodiment.

[0064] This step segments the volume data by using a special volume data segmentation algorithm, effectively removes the interference in the complex biological background through the collaborative action of the point cloud segmentation result, and finally generates volume data containing only the virus region. Cross-modal collaborative segmentation generates a preliminary virus region mask by projecting the point cloud segmentation result into the volume grid space. This mask provides additional spatial constraints for the volume segmentation algorithm, helping the algorithm to more accurately locate the virus region. For example, the virus surface points identified by point cloud segmentation can be mapped to the corresponding voxels of the volume grid, guiding the algorithm to focus on these high-density regions, thereby reducing false positives for background noise.

[0065] Specifically, the virus particles separated by the point cloud segmentation algorithm generate a point cloud subset labeled as virus. Each point in this subset has a classification label indicating that it belongs to a virus particle rather than a background. Next, the volume segmentation algorithm is initialized, which can use a three-dimensional U-Net-based deep learning model. The deep learning model analyzes the density features of each voxel in the volume grid and its spatial neighborhood relationship through multiple layers of convolution and upsampling operations, and predicts whether each voxel belongs to the virus region. The input of the network is the original volume data, and the output is a probability map of the same size as the input, indicating the probability of each voxel belonging to the virus. Then, the point cloud segmentation result is projected into the volume grid space to generate a preliminary virus region mask. Then, the algorithm takes the point cloud projection mask as an additional input channel, inputs it into the network together with the density features, or adjusts the attention area of the network through an attention mechanism. For example, voxels in the region marked as 1 in the mask are given higher attention weights, prompting the network to preferentially learn the features of these regions, while the region marked as 0 is weakened.

[0066] As a preferred implementation, the volume data separation algorithm of the present application can also combine biological prior knowledge to achieve the prior weights generated by the symmetry and surface protein distribution of the virus. These weights are based on the distance of the voxels to the virus center or protein center, and the voxels close to the key region obtain higher weights, and the voxels far away have lower weights.

[0067] The step 205, i.e., "processing the virus volume data using a first neural network to obtain the density features of the virus structure", will be described in detail below in combination with an embodiment.

[0068] Virus volume data is the three-dimensional mesh data processed by volume segmentation algorithm, containing electron density distribution representing only the virus region. These density values reflect the internal structure and surface characteristics of the virus, such as the high density characteristics of the spike protein region and the density changes of the nucleocapsid or membrane region. The task of the first neural network is to analyze this volume mesh and extract density information that can summarize the structural characteristics of the virus.

[0069] The implementation of this step relies on a pre-trained first neural network, which can be obtained through supervised learning on a training set containing labeled virus volume data. The training goal is to enable the network to extract density features that accurately reflect the structure of the virus from the volume data, while ignoring the effects of background noise or segmentation errors.

[0070] The first neural network can be implemented by various neural network types, such as three-dimensional convolutional neural networks, three-dimensional residual networks, etc. Preferably, a three-dimensional U-Net architecture can be used, which is a deep learning model designed specifically for volume data and can efficiently process the voxel information of three-dimensional meshes. The encoder part of the U-Net gradually compresses the volume data and extracts deep features, while the decoder part restores the spatial resolution through upsampling to ensure that the output density features are consistent with the size of the original volume mesh. Techniques such as residual connections or batch normalization can also be added to the network to enhance training stability and feature expression capabilities.

[0071] The above step 206, i.e., "processing the virus point cloud using a second neural network to obtain the geometric features of the virus surface," is described in detail below in conjunction with an embodiment.

[0072] The virus point cloud is a three-dimensional point set processed by a point cloud segmentation algorithm, with each point representing a spatial position on the surface of the virus and reflecting the geometric characteristics of the virus. The task of the second neural network is to analyze this unordered point set and extract high-level features that can summarize the geometric characteristics of the virus surface, such as surface curvature, normal vector, or protein region topology.

[0073] The second neural network usually adopts a deep learning model specifically designed for point cloud data. The second neural network can be implemented by various neural network types, such as dynamic graph convolutional neural networks, multi-view-based convolutional neural networks, etc. Preferably, the second neural network can use PointNet, PointNet++, or PointTransformer. These networks can directly process unordered three-dimensional point sets, generate feature representations describing the geometry of the virus surface through point-by-point feature extraction and global aggregation.

[0074] As an implementable method, the second neural network optimizes the handling of irregular surfaces and dynamic conformations through embedding dynamic topological constraints of viral surface proteins. Irregular surfaces of viral surfaces refer to the complex geometry of viral membranes and protein regions, such as the non-uniform distribution of spike proteins or irregular undulations of membranes. Dynamic conformations reflect the conformational changes of viral surface proteins in natural conditions, such as spike proteins may present different spatial arrangements due to environmental factors. Dynamic topological constraints are based on biological prior knowledge of viruses, such as the spatial distribution and expected topological properties of spike proteins obtained through protein structure prediction tools. These constraints are embedded in the training or inference process of the network as additional guidance information, helping the network more accurately identify and express the complex geometric features of the viral surface.

[0075] In specific implementations, dynamic topological constraints can be implemented by adjusting the loss function or attention mechanism of the network. For example, when training the second neural network, a topological constraint loss term is added to encourage the network to generate geometric features consistent with known protein topological structures, such as ensuring that points in the spike protein region have higher curvature or specific neighborhood distribution. In specific implementations, the algorithm first assigns a topological label to each point in the point cloud based on the prior information of the protein center. If the distance from a point to the nearest protein center is less than a threshold, the point is marked as a spike protein region, which is expected to have higher curvature and dense neighborhood distribution. The topological constraint loss term compares the network's predicted geometric features with these expected properties, such as requiring points in the spike protein region to have higher curvature values or specific neighborhood patterns in the feature space. The training goal is for the network to learn these topological properties and optimize feature extraction for irregular surfaces and dynamic conformations. In the inference stage, the network can prioritize attention on points in the protein region through an attention mechanism, assigning higher weights based on the prior position of the protein center, thereby enhancing feature extraction for these dynamic conformation regions. This optimization enables the network to better handle irregular surfaces and dynamic conformations, avoiding the limitations of traditional point cloud processing algorithms in complex geometry.

[0076] The above step 207, i.e., "fusing density features and geometric features using a multi-modal fusion model to generate a unified feature representation", is described in detail below in conjunction with an embodiment.

[0077] Density features are high-level representations extracted from the separated virion volume data by a first neural network, typically a multi-channel feature map of the same size as the volume grid, describing the high-density regions of the virus internal structure and their spatial distribution patterns; geometric features are features extracted from the separated virus point cloud by a second neural network, typically a high-dimensional feature matrix capturing the curvature, normal vector, or topological structure of protein regions on the virus surface. These two features represent different aspects of the virus: density features focus on internal electron density, and geometric features focus on surface shape, both complementary but different in format and spatial representation.

[0078] The task of the multi-modal fusion model is to integrate these two heterogeneous features into a unified feature representation, to eliminate the differences between modalities and enhance the expressiveness of the features. The model can be implemented based on deep learning architectures such as multi-layer perceptron or attention mechanism. The fusion process first needs to align the spatial representations of density features and geometric features. Since density features are based on volume grids, and geometric features are based on point clouds, the model maps density features to point cloud coordinate space through projection techniques. For example, the algorithm uses trilinear interpolation to assign density feature values of the volume grid to the corresponding positions of the point cloud points, generating a density feature representation of the same number of points as the point cloud, thus achieving spatial alignment of the two.

[0079] The fusion process can be implemented in various ways, such as: using a multi-layer perceptron to concatenate density features and geometric features into a high-dimensional input vector, then extracting comprehensive features through a multi-layer neural network layer by layer to generate a unified feature representation. Another method is to introduce a cross-modal attention mechanism to enhance the feature expression of key regions by calculating the mutual relationship between density features and geometric features.

[0080] As a preferred implementation, the multi-modal fusion model uses a cross-modal attention mechanism to align the density features and the geometric features, and combines the biological prior knowledge of virus symmetry and surface protein distribution to optimize feature extraction; wherein the cross-modal attention mechanism includes: projecting the density features from the volume grid to the point cloud coordinate space, generating a density feature representation aligned with the geometric features through trilinear interpolation; generating a biological prior weight based on the symmetry of the virus and the distribution of the surface protein, the prior weight is obtained by calculating the distance between the point cloud points and the center of the virus or the center of the protein in the Gaussian kernel function; constructing a cross-modal attention module, taking the density features as the query, the geometric features as the key and the value, calculating the attention score, and adjusting the score through the prior weight; taking the geometric features as the query, the density features as the key and the value, calculating the reverse attention score, fusing the prior weight, and generating enhanced geometric features; fusing the enhanced density features and geometric features through a multi-layer perceptron to generate fusion features.

[0081] Specifically, the cross-modal attention mechanism achieves spatial alignment of the two modalities by projecting the density features from the volume grid to the point cloud coordinate space. Due to the different data structures of both the density features and the geometry features, a unified spatial representation is needed before fusion. The algorithm uses a trilinear interpolation method to map the density feature values of the volume grid to the three-dimensional coordinates of the point cloud points. Specifically, for each point cloud point, the algorithm finds the neighboring voxels in the volume grid based on its coordinates, and interpolates the density values of these voxels to generate a density feature representation with the same number of points as the point cloud point. This projection process ensures that the density features are spatially aligned with the geometry features, adapt to the sub-nanometer resolution of the virus, and preserve the fine density information of key regions such as the spike protein.

[0082] Next, the cross-modal attention mechanism generates prior weights using biological prior knowledge of the virus to enhance attention to key regions of the virus. The prior weights are based on the symmetry and surface protein distribution of the virus, and are calculated by a Gaussian kernel function to determine the distance between the point cloud points and the virus center or protein center. The virus center is calculated by the point cloud centroid or the volume data density centroid, reflecting the overall symmetry of the virus, such as the center feature of spherical viruses. The protein center is obtained by a protein structure prediction tool. The algorithm iterates through each point in the point cloud, calculates its distance to the virus center or the nearest protein center, and converts the distance to a weight value through the Gaussian kernel function. Points close to the virus center or protein center have higher weights, as these points are more likely to belong to key regions such as high-density protein regions; points far away have lower weights, which may correspond to background or smooth membrane regions. These prior weights provide biological guidance for the subsequent attention mechanism, highlighting the structural characteristics of the virus.

[0083] Subsequently, the mechanism constructs a cross-modal attention module to calculate multi-head attention scores using density features as queries, geometry features as keys and values, and realizes feature interaction between modalities. In this process, the projected density features serve as queries to inquire relevant information in the geometry features; the geometry features serve as keys and values to provide surface shape details. The attention module calculates the similarity between the density features and the geometry features through a multi-head attention mechanism, generates attention scores, and determines which geometry features are most important for enhancing the density features. For example, the spike protein region may have higher attention scores due to its high density and high curvature characteristics. The algorithm further adjusts these scores using prior weights to enhance feature interaction in high-density protein regions. For example, points close to the protein center have higher attention scores due to high weights, making the network pay more attention to the geometric details of these regions, thereby optimizing the effect of feature fusion.

[0084] To ensure symmetric interaction of information between modalities, the mechanism symmetrically queries the geometric features, density features as keys and values, and calculates the reverse attention score. This process is similar to the previous step, but the roles are reversed: geometric features ask density features for relevant information, and density features provide details of the internal structure. The algorithm also uses a multi-head attention mechanism to calculate the reverse attention score, and adjusts the score through prior weights to enhance the contribution of density information in key areas such as spike proteins to geometric features. For example, the density features of the spike protein region have a stronger enhancing effect on geometric features due to the high weight. This two-way attention mechanism ensures that density features and geometric features complement each other during fusion, generating enhanced geometric features that retain complementary information from both modalities.

[0085] Finally, the mechanism fuses the enhanced density features and enhanced geometric features through a multi-layer perceptron to generate the final fusion features. Specifically, the algorithm concatenates the enhanced density features and geometric features into a high-dimensional input vector, inputs it into a multi-layer perceptron, and extracts comprehensive features layer by layer through a multi-layer neural network. The output fusion features are a unified feature representation, such as a feature matrix, which integrates the internal density distribution of the virus, such as the high-density characteristics of protein regions, and surface geometric information, such as the curvature and topology of the spike protein. This feature representation not only retains the details of both modalities, but also enhances the robustness and completeness of the features through cross-modal interaction.

[0086] As an implementable manner, the cross-modal attention mechanism of the present application can adopt a multi-head attention mechanism. The multi-head attention mechanism is a deep learning technique that enhances feature interaction through parallel computation of multiple attention "heads". Each attention head is responsible for analyzing the relationship between input features from different perspectives, and the calculation of attention scores is based on the concept of Query, Key and Value. Taking cross-modal fusion as an example, the density feature can be used as the query to inquire about the relevant information of the geometric feature; the geometric feature is used as the key and value to provide the details of the surface shape. The attention score measures the similarity between the query and the key, and the higher the score indicates that the corresponding key-value pair contributes more to the query. In the present application, the input density feature and geometric feature are divided into multiple subspaces in the multi-head attention mechanism, and the parallel processing units independently calculate the attention scores in each subspace. Each attention head focuses on a specific aspect of the feature, such as the high-density protein region of the density feature or the spike protein curvature characteristics of the geometric feature. Through parallel computation of multiple heads, the mechanism can analyze the relationship between features from different angles and generate more comprehensive attention scores. Among them, the number of attention heads can be set according to the characteristics of the virus, for example, the cross-modal attention module uses 8 attention heads, with the density feature as the query and the geometric feature as the key and value. The algorithm first divides the density feature and the geometric feature into 8 subspaces, and each head processes 8-dimensional features. The first head focuses on the high correlation between the density feature and the geometric feature in the spike protein region, generating a high attention score, indicating that these points are crucial for fusion. The second head focuses on the correlation between the smooth geometry and the low-density feature in the viral membrane region, generating a lower score. Other heads may capture feature interactions related to protein topology or virus symmetry, respectively. The algorithm adjusts the scores based on the prior weight of the protein center, for example, giving higher weight to points close to the spike protein, so that the scores of these points are further improved in all heads.

[0087] The step 208, i.e., "reconstructing a three-dimensional model of the virus structure based on the unified feature representation to obtain a three-dimensional real scene model of the target virus sample", will be described in detail below in combination with an embodiment.

[0088] The present application reconstructs a three-dimensional model of the virus through a unified feature representation. The process of three-dimensional model reconstruction can convert the unified feature representation into a volume model or a surface mesh model using a deep learning model or a traditional geometric algorithm. For example, a three-dimensional convolutional neural network can be used to process the unified feature representation to generate an initial volume model.

[0089] As an implementable manner, the three-dimensional real scene model of the present application can be implemented in the following way: first, based on the unified feature representation, an initial three-dimensional volume model is generated through a three-dimensional convolutional neural network, and the initial three-dimensional volume model represents the voxel density distribution of the virus structure.

[0090] Secondly, a three-dimensional prior constraint is generated using the prior weights, which calculates the distance weight between the volume grid voxel and the virus center or protein center through the Gaussian kernel function, adapts the sub-nanometer resolution of the virus voxel model, and enhances the voxel expression of the high-density protein region. The virus center is usually calculated by the centroid of the volume data or point cloud, reflecting the overall symmetry of the virus, such as the center characteristics of spherical viruses. The protein center is obtained by protein structure prediction tools, such as the expected position of the spike protein of the new coronavirus. The algorithm assigns a weight to each voxel, and voxels close to the virus center or protein center obtain higher weights because these areas are more likely to correspond to high-density protein structures; voxels far away have lower weights and may belong to the background or low-density region. These weights form a three-dimensional prior constraint, adapting the sub-nanometer resolution.

[0091] Next, based on the three-dimensional prior constraint, the initial three-dimensional volume model is optimized, and the voxel density distribution is adjusted by weighted voxel smoothing, which enhances the surface details of high-density protein regions according to the prior weights. Weighted voxel smoothing is a smoothing algorithm that considers the density value of the voxel and its spatial neighborhood, and adjusts the smoothing strength through the prior weight. Specifically, voxels with high prior weights are subjected to weak smoothing, preserving their surface details and fine structures; voxels with low prior weights are subjected to strong smoothing, suppressing noise contributions.

[0092] Then, surface extraction is performed on the optimized three-dimensional volume model to generate a triangular mesh model of the virus surface, which adapts the sub-nanometer resolution and preserves the geometric features of the virus surface protein. This step generates a triangular mesh model of the virus surface from the volume grid through a surface extraction algorithm. Surface extraction usually uses the Marching Cubes algorithm to analyze the density values of the volume grid, extract an isosurface, and convert it into a three-dimensional mesh composed of triangular patches.

[0093] Finally, according to the triangular model, the final three-dimensional real scene model is generated. The triangular mesh model is a very common model representation form in the field of three-dimensional reconstruction, composed of a group of vertices, edges and triangular patches, used to represent the surface geometry of objects, such as the spike protein and membrane structure of the virus surface.

[0094] Further, before generating the three-dimensional real scene model, the triangular mesh can be further refined, including grid smoothing based on the topological constraints of the viral surface protein and hole filling, to generate the final three-dimensional real scene model. The grid smoothing removes the jagged edges or irregular shapes caused by noise by adjusting the position of the triangular patches, while using the protein topological constraints to ensure that the smoothing process preserves the geometric details of the key regions, such as the protruding structure of the spike protein. The hole filling repairs the grid defects caused by errors in the stereoscopic segmentation or surface extraction, such as missing patches in the viral membrane region. The refined triangular mesh model is smoother and more complete, with both biological authenticity and visual quality, improving the quality of the three-dimensional real scene model of the target virus sample.

[0095] The above method provided by the embodiments of the present application can be applied to various application scenarios, including but not limited to: applied to viral structure analysis to provide high-precision structural data for anti-viral drug development; used for vaccine design to analyze the structural characteristics of viral surface antigens through high-precision three-dimensional models; used for viral variation monitoring to analyze the influence of structural changes on infectivity or pathogenicity by comparing three-dimensional models of different virus strains, and the like.

[0096] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0097] According to another aspect, embodiments of a viral structure three-dimensional reconstruction system are provided. Figure 3 A schematic block diagram of the viral structure three-dimensional reconstruction system according to one embodiment is shown. As shown, the system 300 includes: Figure 3

[0098] A stereoscopic data acquisition unit 301 configured to acquire stereoscopic data containing a target virus sample, the stereoscopic data including an electron density distribution of the target virus sample.

[0099] A point cloud data acquisition unit 302 configured to acquire point cloud data, the point cloud data including a three-dimensional point set representing surface geometric features of the target virus sample.

[0100] A point cloud data segmentation unit 303 configured to separate viral particles from the point cloud data using an optimized point cloud segmentation algorithm that combines biological prior knowledge of viral surface protein distribution and sub-nanometer attention mechanism to remove interference from cell debris and non-viral particles, generating separated viral point cloud.​

[0101] The stereoscopic data segmentation unit 304 is configured to separate the virus region from the stereoscopic data by using a stereoscopic data segmentation algorithm, wherein the stereoscopic data segmentation algorithm aligns the virus point cloud by cross-modal collaborative segmentation, removes the interference of cell debris and non-virus particles, and generates separated virus stereoscopic data.

[0102] The density feature extraction unit 305 is configured to process the virus stereoscopic data by using a first neural network to obtain the density feature of the virus structure.

[0103] The geometric feature extraction unit 306 is configured to process the virus point cloud by using a second neural network to obtain the geometric feature of the virus surface.

[0104] The feature fusion unit 307 is configured to fuse the density feature and the geometric feature by using a multi-modal fusion model to generate a unified feature representation.

[0105] The model reconstruction unit 308 is configured to reconstruct a three-dimensional model of the virus structure based on the unified feature representation to obtain a three-dimensional real scene model of the target virus sample.

[0106] As an implementable manner, the point cloud data segmentation unit 303 can be configured as follows: the optimized point cloud segmentation algorithm is implemented by a point cloud neural network based on a sub-nanometer attention mechanism; the sub-nanometer attention mechanism includes: for the sub-nanometer resolution of the virus point cloud, the local neighborhood range of attention calculation is limited to a radius of 0.5 to 2 nanometers; biological prior knowledge of virus surface protein distribution is used to generate prior weights according to virus symmetry and expected protein positions, and the attention score is adjusted according to the prior weights.

[0107] As an implementable manner, when generating prior weights according to virus symmetry and expected protein positions and adjusting attention scores according to the prior weights, the point cloud data segmentation unit 303 can be configured as follows: the local density of each point in the point cloud data is calculated, wherein the local density is determined by the average distance inverse between the k-neighbor points of each point in the point cloud data; the density weight is generated based on the local density, the weight of the high-density region is enhanced by a nonlinear function, and the weight of the low-density region is attenuated; the prior weight is generated by combining the biological prior of the virus, the prior weight is determined based on the symmetry and surface protein distribution of the virus, and the distance between the point cloud points and the virus center and the protein center is determined by a Gaussian kernel function; the comprehensive weight is generated by fusing the density weight and the prior weight; and the score of the attention module in the point cloud segmentation algorithm is adjusted according to the comprehensive weight.

[0108] As an implementable manner, the first neural network adopts a three-dimensional U-Net based deep learning model.

[0109] As an implementable manner, the second neural network is implemented by using a PointNet model, a PointNet++ model or a PointTransformer model.

[0110] As an implementable manner, the feature fusion unit 307 can be configured to: the multi-modal fusion model aligns the density features and the geometric features by using a cross-modal attention mechanism, and optimizes feature extraction by combining biological prior knowledge of symmetry of viruses and surface protein distribution; wherein the cross-modal attention mechanism comprises: projecting the density features from a voxel grid to a point cloud coordinate space, generating density feature representations aligned with geometric features by trilinear interpolation; generating biological prior weights based on the symmetry of viruses and the distribution of surface proteins, the prior weights are obtained by calculating the distance between the point cloud points and the center of the virus or the center of the protein in the Gaussian kernel function; constructing a cross-modal attention module, taking the density features as queries, the geometric features as keys and values, calculating attention scores, and adjusting the scores by the prior weights; taking the geometric features as queries, the density features as keys and values, calculating reverse attention scores, and generating enhanced geometric features by fusing the prior weights; and generating fused features by fusing the enhanced density features and the geometric features through a multi-layer perception.

[0111] As an implementable manner, the feature fusion unit 307 can be configured to: obtain attention scores by parallel calculation of 4 or more attention heads; wherein the attention heads comprise: an attention head for analyzing the correlation degree of density features and geometric features in the spike protein region, and an attention head for analyzing the correlation degree of smooth geometry and low density features in the viral membrane region.

[0112] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, for the system or system embodiments, since it is basically similar to the method embodiments, it is described more simply, and the related parts can be referred to the part of the method embodiments. The above-described system and system embodiments are only illustrative, wherein the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0113] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant national and regional laws, regulations and standards, and provide corresponding operation portal for user to choose authorization or refusal.

[0114] In addition, the embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method in any one of the preceding method embodiments.

[0115] and an electronic device comprising:

[0116] one or more processors; and a memory associated with the one or more processors, the memory being configured to store program instructions that, when executed by the one or more processors, perform the steps of the method in any one of the preceding method embodiments.

[0117] wherein, Figure 4 The exemplary architecture of the electronic device is shown, which can specifically include a processor 410, a video display adapter 411, a disk drive 412, an input / output interface 413, a network interface 414, and a memory 420. The above-mentioned processor 410, video display adapter 411, disk drive 412, input / output interface 413, network interface 414, and memory 420 can be connected by a communication bus 430.

[0118] The processor 410 can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the present application.

[0119] The memory 420 can be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 420 can store an operating system 421 for controlling the operation of the electronic device 400, a basic input / output system (BIOS) 422 for controlling the low-level operation of the electronic device 400. In addition, a web browser 423, a data storage management system 424, and a virus structure three-dimensional reconstruction system 425, etc. can also be stored. The virus structure three-dimensional reconstruction system 425 described above can be an application program for specifically implementing the operations of the above steps in the embodiments of the present application. In summary, when the technical solutions provided in the present application are implemented by software or firmware, the relevant program codes are stored in the memory 420 and executed by the processor 410.

[0120] The input / output interface 413 is configured to connect an input / output module to realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0121] The network interface 414 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0122] The bus 430 includes a path for transmitting information between various components (such as the processor 410, the video display adapter 411, the disk drive 412, the input / output interface 413, the network interface 414, and the memory 420) of the device.

[0123] It should be noted that although the above device only shows the processor 410, the video display adapter 411, the disk drive 412, the input / output interface 413, the network interface 414, the memory 420, and the bus 430, etc., in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary for implementing the solutions of the present application, and does not have to contain all the components shown in the figure.

[0124] Those skilled in the art can clearly understand the application by the description of the above embodiments that the application can be implemented by means of software plus necessary universal hardware platforms. Based on such an understanding, the technical solutions of the application can be embodied in the form of a computer program product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the application.

[0125] The technical solutions provided by the application are described in detail above, and the principles and implementation manners of the application are described by using specific examples. The above description of the embodiments is only used to help understand the methods and core ideas of the application. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the ideas of the application. In conclusion, the content of the specification should not be understood as a limitation of the application.

Claims

1. A method for three-dimensional reconstruction of virus structure, characterized in that, The method comprises: acquiring stereoscopic data containing a target virus sample, the stereoscopic data including an electron density distribution of the target virus sample; acquiring point cloud data, the point cloud data including a three-dimensional point set representing surface geometric features of the target virus sample; isolating virus particles from the point cloud data using an optimized point cloud segmentation algorithm, the optimized point cloud segmentation algorithm combining biological prior knowledge of virus surface protein distribution and a sub-nanometer attention mechanism to remove interference of cell debris and non-virus particles, generating isolated virus point cloud; isolating a virus region from the stereoscopic data using a stereoscopic data segmentation algorithm, wherein the stereoscopic data segmentation algorithm aligns the virus point cloud through cross-modal collaborative segmentation, removes interference of cell debris and non-virus particles, and generates isolated virus stereoscopic data; processing the virus stereoscopic data using a first neural network to obtain density features of the virus structure; processing the virus point cloud using a second neural network to obtain geometric features of the virus surface; fusing the density features and the geometric features using a multi-modal fusion model to generate unified feature representation; reconstructing a three-dimensional model of the virus structure based on the unified feature representation to obtain a three-dimensional real scene model of the target virus sample.

2. The method of claim 1, wherein, The optimized point cloud segmentation algorithm is implemented by a point cloud neural network based on a sub-nanometer attention mechanism; wherein the sub-nanometer attention mechanism comprises: for the sub-nanometer resolution of the virus point cloud, limiting the local neighborhood range of attention calculation to a radius of 0.5 to 2 nanometers; using biological prior knowledge of virus surface protein distribution, generating prior weights according to virus symmetry and expected protein positions, and adjusting attention scores according to the prior weights.

3. The method of claim 2, wherein, The generation of prior weights according to virus symmetry and expected protein positions and the adjustment of attention scores according to the prior weights comprise: calculating the local density of each point in the point cloud data, wherein the local density is determined by the average distance inverse between the k-nearest neighbors of each point in the point cloud data; generating density weights based on the local density, enhancing the weights of high-density regions and attenuating the weights of low-density regions through a nonlinear function; combining biological priors to generate prior weights, the prior weights being determined based on the symmetry and surface protein distribution of the virus, and the distance between the point cloud points and the virus center and the protein center being determined through a Gaussian kernel function; fusing the density weights and the prior weights to generate comprehensive weights; adjusting the scores of the attention module in the point cloud segmentation algorithm according to the comprehensive weights.

4. The method of claim 1, wherein, The first neural network adopts a three-dimensional U-Net based deep learning model.

5. The method of claim 1, wherein, The second neural network adopts a PointNet model, a PointNet++ model, or a PointTransformer model.

6. The method of claim 1, wherein, The multi-modal fusion model aligns the density features and the geometric features using a cross-modal attention mechanism, and optimizes feature extraction combining biological prior knowledge of virus symmetry and surface protein distribution; wherein the cross-modal attention mechanism comprises: projecting the density features from the volume grid to the point cloud coordinate space, generating density feature representation aligned with geometry features by trilinear interpolation; generating biological prior weights based on symmetry and surface protein distribution of the virus, the prior weights being obtained by calculating the distance between the point cloud points and the center of the virus or the center of the protein in a Gaussian kernel function; constructing a cross-modal attention module to take the density features as queries, the geometry features as keys and values, calculate attention scores, and adjust the scores by the prior weights; taking the geometry features as queries, the density features as keys and values, calculating reverse attention scores, and generating enhanced geometry features by fusing the prior weights; fusing the enhanced density features and geometry features by a multi-layer perception to generate fused features.

7. The method of claim 6, wherein, The calculating of the attention scores comprises: obtaining the attention scores by parallel calculation of 4 or more attention heads; wherein the attention heads comprise: an attention head for analyzing the correlation between the density features and the geometry features of the spike protein region, and an attention head for analyzing the correlation between the smooth geometry and low density features of the virus membrane region.

8. A virus structure three-dimensional reconstruction system, characterized by comprising: a virus structure three-dimensional reconstruction program; and a virus structure three-dimensional reconstruction database. The system comprises: a volume data acquisition unit configured to acquire volume data containing a target virus sample, the volume data comprising an electron density distribution of the target virus sample; a point cloud data acquisition unit configured to acquire point cloud data, the point cloud data comprising a set of three-dimensional points representing surface geometry features of the target virus sample; a point cloud data segmentation unit configured to separate virus particles from the point cloud data using an optimized point cloud segmentation algorithm that combines biological prior knowledge of virus surface protein distribution and sub-nanometer attention mechanisms to remove interference from cell debris and non-virus particles, generating separated virus point cloud data; a volume data segmentation unit configured to separate virus regions from the volume data using a volume data segmentation algorithm that aligns the virus point cloud through cross-modal collaborative segmentation to remove interference from cell debris and non-virus particles, generating separated virus volume data; a density feature extraction unit configured to process the virus volume data using a first neural network to obtain density features of the virus structure; a geometry feature extraction unit configured to process the virus point cloud using a second neural network to obtain geometry features of the virus surface; a feature fusion unit configured to fuse the density features and the geometry features using a multi-modal fusion model to generate a unified feature representation; a model reconstruction unit configured to reconstruct a three-dimensional model of the virus structure based on the unified feature representation, obtaining a three-dimensional real scene model of the target virus sample.

9. An electronic device, comprising: comprise: one or more processors; and a memory associated with the one or more processors, the memory being used to store program instructions, the program instructions being read and executed by the one or more processors to perform the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images