Method for generating a protein image

Abstract

This application provides a protein image generation method, applied in the fields of computational biology image processing and artificial intelligence, comprising: when the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image, obtaining protein semantic features based on reference protein features and reference contour features used as spatial context conditions to guide the reference protein features across modalities; when the protein semantic data includes a reference protein sequence, performing feature extraction on the reference protein sequence to obtain protein semantic features; fusing the protein semantic features and the target contour features of the target cell contour image used as spatial boundary constraints to guide the protein semantic features at the pixel level to obtain target protein fusion features; and decoding the target protein fusion features to generate the target protein image.

Description

Technical Field

[0001] This application relates to the fields of computational biology image processing and artificial intelligence, and in particular to a method for generating protein images. Background Technology

[0002] The spatial distribution of proteins within cells, also known as protein subcellular localization or distribution, is related to factors such as protein biological function, molecular interactions, cell state, and cell tissue morphology. Proteins may be distributed in the nucleus, cytoplasm, cell membrane, mitochondria, endoplasmic reticulum, Golgi apparatus, or other subcellular structures. The same protein may exhibit different spatial distribution patterns under different cell states, disease environments, or drug interventions.

[0003] Protein image quality is of great significance for understanding protein function, analyzing disease-related molecular mechanisms, screening drug targets, or conducting cell phenotype studies. Summary of the Invention

[0004] In view of the above problems, embodiments of this application provide a protein image generation method, apparatus, electronic device, storage medium, and program product.

[0005] According to one aspect of the embodiments of this application, a protein image generation method is provided, comprising: extracting features from a target cell contour image corresponding to a protein image to be generated, to obtain target contour features; when the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image, obtaining protein semantic features based on reference protein features of the reference protein image and reference contour features of the reference cell contour image, wherein the reference contour features are used as spatial context conditions to guide the reference protein features across modalities; when the protein semantic data includes a reference protein sequence, extracting features from the reference protein sequence to obtain protein semantic features; fusing the protein semantic features and the target contour features to obtain target protein fusion features, wherein the target contour features are used as spatial boundary constraints to guide pixel-level reconstruction of the protein semantic features; and decoding the target protein fusion features to generate a target protein image.

[0006] In some implementations, protein semantic features are obtained based on reference protein features of a reference protein image and reference contour features of a reference cell contour image. This includes: performing local affine modulation on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image to achieve cross-modal spatial guidance of the reference protein features by the reference contour features, thereby obtaining reference local modulation features; using a cross-attention strategy to process the reference protein features used as queries and the intermediate reference contour features corresponding to the reference contour features as keys and values, thereby obtaining reference global fusion features. The reference global fusion features are used to capture the long-range dependencies and relative distribution patterns of protein features in the global field of view under the spatial constraints of the intermediate reference contour features; and obtaining protein semantic features based on the reference local modulation features and the reference global fusion features.

[0007] In some implementations, protein semantic features are obtained based on reference local modulation features and reference global fusion features. This includes: concatenating the reference local modulation features and reference global fusion features along the channel dimension to obtain reference concatenated features; processing the reference concatenated features using a gated perceptron strategy to obtain reference local weight vectors and reference global weight vectors; and processing the reference local modulation features and reference global fusion features using the reference local weight vectors and reference global weight vectors to obtain protein semantic features. The reference local weight vector is used to control the contribution distribution of the reference local modulation features to the protein semantic features, and the reference global weight vector is used to control the contribution distribution of the reference global fusion features to the protein semantic features.

[0008] In some implementations, local affine modulation is performed on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image to achieve cross-modal spatial guidance of the reference protein features by the reference contour features, resulting in reference local modulation features. This includes: projecting the reference contour features of the reference cell contour image to obtain intermediate reference contour features; performing linear mapping on the intermediate reference contour features to obtain affine transformation parameters, wherein the affine transformation parameters include scaling parameters and translation parameters. The scaling parameters are used to control the local enhancement or suppression of the reference protein features in the amplitude dimension, and the translation parameters are used to control the offset of the reference protein features in the activation baseline dimension; and performing affine transformation on the reference protein features of the reference protein image using the affine transformation parameters to obtain reference local modulation features.

[0009] In some implementations, a gated perceptron strategy is used to process the reference stitching features to obtain a reference local weight vector and a reference global weight vector. This includes: using a gated perceptron strategy to perform a nonlinear transformation on the reference stitching features to obtain a weight vector, wherein the weight vector is used to control the contribution distribution of the reference local modulation features and the reference global fusion features in feature fusion; and dividing the weight vector along the channel dimension to obtain the reference local weight vector and the reference global weight vector.

[0010] In some implementations, reference local modulation features and reference global fusion features are processed using reference local weight vectors and reference global weight vectors to obtain protein semantic features. This includes: performing linear projections on the reference local modulation features and reference global fusion features respectively to obtain reference local projection features and reference global projection features; and using reference local weight vectors and reference global weight vectors to perform feature fusion on the reference local projection features and reference global projection features to obtain protein semantic features.

[0011] In some implementations, protein semantic features are obtained by processing reference protein features from a reference protein image and reference contour features from a reference cell contour image using a protein feature extraction model. The protein feature extraction model is trained using a sample protein image and a corresponding sample cell contour image to train a first deep learning model. This first deep learning model includes multiple cascaded feature cross-modules, including a protein feature extraction unit, a contour feature extraction unit, and a contour conditional fusion unit. The protein feature extraction unit extracts features from the sample protein image to obtain sample protein features. The contour feature extraction unit extracts features from the sample cell contour image to obtain sample contour features. The contour conditional fusion module uses the sample contour features as spatial context conditions to guide the sample protein features across modalities, resulting in sample protein semantic features.

[0012] In some implementations, the protein feature extraction model is a trained student model. The trained student model is obtained by training the first deep learning model, which includes the teacher model and the student model, using sample protein images and sample cell contour images corresponding to the sample protein images. Multiple cascaded feature cross modules include multiple cascaded feature cross modules corresponding to the teacher model and multiple cascaded feature cross modules corresponding to the student model.

[0013] In some implementations, feature fusion is performed on protein semantic features and target contour features to obtain target protein fusion features. This includes: using a bridging module in a protein image generation model to perform feature fusion on protein semantic features and target contour features to obtain target protein fusion features; wherein, decoding the target protein fusion features to generate a target protein image includes: using a decoder in a protein image generation model to decode the target protein fusion features to obtain a target protein image; wherein, the protein image generation model is obtained by training a second deep learning model using a sample reference protein image and a sample reference cell contour image corresponding to the sample reference protein image, as well as a sample target protein image, a sample target cell contour image corresponding to the sample target protein image, and a sample target protein sequence corresponding to the sample target protein image.

[0014] In some implementations, the protein image generation model is obtained by training a second deep learning model based on a sample target protein image, a protein image generated from a first sample, and a protein image generated from a second sample. The protein image generated from the first sample is obtained using a sample reference protein image, a sample reference cell contour image, and a sample target cell contour image. The protein image generated from the second sample is obtained using a sample reference protein image, a sample reference cell contour image, and a sample target protein sequence.

[0015] According to another aspect of the embodiments of this application, a protein image generation apparatus is provided, comprising: a first obtaining module, configured to extract features from a target cell contour image corresponding to a protein image to be generated, to obtain target contour features; a second obtaining module, configured to, when the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image, obtain protein semantic features based on reference protein features of the reference protein image and reference contour features of the reference cell contour image, wherein the reference contour features are used as spatial context conditions to guide the reference protein features across modalities; a third obtaining module, configured to, when the protein semantic data includes a reference protein sequence, extract features from the reference protein sequence to obtain protein semantic features; a fourth obtaining module, configured to fuse the protein semantic features and the target contour features to obtain target protein fusion features, wherein the target contour features are used as spatial boundary constraints to guide pixel-level reconstruction of the protein semantic features; and a generation module, configured to decode the target protein fusion features to generate a target protein image.

[0016] According to another aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0017] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, which stores computer-executable instructions that, when executed, are used to implement the method described above.

[0018] According to another aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer executable instructions, which, when executed, are used to implement the method described above.

[0019] According to embodiments of this application, target contour features are obtained by extracting features from the target cell contour image. These features are then used as spatial boundary constraints to guide pixel-level reconstruction of protein semantic features. This ensures that the protein semantic features are confined to the internal region defined by the target cell contour, and also improves the fineness of spatial distribution to the pixel level. This achieves alignment between spatial structure and functional semantics while suppressing the cross-modal semantic gap. Since the target contour features express the boundaries, nuclear region, cytoplasmic extent, and spatial morphological relationships of the target cell, fusing the protein semantic features with the target contour features before generation allows the subsequently decoded protein image to be constrained by the spatial structure of the target cell. This reduces the possibility of the generated signal crossing the cell region or being inconsistent with the cell boundary, thus improving the quality of the generated protein image. Furthermore, the protein semantic data may include a reference protein image and its reference cell contour image, or it may include a reference protein sequence. When a reference protein image exists, protein semantic features can be obtained based on the reference cell contour image and the reference protein image. Reference contour features from reference cell contour images can serve as spatial context conditions to guide reference protein features from reference protein images across modalities. This establishes a correspondence between contour morphology and protein distribution at the feature level, binding the protein distribution provided by the reference protein image to its spatial structure. This yields protein semantic features that can include transferable patterns and spatial context information, improving the accuracy of protein semantic features. In the absence of a reference cell contour image, protein semantic features can be obtained based on reference protein sequences. Since protein semantic features from different sources can be fused with target contour features and incorporated into the decoding process, this approach can adapt to protein image generation tasks under various input conditions. Attached Figure Description

[0020] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0021] Figure 1 An exemplary system architecture of a protein image generation method according to an embodiment of this application is illustrated schematically;

[0022] Figure 2 A flowchart illustrating a protein image generation method according to an embodiment of this application is shown schematically.

[0023] Figure 3 The illustration shows a schematic diagram of the principle of a protein image generation method according to an embodiment of this application;

[0024] Figure 4 The illustration shows a schematic diagram of the principle of obtaining protein semantic features according to an embodiment of this application;

[0025] Figure 5 The diagram illustrates the training process of a protein feature extraction model according to an embodiment of this application.

[0026] Figure 6A This illustration schematically shows a diagram of protein image generation based on a reference protein image, a reference cell contour image, and a target cell contour image according to an embodiment of this application;

[0027] Figure 6B This illustration schematically depicts a protein image generation process based on a reference protein sequence and a target cell contour image, according to embodiments of this application; and

[0028] Figure 7 The illustration shows a schematic diagram comparing the target protein image generation method according to an embodiment of this application with other methods.

[0029] Specific implementation method

[0030] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0031] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0033] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0034] With the development of artificial intelligence technology, related technologies utilize protein image generation models to obtain protein images.

[0035] Research has revealed that protein image generation models are prone to boundary distortion or signal overflow in complex cell morphologies. Real cells possess highly complex and irregular geometric shapes. Protein image generation models use cell contour images as additional channels, masks, or simple conditional inputs, and then perform channel stitching with protein images. These methods fail to establish a deep coupling relationship between the spatial structure of the cell contour and the local localization of proteins within the model. This results in generated protein images that may cross cell boundaries or exhibit blurring, misalignment, or distortion in areas such as cell edges, elongated protrusions, and nuclear boundaries.

[0036] Protein image generation models tend to separate protein feature extraction and protein image generation into two relatively independent tasks. The protein features obtained by the protein feature extraction model may not be consistent with the latent space generated by the protein image generation model, leading to semantic shifts during feature transfer.

[0037] Furthermore, for many newly discovered proteins that have not yet been imaged under a microscope (e.g., novel artificially designed proteins or rare mutants), protein image generation models also need to be developed through experiments from scratch.

[0038] Therefore, a key challenge is developing a protein image generation scheme with spatial physical constraints, ensuring that the generated protein images can serve as a reliable substitute or prior reference for wet experiments, thereby reducing resource waste caused by blindly conducting wet experiments. Furthermore, in the absence of protein images, overcoming cross-modal barriers to obtain protein images that meet fidelity requirements is a problem that needs to be solved. Protein images can be used to study the macroscopic spatial distribution patterns of proteins within cell outlines.

[0039] Therefore, this application proposes to decouple the low-level features of protein images from cell contour images, use the contour features of cell contour images as spatial constraints to guide protein features, and based on a cross-modal dual-path protein image generation method, achieves compatibility with different input conditions such as reference protein images and reference protein sequences, and can generate protein images with reasonable space, consistent boundaries and conforming to protein distribution properties under the constraints of reference cell contour images and target cell contours.

[0040] Further research revealed that because the protein image generation model is trained using protein images of a single type of sample, its versatility is poor. Consequently, the model needs to be retrained or fine-tuned for other types of proteins, resulting in high computational costs and complex model deployment, which limits its application in multi-target high-throughput screening.

[0041] Therefore, embodiments of this application propose a method for training a protein feature extraction model based on various types of sample protein images and sample cell contour images corresponding to the sample protein images.

[0042] For ease of understanding, some terms involved in the embodiments of this application will be explained below.

[0043] Protein images, acquired through fluorescence microscopy, are grayscale or pseudo-color images used to visualize the spatial distribution and expression intensity of proteins within cells. The pixel values ​​of a protein image characterize the relative concentration or expression intensity of the protein at that location; higher pixel values ​​indicate greater protein enrichment. Protein images may also include corresponding cell outline images.

[0044] Cell contour images, acquired through methods such as fluorescent labeling, bright-field imaging, or dye staining, are used to characterize the boundaries and spatial morphology of cellular or subcellular structures. Cell contour images can depict the geometry, size, location, or topological relationships of subcellular structures such as the cell membrane, nucleus, mitochondria, or endoplasmic reticulum. The data format of cell contour images can include single-channel grayscale images, where highlighted areas represent the interior or boundary of a cell or subcellular structure, and external areas represent the background. For example, the cell membrane contour can be circular, elliptical, or an irregular polygon, used to define the cell boundary and limit the range of protein expression. The nucleus contour is approximately circular or elliptical, used to distinguish the location of intranuclear and extranuclear proteins. The mitochondrial contour can be elongated, net-like, or spherical, guiding the spatial distribution of mitochondrial proteins within its region. Cell contour images can characterize the spatial structure of the cell in which the protein image is located.

[0045] A protein sequence refers to the primary structure of a linear polypeptide chain composed of multiple standard amino acids linked by peptide bonds. It is related to the protein's folding mechanism, three-dimensional conformation, and functional location within the cell. An amino acid sequence can represent the chemical composition and data expression of a protein sequence. Protein sequences can also correspond to cell contour images. Cell contour images can characterize the spatial structure of the cell containing the protein sequence.

[0046] In this embodiment, based on their function and stage, protein images can be categorized into reference protein images and target protein images used in the model application stage, and sample protein images, sample reference protein images, sample target protein images, first sample generated protein images, and second sample generated protein images used in the model training stage. Cell contour images can be categorized into reference cell contour images and target cell contour images used in the model application stage, and sample cell contour images, sample reference cell contour images, and sample target cell contour images used in the model training stage. Protein sequences can be categorized into reference protein sequences used in the model application stage, and sample reference protein sequences used in the model training stage. The target protein image can be the protein image expected to be generated in the model application stage. The first sample generated protein image and the second sample generated protein image can be protein images generated in the model training stage.

[0047] A reference protein image corresponds to a reference cell contour image. A reference protein sequence corresponds to a reference cell contour image. A target protein image corresponds to a target cell contour image. A sample protein image corresponds to a sample cell contour image. A sample reference protein image corresponds to a sample reference cell contour image. A sample target protein image corresponds to a sample target cell contour image. A sample reference protein sequence corresponds to a sample reference cell contour image. The sample reference protein image, sample reference cell contour image, sample target cell contour image, and sample target protein image can all correspond to the protein image generated from the first sample. The sample reference protein sequence and sample target cell contour image can all correspond to the protein image generated from the second sample.

[0048] A reference protein image can refer to a protein image associated with the target protein image to be generated, used to provide the protein's distribution properties. A reference protein sequence can refer to a protein sequence associated with the protein image to be generated.

[0049] Cell contour images (e.g., reference cell contour images or sample reference cell contour images) can serve as spatial structure priors, providing morphological templates for protein images (e.g., reference protein images corresponding to the reference cell contour images or sample reference protein images corresponding to the sample reference cell contour images), defining the possible regions where proteins may appear. Furthermore, cell contour images (e.g., target cell contour images or sample target cell contour images) can serve as spatial boundary constraints, guiding pixel-level reconstruction of protein semantic features (e.g., protein semantic features or sample protein semantic features).

[0050] Protein semantic data can refer to data used to provide at least one of the following: protein identity, distribution attributes, or spatial localization tendency. Protein semantic data may include a reference protein image and a reference cell outline image corresponding to the reference protein image, or it may include a reference protein sequence.

[0051] Protein semantic features refer to features obtained from protein semantic data, used to characterize a protein's identity, distribution attributes, subcellular localization tendency, fluorescence signal morphology, or other information related to the protein's spatial distribution. Protein semantic features can be derived from a reference protein image and its corresponding reference cell contour image, or from a reference protein sequence.

[0052] A bridging module can refer to a network module used to fuse protein semantic features and target contour features. A bridging module can include linear projection layers, convolutional layers, attention layers, normalization layers, residual connections, spatial modulation layers, conditional fusion layers, or combinations thereof. The bridging module can be used to map protein semantic features from different sources to a generative latent space adapted to the target contour features, and establish a correspondence between protein distribution attributes and the spatial structure of the target cell in this generative latent space.

[0053] Protein language models can encode discrete amino acid symbols into high-dimensional continuous feature vectors that include at least one of evolutionary information, physicochemical properties, or structural predispositions. For example, protein language models can include ESM-2 (Evolutionary Scale Modeling 2).

[0054] The protein image generation method provided in this application embodiment will be described below with reference to the accompanying drawings. The protein image generation method may include a protein semantic feature extraction process and a protein image generation process. The protein semantic feature extraction process may include a process based on a reference protein image and a reference cell contour image corresponding to the reference protein image, or a process based on a reference protein sequence. To achieve protein semantic feature extraction and protein image generation, this application embodiment also provides a protein feature extraction model and a protein image generation model, and provides training processes for the protein feature extraction model and the protein image generation model.

[0055] Figure 1 An exemplary system architecture of a protein image generation method according to an embodiment of this application is illustrated.

[0056] like Figure 1 As shown, the business system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0057] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, or third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, or third terminal device 103.

[0058] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and supporting web browsing, such as smartphones, tablets, laptops, or desktop computers. The server 105 can be a server that provides various services.

[0059] It should be noted that the protein image generation method provided in this application embodiment can be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or it can be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0060] Alternatively, the protein image generation method provided in this application embodiment can also be executed by server 105, or by a server or server cluster that is different from server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 or server 105.

[0061] Furthermore, the training methods for the protein feature extraction model and the protein image generation model described in the embodiments of this application may be the same as or different from the execution subject of the protein image generation method described in the embodiments of this application, and the embodiments of this application do not limit this.

[0062] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0063] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0064] Figure 2 A flowchart illustrating a protein image generation method according to an embodiment of this application is shown schematically.

[0065] like Figure 2 As shown, the protein image generation method may include operations S210~S220.

[0066] In operation S210, feature extraction is performed on the target cell contour image corresponding to the protein image to be generated to obtain the target contour features.

[0067] In operation S220, when the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image, the protein semantic features are obtained based on the reference protein features of the reference protein image and the reference contour features of the reference cell contour image.

[0068] In operation S230, when the protein semantic data includes a reference protein sequence, feature extraction is performed on the reference protein sequence to obtain protein semantic features.

[0069] In operation S240, feature fusion is performed on the semantic features of the protein and the contour features of the target to obtain the fused features of the target protein.

[0070] In operation S250, the fusion features of the target protein are decoded to generate an image of the target protein.

[0071] The target cell contour image can be obtained through a microscopic image acquisition device or provided by an existing cell image database. In one implementation, a microscopic image containing cell contour information can be acquired first, and then the microscopic image can be preprocessed to obtain the target cell contour image. Preprocessing may include size normalization, background subtraction, brightness normalization, cell region segmentation, cell nucleus region segmentation, edge extraction, mask generation, hole filling, connected component filtering, or image registration. After preprocessing, the target cell contour image can be input into the model in the form of a two-dimensional matrix, a multi-channel matrix, or an image tensor.

[0072] Target contour features can be used as spatial boundary constraints to guide pixel-level reconstruction of protein semantic features. These features can be extracted by a contour encoder. The contour encoder can include convolutional neural networks, residual networks, U-shaped network encoders, visual attention networks, image patch embedding networks, Transformer encoders, or combinations thereof. Target contour features can be feature maps with spatial dimensions or feature sequences composed of multiple spatial labels. These features are used to provide spatial structural information of the target cells during the generation process, rather than merely serving as a binary mask for post-processing at the output stage.

[0073] Reference protein images provide existing spatial distribution information of the protein to be generated. Reference cell contour images provide the cellular morphological background of this spatial distribution. Reference protein images and reference cell contour images can be paired data. Feature extraction from the reference protein image yields reference protein features, and feature extraction from the reference cell contour image yields reference contour features. Reference protein features are used to characterize point-like, linear, granular, membrane-like, intranuclear, perinuclear, cytoplasmic, or other distribution morphologies of proteins in the image. Reference contour features are used to characterize the spatial regions, boundaries, nuclear regions, cytoplasmic regions, and local morphology of reference cells. Reference contour features are used as spatial context conditions to guide reference protein features across modalities.

[0074] Having obtained reference protein features and reference contour features, the reference contour features can be used as spatial context conditions for cross-modal guidance of the reference protein features. Cross-modal guidance can include local modulation, global attention, gated fusion, or a combination thereof. Through cross-modal guidance, protein semantic features no longer merely represent the grayscale or texture information of the reference protein image, but further include the relationship between the protein distribution and the spatial structure of the cell contour.

[0075] A reference protein sequence can be input into a sequence encoder to obtain semantic features of the protein from which the sequence originates. The sequence encoder can be a protein language model, a recurrent neural network, a convolutional sequence encoder, a Transformer sequence encoder, or other models capable of processing amino acid sequences. The sequence encoder can output a global sequence vector, a residue-level vector, sequence marker features, or a pooled sequence representation. In some implementations, the sequence features output by the sequence encoder can be subjected to linear transformations, dimensionality transformations, normalization, or projection processing, enabling them to be fused with the target contour features in subsequent bridging modules.

[0076] Therefore, when a reference protein image is available, protein semantic features can be obtained through the reference image path; when a reference protein image is unavailable but a reference protein sequence is available, protein semantic features can be obtained through the sequence path. This setting makes protein semantic features a unified expression for different input modalities entering the image generation process.

[0077] Feature fusion can be performed by the bridging module. The bridging module first projects the protein semantic features, converting them into a dimension or spatial scale that matches the target contour features. Then, it conditionally fuses the projected protein semantic features with the target contour features, reconstructing the protein distribution attributes represented by the protein semantic features within the spatial range defined by the target cell contour. Fusion methods can include splicing fusion, weighted fusion, cross-attention fusion, conditional normalization fusion, spatial modulation fusion, residual fusion, or multi-scale fusion.

[0078] Target contour features are used as spatial boundary constraints to guide pixel-level reconstruction of protein semantic features. This means that target contour features not only provide global cell identity information, but also participate in the generation constraints of each spatial location or each feature location, enabling the model to consider target cell boundaries, nuclear regions, cytoplasmic regions, and other local morphological structures when recovering pixel-level protein images.

[0079] Decoding can be performed by a decoder. The decoder may include upsampling layers, deconvolution layers, convolutional layers, residual blocks, attention layers, normalization layers, and an output layer. The decoder progressively recovers the spatial resolution of the target protein fusion features and outputs a target protein image corresponding to the target cell contour image. The output target protein image can be a single-channel image or a multi-channel image; it can represent predicted fluorescence intensity or the relative distribution intensity of the protein at different locations within the target cell.

[0080] According to embodiments of this application, target contour features are obtained by extracting features from the target cell contour image. These features are then used as spatial boundary constraints to guide pixel-level reconstruction of protein semantic features. This ensures that the protein semantic features are confined to the internal region defined by the target cell contour, and also improves the fineness of spatial distribution to the pixel level. This achieves alignment between spatial structure and functional semantics while suppressing the cross-modal semantic gap. Since the target contour features express the boundaries, nuclear region, cytoplasmic extent, and spatial morphological relationships of the target cell, fusing the protein semantic features with the target contour features before generation allows the subsequently decoded protein image to be constrained by the spatial structure of the target cell. This reduces the possibility of the generated signal crossing the cell region or being inconsistent with the cell boundary, thus improving the quality of the generated protein image. Furthermore, the protein semantic data may include a reference protein image and its reference cell contour image, or it may include a reference protein sequence. When a reference protein image exists, protein semantic features can be obtained based on the reference cell contour image and the reference protein image. Reference contour features from reference cell contour images can serve as spatial context conditions to guide reference protein features from reference protein images across modalities. This establishes a correspondence between contour morphology and protein distribution at the feature level, binding the protein distribution provided by the reference protein image to its spatial structure. This yields protein semantic features that can include transferable patterns and spatial context information, improving the accuracy of protein semantic features. In the absence of a reference cell contour image, protein semantic features can be obtained based on reference protein sequences. Since protein semantic features from different sources can be fused with target contour features and incorporated into the decoding process, this approach can adapt to protein image generation tasks under various input conditions.

[0081] Figure 3 The illustration shows a schematic diagram of the principle of a protein image generation method according to an embodiment of this application.

[0082] like Figure 3 As shown, reference protein images and corresponding reference cell contour images can be fed into a protein feature extraction model to obtain the semantic features of the protein from the image source; reference protein sequences can be extracted using a sequence feature extraction model to obtain the semantic features of the protein from the sequence source. After being selected or transformed, the semantic features of proteins from different sources, along with the target contour features corresponding to the target cell contour image, enter the image generation stage. This overall architecture reflects the decoupled input of protein semantic data and target cell contour images: protein semantic data mainly provides the distribution attributes of the protein to be generated, while the target cell contour image mainly provides the spatial boundaries and morphological conditions of the generated image. After being fused in the bridging module, the target protein image is then generated by the decoder.

[0083] According to embodiments of this application, protein semantic features and target contour features are separated, extracted, and then fused to generate target protein fusion features. Since protein semantic features are used to express the distribution attributes of the protein to be generated, while target contour features are used to express the spatial boundaries and morphological structure of the target cell, after fusion in the bridging module, the protein distribution attributes can be confined within the generation space defined by the target cell contour. This reduces the possibility of inconsistent boundaries, extracellular signal responses, or mismatches with the target cell morphology in the generated image.

[0084] Meanwhile, this application embodiment sets up reference image paths and reference sequence paths in the protein semantic data. The reference image path can utilize the spatial distribution attributes provided by existing protein images; the reference sequence path can utilize the protein identity and potential location information provided by reference protein sequences when a reference protein image is missing. Since both paths ultimately form protein semantic features and are fused with target contour features, the model can generate images of different proteins under the same target contour by replacing the protein semantic data, and can also generate images based on sequence information when a reference image is missing. This technical approach helps improve the model's adaptability to different proteins and different input conditions.

[0085] Figure 4 The illustration shows a schematic diagram of the principle of obtaining protein semantic features according to an embodiment of this application.

[0086] like Figure 4 As shown, when protein semantic data includes a reference protein image and a reference cell contour image, protein semantic features can be obtained through a feature cross-referencing module. The feature cross-referencing module can include a protein feature extraction unit, a contour feature extraction unit, and a contour conditional fusion unit. The protein feature extraction unit processes the reference protein features, the contour feature extraction unit processes the reference contour features, and the contour conditional fusion unit uses the reference contour features as spatial context conditions to guide the reference protein features. The contour conditional fusion unit can include a local feature modulation subunit, a global cross-attention subunit, and a dynamic gating fusion subunit.

[0087] Based on the reference protein features of the reference protein image and the reference contour features of the reference cell contour image, the semantic features of the protein are obtained, which may include the following process.

[0088] First, local affine modulation is performed on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image. This achieves cross-modal spatial guidance of the reference protein features by the reference contour features, resulting in reference locally modulated features. As one implementation method, a local feature modulation subunit is used to perform local affine modulation on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image, thereby achieving cross-modal spatial guidance of the reference protein features by the reference contour features and obtaining reference locally modulated features.

[0089] Local affine modulation is used to adjust reference protein features at local spatial locations or at the level of local feature elements. Reference contour features contain information about cell boundaries, nuclei, cytoplasmic regions, or other local spatial structures. By generating modulation parameters using reference contour features, reference protein features can be enhanced, suppressed, or shifted to varying degrees at different spatial locations, thereby establishing a local correspondence between reference protein features and reference cell morphology.

[0090] Secondly, a cross-attention strategy is used to process the reference protein features used as queries and the intermediate reference contour features corresponding to the reference contour features as keys and values, resulting in a global reference fusion feature. This global reference fusion feature is used to capture the long-range dependencies and relative distribution patterns of protein features in the global view under the spatial constraints of the intermediate reference contour features. As one implementation, a global cross-attention subunit is used to process the reference protein features used as queries and the intermediate reference contour features corresponding to the reference contour features as keys and values, based on a cross-attention strategy, to obtain the global reference fusion feature.

[0091] Cross-attention strategies allow reference protein features to reallocate attention based on spatial information in reference contour features. The reference protein features act as queries, expressing the retrieval needs of protein distribution information within a spatial context; intermediate reference contour features act as keys and values, providing a retrieveable and aggregated contour spatial context. Through attention weight calculation, the model can establish global associations between protein distribution and different regions within the cell contour.

[0092] Next, protein semantic features are obtained based on the reference local modulation features and the reference global fusion features. The reference local modulation features emphasize local spatial alignment, while the reference global fusion features emphasize global relative distribution. Combining the two allows for the simultaneous preservation of local boundary constraints and global distribution patterns in the protein semantic features. As one implementation method, a dynamic gating fusion subunit is used to obtain protein semantic features based on the reference local modulation features and the reference global fusion features.

[0093] In one alternative implementation, let the protein feature of the input conditional fusion unit be P, and the contour feature be M. The local modulation branch can first project the contour feature to obtain the intermediate contour feature H that satisfies the following formula (1).

[0094] H=φ(W mM +b m (1)

[0095] Among them, W mM b represents the learnable projective weights. m Let φ represent the bias term and φ represent the nonlinear activation function. Then, based on the intermediate contour feature H, the scaling parameter γ that satisfies the following formula (2) and the translation parameter β that satisfies the following formula (3) are generated respectively.

[0096] γ=tanh(W γH +b γ (2)

[0097] β=W βH +b β (3)

[0098] Among them, W γH W βH b represents the learnable weights. γ b β γ represents the bias term, and tanh represents the hyperbolic tangent function, used to limit the magnitude of the scaling parameter. Subsequently, the protein feature P is affinely modulated using the scaling parameter γ and the translation parameter β to obtain the locally modulated feature (i.e., the locally constrained feature) P satisfying the following formula (4). local .

[0099] P local =(1+γ)⊙P+β(4)

[0100] Wherein, ⊙ represents element-wise multiplication. This formula is merely an exemplary implementation and does not limit the scope of protection of the embodiments of this application. The role of local modulation is to adjust the local position or local channel of protein features according to contour features, so that the protein features can perceive cell boundaries, cell nucleus regions or other local spatial conditions.

[0101] Secondly, the cross-attention branch uses protein features as query features (i.e., queries) and contour features as key features (i.e., keys) and value features (i.e., values) for cross-attention processing to obtain fused features (i.e., global constraint features). Cross-attention is used to capture the long-range dependencies between protein features and contour features. For example, some proteins may have global localization features such as intranuclear distribution, perimembrane distribution, or diffuse cytoplasmic distribution, and relying solely on local modulation may not be sufficient to fully express their spatial regularity. Through cross-attention, protein features can actively query spatial structural information in contour features, thereby establishing a global correspondence.

[0102] In one alternative implementation, the global constraint features satisfy the following formulas (5) to (9).

[0103] (5)

[0104] (6)

[0105] (7)

[0106] The cross-attention weights satisfy the following formula (8).

[0107] (8)

[0108] The global constraint features that satisfy the following formula (9) are obtained based on the cross-attention weights and value features.

[0109] (9)

[0110] in, Indicates query characteristics, Indicates key features, Indicates value characteristics, , and Indicates learnable mapping parameters, express transpose, Representing feature dimension, Indicates the cross-attention weights. This represents global constraint features.

[0111] For ease of explanation, the above cross-attention can satisfy the following formulas (10) to (11).

[0112] (10)

[0113] (11)

[0114] in, Indicates the first Each query feature Indicates the first Key features, Indicates the first Individual value features, Indicates the first The query features and the first Cross-attention weights among key features Indicates the first The global constraint features corresponding to each feature location.

[0115] This formula is merely one exemplary implementation. Through the above calculations, global spatial information can be obtained for each location or feature marker of a protein feature based on its correlation with the contour features.

[0116] Secondly, the gated fusion branch determines the gating weights based on local and global constraint features, and fuses these features to update the protein features. Since different proteins have different localization patterns and different cell contours vary significantly, the contribution ratios of local modulation and global cross-attention to the final features should not be fixed. The gated fusion branch can adaptively determine the weights of local and global constraints based on the current input.

[0117] In one alternative implementation, local constraint features can be... and global constraint features The data is concatenated and input into the gate network (Gate) to obtain the local gate control weights. and global gating weights It can satisfy the following formula (12).

[0118] (12)

[0119] Then, the local constraint features and global constraint features are weighted and fused to obtain the protein semantic features. It can satisfy the following formula (13).

[0120] (13)

[0121] Gated networks can include linear layers, multilayer perceptrons, SiLU activation functions, normalized layers, or de-energized layers. Gating weights can be normalized to ensure that local and global gating weights contribute proportionally across the feature or channel dimensions.

[0122] According to embodiments of this application, two feature interaction methods are employed: local affine modulation and global cross-attention. Local affine modulation locally enhances, suppresses, or shifts reference protein features based on reference contour features, enabling the protein features to establish correspondences with local structures such as cell boundaries, nuclear regions, and cytoplasmic regions. Global cross-attention allows the reference protein features to capture long-range relationships within the spatial context provided by the reference contour features, such as the relative relationship between perinuclear distribution, perimembrane distribution, diffuse cytoplasmic distribution, or local particle distribution and the overall cell morphology. Thus, protein semantic features can reflect both local spatial constraints and global relative distribution patterns, helping to maintain protein distribution attributes in subsequent target image generation stages.

[0123] In one alternative implementation, local constraint features can be... and global constraint features The vectors are concatenated and input into the gated network (Gate) to obtain the local weight vector (i.e., the local gated weights). And the global weight vector (i.e., the global gated weights). It can satisfy the following formula (14).

[0124] (14)

[0125] Then, the local constraint features and global constraint features are weighted and fused to obtain the protein semantic features. It can satisfy the following formula (15).

[0126] (15)

[0127] A gated perceptron strategy may include at least one linear layer, a nonlinear activation layer, and an output layer. The nonlinear activation layer may employ SiLU, ReLU, GELU, or other activation functions. The weight vector output by the gated perceptron strategy can be divided along the channel dimension into a reference local weight vector and a reference global weight vector.

[0128] According to embodiments of this application, a gated perceptron strategy is used to determine the contribution distribution of reference local modulation features and reference global fused features, rather than simply adding them together. Since different proteins have different spatial distribution patterns and different spatial constraints in different cell contours, a fixed fusion method may over-rely on local or global information in some samples. The gated perceptron generates a weight vector based on the reference splicing features, enabling the model to adaptively adjust the contribution ratio of local and global features according to the current protein distribution and contour context. This reduces information bias during feature fusion and improves the adaptability of protein semantic features to different proteins and cell morphologies.

[0129] Local affine modulation of the reference protein features of a reference protein image based on the reference contour features of a reference cell contour image can include: projecting the reference contour features of the reference cell contour image to obtain intermediate reference contour features; performing linear mapping on the intermediate reference contour features to obtain affine transformation parameters; and using the affine transformation parameters to perform affine transformation on the reference protein features of the reference protein image to obtain reference local modulation features.

[0130] Projecting reference contour features can give them the same or interactive dimensions as the reference protein features. Projection methods can include linear projection, convolutional projection, normalized projection, attention projection, or combinations thereof. After obtaining the intermediate reference contour features, two different linear mapping branches can be used to generate scaling and translation parameters, respectively. The scaling parameters control the local enhancement or suppression of the reference protein features in the amplitude dimension, while the translation parameters control the offset of the reference protein features in the activation baseline dimension.

[0131] In one implementation, the scaling parameter can be processed by a hyperbolic tangent function, a sigmoid function, or other amplitude-limiting activation functions to avoid excessive modulation amplitude. The translation parameter can be obtained directly from a linear mapping or used for translation after normalization. The affine transformation parameter can be a tensor of the same size as the reference protein feature, or it can be a vector or feature map that can be used to generate the reference protein feature through a broadcast.

[0132] According to embodiments of this application, affine transformation parameters are generated from reference contour features, enabling the reference contour features to influence reference protein features at a local spatial level. Since the scaling parameter acts on the amplitude dimension, it can enhance or suppress the protein feature response at locations related to cell structure; the translation parameter acts on the activation baseline dimension, adjusting the baseline activation level of the reference protein features in different regions. Thus, local affine modulation can establish a correspondence between cell contour structure and local protein distribution at the feature level, providing local spatial constraints for subsequent semantic feature generation.

[0133] The reference stitching features are processed using a gated perceptron strategy to obtain a reference local weight vector and a reference global weight vector. This can include: performing a nonlinear transformation on the reference stitching features using a gated perceptron strategy to obtain a weight vector, wherein the weight vector is used to control the contribution distribution of the reference local modulation features and the reference global fusion features in feature fusion; and splitting the weight vector along the channel dimension to obtain the reference local weight vector and the reference global weight vector.

[0134] A gated perceptron strategy may include a first linear layer, a nonlinear activation layer, and a second linear layer. The first linear layer is used to perform dimensionality transformation on the reference stitched features, the nonlinear activation layer is used to introduce nonlinear mapping relationships, and the second linear layer is used to output weight vectors corresponding to the reference local modulation features and the reference global fused features. In some implementations, the gated perceptron may also include a normalization layer, a de-extension layer, or residual connections.

[0135] Segmenting the weight vector along the channel dimension allows the resulting local reference weight vector to correspond to the local reference modulation feature, and the global reference weight vector to correspond to the global reference fusion feature. In some implementations, the segmented weight vector can also be normalized to ensure that the local and global reference weight vectors satisfy preset weight constraints.

[0136] According to embodiments of this application, a weight vector is obtained through nonlinear transformation, and then divided into local weights and global weights along the channel dimension. Since nonlinear transformation can express the complex relationship between reference local modulation features and reference global fusion features, the resulting weight vector can adjust the feature fusion method according to the current input state. After segmentation along the channel dimension, the local weights and global weights correspond to their respective feature branches, enabling a controllable distribution of branch contributions in the feature fusion process. This reduces the channel representation confusion caused by simple concatenation or direct addition.

[0137] The protein semantic features are obtained by processing the reference local modulation features and the reference global fusion features using the reference local weight vector and the reference global weight vector. This can include: performing linear projection on the reference local modulation features and the reference global fusion features respectively to obtain reference local projection features and reference global projection features; and fusing the reference local projection features and the reference global projection features using the reference local weight vector and the reference global weight vector to obtain the protein semantic features.

[0138] Linear projection is used to bring reference local modulation features and reference global fusion features into a unified feature dimension or a unified semantic space. Reference local modulation features originate from local affine modulation, while reference global fusion features originate from cross-attention; their statistical distributions and semantic meanings may differ. By performing linear projection separately, the two types of features can be dimensionally aligned and semantically adjusted before being fused using weight vectors.

[0139] In some implementations, linear projection can be achieved using fully connected layers, 1×1 convolutional layers, normalization layers, or combinations thereof. Feature fusion can be weighted summation, weighted concatenation followed by projection, residual fusion, or other fusion methods that preserve the weight control relationships.

[0140] According to embodiments of this application, the reference local modulation features and the reference global fusion features are first linearly projected, and then fused based on weight vectors. This technique allows features from different sources to first enter a comparable and fusionable feature space, avoiding direct fusion of local and global features due to differences in dimensionality or distribution. Subsequently, weighted fusion is performed using the reference local weight vector and the reference global weight vector, which can form unified protein semantic features while preserving local spatial alignment and global distribution relationships. This improves the stability and usability of protein semantic features for generation.

[0141] Protein semantic features can be obtained by processing reference protein features from a reference protein image and reference contour features from a reference cell contour image using a protein feature extraction model. The protein feature extraction model can be obtained by training a first deep learning model using a sample protein image and the corresponding sample cell contour image. The following section combines... Figure 5 Please provide an explanation.

[0142] Figure 5 The diagram illustrates the training process of a protein feature extraction model according to an embodiment of this application.

[0143] like Figure 5 As shown, the first deep learning model includes multiple cascaded feature cross-modules. These cascaded feature cross-modules can be sequentially connected along the network depth direction, enabling independent updates and conditional interactions between sample protein features and sample contour features at multiple levels. Each feature cross-module includes a protein feature extraction unit, a contour feature extraction unit, and a contour conditional fusion unit.

[0144] The protein feature extraction unit is used to extract features from sample protein images to obtain the protein features. The protein feature extraction unit may include image patch embedding layers, positional encoding layers, self-attention layers, convolutional layers, feedforward networks, normalization layers, or combinations thereof. The protein feature extraction unit is mainly used to learn the distribution pattern of the protein image itself, reducing the interference of the contour background on the protein distribution features in the lower-level feature extraction stage.

[0145] The contour feature extraction unit is used to extract features from the sample cell contour image to obtain the sample contour features. The contour feature extraction unit can have the same or similar structure as the protein feature extraction unit, or it can adopt a convolutional structure or attention structure that is more suitable for morphological structure extraction. The contour feature extraction unit is mainly used to learn spatial context information such as cell boundaries, nuclear regions, cytoplasmic regions, and local morphology.

[0146] The contour conditional fusion unit is used to guide the sample protein features across modalities by using sample contour features as spatial context conditions, thereby obtaining the sample protein semantic features. The contour conditional fusion unit may include the aforementioned local affine modulation branch, global cross-attention branch, and gated fusion branch. After multiple feature cross-modules are cascaded, the model can progressively introduce contour conditions into protein features at multiple scales or multiple semantic levels.

[0147] Protein and contour images can be divided into multiple image patches, which are then embedded and fed into the protein feature extraction unit and contour feature extraction unit, respectively. After self-attention processing, the protein and contour features enter the contour conditional fusion unit for cross-branch information interaction. Subsequently, the fused features can be further processed through a feedforward network and a subsequent feature cross-processing module to obtain protein semantic features.

[0148] According to embodiments of this application, multiple cascaded feature cross-modules are set in the first deep learning model, and protein feature extraction units and contour feature extraction units are set in each feature cross-module. Since protein images and cell contour images have different physical meanings, directly stitching them together at the bottom layer may cause the contour background information to interfere with the protein's own distribution characteristics. Embodiments of this application first extract sample protein features and sample contour features separately, and then use a contour conditional fusion unit for cross-modal guidance, which can introduce spatial context while maintaining the relative independence of protein features. Multiple cascaded feature cross-modules further enable this guidance to occur at multiple levels, thereby improving the ability of protein semantic features to express cell spatial structure and protein distribution attributes.

[0149] The protein feature extraction model can be a trained student model. The trained student model is obtained by training a first deep learning model, including both the teacher and student models, using sample protein images and corresponding sample cell contour images. Multiple cascaded feature cross-modules include multiple cascaded feature cross-modules corresponding to the teacher model and multiple cascaded feature cross-modules corresponding to the student model.

[0150] During training, the teacher and student models can have the same or similar network structures, both including multiple cascaded feature cross-modules. In the initial stage, the parameters of the student model can be copied to the teacher model, aligning their representations within similar feature spaces. Subsequently, the student model is updated using a loss function and backpropagation, while the teacher model is updated based on the exponential moving average of the student model's parameters. This approach allows the teacher model to provide a relatively smooth target representation, while the student model acquires feature extraction capabilities by learning the representation output by the teacher model.

[0151] In one implementation, multi-scale view processing can be performed on sample protein images and sample cell contour images to obtain a global view and a local view. The global view retains a large range of cell and protein distribution information to provide macroscopic spatial relationships; the local view retains a smaller range of local regions to provide local structural information. The global view can be input into both the teacher model and the student model, while the local view can be input into the student model. The teacher model outputs a target probability distribution or target feature representation based on the global view, while the student model outputs a predicted probability distribution or predicted feature representation based on both the global and local views. By making the student model output close to the teacher model output, the student model can be trained to form a consistent protein representation under both local and global perspectives.

[0152] Loss functions can include self-supervised distillation loss functions. Self-supervised distillation loss functions can calculate the difference between the teacher model output and the student model output after temperature scaling and centering.

[0153] First, teacher and student models are constructed. Both models employ a protein representation network containing bi-branch feature cross-modules. The teacher and student models can have the same network structure, including protein branches, contour branches, multiple bi-branch feature cross-modules, and feature projection heads. The student model is used for backpropagation updates via a loss function, while the teacher model provides a stable target feature representation.

[0154] Secondly, multi-scale view processing is performed on the protein image data to obtain a global view and a local view. Protein image data can include protein fluorescence images and their corresponding cell contour images. Multi-scale view processing can include random cropping, scale transformation, image enhancement, masking, intensity perturbation, flipping, rotation, or combinations thereof. The global view can cover a large cellular region to preserve the overall protein distribution pattern; the local view can cover a smaller local region to force the network to learn the relationship between local details and global semantics.

[0155] Next, a global view is input into both the teacher and student models, while a local view is input into the student model. The teacher model outputs a teacher feature representation based on the global view, while the student model outputs a student feature representation based on both the global and local views. Since the local view is not input into the teacher model, the student model needs to learn high-level semantic features consistent with the global representation of the teacher model, given limited local information. This asymmetric input approach enables the student model to extract a more robust protein distribution representation.

[0156] Then, based on the feature representation output by the teacher model using the global view, the feature representation output by the student model using both the global and local views is constrained. Constraint methods can include cross-entropy loss, mean squared error loss, KL divergence loss, contrastive loss, or combinations thereof. In one optional implementation, temperature scaling and normalization can be applied to the teacher and student model outputs separately, and then the distribution difference between the two can be calculated. Through this loss function, the student model is constrained to output a feature representation consistent with or close to that of the teacher model.

[0157] Finally, the student model parameters are updated, and the teacher model parameters are updated with momentum based on these student model parameters. The student model parameters can be updated via backpropagation and gradient descent. The teacher model parameters do not necessarily receive gradients directly; instead, they are updated using an exponential moving average based on the student model parameters. Let the teacher model parameters in the t-th iteration be... The student model parameters are The momentum coefficient is Then the teacher model parameters can be updated using the following formula (16).

[0158] (16)

[0159] Where m can be a preset momentum coefficient. This formula is only one exemplary implementation. Through momentum updates, the teacher model can maintain a relatively stable feature representation, avoiding drastic fluctuations during training.

[0160] In some implementations, the training of the protein characterization network can employ a self-supervised teacher-student distillation strategy based on the DINOv2 concept. Specifically, a teacher model and a student model are constructed, both using a dual-branch feature crossover module structure. The input protein image data can undergo multi-scale view processing to form a global view and a local view; the global view is input into both the teacher and student models, and the local view is input into the student model. The student model is trained based on the feature representation output by the teacher model from the global view, and the teacher model parameters are updated with momentum based on the student model parameters.

[0161] In some implementations, when using self-supervised teacher-student distillation training based on the DINOv2 concept, temperature scaling can be applied to the outputs of the teacher and student models, and the cross-entropy loss function value between their probability distributions can be determined. The cross-entropy loss function can satisfy the following formulas (17) to (19).

[0162] (17)

[0163] (18)

[0164] (19)

[0165] in, This represents the output of the teacher model. This represents the output of the student model. Represents the centralization parameter. This represents the temperature parameter of the teacher model. This represents the temperature parameters of the student model. This represents the target probability distribution output by the teacher model. This represents the predicted probability distribution output by the student model. This indicates the loss during self-monitored distillation.

[0166] In some implementations, the types of sample protein images and corresponding sample cell contour images can be diverse. That is, the protein feature extraction model can be trained using various types of sample protein images and corresponding sample cell contour images. This diversity of training samples improves the versatility of the protein feature extraction model.

[0167] Table 1 schematically illustrates the characterization capability verification results of the protein feature extraction model according to embodiments of this application.

[0168]

[0169] Table 1

[0170] As shown in Table 1, the features output by the trained protein feature extraction model can be used for downstream tasks such as protein identity prediction and subcellular localization to verify the usability of the extracted features. The results in Table 1 demonstrate that the features obtained using the protein feature extraction model with a dual-branch feature cross-module have verifiable representational capabilities in the corresponding downstream tasks. This verification result serves as an explanation of the ability of the model structure in this application's embodiments to extract effective protein semantic features.

[0171] To verify the effectiveness of the protein representation network in extracting features, the parameters of the network can be fixed after training, and its output features can be validated using a linear probing method on downstream tasks. Downstream tasks can include protein identity prediction and subcellular localization tasks, and evaluation metrics can include recall (R), precision (P), macro F1 score (Ma-F1), and micro F1 score (Mi-F1). Table 1 shows the comparison results of the protein representation network with other representation methods on different datasets. As shown in Table 1, compared to the DINOv2 method which directly uses the standard visual Transformer backbone, the protein representation network in this embodiment, which includes a dual-branch feature cross-module, improves the feature representation performance in protein identity prediction and subcellular localization tasks. This indicates that by extracting features from the protein branch and contour branch separately and then conditionally fusing them, more suitable protein conditional features for generating subcellular protein distribution images can be obtained.

[0172] According to embodiments of this application, a protein feature extraction model is trained using a teacher model and a student model, both of which include multiple cascaded feature cross-modules. The student model learns the target representation formed by the teacher model based on a global view through different view inputs, enabling it to establish a consistent protein representation between local and global information. The teacher model is updated using an exponential moving average of the student model parameters, reducing target fluctuations during training. Thus, even with limited sample annotations or complex sample morphologies, a student model for extracting protein semantic features can be obtained. When used in subsequent generation processes, the trained student model can provide stable protein semantic features for reference image paths.

[0173] Figure 6A The illustration shows a schematic diagram of protein image generation based on a reference protein image, a reference cell contour image, and a target cell contour image according to an embodiment of this application.

[0174] Figure 6B The illustration shows a schematic diagram of protein image generation based on a reference protein sequence and a target cell outline image according to an embodiment of this application.

[0175] like Figure 6A and Figure 6B As shown, feature fusion of protein semantic features and target contour features to obtain target protein fusion features can include: using a bridging module in a protein image generation model to fuse protein semantic features and target contour features to obtain target protein fusion features. Decoding the target protein fusion features to generate a target protein image can include: using a decoder in a protein image generation model to decode the target protein fusion features to obtain the target protein image.

[0176] The bridging module can receive protein semantic features from either image or sequence sources. For image-based protein semantic features, the reference protein image and reference cell contour image can be input into a protein feature extraction model to obtain protein semantic features; the target cell contour image can be input into a contour encoder to obtain target contour features; the bridging module then fuses the two and outputs the target protein fusion feature. For sequence-based protein semantic features, the reference protein sequence can be input into a sequence encoder to obtain sequence features; the sequence features are then projected or transformed to form protein semantic features; the target cell contour image is still input into a contour encoder to obtain target contour features; the bridging module then fuses the protein semantic features and target contour features and outputs the target protein fusion feature.

[0177] In some implementations, protein semantic features from image sources and protein semantic features from sequence sources can enter the same bridging module and the same decoder. In other implementations, they can enter a bridging module and decoder that share some parameters, or they can be processed separately by a pre-projection layer before entering a common generative latent space. Shared or partially shared generation paths enable protein semantic features from different sources to be subject to consistent target contour constraints during the generation stage, thereby improving the model's adaptability to different input modalities.

[0178] The protein image generation model can be obtained by training a second deep learning model using a sample reference protein image and a sample reference cell outline image corresponding to the sample reference protein image, as well as a sample target protein image, a sample target cell outline image corresponding to the sample target protein image, and a sample target protein sequence corresponding to the sample target protein image.

[0179] During training, sample reference protein images and sample reference cell contour images are used to form semantic features of sample proteins from image sources; sample target protein sequences are used to form semantic features of sample proteins from sequence sources; sample target cell contour images are used to form sample target contour features; and sample target protein images serve as supervision signals. The second deep learning model may include a bridging module and a decoder, or it may include a contour encoder for processing target contour images and a projection module for processing sequence features.

[0180] The bridging module maps protein semantic features to a generated latent space corresponding to target contour features. This mapping can include the following processes: performing feature transformation on the protein semantic features to obtain bridging conditional features; spatially constraining the bridging conditional features based on the target contour features so that the protein distribution attributes represented by the bridging conditional features correspond to the spatial region defined by the target cell contour; and fusing the spatially constrained bridging conditional features with the target contour features to obtain the target protein fusion features.

[0181] According to embodiments of this application, a bridging module and a decoder are configured in the protein image generation model. The bridging module is used to fuse protein semantic features and target contour features at the feature level, and the decoder is used to reconstruct the fused target protein features into a target protein image. Since image source features, sequence source features, and target contour features differ in their representational forms, direct input to the decoder can easily lead to feature space mismatch. The bridging module establishes a correspondence between protein distribution attributes and target cell contour structures in the latent space of the generation through feature transformation and spatial constraints. Therefore, the decoder can recover the protein image within the constraints of the target cell contour, which helps improve the consistency between the generated image and the target contour and the distribution attributes of the protein to be generated.

[0182] Simultaneously, by jointly training the second deep learning model using sample reference protein images, sample reference cell contour images, sample target protein images, sample target cell contour images, and sample target protein sequences, the image path and sequence path can share the supervision target during training. Since both paths use the sample target protein image as the target output, image source features and sequence source features are guided to similar or compatible generative latent spaces in the bridging module. This reduces semantic bias when semantic features from proteins of different sources enter the generation process.

[0183] The protein image generation model can be obtained by training a second deep learning model based on a sample target protein image, a protein image generated from a first sample, and a protein image generated from a second sample. The protein image generated from the first sample is obtained using a sample reference protein image, a sample reference cell contour image, and a sample target cell contour image. The protein image generated from the second sample is obtained using a sample reference protein image, a sample reference cell contour image, and a sample target protein sequence.

[0184] In one training method, a first generation path and a second generation path can be constructed. The first generation path is a reference image generation path. This path receives a sample reference protein image, a sample reference cell contour image, and a sample target cell contour image. The sample reference protein image and the sample reference cell contour image are processed by a protein feature extraction model to obtain the first protein semantic features; the sample target cell contour image is processed by a contour encoder to obtain the sample target contour features; a bridging module fuses the first protein semantic features and the sample target contour features, and the decoder outputs the first sample generated protein image.

[0185] The second generation path is the sequence generation path. This path receives the target protein sequence and the target cell contour image. The target protein sequence is processed by a sequence encoder or sequence projection module to obtain the second protein semantic features; the target cell contour image is processed by a contour encoder to obtain the target contour features; the bridging module fuses the second protein semantic features and the target contour features, and the decoder outputs the second sample-generated protein image.

[0186] During training, a second deep learning model can be trained based on the differences between the target protein image of the sample and the protein image generated from the first sample, as well as the differences between the target protein image of the sample and the protein image generated from the second sample. Training losses can include pixel-level mean squared error loss, structural similarity loss, perceptual loss, adversarial loss, region constraint loss, or combinations thereof.

[0187] Figure 7 The illustration shows a schematic diagram comparing the target protein image generation method according to an embodiment of this application with other methods.

[0188] exist Figure 7 In the example shown, column (a) can represent the target cell contour image, column (b) can represent the real target protein image, columns (c) and (d) can represent the generation results of the comparison model, and column (e) can represent the generation result obtained by the method of the embodiments of this application. By comparing the real target protein image, the comparison result, and the generation result of the method of the embodiments of this application, the consistency of the generated images in terms of cell boundaries, local protein signals, and the distribution of nuclear or cytoplasmic regions can be observed.

[0189] In some implementations, auxiliary evaluation information can be generated based on the difference between the protein image generated from the first sample and the protein image generated from the second sample, or based on the spatial consistency between the generated image and the target contour features. This auxiliary evaluation information can be used to indicate the reliability of the generated result in different spatial regions. This auxiliary evaluation information may not be a necessary step in generating the target protein image, but rather an optional output for model validation or result display.

[0190] According to embodiments of this application, a second deep learning model is trained jointly through a first generation path and a second generation path. The first generation path generates a first sample generated protein image using a sample reference protein image and a sample reference cell contour image, while the second generation path generates a second sample generated protein image using a sample target protein sequence. Both paths use the sample target protein image as the supervision target. Since protein semantic features from different sources are constrained to generate the same target protein image under the same target contour, the bridging module and decoder can learn the alignment relationship between image source features and sequence source features. Thus, during the inference phase, regardless of whether the input is a reference protein image or a reference protein sequence, the model can map the corresponding protein semantic features to the generative latent space corresponding to the target cell contour and generate the target protein image.

[0191] Furthermore, the first generation path can utilize existing reference protein images to provide more direct spatial distribution attributes, while the second generation path can utilize reference protein sequences to provide protein semantic information when image data is lacking. After joint training of the two paths, the model can share target contour constraints and decoding processes across different input conditions. This technique helps improve the model's adaptability in scenarios involving multiple protein switching, lack of reference images, and generation of different cell morphologies.

[0192] The formulas, network layer types, loss function types, and training procedures given in the embodiments of this application are for illustrating optional implementation methods and are not intended to limit the scope of protection. Those skilled in the art can adjust the number of network layers, feature dimensions, number of attention heads, loss weights, training batch size, optimization algorithms, encoder type, or decoder type according to specific data scale, image resolution, protein category, computing resources, and application scenario.

[0193] This application also provides a protein image generation apparatus.

[0194] The protein image generation device may include a first acquisition module, a second acquisition module, a third acquisition module, and a fourth acquisition module.

[0195] The first acquisition module is used to extract features from the target cell contour image corresponding to the protein image to be generated, and obtain the target contour features.

[0196] The second obtaining module is used to obtain protein semantic features based on the reference protein features of the reference protein image and the reference contour features of the reference cell contour image when the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image.

[0197] The third acquisition module is used to extract features from the reference protein sequence when the protein semantic data includes the reference protein sequence, thereby obtaining the protein semantic features.

[0198] The fourth module is used to perform feature fusion on the semantic features of the protein and the contour features of the target to obtain the fused features of the target protein.

[0199] The generation module is used to decode the fusion features of the target protein to obtain the target protein image.

[0200] Reference contour features can be used as spatial context conditions to guide reference protein features across modalities. Target contour features can be used as spatial boundary constraints to guide pixel-level reconstruction of protein semantic features.

[0201] In some implementations, the third obtaining module may include the first obtaining submodule, the second obtaining submodule, and the third obtaining submodule.

[0202] The first acquisition submodule is used to perform local affine modulation on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image, so as to realize the cross-modal spatial guidance of the reference protein features by the reference contour features and obtain the reference local modulation features.

[0203] The second submodule is used to process the reference protein features used as queries and the intermediate reference contour features corresponding to the reference contour features as keys and values ​​using a cross-attention strategy to obtain reference global fusion features.

[0204] The third submodule is used to obtain protein semantic features based on reference local modulation features and reference global fusion features.

[0205] Reference global fusion features can be used to capture the long-range dependencies and relative distribution patterns of protein features in the global field of view under the spatial constraints of intermediate reference contour features.

[0206] In some implementations, the third obtaining submodule may include the first obtaining unit, the second obtaining unit, and the third obtaining unit.

[0207] The first obtaining unit is used to concatenate the reference local modulation features and the reference global fusion features along the channel dimension to obtain the reference concatenated features.

[0208] The second acquisition unit is used to process the reference stitching features using a gated perceptron strategy to obtain the reference local weight vector and the reference global weight vector.

[0209] The third acquisition unit is used to process the reference local modulation features and the reference global fusion features using the reference local weight vector and the reference global weight vector to obtain the protein semantic features.

[0210] The reference local weight vector can be used to control the contribution distribution of reference local modulation features to protein semantic features. The reference global weight vector can be used to control the contribution distribution of reference global fusion features to protein semantic features.

[0211] In some implementations, the first obtaining submodule may include a fourth obtaining unit, a fifth obtaining unit, and a sixth obtaining unit.

[0212] The fourth obtaining unit is used to project the reference contour features of the reference cell contour image to obtain intermediate reference contour features.

[0213] The fifth unit is used to perform linear mapping on the intermediate reference contour features to obtain affine transformation parameters.

[0214] Unit 6 is used to perform affine transformation on the reference protein features of the reference protein image using affine transformation parameters to obtain reference local modulation features.

[0215] Affine transformation parameters can include scaling and translation parameters. Scaling parameters can be used to control local enhancement or suppression of the reference protein features in the amplitude dimension. Translation parameters can be used to control the offset of the reference protein features in the activation baseline dimension.

[0216] In some implementations, the second obtaining unit may include a first obtaining subunit and a second obtaining subunit.

[0217] The first acquisition sub-unit is used to perform a nonlinear transformation on the reference stitching features using a gated perceptron strategy to obtain a weight vector.

[0218] The second sub-unit is used to divide the weight vector along the channel dimension to obtain a reference local weight vector and a reference global weight vector.

[0219] The weight vector can be used to control the contribution distribution of the reference local modulation features and the reference global fusion features in feature fusion.

[0220] In some implementations, the third obtaining unit may include a third obtaining subunit and a fourth obtaining subunit.

[0221] The third obtaining subunit is used to perform linear projection on the reference local modulation features and the reference global fusion features respectively to obtain the reference local projection features and the reference global projection features.

[0222] The fourth subunit is used to fuse the reference local projection features and the reference global projection features using the reference local weight vector and the reference global weight vector to obtain the protein semantic features.

[0223] In some implementations, protein semantic features can be obtained by processing reference protein features from a reference protein image and reference contour features from a reference cell contour image using a protein feature extraction model.

[0224] The protein feature extraction model can be obtained by training a first deep learning model using sample protein images and sample cell contour images corresponding to the sample protein images.

[0225] The first deep learning model may include multiple cascaded feature cross-modules. These feature cross-modules may include a protein feature extraction unit, a contour feature extraction unit, and a contour conditional fusion unit. The protein feature extraction unit can be used to extract features from sample protein images to obtain sample protein features. The contour feature extraction unit can be used to extract features from sample cell contour images to obtain sample contour features. The contour conditional fusion module can be used to use sample contour features as spatial context conditions to guide sample protein features across modalities, obtaining sample protein semantic features.

[0226] In some implementations, the protein feature extraction model can be a trained student model. The trained student model can be obtained by training a first deep learning model, including both a teacher model and a student model, using sample protein images and corresponding sample cell contour images. Multiple cascaded feature cross-modules can include multiple cascaded feature cross-modules corresponding to the teacher model and multiple cascaded feature cross-modules corresponding to the student model.

[0227] In some implementations, the third obtaining module may include a fourth obtaining submodule.

[0228] The fourth submodule is used to fuse the semantic features of the protein and the contour features of the target protein using the bridging module in the protein image generation model to obtain the fused features of the target protein.

[0229] The generation module may include a fifth sub-module.

[0230] The fifth submodule is used to decode the target protein fusion features using the decoder in the protein image generation model to obtain the target protein image.

[0231] The protein image generation model can be obtained by training a second deep learning model using a sample reference protein image and a sample reference cell outline image corresponding to the sample reference protein image, as well as a sample target protein image, a sample target cell outline image corresponding to the sample target protein image, and a sample target protein sequence corresponding to the sample target protein image.

[0232] In some implementations, the protein image generation model can be obtained by training a second deep learning model based on a target protein image, a protein image generated from a first sample, and a protein image generated from a second sample. The protein image generated from the first sample can be obtained using a reference protein image, a reference cell contour image, and a target cell contour image. The protein image generated from the second sample can be obtained using a reference protein image, a reference cell contour image, and a target protein sequence.

[0233] It should be noted that the protein image generation device in this application corresponds to the protein image generation method in this application. For a detailed description of the protein image generation device, please refer to the protein image generation method section, which will not be repeated here.

[0234] This application also provides an electronic device.

[0235] The electronic device includes a processor that can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or loaded from a storage portion into random access memory (RAM). The processor may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)). The processor may also include onboard memory for caching purposes. Processor 1101 may include a single processing unit or multiple processing units for performing different actions of the data download method flow according to embodiments of this application.

[0236] The RAM stores various programs and data required for the operation of the electronic device. The processor, ROM, and RAM are interconnected via a bus. The processor executes various operations of the method flow according to embodiments of this application by executing programs in the ROM and / or RAM. It should be noted that the programs may also be stored in one or more memories other than ROM and RAM. The processor may also execute various operations of the data download method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0237] Electronic devices may also include input / output (I / O) interfaces, which are also connected to a bus. Electronic devices may also include one or more of the following components connected to the input / output (I / O) interfaces: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard disks, etc.; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the input / output (I / O) interfaces as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as needed.

[0238] According to embodiments of this application, the method flow according to embodiments of this application can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a processor, it performs the functions defined in the system of embodiments of this application. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0239] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0240] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0241] For example, according to embodiments of this application, a computer-readable storage medium may include the ROM and / or RAM and / or one or more memories other than ROM and RAM described above.

[0242] This application also includes a computer program product comprising a computer program containing program code for performing the methods provided in this application. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the methods provided in this application.

[0243] When the computer program is executed by a processor, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0244] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices or magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed as signals over a network medium, and downloaded and installed via a communication component, and / or installed from a removable medium. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0245] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0246] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0247] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method for generating protein images, comprising: Feature extraction is performed on the target cell contour image corresponding to the protein image to be generated to obtain the target contour features; When the protein semantic data corresponding to the protein image to be generated includes a reference protein image and a reference cell contour image corresponding to the reference protein image, protein semantic features are obtained based on the reference protein features of the reference protein image and the reference contour features of the reference cell contour image, wherein the reference contour features are used as spatial context conditions to guide the reference protein features across modalities. When the protein semantic data includes a reference protein sequence, feature extraction is performed on the reference protein sequence to obtain protein semantic features; Feature fusion is performed on the protein semantic features and the target contour features to obtain target protein fusion features, wherein the target contour features are used as spatial boundary constraints to guide pixel-level reconstruction of the protein semantic features; and The fusion features of the target protein are decoded to generate an image of the target protein.

2. The method according to claim 1, wherein, The step of obtaining protein semantic features based on the reference protein features of the reference protein image and the reference contour features of the reference cell contour image includes: Based on the reference contour features of the reference cell contour image, the reference protein features of the reference protein image are locally affine modulated to achieve cross-modal spatial guidance of the reference protein features by the reference contour features, thereby obtaining reference local modulation features. A cross-attention strategy is used to process the reference protein features used as queries and the intermediate reference contour features corresponding to the reference contour features as keys and values, resulting in a reference global fusion feature. This reference global fusion feature is used to capture the long-range dependencies and relative distribution patterns of the protein features in the global field of view under the spatial constraints of the intermediate reference contour features; and The protein semantic features are obtained based on the reference local modulation features and the reference global fusion features.

3. The method according to claim 2, wherein, The step of obtaining the protein semantic features based on the reference local modulation features and the reference global fusion features includes: The reference local modulation feature and the reference global fusion feature are concatenated along the channel dimension to obtain the reference concatenated feature; The reference stitching features are processed using a gated perceptron strategy to obtain a reference local weight vector and a reference global weight vector; and The reference local modulation feature and the reference global fusion feature are processed using the reference local weight vector and the reference global weight vector to obtain the protein semantic feature. The reference local weight vector is used to control the contribution distribution of the reference local modulation feature to the protein semantic feature, and the reference global weight vector is used to control the contribution distribution of the reference global fusion feature to the protein semantic feature.

4. The method according to claim 2 or 3, wherein, The step of performing local affine modulation on the reference protein features of the reference protein image based on the reference contour features of the reference cell contour image to achieve cross-modal spatial guidance of the reference protein features by the reference contour features, and obtaining reference local modulation features, includes: The reference contour features of the reference cell contour image are projected to obtain intermediate reference contour features; A linear mapping is performed on the intermediate reference contour features to obtain affine transformation parameters, wherein the affine transformation parameters include scaling parameters and translation parameters. The scaling parameters are used to control the local enhancement or suppression of the reference protein features in the amplitude dimension, and the translation parameters are used to control the offset of the reference protein features in the activation baseline dimension; and The reference protein features of the reference protein image are obtained by performing an affine transformation on the reference protein image using the affine transformation parameters.

5. The method according to claim 3 or 4, wherein, The process of using a gated perceptron strategy to process the reference stitching features to obtain a reference local weight vector and a reference global weight vector includes: The reference stitched features are nonlinearly transformed using the gated perceptron strategy to obtain a weight vector, wherein the weight vector is used to control the contribution distribution of the reference local modulation features and the reference global fusion features in feature fusion; and The weight vector is divided along the channel dimension to obtain the reference local weight vector and the reference global weight vector.

6. The method according to any one of claims 3 to 5, wherein, The process of processing the reference local modulation features and the reference global fusion features using the reference local weight vector and the reference global weight vector to obtain the protein semantic features includes: Linear projection is performed on the reference local modulation features and the reference global fusion features respectively to obtain reference local projection features and the reference global projection features; and The reference local weight vector and the reference global weight vector are used to fuse the reference local projection features and the reference global projection features to obtain the protein semantic features.

7. The method according to any one of claims 1 to 6, wherein, The protein semantic features are obtained by processing the reference protein features of the reference protein image and the reference contour features of the reference cell contour image using a protein feature extraction model. The protein feature extraction model is obtained by training a first deep learning model using a sample protein image and a corresponding sample cell contour image. The first deep learning model includes multiple cascaded feature cross-modules, each including a protein feature extraction unit, a contour feature extraction unit, and a contour conditional fusion unit. The protein feature extraction unit extracts features from the sample protein image to obtain sample protein features. The contour feature extraction unit extracts features from the sample cell contour image to obtain sample contour features. The contour conditional fusion module uses the sample contour features as spatial context conditions to guide the sample protein features across modalities to obtain sample protein semantic features.

8. The method according to claim 7, wherein, The types of the sample protein image and the sample cell contour image corresponding to the sample protein image include various types; The protein feature extraction model is a trained student model. The trained student model is obtained by training the teacher model and student model included in the first deep learning model using the sample protein image and the sample cell contour image corresponding to the sample protein image. The multiple cascaded feature cross modules include multiple cascaded feature cross modules corresponding to the teacher model and multiple cascaded feature cross modules corresponding to the student model.

9. The method according to any one of claims 1 to 8, wherein, The feature fusion of the protein semantic features and the target contour features to obtain the target protein fused features includes: The bridging module in the protein image generation model is used to fuse the semantic features of the protein and the contour features of the target protein to obtain the fused features of the target protein. The step of decoding the target protein fusion features to generate a target protein image includes: The target protein image is obtained by decoding the target protein fusion features using the decoder in the protein image generation model. The protein image generation model is obtained by training a second deep learning model using a sample reference protein image and a sample reference cell contour image corresponding to the sample reference protein image, as well as a sample target protein image, a sample target cell contour image corresponding to the sample target protein image, and a sample target protein sequence corresponding to the sample target protein image.

10. The method according to claim 9, wherein, The protein image generation model is obtained by training a second deep learning model based on the target protein image of the sample, the protein image generated from the first sample, and the protein image generated from the second sample. The protein image generated from the first sample is obtained using the reference protein image of the sample, the reference cell contour image of the sample, and the target cell contour image of the sample. The protein image generated from the second sample is obtained using the reference protein image of the sample, the reference cell contour image of the sample, and the target protein sequence of the sample.

Images