A multi-frame single-cell sequence RNA transcription site detection method

Abstract

This invention provides a method for detecting RNA transcription sites in multi-frame single-cell sequences. The method includes: first, using a pre-trained 2D U-net model to extract feature maps, temporal location features, horizontal location features, and vertical location features from multi-frame cell image sequences; then, encoding and decoding the feature maps and three-dimensional location features; and finally, predicting class labels and RNA transcription site locations. Compared to existing two-dimensional single-frame cell image detection methods, this invention combines image features with three-dimensional location features between multiple frames to aid in RNA transcription site prediction, effectively improving the detection performance of RNA transcription sites.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for detecting RNA transcription sites in multi-frame single-cell sequences. Background Technology

[0002] In biological gene research, detecting RNA transcription sites is a crucial step in understanding the mechanisms and biological processes of gene expression. An RNA transcription site is a specific region formed during the transcription of a gene into mRNA. This region typically marks the start of mRNA production and is an important component of gene expression regulation. Based on the properties of RNA transcription sites and their morphological characteristics in fluorescence microscopy images, they can also be considered as a type of "particle." Typically, the transcription process is visualized in fluorescence microscopy images as a fluctuating fluorescence signal, characterized by localized intensity maxima that differ from the surrounding regions.

[0003] Existing RNA transcription site detection methods can be categorized into unsupervised and supervised methods. Unsupervised methods refer to models where, under unsupervised conditions, the user specifies parameter values ​​in a parameter template for noise suppression / particle enhancement, as well as thresholds for distinguishing particles from the background. Typically, these methods do not require training and usually assume that particles are isolated, relying on simple image processing techniques, without requiring the availability of labeled training data or a learning framework. However, unsupervised methods are often not only time-consuming but also have very limited applications and are highly sensitive to image quality.

[0004] Supervised methods include training on labeled image data using deep learning methods such as convolutional neural networks (CNNs) like U-Net. These methods not only achieve threshold-independent particle detection but also possess strong generalization performance. However, current particle detection algorithms are limited to the two-dimensional level, training and inference only on single-frame cell images. Due to the low signal-to-noise ratio of RNA transcription sites, their fluorescence intensity varies and may even disappear, sometimes appearing indistinguishable from noise in microscopic images. Two-dimensional detection methods are prone to false positives, resulting in poor performance in RNA transcription site detection. Summary of the Invention

[0005] This invention provides a method for detecting RNA transcription sites in multi-frame single-cell sequences, which overcomes the shortcomings of existing RNA transcription site detection methods that are limited to the two-dimensional level, only training and inference on single-frame cell images, and do not take into account the dynamic changes of RNA transcription sites, which easily leads to false positives and low detection results.

[0006] This invention provides a method for detecting RNA transcription sites in multiple single-cell sequences, the method comprising:

[0007] A multi-frame cell image sequence to be detected is obtained, and a pre-trained 2D U-net model is called to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features. The three-dimensional position features include temporal position features, horizontal position features, and vertical position features.

[0008] The feature map and the three-dimensional position features are encoded to obtain an encoded feature sequence, and the encoded feature sequence is decoded to obtain a high-level pixel feature sequence.

[0009] For each feature sequence in the high-level pixel feature sequence, class label prediction and RNA transcription site location prediction are performed to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each frame of cell image.

[0010] In some embodiments, the step of invoking a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features includes:

[0011] For each cell image in the multi-frame cell image sequence, a pre-trained 2D U-net model is invoked to extract features from the cell image to obtain a feature map of the cell image;

[0012] For the multi-frame cell image sequence, the pre-trained 2D U-net model is invoked to extract the temporal position features, horizontal position features, and vertical position features between cell image frames as three-dimensional position features.

[0013] In some embodiments, encoding the feature map and the three-dimensional position features to obtain an encoded feature sequence includes:

[0014] A convolutional network with a 1×1 kernel is invoked to perform convolution processing on each of the feature maps to obtain a convolutional feature map. The feature dimension of the convolutional feature map is then mapped to the encoder dimension to obtain a two-dimensional feature map.

[0015] The two-dimensional feature map and the three-dimensional position features are input into the multi-head self-attention layer of the encoder for self-attention calculation, and the calculated result is input into the feedforward network layer of the encoder for mapping processing to obtain the encoded feature sequence.

[0016] In some embodiments, decoding the encoded feature sequence to obtain a high-level pixel feature sequence includes:

[0017] Learnable instance features are obtained, wherein the learnable instance features are obtained from the pixel features of cell images in the multi-frame cell image sequence, and the number of features of the learnable instance features is N, where N is n×T, n is the number of cell images in the multi-frame cell image sequence, and T is the number of RNA transcription sites to be detected in each cell image;

[0018] The encoded feature sequence and the learnable instance features are input into the cross-attention layer of the decoder for cross-attention calculation, and the calculated result is input into the feedforward network layer of the decoder for mapping processing to obtain a high-level pixel feature sequence composed of N feature sequences.

[0019] In some embodiments, the RNA transcription site, the cellular background, and the normalized center coordinates of the RNA transcription site are obtained by an RNA transcription site detection model, the training process of which includes:

[0020] Obtain cell image samples, wherein the number of RNA transcription sites to be detected in the cell image samples is n, and the cell image samples carry real labels of RNA transcription sites;

[0021] The cell image samples are input into the RNA transcription site detection model for forward propagation to obtain the predicted labels of RNA transcription sites in the cell image samples;

[0022] The Hungarian algorithm is invoked to determine the optimal binary graph match between the true label and the predicted label;

[0023] The Hungarian loss value between the true label and the predicted label is determined based on the optimal binary graph matching cost, and the Hungarian loss value is used as the training loss value for backpropagation in the RNA transcription site detection model to update the parameters of the RNA transcription site detection model.

[0024] In some embodiments, the step of invoking the Hungarian algorithm to determine the optimal binary graph match between the ground truth label and the predicted label includes:

[0025] The candidate matching order is obtained by performing binary graph matching between the real labels and the predicted labels using the Hungarian algorithm.

[0026] Determine the binary graph matching cost of the candidate matching permutation;

[0027] The target matching arrangement with the lowest binary graph matching cost is determined from the candidate matching arrangement and is taken as the optimal binary graph matching.

[0028] In some embodiments, the predicted labels include RNA transcription sites, category predicted labels corresponding to the cell background, and predicted box labels corresponding to the normalized center coordinates of the RNA transcription sites. The determination of the binary graph matching cost for candidate matching permutations includes:

[0029] In the candidate matching permutation, the category prediction probability corresponding to the category prediction label in the predicted label is determined;

[0030] In the candidate matching arrangement, the true bounding box label corresponding to the normalized center coordinates of the RNA transcription site in the true label is determined, and the bounding box loss value between the true bounding box label and the predicted bounding box label is determined.

[0031] The difference between the bounding box loss value and the predicted probability of the category is used as the binary graph matching cost of the candidate matching permutation.

[0032] The present invention also provides a device for detecting RNA transcription sites in multiple single-cell sequences, the device comprising:

[0033] The image feature extraction module is used to acquire a multi-frame cell image sequence to be detected, and to call a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain the corresponding feature map and three-dimensional position features. The three-dimensional position features include temporal position features, horizontal position features and vertical position features.

[0034] The sequence feature extraction module is used to encode the feature map and the three-dimensional position features to obtain an encoded feature sequence, and to decode the encoded feature sequence to obtain an advanced pixel feature sequence.

[0035] The detection module is used to perform category label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequence, so as to obtain the RNA transcription site, cell background and normalized center coordinates of the RNA transcription site for each frame of cell image.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-frame single-cell sequence RNA transcription site detection method as described above.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-frame single-cell sequence RNA transcription site detection method as described above.

[0038] The present invention also provides a computer program product, comprising a computer program that, when executed by a processor, implements the multi-frame single-cell sequence RNA transcription site detection method as described above.

[0039] The present invention provides a method for detecting RNA transcription sites in multi-frame single-cell sequences. Before detection, a pre-trained 2D U-net model is used to extract feature maps, temporal location features, horizontal location features, and vertical location features from multi-frame cell image sequences. Then, the feature maps and three-dimensional location features are encoded and decoded, and finally, category label prediction and RNA transcription site location prediction are performed. Compared to existing two-dimensional single-frame cell image detection methods, this invention combines three-dimensional location features between multiple frames to aid in RNA transcription site prediction, effectively improving detection results. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a schematic flowchart of the method for detecting RNA transcription sites in multiple single-cell sequences provided by the present invention.

[0042] Figure 2 This is a diagram illustrating the training process of the U-net model provided by this invention.

[0043] Figure 3 This is a schematic diagram of the principle architecture of the multi-frame single-cell sequence RNA transcription site detection method provided by the present invention.

[0044] Figure 4 This is a schematic diagram of the training of the RNA transcription site detection model provided by the present invention.

[0045] Figure 5 This is a schematic diagram illustrating the application of the RNA transcription site detection model provided by this invention.

[0046] Figure 6 This is a schematic diagram of the structure of the multi-frame single-cell sequence RNA transcription site detection device provided by the present invention.

[0047] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The method for detecting RNA transcription sites in multiple single-cell sequences of the present invention will be described below with reference to the accompanying drawings. Figure 1 This is a schematic flowchart of the multi-frame single-cell sequence RNA transcription site detection method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps 101 to 103.

[0050] Step 101: Obtain the multi-frame cell image sequence to be detected, and call the pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain the corresponding feature map and three-dimensional position features.

[0051] First, a multi-frame cell image sequence to be detected is acquired. Considering that RNA transcription sites in cell images are easily affected by the signal-to-noise ratio, and fluorescence intensity can change and even disappear, this embodiment of the invention uses multiple cell images for detection of RNA transcription sites. The time intervals between each cell image are continuous, and these multiple cell images constitute a cell image sequence, denoted as . Where T represents the number of cell images, and 3 represents three channels. The height × width of the cell image.

[0052] The locations of RNA transcription sites can differ between sequences. By extracting the positional differences between sequences, we can help detect RNA transcription sites.

[0053] Before detection, this embodiment of the invention first uses the segmentation mask data of a single frame cell image to...

[0054] Manually label potential RNA transcription sites to create segmentation tags and organize cell image data. For example... Figure 2 As shown, a target segmentation task is then performed in the 2D U-net model using a single-frame cell image to train the 2D U-net model. During training, a corresponding Dice loss function is constructed based on the segmentation mask to optimize the 2D U-net model, resulting in a pre-trained 2D U-net model. In subsequent detection of RNA transcription sites in multi-frame cell image sequences, it is unnecessary to retrain the 2D U-net model, thus reducing training difficulty.

[0055] Next, see Figure 3 For a multi-frame cell image sequence to be detected, a pre-trained 2D U-net model is called to extract features from the multi-frame cell image sequence, and the corresponding feature maps and three-dimensional position features are obtained.

[0056] Here is as Figure 3 As shown, the feature extraction of the 2D U-net model is divided into two parts. The backbone is mainly used to extract feature maps of multiple cell images, while the other part extracts the three-dimensional position features of the multiple cell image sequence.

[0057] Specifically, for each cell image in a multi-frame cell image sequence, a pre-trained 2D U-net model is used to extract features from the cell image, resulting in a feature map of the cell image. The feature map is denoted as... Where T is the number of feature maps and C is the feature dimension. It is the height × width of the feature map.

[0058] For a multi-frame cell image sequence, a pre-trained 2D U-net model is used to extract temporal, horizontal, and vertical positional features between cell image frames, which are then used as three-dimensional positional features. Specifically, extracting three-dimensional positional features essentially involves extracting features of inter-frame positional information in the cell image, including temporal, horizontal, and vertical position.

[0059] Temporal location information characterizes the change in the position of RNA transcription sites over time, while horizontal and vertical location information represent the changes in the horizontal and vertical positions of RNA transcription sites within the cell image over time. After extracting the corresponding features, three-dimensional location features are formed. Each dimension's location feature is represented by sine and cosine functions of different frequencies, as shown in the following formulas:

[0060] (1)

[0061] In the above formula (1), The weight is denoted as , and its value is . , It represents the position in the corresponding dimension, where i represents the i-th cell image, and k is a parameter that takes the integer value. The value d is a parameter that is divisible by 3, because the positional features of the three dimensions will be merged to form the final positional code of channel d.

[0062] Step 102: Encode the feature map and 3D position features to obtain an encoded feature sequence, and decode the encoded feature sequence to obtain a high-level pixel feature sequence.

[0063] After feature extraction in step 101, the feature map and 3D position features are then encoded to obtain the encoded feature sequence. For example... Figure 3 As shown, the encoding process can be implemented by calling the Transformer encoder, which inputs the feature map and 3D position features of the cell image into the Transformer encoder for encoding.

[0064] To reduce the computational load of the encoder, this embodiment of the invention, after encoding, first calls a convolutional network with a 1×1 kernel to perform convolution processing on each feature map, obtaining a convolutional feature map. The feature dimension C is then reduced to d, resulting in the convolutional feature map, denoted as . Then, the feature dimensions of the convolutional feature map are mapped to the encoder dimensions to obtain a two-dimensional feature map. Here, in order to form a feature sequence suitable for the encoder input, [the following steps are taken]. The spatial and temporal dimensions are flattened into a single dimension, forming a shape of size [missing information]. The two-dimensional feature map is generated, and the time sequence is kept consistent with the initial feature map.

[0065] The Transformer encoder consists of a multi-head self-attention layer (MSA) and a feedforward network layer (FFN). During encoding, the two-dimensional feature map and the three-dimensional position features are input into the multi-head self-attention layer of the encoder for self-attention calculation, and the calculated result is input into the feedforward network layer of the encoder for mapping processing to obtain the encoded feature sequence.

[0066] Therefore, the feature dimension of the two-dimensional feature map is d, while the feature dimensions of the temporal, horizontal, and vertical positional features in the three-dimensional positional features are all d / 3. The dimension of the three-dimensional positional features formed by merging these three features is also d. Thus, the two-dimensional feature map and the three-dimensional positional features can be merged. The merged features are then input into the multi-head self-attention layer of the encoder for self-attention calculation, and the calculated results are input into the feedforward network layer of the encoder for mapping processing, resulting in multiple encoded feature sequences, such as... Figure 3 The frames n-1, n-1, and n+1 are shown.

[0067] Next, the encoded feature sequence is decoded to obtain the high-level pixel feature sequence. The decoder here has the same structure as the Transformer encoder and also uses the Transformer decoder, which consists of a self-attention mechanism layer and a feedforward network layer.

[0068] In some embodiments, under normal circumstances, the encoded feature sequence is directly input into the Transformer decoder for decoding, the input features are self-attention calculated through a self-attention mechanism layer, and then output through a feedforward network layer.

[0069] In order to improve the decoding accuracy and the detection information of RNA transcription sites in this embodiment of the invention, learnable instance features are introduced to help with decoding. The decoding process of this embodiment of the invention is described below.

[0070] First, learnable instance features are obtained. These features are queried from the pixel features of cell images in a multi-frame cell image sequence. The number of learnable instance features is N, where N is n×T, where n is the number of cell images in the multi-frame cell image sequence, and T is the number of RNA transcription sites to be detected in each cell image.

[0071] Learnable instance features can essentially be viewed as the final detection result of a multi-frame sequence of cell images to be detected. This final detection result is used as a cue label to help the Transformer decoder decode. Since the final input to the Transformer decoder is the encoded feature sequence and learnable instance features, the multi-head self-attention head of the Transformer decoder's self-attention layer is replaced with a cross-attention head, and the self-attention layer becomes a cross-attention layer.

[0072] Therefore, the encoded feature sequence and the learnable instance features are input into the cross-attention layer of the decoder for cross-attention calculation. During input, the encoded feature sequence and the learnable instance features follow the temporal order of the feature maps and remain consistent to avoid sequence disorder. During cross-attention calculation, the learnable instance features are used as the query, and the encoded feature sequence is used as the key and value, respectively. Finally, the calculated result is input into the feedforward network layer of the Transformer decoder for mapping processing, resulting in a high-level pixel feature sequence composed of N feature sequences.

[0073] Step 103: For each feature sequence in the high-level pixel feature sequence, perform category label prediction and RNA transcription site location prediction to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each frame of cell image.

[0074] After decoding in step 102, class label prediction and RNA transcription site location prediction are performed for each feature sequence in the high-level pixel feature sequence. Here, for example... Figure 3As shown, for the high-level pixel feature sequence obtained after decoding, N feature sequences can be sequentially input into a feedforward neural network (FFN) for prediction processing. The feedforward neural network serves as the prediction head, and the prediction processing is set as a multi-task, including classification and regression tasks. The classification task predicts the category label, obtaining the RNA transcription site and cell background for each frame of the cell image. The regression task predicts the location of the RNA transcription site, obtaining the normalized center coordinates of the RNA transcription site, forming the corresponding RNA site box prediction bounding box in each frame of the cell image.

[0075] In classification tasks, a feedforward neural network (FFN) is used as the prediction head to predict class labels for N feature sequences. The feedforward neural network consists of three fully connected neural networks. The first two layers each include a linear layer and a corresponding ReLU activation function. The last linear layer uses the softmax function to predict the class label, that is, to distinguish between the foreground and the background. The foreground is the RNA transcription site and the background is the cell.

[0076] In the regression task, regression prediction of RNA transcription site locations is performed for N feature sequences to obtain the normalized center coordinates of the RNA transcription sites, including the height and width of the predicted bounding box. This enables the detection of RNA transcription sites in cell images.

[0077] In this embodiment of the invention, a pre-trained 2D U-net model is called before detection to extract feature maps, temporal location features, horizontal location features, and vertical location features from a multi-frame cell image sequence.

[0078] Then, the feature map and 3D location features are encoded and decoded, and finally, class label prediction and RNA transcription site location prediction are performed. This enables the detection of RNA transcription sites in a dynamic environment.

[0079] Compared to existing single-frame cell image detection methods at the two-dimensional level, the embodiments of the present invention combine three-dimensional positional features between multiple frames of images to help predict RNA transcription sites, thereby effectively improving the detection results.

[0080] To further improve the detection performance of RNA transcription sites, this invention designs an RNA transcription site detection model to detect RNA transcription sites, cellular background, and the normalized center coordinates of RNA transcription sites. The RNA transcription site detection model includes a pre-trained 2D U-net network, a Transformer encoder, a Transformer decoder, and a feedforward network. The training process of the RNA transcription site detection model is described below.

[0081] First, cell image samples are acquired, which consist of cell time-series images (including multiple single-frame cell images). The number of RNA transcription sites to be detected in the cell image samples is n, and the cell image samples (i.e., cell time-series images) carry true labels for the RNA transcription sites, denoted as y. .

[0082] Next, the cell image samples are input into the RNA transcription site detection model for forward propagation to obtain the predicted labels of RNA transcription sites in the cell image samples.

[0083] The process of forward propagation can be found in [reference needed]. Figure 4 First, based on single-frame cell images from the cell image samples, a 2D U-net network is trained using manually labeled RNA transcription site mask data to obtain the RNA transcription site detection model. After the cell image samples (i.e., cell time-series images) are input into the RNA transcription site detection model, feature extraction is performed through the 2D U-net network to obtain the corresponding cell image feature map and 3D position encoding. Then, the cell image feature map and 3D position encoding are input into the Transformer encoder of the RNA transcription site detection model for encoding, yielding the corresponding encoding results.

[0084] During decoding, a learnable query (equivalent to the learnable instance features mentioned above) is first acquired. Here, the learnable query can be the true label of the RNA transcription site, which helps the Transformer decoder to decode. The encoding result of the Transformer encoder and the learnable query are input into the Transformer decoder of the RNA transcription site detection model for decoding processing. The decoded sequence is then input into a feedforward network for prediction processing, and finally outputs the predicted label of the RNA transcription site in the cell image sample, denoted as . .

[0085] Then the Hungarian algorithm is called to determine the predicted label. Optimal binary graph matching between the predicted label and the true label y. Specifically, the Hungarian algorithm is used to match the predicted label. Perform binary graph matching with the true label y to obtain a candidate matching ranking. After binary graph matching, multiple candidate matching arrangements are obtained. Forming permutations and combinations In this embodiment of the invention, it is necessary to consider permutations and combinations. The optimal binary graph matching is determined, that is, the candidate matching permutation with the lowest matching cost is determined, denoted as . It can be expressed as the following formula:

[0086] (2)

[0087] in, Represents permutations and combinations Any candidate matching permutation in the given list, Indicates the predicted label in the candidate matching permutation With real labels The binary graph matching cost is given by , where i represents the i-th RNA transcription site among the n RNA transcription sites to be detected.

[0088] According to the above formula (2), it is necessary to calculate the permutation of each candidate match. Bipartite graph matching cost Therefore, the next step is to determine the candidate matching permutation. The cost of matching binary graphs is calculated below, and the process of calculating the cost of matching binary graphs is explained.

[0089] In this embodiment of the invention, the predicted label is... This includes the predicted category labels for RNA transcription sites and cellular background, as well as the predicted bounding box labels corresponding to the normalized center coordinates of the RNA transcription sites. The true labels can be represented as... .in, This represents the category prediction label (i.e., the predicted category) corresponding to the RNA transcription site and the cellular background. The labels represent the ground truth bounding boxes corresponding to the normalized center coordinates of RNA transcription sites. It is a vector that defines the center coordinates of the ground truth bounding box, as well as its relative width and height.

[0090] In candidate matching In the process, the predicted labels are first determined. The predicted probability of the category corresponding to the predicted category label is denoted as . This probability is the probability used by the feedforward network of the RNA transcription site detection model to distinguish between RNA transcription sites and cellular background when performing a classification task.

[0091] Then, in the candidate matching permutation In the process of determining the true label True bounding box labels corresponding to the normalized center coordinates of RNA transcription sites and determine the actual frame label With prediction box labels The bounding box loss value between them is denoted as The bounding box loss can be calculated by means of absolute error or mean square error.

[0092] Finally, the difference between the bounding box loss value and the class prediction probability is used as the binary graph matching cost of the candidate matching arrangement, expressed as the following formula (3):

[0093] (3)

[0094] The candidate matching permutation is calculated using formula (3). Bipartite graph matching cost Then, the target matching arrangement with the lowest binary graph matching cost is determined from the candidate matching arrangements, and is taken as the optimal binary graph matching. Here, the result calculated by formula (3) is substituted into the above formula (2) to obtain the result from the permutation and combination. The target matching permutation with the lowest binary graph matching cost is determined from the candidate matching permutations. .

[0095] Target matching permutation The binary graph matching cost is also known as the optimal binary graph matching cost. The true label is determined based on the optimal binary graph matching cost. With predictive labels Hungarian loss value It is expressed as the following formula (4):

[0096] (4)

[0097] In the above formula (4), Indicates the arrangement of target matches Below, predict labels The predicted category probability corresponding to the predicted category label. Indicates real label The ground truth bounding box labels corresponding to the normalized center coordinates of the RNA transcription site. Indicates the arrangement of target matches Below, predict labels The predicted bounding box labels corresponding to the normalized center coordinates of the RNA transcription site.

[0098] Finally, the Hungarian loss value was calculated. The training loss value is used for backpropagation in the RNA transcription site detection model to update its parameters. During backpropagation, the gradient of the RNA transcription site detection model is calculated using the Hungarian loss value, and the gradient is optimized using a gradient optimization algorithm to update the model's parameters. Training ends when a preset number of iterations is reached or the Hungarian loss value begins to converge.

[0099] like Figure 5 As shown, the trained RNA transcription site detection model can directly detect RNA transcription sites in multiple cell images, ultimately identifying RNA transcription sites and the cell background from the cell images. It also performs location prediction to determine the bounding box of the RNA transcription site, thereby realizing the identification and localization of RNA transcription sites in cell images.

[0100] The following describes the multi-frame single-cell sequence RNA transcription site detection device provided by the present invention. The multi-frame single-cell sequence RNA transcription site detection device described below can be referred to in correspondence with the multi-frame single-cell sequence RNA transcription site detection method described above.

[0101] like Figure 6 As shown, the multi-frame single-cell sequence RNA transcription site detection device includes an image feature extraction module 601, a sequence feature extraction module 602, and a detection module 603. Specifically, the image feature extraction module 601 is used to acquire the multi-frame cell image sequence to be detected, and call a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features. The three-dimensional position features include temporal position features, horizontal position features, and vertical position features. The sequence feature extraction module 602 is used to encode the feature maps and the three-dimensional position features to obtain encoded feature sequences, and decode the encoded feature sequences to obtain high-level pixel feature sequences. The detection module 603 is used to perform category label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequences to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each frame of cell image.

[0102] It should be noted that the beneficial effects of the multi-frame single-cell sequence RNA transcription site detection device here correspond to those of the multi-frame single-cell sequence RNA transcription site detection method mentioned above. Therefore, the beneficial effects of the multi-frame single-cell sequence RNA transcription site detection device will not be elaborated here.

[0103] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a multi-frame single-cell sequence RNA transcription site detection method. This method includes: acquiring a multi-frame cell image sequence to be detected, and calling a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features, wherein the three-dimensional position features include temporal position features, horizontal position features, and vertical position features; encoding the feature maps and the three-dimensional position features to obtain an encoded feature sequence, and decoding the encoded feature sequence to obtain a high-level pixel feature sequence; performing category label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequence to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each frame of the cell image.

[0104] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-frame single-cell sequence RNA transcription site detection method provided by the above methods. The method includes: acquiring a multi-frame cell image sequence to be detected, and calling a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features, wherein the three-dimensional position features include temporal position features, horizontal position features, and vertical position features; encoding the feature maps and the three-dimensional position features to obtain encoded feature sequences, and decoding the encoded feature sequences to obtain high-level pixel feature sequences; and performing class label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequences to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each cell image.

[0106] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for detecting RNA transcription sites in multi-frame single-cell sequences provided by the methods described above. The method includes: acquiring a multi-frame cell image sequence to be detected, and calling a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features, wherein the three-dimensional position features include temporal position features, horizontal position features, and vertical position features; encoding the feature maps and the three-dimensional position features to obtain encoded feature sequences, and decoding the encoded feature sequences to obtain high-level pixel feature sequences; performing category label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequences to obtain the RNA transcription site, cell background, and normalized center coordinates of the RNA transcription site for each cell image frame.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting RNA transcription sites in multi-frame single-cell sequences, characterized in that, The method includes: A multi-frame cell image sequence to be detected is obtained, and a pre-trained 2D U-net model is called to extract features from the multi-frame cell image sequence to obtain corresponding feature maps and three-dimensional position features. The three-dimensional position features include temporal position features, horizontal position features, and vertical position features. The feature map and the three-dimensional position features are encoded to obtain an encoded feature sequence, and the encoded feature sequence is decoded to obtain a high-level pixel feature sequence. For each feature sequence in the high-level pixel feature sequence, class label prediction and RNA transcription site location prediction are performed respectively to obtain the RNA transcription site, cell background and normalized center coordinates of the RNA transcription site for each frame of cell image; The process of using a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence yields corresponding feature maps and three-dimensional position features, including: For each cell image in the multi-frame cell image sequence, a pre-trained 2D U-net model is invoked to extract features from the cell image to obtain a feature map of the cell image; For the multi-frame cell image sequence, the pre-trained 2D U-net model is invoked to extract the temporal position features, horizontal position features, and vertical position features between cell image frames as three-dimensional position features.

2. The method for detecting RNA transcription sites in multiple single-cell sequences according to claim 1, characterized in that, The encoding process of the feature map and the three-dimensional position features to obtain the encoded feature sequence includes: A convolutional network with a 1×1 kernel is invoked to perform convolution processing on each of the feature maps to obtain a convolutional feature map. The feature dimension of the convolutional feature map is then mapped to the encoder dimension to obtain a two-dimensional feature map. The two-dimensional feature map and the three-dimensional position features are input into the multi-head self-attention layer of the encoder for self-attention calculation, and the calculated result is input into the feedforward network layer of the encoder for mapping processing to obtain the encoded feature sequence.

3. The method for detecting RNA transcription sites in multiple single-cell sequences according to claim 1, characterized in that, Decoding the encoded feature sequence to obtain a high-level pixel feature sequence includes: Learnable instance features are obtained, wherein the learnable instance features are obtained from the pixel features of cell images in the multi-frame cell image sequence, and the number of features of the learnable instance features is N, where N is n×T, n is the number of cell images in the multi-frame cell image sequence, and T is the number of RNA transcription sites to be detected in each cell image; The encoded feature sequence and the learnable instance features are input into the cross-attention layer of the decoder for cross-attention calculation, and the calculated result is input into the feedforward network layer of the decoder for mapping processing to obtain a high-level pixel feature sequence composed of N feature sequences.

4. The method for detecting RNA transcription sites in multiple single-cell sequences according to claim 1, characterized in that, The RNA transcription site, the cellular background, and the normalized center coordinates of the RNA transcription site were obtained using an RNA transcription site detection model. The training process of the RNA transcription site detection model includes: Obtain cell image samples, wherein the number of RNA transcription sites to be detected in the cell image samples is n, and the cell image samples carry real labels of RNA transcription sites; The cell image samples are input into the RNA transcription site detection model for forward propagation to obtain the predicted labels of RNA transcription sites in the cell image samples; The Hungarian algorithm is invoked to determine the optimal binary graph match between the true label and the predicted label; The Hungarian loss values ​​of the true label and the predicted label are determined based on the optimal binary graph matching, and the Hungarian loss values ​​are used as training loss values ​​for backpropagation in the RNA transcription site detection model to update the parameters of the RNA transcription site detection model.

5. The method for detecting RNA transcription sites in multiple single-cell sequences according to claim 4, characterized in that, The step of using the Hungarian algorithm to determine the optimal binary graph match between the true label and the predicted label includes: The candidate matching order is obtained by performing binary graph matching between the real labels and the predicted labels using the Hungarian algorithm. Determine the binary graph matching cost of the candidate matching permutation; The target matching arrangement with the lowest binary graph matching cost is determined from the candidate matching arrangement and is taken as the optimal binary graph matching.

6. The method for detecting RNA transcription sites in multiple single-cell sequences according to claim 5, characterized in that, The predicted labels include RNA transcription sites, category predicted labels corresponding to the cell background, and predicted box labels corresponding to the normalized center coordinates of the RNA transcription sites. The binary graph matching cost for determining the candidate matching arrangement includes: In the candidate matching permutation, the category prediction probability corresponding to the category prediction label in the predicted label is determined; In the candidate matching arrangement, the true bounding box label corresponding to the normalized center coordinates of the RNA transcription site in the true label is determined, and the bounding box loss value between the true bounding box label and the predicted bounding box label is determined. The difference between the bounding box loss value and the predicted probability of the category is used as the binary graph matching cost of the candidate matching permutation.

7. A device for detecting RNA transcription sites in multi-frame single-cell sequences, characterized in that, The device includes: The image feature extraction module is used to acquire a multi-frame cell image sequence to be detected, and to call a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence to obtain the corresponding feature map and three-dimensional position features. The three-dimensional position features include temporal position features, horizontal position features and vertical position features. The sequence feature extraction module is used to encode the feature map and the three-dimensional position features to obtain an encoded feature sequence, and to decode the encoded feature sequence to obtain an advanced pixel feature sequence. The detection module is used to perform category label prediction and RNA transcription site location prediction for each feature sequence in the high-level pixel feature sequence, so as to obtain the RNA transcription site, cell background and normalized center coordinates of the RNA transcription site for each frame of cell image; The process of using a pre-trained 2D U-net model to extract features from the multi-frame cell image sequence yields corresponding feature maps and three-dimensional position features, including: For each cell image in the multi-frame cell image sequence, a pre-trained 2D U-net model is invoked to extract features from the cell image to obtain a feature map of the cell image; For the multi-frame cell image sequence, the pre-trained 2D U-net model is invoked to extract the temporal position features, horizontal position features, and vertical position features between cell image frames as three-dimensional position features.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method for detecting RNA transcription sites in multiple frames of single-cell sequences as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for detecting RNA transcription sites in multiple frames of single-cell sequences as described in any one of claims 1 to 6.

Images