A processing method and device of an APA prediction model combined with a pre-training model

Abstract

The embodiment of the application relates to a processing method and device of an APA prediction model combined with a pre-training model, the method comprising: constructing an APA prediction model; and training the APA prediction model; after the model training is completed, receiving a model application mode and a first RNA sequence input by a user; and identifying the model application mode; if the model application mode is a first mode, performing APA phenomenon prediction on the input first RNA sequence based on the APA prediction model; and if the model application mode is a second mode, performing APA phenomenon prediction on the input first RNA sequence based on the APA prediction model and performing APA site screening based on a prediction result. Through the application, an APA site prediction means based on artificial intelligence deep learning technology can be provided, and a pre-training model is introduced to reduce model training difficulty and improve model prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a processing method and apparatus for an APA prediction model that combines a pre-trained model. Background Technology

[0002] Alternative polyadenylation (APA) is an important mechanism for regulating the 3' end of ribonucleic acid (RNA) molecules. An APA site is the location where a poly(A) tail is selectively added to the 3' end of messenger RNA (mRNA) during gene expression. APA can derive multiple RNA isoforms with different 3' ends from a single gene. APA plays a crucial role in mRNA stability, cellular localization, and protein translation regulation. Accurate prediction of APA sites helps reveal the complexity and dynamics of gene expression regulation, providing key information for further research on biological processes. In recent years, with the in-depth development of artificial intelligence (AI) technology, more and more research has begun to focus on AI-based APA site prediction methods. The technical problem this invention aims to solve is how to combine AI technology to provide a predictive model that can perform APA site prediction tasks. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing a processing method, apparatus, electronic device, and computer-readable storage medium for an APA prediction model that combines a pre-trained model. This invention constructs an APA prediction model by combining a pre-trained model and a Deep Nueral Network (DNN) model. Based on this APA prediction model, it predicts APA phenomena in input RNA sequences and outputs two types of prediction results: isoform abundance and splice site probability distribution. Furthermore, it screens APA sites based on the splice site probability distribution. This invention provides an APA site prediction method based on artificial intelligence deep learning technology by constructing an APA prediction model. Introducing a mature pre-trained model into the prediction model reduces the difficulty of model training and improves the accuracy of model prediction.

[0004] To achieve the above objectives, a first aspect of the present invention provides a method for processing an APA prediction model that combines a pre-trained model, the method comprising:

[0005] An APA prediction model is constructed and trained. The APA prediction model is used to predict APA phenomena from input target RNA sequences. The APA prediction model includes a pre-trained model, a feature reduction network, and a DNN model. The input of the pre-trained model is connected to the input of the APA prediction model, and its output is connected to the input of the feature reduction network. The output of the feature reduction network is connected to the input of the DNN model. The output of the DNN model is connected to the output of the APA prediction model.

[0006] After the model training is completed, the user inputs the model application mode and the first RNA sequence; and the model application mode is identified; if the model application mode is the first mode, the APA phenomenon is predicted on the input first RNA sequence based on the APA prediction model; if the model application mode is the second mode, the APA phenomenon is predicted on the input first RNA sequence based on the APA prediction model and APA site screening is performed based on the prediction results.

[0007] Preferably, the pre-trained model is implemented based on the transformer model structure; the pre-trained model is used to perform semantic feature encoding processing on the sequence bases in the target RNA sequence to generate corresponding encoded feature tensors and send them to the encoded feature dimensionality reduction network.

[0008] Preferably, the encoded feature dimensionality reduction network is implemented based on a linear neural network; the encoded feature dimensionality reduction network is used to perform feature dimensionality reduction processing on the encoded feature tensor to generate a corresponding dimensionality-reduced feature tensor and send it to the DNN model.

[0009] Preferably, the DNN model is implemented based on the APARENT model structure; the DNN model includes a first convolutional network, a second convolutional network, a fully connected network, a first prediction network, and a second prediction network; the first and second convolutional networks are each composed of two layers of convolutional neural networks; the inputs of the first and second convolutional networks are connected to the output of the encoding feature dimensionality reduction network; the outputs of the first and second convolutional networks are connected to the input of the fully connected network; the output of the fully connected network is connected to the input of the first and second prediction networks respectively; the outputs of the first and second prediction networks are connected to the output of the APA prediction model.

[0010] The first convolutional network is used to learn isomer features based on the reduced-dimensional feature tensor and output a corresponding first feature tensor to be sent to the fully connected network; the second convolutional network is used to learn Poly(A) signal features based on the reduced-dimensional feature tensor and output a corresponding second feature tensor to be sent to the fully connected network; the fully connected network is used to perform feature fusion processing on the first and second feature tensors to generate a corresponding fused feature tensor to be sent to the first and second prediction networks; the first prediction network is used to predict isomer abundance based on the fused feature tensor to generate a corresponding isomer abundance prediction vector as the model output; the second prediction network is used to predict the cleavage probability of sequence bases based on the fused feature tensor to generate a corresponding sequence cleavage probability prediction vector as the model output.

[0011] Preferably, training the APA prediction model specifically includes:

[0012] The pre-trained model and the DNN model of the APA prediction model are jointly trained based on the training dataset and training loss function of the APARENT model.

[0013] Preferably, the step of predicting the APA phenomenon of the input first RNA sequence based on the APA prediction model specifically includes:

[0014] The first RNA sequence is input into the APA prediction model; the pre-trained model performs semantic feature encoding on the sequence bases in the first RNA sequence to generate a corresponding first encoding feature tensor; the encoding feature dimensionality reduction network performs feature dimensionality reduction on the first encoding feature tensor to generate a corresponding first dimensionality-reduced feature tensor; the first convolutional network performs isomer feature learning based on the first dimensionality-reduced feature tensor and outputs a corresponding third feature tensor; the second convolutional network performs Poly(A) signal feature learning based on the first dimensionality-reduced feature tensor and outputs a corresponding fourth feature tensor; the fully connected network performs feature fusion processing on the third and fourth feature tensors to generate a corresponding first fusion feature tensor; the first prediction network performs isomer abundance prediction based on the first fusion feature tensor to generate a corresponding first isomer abundance prediction vector; the second prediction network predicts the cleavage probability of sequence bases based on the first fusion feature tensor to generate a corresponding first sequence cleavage probability prediction vector; and the first isomer abundance prediction vector and the first sequence cleavage probability prediction vector are output as the results of this APA phenomenon prediction.

[0015] Preferably, the step of predicting APA phenomena based on the APA prediction model for the input first RNA sequence and screening APA sites based on the prediction results specifically includes:

[0016] The first RNA sequence is input into the APA prediction model; the pre-trained model performs semantic feature encoding on the sequence bases in the first RNA sequence to generate a corresponding first encoding feature tensor; the encoding feature dimensionality reduction network performs feature dimensionality reduction on the first encoding feature tensor to generate a corresponding first dimensionality-reduced feature tensor; the first convolutional network learns isomer features based on the first dimensionality-reduced feature tensor and outputs a corresponding third feature tensor; the second convolutional network learns Poly(A) signal features based on the first dimensionality-reduced feature tensor and outputs a corresponding fourth feature tensor; the fully connected network performs feature fusion on the third and fourth feature tensors to generate a corresponding first fused feature tensor; the first prediction network predicts isomer abundance based on the first fused feature tensor to generate a corresponding second isomer abundance prediction vector; and the second prediction network predicts the cleavage probability of sequence bases based on the first fused feature tensor to generate a corresponding second sequence cleavage probability prediction vector.

[0017] In the second sequence cleavage point probability prediction vector, the probability of a cleavage point whose probability value exceeds the preset APA site probability threshold is recorded as the corresponding first probability; the sequence bases in the first RNA sequence corresponding to each of the first probabilities are recorded as the corresponding first bases; the position index of each of the first bases in the first RNA sequence is recorded as the corresponding first index; and all the first indices are sorted in descending order of the first probability to obtain the corresponding first index sequence; and the first index sequence is output as the APA site screening result for this time.

[0018] A second aspect of the present invention provides an apparatus for implementing the processing method of the APA prediction model combined with the pre-trained model described in the first aspect above, the apparatus comprising: a model building and training module and a model application module;

[0019] The model building and training module is used to build an APA prediction model and train the APA prediction model. The APA prediction model is used to predict the APA phenomenon from an input target RNA sequence. The APA prediction model includes a pre-trained model, an encoding feature reduction network, and a DNN model. The input of the pre-trained model is connected to the input of the APA prediction model, and its output is connected to the input of the encoding feature reduction network. The output of the encoding feature reduction network is connected to the input of the DNN model. The output of the DNN model is connected to the output of the APA prediction model.

[0020] The model application module is used to receive the model application mode and the first RNA sequence input by the user after the model training is completed; and to identify the model application mode; if the model application mode is the first mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model; if the model application mode is the second mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model and APA site screening is performed based on the prediction results.

[0021] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0022] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0023] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0024] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0025] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing an APA prediction model that combines a pre-trained model. The invention constructs an APA prediction model by combining a pre-trained model and a DNN model; based on this APA prediction model, it predicts APA phenomena in an input RNA sequence and outputs two types of prediction results: isoform abundance and splice site probability distribution; and it screens APA sites based on the splice site probability distribution. This invention provides an APA site prediction method based on artificial intelligence deep learning technology by constructing an APA prediction model; the invention introduces a mature pre-trained model into the prediction model, reducing the difficulty of model training and improving the model's prediction accuracy. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a processing method for an APA prediction model that combines a pre-trained model, as provided in Embodiment 1 of the present invention.

[0027] Figure 2 This is a block diagram of the APA prediction model provided in Embodiment 1 of the present invention;

[0028] Figure 3 This is a module structure diagram of a processing device for an APA prediction model that combines a pre-trained model, provided in Embodiment 2 of the present invention.

[0029] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0031] Embodiment 1 of the present invention provides a processing method for an APA prediction model that combines a pre-trained model, such as... Figure 1 The diagram illustrates a processing method for an APA prediction model combining a pre-trained model, as provided in Embodiment 1 of the present invention. This method mainly includes the following steps:

[0032] Step 1: Construct an APA prediction model and train the APA prediction model.

[0033] Specifically, this includes: Step 11, constructing an APA prediction model;

[0034] Here, the APA prediction model of this embodiment of the invention is used to predict the APA phenomenon of the input target RNA sequence;

[0035] The APA prediction model of this invention is as follows: Figure 2 The module structure diagram of the APA prediction model provided in Embodiment 1 of the present invention is shown, including: a pre-trained model, an encoding feature reduction network, and a DNN model; the input end of the pre-trained model is connected to the input end of the APA prediction model, and the output end is connected to the input end of the encoding feature reduction network; the output end of the encoding feature reduction network is connected to the input end of the DNN model; the output end of the DNN model is connected to the output end of the APA prediction model.

[0036] The pre-trained model in this embodiment of the invention is based on the transformer model structure. This pre-trained model includes models such as Uni-Fold, ESM series models, BERT models, and GPT models, whose model structures and parameters are publicly disclosed. The pre-trained model in this embodiment of the invention is used to perform semantic feature encoding on sequence bases in the target RNA sequence, generating corresponding encoding feature tensors which are then sent to the encoding feature dimensionality reduction network. It should be noted that when constructing the APA prediction model, this embodiment of the invention initializes the parameters of the pre-trained model based on the publicly available model parameters. Since these models are matured through large-scale data training, this operation not only improves the overall prediction accuracy of the APA prediction model but also reduces the overall training difficulty and improves training efficiency.

[0037] The coding feature dimensionality reduction network in this embodiment of the invention is implemented based on a linear neural network; the coding feature dimensionality reduction network is used to perform feature dimensionality reduction processing on the coding feature tensor to generate the corresponding dimensionality-reduced feature tensor and send it to the DNN model;

[0038] The DNN model in this embodiment of the invention refers to the model structure and function implementation of the APARENT model. For details on the model structure, inference process, and function description of the APARENT model, please refer to the technical document "A Deep Neural Network for Predicting and Engineering Alternative Polyadenylation," which will not be repeated here. The DNN model in this embodiment of the invention includes a first convolutional network, a second convolutional network, a fully connected network, a first prediction network, and a second prediction network. The inputs of the first and second convolutional networks are both connected to the output of the encoding feature reduction network. The outputs of the first and second convolutional networks are both connected to the input of the fully connected network. The output of the fully connected network is connected to the inputs of the first and second prediction networks, respectively. The outputs of the first and second prediction networks are connected to the output of the APA prediction model.

[0039] The first and second convolutional networks in this embodiment of the invention are both composed of two-layer convolutional neural networks. The network design of the two-layer convolutional neural network in the first convolutional network is similar to that in the APARENT model for predicting isoform abundance, and the network design of the two-layer convolutional neural network in the second convolutional network is similar to that in the APARENT model for predicting cleavage distribution. Similar to the APARENT model, the first convolutional network in this embodiment of the invention is used to learn isoform features based on the dimensionality-reduced feature tensor and output the corresponding first feature tensor to be sent to the fully connected network. The second convolutional network in this embodiment of the invention is used to learn Poly(A) signal (PAS) features based on the dimensionality-reduced feature tensor and output the corresponding second feature tensor to be sent to the fully connected network.

[0040] The network design of the fully connected network in this embodiment of the invention is similar to the Dense Layer in the APARENT model. It is used to perform feature fusion processing on the first and second feature tensors to generate corresponding fused feature tensors and send them to the first and second prediction networks.

[0041] The first prediction network in this embodiment of the invention is based on a linear regression network and is used to implement the isomer abundance processing in the APARENT model; that is, the first prediction network in this embodiment of the invention is used to generate the corresponding isomer abundance prediction vector as the model output based on the fusion feature tensor to predict isomer abundance.

[0042] The second prediction network in this embodiment of the invention is based on a linear regression network and is used to process the distribution of cut points in the APARENT model; that is, the second prediction network in this embodiment of the invention is used to predict the probability of cut points of sequence bases based on the fused feature tensor and generate the corresponding sequence cut point probability prediction vector as the model output.

[0043] Step 12: Train the APA prediction model;

[0044] Here, in the embodiment of the present invention, when training the APA prediction model, the pre-trained model and the DNN model of the APA prediction model are jointly trained based on the training dataset and training loss function of the APARENT model. The so-called joint training means that after calculating the training loss value based on the training loss function of the APARENT model, the parameters of the DNN model and the pre-trained model are modulated based on the inverse modulation method of the APARENT model.

[0045] Step 2: After model training is complete, the system receives the user-input model application mode and the first RNA sequence; and identifies the model application mode. If the model application mode is the first mode, the system predicts the APA phenomenon of the input first RNA sequence based on the model application mode; if the model application mode is the second mode, the system predicts the APA phenomenon of the input first RNA sequence based on the model application mode and screens for APA sites based on the prediction results; if the model application mode is the third mode, the system modifies the sequence of the input first RNA sequence based on the APA prediction model.

[0046] Specifically, this includes: Step 21, receiving the user-inputted model application mode and first RNA sequence;

[0047] Here, this embodiment of the invention also provides users with three application tools based on the APARENT model by configuring model application modes and corresponding processing flows; the model application modes include a first mode, a second mode, and a third mode. Under normal circumstances, the first and second modes are mandatory configurations, while the third mode is an optional configuration; when the model application mode is the first mode, the user is provided with an APARENT model prediction application tool; when the model application mode is the second mode, the user is provided with an APA site screening application tool; when the model application mode is the optional third mode, the user is provided with an RNA sequence modification application tool.

[0048] Step 22: Identify the application patterns of the model;

[0049] Step 23: If the model application mode is the first mode, then predict the APA phenomenon of the input first RNA sequence based on the APA prediction model.

[0050] Specifically, the process includes: inputting a first RNA sequence into an APA prediction model; using a pre-trained model to perform semantic feature encoding on the sequence bases in the first RNA sequence to generate a corresponding first encoded feature tensor; using an encoded feature dimensionality reduction network to perform feature dimensionality reduction on the first encoded feature tensor to generate a corresponding first dimensionality-reduced feature tensor; using a first convolutional network to learn isomer features based on the first dimensionality-reduced feature tensor and output a corresponding third feature tensor; using a second convolutional network to learn Poly(A) signal features based on the first dimensionality-reduced feature tensor and output a corresponding fourth feature tensor; using a fully connected network to perform feature fusion on the third and fourth feature tensors to generate a corresponding first fused feature tensor; using a first prediction network to predict isomer abundance based on the first fused feature tensor to generate a corresponding first isomer abundance prediction vector; using a second prediction network to predict the cleavage probability of sequence bases based on the first fused feature tensor to generate a corresponding first sequence cleavage probability prediction vector; and outputting the first isomer abundance prediction vector and the first sequence cleavage probability prediction vector as the results of this APA phenomenon prediction.

[0051] Here, in this embodiment of the invention, the first RNA sequence, the first isomer abundance prediction vector, and the first sequence splice point probability prediction vector will be visualized to the user through text, tables, or charts;

[0052] Step 24: If the model application mode is the second mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model and the APA site is screened based on the prediction results.

[0053] Specifically, the steps include: Step 241, inputting the first RNA sequence into the APA prediction model; and using the pre-trained model to perform semantic feature encoding on the sequence bases in the first RNA sequence to generate the corresponding first encoding feature tensor; and using the encoding feature dimensionality reduction network to perform feature dimensionality reduction on the first encoding feature tensor to generate the corresponding first dimensionality-reduced feature tensor; and using the first convolutional network to learn isomer features based on the first dimensionality-reduced feature tensor and output the corresponding third feature tensor; and using the second convolutional network to learn Poly(A) signal features based on the first dimensionality-reduced feature tensor and output the corresponding fourth feature tensor; and using the fully connected network to perform feature fusion on the third and fourth feature tensors to generate the corresponding first fused feature tensor; and using the first prediction network to predict isomer abundance based on the first fused feature tensor to generate the corresponding second isomer abundance prediction vector; and using the second prediction network to predict the cleavage point probability of the sequence bases based on the first fused feature tensor to generate the corresponding second sequence cleavage point probability prediction vector.

[0054] Step 242: In the second sequence splice site probability prediction vector, the splice site probability that exceeds the preset APA site probability threshold is recorded as the corresponding first probability; the sequence bases in the first RNA sequence corresponding to each first probability are recorded as the corresponding first bases; the position index of each first base in the first RNA sequence is recorded as the corresponding first index; and all first indices are sorted in descending order of first probability to obtain the corresponding first index sequence; and the first index sequence is output as the APA site screening result for this time.

[0055] Here, in this embodiment of the invention, the obtained first RNA sequence, first isomer abundance prediction vector, first sequence cut point probability prediction vector, first index sequence, and alignment markers of each first index in the first RNA sequence will be visualized to the user through text, tables, or charts.

[0056] Step 25: If the model application mode is the third mode, then the first RNA sequence is sequence modified based on the APA prediction model.

[0057] Here, this embodiment of the invention also provides users with an RNA sequence modification application tool based on the SeqProp algorithm, namely: a target function F (PWM, sequence cut-point probability prediction vector) related to the position weight matrix (PWM) of the input RNA sequence and the output sequence cut-point probability prediction vector is pre-set; then, based on the differentiability of the neural network, a gradient calculation method for the target function F with respect to PWM is set; then, the PWM is optimized iteratively in the direction that maximizes the target function F through gradient ascent iterative method, and each iteration outputs an optimized PWM corresponding to a new RNA sequence; the RNA sequence corresponding to the optimized PWM obtained at the end of the iteration is taken as the final output modified sequence; after obtaining the modified sequence, the first RNA sequence and the modified sequence are displayed by head base alignment; it should be noted that this embodiment of the invention gives a constraint in the optimization iteration process: only the part of the PWM corresponding to the 3' end structure of the RNA sequence can be optimized.

[0058] It should also be noted that, in this embodiment of the invention, the position weight matrix PWM can be replaced with a weight matrix generated based on other rules, such as a sensitivity weight matrix generated based on sensitivity analysis, a perturbation probability weight matrix generated based on adversarial training with artificial perturbation, or a weight matrix constructed based on feature elimination probability weight or posable probability interchange weight, etc. When using other weight matrices for sequence modification, similar to using positional weight matrices (PWM): a target function is pre-set that is related to other regular weight matrices of the input RNA sequence and the output sequence cut-point probability prediction vector and / or isomer abundance prediction vector; then, based on the differentiability of the neural network, a gradient calculation method for the target function relative to other regular weight matrices is set; then, other weight matrices are optimized iteratively in the direction that makes the target function F reach the set extremum (maximum or minimum value) through a specified gradient ascent or descent iteration method, and the optimized weight matrix output in each iteration corresponds to a new RNA sequence; the RNA sequence corresponding to the optimized weight matrix obtained at the end of the iteration is taken as the final modified sequence; and a constraint is given in the optimization process: only the part of other weight matrices corresponding to the 3' end structure of the RNA sequence can be optimized; and after obtaining the modified sequence, the first RNA sequence and the modified sequence are aligned at the head bases for display.

[0059] Figure 3 This is a block diagram of a processing device for an APA prediction model combined with a pre-trained model, provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiment, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiment. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes a model building and training module 201 and a model application module 202.

[0060] The model building and training module 201 is used to build an APA prediction model and train the APA prediction model. The APA prediction model is used to predict the APA phenomenon of the input target RNA sequence. The APA prediction model includes a pre-trained model, an encoding feature reduction network, and a DNN model. The input of the pre-trained model is connected to the input of the APA prediction model, and the output is connected to the input of the encoding feature reduction network. The output of the encoding feature reduction network is connected to the input of the DNN model. The output of the DNN model is connected to the output of the APA prediction model.

[0061] The model application module 202 is used to receive the model application mode and the first RNA sequence input by the user after the model training is completed; and to identify the model application mode; if the model application mode is the first mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model; if the model application mode is the second mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model and APA site screening is performed based on the prediction results.

[0062] The processing device for an APA prediction model combined with a pre-trained model provided in this embodiment of the invention can execute the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0063] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing elements; they can be fully implemented in hardware; or some modules can be implemented by processing elements calling software, while others are implemented in hardware. For example, the model building and training module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0064] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0065] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0066] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 4As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0067] exist Figure 4 The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0068] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0069] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0070] This invention also provides a chip for executing instructions, which is used to perform the processing steps described in the foregoing method embodiments.

[0071] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for processing an APA prediction model that combines a pre-trained model. The invention constructs an APA prediction model by combining a pre-trained model and a DNN model; based on this APA prediction model, it predicts APA phenomena in an input RNA sequence and outputs two types of prediction results: isoform abundance and splice site probability distribution; and it screens APA sites based on the splice site probability distribution. This invention provides an APA site prediction method based on artificial intelligence deep learning technology by constructing an APA prediction model; the invention introduces a mature pre-trained model into the prediction model, reducing the difficulty of model training and improving the model's prediction accuracy.

[0072] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0073] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0074] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing APA prediction models that combine pre-trained models, characterized in that, The method includes: An APA prediction model is constructed and trained. The APA prediction model is used to predict APA phenomena from input target RNA sequences. The APA prediction model includes a pre-trained model, a feature reduction network, and a DNN model. The input of the pre-trained model is connected to the input of the APA prediction model, and its output is connected to the input of the feature reduction network. The output of the feature reduction network is connected to the input of the DNN model. The output of the DNN model is connected to the output of the APA prediction model. After the model training is completed, the user inputs the model application mode and the first RNA sequence; and the model application mode is identified; if the model application mode is the first mode, the APA phenomenon is predicted on the input first RNA sequence based on the APA prediction model; if the model application mode is the second mode, the APA phenomenon is predicted on the input first RNA sequence based on the APA prediction model and APA site screening is performed based on the prediction results. The pre-trained model is implemented based on the transformer model structure. It is used to perform semantic feature encoding on the sequence bases of the target RNA sequence to generate corresponding encoded feature tensors, which are then sent to the encoded feature dimensionality reduction network. The encoded feature dimensionality reduction network is implemented based on a linear neural network. It is used to perform feature dimensionality reduction on the encoded feature tensors to generate corresponding dimensionality-reduced feature tensors, which are then sent to the DNN model. The DNN model is implemented based on the APARENT model structure and includes a first convolutional network, a second convolutional network, a fully connected network, a first prediction network, and a second prediction network. If the model application mode is the first mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model, specifically including: The first RNA sequence is input into the APA prediction model; the pre-trained model performs semantic feature encoding on the sequence bases in the first RNA sequence to generate a corresponding first encoding feature tensor; the encoding feature dimensionality reduction network performs feature dimensionality reduction on the first encoding feature tensor to generate a corresponding first dimensionality-reduced feature tensor; the first convolutional network learns isomer features based on the first dimensionality-reduced feature tensor and outputs a corresponding third feature tensor; the second convolutional network learns Poly(A) signal features based on the first dimensionality-reduced feature tensor and outputs a corresponding fourth feature tensor; the fully connected network performs feature fusion on the third and fourth feature tensors to generate a corresponding first fused feature tensor; the first prediction network predicts isomer abundance based on the first fused feature tensor to generate a corresponding first isomer abundance prediction vector; the second prediction network predicts the cleavage probability of sequence bases based on the first fused feature tensor to generate a corresponding first sequence cleavage probability prediction vector; and the first isomer abundance prediction vector and the first sequence cleavage probability prediction vector are output as the results of this APA phenomenon prediction. If the model application mode is the second mode, then based on the APA prediction model, the input first RNA sequence is used to predict the APA phenomenon and the APA site is screened based on the prediction results, specifically including: The first RNA sequence is input into the APA prediction model; the pre-trained model performs semantic feature encoding on the sequence bases in the first RNA sequence to generate a corresponding first encoding feature tensor; the encoding feature dimensionality reduction network performs feature dimensionality reduction on the first encoding feature tensor to generate a corresponding first dimensionality-reduced feature tensor; the first convolutional network learns isomer features based on the first dimensionality-reduced feature tensor and outputs a corresponding third feature tensor; the second convolutional network learns Poly(A) signal features based on the first dimensionality-reduced feature tensor and outputs a corresponding fourth feature tensor; the fully connected network performs feature fusion on the third and fourth feature tensors to generate a corresponding first fused feature tensor; the first prediction network predicts isomer abundance based on the first fused feature tensor to generate a corresponding second isomer abundance prediction vector; and the second prediction network predicts the cleavage probability of sequence bases based on the first fused feature tensor to generate a corresponding second sequence cleavage probability prediction vector. In the second sequence cleavage point probability prediction vector, the probability of a cleavage point whose probability value exceeds the preset APA site probability threshold is recorded as the corresponding first probability; the sequence bases in the first RNA sequence corresponding to each of the first probabilities are recorded as the corresponding first bases; the position index of each of the first bases in the first RNA sequence is recorded as the corresponding first index; and all the first indices are sorted in descending order of the first probability to obtain the corresponding first index sequence; and the first index sequence is output as the APA site screening result for this time. The model application modes include a first mode, a second mode, and a third mode. The first and second modes are mandatory configurations, while the third mode is an optional configuration. If the model application mode is the third mode, then the first input RNA sequence is modified based on the APA prediction model. Specifically, the following steps are taken: First, a position weight matrix PWM related to the input RNA sequence and an objective function F (PWM, sequence cut-point probability prediction vector) related to the sequence cut-point probability prediction vector are set. Then, based on the differentiability of the neural network, a gradient calculation method for the objective function F on the weight matrix PWM is set. Then, the weight matrix PWM is optimized iteratively in the direction that maximizes the objective function F through gradient ascent. Each iteration outputs an optimized PWM corresponding to a new RNA sequence. During the optimization iteration, a constraint is given: only the part of the weight matrix PWM corresponding to the 3' end structure of the RNA sequence can be optimized. The RNA sequence corresponding to the optimized PWM obtained at the end of the iteration is taken as the final output modified sequence. After obtaining the modified sequence, the first RNA sequence and the modified sequence are displayed with head base alignment. In addition to the position weight matrix, the weight matrix type of the weight matrix PWM also includes weight matrices generated based on other rules; the other weight matrices specifically include sensitivity weight matrices generated based on sensitivity analysis, perturbation probability weight matrices generated based on adversarial training with artificial perturbation, and weight matrices constructed based on feature elimination probability weights or permutable probability weights; When using the other weight matrices for sequence modification: first, set an objective function related to the current other weight matrices, the sequence cut-point probability prediction vector, and / or the isomer abundance prediction vector; then, based on the differentiability of the neural network, set a gradient calculation method for the current objective function relative to the current weight matrix; then, optimize the current weight matrix by using a specified gradient ascent or descent iteration method in the direction that makes the current objective function reach a set extremum, with each iteration outputting an optimized weight matrix corresponding to a new RNA sequence; and give constraints during the optimization iteration process: only the part of the current weight matrix corresponding to the 3' end structure of the RNA sequence can be optimized; and take the RNA sequence corresponding to the optimized weight matrix obtained at the end of the iteration as the final output modified sequence; and after obtaining the modified sequence, perform head base alignment display on the first RNA sequence and the modified sequence.

2. The processing method for the APA prediction model combined with the pre-trained model according to claim 1, characterized in that, Both the first and second convolutional networks consist of two layers of convolutional neural networks; the inputs of both the first and second convolutional networks are connected to the output of the encoding feature dimensionality reduction network; the outputs of both the first and second convolutional networks are connected to the input of the fully connected network; the output of the fully connected network is connected to the inputs of the first and second prediction networks respectively; and the outputs of the first and second prediction networks are connected to the output of the APA prediction model. The first convolutional network is used to learn isomer features based on the reduced-dimensional feature tensor and output a corresponding first feature tensor to be sent to the fully connected network; the second convolutional network is used to learn Poly(A) signal features based on the reduced-dimensional feature tensor and output a corresponding second feature tensor to be sent to the fully connected network; the fully connected network is used to perform feature fusion processing on the first and second feature tensors to generate a corresponding fused feature tensor to be sent to the first and second prediction networks; the first prediction network is used to predict isomer abundance based on the fused feature tensor to generate a corresponding isomer abundance prediction vector as the model output; the second prediction network is used to predict the cleavage probability of sequence bases based on the fused feature tensor to generate a corresponding sequence cleavage probability prediction vector as the model output.

3. The processing method for the APA prediction model combined with the pre-trained model according to claim 2, characterized in that, The training of the APA prediction model specifically includes: The pre-trained model and the DNN model of the APA prediction model are jointly trained based on the training dataset and training loss function of the APARENT model.

4. An apparatus for performing the processing method of the APA prediction model combined with the pre-trained model as described in any one of claims 1-3, characterized in that, The device includes: a model building and training module and a model application module; The model building and training module is used to build an APA prediction model and train the APA prediction model. The APA prediction model is used to predict the APA phenomenon from an input target RNA sequence. The APA prediction model includes a pre-trained model, an encoding feature reduction network, and a DNN model. The input of the pre-trained model is connected to the input of the APA prediction model, and its output is connected to the input of the encoding feature reduction network. The output of the encoding feature reduction network is connected to the input of the DNN model. The output of the DNN model is connected to the output of the APA prediction model. The model application module is used to receive the model application mode and the first RNA sequence input by the user after the model training is completed; and to identify the model application mode; if the model application mode is the first mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model; if the model application mode is the second mode, then the APA phenomenon prediction is performed on the input first RNA sequence based on the APA prediction model and APA site screening is performed based on the prediction results.

5. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-3; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-3.

Images