Neural network model training method and device, electronic equipment and storage medium

Abstract

The invention provides a neural network model training method and device, electronic equipment and a storage medium, and relates to the technical field of biological information. The method specifically comprises the following steps: acquiring a data set, wherein data in the data set is protein sequence data containing a Kcr site; and according to the data set, based on an initial model, obtaining a target neural network model with a function of identifying the Kcr site in the protein sequence, the initial model comprising an ESM2 model. According to the scheme, the neural network model with the function of identifying the Kcr site in the protein sequence can be obtained through training based on the ESM2 model, so that the Kcr site in the corresponding protein sequence can be quickly and efficiently identified according to the neural network model.

Description

Technical Field

[0001] The present disclosure relates to the field of bioinformatics, and in particular to a neural network model training method, device, electronic device and storage medium. Background Art

[0002] Post-translational modifications (PTMs) of proteins are one of the important mechanisms for regulating protein function. PTMs change the properties of proteins by covalently modifying one or more residues of proteins or by removing groups through proteases. These modifications include the widely studied phosphorylation, glycosylation, acetylation, and ubiquitination, as well as the recently discovered lysine crotonylation (Kcr). In 2011, Tan et al. first identified Kcr through mass spectrometry analysis and discovered its role in histones, which attracted widespread attention.

[0003] Kcr is of great significance in many biological fields. For example, in the field of epigenetics, Kcr modification is closely related to reproductive regulation; in disease research, Kcr modification is involved in a variety of signaling pathways and nuclear-related processes, especially in the pathogenesis of lung cancer, pancreatic cancer and other tumors; in botany, Kcr is involved in photosynthesis and a variety of metabolic pathways, and plays a key role in plant growth and development and metabolic regulation; in microbiology, Kcr is related to the movement of bacteria and parasites, host invasion and metabolic regulation. Therefore, accurate identification of Kcr sites from protein sequence data is of great significance for basic research, drug development and disease treatment.

[0004] However, traditional experimental methods for identifying Kcr sites are time-consuming and costly, limiting the efficiency and scope of research. Summary of the invention

[0005] The present disclosure provides a neural network model training method, device, electronic device and storage medium, which can obtain a neural network model with the function of identifying Kcr sites in protein sequences based on ESM2 model training, so that Kcr sites in corresponding protein sequences can be quickly and efficiently identified according to the neural network model.

[0006] According to a first aspect of the present disclosure, a neural network model training method is provided, the method comprising: obtaining a data set, wherein the data in the data set are protein sequence data containing Kcr sites; and obtaining a target neural network model having the function of identifying Kcr sites in protein sequences based on an initial model according to the data set, wherein the initial model comprises an ESM2 model.

[0007] In a possible implementation, the initial model is an ESM2 model that only retains the first four layers of attention heads.

[0008] In another possible implementation, before obtaining a target neural network model having a function of identifying Kcr sites in a protein sequence based on the data set and the initial model, the method further includes:

[0009] Splitting the data set into a training set and a test set;

[0010] The method of obtaining a target neural network model having the function of identifying Kcr sites in protein sequences based on the data set and the initial model comprises:

[0011] According to the training set and based on the initial model, a target neural network model having the function of identifying Kcr sites in protein sequences is obtained.

[0012] In another possible implementation, splitting the data set into a training set and a test set includes:

[0013] According to a preset ratio, the data set is split into the training set and the test set.

[0014] In another possible implementation, obtaining a target neural network model having a function of identifying Kcr sites in a protein sequence based on the initial model according to the training set includes:

[0015] Extracting features from the training set according to the initial model to obtain a first feature;

[0016] Perform feature extraction on the training set according to the K-mer algorithm to obtain a second feature;

[0017] Outputting, according to the main classifier based on the first feature and the second feature, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site;

[0018] Outputting, according to the first auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature;

[0019] Outputting, according to the second auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature;

[0020] Obtaining a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier, and the classification result of the second auxiliary classifier;

[0021] Based on the loss function, the initial model and the classifier are optimized to obtain the target neural network model, and the classifier includes the main classifier, the first auxiliary classifier and the second auxiliary classifier.

[0022] According to a second aspect of the present disclosure, there is provided a neural network model training device, comprising: a data processing module, for acquiring a data set, wherein the data in the data set is protein sequence data containing a Kcr site;

[0023] The model training module is used to obtain a target neural network model having the function of identifying Kcr sites in protein sequences based on the data set and the initial model, wherein the initial model includes the ESM2 model.

[0024] In a possible implementation, the initial model is an ESM2 model that only retains the first four layers of attention heads.

[0025] In another possible implementation, the data processing module is further used to split the data set into a training set and a test set;

[0026] The model training module is specifically used to obtain a target neural network model having the function of identifying Kcr sites in protein sequences based on the initial model according to the training set.

[0027] In another possible implementation manner, the data processing module is specifically configured to split the data set into the training set and the test set according to a preset ratio.

[0028] In another possible implementation, the model training module is specifically used to perform feature extraction on the training set according to the initial model to obtain a first feature; perform feature extraction on the training set according to the K-mer algorithm to obtain a second feature; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature and the second feature according to the main classifier; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature according to the first auxiliary classifier; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature according to the second auxiliary classifier; obtain a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier and the classification result of the second auxiliary classifier; based on the loss function, optimize the initial model and the classifier to obtain the target neural network model, the classifier including the main classifier, the first auxiliary classifier and the second auxiliary classifier.

[0029] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided in the first aspect.

[0030] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method provided according to the first aspect.

[0031] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the method provided according to the first aspect when executed by a processor.

[0032] The present invention can obtain a neural network model with the function of identifying Kcr sites in protein sequences based on ESM2 model training, so that the Kcr sites in the corresponding protein sequences can be quickly and efficiently identified according to the neural network model, thereby improving the efficiency of Kcr site identification in protein sequences and reducing identification costs.

[0033] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0035] Figure 1 One of the flowcharts of the neural network model training method provided in the embodiment of the present disclosure;

[0036] Figure 2 A second flowchart of a neural network model training method provided in an embodiment of the present disclosure;

[0037] Figure 3 A schematic diagram of the composition of a neural network model training device provided in an embodiment of the present disclosure;

[0038] Figure 4 A schematic block diagram of an example electronic device 400 that may be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] The neural network model training method and neural network model training device provided by the present disclosure are applicable to the case of training a target neural network model based on a data set consisting of protein sequence data containing Kcr sites. The neural network model training method provided by the present disclosure can be performed by a neural network model training device, which can be implemented by software and / or hardware and specifically configured in an electronic device, which can be a computer, a single-chip microcomputer, a server, a mobile terminal, a laptop computer, or other computing devices, which are not limited here.

[0041] The following first describes in detail the neural network model training method provided by the present disclosure.

[0042] Post-translational modifications (PTMs) of proteins are one of the important mechanisms for regulating protein function. PTMs change the properties of proteins by covalently modifying one or more residues of proteins or by removing groups through proteases. These modifications include the widely studied phosphorylation, glycosylation, acetylation, and ubiquitination, as well as the recently discovered lysine crotonylation (Kcr). In 2011, Tan et al. first identified Kcr through mass spectrometry analysis and discovered its role in histones, which attracted widespread attention.

[0043] Kcr is of great significance in many biological fields. For example, in the field of epigenetics, Kcr modification is closely related to reproductive regulation; in disease research, Kcr modification is involved in a variety of signaling pathways and nuclear-related processes, especially in the pathogenesis of lung cancer, pancreatic cancer and other tumors; in botany, Kcr is involved in photosynthesis and a variety of metabolic pathways, and plays a key role in plant growth and development and metabolic regulation; in microbiology, Kcr is related to the movement of bacteria and parasites, host invasion and metabolic regulation. Therefore, accurate identification of Kcr sites from protein sequence data is of great significance for basic research, drug development and disease treatment.

[0044] However, traditional experimental methods for identifying Kcr sites are time-consuming and costly, limiting the efficiency and scope of research.

[0045] In this regard, the present disclosure provides a neural network model training method, which includes: obtaining a data set, wherein the data in the data set are protein sequence data containing Kcr sites; based on the data set and based on an initial model, obtaining a target neural network model with the function of identifying Kcr sites in protein sequences, wherein the initial model includes an ESM2 model.

[0046] The present invention can obtain a neural network model with the function of identifying Kcr sites in protein sequences based on ESM2 model training, so that the Kcr sites in the corresponding protein sequences can be quickly and efficiently identified according to the neural network model, thereby improving the efficiency of Kcr site identification in protein sequences and reducing identification costs.

[0047] The neural network model training method provided by the present invention will be described in detail below.

[0048] Figure 1 The flowchart of the neural network model training method provided by the embodiment of the present disclosure is as follows. Figure 1 As shown, the method may include the following S101-S102.

[0049] S101. Obtain a data set.

[0050] Among them, the data in the dataset are protein sequence data containing Kcr sites;

[0051] For example, protein sequence data containing Kcr sites can be collected from various species (such as plants, animals, humans and microorganisms) to form a data set. For example, corresponding protein sequence data containing Kcr sites can be collected from plants, animals, humans and microorganisms, respectively, wherein plant data are from rice, tea trees and tobacco, animal data are from zebrafish, human data include lung cancer and pancreatic cancer samples, and microbial data are from Toxoplasma gondii.

[0052] As a possible implementation, after obtaining the data set, the data in the data set may be preprocessed to ensure the high quality of the data in the data set. For example, the data set may be deduplicated to remove repeated redundant data in the data set.

[0053] S102. Based on the data set and the initial model, a target neural network model having the function of identifying Kcr sites in protein sequences is obtained, and the initial model includes an ESM2 model.

[0054] For example, based on the above data set, the ESM2 model can be used as the initial model for training (for example, using supervised learning related techniques for training), thereby obtaining a target neural network model. Alternatively, the ESM2 model and the classifier can be used for training according to the above data set, thereby obtaining a target neural network.

[0055] As an example, the ESM2 model may specifically be esm2_t33_650M_UR50D, and of course may also be other models in the ESM2 series models, which is not limited here.

[0056] Through the neural network model training method, a neural network model with the function of identifying Kcr sites in protein sequences can be obtained based on ESM2 model training, so that the Kcr sites in the corresponding protein sequences can be quickly and efficiently identified according to the neural network model, thereby improving the efficiency of Kcr site identification in protein sequences and reducing identification costs.

[0057] Optionally, the initial model is a model consisting of the ESM2 model with only the first four layers of attention heads retained.

[0058] In this way, by modifying the ESM2 model and retaining only the first four layers of attention heads, the overfitting during neural network model training can be reduced, thereby improving the performance of the final target neural network model.

[0059] Optionally, before obtaining a target neural network model having a function of identifying Kcr sites in a protein sequence based on the initial model according to the data set, the method further comprises:

[0060] Split the dataset into training and testing sets.

[0061] For example, the data set can be randomly split into two data sets, a training set and a test set. Of course, in some possible implementations, the data set can also be split in other ways to obtain a training set and a test set, which is not limited here. For example, the data set can be evenly split according to the species corresponding to the protein sequences in the data set, so that the obtained training set and test set both have protein sequences from various species.

[0062] As an example, splitting a dataset into training and testing sets could include:

[0063] The dataset is split into training and testing sets according to the preset ratio.

[0064] For example, the preset ratio may be 7:3, that is, 70% of the data in the data set may be used as a training set, and the remaining 30% of the data may be used as a test set. Of course, in actual applications, other preset ratios may be set according to actual needs or requirements, and are not limited here.

[0065] In this way, it is convenient to subsequently train the neural network model according to the training set, and to test the trained neural network model according to the test set, thereby improving the training quality of the target neural network model.

[0066] Accordingly, according to the data set and based on the initial model, a target neural network model having the function of identifying Kcr sites in protein sequences can be obtained, which may include:

[0067] Based on the initial model according to the training set, a target neural network model with the function of identifying Kcr sites in protein sequences is obtained.

[0068] Optionally, based on the initial model according to the training set, a target neural network model having the function of identifying Kcr sites in protein sequences is obtained, such as Figure 2 As shown, the following S201-S207 may be included.

[0069] S201. Extract features from a training set according to an initial model to obtain a first feature.

[0070] For example, taking the ESM2 model in which only the first four layers of attention heads are retained as the initial model, the data in the training set is subjected to feature extraction according to the initial model, and the expression of the first feature obtained is as follows:

[0071] f(x) = ReLU(W1·ESM 2'(x)+b1)

[0072] Among them, ESM 2'(x) is the feature extracted from the protein sequences in the training set by the ESM2 model that only retains its first four attention heads.

[0073] Of course, in practical applications, when the initial model is another ESM2 model, ESM 2'(x) in the above expression can also be replaced by ESM 2(x), that is, replaced by the features extracted from the corresponding ESM2 model, and there is no limitation here.

[0074] S202. Perform feature extraction on the training set according to the K-mer algorithm to obtain a second feature.

[0075] Among them, the K-mer algorithm can divide the protein sequence into multiple short sequences of length K according to the K-mer analysis method, and extract features from these short sequences to obtain the corresponding frequency vectors.

[0076] For example, taking the 1-mer algorithm and the 2-mer algorithm as an example to extract features from the data in the training set, the expression of the obtained second feature can be:

[0077] g(x)=ReLU(W2·[Count1mer(x); Count2mer(x)]+b2)

[0078] Among them, Count1mer(x); Count2mer(x) represent the frequency vectors extracted by the 1-mer algorithm and the 2-mer algorithm respectively.

[0079] S203. Outputting a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site according to the main classifier based on the first feature and the second feature.

[0080] For example, the main classifier makes a prediction based on the first feature and the second feature, and the obtained classification result can be expressed as follows:

[0081]

[0082] The sigmoid function is used as the activation function.

[0083] S204. Outputting a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature according to the first auxiliary classifier.

[0084] For example, the first auxiliary classifier makes a prediction based on the first feature, and the expression of the obtained classification result can be as follows:

[0085]

[0086] The sigmoid function is used as the activation function.

[0087] S205. Outputting a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature according to the second auxiliary classifier.

[0088] For example, the second auxiliary classifier makes a prediction based on the second feature, and the obtained classification result can be expressed as follows:

[0089]

[0090] The sigmoid function is used as the activation function.

[0091] S206: Obtain a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier, and the classification result of the second auxiliary classifier.

[0092] For example, based on the expressions of the prediction results of the above classifiers, the loss function can be obtained as follows:

[0093]

[0094] Among them, λ1 and λ2 are the weights of the loss functions corresponding to the first auxiliary classifier and the second auxiliary classifier, respectively, which are usually small (in this disclosure, they are set to 0.3 by way of example, and of course in practical applications, they can also be set according to actual conditions). These weights help adjust the learning direction of the model to make it more inclined to the main classifier, thereby preventing overfitting.

[0095] S207. Based on the loss function, optimize the initial model and the classifier to obtain a target neural network model, where the classifier includes a main classifier, a first auxiliary classifier, and a second auxiliary classifier.

[0096] For example, based on the aforementioned loss function, the parameters of the initial model, i.e., the ESM2 model, the main classifier, the first auxiliary classifier, and the second auxiliary classifier can be optimized to obtain a target neural network model consisting of the optimized initial model, the main classifier, the first auxiliary classifier, and the second auxiliary classifier.

[0097] Optionally, in the aforementioned method, the order of S203 - S205 may not be restricted and may be performed in parallel or in any order.

[0098] Based on the aforementioned implementation, classification prediction can be performed by the first auxiliary classifier and the second auxiliary classifier as auxiliary learning, which can enrich the learning path of the neural network model training, thereby improving the accuracy and reliability of the final target neural network model in identifying the Kcr site.

[0099] In an exemplary embodiment, the present disclosure also provides a neural network model training device, which can be used to implement the neural network model training method as described in the above embodiments.

[0100] Figure 3 A schematic diagram of the composition of a neural network model training device provided in an embodiment of the present disclosure.

[0101] like Figure 3 As shown, the device may include:

[0102] The data processing module 301 is used to obtain a data set, wherein the data in the data set is protein sequence data containing Kcr sites; the model training module 302 is used to obtain a target neural network model having the function of identifying Kcr sites in protein sequences based on the data set and an initial model, wherein the initial model includes an ESM2 model.

[0103] In some possible implementations, the initial model is a model consisting of an ESM2 model that only retains the first four layers of attention heads.

[0104] In some possible embodiments, the data processing module 301 is also used to split the data set into a training set and a test set; the model training module 302 is specifically used to obtain a target neural network model having the function of identifying Kcr sites in protein sequences based on the initial model according to the training set.

[0105] In some possible implementations, the data processing module 301 is specifically configured to split the data set into the training set and the test set according to a preset ratio.

[0106] In some possible embodiments, the model training module 302 is specifically used to perform feature extraction on the training set according to the initial model to obtain a first feature; perform feature extraction on the training set according to the K-mer algorithm to obtain a second feature; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature and the second feature according to the main classifier; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature according to the first auxiliary classifier; output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature according to the second auxiliary classifier; obtain a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier and the classification result of the second auxiliary classifier; based on the loss function, optimize the initial model and the classifier to obtain the target neural network model, the classifier including the main classifier, the first auxiliary classifier and the second auxiliary classifier.

[0107] In the technical solution disclosed in the present invention, the acquisition, storage and application of protein sequence data involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0108] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0109] In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the neural network model training method as described in the above embodiments.

[0110] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to the above embodiments.

[0111] In an exemplary embodiment, a computer program product includes a computer program, and when the computer program is executed by a processor, the method according to the above embodiments is implemented.

[0112] Figure 4 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, other suitable computers, servers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, etc. and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0113] like Figure 4 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0114] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0115] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as the neural network model training method described above. For example, in some embodiments, the neural network model training method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the neural network model training method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to perform the neural network model training method in any other appropriate manner (e.g., by means of firmware).

[0116] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0118] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0121] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0123] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A neural network model training method, characterized in that: include: Acquire a data set, wherein the data in the data set is protein sequence data including a Kcr site; According to the data set and based on the initial model, a target neural network model with the function of identifying Kcr sites in protein sequences is obtained, and the initial model includes the ESM2 model.

2. The method according to claim 1, characterized in that The initial model is a model consisting of the ESM2 model retaining only the first four layers of attention heads.

3. The method according to claim 1, characterized in that Before obtaining a target neural network model having a function of identifying Kcr sites in a protein sequence based on the data set and the initial model, the method further comprises: Splitting the data set into a training set and a test set; The method of obtaining a target neural network model having the function of identifying Kcr sites in protein sequences based on the data set and the initial model comprises: According to the training set and based on the initial model, a target neural network model having the function of identifying Kcr sites in protein sequences is obtained.

4. The method according to claim 3, characterized in that The step of splitting the data set into a training set and a test set comprises: According to a preset ratio, the data set is split into the training set and the test set.

5. The method according to claim 3, characterized in that: The method of obtaining a target neural network model having a function of identifying Kcr sites in a protein sequence based on the training set and the initial model comprises: Extracting features from the training set according to the initial model to obtain a first feature; Perform feature extraction on the training set according to the K-mer algorithm to obtain a second feature; Outputting, according to the main classifier based on the first feature and the second feature, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site; Outputting, according to the first auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature; Outputting, according to the second auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature; Obtaining a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier, and the classification result of the second auxiliary classifier; Based on the loss function, the initial model and the classifier are optimized to obtain the target neural network model, and the classifier includes the main classifier, the first auxiliary classifier and the second auxiliary classifier.

6. A neural network model training device, characterized in that: include: A data processing module, used for acquiring a data set, wherein the data in the data set is protein sequence data containing a Kcr site; The model training module is used to obtain a target neural network model with the function of identifying Kcr sites in protein sequences based on the data set and the initial model, wherein the initial model includes the ESM2 model.

7. The device according to claim 6, characterized in that The initial model is a model consisting of the ESM2 model retaining only the first four layers of attention heads.

8. The device according to claim 6, characterized in that The data processing module is further used to split the data set into a training set and a test set; The model training module is specifically used to obtain a target neural network model having the function of identifying Kcr sites in protein sequences based on the initial model according to the training set.

9. The device according to claim 8, characterized in that The data processing module is specifically used to split the data set into the training set and the test set according to a preset ratio.

10. The device according to claim 8, characterized in that The model training module is specifically used to extract features from the training set according to the initial model to obtain a first feature; extract features from the training set according to the K-mer algorithm to obtain a second feature; and output a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature and the second feature according to the main classifier; Outputting, according to the first auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the first feature; Outputting, according to the second auxiliary classifier, a classification result of whether the protein sequence site corresponding to the current feature is a Kcr site based on the second feature; Obtaining a loss function based on the classification result of the main classifier, the classification result of the first auxiliary classifier, and the classification result of the second auxiliary classifier; Based on the loss function, the initial model and the classifier are optimized to obtain the target neural network model, and the classifier includes the main classifier, the first auxiliary classifier and the second auxiliary classifier.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

Images