Feature extraction method, device and equipment of chemical reaction data and medium

Abstract

The invention discloses a chemical reaction data feature extraction method and device, equipment and a medium, and relates to the technical field of computers, the method comprises the following steps: obtaining a target character string of current chemical reaction data, and using a word segmentation device and a preset vocabulary to mark the target character string to obtain a target mark sequence; inputting the target marker sequence into a target mask language model obtained by performing model pre-training by using a historical molecular data character string and a historical chemical reaction data character string, so that the target mask language model outputs a target hidden representation of the target marker sequence; and determining a feature vector extracted from the target hidden representation as a target feature of the current chemical reaction data. Through the scheme, the deep semantic features of the chemical reaction data are extracted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a feature extraction method, device and equipment of chemical reaction data and medium. BACKGROUND

[0002] The chemical reaction data refers to various numerical values and parameters generated in the chemical reaction process, including the basic information of the chemical formula, structural formula, molecular weight and molar mass of the reactants and products, is the SMILES (Simplified Molecular Input Line Entry System) data used to describe the chemical reaction. Generally speaking, a molecule can be represented by a molecular description language, and the chemical reaction data is represented by connecting several reactants and products with symbols "." and ">". The features extracted from the chemical reaction data can be used for chemical reaction classification, enzyme kinetics parameter prediction and other tasks. The existing chemical reaction data feature extraction method mainly depends on the traditional molecular fingerprint and chemical reaction fingerprint extraction technology. These methods generate a series of fixed-length binary bits or digital features, called molecular fingerprints, such as MACCS (Molecular Access System Codes) and Morgan fingerprint (molecular fingerprint).

[0003] The chemical reaction fingerprint is based on the structural features of the reactants and products to extract the change information in the reaction process, but since it mainly captures the shallow information of the molecular structure, it is difficult to extract the deep semantic information of the chemical reaction data.

[0004] From the above, it can be seen that how to extract the deep semantic features of the chemical reaction data is a problem to be solved in the field. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a feature extraction method, device and equipment of chemical reaction data and medium, which can extract the deep semantic features of the chemical reaction data. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a feature extraction method of chemical reaction data, comprising:

[0007] obtaining a target string of the current chemical reaction data, and using a tokenizer and a preset vocabulary to mark the target string to obtain a target mark sequence;

[0008] inputting the target token sequence into a target mask language model pre-trained by using a historical molecular data string and a historical chemical reaction data string respectively, so that the target mask language model outputs a target hidden representation of the target token sequence;

[0009] determining a feature vector extracted from the target hidden representation as a target feature of the current chemical reaction data.

[0010] Optionally, the target mask language model pre-trained by using the historical molecular data string and the historical chemical reaction data string respectively comprises:

[0011] collecting first initial data from a data set of the historical molecular data string, and collecting second initial data from a data set of the historical chemical reaction data string;

[0012] respectively performing data enhancement processing on the first initial data and the second initial data to obtain first enhanced data and second enhanced data;

[0013] respectively tokenizing the first enhanced data and the second enhanced data to obtain first tokenized data and second tokenized data;

[0014] iteratively training an initial mask language model by using the first tokenized data to obtain a first trained mask language model;

[0015] iteratively training the first trained mask language model by using the second tokenized data to obtain a second trained mask language model, and determining the second trained mask language model as the target mask language model.

[0016] Optionally, the data enhancement processing on the first initial data and the second initial data respectively to obtain the first enhanced data and the second enhanced data comprises:

[0017] respectively adding the first initial data and the second initial data as to-be-enhanced data into a preset empty set to obtain a current set;

[0018] determining whether a preset enhancement stop condition is met at present;

[0019] if the preset enhancement stop condition is not met at present, then the representation order of each to-be-enhanced data in the current set is disturbed, and the disturbed data is added into the current set to obtain a next set, the next set is updated as the current set, and then the step of determining whether the preset enhancement stop condition is met at present is re-executed;

[0020] If the preset enhancement stop condition is met at present, the first enhanced data and the second enhanced data are determined from the current set.

[0021] Optionally, the preset enhancement stop condition is that the number of current enhancement processing reaches a first preset threshold or the size of the current set reaches a second preset threshold.

[0022] Optionally, before the first enhanced data and the second enhanced data are tokenized respectively, the method further comprises:

[0023] A tokenizer is constructed, and each string in the data set of the historical molecular data string and the data set of the historical chemical reaction data string is segmented into a single character or a substructure unit by using the tokenizer to obtain a preset vocabulary table;

[0024] Correspondingly, the tokenization of the first enhanced data and the second enhanced data respectively comprises:

[0025] The first enhanced data and the second enhanced data are tokenized respectively by using the tokenizer and the preset vocabulary table.

[0026] Optionally, the iterative training of the initial mask language model by using the first tokenized data comprises:

[0027] The initial mask language model is determined as a current mask language model;

[0028] The first tokenized data is input into the current mask language model to obtain a first hidden representation output by the current mask language model;

[0029] A first loss value of the current mask language model is determined by using the first hidden representation and a target loss function, a first evaluation result of the current mask language model is determined, and parameters and hyperparameters of the current mask language model are updated respectively by using the first loss value and the first evaluation result to obtain a next mask language model;

[0030] Correspondingly, the iterative training of the first trained mask language model by using the second tokenized data comprises:

[0031] The first trained mask language model is determined as a current trained mask language model;

[0032] The second tokenized data is input into the current trained mask language model to obtain a second hidden representation output by the current trained mask language model;

[0033] determining a second loss value of the current trained mask language model by using the second hidden representation and the target loss function, determining a second evaluation result of the current trained mask language model, and updating parameters and hyperparameters of the current trained mask language model by using the second loss value and the second evaluation result respectively, to obtain a next trained mask language model.

[0034] Optionally, the inputting the first marked data into the current mask language model to obtain a first hidden representation output by the current mask language model comprises:

[0035] inputting the first marked data into the current mask language model, so that the current mask language model maps the first marked data to a high-dimensional vector space to obtain a first target matrix, encoding the first target matrix to obtain a first hidden representation;

[0036] Correspondingly, the inputting the second marked data into the current trained mask language model to obtain a second hidden representation output by the current trained mask language model comprises:

[0037] inputting the second marked data into the current trained mask language model, so that the current trained mask language model maps the second marked data to a high-dimensional vector space to obtain a second target matrix, and encoding the second target matrix to obtain a second hidden representation.

[0038] In a second aspect, the present application discloses a feature extraction device for chemical reaction data, comprising:

[0039] a mark sequence acquisition module configured to acquire a target string of current chemical reaction data, and tokenize the target string by using a word segmenter and a preset vocabulary table to obtain a target mark sequence;

[0040] a hidden representation output model configured to input the target mark sequence into a target mask language model pre-trained by using historical molecular data strings and historical chemical reaction data strings, so that the target mask language model outputs a target hidden representation of the target mark sequence;

[0041] a target feature extraction module configured to determine a feature vector extracted from the target hidden representation as a target feature of the current chemical reaction data.

[0042] In a third aspect, the present application discloses an electronic device, comprising:

[0043] a memory configured to save a computer program;

[0044] A processor is configured to execute the computer program to implement the steps of the feature extraction method of chemical reaction data disclosed above.

[0045] In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program, wherein the computer program is configured to implement the steps of the feature extraction method of chemical reaction data disclosed above when executed by a processor.

[0046] The present application has the following beneficial effects: the present application obtains a target string of current chemical reaction data, and labels the target string by using a tokenizer and a preset vocabulary table to obtain a target token sequence; the target token sequence is input into a target mask language model obtained by respectively pre-training a model by using historical molecular data strings and historical chemical reaction data strings, so that the target mask language model outputs a target hidden representation of the target token sequence; and a feature vector extracted from the target hidden representation is determined as a target feature of the current chemical reaction data. As can be seen, the present application labels the target string by using the tokenizer and the preset vocabulary table, which can divide the target string into the smallest meaningful unit, and the target mask language model is obtained by respectively pre-training a model by using the historical molecular data strings and the historical chemical reaction data strings, so that the target mask language model can capture long-distance dependency relationships and context information in the input target token sequence, and thus does not ignore the complex relationships and context information between molecules, thereby being able to extract deep semantic features of the chemical reaction data. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0048] Figure 1 A feature extraction method of chemical reaction data disclosed by the present application is shown in the flowchart;

[0049] Figure 2 A specific hidden representation acquisition schematic diagram disclosed by the present application is shown in the flowchart;

[0050] Figure 3 A specific model training schematic diagram disclosed by the present application is shown in the flowchart;

[0051] Figure 4 A feature extraction device structure schematic diagram of chemical reaction data disclosed by the present application is shown in the flowchart;

[0052] Figure 5A structural diagram of an electronic device is disclosed in the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0054] Chemical reaction data refers to various numerical values and parameters generated in the process of chemical reaction, including basic information such as chemical formula, structural formula, molecular weight and molar mass of reactants and products, is SMILES data used to describe chemical reaction. Generally speaking, a molecule can be represented by molecular description language, and chemical reaction data is represented by connecting several reactants and products with symbols “.” and “>”. The features extracted from the chemical reaction data can be used for chemical reaction classification, enzyme kinetics parameter prediction and other tasks. The existing chemical reaction data feature extraction method mainly depends on traditional molecular fingerprint and chemical reaction fingerprint extraction technology. These methods generate a series of fixed-length binary bits or digital features, called molecular fingerprints, such as MACCS, Morgan fingerprint, through the topological features of molecular structure.

[0055] Chemical reaction fingerprint extracts the change information in the reaction process based on the structural features of reactants and products. However, since it mainly captures the shallow information of molecular structure, it is difficult to extract the deep semantic information of chemical reaction data.

[0056] Therefore, the present application correspondingly provides a chemical reaction data feature extraction scheme to extract the deep semantic features of chemical reaction data.

[0057] Referring to Figure 1 The embodiments of the present application disclose a chemical reaction data feature extraction method, which comprises the following steps:

[0058] Step S11: Obtain a target string of current chemical reaction data, and perform tokenization on the target string by using a tokenizer and a preset vocabulary table to obtain a target token sequence.

[0059] The current chemical reaction data is chemical reaction data to be characterized, and a target string of the current chemical reaction data is obtained, that is, the current chemical reaction data is represented in the form of a string, and the string representation method can be SMILES, SMARTS (SMILES ARbitrary Target Specification, SMILES Arbitrary Target Specification) or SELFIES (Self-referencing Embedded Strings, Self-referencing Embedded Strings) or the like.

[0060] The target string is tokenized by using a tokenizer and a preset vocabulary to obtain a target token sequence. Token is the smallest meaningful unit in text, tokenization refers to the process of dividing text into the smallest meaningful unit, and the preset vocabulary can be constructed based on a historical data set, and the tokenizer tokenizes the target string based on the preset vocabulary.

[0061] Step S12: inputting the target token sequence into a target mask language model pre-trained by using a historical molecular data string and a historical chemical reaction data string, so that the target mask language model outputs a target hidden representation of the target token sequence.

[0062] The historical molecular data string and the historical chemical reaction data string are pre-trained to obtain the target mask language model, and then the target token sequence is input into the target mask language model, and the target mask language model outputs the target hidden representation of the target token sequence.

[0063] In the embodiment, the target mask language model is obtained by pre-training the model using historical molecular data strings and historical chemical reaction data strings, including: collecting first initial data from a data set of historical molecular data strings, and collecting second initial data from a data set of historical chemical reaction data strings; performing data enhancement processing on the first initial data and the second initial data respectively to obtain first enhanced data and second enhanced data; performing tokenization on the first enhanced data and the second enhanced data respectively to obtain first tokenized data and second tokenized data; iteratively training an initial mask language model using the first tokenized data to obtain a first trained mask language model; iteratively training the first trained mask language model using the second tokenized data to obtain a second trained mask language model, and determining the second trained mask language model as the target mask language model. According to a specific scene, a suitable pre-training language model is selected as the mask language model, where the mask language model can be RoBERTa (Robustly optimized BERT approach), T5 (Text-To-Text Transfer Transformer), etc. The data set of historical molecular data strings is, for example, a small molecule SMILES data set PubChem 77M, which contains about 77 million small molecule SMILES data, and the data set of historical chemical reaction data strings is, for example, a chemical reaction SMILES data set USPTO-MIT, which contains about 480,000 chemical reaction SMILES data; the process of obtaining data for training the initial mask language model includes collecting the first initial data and tokenizing the enhanced first initial data, and the process of obtaining data for training the first trained mask language model includes collecting the second initial data and tokenizing the enhanced second initial data. It can be understood that the process of obtaining data for training the initial mask language model and the process of obtaining data for training the first trained mask language model can be performed synchronously or asynchronously.

[0064] In this embodiment, the data augmentation processing is performed on the first initial data and the second initial data respectively to obtain first augmented data and second augmented data, which includes: adding the first initial data and the second initial data as to-be-augmented data into a preset empty set to obtain a current set; determining whether a preset augmentation stop condition is met; if the preset augmentation stop condition is not met, the representation order of each to-be-augmented data in the current set is disturbed, and the disturbed data is added to the current set to obtain a next set, and the next set is updated as the current set, and then the step of determining whether the preset augmentation stop condition is met is re-executed; if the preset augmentation stop condition is met, the first augmented data and the second augmented data are determined from the current set.

[0065] Although the masked language model can be used for feature extraction of small molecules, when directly extended to reaction data, there are problems in representation and processing of complex structures of reactants and products, and the generalization ability of the model needs to be improved. Therefore, the first initial data and the second initial data are respectively subjected to data augmentation processing, and in the pre-training process, the representation order of the first initial data and the second initial data is disturbed for data augmentation, which expands the diversity of training data and improves the generalization ability of the model. This data augmentation method can effectively improve the adaptability of the model to data in different arrangements, so as to better cope with data changes in actual application scenarios.

[0066] The first initial data is taken as to-be-augmented data, and the specific augmentation steps are as follows:

[0067] (1.1) An empty set s1 is created as a preset empty set s1, and the first initial data is added as to-be-augmented data m into the preset empty set s1 to obtain a current set S1;

[0068] (1.2) Determine whether a preset augmentation stop condition is met;

[0069] (1.3) If yes, save the current set S1, and exit the program;

[0070] (1.4) If no, the representation order of the to-be-augmented data m is randomly disturbed to obtain disturbed data, and the disturbed data not existing in the current set S1 is saved to the current set S1, and step (1.2) is re-executed;

[0071] Since the order of the reactants in the chemical reaction SMILES data does not affect the occurrence of the reaction, the chemical reaction SMILES data can be subjected to data augmentation, and the second initial data is taken as to-be-augmented data, and the specific augmentation steps are as follows:

[0072] (2.1) creating an empty set s2 as a preset empty set s2, and adding the second initial data as the data to be enhanced r into the preset empty set s2 to obtain a current set S2;

[0073] (2.2) judging whether the preset enhancement stop condition is met at present;

[0074] (2.3) if yes, saving the current set S2 and exiting the program;

[0075] (2.4) if no, randomly shuffling the representation order of the data to be enhanced r to obtain shuffled data, saving the shuffled data not existing in the current set S2 into the current set S2, and re-executing step (2.2).

[0076] In the embodiment, the preset enhancement stop condition is that the number of current enhancement processing reaches a first preset threshold or the size of the current set reaches a second preset threshold. Specifically, if the number of current enhancement processing reaches the first preset threshold, the data enhancement program is exited, or if the size of the current set reaches the second preset threshold, the data enhancement program is exited. For example, the first preset threshold is set to 10 and the second preset threshold is set to 5, so if the number of current enhancement processing reaches 10 or the size of the current set reaches 5, the current set is saved and the data enhancement program is exited. That is, if the first initial data is used as the data to be enhanced, the data in the current set is the first enhanced data. By executing the data enhancement step on each small molecule SMILES in PubChem 77M, the amount of small molecule SMILES data can be expanded from about 77 million to about 385 million. If the second initial data is used as the data to be enhanced, the data in the current set is the second enhanced data. By executing the data enhancement step on each chemical reaction SMILES data in USPTO-MIT, the amount of chemical reaction SMILES data can be expanded from 480,000 to about 1.8 million.

[0077] Before the tokenization of the first enhanced data and the second enhanced data, the method further includes: constructing a tokenizer, and using the tokenizer to divide each string in the data set of the historical molecular data string and the data set of the historical chemical reaction data string into a single character or a substructure unit to obtain a preset vocabulary. Token is the smallest meaningful unit in text, and tokenization refers to the process of dividing text into the smallest meaningful unit. In the SMILES string, token can be the smallest unit such as atom and ion, or the substructure unit such as hydroxyl and benzene ring. The embodiment adopts an atom and ion based tokenization method. A tokenizer is constructed, which divides each SMILES string into a single character or a substructure unit, such as atom C, O, N, ion [Mg2+] and [Cl-], and other symbols (,), =, #, etc. A vocabulary table containing all possible SMILES tokens is constructed based on the data set PubChem 77M and the data set USPTO-MIT.

[0078] In the embodiment, the tokenization of the first enhanced data and the second enhanced data includes: using the tokenizer and the preset vocabulary to tokenize the first enhanced data and the second enhanced data, respectively. Each SMILES string is converted into a corresponding token sequence. For example, the SMILES string “C(C(=O)O)c1ccc(cc1)N” is converted into the Token sequence “C, (, C, =, O, ), O, c, 1, c, c, c, (, c, c, 1, ), N”.

[0079] In the embodiment, the iterative training of the initial mask language model using the first tokenized data includes: determining the initial mask language model as a current mask language model; inputting the first tokenized data into the current mask language model to obtain a first hidden representation output by the current mask language model; determining a first loss value of the current mask language model using the first hidden representation and a target loss function, determining a first evaluation result of the current mask language model, and updating parameters and hyperparameters of the current mask language model using the first loss value and the first evaluation result, respectively, to obtain a next mask language model. First, the initial mask model is pre-trained using the first tokenized data, i.e., the initial mask model is pre-trained using molecular data. During the training process, the parameters and hyperparameters of the mask language model are updated using the loss value and the evaluation result of the mask language model, respectively.

[0080] In the embodiment, the iterative training of the first trained mask language model by using the second marked data comprises: determining the first trained mask language model as a current trained mask language model; inputting the second marked data into the current trained mask language model to obtain a second hidden representation output by the current trained mask language model; determining a second loss value of the current trained mask language model by using the second hidden representation and the target loss function, determining a second evaluation result of the current trained mask language model, and updating parameters and hyperparameters of the current trained mask language model by using the second loss value and the second evaluation result respectively to obtain a next trained mask language model. When the first trained mask language model trained by the first marked data is obtained, the first trained mask language model is trained by using the second marked data, i.e., the first trained mask language model is pre-trained by using the chemical reaction data, and in the training process, the parameters and the hyperparameters of the mask language model are updated by using the loss value and the evaluation result of the mask language model respectively.

[0081] It should be noted that the iterative training of the initial mask language model by using the first marked data to obtain the first trained mask language model comprises: replacing the marks in the first marked data by using a first preset mask mark based on a first preset proportion to obtain a first mask marked sequence; and iteratively training the initial mask language model by using the first mask marked sequence to obtain the first trained mask language model.

[0082] Further, the iterative training of the first trained mask language model by using the second marked data to obtain the second trained mask language model comprises: replacing the marks in the second marked data by using a second preset mask mark based on a second preset proportion to obtain a second mask marked sequence; and iteratively training the first trained mask language model by using the second mask marked sequence to obtain the first trained mask language model.

[0083] Specifically, after obtaining the first marked data and the second marked data, some marks in the first marked data and the second marked data are masked, usually replaced by special mask marks, and the first preset mask mark and the second preset mask mark are, for example, both [MASK]. For example, the first marked data is “[C, (, C, =, O,), O, c, 1, c, c, c, (, c, c, 1,), N]”, and after being replaced by the mask mark, the first mask marked sequence can be “[C, (, [MASK], =, O,), [MASK], c, 1, c, c, c, (, [MASK], c, 1,), N]”. In this embodiment, 15% of the marks in each input mark sequence are replaced by mask marks, that is, the first preset proportion and the second preset proportion can each be 15%. The mask marked sequence is input into the RoBERTa model, and the model predicts the masked marks according to the context, that is, the hidden representation.

[0084] In the process of updating the masked language model, the loss value is determined by using the target loss function, the parameters of the model are updated by using the loss value, the evaluation result of the model is determined, and the hyperparameters of the model are updated by using the evaluation result. In this way, the next model is obtained, and then the training of the model is performed again. The loss function of the masked language model is used to evaluate the performance of the model in predicting the masked marks. Assuming that the input sequence is S, part of the marks are randomly masked, and the masked sequence is The model outputs a prediction distribution to estimate the original word at each masked position. Assuming that the set of masked word indices is M, the loss function of the masked language model for a sequence can be represented as:

[0085] ;

[0086] In the formula, W i represents the i-th mark in the set M, and P represents the probability that W i is correctly predicted.

[0087] Further, the model needs to be evaluated to ensure its performance and generalization ability, and the hyperparameters of the model are updated by using the evaluation result. The evaluation methods include:

[0088] (1) Accuracy: The accuracy of the model on the validation set is evaluated to measure the correctness of the model in predicting the masked marks.

[0089] (2) Perplexity: Perplexity is a commonly used evaluation index for language models, which reflects the prediction ability of the model on the data set. A lower perplexity indicates that the model better captures the statistical characteristics of the data. For a predicted sequence, the definition of perplexity is:

[0090] ;

[0091] where S represents a sequence, L represents a number of sequence tokens, W i represents the i-th token in the sequence S, is the probability of W given the first through i-1 tokens. The probability of the first token is i , W0 is START, indicating the start of a sentence, is a placeholder.

[0092] In this embodiment, the inputting the first token data into the current mask language model to obtain the first hidden representation output by the current mask language model comprises: inputting the first token data into the current mask language model, so that the current mask language model maps the first token data to a high-dimensional vector space to obtain a first target matrix, and encoding the first target matrix to obtain the first hidden representation. Taking the RoBERTa model as an example, the RoBERTa model adopts a stacked multi-layer Transformer encoder, each layer of which contains a multi-head self-attention mechanism and a feedforward neural network, which are used to capture long-distance dependencies and context information in the input sequence, as shown in a specific hidden representation acquisition schematic diagram as shown in Figure 2 . Specifically, for an input sequence S with a token number of L, the model first uses an Embedding module to embed the input sequence S into a high-dimensional vector space to obtain a matrix , where D is the dimension of the vector space, is a real number, and then encodes through a multi-layer Transformer to finally output the first hidden representation, where the specific formula is as follows:

[0093] .

[0094] In this embodiment, the inputting the second token data into the current trained mask language model to obtain the second hidden representation output by the current trained mask language model comprises: inputting the second token data into the current trained mask language model, so that the current trained mask language model maps the second token data to a high-dimensional vector space to obtain a second target matrix, and encoding the second target matrix to obtain the second hidden representation. It can be understood that in the process of training the current trained mask language model with the second token data as the training data, the current trained mask language model also maps the second token data to a high-dimensional vector space to obtain a second target matrix, and encodes the second target matrix to obtain the second hidden representation.

[0095] ​Step S13: determining the feature vector extracted from the target hidden representation as the target feature of the current chemical reaction data.

[0096] The feature vector is extracted from the last layer hidden representation output by the target mask language model, and the feature vector can represent the feature information of the chemical reaction, so the feature vector is determined as the target feature of the current chemical reaction data, and is used for subsequent analysis and application, for example, the target feature of the current chemical reaction data can provide support for tasks such as chemical reaction classification and enzyme kinetics parameter prediction.

[0097] The beneficial effects of the present application are: the present application obtains the target string of the current chemical reaction data, and uses the tokenizer and the preset vocabulary table to tokenize the target string to obtain the target token sequence; the target token sequence is input into the target mask language model obtained by pre-training the model using the historical molecular data string and the historical chemical reaction data string, so that the target mask language model outputs the target hidden representation of the target token sequence; the feature vector extracted from the target hidden representation is determined as the target feature of the current chemical reaction data. As can be seen, the present application tokenizes the target string using the tokenizer and the preset vocabulary table, which can divide the target string into the smallest meaningful unit, and the target mask language model is obtained by pre-training the model using the historical molecular data string and the historical chemical reaction data string, so that the target mask language model can capture the long-distance dependency relationship and context information in the input target token sequence, so as to not ignore the complex relationship and context information between molecules, thereby being able to extract the deep semantic features of the chemical reaction data.

[0098] The following will take a specific model training diagram as shown in Figure 3 to illustrate the mask language model training process of the present application.

[0099] First, the first stage of model training is performed, that is, the model is trained using molecular training data, and the specific process is as follows:

[0100] (1.1) Collecting first initial data from the data set of historical molecular data strings;

[0101] (1.2) performing data augmentation processing on the first initial data to obtain first augmented data:

[0102] (1.2.1) creating an empty set s1 as a preset empty set s1, and adding the first initial data as the data to be augmented m into the preset empty set s1 to obtain the current set S1;

[0103] (1.2.2) judging whether the current satisfies the preset augmentation stop condition;

[0104] (1.2.3) if yes, save the current set S1, and exit the program;

[0105] (1.2.4) if no, randomly shuffle the representation order of the to-be-enhanced data m to obtain shuffled data, save the shuffled data that does not exist in the current set S1 to the current set S1, and re-execute step (1.2.2);

[0106] (1.3) tokenize the first enhanced data to obtain first tokenized data;

[0107] (1.4) based on the first preset proportion, and using the first preset mask token, replace the tokens in the first tokenized data to obtain a first masked token sequence;

[0108] (1.5) use the first masked token sequence to iteratively train the initial masked language model to obtain a first trained masked language model; wherein in the training process, the loss value and the evaluation result are used to update the parameters and hyperparameters of the model, respectively.

[0109] Secondly, the second stage of model training is performed, that is, the model is trained using the chemical reaction training data, and the specific process is as follows:

[0110] (2.1) collect second initial data from the data set of historical chemical reaction data strings;

[0111] (2.2) perform data enhancement processing on the first initial data to obtain second enhanced data:

[0112] (2.2.1) create an empty set s2 as a preset empty set s2, and add the second initial data as to-be-enhanced data r to the preset empty set s2 to obtain a current set S2;

[0113] (2.2.2) determine whether the preset enhancement stop condition is met;

[0114] (2.2.3) if yes, save the current set S2, and exit the program;

[0115] (2.2.4) if no, randomly shuffle the representation order of the to-be-enhanced data r to obtain shuffled data, save the shuffled data that does not exist in the current set S1 to the current set S2, and re-execute step (2.2.2);

[0116] (2.3) tokenize the second enhanced data to obtain second tokenized data;

[0117] (2.4) based on the second preset proportion, and using the first preset mask token, replace the tokens in the second tokenized data to obtain a second masked token sequence;

[0118] (2.5) iteratively training the first post-training mask language model using the second post-masked sequence to obtain a second post-training mask language model; wherein in the training process, the parameters and hyperparameters of the model are updated using the loss value and the evaluation result, respectively.

[0119] Referring to Figure 4 As shown in the figure, the embodiment of the present application discloses a chemical reaction data feature extraction device, which comprises:

[0120] The target string of the current chemical reaction data is obtained, and the tokenizer and the preset vocabulary table are used for tokenization of the target string to obtain a target token sequence.

[0121] The target token sequence is input into the target mask language model pre-trained using the historical molecular data string and the historical chemical reaction data string, so that the target mask language model outputs the target hidden representation of the target token sequence.

[0122] The feature vector extracted from the target hidden representation is determined as the target feature of the current chemical reaction data.

[0123] The target string of the current chemical reaction data is obtained, and the tokenizer and the preset vocabulary table are used for tokenization of the target string to obtain a target token sequence. The target token sequence is input into the target mask language model pre-trained using the historical molecular data string and the historical chemical reaction data string, so that the target mask language model outputs the target hidden representation of the target token sequence. The feature vector extracted from the target hidden representation is determined as the target feature of the current chemical reaction data. As can be seen, the tokenizer and the preset vocabulary table are used for tokenization of the target string, which can divide the target string into the smallest meaningful unit. The target mask language model is pre-trained using the historical molecular data string and the historical chemical reaction data string, so that the target mask language model can capture the long-distance dependency relationship and the context information in the input target token sequence, and thus the complex relationship between molecules and the context information are not ignored, so that the deep semantic features of the chemical reaction data can be extracted.

[0124] Further, the embodiment of the present application also provides an electronic device. Figure 5 The electronic device 20 structure diagram shown in the figure cannot be considered as any limitation on the use range of the present application.

[0125] Figure 5 A structural schematic diagram of an electronic device is provided in the embodiments of the present application. Specifically, it can include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the feature extraction method of chemical reaction data performed by the electronic device disclosed in any of the preceding embodiments.

[0126] In the embodiments, the power supply 23 is configured to provide operating voltage for each hardware device on the electronic device; the communication interface 24 is capable of creating a data transmission channel between the electronic device and external devices, and the communication protocol followed by the communication interface 24 is any communication protocol applicable to the technical solution of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.

[0127] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0128] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc. The resources stored thereon include an operating system 221, a computer program 222, and data 223, etc. The storage mode can be temporary storage or permanent storage.

[0129] The operating system 221 is used to manage and control each hardware device and the computer program 222 on the electronic device, so as to realize the operation and processing of the processor 21 on the mass data 223 in the memory 22, and can be Windows, Unix, Linux, etc. In addition to the computer program capable of completing the feature extraction method of the chemical reaction data executed by the electronic device disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work. The data 223 can include data transmitted by an external device received by the electronic device, and can also include data collected by the self input / output interface 25 and the like.

[0130] Further, the application also discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by the processor to realize the feature extraction method of the chemical reaction data disclosed in the foregoing. For the specific steps of the method, refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0131] In the specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. For the same or similar parts between each embodiment, refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part is described in the method part.

[0132] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their functionality, which has been described generally and symbolically in flow charts. Having thus described the functionality of the examples in terms of a process, it is appreciated that this functionality can be implemented by one or more types of electrical circuits or computer software, which are collectively referred to herein as a "circuit" that can carry out a variety of operations described herein. The circuit can include a variety of different types of general purpose or special purpose circuits, or combinations thereof. In addition, it is further noted that the embodiments disclosed herein can be modified to comprise more or less steps or operations than those disclosed herein, and such modifications are contemplated and considered within the scope of embodiments of the present application. The steps or operations of the methods or algorithms described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in Random Access Memory (RAM), flash memory, Read-only memory (ROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, hard disk can be used as a non-transitory storage medium to store software modules.

[0133] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of identification in claims. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0134] The above describes in detail the chemical reaction data feature extraction method, device, equipment and medium provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above example is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for feature extraction from chemical reaction data, characterized in that, include: Obtain the target string of the current chemical reaction data, and use a word segmenter and a preset vocabulary to tokenize the target string to obtain the target token sequence; The target label sequence is input into a target mask language model obtained by pre-training the model using historical molecular data strings and historical chemical reaction data strings, so that the target mask language model outputs the target hidden representation of the target label sequence; The feature vector extracted from the target hidden representation is determined as the target feature of the current chemical reaction data.

2. The feature extraction method for chemical reaction data according to claim 1, characterized in that, The target mask language model is obtained by pre-training the model using historical molecular data strings and historical chemical reaction data strings, including: First initial data was collected from the dataset of historical molecular data strings, and second initial data was collected from the dataset of historical chemical reaction data strings; Data augmentation processing is performed on the first initial data and the second initial data respectively to obtain the first augmented data and the second augmented data; The first enhanced data and the second enhanced data are respectively labeled to obtain the first labeled data and the second labeled data; The initial masked language model is iteratively trained using the first labeled data to obtain the first trained masked language model. The first post-trained masked language model is iteratively trained using the second labeled data to obtain a second post-trained masked language model, and the second post-trained masked language model is determined as the target masked language model.

3. The feature extraction method for chemical reaction data according to claim 2, characterized in that, The step of performing data augmentation processing on the first initial data and the second initial data respectively to obtain the first augmented data and the second augmented data includes: The first initial data and the second initial data are respectively added to a preset empty set as data to be enhanced, so as to obtain the current set; Determine whether the preset enhanced stop condition is met; If the preset enhancement stop condition is not met, the representation order of each data to be enhanced in the current set is shuffled, and the shuffled data is added to the current set to obtain the next set. The next set is then updated to the current set, and the process jumps back to the step of determining whether the preset enhancement stop condition is met. If the preset enhancement stop condition is met, then the first enhanced data and the second enhanced data are determined from the current set.

4. The feature extraction method for chemical reaction data according to claim 3, characterized in that, The preset enhancement stopping condition is when the number of current enhancement processes reaches a first preset threshold or when the size of the current set reaches a second preset threshold.

5. The feature extraction method for chemical reaction data according to claim 2, characterized in that, Before the first enhanced data and the second enhanced data are respectively tokenized, the method further includes: Construct a word segmenter, and use the word segmenter to divide each string in the dataset of historical molecular data strings and the dataset of historical chemical reaction data strings into individual characters or sub-structural units to obtain a preset vocabulary; Accordingly, the step of tokenizing the first enhanced data and the second enhanced data respectively includes: The first enhanced data and the second enhanced data are tokenized using the word segmenter and the preset vocabulary, respectively.

6. The feature extraction method for chemical reaction data according to any one of claims 2 to 5, characterized in that, The iterative training of the initial masked language model using the first labeled data includes: The initial mask language model is determined as the current mask language model; The first tagged data is input into the current mask language model to obtain the first hidden representation output by the current mask language model; The first loss value of the current masked language model is determined using the first hidden representation and the target loss function, and the first evaluation result of the current masked language model is determined. The parameters and hyperparameters of the current masked language model are updated using the first loss value and the first evaluation result, respectively, to obtain the next masked language model. Accordingly, the iterative training of the first trained masked language model using the second labeled data includes: The first post-trained masked language model is determined as the current post-trained masked language model; The second labeled data is input into the currently trained masked language model to obtain the second hidden representation output by the currently trained masked language model; The second loss value of the currently trained post-masked language model is determined using the second hidden representation and the target loss function, and the second evaluation result of the currently trained post-masked language model is determined. The parameters and hyperparameters of the currently trained post-masked language model are updated using the second loss value and the second evaluation result, respectively, to obtain the next trained post-masked language model.

7. The feature extraction method for chemical reaction data according to claim 6, characterized in that, The step of inputting the first tagged data into the current mask language model to obtain the first hidden representation output by the current mask language model includes: The first labeled data is input into the current mask language model so that the current mask language model maps the first labeled data to a high-dimensional vector space to obtain a first target matrix. The first target matrix is ​​then encoded to obtain a first hidden representation. Accordingly, the step of inputting the second labeled data into the currently trained masked language model to obtain the second hidden representation output by the currently trained masked language model includes: The second labeled data is input into the currently trained post-mask language model so that the currently trained post-mask language model maps the second labeled data to a high-dimensional vector space to obtain a second target matrix. The second target matrix is ​​then encoded to obtain a second hidden representation.

8. A feature extraction device for chemical reaction data, characterized in that, include: The tag sequence acquisition module is used to acquire the target string of the current chemical reaction data, and to tag the target string using a word segmenter and a preset vocabulary to obtain the target tag sequence; A hidden representation output model is used to input the target label sequence into a target mask language model obtained by pre-training the model using historical molecular data strings and historical chemical reaction data strings, so that the target mask language model outputs the target hidden representation of the target label sequence; The target feature extraction module is used to determine the feature vector extracted from the target hidden representation as the target feature of the current chemical reaction data.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the feature extraction method for chemical reaction data as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the feature extraction method for chemical reaction data as described in any one of claims 1 to 7.

Images