A method for adverse drug reaction detection based on contrastive learning

Abstract

The invention discloses a drug adverse reaction detection method based on contrastive learning. For drug adverse reaction information obtained from social media data, a biomedical text mining method is used to mine valuable information therefrom to obtain structured data; a word vector is obtained as an initial value by pre-training words using a Word2Vector method with words as fine granularity, and external knowledge obtained from a medical dictionary is introduced into the word vector; the word vector is encoded using the most advanced BERT algorithm in deep learning to obtain a latent layer representation of the text; a hybrid contrastive learning method is used to optimize the fusion feature representation, and a new loss function is designed to train the model by combining the loss functions of unsupervised contrastive learning (UCL) and supervised contrastive learning (SCL) to improve the robustness of the model; finally, a Softmax function is used to predict the possibility of the existence of drug adverse reactions to achieve detection of drug adverse reactions.

Description

Technical Field

[0001] The present invention is to detect adverse drug reactions in social media data and judge whether a text contains content related to adverse drug reactions. Specifically, it is a method for detecting adverse drug reactions based on contrastive learning. Background Art

[0002] Adverse drug reaction (ADR) refers to harmful reactions that are unrelated to the purpose of medication when qualified drugs are used to prevent, diagnose or treat diseases. As one of the public health issues associated with high morbidity and mortality, adverse drug reactions can lead to prolonged hospital stays, unnecessary readmissions and increased health care expenditures. According to statistics from global public health organizations, the incidence of ADR in hospitalized patients worldwide is 10% to 20% per year, of which about 5% die from severe ADR. According to data analysis results from developed countries, the ADR-related hospitalization rate is 5.6% in the United States, 4.8% in Germany, and 3.2% in the United Kingdom. According to statistics, my country is a region where adverse drug reactions are more serious. According to statistics, at least 2.5 million of the more than 50 million hospitalized patients in my country are admitted to hospital due to adverse drug reactions each year, of which 250,000 to 500,000 are severe adverse reactions and about 190,000 die as a result. Therefore, monitoring and research on adverse drug reactions are very important.

[0003] With the rapid development of social networks, people tend to share their medication experiences through social media, such as Twitter, Weibo, and major forums, or consult health bloggers and online doctors about medication issues through the Internet. The information posted on social media contains the latest medication information, but this information is very scattered. If doctors and pharmacists rely on manual extraction from massive data, it will take a lot of time and be prone to errors. Therefore, it is more convenient and flexible to use natural language processing methods to extract adverse drug reaction information from social media.

[0004] Textual research on adverse drug reactions mainly mines relevant knowledge from massive biomedical literature, clinical electronic medical records, social media and other texts, involving multiple natural language processing technologies such as information retrieval, text classification, named entity recognition, and relationship extraction. Among them, monitoring adverse drug reactions and dividing massive text data to determine whether it involves specific types of adverse drug reactions is a key technology in current theoretical research. However, there are currently problems with text mining of adverse drug reactions, such as low accuracy, inability to accurately identify proprietary vocabulary, and inaccurate semantic features contained in word vectors. Therefore, in order to improve the accuracy of recognition and the robustness of the model, external medical information can be introduced into the model, and the idea of ​​contrastive learning can be used to improve the similarity between the same samples. Summary of the invention

[0005] The present invention provides a drug adverse reaction detection method based on contrastive learning. For drug adverse reaction information obtained from social media data, a biomedical text mining method is used to mine valuable information therefrom to obtain structured data; a word vector is obtained as an initial value by pre-training words using a Word2Vector method with words as fine granularity, and external knowledge obtained from a medical dictionary is introduced into the word vector; the word vector is encoded using the most advanced BERT algorithm in deep learning to obtain a latent representation of the text; a hybrid contrastive learning method is used to optimize the fusion feature representation, and a new loss function is designed to train the model by combining the loss functions of unsupervised contrastive learning (UCL) and supervised contrastive learning (SCL), so as to realize the detection of drug adverse reactions.

[0006] To achieve the above object, the technical solution of the present invention is:

[0007] A method for detecting adverse drug reactions based on contrastive learning comprises the following steps:

[0008] S1. Obtain adverse drug reaction information from social media data, and use biomedical text mining methods to mine valuable information from it to obtain structured data and keywords;

[0009] S2. Use the Word2vec method to train words at a fine-grained level to obtain word vectors, train keywords to obtain keyword vectors, align the word vectors with the keyword vectors, and then perform weighted summation to fuse them;

[0010] S3. Use the most advanced BERT algorithm in deep learning to encode word vectors and obtain the hidden layer representation of the text;

[0011] S4. Use the hybrid contrastive learning method to optimize the fusion feature representation, combine the loss functions of unsupervised contrastive learning (UCL) and supervised contrastive learning (SCL), design a new loss function to train the model, and improve the robustness of the model;

[0012] S5. Finally, the Softmax function is used to predict the possibility of adverse drug reactions and realize the detection of adverse drug reactions.

[0013] Furthermore, the step S1 is specifically as follows:

[0014] S11. Use crawler tools to crawl medication data shared by users from publicly available social media such as Twitter, Weibo, and medical forums, and perform preliminary data cleaning to unify the language and storage format;

[0015] S12. Search online to obtain three public data sets related to adverse drug reactions: ADE (Adverse Drug Effect), CADEC (CSIRO Adverse Drug Event Corpus), and PubMED;

[0016] S13. Combine the data obtained from social media with the public dataset to form text data. Each piece of data is composed of n words to get a sentence s. w ={w1,w2,…,w i ,…,w n}, where: s w represents a sentence consisting of words, w1 represents the first word in the sentence, w2 represents the second word in the sentence, and w i represents the i-th word in the sentence, w n Represents the nth word in the sentence. Each piece of data is similarly matched with the MedDRA International Medical Dictionary for Regulatory Activities M published by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use. Text classification technology is used to determine whether the text contains adverse drug reaction information. At the same time, the matched words are added as keywords after each sample text as additional features, recorded as k ij , k ij Represents a match for word w i The jth keyword of , one word may match multiple keywords, and each sentence is converted into a sequence of word-keyword pairs, that is, s wk ={(w1,k 1j ),(w2,k 2j ),…,(w i ,k ij ),…,(w n ,k nj )}Where: s wk Represents a sentence composed of words and keywords. i represents the i-th word in the sentence, k 1j represents the jth keyword that matches word w1, k 2j represents the jth keyword that matches word w2, k ij Represents a match for word w i The jth keyword, k nj Represents a match for word w n The jth keyword of

[0017] S14. Based on the keywords of whether the sample has adverse drug reactions, combined with the original labels of some samples, the original samples are divided into two categories, and the samples containing adverse drug reaction information are marked as 1, and the remaining samples are marked as 0, that is, the label is recorded as y, y=1 or y=0.

[0018] Furthermore, the step S2 is specifically as follows:

[0019] S21.Word2Vce is an efficient word vector learning method. It imports Word2Vec from the Gensim library, loads the GoogleNews-vectors-negative300 pre-trained word vector model, and converts the original text in the sample into word vectors. Its output is in: is the word vector of the i-th word, e w is the pre-trained word embedding lookup table, w i is the i-th word in the sentence, and the dimension of each word vector is d;

[0020] S22. Repeat step S21 and use the Word2Vec pre-trained word vector model to convert the keywords in some samples with keywords into word vectors in: is the kth ij The word vector corresponding to the keyword, e w is the pre-trained word embedding lookup table, k ij is matched to word w i The j-th keyword of , the dimension of each word vector is d;

[0021] S23. Pair each word with a keyword. Since each keyword contributes differently to the word, in order to better fuse the two, we adopt a bilinear attention mechanism; specifically, we assign all keywords k assigned to the i-th word ij Expressed as Its size is m times d, where m is the total number of assigned keywords and d is the dimension of each word vector. i is the set of all keyword vectors of the i-th word, is the word vector corresponding to the first keyword, is the word vector corresponding to the mth keyword, and the relevance of each word is calculated Among them: a i Represents the relevance between the ith word and its corresponding keyword, is the word vector of the i-th word, W attn is the weight matrix of bilinear attention, It's K i The transpose of is initialized according to the normal distribution; then, the contribution of the keyword to its matching word is obtained by the following method represents the keyword contribution obtained by the i-th word; finally, the weighted dictionary information is injected into the word vector in the following way It represents the final word vector of the i-th word that incorporates keyword information, and finally adds a dropout layer and a normalization layer.

[0022] Furthermore, the step S3 is specifically as follows:

[0023] S31. The processed word vector The input is sent to the Input Embedding layer, which adds the word vector, position vector, and text vector, inserts a [CLS] symbol before each text, and then outputs the encoded vector E = {e1, e2, …, e i ,…,e n}, E represents the vector set after a sentence is input into the Embedding layer, e1 represents the vector result after the first word position in the sentence is input into the Embedding layer, e2 represents the vector result after the second word position in the sentence is input into the Embedding layer, e i Represents the vector result after the i-th word position in the sentence is input into the Embedding layer, e n Represents the vector result after the nth word position in the sentence is input into the Embedding layer;

[0024] S32. Input E into the encoder, where Bert is a bidirectional encoding representation algorithm based on Transformer. The Bert network structure is formed by stacking multiple layers of TransformerEncoder. Inputting E into the Bert encoder means directly entering the Transformer encoder. The functions of each Transformer layer are as follows:

[0025] G=LN(H l-1 +MHAttn(H l-1 ))

[0026] H l =LN(G+FFN(G))

[0027] Among them: G represents the attention score of the previous hidden layer, represents the lth hidden layer, The hidden layer vector representing the position of the first word in the lth hidden layer, The hidden layer vector representing the position of the second word in the lth hidden layer, The hidden layer vector representing the position of the i-th word in the l-th hidden layer, The hidden layer vector representing the position of the nth word in the lth hidden layer, H l-1 represents the l-1th layer, which is the previous hidden layer, and H 0 =E,H 0is the result of the 0th hidden layer, E represents the initial input result, LN is layer normalization, MHAtn is the multi-head attention mechanism, and FFN is a two-layer feedforward network with RELU as the hidden activation function;

[0028] There are 12 Transformer layers in S33.Bert, which continuously input the H obtained in the previous step. l To the remaining Transformer layers, finally, get the output of the last layer of Transformer. The default output includes 2, last_hidden_state, size is (batch_size, sequence_length, hidden_size); pooler_output, size is (batch_size, hidden_size); select last_hidden_state[:,0] as the final result, which is the output result of the last text [CLS] position, representing the summary information of the entire sequence, and is a tensor with a shape of (batch_size, hidden_size).

[0029] Furthermore, the step S4 is specifically as follows:

[0030] S41. Sample construction for unsupervised contrastive learning: Double the text sample, that is, convert the processed word vector After being copied once, it is input into S31 again. Since the Transformer contains random mask Dropout, the output obtained after the same sample is input into Bert is different. The feature vector obtained by inputting the same sample twice in a batch is recorded as and Treat them as a pair of positive samples and the rest as negative samples, construct new labels and send them to loss calculation;

[0031] S42. Loss construction of unsupervised contrastive learning: According to the constructed samples, infoNCE loss is used:

[0032]

[0033] Among them, l p-ucl represents the unsupervised loss of the p-th sample, and Represents the two output results after the p-th sample is input twice into Bert coding, represents the cosine similarity of these two output results, represents the result of the qth sample after Bert encoding, N represents the total number of samples in a batch, e represents exponential calculation, and τ is a temperature constant used to control the degree of attention paid by the adjustment model to difficult samples. It is not good to be too large or too small. τ is set to 0.05;

[0034] S43. Sample construction for supervised contrastive learning: In the samples of the same round, samples of the same category are set as positive sample pairs, that is, samples with labels of 1 or 0, and samples of different categories in the same round are set as negative sample pairs, that is, samples with different labels; by comparing the labels of each sample in the same round, a new label is set for loss calculation;

[0035] S44. Loss construction of supervised contrastive learning: The loss of the supervised loss function is set to:

[0036]

[0037] Among them, l p-scl represents the supervised loss of the p-th sample, h p , is the result of a pair of positive samples after Bert encoding, h p , is the result of a pair of negative samples after Bert encoding, τ is the temperature constant, and τ is set to 0.05;

[0038] S45. Construct loss and calculate the model results: record the cross entropy loss as The unsupervised contrastive learning loss is denoted by L UCL , the supervised contrastive learning loss function is recorded as final λ1 is the final weight of the unsupervised contrastive learning loss function, and λ2 is the final weight of the supervised contrastive learning loss function. The batch_size in the model parameters is set to 128, the epoch is set to 5, and the data is sent to the model for repeated iterations. According to calculations, the model accuracy is highest when λ1=0.06 and λ2=0.1.

[0039] Furthermore, the step S5 is specifically as follows:

[0040] S51. After the model training is completed, the test set is sent to the predictor to obtain the last_hidden_state[:,0] in the output, which is the output result of the last text [cls] position and represents the summary information of the entire sequence;

[0041] S52. Use softmax to perform normalization to obtain the predicted probability. The predicted probability value is the probability ratio of containing adverse drug reactions. Finally, the maximum value of the probability is selected as the sample label.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The present invention applies the algorithm based on contrastive learning to the detection of adverse drug reactions for the first time, introduces external information of MedDRA International Medical Dictionary into the original text information, fuses word vectors and keyword vectors, expands the semantic information of word vectors, and inputs the expanded word vectors into the most advanced Bert algorithm for model encoding; using the idea of ​​contrastive learning, constructs new sample pairs that can be used for contrastive learning, designs and integrates unsupervised contrastive learning loss function and supervised contrastive learning loss function, combines the two losses for model training, improves the quality of sentence representation vectors, effectively improves the detection accuracy of adverse drug reactions, and increases the robustness of the model. The algorithm greatly retains the semantic information of sentence vectors, and shortens the spatial distance between the same samples, and is fully suitable for the relevant detection of adverse drug reactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the present invention.

[0045] Figure 2 This is a flowchart of the contrastive learning (including unsupervised contrastive learning and supervised contrastive learning) part. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the embodiments.

[0047] like Figure 1 and 2 As shown, a method for detecting adverse drug reactions based on contrastive learning comprises the following steps:

[0048] S1. Obtain adverse drug reaction information from social media data, and use biomedical text mining methods to mine valuable information from it to obtain structured data and keywords;

[0049] The step S1 is specifically as follows:

[0050] S11. Use crawler tools to crawl medication data shared by users from publicly available social media such as Twitter, Weibo, and medical forums, and perform preliminary data cleaning to unify the language and storage format;

[0051] S12. Search online to obtain three public data sets related to adverse drug reactions: ADE (Adverse Drug Effect), CADEC (CSIRO Adverse Drug Event Corpus), and PubMED;

[0052] S13. Combine the data obtained from social media with the public dataset to form text data. Each piece of data is composed of n words to get a sentence s. w ={w1,w2,…,w i ,…,w n}, where: s w represents a sentence consisting of words, w1 represents the first word in the sentence, w2 represents the second word in the sentence, and w i represents the i-th word in the sentence, w n Represents the nth word in the sentence. Each piece of data is similarly matched with the MedDRA International Medical Dictionary for Regulatory Activities M published by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use. Text classification technology is used to determine whether the text contains adverse drug reaction information. At the same time, the matched words are added as keywords after each sample text as additional features, recorded as k ij , k ij Represents a match for word w i The jth keyword of , one word may match multiple keywords, and each sentence is converted into a sequence of word-keyword pairs, that is, s wk ={(w1,k 1j ),(w2,k2j),…,(w i ,k ij ),…,(w n ,k nj )}Where: s wk Represents a sentence composed of words and keywords. i represents the i-th word in the sentence, k 1j represents the jth keyword that matches word w1, k 2j represents the jth keyword that matches word w2, k ij Represents a match for word w i The jth keyword, k nj Represents a match for word w n The jth keyword of

[0053] S14. Based on the keywords of whether the sample has adverse drug reactions, combined with the original labels of some samples, the original samples are divided into two categories, and the samples containing adverse drug reaction information are marked as 1, and the remaining samples are marked as 0, that is, the label is recorded as y, y=1 or y=0.

[0054] S2. Use the Word2vec method to train words at a fine-grained level to obtain word vectors, train keywords to obtain keyword vectors, align the word vectors with the keyword vectors, and then perform weighted summation to fuse them;

[0055] The step S2 is specifically as follows:

[0056] S21.Word2Vce is an efficient word vector learning method. It imports Word2Vec from the Gensim library, loads the GoogleNews-vectors-negative300 pre-trained word vector model, and converts the original text in the sample into word vectors. Its output is in: is the word vector of the i-th word, e w is the pre-trained word embedding lookup table, w i is the i-th word in the sentence, and the dimension of each word vector is d;

[0057] S22. Repeat step S21 and use the Word2Vec pre-trained word vector model to convert the keywords in some samples with keywords into word vectors in: is the kth ij The word vector corresponding to the keyword, e w is the pre-trained word embedding lookup table, k ij is matched to word w i The j-th keyword of , the dimension of each word vector is d;

[0058] S23. Pair each word with a keyword. Since each keyword contributes differently to the word, in order to better fuse the two, we adopt a bilinear attention mechanism; specifically, we assign all the keywords k assigned to the i-th word ij Expressed as Its size is m times d, where m is the total number of assigned keywords and d is the dimension of each word vector. i is the set of all keyword vectors of the i-th word, is the word vector corresponding to the first keyword, is the word vector corresponding to the mth keyword, and the relevance of each word is calculated Among them: a i Represents the relevance between the ith word and its corresponding keyword, is the word vector of the i-th word, W attn is the weight matrix of bilinear attention, It's K i The transpose of is initialized according to the normal distribution; then, the contribution of the keyword to its matching word is obtained by the following method represents the keyword contribution obtained by the i-th word; finally, the weighted dictionary information is injected into the word vector in the following way It represents the final word vector of the i-th word that incorporates keyword information, and finally adds a dropout layer and a normalization layer.

[0059] S3. Use the most advanced BERT algorithm in deep learning to encode word vectors and obtain the hidden layer representation of the text;

[0060] The step S3 is specifically as follows:

[0061] S31. The processed word vector The input is sent to the Input Embedding layer, which adds the word vector, position vector, and text vector, inserts a [CLS] symbol before each text, and then outputs the encoded vector E = {e1, e2, …, e i ,…,e n}, E represents the vector set after a sentence is input into the Embedding layer, e1 represents the vector result after the first word position in the sentence is input into the Embedding layer, e2 represents the vector result after the second word position in the sentence is input into the Embedding layer, e i Represents the vector result after the i-th word position in the sentence is input into the Embedding layer, e n Represents the vector result after the nth word position in the sentence is input into the Embedding layer;

[0062] S32. Input E into the encoder, where Bert is a bidirectional encoding representation algorithm based on Transformer. The Bert network structure is formed by stacking multiple layers of TransformerEncoder. Inputting E into the Bert encoder means directly entering the Transformer encoder. The functions of each Transformer layer are as follows:

[0063] G=LN(H l-1 +MHAttn(H l-1 ))

[0064] H l =LN(G+FFN(G))

[0065] Among them: G represents the attention score of the previous hidden layer, represents the lth hidden layer, The hidden layer vector representing the position of the first word in the lth hidden layer, The hidden layer vector representing the position of the second word in the lth hidden layer, The hidden layer vector representing the position of the i-th word in the l-th hidden layer, The hidden layer vector representing the position of the nth word in the lth hidden layer, H l-1 represents the l-1th layer, which is the previous hidden layer, and H 0 =E,H 0is the result of the 0th hidden layer, E represents the initial input result, LN is layer normalization, MHAtn is the multi-head attention mechanism, and FFN is a two-layer feedforward network with RELU as the hidden activation function;

[0066] There are 12 Transformer layers in S33.Bert, which continuously input the H obtained in the previous step. l To the remaining Transformer layers, finally, get the output of the last layer of Transformer. The default output includes 2, last_hidden_state, size is (batch_size, sequence_length, hidden_size); pooler_output, size is (batch_size, hidden_size); select last_hidden_state[:,0] as the final result, which is the output result of the last text [CLS] position, representing the summary information of the entire sequence, and is a tensor with a shape of (batch_size, hidden_size).

[0067] S4. Use the hybrid contrastive learning method to optimize the fusion feature representation, combine the loss functions of unsupervised contrastive learning (UCL) and supervised contrastive learning (SCL), design a new loss function to train the model, and improve the robustness of the model;

[0068] The step S4 is specifically as follows:

[0069] S41. Sample construction for unsupervised contrastive learning: Double the text sample, that is, convert the processed word vector After being copied once, it is input into S31 again. Since the Transformer contains random mask Dropout, the output obtained after the same sample is input into Bert is different. The feature vector obtained by inputting the same sample twice in a batch is recorded as and Treat them as a pair of positive samples and the rest as negative samples, construct new labels and send them to loss calculation;

[0070] S42. Loss construction of unsupervised contrastive learning: According to the constructed samples, infoNCE loss is used:

[0071]

[0072] Among them, l p-ucl represents the unsupervised loss of the p-th sample, and Represents the two output results after the p-th sample is input twice into Bert coding, represents the cosine similarity of these two output results, represents the result of the qth sample after Bert encoding, N represents the total number of samples in a batch, e represents exponential calculation, and τ is a temperature constant used to control the degree of attention paid by the adjustment model to difficult samples. It is not good to be too large or too small. τ is set to 0.05;

[0073] S43. Sample construction for supervised contrastive learning: In the samples of the same round, samples of the same category are set as positive sample pairs, that is, samples with labels of 1 or 0, and samples of different categories in the same round are set as negative sample pairs, that is, samples with different labels; by comparing the labels of each sample in the same round, a new label is set for loss calculation;

[0074] S44. Loss construction of supervised contrastive learning: The loss of the supervised loss function is set to:

[0075]

[0076] Among them, l p-scl represents the supervised loss of the p-th sample, h p , is the result of a pair of positive samples after Bert encoding, h p , is the result of a pair of negative samples after Bert encoding, τ is the temperature constant, and τ is set to 0.05;

[0077] S45. Construct loss and calculate the model results: record the cross entropy loss as The unsupervised contrastive learning loss is denoted by The supervised contrastive learning loss function is denoted as final λ1 is the final weight of the unsupervised contrastive learning loss function, and λ2 is the final weight of the supervised contrastive learning loss function. The batch_size in the model parameters is set to 128, the epoch is set to 5, and the data is sent to the model for repeated iterations. According to calculations, the model accuracy is highest when λ1=0.06 and λ2=0.1.

[0078] S5. Finally, the Softmax function is used to predict the possibility of adverse drug reactions and detect adverse drug reactions;

[0079] The step S5 is specifically as follows:

[0080] S51. After the model training is completed, the test set is sent to the predictor to obtain the last_hidden_state[:,0] in the output, which is the output result of the last text [cls] position and represents the summary information of the entire sequence;

[0081] S52. Use softmax to perform normalization to obtain the predicted probability. The predicted probability value is the probability ratio of containing adverse drug reactions. Finally, the maximum value of the probability is selected as the sample label.

[0082] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Without departing from the spirit and principles of the present invention, any modifications, equivalent substitutions, improvements, etc. made to the present invention by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A method for detecting adverse drug reactions based on contrastive learning, characterized in that: The following steps are involved: S1. Obtain adverse drug reaction information from social media data, and use biomedical text mining methods to mine valuable information from it to obtain structured data and keywords; S2. Use the Word2vec method to train words at a fine-grained level to obtain word vectors, train keywords to obtain keyword vectors, align the word vectors with the keyword vectors, and then perform weighted summation to fuse them; S3. Use the most advanced BERT algorithm in deep learning to encode word vectors and obtain the hidden layer representation of the text; S4. Use the hybrid contrastive learning method to optimize the fusion feature representation, combine the loss functions of unsupervised contrastive learning (UCL) and supervised contrastive learning (SCL), design a new loss function to train the model, and improve the robustness of the model; S5. Finally, the Softmax function is used to predict the possibility of adverse drug reactions and detect adverse drug reactions; The step S4 is specifically as follows: S41. Sample construction for unsupervised contrastive learning: Double the text sample and transform the processed word vectors After being copied once, it is input again into step S31. Since the Transformer includes random mask Dropout, the output obtained after the same sample is input into Bert is different. The feature vector obtained by inputting the same sample twice in a batch is recorded as and Treat them as a pair of positive samples and the rest as negative samples, construct new labels and send them to loss calculation; S42. Loss construction of unsupervised contrastive learning: According to the constructed samples, infoNCE loss is used: Among them, l p-ucl represents the unsupervised loss of the p-th sample, and Represents the two output results after the p-th sample is input twice into Bert coding, represents the cosine similarity of these two output results, represents the result of the qth sample after Bert encoding, N represents the total number of samples in a batch, e represents the exponential calculation, and τ is the temperature constant used to control the degree of attention paid by the adjustment model to difficult samples. τ is set to 0.05; S43. Sample construction for supervised contrastive learning: In the samples of the same round, samples of the same category are set as positive sample pairs, that is, samples with labels of 1 or 0, and samples of different categories in the same round are set as negative sample pairs, that is, samples with different labels; by comparing the labels of each sample in the same round, a new label is set for loss calculation; S44. Loss construction of supervised contrastive learning: The loss of the supervised loss function is set to: Among them, l p-scl represents the supervised loss of the p-th sample, h p , is the result of a pair of positive samples after Bert encoding, h p , is the result of a pair of negative samples after Bert encoding, τ is the temperature constant, τ is set to 0.05; S45. Construct loss and calculate the model results: record the cross entropy loss as The unsupervised contrastive learning loss is denoted by The supervised contrastive learning loss function is denoted as final λ1 is the final weight of the unsupervised contrastive learning loss function, and λ2 is the final weight of the supervised contrastive learning loss function. The batch_size in the model parameters is set to 128, the epoch is set to 5, and the data is sent to the model for repeated iterations. According to calculations, the model accuracy is highest when λ1=0.06 and λ2=0.

1.

2. The method for detecting adverse drug reactions based on contrastive learning according to claim 1, characterized in that: The step S1 is specifically as follows: S11. Use crawler tools to crawl medication data shared by users from publicly published social media, and perform preliminary data cleaning to unify the language and storage format; S12. Search the Internet to obtain three public data sets related to adverse drug reactions: ADE, CADEC, and PubMED; S13. Combine the data obtained from social media with the public dataset to form text data. Each piece of data is composed of n words to get a sentence s. w ={w1, w2, ..., w i , ..., w n }, where: s w represents a sentence consisting of words, w1 represents the first word in the sentence, w2 represents the second word in the sentence, and w i represents the i-th word in the sentence, w n Represents the nth word in the sentence. Each piece of data is similarly matched with the MedDRA International Medical Dictionary for Regulatory Activities M published by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use. Text classification technology is used to determine whether the text contains adverse drug reaction information. At the same time, the matched words are added as keywords after each sample text as additional features, recorded as k ij , k ij Represents a match for word w i The jth keyword of , one word can match multiple keywords, and each sentence is converted into a sequence of word-keyword pairs, that is, s wk ={(w1, k 1j ),(w2,k 2j ),...,(w i , k ij ),...,(w n , k nj )}Where: s wk Represents a sentence composed of words and keywords. i represents the i-th word in the sentence, k 1j represents the jth keyword that matches word w1, k 2j represents the jth keyword that matches word w2, k ij Represents a match for word w i The jth keyword, k nj Represents a match for word w n The jth keyword of S14. Based on the keywords of whether the sample has adverse drug reactions, combined with the original labels of some samples, the original samples are divided into two categories, and the samples containing adverse drug reaction information are marked as 1, and the remaining samples are marked as 0, that is, the label is recorded as y, y=1 or y=0.

3. The method for detecting adverse drug reactions based on contrastive learning according to claim 1, characterized in that: The step S2 is specifically as follows: S21.Word2Vce is a word vector learning method. It imports Word2Vec from the Gensim library, loads the GoogleNews-vectors-negative300 pre-trained word vector model, and converts the original text in the sample into a word vector. Its output is in: is the word vector of the i-th word, e w is the pre-trained word embedding lookup table, w i is the i-th word in the sentence, and the dimension of each word vector is d; S22. Repeat step S21 and use the Word2Vec pre-trained word vector model to convert the keywords in some samples with keywords into word vectors in: is the kth ij The word vector corresponding to the keyword, e w is the pre-trained word embedding lookup table, k ij is matched to word w i The j-th keyword of , the dimension of each word vector is d; S23. Pair each word with a keyword. Since each keyword contributes differently to the word, a bilinear attention mechanism is used. All keywords k assigned to the i-th word are combined. ij Expressed as Its size is m times d, where m is the total number of assigned keywords and d is the dimension of each word vector. i is the set of all keyword vectors of the i-th word, is the word vector corresponding to the first keyword, is the word vector corresponding to the mth keyword, and the relevance of each word is calculated Among them: a i Represents the relevance between the ith word and its corresponding keyword, is the word vector of the i-th word, W attn is the weight matrix of bilinear attention, It's K i The transpose of is initialized according to the normal distribution; then, the contribution of the keyword to its matching word is obtained by the following method represents the keyword contribution obtained by the i-th word; finally, the weighted dictionary information is injected into the word vector in the following way It represents the final word vector of the i-th word that incorporates keyword information, and finally adds a dropout layer and a normalization layer.

4. The method for detecting adverse drug reactions based on contrastive learning according to claim 1, characterized in that: The step S3 is specifically as follows: S31. The processed word vector The input is sent to the Input Embedding layer, which adds the word vector, position vector, and text vector, inserts a [CLS] symbol before each text, and then outputs the encoded vector E = {e1, e2, ..., e i , ..., e n }, E represents the vector set after a sentence is input into the Embedding layer, e1 represents the vector result after the first word position in the sentence is input into the Embedding layer, e2 represents the vector result after the second word position in the sentence is input into the Embedding layer, e i Represents the vector result after the i-th word position in the sentence is input into the Embedding layer, e n Represents the vector result after the nth word position in the sentence is input into the Embedding layer; S32. Input E into the encoder, where Bert is a bidirectional encoding representation algorithm based on Transformer. The Bert network structure is formed by stacking multiple layers of Transformer Encoders. Inputting E into the Bert encoder means directly entering the Transformer encoder. The functions of each Transformer layer are as follows: G=LN(H l-1 +MHAttn(H l-1 )) H l =LN(G+FFN(G)) Among them: G represents the attention score of the previous hidden layer, represents the lth hidden layer, The hidden layer vector representing the position of the first word in the lth hidden layer, The hidden layer vector representing the position of the second word in the lth hidden layer, The hidden layer vector representing the position of the i-th word in the l-th hidden layer, The hidden layer vector representing the position of the nth word in the lth hidden layer, H l-1 represents the l-1th layer, which is the previous hidden layer, and H 0 =E,H 0 is the result of the 0th hidden layer, E represents the initial input result, LN is layer normalization, MHAtn is the multi-head attention mechanism, and FFN is a two-layer feedforward network with RELU as the hidden activation function; There are 12 Transformer layers in S33.Bert, which continuously input the H obtained in the previous step. l To the remaining Transformer layers, finally, get the output of the last layer of Transformer. The default output includes 2, last_hidden_state, the size is batch_size, sequence_length, hidden_size; pooler_output, the size is batch_size, hidden_size; select last_hidden_state[:, 0] as the final result, which is the output result of the last text [CLS] position, representing the summary information of the entire sequence, and is a tensor with a shape of batch_size, hidden_size.

5. The method for detecting adverse drug reactions based on contrastive learning according to claim 1, characterized in that: The step S5 is specifically as follows: S51. After the model training is completed, the test set is sent to the predictor to obtain the last_hidden_state[:,0] in the output, which is the output result of the last text [cls] position and represents the summary information of the entire sequence; S52. Use softmax to perform normalization to obtain the predicted probability. The predicted probability value is the probability ratio of containing adverse drug reactions. Finally, the maximum value of the probability is selected as the sample label.

Images