Adverse reaction signal analysis method and system for drug safety

Abstract

This application provides an adverse reaction signal analysis method for pharmacovigilance, applicable to the fields of pharmacovigilance and artificial intelligence. The method includes: acquiring spontaneously reported adverse drug reaction data; generating a candidate set of adverse drug reaction signals based on the spontaneously reported data; extracting multidimensional features for each candidate signal in the candidate signal set; inputting the multidimensional features, including historical stratified verification features, into a machine learning model to calculate a confidence score for each candidate signal; adjusting a judgment threshold based on at least one of the following: confidence score distribution information, historical signal verification results, and system feedback information; identifying candidate signals with confidence scores higher than the judgment threshold as target adverse reaction signals; and outputting the system feedback information as the error between the prediction and verification results; sorting the target adverse reaction signals according to their confidence scores. This application also provides an adverse reaction signal analysis system for pharmacovigilance.

Description

Technical Field

[0001] This application relates to the field of pharmacovigilance technology, specifically to a method and system for analyzing adverse reaction signals for pharmacovigilance. Background Technology

[0002] Adverse drug reaction monitoring is a crucial aspect of ensuring public medication safety. Currently, the field of pharmacovigilance generally relies on the large-scale data accumulated by spontaneous reporting systems (such as the FDA's Adverse Drug Reaction Reporting System FAERS) to analyze the potential associations between drugs and adverse reactions using statistical methods in order to identify potential safety risk signals.

[0003] Existing analytical methods typically employ pre-defined statistical models and fixed thresholds. For example, by constructing drug-adverse reaction contingency tables, calculating statistical indicators such as the reporting odds ratio (ROR) and proportional reporting ratio (PRR), drug-adverse reaction combinations that meet fixed threshold conditions (e.g., ROR > 1 and p < 0.05) are identified as risk signals. However, because the scale, structure, and distribution of spontaneously reported data dynamically change over time—for example, the number of reports is low in the early stages of a new drug's market launch and surges later—fixed thresholds are prone to generating false positives or false negatives under different data environments, leading to inconsistent accuracy in signal screening. Cases that have been confirmed as real risks or false signals by clinical studies or regulatory agencies are not systematically fed back into subsequent analytical processes, making each analysis a "start from scratch" process, hindering the accumulation of experience, and making it difficult to continuously optimize the judgment strategy.

[0004] Therefore, there is an urgent need for an adverse reaction signal analysis method for pharmacovigilance to improve the accuracy, stability and intelligence of signal recognition. Summary of the Invention

[0005] In view of the above problems, this application provides a method and system for pharmacovigilance-oriented adverse reaction signal analysis, which is used for automated analysis, risk signal identification and dynamic assessment of adverse reaction data after drug marketing.

[0006] According to a first aspect of this application, a method for analyzing adverse reaction signals for pharmacovigilance is provided, comprising: acquiring spontaneously reported adverse drug reaction data; generating a set of candidate adverse drug reaction signals based on the spontaneously reported data; extracting multidimensional features for each candidate signal in the candidate signal set, the multidimensional features including historical stratified verification features, the historical stratified verification features including historical signal verification result features stratified according to verification source, verification intensity, and / or verification time stage; inputting the multidimensional features containing the historical stratified verification features into a machine learning model to calculate a confidence score corresponding to each candidate signal; the confidence score being used to characterize the probability that the candidate signal is a real adverse reaction risk; adjusting a judgment threshold based on at least one of the distribution information of the confidence score, the historical signal verification results, and system feedback information, and determining candidate signals with confidence scores higher than the judgment threshold as target adverse reaction signals, the system feedback information being the error between the prediction result and the verification result; and sorting the target adverse reaction signals according to the confidence scores and outputting them.

[0007] According to embodiments of this application, the historical signal verification result features are obtained by hierarchically labeling the historical verification results according to the verification source, verification intensity, and / or verification time stage. This includes: firstly stratifying the historical verification results according to the verification source to obtain verification source level features, where the verification source includes at least one of regulatory agency announcements, system reviews, real-world studies, or expert feedback; secondly stratifying the historical verification results according to the verification intensity to obtain verification intensity level features, where the verification intensity includes at least one of strong verification, medium verification, or weak verification; thirdly stratifying the historical verification results according to the verification time stage to obtain verification time stage features, where the verification time stage includes at least one of early verification, mature verification, or long-term verification; and finally, obtaining the historical signal verification result features based on the verification source level features, the verification intensity level features, and the verification time stage features.

[0008] According to an embodiment of this application, adjusting the judgment threshold based on at least one of the distribution information of the credibility score, the historical signal verification results, and the system feedback information includes: determining the credibility score corresponding to a preset quantile as the judgment threshold; determining the judgment threshold based on the mean and standard deviation of the credibility score; determining the threshold that optimizes the classification performance index in the historical verification data as the judgment threshold; and adjusting the judgment threshold based on the error value between the prediction result and the verification result.

[0009] According to embodiments of this application, the multidimensional features also include at least one of the following features: time distribution features of adverse reaction reports, reporter type features, report source features, combination drug complexity features, or adverse reaction severity features.

[0010] According to an embodiment of this application, the calculation of the credibility score corresponding to each candidate signal includes: concatenating the multidimensional features into a feature vector; inputting the feature vector into a pre-trained machine learning model, wherein the model outputs a continuous value as the credibility score through weighted summation and nonlinear mapping; wherein the parameters of the machine learning model are pre-trained using historical verification results as training samples.

[0011] According to embodiments of this application, the machine learning model includes one or more of supervised learning models, semi-supervised learning models, and incremental learning models.

[0012] According to an embodiment of this application, the method further includes: performing artificial intelligence analysis on the sorted target adverse reaction signals and their corresponding multidimensional features and credibility scores to generate a natural language analysis report.

[0013] According to an embodiment of this application, the method further includes: using the acquired historical verification results and / or expert feedback information as learning samples to update the parameters of the machine learning model and the judgment threshold.

[0014] According to an embodiment of this application, when updating the parameters of the machine learning model, an incremental learning approach is adopted, which updates the model parameters using newly obtained historical verification results or expert feedback information without completely retraining the machine learning model.

[0015] A second aspect of this application provides an adverse reaction signal analysis system for pharmacovigilance, comprising: a data acquisition module for acquiring spontaneously reported adverse drug reaction data; a candidate signal generation module for generating a set of candidate adverse drug reaction signals based on the spontaneously reported data; a feature extraction module for extracting multidimensional features for each candidate signal in the candidate signal set, the multidimensional features including historical stratified verification features, the historical stratified verification features including historical signal verification result features stratified according to verification source, verification intensity, and / or verification time stage; and a credibility score calculation module for inputting the multidimensional features containing the historical stratified verification features into a machine learning model to calculate the credibility score corresponding to each candidate signal. The system includes: a reliability score, which characterizes the probability that a candidate signal represents a real adverse reaction risk; a threshold adjustment module, which adjusts a judgment threshold based on at least one of the distribution information of the reliability score, the historical signal verification results, and system feedback information, and filters candidate signals with reliability scores higher than the judgment threshold as target adverse reaction signals; a target adverse reaction signal output module, which sorts the target adverse reaction signals according to the reliability score and outputs them; and an artificial intelligence analysis module, which performs artificial intelligence analysis on the sorted target adverse reaction signals and their corresponding multidimensional features and reliability scores to generate a natural language analysis report.

[0016] Compared with related technologies, this application has the following advantages:

[0017] By introducing historical signal verification results features that are hierarchically labeled according to verification source, verification strength, and / or verification time stage, the machine learning model can distinguish the reliability of verification evidence at different levels, improving the accuracy of signal scoring. By dynamically adjusting the judgment threshold based on the distribution of credibility scores, historical signal verification results, and system feedback information (the error between prediction and verification results), the signal screening criteria can adaptively change with data distribution and system operating experience, avoiding misjudgments caused by fixed thresholds. By using historical verification results and expert feedback information to update model parameters and judgment thresholds, the system achieves closed-loop learning and continuous evolution. Attached Figure Description

[0018] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 A flowchart illustrating an adverse reaction signal analysis method for pharmacovigilance according to an embodiment of this application is shown schematically.

[0020] Figure 2A schematic diagram illustrating multidimensional features according to an embodiment of this application is shown.

[0021] Figure 3 A flowchart illustrating a method for determining the characteristics of historical signal verification results according to an embodiment of this application is shown.

[0022] Figure 4 The illustration shows a data forest diagram of adverse reaction signals using ribavirin as an example, according to an embodiment of this application.

[0023] Figure 5 A schematic diagram of an adverse reaction signal analysis method for pharmacovigilance according to another embodiment of this application is shown.

[0024] Figure 6 The diagram illustrates the architecture of an adverse reaction signal analysis system for pharmacovigilance according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] First, let's explain the technical terms that appear in the embodiments:

[0030] Spontaneously reported data refers to records submitted voluntarily to regulatory agencies or pharmaceutical manufacturers by healthcare professionals, patients, or marketing authorization holders regarding adverse events following medication use. These reports are not derived from organized research or clinical trials but are based on real-world observations during actual clinical medication use. Spontaneously reported data is one of the primary data sources for signal detection in the field of pharmacovigilance; typical databases include the FDA Adverse Event Reporting System (FAERS).

[0031] Candidate signals: These are data units constructed based on spontaneously reported data to represent potential associations between drugs and adverse reactions. Each candidate signal contains at least one combination of a drug identifier and an adverse reaction identifier, and may further include statistical information related to that combination (such as the number of reports, the distribution of reporting time, etc.). For example, "Drug A + Rash" is a candidate signal. The candidate signal set is the collection of all drug-adverse reaction pairs to be analyzed.

[0032] Multidimensional features refer to numerical indicators extracted from candidate signals to describe various aspects of the signal's attributes. Multidimensional features are typically represented as vectors, with each dimension corresponding to a specific attribute. In the embodiments of this application, multidimensional features include at least historical stratified verification features, and may also include features such as the temporal distribution of adverse reaction reports, reporter type features, report source features, complexity of combined medication use, and severity of adverse reactions.

[0033] Historical Stratified Validation Features: These refer to the features formed by structurally labeling the validation results of signals that have been validated in the past (e.g., confirmed as real risks or false signals by clinical studies, regulatory announcements, or expert review) according to preset stratification dimensions. Stratification dimensions include: Validation Source: This refers to the origin or method of generation of the validation result. Validation results from different sources have different levels of credibility. In this application embodiment, validation sources may include regulatory agency announcements (such as safety notifications from the FDA and EMA), systematic reviews (peer-reviewed meta-analyses or systematic reviews), real-world studies (studies based on large-scale observational data), and expert feedback (manual review conclusions from pharmacovigilance experts), etc. Typically, validation results from regulatory agency announcements have the highest credibility weight. Validation Strength: This refers to the strength of the evidence upon which the validation result is based. In this application embodiment, validation strength is divided into strong validation, moderate validation, and weak validation. Strong validation refers to evidence of a clear causal relationship (e.g., positive de-excitation / re-excitation, clear pathophysiological mechanism); moderate validation refers to statistically significant association but the causal mechanism is not yet clear; weak validation refers to only case reports or a lack of controlled analysis. Validation Time Period: This refers to the drug's market launch period corresponding to the validation conclusion. This application's embodiments divide the validation time phases into early validation (within 2-3 years after drug launch), mature validation (3-5 years after launch), and long-term validation (more than 5 years after launch). Introducing the time dimension can reflect the timeliness of validation results; recent validation results generally have higher reference value than long-term validation results. The essence of historical stratified validation features is to quantify the above-mentioned stratification information into a calculable numerical form (e.g., the weight, proportion, or statistical indicator of different strata) as input to the machine learning model.

[0034] Credibility score: This refers to a continuous numerical value output by a machine learning model, representing the probability that each candidate signal represents a real adverse reaction risk. The score is typically defined in the range of 0 to 1 (or 0 to 100), with higher values ​​indicating a greater likelihood that the signal represents a real risk. The credibility score is the core basis for subsequent threshold adjustments and signal priority ranking.

[0035] Judgment threshold: This refers to the boundary value used to filter target signals from candidate signals. In this embodiment, the judgment threshold is not a fixed constant, but is dynamically determined based on the confidence score distribution of the current batch of candidate signals, historical signal verification results, and system feedback information. Only candidate signals with a confidence score higher than this threshold will be identified as target adverse reaction signals. The dynamic threshold mechanism avoids the misjudgment problem caused by using a fixed threshold (such as ROR>1 and p<0.05) in traditional methods.

[0036] System feedback information refers to information generated by the system itself during the operation of the method in this embodiment, reflecting the difference between the prediction result and the objective verification result. Specifically, system feedback information is the error between the prediction result and the verification result. For example, when the system determines a candidate signal as a high-confidence signal (predicted as a true risk), but subsequent clinical research or expert review confirms that the signal is a false signal, the system records the inconsistency (error) between the prediction and verification and uses it as feedback information to adjust the judgment threshold. System feedback information is one of the important bases for achieving adaptive threshold adjustment.

[0037] Target adverse reaction signals: These refer to adverse reaction signals that, after being screened by the method provided in the embodiments of this application, are deemed to have high credibility and warrant further attention or reporting. Target adverse reaction signals are candidate signals with credibility scores higher than the dynamic judgment threshold, which are then sorted and output for use in pharmacovigilance analysis, risk assessment, or regulatory decision-making.

[0038] Supervised learning models, semi-supervised learning models, and incremental learning models: Supervised learning models are machine learning models trained using historical data with real labels (i.e., confirmed as real risks or false signals) to learn the mapping relationship between features and labels, thereby predicting unlabeled candidate signals. Semi-supervised learning models are machine learning models capable of being trained using both a small amount of labeled data and a large amount of unlabeled data. This model is suitable for scenarios with limited historical validation data but a large number of unlabeled candidate signals, and can reduce labeling costs to some extent. Incremental learning models are machine learning models capable of locally updating the current model parameters (e.g., through one or several gradient descent iterations) to absorb new knowledge when new data arrives, without completely retraining the entire model. Incremental learning is a key technology for achieving continuous system evolution and improving operational efficiency.

[0039] Incremental learning refers to a model update strategy where, whenever new training samples are obtained (e.g., new historical validation results or expert feedback), the model only fine-tunes its current parameters based on these new samples, without needing to train from scratch using all historical data. Incremental learning enables models to quickly adapt to changes in data distribution and significantly reduces computational resource consumption, making it an important way to achieve closed-loop self-learning.

[0040] Based on the aforementioned technical problems, this application provides a method for adverse reaction signal analysis for pharmacovigilance, comprising: acquiring spontaneously reported adverse drug reaction data; generating a candidate set of adverse drug reaction signals based on the spontaneously reported data; extracting multidimensional features for each candidate signal in the candidate signal set, the multidimensional features including historical stratified verification features, the historical stratified verification features including historical signal verification result features stratified according to verification source, verification intensity and / or verification time stage; inputting the multidimensional features containing historical stratified verification features into a machine learning model to calculate the credibility score corresponding to each candidate signal; the credibility score is used to characterize the probability that the candidate signal is a real adverse reaction risk; adjusting the judgment threshold according to at least one of the credibility score distribution information, historical signal verification results and system feedback information, and determining the candidate signals with credibility scores higher than the judgment threshold as target adverse reaction signals, the system feedback information being the error between the prediction result and the verification result; and sorting the target adverse reaction signals according to the credibility score and outputting the results.

[0041] Figure 1 A flowchart illustrating an adverse reaction signal analysis method for pharmacovigilance according to an embodiment of this application is shown schematically. Figure 1 As shown, the adverse reaction signal analysis method 300 for pharmacovigilance according to an embodiment of this application may include steps S110 to S160.

[0042] In step S110, spontaneous reporting data of adverse drug reactions are obtained.

[0043] In one example, the system retrieves raw report data from a spontaneously reported database (such as FAERS). Spontaneously reported data consists of records of medication-related adverse events voluntarily submitted by healthcare professionals, patients, or marketing authorization holders. Each report includes at least the drug name, a description of the adverse reaction, the reporting time, the reporter type, and geographic information. Preprocessing operations include deduplication, missing value imputation, and standardization coding of adverse reaction terms using the MedDRA dictionary. This step lays the data foundation for subsequent signal generation and ensures data quality.

[0044] Specifically, the system first connects to one or more spontaneous reporting data sources via API interfaces, database connections, or file imports, such as the FDA Adverse Event Reporting System (FAERS), the WHO Global Case Safety Reporting Database (VigiBase), or the data platform of the domestic Adverse Drug Reaction Monitoring Center. The system supports periodic incremental fetching (such as quarterly updates) or real-time streaming access. For each original report, the system parses the following core fields: drug name (including generic name, brand name, and batch number, normalized using a standard drug dictionary such as the WHO Drug Dictionary, mapping different names to a unified standard drug identifier); adverse reaction description (original text description, first using natural language processing technology to extract key medical entities such as rash and liver damage, then mapping to the MedDRA (International Medical Terminology Dictionary) preferred terminology level to generate a standard adverse reaction code PT code); report time (formatted as a standard date YYYY-MM-DD, used for subsequent time distribution feature calculations); reporter type (classified as doctor, pharmacist, other medical professionals, consumer, etc., categorized through keyword matching or preset field values); geographic information (extracting country and region codes for geographic clustering analysis); combined medication information (identifying other drugs used in the report besides the target drug, recording their names and quantities); severity marker (judging the severity of the adverse reaction based on regulatory standards such as whether it leads to hospitalization, endangers life, or causes death). The system then automatically performs data cleaning and quality control: Duplicate reports are removed based on unique report identifiers (such as the ISR number for FAERS); if no unique identifier exists, deduplication is performed based on the similarity of fields such as patient age, gender, medication, adverse reactions, and report date; for records missing report times, if they can be inferred from other fields, they are added; otherwise, they are marked as unknown and assigned a default value (such as median padding) in subsequent feature construction; records missing reporter type can be categorized as "unknown"; the system checks if the drug name is in the standard dictionary; if not, it attempts fuzzy matching or transfers the case to manual review; it checks if the adverse reaction code is valid; and it removes obviously unreasonable data (such as reports with a date earlier than the drug's market launch date). The cleaned data is stored in an intermediate database table in a unified format. Each record contains a standard drug ID, a standard adverse reaction MedDRA code, a report date, a reporter type classification code, a country / region code, a list of combined medications, a severity indicator, and the original report source ID.

[0045] Through the standardization, cleaning, and structuring processes described above, the data acquisition unit transforms raw, unstructured spontaneous reports into a clean, computable format, ensuring the accuracy and consistency of subsequent feature extraction and signal analysis. In particular, the MedDRA encoding of adverse reaction terms and the standardized mapping of drugs enable comparison and aggregation of reports from different sources and in different languages ​​within the same semantic space.

[0046] In step S120, a set of candidate signals for adverse drug reactions is generated based on spontaneously reported data.

[0047] In one example, the system treats each unique "drug-adverse reaction" pair as a candidate signal. For example, "drug A + rash". The system collects all reports involving this combination, removes duplicates and incomplete entries, forming a candidate signal. The set of all candidate signals is the candidate signal set. This step transforms the raw reports into analyzable structured units, each representing a potential drug-adverse reaction association. By generating candidate signals, the system can batch process thousands of drug-adverse reaction combinations, improving analytical efficiency.

[0048] Based on the spontaneous adverse drug reaction report data obtained in step S110, all report data are traversed, and each unique (drug ID, adverse reaction code) combination is grouped and aggregated. For example, "ribavirin + anemia" and "ribavirin + rash" are two different drug-adverse reaction pairs, each corresponding to a candidate signal. To reduce computational load, the system can set a minimum report threshold (e.g., at least 3 independent reports) to filter out overly sparse candidate signals; this threshold can be adaptively adjusted according to the drug's market launch time and the total number of reports (e.g., for new drugs, it can be set to 1 report). In this embodiment, all candidate signals with at least 1 report are retained to ensure that new rare signals are not missed.

[0049] Because the same drug-adverse reaction pair may generate multiple similar entries due to spelling differences or synonyms in the original reports, the system performs further deduplication and merging: different brand names of the same drug (such as "ribavirin capsules", "Ribavirin", "virazole") are automatically merged after being mapped to the same standard drug ID; different descriptions of the same adverse reaction (such as "nausea", "vomiting", "nausea") are merged into the same PT code through the MedDRA hierarchy (PT to LLT); for the same patient, the same drug-adverse reaction, and multiple reports within a similar time period (which may be follow-up reports), the system merges them according to preset rules (such as retaining the earliest or latest one) to avoid duplicate counting. Each candidate signal is assigned a set of metadata, including: total number of reports, report time series (earliest report date, latest report date), reporter type distribution (such as number of physician reports, number of patient reports), geographic distribution (number of countries / regions involved and major contributing regions), frequency of combination therapy (average number of combination drugs involved in the report), and number and proportion of severe cases. This statistical information will serve as input for the subsequent feature extraction module. All candidate signals and their accompanying statistics are organized into an index table (e.g., a JSON array or a database table). Each record contains an internally generated unique signal identifier, a standard drug name, a standard adverse reaction name, a total number of reports, the earliest report date, the latest report date, reporter statistics (JSON object), geographic statistics (JSON object), the proportion of severe cases, and the complexity of combination therapy.

[0050] In step S130, multidimensional features are extracted for each candidate signal in the candidate signal set.

[0051] According to embodiments of this application, the multidimensional features include historical hierarchical verification features, which include historical signal verification result features that are hierarchically labeled according to verification source, verification intensity, and / or verification time stage.

[0052] In one example, for each candidate signal, the system extracts a set of numerical feature vectors. These include at least "historical stratified verification features." These features refer to structured features formed by stratifying and labeling historically verified (e.g., confirmed or denied by regulatory agencies) signal verification results according to their verification source (e.g., regulatory announcements, system reviews, real-world studies, expert feedback), verification strength (strong, medium, weak), and / or verification time stage (early, mature, long-term). For example, for the candidate signal "drug B+ liver injury," the system can statistically analyze the frequency and verification conclusions of the signal in regulatory announcements, the statistical results in system reviews, and the support in case reports, using these statistics as components of the feature vector. By introducing stratified labeling, the model can distinguish historical evidence from different sources and with different strengths, avoiding treating all verification results the same, thereby improving the accuracy of subsequent scoring. The technical effect of this step is to transform messy historical knowledge into computable numerical features, enabling the machine learning model to "learn" the reliability differences of evidence at different levels.

[0053] In step S140, the multidimensional features containing historical stratified verification features are input into the machine learning model to calculate the credibility score corresponding to each candidate signal; the credibility score is used to characterize the probability that the candidate signal is a real adverse reaction risk.

[0054] In one example, the system inputs the multidimensional feature vector constructed in step S130 (which necessarily includes historical stratified verification features) into a pre-trained machine learning model. This model can be a supervised learning model, a semi-supervised learning model, or an incremental learning model. The model outputs a continuous numerical value as a confidence score by performing weighted summation and nonlinear mapping (e.g., using logistic regression or a neural network) on the feature vector, typically ranging from 0 to 1. The higher the score, the greater the likelihood that the candidate signal represents a genuine adverse reaction risk. This step integrates multidimensional features into an intuitive confidence index, facilitating subsequent threshold comparison and ranking. By using multidimensional features that include historical stratified verification features, the model can comprehensively consider multiple attributes of the signal, outputting a more reliable score than traditional single statistical indicators.

[0055] In step S150, the judgment threshold is adjusted based on at least one of the following: the distribution information of the credibility score, the historical signal verification results, and the system feedback information. Candidate signals with credibility scores higher than the judgment threshold are identified as target adverse reaction signals. The system feedback information is the error between the prediction result and the verification result.

[0056] In one example, the system does not use a fixed threshold but dynamically calculates the judgment threshold. Specific adjustment criteria include: the overall distribution of the confidence scores of all current candidate signals (e.g., mean, standard deviation, quantiles); historical signal verification results (i.e., the true state of previously confirmed signals and their corresponding scores); and system feedback information, i.e., the error between the predicted result (judgment based on the current threshold) and subsequent actual verification results (e.g., false positive rate, false negative rate, or error value). Based on at least one of these criteria, the system calculates a suitable threshold. For example, when historical verification results show a high false positive rate, the system can increase the threshold; when the system feedback error is large, the threshold can be adjusted through an error-driven approach. Then, the confidence score of each candidate signal is compared with this dynamic threshold, and signals exceeding the threshold are identified as target adverse reaction signals (i.e., high-risk signals). The technical advantage of this step is that it avoids the problem of signal screening deviating from the actual data distribution caused by a fixed threshold, allowing the threshold to adaptively change with data distribution, historical experience, and system performance, thus improving the adaptability and accuracy of signal screening.

[0057] In step S160, the target adverse reaction signals are sorted according to the confidence score and then output.

[0058] In one example, the system sorts identified adverse reaction signals from highest to lowest confidence score, generating a ranked list. This list can be output as a visual table, data file, or report for use by pharmacovigilance analysts or regulatory decision-makers. The ranked list ensures that high-risk signals are prioritized, improving the efficiency of manual review and reducing the risk of missed detections.

[0059] The method provided in this application, by introducing historical hierarchical verification features and a dynamic threshold adjustment mechanism, achieves credibility scoring and adaptive screening of candidate signals. Compared with existing fixed threshold methods, this application can improve scoring accuracy by utilizing hierarchical historical knowledge and dynamically adjust the screening criteria based on data distribution and system feedback, significantly reducing false positive and false negative rates.

[0060] Figure 2 A schematic diagram illustrating multidimensional features according to an embodiment of this application is shown.

[0061] According to embodiments of this application, the multidimensional features also include at least one of the following features: time distribution features of adverse reaction reports, reporter type features, report source features, combination drug complexity features, or adverse reaction severity features.

[0062] like Figure 2 As shown, based on the foregoing embodiments, the multidimensional features may further include at least one of the following features:

[0063] The temporal distribution characteristics of adverse reaction reports reflect the degree of signal aggregation within different time windows and can be calculated as follows: T_score = (number of reports of this signal within a specific time window) / (historical average number of reports of this signal). If T_score is significantly greater than 1, it suggests that there may be abnormal aggregation in the near future.

[0064] Reporter type characteristics: Reflects the professional level of the report source. For example, the proportion of physician reports can be calculated: Physician_ratio = Number of physician reports for this signal / Total number of reports for this signal. Physician reports often have higher clinical evaluation value.

[0065] Combination drug complexity characteristic: This reflects the degree of complexity of a patient using multiple medications simultaneously in this signal report, and can be defined as: Complexity = log(number of combined medications + 1), where the number of combined medications is the average or median of all reports for this signal. Higher complexity indicates greater uncertainty in attributing the signal to a single medication.

[0066] Severity characteristics of adverse reactions: Reflects the degree of impact of the adverse reactions involved in the signal on the patient's health. It can be calculated as: Severity_score = number of severe cases / total number of cases. Severe cases include outcomes such as hospitalization, life-threatening, death, or congenital malformations.

[0067] Historical signal validation result features (non-stratified version): can be calculated as: Validation_score = number of times the signal (or similar signals) has been validated as true in history / total number of times the signal has appeared in history. This feature can be directly used as a summary metric for stratified validation features, but stratified validation features further distinguish the source, intensity, and stage of validation. These features, together with historical stratified validation features, constitute a complete feature vector, which is input into the machine learning model for scoring. By introducing more dimensional features, the model can more comprehensively capture the multifaceted attributes of the signal, thereby further improving the accuracy of the credibility score.

[0068] Based on the above embodiments, by Figure 3 It further defines the specific method for obtaining the characteristics of historical signal verification results by hierarchical labeling according to the verification source, verification intensity, and / or verification time stage.

[0069] Figure 3 A flowchart illustrating a method for determining the features of historical signal verification results according to an embodiment of this application is shown schematically. Figure 3 As shown, it includes steps S210 to S240.

[0070] In step S210, the historical verification results are first stratified according to the verification source to obtain the verification source hierarchical characteristics. The verification source includes at least one of regulatory agency announcements, system reviews, real-world studies, or expert feedback.

[0071] In step S220, the historical verification results are stratified in a second way according to the verification strength to obtain the verification strength level characteristics. The verification strength includes at least one of strong verification, medium verification, or weak verification.

[0072] In step S230, the historical verification results are stratified in a third way according to the verification time stage to obtain the verification time stage characteristics. The verification time stage includes at least one of early verification, mature verification, or long-term verification.

[0073] In step S240, the historical signal verification result characteristics are obtained based on the verification source level characteristics, verification intensity level characteristics, and verification time stage characteristics.

[0074] In one example, hierarchical annotation includes the following operations:

[0075] The first tier: Historical validation results are tiered according to their validation source to obtain the validation source hierarchy characteristics. Validation sources include at least one of the following: regulatory agency announcements, systematic reviews, real-world studies, or expert feedback. For example, validation results from official safety announcements from the FDA or EMA are set at the highest level, those from peer-reviewed systematic reviews are set at the second highest level, those from real-world studies or registry studies are set at the middle level, and those from case reports or individual expert experience are set at the lowest level.

[0076] Second stratification: Historical validation results are stratified according to validation strength to obtain validation strength level characteristics. Validation strength includes at least one of strong validation, moderate validation, or weak validation. For example, strong validation is supported by positive evidence of de-excitation / re-excitation or a clear pathophysiological mechanism; moderate validation is supported by statistically significant association but with an unclear mechanism; and weak validation is supported by only case reports and a lack of control analysis.

[0077] The third stratification: Historical validation results are stratified according to the validation time stage to obtain the validation time stage characteristics. The validation time stage includes at least one of the following: early validation (within 2-3 years after launch), mature validation (3-5 years after launch), or long-term validation (more than 5 years).

[0078] Based on the verification source hierarchy features, verification intensity level features, and verification time stage features obtained above, the characteristics of historical signal verification results are finally determined. For example, a multi-dimensional vector can be constructed, with each dimension corresponding to a statistical value under a hierarchical category (such as the number of times a certain signal is verified as true in the regulatory announcement level / the total number of times), or a comprehensive feature value can be obtained by weighted summation.

[0079] In one exemplary implementation, the system can evaluate the historical performance of different layers. For example, for each layer (such as the "Regulatory Announcement" layer), its historical validation accuracy is calculated: Accuracy_layer = Number of correctly validated signals in this layer / Total number of signals in this layer. Based on the accuracy, the weight of this layer can be further calculated: w_layer = Accuracy_layer / (Accuracy_layer + Error_layer), where Error_layer is the error rate. In this way, validation information from different layers is given different influences in subsequent models.

[0080] The technical advantage of this application's embodiments lies in the fact that by performing multi-dimensional stratification of historical verification results, the system can make refined use of verification evidence from different sources, with different intensities, and different timeframes, avoiding a simplistic and crude "one-size-fits-all" approach. The stratified features allow the machine learning model to learn which levels of verification results are more reliable, thus assigning them higher weight during scoring. Simultaneously, the stratified historical performance evaluation results can also be used for threshold updates and risk level labeling (such as distinguishing between "high-risk signals that have been verified by multiple sources" and "weak signals that require continuous observation"), improving the credibility of signal evaluation and decision support capabilities.

[0081] According to embodiments of this application, adjusting the judgment threshold based on at least one of the following: distribution information of the confidence score, historical signal verification results, and system feedback information includes: determining the confidence score corresponding to a preset quantile as the judgment threshold; determining the judgment threshold based on the mean and standard deviation of the confidence score; determining the threshold that optimizes the classification performance index in historical verification data as the judgment threshold; and adjusting the judgment threshold based on the error value between the prediction result and the verification result.

[0082] This application further defines a specific method for adjusting the judgment threshold based on at least one of the following: distribution information of the confidence score, historical signal verification results, and system feedback information. Adjusting the judgment threshold includes at least one of the following methods:

[0083] (1) Quantile method: The confidence score corresponding to the preset quantile is determined as the judgment threshold. For example, the system calculates the 90th quantile of the confidence score distribution of all candidate signals and uses the score value corresponding to this quantile as the threshold. This method is suitable for scenarios where it is desirable to filter out a fixed proportion of signals with the highest scores. It is simple to operate and can ensure that the number of signals filtered in each round is relatively stable.

[0084] (2) Statistical distribution method: The judgment threshold is determined based on the mean and standard deviation of the credibility score. Specifically, the mean μ and standard deviation σ of the score are calculated, and the threshold is set as μ + k*σ, where k is an adjustable parameter (e.g., 2 or 3). This method assumes that the score follows a unimodal distribution, and signals that exceed the mean by several times the standard deviation can be regarded as abnormally high-risk signals. It is suitable for scenarios where the score distribution is relatively concentrated and the extreme values ​​have practical significance.

[0085] (3) Performance optimization method: The threshold that optimizes the classification performance metrics in the historical validation data is determined as the decision threshold. The system iterates through possible thresholds on the existing historical validation dataset (i.e., the set of signals with known true labels) and selects the threshold that maximizes the classification performance metrics (such as F1 score, accuracy, Youden index, etc.) as the decision threshold for the current batch of signals. This method directly uses historical data as a benchmark, pursuing the best classification effect on known samples, and has strong empirical evidence.

[0086] (4) Error-driven approach: The judgment threshold is adjusted based on the error between the prediction result and the verification result. The system records the error between each prediction result (judgment conclusion based on the current threshold) and the subsequent actual verification result (e.g., an increase in the false positive rate or false negative rate), and then updates the threshold according to a predetermined adjustment strategy. For example, an iterative formula is used: Threshold_new = Threshold_old + α * Δerror, where α is the adjustment coefficient and Δerror is the error (e.g., the difference between the false positive rate and the target false positive rate). This approach allows the threshold to "learn" based on real-time feedback from the system, adapting to changes in data distribution.

[0087] In some feasible examples, the above four methods can be used individually or in combination. For example, in a composite implementation, the system calculates the decision threshold by weighting and summing the data size term, the score distribution term, and the feedback adjustment term: Threshold = w1*T_data + w2*T_distribution + w3*T_feedback, where T_data is based on the data size or time dimension, T_distribution is a threshold term based on the score distribution (such as quantiles or μ+kσ), and T_feedback is an adjustment term based on historical validation or system feedback. The weight parameters can be dynamically determined based on the data size (such as sample size proportion), time decay (exp(-λ*Δt)), or feedback performance (accuracy / (accuracy+error)).

[0088] This application provides various flexible threshold determination methods, enabling the system to select the most suitable strategy based on specific data characteristics, historical verification results, and real-time feedback, or to combine the advantages of multiple strategies, further improving the adaptability and accuracy of threshold adjustment. Through weighted fusion and time decay mechanisms, the system can better cope with concept drift and long-term evolution.

[0089] According to an embodiment of this application, calculating the credibility score corresponding to each candidate signal includes: concatenating multi-dimensional features into a feature vector; inputting the feature vector into a pre-trained machine learning model, and the model outputting a continuous value as the credibility score through weighted summation and nonlinear mapping; wherein, the parameters of the machine learning model are pre-trained using historical verification results as training samples.

[0090] In one example, a confidence score is used to characterize the probability or relative confidence level of each candidate signal as a genuine adverse reaction signal, and serves as the basis for selecting high-confidence signals. Specifically, the signal confidence score can be calculated by performing a function mapping on the feature vector. For example, the system organizes all features extracted from each candidate signal (including historical stratified verification features and other optional features) into a numerical vector in a predetermined order. Suppose the feature vector of a candidate signal is represented as: X = [x1, x2, x3, x4, x5], where each component corresponds to the time trend feature, the reporter type feature (such as the proportion of doctors), the complexity of combined medication, the severity of adverse reactions, and the historical verification score feature, respectively. The splicing process requires that all features have been standardized or normalized to avoid differences in units affecting model performance.

[0091] The feature vector is input into a pre-trained machine learning model. The model outputs a continuous value as a confidence score through weighted summation and nonlinear mapping. Specifically, the model can be represented as Score = f(w*X + b), where w is the weight vector, b is the bias, and f is a nonlinear mapping function (such as the sigmoid function f(z) = 1 / (1 + e^(-z))), with an output range of (0,1)). The weighted summation and nonlinear mapping here are the most common computational forms in machine learning and can be regarded as a single-layer neural network or logistic regression model. Deeper neural networks or other regression models can also be used. The parameters of the machine learning model (such as the weight vector w and the bias b) are pre-trained using historical validation results as training samples: during training, the feature vectors of historical candidate signals and their corresponding true labels (such as "real risk" or "false signal") are used, and the parameters are optimized by minimizing the loss function (such as cross-entropy loss).

[0092] In a specific example, X can be taken as: [1.2, 0.65, 2.1, 0.30, 0.80]. The scoring model can be expressed as: Score = f(w·X + b); where w = [w1, w2, w3, w4, w5] are the model weight parameters, b is the bias term, and f is the nonlinear mapping function. In one implementation, the function f can be the Sigmoid function: f(z) = 1 / (1 + e^(-z)). For example, in an exemplary parameter setting: w = [0.8, 1.2, -0.5, 1.5, 2.0], b = -1.0. Inputting the above feature vector into the model for calculation yields: Score ≈ 0.87. The scoring result indicates that the candidate signal has high credibility, that is, it is highly likely to be a real adverse reaction signal.

[0093] The embodiments of this application clarify the mathematical implementation of credibility score calculation, enabling those skilled in the art to train and score models based on this description. By concatenating multidimensional features into vectors and performing weighted nonlinear mapping, the model can automatically learn the influence weights of each feature on the final score, without the need for manual rule setting. Parameters are obtained through training with historical data, ensuring the objectivity and repeatability of the scoring.

[0094] It should be noted that the above-mentioned feature forms, model structures and parameter settings are only examples, and the embodiments of this application do not limit the specific calculation methods, model types or parameter values.

[0095] Figure 4 The illustration shows a data forest diagram of adverse reaction signals using ribavirin as an example, according to an embodiment of this application.

[0096] This application uses the exemplary drug ribavirin as an example to demonstrate the adverse reaction signal analysis results output by the system in actual operation. Specifically, as shown below... Figure 4 As shown.

[0097] Figure 4The graph is a forest plot, with the horizontal axis representing the confidence score (represented on logarithmic or linear coordinates, with values ​​in the graph being the score values), and the vertical axis representing the adverse reaction terms (ADRs). The graph lists 15 candidate adverse reaction signals related to ribavirin, including Hepatitis C, Hepatocellular carcinoma, Anaemia, and Rash. Using the method described in this application, spontaneously reported ribavirin data is analyzed repeatedly (e.g., using cross-validation or bootstrap resampling), with each analysis outputting a confidence score for each candidate signal. Each point in the graph represents the average confidence score of that signal across multiple analyses, and the horizontal lines or numbers in parentheses on either side of the point represent the minimum and maximum scores (i.e., confidence intervals or ranges). For example, the score for Hepatitis C is 104.98 (range 98.1 to 112.35), the score for Anaemia is 9.55 (range 9.11 to 10.02), and the score for Rash is 1.57 (range 1.44 to 1.71). The score corresponding to each adverse reaction in the figure is the confidence score in this embodiment, used to characterize the likelihood that the candidate signal is a genuine adverse reaction risk. A higher score indicates that ribavirin is more likely to cause the adverse reaction. For example, the scores for Hepatitis C and Hepatocellular carcinoma are much higher than other signals, suggesting that ribavirin has the strongest association with liver-related adverse reactions, which is consistent with the known safety profile of ribavirin. The system can dynamically adjust the judgment threshold based on the confidence score distribution of all candidate signals. For example, if the current threshold is set to 5.0, signals with scores higher than 5.0 (Hepatitis c, Hepatocellular carcinoma, Anaemia) will be identified as target adverse reaction signals; if the threshold is set to 2.0, signals such as treatment failure and Haemoglobin decreased will also be included. By adjusting the threshold, the system can flexibly control the sensitivity of the screening. The signals are sorted from highest to lowest confidence score, and the order from top to bottom in the figure represents the sorting results. The highest-risk signal (Hepatitis c) is listed first, making it easier for pharmacovigilance personnel to prioritize it. Some signals in the figure (such as Anaemia and Rash) have been repeatedly verified in literature or regulatory announcements. When extracting historical stratified verification features, the system will give these signals higher weight, thus making their confidence scores more accurate.

[0098] pass Figure 4The forest plot shown demonstrates how the system outputs the confidence score and uncertainty range of each candidate signal in an intuitive and visual format, helping pharmacovigilance experts quickly identify high-risk signals. Compared to traditional single statistical indicators (such as ROR value), the confidence score in this embodiment integrates multi-dimensional features (including historical stratified verification features), resulting in more reliable scores. Simultaneously, the dynamic threshold mechanism automatically determines the screening boundary based on the score distribution, avoiding the subjectivity and blindness of manually setting fixed thresholds.

[0099] According to embodiments of this application, the machine learning model includes one or more of supervised learning models, semi-supervised learning models, and incremental learning models.

[0100] This application further defines the types of machine learning models based on the above embodiments. Machine learning models include one or more of supervised learning models, semi-supervised learning models, and incremental learning models. Supervised learning models are suitable for situations where a large amount of labeled historical validation data is available. The model learns the mapping relationship between features and labels by minimizing the prediction error on the training set. Its advantage is high accuracy, but its disadvantage is the need for a large amount of labeled data. Semi-supervised learning models are suitable for situations where only a small number of signals in the historical validation data are labeled, while most signals are unlabeled. This type of model can utilize the distribution information in unlabeled data to assist training, reducing dependence on labeled data. In the field of pharmacovigilance, most candidate signals lack conclusive validation conclusions, therefore semi-supervised learning has high practical value. Incremental learning models are suitable for scenarios that require continuous absorption of new validation results. This type of model can perform local updates based on the current parameters (such as one or several gradient descent iterations) without complete retraining when receiving new data, thereby achieving continuous model evolution. Incremental learning enables the system to quickly adapt to changes in data distribution while saving computational resources.

[0101] Depending on different data conditions and application requirements, the most suitable model type or combination thereof can be selected, enabling flexible deployment of the system in different application scenarios.

[0102] Figure 5 A schematic diagram of an adverse reaction signal analysis method for pharmacovigilance according to another embodiment of this application is shown.

[0103] According to an embodiment of this application, the method further includes step S170.

[0104] In step S170, the sorted target adverse reaction signals and their corresponding multidimensional features and credibility scores are subjected to artificial intelligence analysis to generate a natural language analysis report.

[0105] In one example, such as Figure 5As shown, the system inputs a sorted list of high-priority target signals, along with the characteristics and scores of each signal, into a pre-built artificial intelligence analysis module. This module, built on a large language model, can automatically generate highly readable natural language reports. Report content may include: an overview of high-risk drug-adverse reaction pairs (e.g., listing the top 10 most suspicious signals and their scores); historical validation references (e.g., indicating which signals have been confirmed by regulatory agencies or refuted by studies); risk warnings for combined drug use (e.g., for signals involving multiple drugs, indicating possible drug interactions); signal trend analysis (e.g., warnings of signals with a recent surge in reports); and alerts to abnormal signals (e.g., signals with scores significantly higher than similar historical signals). The generated reports can be output in a structured format (text plus charts) for direct use by regulatory or research personnel.

[0106] By translating machine learning scores into human-readable, easily understandable natural language reports, the interpretability and usability of the system are greatly improved. Pharmacovigilance experts no longer need to directly view complex scoring tables; instead, they can directly read comprehensive reports containing risk overviews, historical references, and trend analyses, enabling them to make more efficient regulatory or research decisions.

[0107] According to embodiments of this application, the method further includes: using the acquired historical verification results and / or expert feedback information as learning samples to update the parameters and judgment threshold of the machine learning model.

[0108] In one example, after completing a signal analysis and outputting the target signal, the system continuously collects external validation information. This information includes: historical validation results: the true status of the signal confirmed through subsequent clinical studies, literature updates, or regulatory announcements (e.g., a signal being confirmed as a real risk or a false signal). Expert feedback: the manual review conclusions of pharmacovigilance experts on the system's output signals, such as confirmation, denial, or delayed assessment.

[0109] The system adds these newly acquired samples (feature vectors plus corresponding ground truth or feedback labels) to the training dataset and periodically or in real-time retrains or fine-tunes the machine learning model, updating model parameters (such as weights w and biases b). During model updates, a loss function can be constructed based on historical validation results, for example, cross-entropy loss can be used for binary classification problems: L = -[y·log(Score) + (1-y)·log(1-Score)], where y is the ground truth label of the signal (1 represents real risk, 0 represents false signal), and Score is the confidence score of the model output. The model parameters are updated using gradient descent: w_new = w_old - η·∂L / ∂w, b_new = b_old - η·∂L / ∂b, where η is the learning rate. Simultaneously, the system recalculates or adjusts the decision threshold based on these new validation results (e.g., selecting the optimal threshold on the updated validation set through performance optimization, or updating the threshold through error-driven methods: Threshold_new = Threshold_old + α·Δerror).

[0110] The technical effect of this application embodiment is that it forms a closed-loop self-learning mechanism, which enables the system to learn from each verification feedback and continuously optimize its scoring model and threshold strategy, so that the accuracy and stability of signal judgment can be continuously improved with time and data accumulation, avoiding performance degradation caused by unchanging behavior.

[0111] According to embodiments of this application, when updating the parameters of a machine learning model, an incremental learning approach is adopted, which updates the model parameters using newly obtained historical validation results or expert feedback information without completely retraining the machine learning model.

[0112] In one example, based on the above embodiment, an incremental learning approach is further specified when updating the parameters of the machine learning model. Incremental learning means that when new validation results or expert feedback are obtained, the system does not retrain the model from scratch using all historical data. Instead, based on the current model parameters, it performs one or more gradient descent updates (or other online learning algorithms) using new samples, allowing the model parameters to gradually evolve to adapt to the new data patterns. For example, stochastic gradient descent (SGD) can be used for updates: w_new = w_old - η·∂L / ∂w, where η is the learning rate and L is the loss function (such as cross-entropy). Similarly, thresholds can also be updated incrementally, such as Threshold_new = Threshold_old + α·Δerror.

[0113] The technical advantage of this application's embodiments lies in the fact that incremental learning avoids the enormous computational overhead of retraining the entire model for each update, enabling rapid model evolution. This is particularly important for pharmacovigilance systems because data is continuously flowing in; incremental learning allows the system to absorb new knowledge in near real-time, maintaining the model's up-to-date state while reducing the demand for computational resources.

[0114] Based on the above-described adverse reaction signal analysis method for pharmacovigilance, embodiments of this application also provide an adverse reaction signal analysis system for pharmacovigilance. The following will be combined with... Figure 6 The device is described in detail.

[0115] Figure 6 The diagram illustrates the architecture of an adverse reaction signal analysis system for pharmacovigilance according to an embodiment of this application.

[0116] like Figure 6 As shown, the adverse reaction signal analysis system 600 for pharmacovigilance in this embodiment includes a data acquisition module 610, a candidate signal generation module 620, a feature extraction module 630, a credibility score calculation module 640, a threshold adjustment module 650, a target adverse reaction signal output module 660, and an artificial intelligence analysis module 670.

[0117] The data acquisition module 610 is used to acquire spontaneously reported adverse drug reaction data. In one embodiment, the data acquisition module 610 can be used to perform step S110 described above, which will not be repeated here.

[0118] The candidate signal generation module 620 is used to generate a set of candidate signals for adverse drug reactions based on spontaneously reported data. In one embodiment, the candidate signal generation module 620 can be used to perform step S120 described above, which will not be repeated here.

[0119] The feature extraction module 630 is used to extract multi-dimensional features for each candidate signal in the candidate signal set. The multi-dimensional features include historical hierarchical verification features, which include historical signal verification result features that are hierarchically labeled according to verification source, verification strength, and / or verification time stage. In one embodiment, the feature extraction module 630 can be used to perform step S130 described above, which will not be repeated here.

[0120] The credibility score calculation module 640 is used to input multidimensional features including historical hierarchical verification features into a machine learning model to calculate the credibility score corresponding to each candidate signal; the credibility score is used to characterize the probability that the candidate signal is a real adverse reaction risk. In one embodiment, the credibility score calculation module 640 can be used to perform step S140 described above, which will not be repeated here.

[0121] The threshold adjustment module 650 is used to adjust the judgment threshold based on at least one of the following: the distribution information of the credibility score, the historical signal verification results, and the system feedback information. Candidate signals with credibility scores higher than the judgment threshold are then selected as target adverse reaction signals. The system feedback information is the error between the prediction result and the verification result. In one embodiment, the threshold adjustment module 650 can be used to perform step S150 as described above, which will not be repeated here.

[0122] The target adverse reaction signal output module 660 is used to sort and output the target adverse reaction signals according to their confidence scores. In one embodiment, the target adverse reaction signal output module 660 can be used to perform step S160 described above, which will not be repeated here.

[0123] The artificial intelligence analysis module 670 is used to perform artificial intelligence analysis on the sorted target adverse reaction signals and their corresponding multidimensional features and credibility scores to generate a natural language analysis report. In one embodiment, the artificial intelligence analysis module 670 can be used to perform step S170 described above, which will not be repeated here.

[0124] The data acquisition module connects to spontaneously generated adverse drug reaction (ADR) reports, such as those from the FDA Adverse Event Reporting System (FAERS). This unit periodically or in real-time acquires raw reports, each containing at least the following fields: drug name, ADR description, reporting time, reporter type (e.g., physician, pharmacist, patient), and geographic information. Preprocessing is then performed: deduplication (removing identical duplicate records), missing value imputation (e.g., filling in missing reporter types with "unknown"), and terminology standardization (encoding ADR descriptions using the MedDRA dictionary). The preprocessed data forms a structured intermediate table for use by subsequent units. This step ensures data quality and lays the foundation for signal analysis.

[0125] The signal candidate generation module receives the preprocessed data and defines each unique "drug-adverse reaction" pair as a candidate signal. For example, "ribavirin + rash" is a candidate signal. The system iterates through all reports, extracts all occurrences of drug-adverse reaction combinations, removes duplicate entries due to spelling errors or synonyms, and aggregates information from multiple reports under the same combination (such as the number of reports, time distribution, etc.) to form a candidate signal set. This set covers all potential risk associations to be analyzed, providing objects for subsequent feature extraction.

[0126] The feature extraction module extracts numerical, multi-dimensional features for each candidate signal. In addition to conventional features, the core feature is the historical hierarchical verification feature. This feature is generated by hierarchically labeling historical verification results. Specifically, the system maintains a historical verification knowledge base, which records signals that have been confirmed or refuted by regulatory agencies, system reviews, real-world studies, or expert feedback. During feature construction, for the current candidate signal, the system statistically analyzes the frequency of its occurrence and verification conclusions in history according to the verification source (e.g., regulatory announcements, system reviews, case reports), verification strength (strong, medium, weak), and verification time stage (early, mature, long-term), forming a multi-dimensional vector. For example, for "ribavirin + anemia," the system might find that: it appeared 3 times in the history of regulatory announcements, all of which were confirmed; it appeared 5 times in the system review level, with 4 of those being confirmed; and it appeared 20 times in the case report level, with only 10 of those being confirmed. These statistics serve as feature components. Other common features include: time distribution features (such as the ratio of recent reports to historical average reports, T_score = number of reports in a specific window / historical average number of reports), reporter type features (such as the proportion of physician reports, Physician_ratio = number of physician reports / total number of reports), combination drug complexity (Complexity = log(number of combination drugs + 1)), and adverse reaction severity (Severity_score = number of severe cases / total number of cases). All features are concatenated into a feature vector.

[0127] The feature vector of each candidate signal is input into the credibility score calculation module, which outputs a continuous value between 0 and 1 as the credibility score. The model parameters (weights w and bias b) are obtained through training on historical validation data, using the cross-entropy loss function during training. This score quantifies the probability that the candidate signal represents a real adverse reaction risk; the higher the score, the greater the risk.

[0128] The threshold adjustment module receives the confidence scores of all candidate signals, as well as historical signal verification results and system feedback information (the error between the prediction result and subsequent verification results). It dynamically calculates the judgment threshold and compares the confidence score of each signal with this threshold. Signals with scores higher than the threshold are identified as target adverse reaction signals (i.e., high-confidence risk signals). This dynamic threshold mechanism avoids the data mismatch problem caused by fixed thresholds.

[0129] The target adverse reaction signal output module sorts the target adverse reaction signals from highest to lowest according to their confidence scores, generating a sorted list. Each signal in the list can be accompanied by a feature summary and score value. The artificial intelligence analysis module receives the sorted target adverse reaction signals and their corresponding multidimensional features and confidence scores, and automatically generates a natural language analysis report using a large language model.

[0130] According to embodiments of this application, any and multiple modules among the data acquisition module 610, candidate signal generation module 620, feature extraction module 630, credibility score calculation module 640, threshold adjustment module 650, target adverse reaction signal output module 660, and artificial intelligence analysis module 670 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the data acquisition module 610, candidate signal generation module 620, feature extraction module 630, credibility score calculation module 640, threshold adjustment module 650, target adverse reaction signal output module 660, and artificial intelligence analysis module 670 can be at least partially implemented as hardware circuits, such as field-programmable gate arrays, programmable logic arrays, systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits, or any other reasonable means of integrating or packaging circuits, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the data acquisition module 610, candidate signal generation module 620, feature extraction module 630, credibility score calculation module 640, threshold adjustment module 650, target adverse reaction signal output module 660, and artificial intelligence analysis module 670 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0132] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

Claims

1. A method of adverse drug reaction signal analysis for pharmacovigilance, characterized by, include: Obtain spontaneously reported data on adverse drug reactions; A set of candidate signals for adverse drug reactions is generated based on the spontaneously reported data; Multidimensional features are extracted for each candidate signal in the candidate signal set. The multidimensional features include historical hierarchical verification features, which include historical signal verification result features that are hierarchically labeled according to verification source, verification intensity and / or verification time stage. The multidimensional features containing the historical hierarchical verification features are input into the machine learning model to calculate the credibility score corresponding to each candidate signal; the credibility score is used to characterize the probability that the candidate signal is a real adverse reaction risk; The judgment threshold is adjusted based on at least one of the distribution information of the credibility score, the historical signal verification results, and the system feedback information, and candidate signals with credibility scores higher than the judgment threshold are identified as target adverse reaction signals. The system feedback information is the error between the prediction result and the verification result. The target adverse reaction signals are sorted and output according to the credibility score.

2. The method of claim 1, wherein, Historical signal verification results are characterized by hierarchical labeling based on verification source, verification intensity, and / or verification time stage. The historical verification results are first-leveled according to the verification source to obtain the verification source hierarchical characteristics. The verification source includes at least one of regulatory agency announcements, system reviews, real-world studies or expert feedback. The historical verification results are stratified according to the verification strength to obtain the verification strength level characteristics. The verification strength includes at least one of strong verification, medium verification, or weak verification. The historical verification results are stratified into a third layer according to the verification time stage to obtain verification time stage characteristics. The verification time stage includes at least one of early verification, maturity verification, or long-term verification. Historical signal verification result features are obtained based on the verification source level features, the verification intensity level features, and the verification time stage features.

3. The method of claim 1, wherein, Adjusting the judgment threshold based on at least one of the following: the distribution information of the credibility score, the historical signal verification results, and the system feedback information: The confidence score corresponding to the preset quantile is determined as the judgment threshold; The judgment threshold is determined based on the mean and standard deviation of the credibility score; The threshold that optimizes the classification performance index in the historical validation data is determined as the decision threshold. The judgment threshold is adjusted based on the error value between the prediction result and the verification result.

4. The method of claim 1, wherein, The multidimensional features also include at least one of the following features: time distribution features of adverse reaction reports, reporter type features, report source features, combination drug complexity features, or adverse reaction severity features.

5. The method of claim 1, wherein, The calculation of the credibility score for each candidate signal includes: The multidimensional features are concatenated into a feature vector; The feature vector is input into a pre-trained machine learning model, which outputs a continuous value as the credibility score through weighted summation and nonlinear mapping; wherein the parameters of the machine learning model are pre-trained using historical verification results as training samples.

6. The method of claim 5, wherein, The machine learning model includes one or more of the following: supervised learning model, semi-supervised learning model, and incremental learning model.

7. The method of claim 1, wherein, The method further includes: The sorted target adverse reaction signals and their corresponding multidimensional features and credibility scores are analyzed using artificial intelligence to generate a natural language analysis report.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The parameters of the machine learning model and the judgment threshold are updated using the acquired historical verification results and / or expert feedback information as learning samples.

9. The method according to claim 8, characterized in that, When updating the parameters of the machine learning model, an incremental learning approach is adopted, which updates the model parameters using newly obtained historical validation results or expert feedback information without completely retraining the machine learning model.

10. An adverse reaction signal analysis system for pharmacovigilance, characterized in that, The system includes: The data acquisition module is used to acquire spontaneous reports of adverse drug reactions. A candidate signal generation module is used to generate a set of candidate signals for adverse drug reactions based on the spontaneously reported data. The feature extraction module is used to extract multi-dimensional features for each candidate signal in the candidate signal set. The multi-dimensional features include historical hierarchical verification features, which include historical signal verification result features that are hierarchically labeled according to verification source, verification intensity and / or verification time stage. The credibility score calculation module is used to input the multidimensional features containing the historical hierarchical verification features into the machine learning model to calculate the credibility score corresponding to each candidate signal; the credibility score is used to characterize the probability that the candidate signal is a real adverse reaction risk; The threshold adjustment module is used to adjust the judgment threshold according to at least one of the distribution information of the credibility score, the historical signal verification results, and the system feedback information, and to filter candidate signals with credibility scores higher than the judgment threshold as target adverse reaction signals. The system feedback information is the error between the prediction result and the verification result. The target adverse reaction signal output module is used to sort the target adverse reaction signals according to the confidence score and then output them. The artificial intelligence analysis module is used to perform artificial intelligence analysis on the sorted target adverse reaction signals and their corresponding multidimensional features and credibility scores to generate a natural language analysis report.

Images