Event labeling method, device, equipment and product for treatment causal inference

Abstract

The application relates to the field of information communication technology specially applicable to brain medical data mining, and particularly relates to an event labeling method and device for therapeutic causal inference, equipment and products. The method comprises the following labeling steps of clinical case report table data: performing keyword matching on clinical case report table data, extracting records containing preset medical entities as to-be-labeled records, and acquiring context data corresponding to the records; loading a state machine model, wherein the state machine model defines the treatment state of a subject and state conversion rules; inputting the conversion rules of the state machine model and the context data into a large language model to obtain a labeling result of the to-be-labeled records, wherein the labeling result indicates whether the record is a concomitant event or background medication; and outputting the labeling result. The application can improve the accuracy of therapeutic benefit analysis.

Description

Technical Field

[0001] This application relates to the field of information and communication technology, specifically applicable to brain medical data mining, and particularly to an event labeling method, apparatus, device, and product for causal inference in treatment. Background Technology

[0002] In clinical trials for neurodegenerative diseases such as Alzheimer's, impaired cognitive function in participants leads to significant adherence problems, resulting in frequent comorbidities during the trials. These events include the temporary use of psychotropic medications due to worsening psychiatric symptoms, or withdrawal from the trial due to insufficient efficacy. The ICH E9 (R1) guideline requires causal inference for comorbidities, clarifying the estimation target of treatment effect through the Estimand framework. Traditional methods rely on statisticians to classify and label events according to pre-defined rules when analyzing clinical case report data. In Alzheimer's trials, the event types are complex and the data volume is large, making manual labeling problematic in terms of efficiency and consistency.

[0003] To improve processing efficiency, we began exploring the use of automated techniques to classify events in the data. These techniques primarily rely on keyword matching or pre-defined labels for identification, lacking a dynamic detection mechanism for the dependency between events and the subject's treatment status. In real-world data, the same medical entity can possess different causal attributes under different temporal conditions. For example, memantine was a background medication before randomization, but became salvage therapy when it was newly initiated after cognitive score deterioration. The former is used as a covariate in statistical adjustment, while the latter, as a concomitant event, requires defined treatment strategies. Keyword matching cannot determine whether an event triggered a subject's transition from an in-treatment state to a protocol deviation state based on the temporal relationships reflected in the data, easily confusing the two and leading to insufficient accuracy in subsequent treatment benefit analyses. Summary of the Invention

[0004] To address, or at least partially address, the aforementioned technical problems, this application provides an event labeling method for causal inference of treatment, which can improve the accuracy of treatment benefit analysis.

[0005] Firstly, this application provides an event annotation method for causal inference in treatment, including the following annotation steps for clinical case report form data: Perform keyword matching on clinical case report data to extract records containing preset medical entities; For each extracted record, it is used as a record to be labeled, and the context data corresponding to that record is obtained. Load a state machine model, which defines the treatment state of the subject and the rules for state transitions; The transformation rules of the state machine model and the context data are input into the large language model to obtain the annotation results of the record to be annotated. The annotation results indicate that the record is a concomitant event or background medication. Output the annotation results to complete the annotation of the record to be annotated.

[0006] Optionally, obtaining the context data corresponding to the record includes: Extract the event features of the records to be labeled; Based on the event characteristics, query the preset prior probability of the trigger signal time distribution and calculate the posterior probability of multiple candidate time windows; Based on the posterior probability and the preset upper limit of the lexical budget, the lexical budget is allocated to the multiple candidate time windows; According to the lexical budget allocated to each time window, data records are clipped from the corresponding time range, and the clipped data records are spliced ​​together to form a multi-scale combined context, which serves as the context data.

[0007] Optionally, the posterior probability is calculated using Bayes' theorem based on the prior probability and the event features; The lexical budget allocation is calculated according to the following rules: the lexical budget allocated to each time window is equal to the upper limit of the lexical budget multiplied by the proportion of the posterior probability of that window to the sum of the posterior probabilities of all windows.

[0008] Optionally, for each time window, when the number of tokens in the original data records within the corresponding time range exceeds the token budget allocated to that window, the original data records are sorted by relevance, and records with higher relevance are loaded first, until the cumulative number of tokens reaches the token budget. When splicing the cropped data records, the data of different time windows are labeled according to the time scale, and the time range of each time window is marked in the multi-scale combination context.

[0009] Optionally, the event labeling method for treating causal inference further includes: recording the information value of each time window in the actual labeling, and adjusting the prior probability using a Bayesian update rule, including: The information value of each time window is evaluated based on whether it contains key triggering signals and the confidence level of the annotation results. When the cumulative number of labeled samples of the same event type reaches the preset sample quantity threshold, the empirical distribution of each time window in actual labeling is statistically analyzed. The prior probability is adjusted using a Bayesian update formula, which weights the prior probability with the empirical distribution, wherein a smoothing coefficient is used to balance the weights of historical priors and new observation data.

[0010] Optionally, the step of inputting the transformation rules of the state machine model and the context data into the large language model includes: Construct structured prompt words, which include the time and description information of the record to be labeled, the multi-scale combined context, and the transformation rule description of the state machine model; The structured prompts require the large language model to output the annotation results and corresponding confidence scores of the records to be annotated according to the transformation rules.

[0011] Optionally, the state machine model includes three state nodes: maintaining the experimental scheme state, exiting the observation state, and switching to the non-experimental scheme state; The state transition rule is defined as follows: If the record to be labeled triggers the subject's transition from maintaining the trial protocol to exiting the observation protocol or switching to a non-trial protocol state, then the record to be labeled will be marked as a concomitant event. If the record to be labeled does not trigger a state transition, then the record to be labeled will be labeled as background medication.

[0012] In a second aspect, this application provides an event labeling apparatus for treating causal inference, characterized in that the apparatus includes at least one module for performing the event labeling method for treating causal inference as described in any of the first aspects.

[0013] Thirdly, this application provides a computer device, characterized in that the computer device includes a processor for executing a computer program stored in a memory to implement the event labeling method for treating causal inference as described in any of the first aspects.

[0014] Thirdly, this application provides a computer program product containing instructions, characterized in that, when the instructions are executed by a computer device, the computer device performs the event labeling method for treating causal inference as described in any of the first aspects.

[0015] The technical solution provided in this application has the following advantages compared with the prior art: One of its beneficial effects and its working principle is as follows: The causal nature of an event depends on whether it triggers a transition in the subject's treatment status. A medication record that does not change the subject's treatment status is considered background medication. If the medication record triggers a transition from a maintenance treatment status to an out-of-protocol treatment status or withdrawal from observation, the record is considered a concomitant event. Existing keyword matching technologies can only identify entity names and cannot determine whether a record triggered a status transition, making it impossible to distinguish between background medication and concomitant events.

[0016] This application explicitly defines the treatment state of the subject and the transition conditions between states using a state machine model. The state machine transforms the implicit criterion of whether a state transition is triggered into a verifiable logical rule. After receiving the state machine transition rule, the large language model no longer relies on the semantic similarity of entity names for classification, but instead determines whether the record meets the triggering conditions for a state transition based on contextual data. If the context indicates that a clinical condition meeting the transition conditions existed when the record occurred, the model determines that the record triggered a state transition and marks it as an accompanying event. If the context does not indicate that the transition conditions were met, the model determines that the record did not trigger a state transition and marks it as background medication.

[0017] This mechanism, which shifts the basis of judgment from entity names to state transition detection, enables large language models to determine the causal attributes of records. By introducing structured constraints from state machines, the annotation process transforms from fuzzy judgments relying on semantic matching to precise reasoning based on logical rules, providing an accurate data foundation for subsequent causal inference analysis.

[0018] Its second beneficial effect and its working principle are as follows: The distribution of trigger signals along the timeline is uncertain. Especially for events corresponding to different medical entities, trigger signals may be concentrated in the days leading up to the event, or they may be scattered throughout the gradual process over several weeks. Existing technologies use a fixed time window strategy to load contextual data, which cannot adapt to this distributional variation. A fixed short window truncates the scattered chain of trigger signals, while a fixed long window loads a large number of irrelevant long-term records. Under the lexical budget constraint of a large language model, trigger signals are diluted by irrelevant data, reducing the efficiency of acquiring key information.

[0019] This application transforms fixed window selection into multi-scale dynamic allocation by establishing a probabilistic model of the temporal distribution of trigger signals. Based on event characteristics, the posterior probability of different time windows is calculated, and the lexical budget is allocated to each window according to the probability ratio. For events with trigger signals concentrated in the recent past, more budget is allocated to shorter windows to load more recent data. For events with dispersed trigger signals, budget is allocated to multiple windows simultaneously to cover a longer time range with appropriate precision. Data from each window is pruned according to the allocated budget and then concatenated into a multi-scale combined context, enabling the large language model to simultaneously obtain information from multiple time scales within a fixed lexical budget.

[0020] This probability-driven budget allocation mechanism optimizes the information density of the lexical budget. For events with high trigger time uncertainty, it can explore multiple possible time regions under budget constraints, avoiding information omissions or noise interference caused by incorrect single-window selection. By transforming the probability distribution of trigger signals into a data loading strategy, the annotation process achieves adaptive adjustment of information acquisition under resource-constrained conditions, providing high-hit-rate contextual data for large language models. Attached Figure Description

[0021] Figure 1 A schematic diagram illustrating an application scenario of the event labeling method for causal inference in treatment provided in the embodiments of this application; Figure 2 This is one of the flowcharts illustrating an event labeling method for causal inference in treatment provided in an embodiment of this application; Figure 3 This is a second flowchart illustrating the event labeling method for causal inference in treatment provided in an embodiment of this application. Figure 4 This is the third flowchart illustrating the event labeling method for causal inference in treatment provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0024] Before providing a detailed explanation of the embodiments of this application, let's first introduce the application scenarios involved in the embodiments of this application.

[0025] like Figure 1 As shown, Figure 1 This is a schematic diagram illustrating an application scenario of the event labeling method for causal inference in treatment provided in the embodiments of this application. For example... Figure 1 As shown in the embodiment of this application, taking a clinical trial of Alzheimer's disease as an example, when statisticians process clinical case report data, they use an event labeling device for treatment causal inference to automatically label the medical event records in the data. The device uses a state machine-guided large language model to determine whether the event triggers the transformation of the subject's treatment state and outputs the labeling results for statisticians to use for subsequent treatment causal inference analysis.

[0026] The method for event labeling for treating causal inference provided in this application embodiment can be loaded and executed by an event labeling device for treating causal inference, the device including at least one module, the at least one module being used to execute the event labeling method for treating causal inference as described in the following embodiments.

[0027] like Figures 2-4 As shown in the embodiments of this application, the event annotation method for causal inference in treatment includes the following annotation steps for clinical case report form data: S201: Perform keyword matching on clinical case report data to extract records containing preset medical entities.

[0028] Clinical case report data is stored in a structured table format, with each record containing fields such as subject number, record time, event type, event description, and relevant numerical values. First, a pre-defined medical entity dictionary is loaded. This dictionary contains entities commonly found in clinical trials for specific disease types (such as Alzheimer's disease), including drug names, surgical procedures, and termination events. The dictionary can be built based on standard medical terminology databases (such as UMLS and SNOMED CT) or manually customized according to the trial protocol.

[0029] An exact matching algorithm is used to scan the data. Specifically, the event description field in the clinical case report table is traversed, and the text content of this field is matched against each entity in the medical entity dictionary. When the event description field of a record contains a medical entity defined in the dictionary, that record is extracted and designated as the event record to be labeled.

[0030] For scenarios requiring variant matching (such as the correspondence between risperidone and risperdone), regular expression matching or fuzzy matching algorithms based on edit distance (e.g., the number of characters between them) can be used. This application does not limit the specific matching algorithm; those skilled in the art can choose an appropriate text matching method according to actual needs.

[0031] The output of keyword matching is a set of event records to be labeled. Each record retains all original field information, including subject number, recording time, event description, etc., for use in subsequent steps. In this embodiment, the role of the keyword matching step is to quickly locate records that may require causal attribute judgment, narrowing the data range for subsequent processing and improving overall processing efficiency.

[0032] S202: For the extracted records, treat them as records to be labeled and obtain the context data corresponding to those records; Specifically, obtaining the context data corresponding to the record includes: S301: Extract the event features of the record to be labeled; In step S301, event features are extracted from the records to be labeled.

[0033] The event features extracted in this application include three categories: drug or treatment category features, operation type features, and subject baseline features.

[0034] Drug or treatment category characteristics are extracted from the event description field of the record.

[0035] Based on pre-defined classification rules, identified medical entities are mapped to corresponding categories. For example, in Alzheimer's disease clinical trials, drug categories may include cognitive enhancers (such as memantine and donepezil), antipsychotics (such as risperidone and olanzapine), and sedative-hypnotic drugs. Classification rules can be established based on the drug's ATC code (Anatomy, Therapeutics and Chemistry Classification System) or the WHO drug classification standards. For surgical or other treatment measures, classification is based on the ICD code or the classification system defined in the trial protocol.

[0036] Operation type characteristics reflect the nature of the medical operation corresponding to the record. Common operation types include: initial prescription, dosage adjustment, actual use of medication as needed, and treatment termination. Operation types are identified by analyzing the event type field of the record and keywords (such as initial, adjusted to, discontinued) in the event description. Operation type characteristics are important for determining the temporal distribution of trigger signals. For example, trigger signals for initial prescriptions are usually concentrated around the time of prescription, while trigger signals for dosage adjustments may be scattered over a longer observation period.

[0037] Baseline characteristics of the subjects were extracted from their baseline data. Baseline data included clinical indicators measured at randomization or at the start of the trial, such as ADAS-Cog cognitive scores, BPRS behavioral and mental health scores, and disease severity grading. These characteristics reflect the subjects' initial health status and are correlated with the temporal distribution of trigger signals. For example, subjects with lower baseline cognitive scores may be more prone to rapid disease exacerbation, while those with higher baseline scores may experience relatively slower disease progression. Baseline records were retrieved from the clinical database based on the subject's ID, and relevant indicator values ​​were extracted.

[0038] The three types of features mentioned above are organized into an event feature vector. This event feature vector serves as the input for subsequent probability calculations, used to query the prior probability distribution and calculate the posterior probability.

[0039] S302: Based on the event characteristics, query the preset prior probability of the trigger signal time distribution and calculate the posterior probability of multiple candidate time windows; The prior probabilities of the trigger signal time distribution are stored in the form of a probability distribution table. This distribution table is indexed by event type and records the probability of trigger signals of different types of events occurring in each time window.

[0040] For example, for the event type "antipsychotic medication used as needed," the prior distribution might be: 0.20 for a 1-week window, 0.45 for a 2-week window, 0.30 for a 4-week window, and 0.05 for an 8-week window. These prior probabilities can be obtained based on statistical analysis of historical labeled data, or initial values ​​can be set by domain experts based on clinical experience.

[0041] First, based on the drug category features and operation type features extracted in step S301, the event type corresponding to the current record is determined. The event type is defined in a hierarchical structure; for example, "antipsychotic drugs - on-demand use" is a specific event type, and "antipsychotic drugs - dosage adjustment" is another event type. The prior probability distribution table is then consulted to find the probability distribution matching the event type, obtaining the prior probability value for each candidate time window.

[0042] After obtaining the prior probability, the posterior probability is calculated using Bayes' theorem. The aim is to incorporate the individual characteristics of the subjects, making the probability estimate more accurate.

[0043] First, based on the baseline characteristics of the subjects, the likelihood of each time window under those characteristics is calculated. The likelihood reflects the probability that the trigger signal will appear in a certain time window under specific subject characteristics.

[0044] Likelihood calculation is based on conditional probability statistics of historical data. By maintaining a conditional probability table, the frequency of trigger signals occurring in each time window under different subject characteristics (baseline score range in this embodiment) is recorded.

[0045] For example, for subjects with a baseline BPRS score of 24-28 (moderate psychiatric symptoms), statistics may show that triggers for worsening psychiatric symptoms occur more frequently than average within a 2-week window. Based on the current baseline characteristics of the subjects, the corresponding likelihood values ​​are retrieved from the conditional probability table.

[0046] The posterior probability is calculated using Bayes' theorem: in, For time window The posterior probability, Let be the prior probability of this window. In the window Event characteristics under conditions likelihood This represents the total number of candidate time windows.

[0047] In this embodiment, four candidate time windows are defined: a 1-week window, a 2-week window, a 4-week window, and an 8-week window. The summation term in the denominator is used to normalize all candidate time windows, ensuring that the sum of the posterior probabilities of all windows is 1.

[0048] S303: Based on the posterior probability and the preset upper limit of the lexical budget, allocate the lexical budget to the plurality of candidate time windows.

[0049] Specifically, based on the posterior probability calculated in step S302 and the preset upper limit of the lexical budget, the lexical budget is allocated to each candidate time window.

[0050] The lexical budget limit refers to the maximum number of lexical units allowed as input in a single call to a large language model. This limit is generally determined manually based on a combination of factors, including the interface limitations of the large language model, response time requirements, and cost control strategies.

[0051] For example, in this embodiment of the application, the token budget cap is set to 10,000 tokens. This cap includes the fixed overhead for constructing the cue word framework and state machine rule description, as well as the allocable budget for loading context data.

[0052] Generally, about 950 lexical units are reserved for the cue word frame, and the remaining 9050 lexical units are allocated as an allocable budget for loading data from various time windows.

[0053] The lexical budget allocation is calculated according to the following rule: the lexical budget allocated to each time window is equal to the upper limit of the lexical budget multiplied by the proportion of the posterior probability of that window to the sum of the posterior probabilities of all windows. This rule transforms the posterior probability into a budget allocation weight, ensuring that windows with higher probabilities receive more budget and windows with lower probabilities receive less budget.

[0054] The allocation formula is: in, To allocate to time windows Lexical budget, The upper limit of the lexical budget (the allocable budget after deducting the cost of the cue word frame) is rounded down to ensure that the allocated budget is an integer number of lexical units.

[0055] In a preferred embodiment, a minimum allocation threshold is set. When the lexical budget allocated to a certain time window is lower than the minimum allocation threshold, the data for that time window is not loaded, and the lexical budget for that window is reallocated to other time windows.

[0056] Specifically, after the initial allocation is completed, a minimum threshold filtering is performed. A minimum allocation threshold is set. When the allocated budget for a certain time window is lower than the threshold, it is determined that the budget for that window is insufficient to load meaningful data. Therefore, the window is not loaded, and the budget for that window is reallocated to other windows (such as the second smallest window).

[0057] The minimum threshold is set based on the following considerations: large language models require sufficient contextual information to make accurate judgments. If the budget allocated to a window is too small (e.g., only 100-200 lexical units), only 1-2 data records can be loaded, which cannot provide complete information on clinical changes and may even introduce noise. In this embodiment, the minimum threshold is set to 1000 lexical units. This value can load approximately 4-5 or more records containing score changes or symptom descriptions, which is sufficient to reflect the basic clinical situation within that time window.

[0058] S304: According to the lexical budget allocated to each time window, cut data records from the corresponding time range and splice the cut data records into a multi-scale combined context, which serves as the context data.

[0059] Specifically, according to the lexical budget allocated to each time window in step S303, data records are clipped from the corresponding time range, and the clipped data records are spliced ​​together to form a multi-scale combined context.

[0060] For each time window with an allocated budget, first determine the time range corresponding to that window.

[0061] The time range is based on the occurrence time of the record to be annotated and traces backward.

[0062] For example, if the record to be labeled occurs on day 58, the time range of the 1-week window is from day 51 to day 58, the time range of the 2-week window is from day 44 to day 58, and the time range of the 4-week window is from day 30 to day 58.

[0063] Extract all data records of the subject within the corresponding time range from the clinical case report data to serve as the raw dataset for this window.

[0064] The raw data records contain various types of clinical information, such as scoring measurement records (ADAS-Cog cognitive score, BPRS mental and behavioral score, etc.), symptom observation records (agitation, auditory hallucinations, aggressive behavior, etc.), medication records, adverse event records, routine visit records, etc.

[0065] The original dataset is used to count the number of lexical units. This calculation is based on the word segmenter used in the large language model. The text content of each record is input into the word segmenter to obtain the corresponding lexical sequence. The number of lexical units is counted, and these sequences are summed to obtain the total number of lexical units in the original dataset.

[0066] If the total number of tokens in the original dataset does not exceed the token budget allocated to this window, load all original data records. If the total number of tokens in the original dataset exceeds the allocated budget, perform data pruning and filter records by relevance.

[0067] Data pruning employs a relevance-based ranking method. Each record is scored for relevance based on its content type. Specifically, a priority order is preset for record types; for example, records with scoring measurement keywords have the highest priority, followed by symptom description records (keywords related to agitation, auditory hallucinations, etc.), then medication records and adverse event records, with routine visit records having the lowest priority. The original dataset is sorted according to record type priority, with records containing high-priority keywords loaded first. Within the same priority level, records are arranged in reverse chronological order, loading records more recently than the event to be labeled first. The sorted records are loaded sequentially, accumulating the number of terms loaded. Loading stops when the accumulated number of terms reaches the term budget allocated to that window.

[0068] Then, the data from different windows are concatenated into a multi-scale combined context. During the concatenation process, the data from different windows are labeled according to the time scale, and the time range of each window is marked in the multi-scale combined context.

[0069] The specific concatenation format is as follows: A data block is created for each time window. This data block contains a window identifier, a time range description, and the data records within that window. The window identifier uses hierarchical naming, such as: "Time Scale 1-Week Range" or "Time Scale 2-Week Range." The time range description clearly indicates the start and end times covered by the window, such as "Day 44 to Day 58." The data records within the window are arranged chronologically, and each record retains its timestamp, event type, and event description.

[0070] The overall structure of the multi-scale combined context is as follows: First, there is a data block with a 1-week window (if the window is allocated a budget), containing information from the most recent week; second, there is a data block with a 2-week window, containing information from the most recent 2 weeks; then, there is a data block with a 4-week window, containing information from the most recent 4 weeks; and finally, there is a data block with an 8-week window, containing information from the most recent 8 weeks.

[0071] Each data block is distinguished by a clear delimiter, which makes it easier for large language models to identify information sources at different time scales.

[0072] Through this multi-scale annotation and concatenation method, large language models can clearly understand the time range corresponding to each part of the data when receiving contextual data. This organization method enables the model to obtain multiple time scale perspectives simultaneously under a fixed lexical budget constraint, capturing both recent detailed changes and understanding long-term trend backgrounds.

[0073] Finally, the multi-scale combined context, as the context data, is passed to step S204 for the large language model to perform event labeling in conjunction with state machine transformation rules. Through probability-driven budget allocation and multi-scale labeled data concatenation, this application achieves adaptive information acquisition under lexical budget constraints, providing context input with optimized information density for the large language model.

[0074] In this embodiment of the application, it further includes: recording the information value of each time window in the actual annotation, and adjusting the prior probability using a Bayesian update rule, including: S401: Evaluate the information value of each time window based on whether it contains key triggering signals and the confidence level of the annotation results. Information value reflects the degree to which a given time window contributes to accurate annotation.

[0075] The assessment of information value is based on two indicators: whether the window contains key triggering signals and the confidence level of the annotation results.

[0076] Key trigger signals refer to clinical information that directly supports the annotation judgment. For records labeled as comorbid events, key trigger signals include state transition triggers such as significant changes in scores, new symptoms, and the occurrence of adverse events. For records labeled as background medication, key trigger signals refer to triggers where no state transition trigger exists in the context. After completing steps S204 and S205, the data of each time window in the multi-scale combined annotation context can be examined through a large model or manual backtracking to determine which time window contains the key information supporting the final annotation result.

[0077] Specifically, for records labeled as accompanying events and verified, their multi-scale combined context is analyzed to identify time windows containing triggering signals. For example, if the context of a record shows that a 2-week window includes records of new auditory hallucinations and increased agitation frequency, while a 1-week window only includes the final medication records, and a 4-week window includes earlier, minor score changes, then the 2-week window is determined to contain key triggering signals and has high information value. A 1-week window has insufficient information, and a 4-week window has low information density and relatively low information value.

[0078] The confidence level of the annotation results reflects the certainty of the large language model's judgment. It is obtained through S204 below.

[0079] Combining these two indicators, an information value score is calculated for each time window. In this embodiment, the information value score is calculated as follows: If a window contains a key trigger signal and the labeled confidence level is higher than 0.8, the information value score for that window is high (assigned a value of 1.0). If a window contains a key trigger signal but the labeled confidence level is between 0.6 and 0.8, the information value score for that window is medium (assigned a value of 0.6). If a window does not contain a key trigger signal, the information value score for that window is low (assigned a value of 0.2).

[0080] The information value score of each time window during each annotation process is recorded in the annotation log and stored together with the event type and subject characteristics of the record for statistical analysis in step S402.

[0081] S402: When the cumulative number of labeled samples of the same event type reaches the preset sample quantity threshold, statistically analyze the empirical distribution of each time window in the actual labeling. The setting of a preset sample size threshold is generally based on considerations of statistical significance. In this embodiment, the preset sample size threshold is set to 20 samples. That is, when 20 cases of a certain event (such as "on-demand use of antipsychotic drugs") have been accumulated, the sample size is considered sufficient for statistical analysis of empirical distribution.

[0082] The statistical analysis of the experience distribution is based on the information value scores recorded in step S401. All labeled samples of this event type are extracted, and the frequency of being rated as having high information value in each time window is counted.

[0083] In this embodiment of the application, the proportion of samples with high information value scores in each time window is calculated as the empirical probability of that window.

[0084] The statistical results need to be normalized to ensure that the sum of the empirical probabilities for each window is 1. The normalization formula is: in, For time window The number of samples rated as having high information value, where n is the total number of candidate time windows.

[0085] S403: The prior probability is adjusted using the Bayesian update formula, which weights the prior probability with the empirical distribution, wherein the smoothing coefficient is used to balance the weights of historical priors and new observation data.

[0086] The empirical distribution statistically obtained in step S402 is weighted and combined with the current prior probability to obtain the updated prior probability.

[0087] The purpose of Bayesian updating is to integrate empirical knowledge learned from actual labeled data into the prior probability, so that the prior probability gradually approximates the actual temporal distribution characteristics of the trigger signals in the dataset. The update formula uses a weighted average form: in, This represents the updated prior probability. The prior probability before the update. The empirical distribution is calculated in step S402. This is the smoothing coefficient.

[0088] The smoothing coefficient is a manually set value. In this embodiment, the smoothing coefficient is set to 0.7, meaning that the weight of historical prior data is 70% and the weight of new observation data is 30%. This setting maintains prior stability while allowing learning and adjustment from new data.

[0089] The updated prior probabilities are stored in the probability distribution table, replacing the original prior probability values. The updated prior probabilities will be queried and used in step S302 of the subsequent annotation process, affecting the calculation of the posterior probability of the new record's time window and the allocation of the lexical budget.

[0090] S203: Load the state machine model, which defines the treatment state of the subject and the state transition rules.

[0091] Specifically, in step S203, a pre-built state machine model is loaded. This state machine model is used to formally define the treatment states of subjects during clinical trials and the conditions for transitions between states, providing a structured basis for judgment in the large language model.

[0092] Specifically, the state machine model described in this application embodiment includes three state nodes: maintaining the experimental scheme state, exiting the observation state, and switching to the non-experimental scheme state; Maintaining the trial protocol status means that the subject is receiving treatment according to the trial protocol and is within the effective observation period. Withdrawal from observation status means that the subject is no longer participating in the trial observation for various reasons (such as death, loss to follow-up, or voluntary withdrawal). Switching to a non-trial protocol status means that although the subject has not withdrawn from the trial, they have received treatment interventions outside the trial protocol, such as initiating salvage therapy or changing the medication regimen due to adverse reactions.

[0093] The state machine model is implemented using a directed graph data structure. Each state node contains a state identifier and a state description. State nodes are connected by transition edges, and each transition edge contains a logical expression for the trigger condition.

[0094] Triggering conditions are defined based on the characteristics of clinical events. For example, the transition from a state of maintaining the trial protocol to a state of exiting observation may be triggered by conditions such as the death of a subject or the withdrawal of informed consent by a subject.

[0095] The transition from maintaining the trial protocol to switching to a non-trial protocol may be triggered by conditions such as initiating off-protocol salvage therapy or switching to a non-trial drug due to adverse reactions.

[0096] The state transition rule defines the correspondence between event labeling and state transition. If an event described in a record triggers a subject's transition from the state of maintaining the trial protocol to the state of exiting observation or switching to a non-trial protocol state, then the record should be labeled as a concomitant event.

[0097] If an event described in a record does not trigger a state transition, meaning the subject remains in the original state, then the record should be marked as background medication.

[0098] Concomitant events alter a subject's treatment status and can affect the estimation of treatment effects in subsequent causal inference analyses. Background medication events are only considered as covariates in statistical adjustment and do not affect the subject's treatment status.

[0099] The state machine model is stored in the form of a data structure (graph structure and state transition matrix). This model is subsequently used to guide the judgment of large language models.

[0100] S204: Input the transformation rules of the state machine model and the context data into the large language model to obtain the annotation result of the record to be annotated, wherein the annotation result indicates that the record is a concomitant event or background medication; Specifically, it includes the following steps: Structured prompts are constructed by organizing transformation rules, contextual data, and information about the records to be labeled into an input format that can be understood by a large language model. Structured prompts consist of three core components: basic information about the records to be labeled, multi-scale combined context, and a description of the transformation rules from a state machine model.

[0101] The basic information for the record to be labeled includes the date of occurrence, event description, and subject number. For example, for the record to be labeled, day 58: risperidone 1mg was used, the date (day 58), event description (risperidone 1mg used), and the subject's medication history information (e.g., first prescription of risperidone 1mg on day 15, used as needed) would be extracted. This information clarifies the target of the labeling task.

[0102] The multi-scale combined context, namely the data generated in step S304, contains data blocks across multiple time windows. This context is fully embedded in the prompt words, providing a basis for judgment for the large language model. Since the context data has been labeled with time scales and window ranges, the model can clearly identify the time source of each part of the data.

[0103] The state machine model transition rule description transforms the state machine model loaded in step S203 into natural language form. Specifically, it describes the three state nodes of the state machine and their meanings in words, and expresses the triggering conditions for state transitions as judgment rules. For example, the transition rule description can be expressed as follows: If, before the record occurs, the subject experiences a significant deterioration in psychiatric symptoms (e.g., a BPRS score increase of more than 5 points from baseline, or the addition of hallucinations or delusions, or a significant increase in agitation frequency), then the record is determined to have triggered the subject's transition from the maintenance protocol state to the non-protocol state, and should be marked as a concomitant event. If, before the record occurs, the subject does not exhibit any clinical changes that meet the above conditions, then the record is determined not to have triggered a state transition, and should be marked as background medication.

[0104] The structured prompts are organized in the following order: task description, object to be labeled, contextual data, judgment rules, and output requirements. The prompt begins by stating the goal of the labeling task: determining whether the record is a concomitant event or background medication. It then provides basic information about the record to be labeled, clarifying the object for the model. Next, it provides multi-scale combined context, giving the model a basis for judgment. Then, it provides a textual description of the state machine transition rules, clarifying the judgment criteria for the model. Finally, it specifies the output requirements, instructing the large language model to output the labeling result of the record and its corresponding confidence score based on the transition rules.

[0105] Confidence score is a self-assessment of the reliability of the annotation results by the large language model, ranging from 0 to 1. Confidence score reflects the degree of certainty with which the model makes judgments based on a given context and rules. If the context contains clear trigger signals for state transitions (such as the addition of auditory hallucinations and a significant increase in agitation frequency), the model outputs a higher confidence score (e.g., above 0.9). If the signals in the context are ambiguous or contain contradictory information, the model outputs a lower confidence score (e.g., 0.6-0.7).

[0106] The confidence level information is used for the information value assessment in step S401.

[0107] The constructed structured prompts are sent to the large language model via an API interface. In this embodiment, commercial large language models such as Wenxin Yiyan and Tongyi Qianwen can be used. This application does not limit the specific type and version of the large language model; those skilled in the art can choose a suitable model according to actual needs.

[0108] After receiving prompt words, the large language model, based on its pre-trained language understanding and reasoning abilities, analyzes clinical information in the contextual data to determine whether there are triggering conditions that conform to the state machine transition rules. The model's reasoning process utilizes its medical knowledge and logical reasoning abilities learned on large-scale text data. By embedding state machine transition rules as explicit judgment criteria into the prompt words, this application guides the model to reason according to structured logic, rather than relying solely on semantic similarity for classification.

[0109] After the large language model completes inference, it returns the annotation results in a structured format. The returned results include the annotation type (accompanying event or background medication) and the confidence score.

[0110] By analyzing the results returned by the model, the label type and confidence level are extracted and used as the labeling result for that record.

[0111] This application implements rule-guided semantic annotation in this way. Compared with directly letting the model classify events, the method of this application provides the model with clear judgment criteria (appropriate contextual data) and judgment standards (transformation rules), making the model's reasoning process more controllable and improving the accuracy and consistency of the annotation results.

[0112] S205: Output the annotation results to complete the annotation of the record to be annotated.

[0113] Output the annotation results after step S204.

[0114] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0115] In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments are consistent and can be referenced mutually. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. In the textual description of the embodiments of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. In this application, "first," "second," and various numerical designations are only for ease of description and are not used to limit the scope of the embodiments of this application. For example, they are used to distinguish different messages, rather than to describe a specific order or sequence.

[0116] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0117] Finally, it should be noted that the above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An event labeling method for treating causal inference, characterized in that, The following are the steps for labeling clinical case report form data: Perform keyword matching on clinical case report data to extract records containing preset medical entities; For each extracted record, it is used as a record to be labeled, and the context data corresponding to that record is obtained. Load a state machine model, which defines the treatment state of the subject and the rules for state transitions; The transformation rules of the state machine model and the context data are input into the large language model to obtain the annotation results of the record to be annotated. The annotation results indicate that the record is a concomitant event or background medication. Output the annotation results to complete the annotation of the record to be annotated; The step of obtaining the context data corresponding to the record includes: Extract the event features of the records to be labeled; Based on the event characteristics, query the preset prior probability of the trigger signal time distribution and calculate the posterior probability of multiple candidate time windows; Based on the posterior probability and the preset upper limit of the lexical budget, the lexical budget is allocated to the multiple candidate time windows; According to the lexical budget allocated to each time window, data records are clipped from the corresponding time range, and the clipped data records are spliced ​​together to form a multi-scale combined context, which serves as the context data. The event labeling method for treating causal inference further includes: recording the information value of each time window in the actual labeling, and adjusting the prior probability using a Bayesian update rule, including: The information value of each time window is evaluated based on whether it contains key triggering signals and the confidence level of the annotation results. When the cumulative number of labeled samples of the same event type reaches the preset sample quantity threshold, the empirical distribution of each time window in actual labeling is statistically analyzed. The prior probability is adjusted using a Bayesian update formula, which weights the prior probability with the empirical distribution, wherein a smoothing coefficient is used to balance the weights of historical priors and new observation data.

2. The event labeling method for treating causal inference according to claim 1, characterized in that: The posterior probability is calculated using Bayes' theorem based on the prior probability and the event characteristics. The lexical budget allocation is calculated according to the following rules: the lexical budget allocated to each time window is equal to the upper limit of the lexical budget multiplied by the proportion of the posterior probability of that window to the sum of the posterior probabilities of all windows.

3. The event labeling method for treating causal inference according to claim 1, characterized in that, For each time window, when the number of tokens in the original data records within the corresponding time range exceeds the token budget allocated to that window, the original data records are sorted by relevance, and records with higher relevance are loaded first, until the cumulative number of tokens reaches the token budget. When splicing the cropped data records, the data of different time windows are labeled according to the time scale, and the time range of each time window is marked in the multi-scale combination context.

4. The event labeling method for treating causal inference according to claim 1, characterized in that, The step of inputting the transformation rules of the state machine model and the context data into the large language model includes: Construct structured prompt words, which include the time and description information of the record to be labeled, the multi-scale combined context, and the transformation rule description of the state machine model; The structured prompts require the large language model to output the annotation results and corresponding confidence scores of the records to be annotated according to the transformation rules.

5. The event labeling method for treating causal inference according to claim 1, characterized in that, The state machine model contains three state nodes: maintaining the experimental scheme state, exiting the observation state, and switching to the non-experimental scheme state. The state transition rule is defined as follows: If the record to be labeled triggers the subject's transition from maintaining the trial protocol to exiting the observation protocol or switching to a non-trial protocol state, then the record to be labeled will be marked as a concomitant event. If the record to be labeled does not trigger a state transition, then the record to be labeled will be labeled as background medication.

6. An event labeling device for treating causal inference, characterized in that, The apparatus includes at least one module for performing the event labeling method for treating causal inference as described in any one of claims 1-5.

7. A computer device, characterized in that, The computer device includes a processor for executing a computer program stored in a memory to implement the event labeling method for causal inference of treatment as described in any one of claims 1-5.

8. A computer program product containing instructions, characterized in that, When the instructions are executed by a computer device, the computer device performs the event labeling method for treating causal inference as described in any one of claims 1-5.

Images