A gene sequencing process parameter optimization method based on reinforcement learning

Abstract

This invention discloses a gene sequencing workflow parameter optimization method based on reinforcement learning, belonging to the field of gene sequencing data processing technology, and includes the following steps: Step 1: Receive gene sequencing task information and generate a sequencing task constraint profile; Step 2: Read sequencing workflow observation records and generate sequencing workflow observation sequences; Step 3: Regularize the observation sequences to generate sequencing workflow state sequences; Step 4: Input an improved GRU-D model to obtain a sequencing parameter decision state profile; Step 5: Map and generate candidate parameter action chains; Step 6: Use an improved P-DQN hybrid action parameter optimization algorithm to generate a candidate parameter strategy set; Step 7: Filter and generate target parameter schemes and backup parameter schemes; Step 8: Issue the target scheme and update the experience replay library and parameter action feasible region. This invention uses an improved P-DQN algorithm and an improved GRU-D model to achieve closed-loop optimization of gene sequencing workflow parameters.

Description

Technical Field

[0001] This invention relates to the field of gene sequencing data processing technology, and in particular to a method for optimizing gene sequencing process parameters based on reinforcement learning. Background Technology

[0002] As the application of gene sequencing technology continues to expand in scenarios such as genetic disease screening, tumor detection, pathogen identification, and population genetic research, sequencing tasks place higher demands on the quality, efficiency, and stability of process parameter configuration.

[0003] Current gene sequencing workflows typically involve pre-configuring sequencing parameters, read processing parameters, and analysis process parameters based on task type, sample conditions, and platform experience. These parameters are then manually reviewed or adjusted for the next batch based on quality assessment results after sequencing. However, the following problems commonly exist in practical applications:

[0004] Different sequencing tasks have different quality constraints, execution stages and feedback cycles, and fixed parameter templates are difficult to adapt to the stage state changes in the sequencing process; the sources of sequencing process observation records are scattered, and missing states and observation intervals will affect the judgment of quality changes, and existing methods are difficult to transform process observation sequences into state profiles that can be used for parameter decision-making.

[0005] Sequencing quality feedback typically lags behind parameter action execution, and historical execution feedback is difficult to be fed back into the parameter strategy generation process in a timely manner. At the same time, there is a lack of a unified hybrid action optimization method between discrete process actions and continuous parameter values, making it difficult for the target parameter scheme and backup parameter scheme to form an executable, switchable, and continuously updated closed loop.

[0006] Therefore, how to provide a method for optimizing gene sequencing process parameters based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a gene sequencing process parameter optimization method based on reinforcement learning. This invention generates a sequencing task constraint profile by receiving gene sequencing task information, reads sequencing process observation records and organizes them to generate a sequencing process state sequence, so that the gene sequencing task requirements and process observation states enter a unified parameter decision chain.

[0008] This invention inputs the sequencing process status sequence into an improved GRU-D sequencing state drift prediction model to generate a sequencing parameter decision state profile. It also embeds a stage lag drift calibration mechanism in the quality drift calibration layer to correct the impact of missing observations, observation intervals, and stage quality drift on the parameter decision state.

[0009] This invention employs an improved P-DQN hybrid action parameter optimization algorithm to process candidate parameter action chains, and calibrates the value of candidate actions through a sequencing quality delay reward reinjection mechanism to generate a candidate parameter strategy set;

[0010] Then, through constraint screening and strategy sorting, target sequencing process parameter schemes and backup sequencing process parameter schemes are generated. The historical sequencing experience replay library and parameter action feasible domain are updated through execution feedback, forming a learnable, executable and continuously updatable gene sequencing process parameter optimization closed loop.

[0011] A method for optimizing gene sequencing workflow parameters based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0012] Step 1: Receive gene sequencing task information, analyze the task constraints of the gene sequencing task information, and generate a sequencing task constraint profile;

[0013] Step 2: Based on the sequencing task constraint profile, read the sequencing process observation records and generate sequencing process observation sequences;

[0014] Step 3: Perform state sequence normalization on the sequencing process observation sequences to generate sequencing process state sequences;

[0015] Step 4: Input the sequencing process state sequence into the improved GRU-D sequencing state drift prediction model to obtain a state profile of sequencing parameter decisions. The improved GRU-D sequencing state drift prediction model includes a process observation attenuation embedding layer, a stage recursive characterization layer, a quality drift calibration layer, and a strategy state output layer. The quality drift calibration layer embeds a stage hysteresis drift calibration mechanism.

[0016] Step 5: Map the feasible domain of the sequencing parameter decision state profile to generate candidate parameter action chains;

[0017] Step 6: Based on the historical sequencing experience replay library, the improved P-DQN hybrid action parameter optimization algorithm is used to process the candidate parameter action chain, and the candidate action value is calibrated through the sequencing quality delay reward reinjection mechanism to generate a candidate parameter strategy set;

[0018] Step 7: Perform execution constraint screening and strategy ranking on the candidate parameter strategy set to generate the target sequencing process parameter scheme and the alternative sequencing process parameter scheme;

[0019] Step 8: Send the target sequencing process parameter scheme to the sequencing process execution end, trigger the backup sequencing process parameter scheme based on the parameter execution feedback data, and update the historical sequencing experience replay library and parameter action feasible domain.

[0020] Optionally, step one specifically includes:

[0021] Read the gene sequencing task information, perform integrity verification on the gene sequencing task information, remove descriptive items that are not related to the optimization of sequencing process parameters, and obtain valid task records;

[0022] A task index is generated based on valid task records. The task index is used to associate the records of the same sequencing task in the sequencing workflow observation sequences, candidate parameter strategy sets, and parameter execution feedback data.

[0023] The sequencing object information in the valid task record is converted into an object constraint vector, the target requirements are converted into a quality constraint vector, and the execution requirements are converted into a process constraint vector.

[0024] Write the task index, object constraint vector, quality constraint vector, and process constraint vector into a unified profile structure, and configure observation read markers and parameter constraint call markers for the unified profile structure to generate a sequencing task constraint profile.

[0025] Optionally, step two specifically includes:

[0026] Read the task index and observation read marker from the sequencing task constraint profile, and retrieve the corresponding sequencing process observation records according to the task index;

[0027] The effective observation records are obtained by filtering out sequencing process observation records that are irrelevant to process parameter optimization according to the observation reading markers;

[0028] The valid observation records are verified for consistency with their source. The valid observation records that pass the consistency verification are then merged according to the process stages to obtain the stage observation records.

[0029] Configure stage sequence identifiers and observation time identifiers for stage observation records, sort and deduplicate stage observation records according to stage sequence identifiers and observation time identifiers, and generate observation sequences for sequencing process.

[0030] Optionally, step three specifically includes:

[0031] Read the stage observation records, stage sequence identifiers, and observation time identifiers from the sequencing process observation sequence, and establish stage observation windows according to the stage sequence identifiers;

[0032] The stage observation records within the stage observation window are encoded, converted, and uniformly scaled to generate a stage observation matrix.

[0033] A missing status marker is generated based on the missing location of the stage observation record within the stage observation window, and a status interval vector is generated based on the observation time identifier of the adjacent stage observation window.

[0034] The stage observation matrix is ​​mapped to a stage state vector, and the stage state vector, missing state marker, and state interval vector are combined according to the stage order identifier to generate the sequencing process state sequence.

[0035] Optionally, step four specifically includes:

[0036] The process observation attenuation embedding layer reads the stage state vector, missing state marker and state interval vector in the sequencing process state sequence, and uses a learnable attenuation mapping to perform a nonlinear transformation on the state interval vector to generate the observation attenuation coefficient.

[0037] The effective observation positions and missing observation positions in the stage state vector are determined according to the missing state marker. The stage state vector corresponding to the missing observation position is attenuated and compensated using the observation attenuation coefficient. The attenuated and compensated stage state vector is then mapped to the process observation embedding vector.

[0038] The stage recursive representation layer reads the process observation embedding vector and uses the hidden state of the previous process stage as a recursive reference. It performs non-linear recursive fusion of the process observation embedding vector and the hidden state of the previous process stage through a loop state update function to generate the hidden state of the current process stage. It iteratively updates the hidden state of the current process stage according to the process stage order to generate the stage recursive state vector.

[0039] The quality drift calibration layer reads the recursive state vector, state interval vector, and missing state marker during the reading phase, and executes the hysteresis drift calibration mechanism during the execution phase.

[0040] The stage lag drift calibration mechanism generates stage lag coefficients based on the state interval vector, generates stage drift residuals based on the hidden state differences between adjacent process stages, and corrects the stage drift residuals with effective observation weights based on the missing state labels to obtain corrected drift residuals.

[0041] The stage lag drift calibration mechanism performs a weighted mapping between the stage lag coefficient and the corrected drift residual to generate a stage drift calibration term; the mass drift calibration layer performs residual fusion between the stage drift calibration term and the stage recursive state vector, and performs nonlinear calibration on the residual fusion result to generate a mass drift calibration vector.

[0042] The policy state output layer performs nonlinear projection, state normalization, and policy state compression on the quality drift calibration vector to generate a sequencing parameter decision state profile for constructing candidate parameter action chains.

[0043] Optionally, step five specifically includes:

[0044] Read the feasible domain of parameter actions from the historical sequencing experience replay library, read the sequencing parameter decision state profile, and perform state component analysis on the sequencing parameter decision state profile to obtain the stage state component, drift calibration component and strategy constraint component.

[0045] Match the stage state components with the stage action indexes in the parameter action feasible domain to determine the set of actions to be executed in the stage;

[0046] The drift calibration component is associated with the set of stage execution actions to generate action adjustment direction markers;

[0047] Match the policy constraint components with the action value boundaries in the feasible domain of parameter actions to generate a continuous parameter boundary vector;

[0048] Discrete process actions are generated according to the set of actions to be executed in stages, and continuous parameter vectors are generated according to the action adjustment direction markers and continuous parameter boundary vectors.

[0049] Discrete process actions, continuous parameter vectors, and execution stage markers are linked together to generate candidate parameter action chains.

[0050] Optionally, step six specifically includes:

[0051] Read the discrete process actions, continuous parameter vectors, and execution stage markers in the candidate parameter action chain, and retrieve sequencing experience samples of the same stage from the historical sequencing experience replay library according to the execution stage markers;

[0052] The historical state profiles, historical parameter actions, and historical execution feedback in the sequencing experience samples of the same stage are matched with the candidate parameter action chains to generate reinforcement learning state action samples.

[0053] A hybrid action evaluation object is established based on reinforcement learning state-action samples. The hybrid action evaluation object includes discrete process actions, continuous parameter vectors, execution stage labels, and historical execution feedback references.

[0054] An improved P-DQN hybrid motion parameter optimization algorithm is used to process the hybrid motion evaluation object. The continuous parameter vector is corrected by the continuous parameter generation function to generate candidate continuous parameter values.

[0055] The discrete process actions, candidate continuous parameter values, and execution stage markers are jointly evaluated to generate discrete action value and parameter action combination value.

[0056] The delayed reward records returned by the sequencing quality delayed reward reinjection mechanism are invoked to calibrate the value of discrete actions and the value of parameter action combinations, and generate calibrated candidate action values.

[0057] The mixed action evaluation objects are sorted according to the value of the candidate actions after calibration, and parameter action combinations that do not meet the execution stage markers are filtered out to generate a candidate parameter strategy set.

[0058] Optionally, the sequencing quality delay reward refeeding mechanism specifically includes:

[0059] Read historical parameter execution feedback records and historical candidate parameter strategy records from the historical sequencing experience replay library, and extract historical task completion feedback, historical sequencing quality feedback and historical execution anomaly feedback;

[0060] Based on the execution stage markers in the historical candidate parameter strategy records, the historical parameter actions and historical parameter execution feedback records are matched for stage attribution to generate historical stage feedback association records.

[0061] Reward components are converted from historical task completion feedback, historical sequencing quality feedback, and historical execution anomaly feedback to generate completion reward components, quality reward components, and anomaly penalty components.

[0062] Based on the historical stage feedback and correlation records, the completion reward component, quality reward component, and anomaly penalty component are allocated to the corresponding historical parameter actions to generate stage-delayed rewards.

[0063] The stage-delayed reward is bound to the discrete action value and parameter action combination value in the historical candidate parameter strategy record to generate a delayed reward record;

[0064] The delayed reward record is returned to the improved P-DQN hybrid action parameter optimization algorithm to calibrate the value of candidate actions. After the target sequencing procedure parameter scheme is executed, the newly generated parameter execution feedback data is written into the historical sequencing experience replay library for the generation of the next round of delayed reward record.

[0065] Optionally, step seven specifically includes:

[0066] Read the parameter action combination, execution phase marker, and calibrated candidate action value from the candidate parameter strategy set;

[0067] The parameter action combinations are filtered for execution constraints according to the execution stage markers. Parameter action combinations that do not match the current sequencing process stage, exceed the feasible domain of parameter actions, or have execution conflicts are deleted to obtain an executable parameter strategy set.

[0068] The executable parameter strategy set is sorted according to the value of the candidate actions after calibration, and the parameter strategy sorting result is generated.

[0069] Select the highest-ranked parameter action combination from the parameter strategy ranking results to generate the target sequencing workflow parameter scheme;

[0070] Select the second-highest ranked parameter action combination from the parameter strategy ranking results that meets the substitution conditions of the same execution stage, and generate a backup sequencing process parameter scheme.

[0071] Optionally, step eight specifically includes:

[0072] Read the target sequencing process parameter scheme and the backup sequencing process parameter scheme, convert the target sequencing process parameter scheme into parameter issuance instructions with execution stage markers, send the parameter issuance instructions to the sequencing process execution end, and receive the parameter execution feedback data returned by the sequencing process execution end;

[0073] Extract the protocol execution status, sequencing quality feedback and abnormal feedback markers from the parameter execution feedback data, and determine whether the target sequencing process parameter protocol meets the abnormal switching conditions according to the abnormal feedback markers.

[0074] When the target sequencing process parameter scheme meets the abnormal switching conditions, the backup sequencing process parameter scheme corresponding to the target sequencing process parameter scheme is invoked and sent to the sequencing process execution terminal.

[0075] Write the target sequencing process parameter scheme, the alternative sequencing process parameter scheme, parameter execution feedback data, and scheme switching results into the historical sequencing experience playback library.

[0076] The action boundaries of the corresponding execution stage in the feasible domain of the parameter action are corrected based on the feedback data.

[0077] The beneficial effects of this invention are:

[0078] This invention processes sequencing process state sequences by improving the GRU-D sequencing state drift prediction model. It utilizes a process observation attenuation embedding layer, a stage recursive characterization layer, a quality drift calibration layer, and a strategy state output layer to sequentially complete missing observation compensation, stage state recursion, quality drift calibration, and strategy state output. This enables scattered, unequally spaced, and missing sequencing process observation records to be transformed into sequencing parameter decision state profiles that can be used for parameter decision-making.

[0079] By improving the P-DQN hybrid action parameter optimization algorithm to process candidate parameter action chains, discrete process actions and continuous parameter vectors are placed in the same reinforcement learning decision process. Combined with the sequencing quality delay reward backfeed mechanism, the value of candidate actions is calibrated, so that the historical quality feedback formed after sequencing can be backfeed to the parameter strategy generation process, thus solving the problem of strategy learning lag caused by sequencing parameter actions being executed first and quality feedback appearing later.

[0080] By continuously generating sequencing task constraint profiles, sequencing process observation sequences, sequencing process status sequences, candidate parameter action chains, candidate parameter strategy sets, target sequencing process parameter schemes, and backup sequencing process parameter schemes, this invention forms a closed-loop parameter optimization process from task parsing, status prediction, strategy generation, execution screening to feedback updates, enabling gene sequencing process parameter configuration to have stage adaptation, master / backup switching, and continuous update capabilities. Attached Figure Description

[0081] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0082] Figure 1 This is an overall flowchart of a gene sequencing process parameter optimization method based on reinforcement learning proposed in this invention.

[0083] Figure 2 This is a flowchart illustrating the working principle of the improved GRU-D sequencing state drift prediction model proposed in this invention.

[0084] Figure 3 This is a flowchart illustrating the working principle of the improved P-DQN hybrid action parameter optimization algorithm and sequencing quality delay reward reinjection mechanism proposed in this invention. Detailed Implementation

[0085] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0086] refer to Figures 1-3 A reinforcement learning-based method for optimizing gene sequencing workflow parameters includes the following steps:

[0087] Step 1: Receive gene sequencing task information, analyze the task constraints of the gene sequencing task information, and generate a sequencing task constraint profile;

[0088] Step 2: Based on the sequencing task constraint profile, read the sequencing process observation records and generate sequencing process observation sequences;

[0089] Step 3: Perform state sequence normalization on the sequencing process observation sequences to generate sequencing process state sequences;

[0090] Step 4: Input the sequencing process state sequence into the improved GRU-D sequencing state drift prediction model to obtain a state profile of sequencing parameter decisions. The improved GRU-D sequencing state drift prediction model includes a process observation attenuation embedding layer, a stage recursive characterization layer, a quality drift calibration layer, and a strategy state output layer. The quality drift calibration layer embeds a stage hysteresis drift calibration mechanism.

[0091] Step 5: Map the feasible domain of the sequencing parameter decision state profile to generate candidate parameter action chains;

[0092] Step 6: Based on the historical sequencing experience replay library, the improved P-DQN hybrid action parameter optimization algorithm is used to process the candidate parameter action chain, and the candidate action value is calibrated through the sequencing quality delay reward reinjection mechanism to generate a candidate parameter strategy set;

[0093] Step 7: Perform execution constraint screening and strategy ranking on the candidate parameter strategy set to generate the target sequencing process parameter scheme and the alternative sequencing process parameter scheme;

[0094] Step 8: Send the target sequencing process parameter scheme to the sequencing process execution end, trigger the backup sequencing process parameter scheme based on the parameter execution feedback data, and update the historical sequencing experience replay library and parameter action feasible domain.

[0095] In this embodiment, step one specifically includes:

[0096] After receiving gene sequencing task information, the system performs integrity verification on the gene sequencing task information. The integrity verification objects include sequencing object information, target requirements, and execution requirements. Task records that are missing any of these objects are marked as records to be completed and will not enter the current round of parameter optimization process. For task records that pass the integrity verification, valid items are filtered out, and notes and descriptions that are not related to sequencing process parameter optimization are deleted to obtain valid task records.

[0097] A task index is generated based on valid task records. The task index is used to associate the records of the same gene sequencing task in the sequencing workflow observation sequence, candidate parameter strategy set, and parameter execution feedback data. Constraint vector transformation is performed on valid task records to convert sequencing object information into object constraint vectors, target requirements into quality constraint vectors, and execution requirements into workflow constraint vectors. The object constraint vector, quality constraint vector, and workflow constraint vector are generated by a combination of category coding and normalized numerical coding, and written into a unified profile structure according to the task index.

[0098] The unified profile structure is configured with observation read markers and parameter constraint call markers. The observation read markers are used to limit the reading range of the observation records in the sequencing process, and the parameter constraint call markers are used to limit the calling of the quality constraint vector and process constraint vector when mapping the feasible domain of parameter actions, so as to generate a sequencing task constraint profile.

[0099] Through the above processing, gene sequencing task information is transformed into a sequencing task constraint profile with task association, constraint expression, and call markers, enabling the sequencing process observation record reading, candidate parameter action chain construction, candidate parameter strategy screening, and execution feedback update to be executed continuously around the same task.

[0100] In this embodiment, step two specifically includes:

[0101] The system reads the task index and observation read marker from the sequencing task constraint profile; the task index is used to determine the set of observation records corresponding to the same gene sequencing task, and the observation read marker is used to limit the range of observation records participating in the optimization of process parameters; the system retrieves the sequencing process observation records according to the task index, and filters out records that are not related to parameter action mapping, strategy generation and execution feedback updates according to the observation read marker, so as to obtain the effective observation records;

[0102] Valid observation records are checked for source consistency. The source consistency check uses the task index, sequencing batch identifier, and record source identifier as the verification objects. Observation records with inconsistent task indexes, inconsistent batch identifiers, or untraceable record sources are not written into the sequencing workflow observation sequence. Valid observation records that pass the source consistency check are merged according to the workflow stage to obtain the stage observation record.

[0103] The system configures stage sequence identifiers and observation time identifiers for each stage observation record. The stage sequence identifier uses a two-digit code and increments sequentially according to the workflow of task configuration, sequencing operation, read processing, coverage analysis, and quality monitoring. The observation time identifier uses a millisecond-level timestamp and, together with the task index and stage sequence identifier, forms the observation unit index. The system sorts the stage observation records according to the stage sequence identifier and observation time identifier, and removes duplicate records with the same task index, stage sequence identifier, and observation time identifier to generate the sequencing workflow observation sequence.

[0104] Through the above processing, sequencing process observation records are converted into sequencing process observation sequences with task affiliation, source consistency, stage order, and time order, enabling the state sequence regularization step to generate sequencing process state sequences based on continuous observation records under the same task.

[0105] In this embodiment, step three specifically includes:

[0106] The system reads the stage observation records, stage sequence identifiers, and observation time identifiers from the sequencing workflow observation sequence, and establishes stage observation windows according to the stage sequence identifiers. The stage observation windows are arranged in the workflow order of task configuration, sequencing operation, read processing, coverage analysis, and quality monitoring. Each stage observation window retains 32 observation units. If there are more than 32 observation units, the observation units with the later time sequence are retained according to the observation time identifier. If there are fewer than 32 observation units, placeholder units are written in the empty positions.

[0107] The stage observation records within the stage observation window are encoded, converted, and coded to a unified scale. Categorical records are encoded using one-hot encoding, numerical records are encoded using min-max normalization, and missing placeholder cells are written with zero-value encoding. Each observation unit is encoded as a 24-dimensional observation encoding vector. The 32 observation units within the stage observation window are combined in chronological order to generate a 32×24-dimensional stage observation matrix.

[0108] A missing status marker is generated based on the missing position of the stage observation record within the stage observation window. The missing position is marked as 0, and the valid observation position is marked as 1. A status interval vector is generated based on the observation time marker of the adjacent stage observation window. The status interval vector is generated based on the time difference between the last valid observation time marker of the adjacent stage, and the unit is set to seconds. The time difference exceeding 7200 seconds is truncated and normalized according to 7200 seconds.

[0109] The stage observation matrix is ​​input into a linear mapping function and a nonlinear activation function to generate a 64-dimensional stage state vector; the missing state marker and the state interval vector are expanded into 64-dimensional vectors respectively, and combined with the 64-dimensional stage state vector according to the stage order identifier to generate the sequencing process state sequence.

[0110] Through the above processing, the sequencing process observation sequence is converted into a sequencing process state sequence containing stage state vectors, missing state markers, and state interval vectors, enabling the improved GRU-D sequencing state drift prediction model to read state inputs with the same dimension, the same order, and containing missing and time interval information.

[0111] In this embodiment, step four specifically includes:

[0112] The improved GRU-D sequencing state drift prediction model uses the sequencing process state sequence as input. The sequencing process state sequence includes a 64-dimensional stage state vector, a 64-dimensional missing state marker, and a 64-dimensional state interval vector. The original GRU-D model handles missing states through observation decay and updates hidden states through a cyclic recursive structure, making it suitable for time series modeling with missing observations and unequal interval observations. This implementation retains the observation decay handling and hidden state recursive update method of the original GRU-D model, and changes the general time series input of the original model to the sequencing process state sequence, and changes the ordinary prediction output of the original model to a sequencing parameter decision state profile.

[0113] The process observation attenuation embedding layer reads the stage state vector, missing state markers, and state interval vectors. It inputs the state interval vector into a learnable attenuation mapping function, which generates 64-dimensional observation attenuation coefficients through linear transformation, ReLU nonlinear activation, and exponential attenuation transformation. The observation attenuation coefficients are applied to the missing observation positions in the stage state vector according to the missing state markers. The effective observation positions retain their original encoded values, and the missing observation positions converge to the historical average state of the same stage according to the observation attenuation coefficients, resulting in the attenuation-compensated stage state vector. The process observation attenuation embedding layer then maps the attenuation-compensated stage state vector into a 128-dimensional process observation embedding vector.

[0114] The stage recursive representation layer follows the cyclic state update idea of ​​the original GRU-D model, using the 128-dimensional process observation embedding vector and the 128-dimensional hidden state of the previous process stage as recursive inputs. It calculates the hidden state of the current process stage through update gates, reset gates, and candidate hidden states. The stage recursive representation layer iterates recursively according to the stage order of task configuration, sequencing operation, read processing, coverage analysis, and quality monitoring to generate a 128-dimensional stage recursive state vector. Unlike the original GRU-D model, this implementation introduces stage order constraints in the recursive update process, so that the hidden state update no longer advances only with time points, but advances according to the stages of the gene sequencing process.

[0115] The quality drift calibration layer is an improvement on the output of the original GRU-D model. The original GRU-D model usually outputs the prediction result directly after the recursive hidden state. In this implementation, a quality drift calibration layer is added between the stage recursive state vector and the policy state output layer, and a stage lag drift calibration mechanism is embedded in the quality drift calibration layer. The stage lag drift calibration mechanism reads the state interval vector and generates the stage lag coefficient, reads the hidden state difference between adjacent process stages and generates the stage drift residual, and then corrects the stage drift residual with effective observation weights according to the missing state label to obtain the corrected drift residual.

[0116] The stage lag drift calibration mechanism performs a weighted mapping between the stage lag coefficient and the corrected drift residual to generate a 128-dimensional stage drift calibration term. The quality drift calibration layer fuses the stage drift calibration term with the 128-dimensional stage recursive state vector and generates a 128-dimensional quality drift calibration vector through a nonlinear calibration function. Through the above improvements, the model can further describe the problems of quality changes lagging behind the progress of the process stages, the impact of missing observations on drift judgment, and the impact of state differences between adjacent stages on parameter decisions in the gene sequencing process, based on the original GRU-D deletion decay and recursive modeling.

[0117] The strategy state output layer performs nonlinear projection, state normalization, and strategy state compression on the 128-dimensional quality drift calibration vector to generate a 128-dimensional sequencing parameter decision state profile; the sequencing parameter decision state profile is written into the parameter action mapping cache and used as the state input for constructing candidate parameter action chains;

[0118] Through the above processing, the improved GRU-D sequencing state drift prediction model retains the original GRU-D model's ability to handle missing states and recursive prediction. It introduces sequencing process state sequences at the input end, introduces stage sequence recursive representations in the intermediate layer, and introduces a stage lag drift calibration mechanism before output. This enables the sequencing process states that are staged, unequally spaced, and have missing states to be transformed into sequencing parameter decision state profiles suitable for reinforcement learning parameter action generation.

[0119] In this embodiment, step five specifically includes:

[0120] The system reads the feasible domain of parameter actions from the historical sequencing experience replay library. The feasible domain of parameter actions includes the stage action index and the action value boundary. When there is no feasible domain of parameter actions of the same task type or execution stage in the historical sequencing experience replay library, the system reads the initial safety parameter boundary to generate an initial feasible domain of parameter actions, and uses the initial feasible domain of parameter actions for the current round of parameter action mapping. The stage action index is used to limit the discrete process actions that can be generated in the current process stage, and the action value boundary is used to limit the value range of continuous parameter vectors. The system performs task consistency verification between the feasible domain of parameter actions and the sequencing parameter decision state profile. Feasible domains of parameter actions that are inconsistent with the task do not participate in the current round of mapping.

[0121] The system reads a 128-dimensional sequencing parameter decision state profile and performs state component analysis on the profile. The first 40 dimensions are used as stage state components, the middle 40 dimensions as drift calibration components, and the last 48 dimensions as policy constraint components. The stage state components are matched with the stage action index to determine the set of stage execution actions. The drift calibration components are associated with the set of stage execution actions to generate action adjustment direction markers. The policy constraint components are matched with the action value boundaries to generate continuous parameter boundary vectors.

[0122] The system generates discrete process actions according to the set of actions to be executed in each stage, and generates a 16-dimensional continuous parameter vector according to the action adjustment direction marker and the continuous parameter boundary vector. The continuous parameter vector is restricted to the value boundary of the corresponding action. The upward adjustment marker is generated in the direction of the upper limit of the boundary, and the downward adjustment marker is generated in the direction of the lower limit of the boundary. The marker is kept to use the most recently executable value of the action in the same stage. If there is no most recently executable value, the midpoint of the value boundary of the corresponding action is taken as the value of the continuous parameter vector.

[0123] The system links discrete process actions, continuous parameter vectors, and execution stage markers in a chain to generate candidate parameter action chains;

[0124] Through the above processing, the sequencing parameter decision state profile is transformed into a candidate parameter action chain with stage executability, boundary constraints, and continuous parameter values, enabling the improved P-DQN hybrid action parameter optimization algorithm to simultaneously handle discrete process actions and continuous parameter vectors.

[0125] In this embodiment, step six specifically includes:

[0126] The improved P-DQN hybrid action parameter optimization algorithm takes candidate parameter action chains as the processing object. The original P-DQN algorithm is used to process the parameterized action space where discrete actions and continuous action parameters coexist. The continuous parameter generation function generates corresponding continuous parameters for discrete actions, and the action value function evaluates the combined effect of discrete actions and continuous parameters.

[0127] This implementation retains the continuous parameter generation and action value evaluation method of the original P-DQN algorithm, and changes the general state action input of the original algorithm to the candidate parameter action chain in the gene sequencing process, and changes the ordinary action value output of the original algorithm to the candidate parameter strategy set.

[0128] The system reads the discrete process actions, 16-dimensional continuous parameter vectors, and execution stage markers from the candidate parameter action chain, and retrieves sequencing experience samples of the same stage from the historical sequencing experience replay library according to the execution stage markers. The sequencing experience samples of the same stage include historical state profiles, historical parameter actions, and historical execution feedback. The system performs task index matching and execution stage matching between the sequencing experience samples of the same stage and the candidate parameter action chain to generate reinforcement learning state action samples.

[0129] To adapt the original P-DQN algorithm to the stage execution characteristics of the gene sequencing workflow, a hybrid action evaluation object is constructed at the algorithm input. The hybrid action evaluation object consists of discrete workflow actions, a 16-dimensional continuous parameter vector, execution stage markers, and historical execution feedback references.

[0130] Execution phase markers are used to distinguish the execution meaning of the same continuous parameter in different sequencing process phases, and historical execution feedback references are used to call the execution results of historical strategies in the same phase;

[0131] The continuous parameter generation process reads the discrete process actions and 16-dimensional continuous parameter vectors from the mixed action evaluation object, performs boundary preservation and direction correction on the continuous parameter vectors, and generates candidate continuous parameter values. Boundary preservation is used to limit the candidate continuous parameter values ​​from exceeding the feasible domain of the parameter action, and direction correction is used to make the candidate continuous parameter values ​​consistent with the action adjustment direction mark.

[0132] The action value evaluation process reads discrete process actions, candidate continuous parameter values, and execution stage markers, and generates discrete action value and parameter action combination value respectively. The discrete action value is used to represent the selection value of the process action in the current stage, and the parameter action combination value is used to represent the execution value after the discrete process action is combined with the candidate continuous parameter value.

[0133] Unlike the original P-DQN algorithm, which directly corrects values ​​based on immediate rewards, this implementation calls the delayed reward records returned by the sequencing quality delayed reward reinjection mechanism at the value output end to calibrate the values ​​of discrete actions and parameter action combinations, generating calibrated candidate action values. In this implementation, the sequencing quality delayed reward reinjection mechanism only provides delayed reward records, and the generation process of the delayed reward records will be explained separately.

[0134] The system sorts the mixed action evaluation objects according to the value of the candidate actions after calibration, deletes parameter action combinations that do not meet the execution stage markers, and converts the remaining mixed action evaluation objects into a set of candidate parameter strategies.

[0135] Through the above processing, the improved P-DQN hybrid action parameter optimization algorithm retains the hybrid action processing capability of the original P-DQN algorithm, introduces execution stage markers and historical execution feedback references at the input end, introduces boundary preservation and direction correction at the continuous parameter generation end, and introduces delayed reward calibration at the value output end. This enables the candidate parameter action chain to be transformed into a candidate parameter strategy set that fits the gene sequencing process stages, continuous parameter boundaries, and historical quality feedback.

[0136] In this embodiment, the sequencing quality delay reward reinjection mechanism is specifically as follows:

[0137] The sequencing quality delay reward refeeding mechanism reads historical parameter execution feedback records and historical candidate parameter strategy records from the historical sequencing experience replay library and matches them according to the task index and execution stage marker; historical records that have not formed an execution feedback closed loop do not participate in the generation of delay reward records;

[0138] The mechanism extracts historical task completion feedback, historical sequencing quality feedback, and historical execution anomaly feedback from historical parameter execution feedback records;

[0139] Historical task completion feedback is converted into a completion reward component, historical sequencing quality feedback is converted into a quality reward component, and historical execution anomaly feedback is converted into an anomaly penalty component.

[0140] The completion reward component and quality reward component are normalized to the range of 0 to 1, and the anomaly penalty component is normalized to the range of -1 to 0. They are then combined to generate a historical feedback reward vector.

[0141] The mechanism performs stage attribution matching based on the execution stage markers in the historical candidate parameter strategy records; when there are multiple historical parameter actions within the same execution stage, stage attribution coefficients are generated according to the execution time order of the historical parameter actions and the scheme switching results, and the sum of the stage attribution coefficients is set to 1.

[0142] Historical feedback reward vectors are allocated to corresponding historical parameter actions according to stage attribution coefficients to generate stage-delayed rewards.

[0143] The mechanism binds the stage-delayed reward to the discrete action value and parameter action combination value in the historical candidate parameter strategy record to generate a delayed reward record;

[0144] Delayed reward records are written into the reward refill region of the historical sequencing experience replay library and returned to the improved P-DQN hybrid action parameter optimization algorithm for value calibration of discrete action value and parameter action combination value;

[0145] After the target sequencing workflow parameter scheme is executed, the system writes the newly generated parameter execution feedback data into the historical sequencing experience replay library according to the task index and marks it as a record to be re-injected;

[0146] The records to be reinjected will be transformed into historical parameter execution feedback records during the next round of candidate parameter strategy generation, and will participate in the generation of delayed reward records.

[0147] Through the above processing, the quality feedback and abnormal feedback that appear after sequencing are completed are transformed into delayed reward records that can calibrate the value of candidate actions, enabling the improved P-DQN hybrid action parameter optimization algorithm to continuously learn the correspondence between historical parameter actions and sequencing execution results.

[0148] In this embodiment, step seven specifically includes:

[0149] The system reads the parameter action combinations, execution stage markers, and calibrated candidate action values ​​from the candidate parameter strategy set. The parameter action combination consists of discrete process actions and candidate continuous parameter values. The calibrated candidate action values ​​are uniformly normalized to the range of 0 to 1. The system determines the current sequencing process stage according to the execution stage marker, matches the parameter action combination with the parameter action feasible region, and deletes parameter action combinations that are inconsistent with the current sequencing process stage.

[0150] The system performs boundary checks and conflict checks on the retained parameter action combinations. Boundary checks are used to determine whether the candidate consecutive parameter values ​​are within the action value boundary corresponding to the feasible domain of the parameter action. Conflict checks are used to determine whether there are parameter action combinations with opposite action directions, duplicate parameter objects, or mutually exclusive execution orders within the same execution phase. Parameter action combinations that fail the boundary checks or conflict checks are not included in the strategy sorting.

[0151] The system will write the parameter action combination filtered by the execution constraints into the executable parameter strategy set, and sort the strategies from high to low according to the value of the calibrated candidate actions;

[0152] When the difference in candidate action value after calibration is less than 0.01, they are treated as having equal value. Priority is given to retaining parameter action combinations with fewer execution conflicts and whose candidate continuous parameter values ​​are closer to the midpoint of the corresponding action value boundary, thus generating parameter strategy ranking results.

[0153] The system selects the highest-ranked parameter action combination from the parameter strategy ranking results to generate the target sequencing process parameter scheme;

[0154] From the parameter action combinations that are ranked in the top 5 of the parameter strategy ranking results and have not been selected as the target sequencing process parameter scheme, reserve candidates are selected. If the parameter strategy ranking results are less than 5, all remaining parameter action combinations are selected. From the reserve candidates, parameter action combinations that have the same execution stage and do not conflict with the target sequencing process parameter scheme are selected to generate reserve sequencing process parameter schemes.

[0155] Through the above processing, the candidate parameter strategy set is selected as the target sequencing process parameter scheme and the backup sequencing process parameter scheme that meet the constraints of execution stage, parameter boundary and action conflict. This enables the sequencing process execution end to prioritize the execution of the target scheme and call the alternative backup scheme in the same stage when the target scheme is abnormal.

[0156] In this embodiment, step eight specifically includes:

[0157] The system reads the target sequencing workflow parameter scheme and the backup sequencing workflow parameter scheme; the target sequencing workflow parameter scheme includes task index, execution stage marker, discrete workflow actions and a 16-dimensional continuous parameter vector.

[0158] The system generates parameter delivery instructions based on the task index and execution stage marker. The parameter delivery instructions include the execution stage marker, parameter action content, and check code. The check code is used by the sequencing process execution end to confirm that the parameter delivery instructions have not been missing any content.

[0159] The system sends parameter distribution instructions to the sequencing process execution end and receives parameter execution feedback data returned by the sequencing process execution end. The parameter execution feedback data includes protocol execution status, sequencing quality feedback, and anomaly feedback flags. Sequencing quality feedback is uniformly normalized to the 0 to 1 range, and anomaly feedback flags are represented by 0 and 1, where 0 indicates no anomaly was triggered and 1 indicates an anomaly was triggered.

[0160] When the execution status of the protocol is failed, the abnormal feedback is marked as 1, the sequencing quality feedback is lower than the lower limit of the quality constraint vector, or the parameter issuance command verification fails, the target sequencing process parameter protocol is determined to meet the abnormal switching conditions. When the abnormal switching conditions are met, the system calls the corresponding backup sequencing process parameter protocol and converts the backup sequencing process parameter protocol into a new parameter issuance command before sending it to the sequencing process execution end.

[0161] The system writes the target sequencing workflow parameter scheme, the backup sequencing workflow parameter scheme, parameter execution feedback data, and scheme switching results into the historical sequencing experience replay library according to the task index, and marks them as records to be re-injected; the scheme switching results are used to distinguish between three states: the target scheme is executed normally, the backup scheme is triggered to execute, and the scheme execution fails.

[0162] The system corrects the action boundaries of the corresponding execution stage in the feasible domain of parameter actions according to the parameter execution feedback data; when the parameter action combination is executed successfully and the sequencing quality feedback meets the lower limit corresponding to the quality constraint vector, the corresponding action boundary is retained and the executable value is recorded.

[0163] When the combination of parameter actions triggers an abnormal switch or the sequencing quality feedback is lower than the lower limit corresponding to the quality constraint vector, the value range near the corresponding continuous parameter vector is marked as a risk boundary region, and the priority of calling the risk boundary region in the next round of parameter action feasible domain mapping is reduced.

[0164] Through the above processing, the target sequencing process parameter scheme and the backup sequencing process parameter scheme form an execution closed loop that can be distributed, switched, and updated with feedback, so that the historical sequencing experience replay library and the parameter action feasible domain can be continuously updated with the sequencing process execution results.

[0165] Example 1: To verify the feasibility of the present invention in practice, the present invention was applied to the optimization of gene sequencing process parameters in a gene testing laboratory; the experimental subjects were 96 continuously received gene sequencing tasks, the types of which included genetic disease screening, tumor targeted detection and pathogen identification;

[0166] The system reads the sequencing process observation records according to the task index, generates the sequencing process status sequence, and uses an improved GRU-D sequencing state drift prediction model to generate a sequencing parameter decision status profile. Then, it generates a candidate parameter strategy set through an improved P-DQN hybrid action parameter optimization algorithm. Finally, it obtains the target sequencing process parameter scheme and the backup sequencing process parameter scheme through execution constraint screening.

[0167] The experiment set up three comparison methods: fixed template parameter method, manual experience adjustment method, and ordinary P-DQN parameter optimization method. The fixed template parameter method uses fixed laboratory process parameters, the manual experience adjustment method is adjusted by technicians based on intermediate sequencing results, and the ordinary P-DQN parameter optimization method does not introduce stage lag drift calibration mechanism and sequencing quality delay reward reinjection mechanism.

[0168] All methods were run under the same task set, the same sequencing platform, and the same analysis workflow. Abnormal tasks were confirmed by the joint confirmation of equipment alarm logs, quality control results, and manual review records. The effective read ratio, target coverage completion rate, average number of parameter corrections, average workflow time, and anomaly handling success rate were statistically analyzed. The results are shown in Table 1.

[0169] Table 1. Application results of the present invention and comparative methods in the optimization of gene sequencing process parameters.

[0170] method Effective segment reading ratio / % Target coverage completion rate / % Average parameter correction times / times Average process time / h Exception handling success rate / % Fixed template parameter method 84.7 88.1 3.6 8.2 58.9 Manual experience adjustment method 88.2 91.0 2.8 7.4 69.3 Common P-DQN parameter optimization methods 90.5 93.7 2.1 6.6 78.4 Method of the present invention 93.6 96.0 1.3 5.9 88.7

[0171] As shown in Table 1, the effective read ratio of the method of the present invention is 93.6%, and the target coverage completion rate is 96.0%, both of which are higher than the fixed template parameter method, the manual experience adjustment method, and the ordinary P-DQN parameter optimization method. The average number of parameter corrections is reduced to 1.3 times, the average process time is reduced to 5.9 hours, and the anomaly handling success rate reaches 88.7%, indicating that the target sequencing process parameter scheme and the backup sequencing process parameter scheme can form a more stable execution closed loop in different sequencing tasks.

[0172] Effective read ratio reflects the impact of parameter scheme on read quality and read retention; target coverage completion rate reflects the coverage of the target region of the sequencing task; average number of parameter corrections reflects whether the parameter scheme needs frequent correction; average process time reflects the overall efficiency of sequencing operation and analysis; anomaly handling success rate reflects the ability to complete task recovery by adjusting parameter scheme, calling backup schemes, or manual intervention after a process anomaly occurs.

[0173] The reason why the method of the present invention achieves the above results is that the improved GRU-D sequencing state drift prediction model can transform missing observations, state intervals, and stage quality drift into a state profile of sequencing parameter decision; the improved P-DQN hybrid action parameter optimization algorithm can simultaneously handle discrete process actions and continuous parameter vectors; the sequencing quality delay reward reinjection mechanism can transform historical sequencing quality feedback into delay reward records, so as to continuously calibrate the value of candidate actions; the above mechanisms together transform gene sequencing process parameter optimization from a fixed configuration into a closed-loop optimization process based on state prediction, strategy generation, master-slave execution, and feedback update.

[0174] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing gene sequencing workflow parameters based on reinforcement learning, characterized in that, Includes the following steps: Step 1: Receive gene sequencing task information, analyze the task constraints of the gene sequencing task information, and generate a sequencing task constraint profile; Step 2: Based on the sequencing task constraint profile, read the sequencing process observation records and generate sequencing process observation sequences; Step 3: Perform state sequence normalization on the sequencing process observation sequences to generate sequencing process state sequences; Step 4: Input the sequencing process state sequence into the improved GRU-D sequencing state drift prediction model to obtain a state profile of sequencing parameter decisions. The improved GRU-D sequencing state drift prediction model includes a process observation attenuation embedding layer, a stage recursive characterization layer, a quality drift calibration layer, and a strategy state output layer. The quality drift calibration layer embeds a stage hysteresis drift calibration mechanism. Step 5: Map the feasible domain of the sequencing parameter decision state profile to generate candidate parameter action chains; Step 6: Based on the historical sequencing experience replay library, the improved P-DQN hybrid action parameter optimization algorithm is used to process the candidate parameter action chain, and the candidate action value is calibrated through the sequencing quality delay reward reinjection mechanism to generate a candidate parameter strategy set; Step 7: Perform execution constraint screening and strategy ranking on the candidate parameter strategy set to generate the target sequencing process parameter scheme and the alternative sequencing process parameter scheme; Step 8: Send the target sequencing process parameter scheme to the sequencing process execution end, trigger the backup sequencing process parameter scheme based on the parameter execution feedback data, and update the historical sequencing experience replay library and parameter action feasible domain.

2. The method for optimizing gene sequencing process parameters based on reinforcement learning according to claim 1, characterized in that, Step one specifically involves: Read the gene sequencing task information, perform integrity verification on the gene sequencing task information, remove descriptive items that are not related to the optimization of sequencing process parameters, and obtain valid task records; A task index is generated based on valid task records. The task index is used to associate the records of the same sequencing task in the sequencing workflow observation sequences, candidate parameter strategy sets, and parameter execution feedback data. The sequencing object information in the valid task record is converted into an object constraint vector, the target requirements are converted into a quality constraint vector, and the execution requirements are converted into a process constraint vector. Write the task index, object constraint vector, quality constraint vector, and process constraint vector into a unified profile structure, and configure observation read markers and parameter constraint call markers for the unified profile structure to generate a sequencing task constraint profile.

3. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step two specifically involves: Read the task index and observation read marker from the sequencing task constraint profile, and retrieve the corresponding sequencing process observation records according to the task index; The effective observation records are obtained by filtering out sequencing process observation records that are irrelevant to process parameter optimization according to the observation reading markers; The valid observation records are verified for consistency with their source. The valid observation records that pass the consistency verification are then merged according to the process stages to obtain the stage observation records. Configure stage sequence identifiers and observation time identifiers for stage observation records, sort and deduplicate stage observation records according to stage sequence identifiers and observation time identifiers, and generate observation sequences for sequencing process.

4. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step three specifically involves: Read the stage observation records, stage sequence identifiers, and observation time identifiers from the sequencing process observation sequence, and establish stage observation windows according to the stage sequence identifiers; The stage observation records within the stage observation window are encoded, converted, and uniformly scaled to generate a stage observation matrix. A missing status marker is generated based on the missing location of the stage observation record within the stage observation window, and a status interval vector is generated based on the observation time identifier of the adjacent stage observation window. The stage observation matrix is ​​mapped to a stage state vector, and the stage state vector, missing state marker, and state interval vector are combined according to the stage order identifier to generate the sequencing process state sequence.

5. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step four specifically involves: The process observation attenuation embedding layer reads the stage state vector, missing state marker and state interval vector in the sequencing process state sequence, and uses a learnable attenuation mapping to perform a nonlinear transformation on the state interval vector to generate the observation attenuation coefficient. The effective observation positions and missing observation positions in the stage state vector are determined according to the missing state marker. The stage state vector corresponding to the missing observation position is attenuated and compensated using the observation attenuation coefficient. The attenuated and compensated stage state vector is then mapped to the process observation embedding vector. The stage recursive representation layer reads the process observation embedding vector and uses the hidden state of the previous process stage as a recursive reference. It then performs non-linear recursive fusion of the process observation embedding vector and the hidden state of the previous process stage through a loop state update function to generate the hidden state of the current process stage. Iteratively update the hidden state of the current process stage according to the process stage sequence, and generate a stage recursive state vector; The quality drift calibration layer reads the recursive state vector, state interval vector, and missing state marker during the reading phase, and executes the hysteresis drift calibration mechanism during the execution phase. The stage lag drift calibration mechanism generates stage lag coefficients based on the state interval vector, generates stage drift residuals based on the hidden state differences between adjacent process stages, and corrects the stage drift residuals with effective observation weights based on the missing state labels to obtain corrected drift residuals. The stage lag drift calibration mechanism weights and maps the stage lag coefficient with the corrected drift residual to generate a stage drift calibration term. The mass drift calibration layer performs residual fusion of the stage drift calibration term and the stage recursive state vector, and performs nonlinear calibration on the residual fusion result to generate the mass drift calibration vector. The policy state output layer performs nonlinear projection, state normalization, and policy state compression on the quality drift calibration vector to generate a sequencing parameter decision state profile for constructing candidate parameter action chains.

6. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step five specifically involves: Read the feasible domain of parameter actions from the historical sequencing experience replay library, read the sequencing parameter decision state profile, and perform state component analysis on the sequencing parameter decision state profile to obtain the stage state component, drift calibration component and strategy constraint component. Match the stage state components with the stage action indexes in the parameter action feasible domain to determine the set of actions to be executed in the stage; The drift calibration component is associated with the set of stage execution actions to generate action adjustment direction markers; Match the policy constraint components with the action value boundaries in the feasible domain of parameter actions to generate a continuous parameter boundary vector; Discrete process actions are generated according to the set of actions to be executed in stages, and continuous parameter vectors are generated according to the action adjustment direction markers and continuous parameter boundary vectors. The discrete process actions, continuous parameter vectors, and execution stage markers are linked together to generate candidate parameter action chains.

7. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step six specifically involves: Read the discrete process actions, continuous parameter vectors, and execution stage markers in the candidate parameter action chain, and retrieve sequencing experience samples of the same stage from the historical sequencing experience replay library according to the execution stage markers; By matching the historical state profiles, historical parameter actions, and historical execution feedback in the sequencing experience samples of the same stage with the candidate parameter action chain, reinforcement learning state action samples are generated. A hybrid action evaluation object is established based on reinforcement learning state-action samples. The hybrid action evaluation object includes discrete process actions, continuous parameter vectors, execution phase labels, and historical execution feedback references. An improved P-DQN hybrid motion parameter optimization algorithm is used to process the hybrid motion evaluation object. The continuous parameter vector is corrected by the continuous parameter generation function to generate candidate continuous parameter values. The discrete process actions, candidate continuous parameter values, and execution stage markers are jointly evaluated to generate discrete action value and parameter action combination value. The delayed reward records returned by the sequencing quality delayed reward reinjection mechanism are invoked to calibrate the value of discrete actions and the value of parameter action combinations, and generate calibrated candidate action values. The mixed action evaluation objects are sorted according to the value of the candidate actions after calibration, and parameter action combinations that do not meet the execution stage markers are filtered out to generate a candidate parameter strategy set.

8. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 7, characterized in that, The sequencing quality delay reward refeeding mechanism is as follows: Read historical parameter execution feedback records and historical candidate parameter strategy records from the historical sequencing experience replay library, and extract historical task completion feedback, historical sequencing quality feedback and historical execution anomaly feedback; Based on the execution stage markers in the historical candidate parameter strategy records, the historical parameter actions and historical parameter execution feedback records are matched for stage attribution to generate historical stage feedback association records. Reward components are converted from historical task completion feedback, historical sequencing quality feedback, and historical execution anomaly feedback to generate completion reward components, quality reward components, and anomaly penalty components. Based on the historical stage feedback and correlation records, the completion reward component, quality reward component, and anomaly penalty component are allocated to the corresponding historical parameter actions to generate stage-delayed rewards. The stage-delayed reward is bound to the discrete action value and parameter action combination value in the historical candidate parameter strategy record to generate a delayed reward record; The delayed reward record is returned to the improved P-DQN hybrid action parameter optimization algorithm to calibrate the value of candidate actions. After the target sequencing procedure parameter scheme is executed, the newly generated parameter execution feedback data is written into the historical sequencing experience replay library for the generation of the next round of delayed reward record.

9. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step seven specifically involves: Read the parameter action combination, execution phase marker, and calibration value of the candidate action from the candidate parameter strategy set; The parameter action combinations are filtered for execution constraints according to the execution stage markers. Parameter action combinations that do not match the current sequencing process stage, exceed the feasible domain of parameter actions, or have execution conflicts are deleted to obtain an executable parameter strategy set. The executable parameter strategy set is sorted according to the value of the candidate actions after calibration, and the parameter strategy sorting result is generated. Select the highest-ranked parameter action combination from the parameter strategy ranking results to generate the target sequencing workflow parameter scheme; Select the second-highest ranked parameter action combination from the parameter strategy ranking results that meets the substitution conditions of the same execution stage, and generate a backup sequencing process parameter scheme.

10. The method for optimizing gene sequencing workflow parameters based on reinforcement learning according to claim 1, characterized in that, Step eight specifically involves: Read the target sequencing process parameter scheme and the backup sequencing process parameter scheme, convert the target sequencing process parameter scheme into parameter issuance instructions with execution stage markers, send the parameter issuance instructions to the sequencing process execution end, and receive the parameter execution feedback data returned by the sequencing process execution end; Extract the protocol execution status, sequencing quality feedback and abnormal feedback markers from the parameter execution feedback data, and determine whether the target sequencing process parameter protocol meets the abnormal switching conditions according to the abnormal feedback markers. When the target sequencing process parameter scheme meets the abnormal switching conditions, the backup sequencing process parameter scheme corresponding to the target sequencing process parameter scheme is invoked and sent to the sequencing process execution terminal. Write the target sequencing process parameter scheme, the alternative sequencing process parameter scheme, parameter execution feedback data, and scheme switching results into the historical sequencing experience playback library. The action boundaries of the corresponding execution stage in the feasible domain of the parameter action are corrected based on the feedback data.

Images