FPVA biochip layout method considering reagent type difference in unit multiplexing

Abstract

The invention provides an FPVA biochip layout method considering reagent type difference in unit multiplexing, which comprises the following steps of: modeling a biochip layout task as a Markov decision process: according to the size of an input FPVA chip, the dependency relationship of each operation in a biochemical reaction sequence diagram, the reagent type contained in each operation and a binding scheduling scheme, determining the FPVA biochip layout task; initializing an FPVA mesh model and defining a state space, an action space and a reward function; wherein the reward function is set according to the total unit multiplexing complexity of the current layout scheme, the legality of the current component layout and the distance between the component and the center point of the FPVA chip; and finally, completing the FPVA biochip layout through the dual deep Q network based on a Markov decision process. By designing a reward function, the unit multiplexing complexity is minimized, and the distance between components containing the same type of reagents is reduced, so that the purposes of minimizing the cross contamination degree of a final layout scheme, the total length of a fluid transportation path and the bioassay completion time are achieved.

Description

Technical Field

[0001] This invention relates to the field of automated electronic design of microfluidic biochips, and more specifically to an FPVA biochip layout method that takes into account the differences in reagent types during unit reuse. Background Technology

[0002] As a next-generation flow microfluidic chip architecture, the Fully Programmable Valve Array (FPVA) biochip combines the flexibility of digital chips with the high-throughput characteristics of flow chips, enabling more efficient completion of complex biochemical experimental tasks. The core of the FPVA chip is a two-dimensional array of multiple microvalves. These valves are programmed to switch between open and closed states, and each valve can be independently controlled. These valves separate multiple adjacent physical units, allowing for precise control of fluid flow between adjacent units. The unique feature of the FPVA biochip lies in its high flexibility and programmability. It can dynamically adjust fluid paths and reaction components according to needs, giving it stronger control capabilities and greater reconfigurability. Its design process can be divided into two stages: high-level synthesis and physical design. High-level synthesis includes bonding and scheduling, while physical design includes placement and routing. Layout is a critical step in the physical design of FPVA biochips, which determines the placement of all components. The layout algorithm affects a range of metrics, including bioassay completion time, total length of fluid transport paths, reagent consumption, and degree of cross-contamination, thus impacting the overall performance of the chip.

[0003] Cell reuse in FPVA refers to the ability of multiple fluids or components to share certain identical cells at different times, which is a key aspect of FPVA's flexibility and programmability. In FPVA layout, cell reuse primarily manifests as the overlapping of non-concurrent components. The way components overlap affects chip performance and reliability in several ways. First, it affects the degree of cross-contamination; the more types of reagents passing through the same cell, the greater the contamination. Second, it affects biochemical detection performance indicators such as the total length of the fluid transport path and the bioassay completion time. This is mainly reflected in the fact that the closer components containing the same reagent are, the shorter the total fluid transport path and the shorter the bioassay completion time. However, existing FPVA layout research has not considered the impact of reagent type differences during cell reuse on these important indicators. Therefore, this invention focuses on the issue of reagent type differences in cell reuse during FPVA layout and proposes an FPVA layout method based on deep reinforcement learning that considers reagent type differences in cell reuse, thereby effectively improving chip performance and reliability. Summary of the Invention

[0004] The purpose of this invention is to propose an FPVA biochip layout method that considers the differences in reagent types during unit reuse. The biochip layout task is modeled as a Markov decision process, and the reward function is designed to minimize the complexity of unit reuse and reduce the distance between components containing the same type of reagent. This achieves the goal of minimizing the degree of cross-contamination, the total length of fluid transport paths, and the completion time of bioassays in the final layout scheme, thereby effectively improving the performance and reliability of the chip.

[0005] To achieve the above objectives, the technical solution of the present invention is: an FPVA biochip layout method considering the differences in reagent types during unit reuse, specifically including the following steps:

[0006] Modeling the biochip placement task as a Markov decision process:

[0007] Based on the input FPVA chip size, the dependencies of each operation in the biochemical reaction timing diagram, the types of reagents contained in each operation, and the binding scheduling scheme, the FPVA grid model is initialized and the state space, action space, and reward function are defined. The binding scheduling scheme includes the component number and size bound to each operation and the timing information of each operation.

[0008] Initializing the FPVA mesh model includes: modeling the FPVA biochip as a two-dimensional matrix structure according to the size of the FPVA chip, in order to obtain the layout information of all components on the FPVA chip based on the two-dimensional matrix structure, including the coordinates and dimensions of the components;

[0009] Define the state space: The environment state of the agent at each step includes the total unit reuse complexity of the current layout scheme and the layout information of all components;

[0010] Define the action space: the agent's action corresponds to selecting a component and determining the direction of the component's movement;

[0011] The environment updates the layout scheme based on the agent's actions and determines whether the current layout scheme is valid. If the current layout scheme is valid, the layout state is updated and the total unit reuse complexity of the new layout scheme is calculated. Otherwise, the action is invalid and the layout scheme is rolled back to the state before the action.

[0012] Define the reward function: Set the reward function based on the total cell reuse complexity of the current layout scheme, the legality of the current component layout, and the distance of the component from the center point of the FPVA chip;

[0013] The FPVA biochip layout was completed using a dual deep Q-network based on Markov decision processes.

[0014] Preferably, the modeling of FPVA biochips is as follows:

[0015] Based on the size of the FPVA biochip, the FPVA biochip is modeled as an m×n two-dimensional matrix structure. Each position in the two-dimensional matrix structure corresponds to a basic functional unit on the FPVA chip. The position of the basic functional unit is represented by coordinates (x, y), where 0 ≤ x ≤ m-1 and 0 ≤ y ≤ n-1. x represents the unit being located in the x-th column from left to right in the two-dimensional matrix structure, and y represents the unit being located in the y-th row from bottom to top in the two-dimensional matrix structure. Each basic functional unit includes several microvalves and flow channels.

[0016] The fluid transport path is constructed by setting the opening and closing of microvalves between adjacent basic functional units;

[0017] The basic functional unit with coordinates (0,0) is set as the input unit, and the basic functional unit with coordinates (m,n) is set as the output unit. The fluid is input from the (0,0) unit and output from the (m,n) unit.

[0018] The position and size of components in a two-dimensional matrix structure using (locx) i ,locy i ,lenx i ,leny i ) indicates that locx i locy i lenx i leny i These represent the horizontal and vertical coordinates of component i, and the number of units occupied by component i in the horizontal and vertical directions, respectively.

[0019] Preferably, the calculation of the total unit reuse complexity of the current layout scheme is as follows:

[0020]

[0021] Where P represents the total unit reuse complexity of the current layout scheme; p i,j Measure the unit reuse complexity of the cell at coordinate (i,j); N a,b α represents the number of different types of reagents in the two components numbered a and b; a,b α is a weight parameter; when the blend operations bound to the two components have no dependency, α a,b =1, when there is a dependency between the mixed operations bound to the two components, 0 < α a,b <1, and the smaller the distance between the operations bound to the two components in the sequence diagram, the lower α is. a,b The smaller the value, the better; c i,j,a,b Let c be a binary variable. When both components numbered a and b have occupied the (i,j) unit, c i,j,a,b =1, otherwise c i,j,a,b =0.

[0022] Preferably, the state space S is represented as:

[0023] S={s t |s t =[P,x0,y0,x1,y1,x2,y2,…,x i ,y i ,…,x l-1 ,y l-1 ], 0≤x i <m,0≤y i <n}

[0024] Among them, s t Let x represent the environment state of the agent at step t, P represent the total unit reuse complexity of the current layout scheme, and x represent the environment state of the agent at step t. i y i Let x and y be the x and y coordinates of component i, respectively, where 0 ≤ i < l, and l is the number of components.

[0025] Preferably, the action space A is represented as:

[0026] A={a t |0≤a t <l×5}

[0027] Among them, a t This represents the component selected by the agent at step t and its direction of movement; each component can move or remain stationary in one of the four directions: up, down, left, or right, and can move at most one unit at a time.

[0028] Preferably, the current layout scheme is legal if it meets all constraints, including layout area restriction constraints and concurrent component non-overlapping constraints;

[0029] The layout area constraint is that components cannot occupy units that are not suitable for component layout, as expressed as:

[0030] (locx,locy)≠(0,0)

[0031] locx + lenx < m || locy + leny < n

[0032] Where locx and locy represent the horizontal and vertical coordinates of the component; lenx and leny represent the number of units occupied by the component in the horizontal and vertical directions;

[0033] The constraint that concurrent components do not overlap means that concurrent components in the scheduling scheme cannot overlap with each other.

[0034] Preferably, the non-overlapping constraint of concurrent components specifically means that for any two concurrent components (locx) in the scheduling scheme... i ,locyi ,lenx i ,leny i ) and (locx j ,locy j ,lenx j ,leny j It needs to satisfy any one of the following four formulas:

[0035] locx i +lenx i ≤locx j

[0036] locy i +leny i ≤locy j

[0037] locx j +lenx j ≤locx i

[0038] locy j +leny j ≤locy i

[0039] Where locx j locy j lenx j leny j These represent the horizontal and vertical coordinates of component j, and the number of units occupied by component j in the horizontal and vertical directions, respectively.

[0040] Preferably, the reward function is specifically set as follows:

[0041]

[0042] Where, r t This indicates that the agent is in state s. t Take action a t The immediate reward obtained afterward; P pre and P now λ represents the total unit reuse complexity corresponding to the layout schemes before and after the action; λ is a weighting coefficient used to balance the relative importance of unit reuse complexity and component distance from the center point; β is a ternary variable, which is the value of the component's position when action a... t When the component is moved away from the center of the chip, β = -1; when action a t Move the component closer to the center of the chip, β = 1; when action a t Keep the components stationary, β = 0; C2 and C1 are both constants.

[0043] Preferably, the FPVA biochip layout based on Markov decision processes using dual deep Q-networks specifically includes the following steps:

[0044] S1. Initialization: Initialize the main network parameters and target network parameters, and clear the experience replay pool;

[0045] S2, Environment Interaction: In the current state s t Below, the main network uses the ε-greedy policy to select action a. t The exploration rate decreases linearly; after the selected action is performed, the environment returns to a new state s. t+1 and instant rewards t , will (s t ,a t ,r t ,s t+1 Stored in the experience replay pool;

[0046] S3. Sampling and Training: Once the number of samples in the experience replay pool reaches a certain size, a small batch of samples is randomly sampled for training, and the target Q-value for each sample is calculated.

[0047]

[0048] Where, r t It is the immediate reward obtained at the current moment; γ is the discount factor; s t+1 The next state; Q target It is the Q-value estimated by the target network; Q online θ is the Q-value estimated by the current main network; a′ is the possible next action; θ is the main network parameter. - These are the target network parameters;

[0049] The main network parameters are updated by minimizing the loss function of the main network based on the calculated target Q value:

[0050]

[0051] in, Let Q be the target Q-value for the k-th sample; Q is the Q-value estimate of the current main network for the current state and action. Let N represent the current state and action of the k-th sample; N is the sample size.

[0052] Every C steps, the main network parameters are copied to the target network parameters to ensure the stability of the target value estimation.

[0053] S4. Inference Phase and Layout Output: After training is completed, the trained final target network is used for layout inference; the current state is initialized, and then a loop is entered to continuously make layout decisions until the conditions for layout completion are met;

[0054] In each loop, the agent selects an optimal action based on the current state and the trained target network, that is, determines the layout movement direction of a certain component; after executing the optimal action, a new state is obtained, and the optimal action is recorded for subsequent construction of a complete layout scheme, and then the current state is updated; then the next round of state evaluation and action selection begins.

[0055] After reasoning is complete, the final layout scheme, the total unit reuse complexity of the final layout scheme, the total length of the fluid transport path, and the completion time of the bioassay are output.

[0056] Preferably, the total length of the fluid transport path and the bioassay completion time are estimated by calculating the Manhattan distance.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] (1) This invention designs an FPVA layout algorithm based on deep reinforcement learning (DRL) that considers the differences in reagent types in unit reuse. This method models the FPVA layout task as a Markov decision process, and then applies reinforcement learning to solve it. At the same time, it considers the differences in reagent types in the unit reuse process, and finally generates a high-quality FPVA layout scheme.

[0059] (2) By considering the differences between reagents and the impact of component overlap on cross-contamination, a unit reuse complexity index and an average distance index containing the same reagent components were designed to quantify the unit reuse complexity and the degree of aggregation of units containing the same reagent components in the layout scheme, respectively. In addition, layout scheme constraints, including layout area restriction constraints and concurrent component non-overlap constraints, were introduced to determine the legality of the layout scheme.

[0060] (3) Design an effective DRL reward function to reduce the complexity of unit reuse and the distance between components containing the same type of reagents. Then, during the training process, continuously update the target network parameters of the Double Deep Q-Network (DDQN) based on the reward value. Finally, optimize the overlap between components through model inference, thereby minimizing the total unit reuse complexity, total length of fluid transport path and bioassay completion time of the final layout scheme. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the FPVA biochip modeling of the present invention;

[0062] Figure 2The diagram below is an example of the complexity calculation process for unit reuse in this invention, where (a) represents a biochemical detection timing diagram and (b) represents an example of the complexity calculation for unit reuse.

[0063] Figure 3 This is a schematic diagram showing the movement direction of the component of the present invention;

[0064] Figure 4 This is a schematic diagram of the DDQN training process of the present invention. Detailed Implementation

[0065] The following is in conjunction with the appendix Figure 1-4 The technical solution of the present invention will be described in detail below.

[0066] As a novel flow microfluidic biochip, the Fully Programmable Valve Array (FPVA) biochip boasts excellent flexibility and programmability, capable of meeting a variety of complex experimental needs. As a crucial stage in architecture synthesis, FPVA layout affects several performance metrics, including bioassay completion time, total fluid transport path length, and cross-contamination levels. Furthermore, unit reuse is a significant indicator of the flexibility and programmability of FPVA chips. However, existing research on FPVA layout has not considered the impact of reagent type differences during unit reuse on these important metrics.

[0067] To address this issue, this invention focuses on the problem of reagent type differences in unit reuse during FPVA layout and proposes an FPVA layout method based on deep reinforcement learning that considers reagent type differences in unit reuse. First, by considering the differences between reagents and the impact of component overlap on cross-contamination, two indices are designed: unit reuse complexity and average distance between components containing the same reagent. These are used to quantify the unit reuse complexity and the clustering degree of components containing the same reagent in each unit of the layout scheme, respectively. Second, constraints, including layout region restrictions and concurrent component non-overlapping constraints, are introduced to ensure the legality of the layout scheme. Finally, a reward function is designed to minimize unit reuse complexity and reduce the distance between components containing the same type of reagent, thereby minimizing the cross-contamination degree, total fluid transport path length, and bioassay completion time of the final layout scheme. The proposed layout method is evaluated through simulation experiments. Experimental results show that the layout method designed in this invention optimizes the total unit reuse complexity, bioassay completion time, and total fluid transport path length by an average of 34.2%, 2.8%, and 9.2%, respectively. Therefore, this invention can obtain a high-quality FPVA layout scheme that considers reagent type differences in unit reuse.

[0068] The specific solution of the present invention is as follows:

[0069] 1. Environmental Modeling

[0070] The FPVA chip can be viewed as a regular matrix, with each matrix unit actually corresponding to a functional unit in the FPVA chip.

[0071] like Figure 1 As shown, the FPVA chip is modeled as a regular two-dimensional matrix structure, denoted as an m×n grid, where each matrix cell corresponds to a basic functional module on the chip. Depending on the specific requirements, these cells can be configured to perform different functions, such as storing droplets, performing mixing, separation, or transport operations. Each functional unit typically consists of several microvalves and flow channels, whose opening and closing states are controlled by programming instructions.

[0072] The present invention models the FPVA biochip as follows:

[0073] (1) Functional unit abstraction: Each position in the matrix is ​​either an empty unit (available for use) or an occupied unit (already assigned to a biochemical operation or a fluid is passing through it, and cannot be used by other operations or fluids at the same time).

[0074] (2) Programmable fluid paths: Fluid movement between adjacent functional units is achieved by controlling the opening and closing of microvalves, and movement is limited to the four directly adjacent units: up, down, left, and right. The control granularity is the distance of one unit at a time. By flexibly setting the opening and closing of valves, any legal fluid transport path can be dynamically constructed within the matrix, thereby supporting complex bioassay processes.

[0075] (3) Locating unit coordinates: Use coordinates (x, y) to locate the unit position, where 0≤x≤m-1, 0≤y≤n-1, x represents the unit being located in the xth column from left to right, and y represents the unit being located in the yth row from bottom to top.

[0076] (4) Input / output unit: Set the (0,0) unit as the input unit and the (m,n) unit as the output unit. Fluid can only be input from the (0,0) unit and output from the (m,n) unit.

[0077] (5) Component attribute recording: using (locx) i ,locy i ,lenx i ,leny i The records represent the coordinates and dimensions of component i, respectively, where the component dimensions are represented by the number of units occupied horizontally and vertically.

[0078] The environmental inputs include: the size of the FPVA chip, the dependencies of each operation in the biochemical reaction timing diagram, the types of reagents contained in each operation, and the binding scheduling scheme, including the component number and size bound to each operation and the timing information of each operation.

[0079] This information is used for initializing the FPVA grid model and defining the state space, action space, and reward function. In addition, to support subsequent agent training, the environment also provides the following functions:

[0080] (1) Component layout validity check: Check whether the current layout scheme meets the constraints. The constraints are designed as follows:

[0081] Layout area constraints: The component does not occupy units that are not suitable for component layout, including units occupied by input ports and output ports.

[0082] (locx,locy)≠(0,0) (1)

[0083] locx+lenx<m||locy+leny<n (2)

[0084] Where locx and locy represent the horizontal and vertical coordinates of the component; lenx and leny represent the number of units occupied by the component in the horizontal and vertical directions;

[0085] Concurrent component non-overlap constraint: Concurrent components in the scheduling scheme cannot overlap. That is, for any two concurrent components (locx) in the scheduling scheme... i ,locy i ,lenx i ,leny i ) and (locx j ,locy j ,lenx j ,leny j It needs to satisfy any one of the following four formulas.

[0086] locx i +lenx i ≤locx j (3)

[0087] locy i +leny i ≤locy j (4)

[0088] locx j +lenx j ≤locx i (5)

[0089] locy j +leny j ≤locy i (6)

[0090] Where locx j locy j lenx j leny j These represent the horizontal and vertical coordinates of component j, and the number of units occupied by component j in the horizontal and vertical directions, respectively.

[0091] (2) Component placement rollback mechanism: If the agent's action is illegal (i.e. it does not meet the layout area constraint and the concurrent component non-overlap constraint), the environment will roll back the layout state to the previous step to maintain learning stability.

[0092] (3) Calculation of relevant indicators: Calculate the total unit reuse complexity of the current layout scheme, the estimated bioassay completion time, and the total length of the fluid transport path.

[0093] The total unit reuse complexity P is used to measure the degree of unit reuse complexity of a layout scheme:

[0094]

[0095] Where, p i,j The unit reuse complexity, N, is measured by the unit at coordinate (i,j). a,b Let α represent the number of different types of reagents for the two components numbered a and b. a,b It is a weight parameter, α, which is used when there is no dependency between the blend operations bound to the two components. a,b =1, when there is a dependency between the mixed operations bound to the two components, 0 < α a,b <1, the smaller the distance between the operations bound to the two components in the sequence diagram, the lower α. a,b The smaller it is. i,j,a,b Let c be a binary variable. When both components numbered a and b have occupied the (i,j) unit, c i,j,a,b =1, otherwise c i,j,a,b =0.

[0096] A smaller P-value indicates lower cell reuse complexity of the component, resulting in less contamination in the layout scheme and closer proximity of components containing the same reagent when chip area is limited. Conversely, a larger P-value indicates higher cell reuse complexity of the component, resulting in greater contamination in the layout scheme and greater distance between components containing the same reagent when chip area is limited.

[0097] Figure 2 (a) is a time series diagram of the biochemical detection of a certain biochemical reaction. Figure 2Figure (b) shows an example of a layout scheme and its P-value calculation process. Assume that when two components have a dependency on each other in their hybrid operations and the distance between them in the dependency graph is 1, α = 0.5; when the distance is 2, α = 0.8.

[0098] The total length of the fluid transport path and the completion time of the bioassay were estimated by calculating the Manhattan distance.

[0099] 2. State Space and Action Space Design

[0100] The entire FPVA chip placement process is modeled as a Markov decision process. This process consists of the following elements:

[0101] (1) State space

[0102] The environment state s of the agent at step t t Including the P-value and the component's coordinate information, the entire state space S can be represented as:

[0103] S={s t |s t =[P,x0,y0,x1,y1,x2,y2,……,x l-1 ,y l-1 ], 0≤x i <m,0≤y i <n} (9)

[0104] Where l is the number of components, x i y i The x and y coordinates are respectively the x and y coordinates of component i, where 0 ≤ i < l; the P value of the current layout scheme and the chip layout information are flattened into a one-dimensional array and used as state input to the agent.

[0105] (2) Action space

[0106] The agent's action a t This corresponds to selecting a component and determining its direction of movement. (Reference) Figure 3 Each component can move in one of four directions: up, down, left, or right, or remain stationary, and can move at most one unit at a time.

[0107] Therefore, each component has 5 possible directions of movement, and the motion space of all components is l×5.

[0108] A={a t |0≤a t <l×5} (10)

[0109] The environment updates the layout scheme based on the agent's actions, and then determines whether the current layout scheme is valid. If the current layout scheme is valid (i.e., satisfies all constraints), the layout state is updated, and the unit reuse complexity of the new layout scheme is calculated. Otherwise, the action is invalid, and the layout scheme is rolled back to the state before the action.

[0110] 3. Reward function design:

[0111] The reward function guides the agent to learn and optimize layouts. After each action, the agent calculates the change in total unit reuse complexity based on the current layout and awards a corresponding reward. Specifically, the model first determines the validity of the current layout. If invalid, a negative reward of constant C² is awarded. If valid, the model continues to calculate the change in total unit reuse complexity. If the total unit reuse complexity of the current layout decreases compared to before the action, a positive reward is awarded to encourage the agent to choose this action in the current state. If the total unit reuse complexity increases, a negative reward is awarded to discourage the agent from choosing this action in the current state. If the total unit reuse complexity remains unchanged, a small positive reward is awarded.

[0112] When the chip size is large, the agent tends to distribute the components as widely as possible, which increases the total flow path length and biometric measurement completion time. To address this issue, a constant C1 is set. After an action, if the component moves away from the chip center, the agent receives negative feedback of size C1; if the component moves closer to the center, the agent receives positive feedback of size C1. Therefore, the reward function is set as follows:

[0113]

[0114] Where, r t This indicates that the agent is in state s. t Take action a t The immediate reward received afterward. P pre and P now These represent the cell reuse complexity of the layout schemes before and after the action, respectively. λ is a weighting coefficient used to balance the relative importance of cell reuse complexity and the distance of the component from the center point. β is a ternary variable: when the action moves the component away from the chip center, β = -1; when the action moves the component closer to the chip center, β = 1; when the action keeps the component stationary, β = 0.

[0115] 4. Agent training:

[0116] In the FPVA chip placement problem, component movement operations belong to a discrete action space, meaning each component can move in four directions (up, down, left, and right) or choose to remain stationary. Given that the structure of DDQN can provide a more stable action value assessment, improving the model's convergence and decision quality, and is particularly well-suited to the task's requirement for precise placement, this invention selects DDQN as the agent.

[0117] The training process of DDQN is divided into two parts: initialization and layout simulation for each round. The specific contents of each part are as follows:

[0118] (1) Initialization: Initialize the parameters of the main network and the target network, and construct the FPVA chip mesh model and state space. Set the reuse complexity index and reward function parameters. Then, establish the experience replay pool and set hyperparameters such as learning rate, discount factor, target network update frequency, and exploration rate.

[0119] (2) Layout simulation per round: This part consists of two loops: the outer loop represents the number of rounds, and the inner loop represents the number of steps in each round. In each step, the current state s is first determined... t Using the ε-greedy strategy to select action a t In this approach, an action is randomly selected with a certain probability to increase exploratory power; otherwise, the action with the highest evaluation value in the current main network is selected to enhance utilization. After executing the selected action, the environment returns to a new state s. t+1 and instant rewards t Then the model will (s t ,a t ,r t ,s t+1 The samples are stored in the experience replay pool. Once the number of samples in the experience replay pool reaches a certain size, the model randomly samples a small batch of samples for training. The model calculates the target Q-value for each sample.

[0120]

[0121] Where, r t γ is the immediate reward received at the current moment; γ is the discount factor (typically between 0.9 and 0.99); s t+1 The next state; Q target It is the Q-value estimated by the target network; Q online θ is the Q-value estimated by the current main network; a′ is the possible next action; θ is the main network parameter; θ - These are the target network parameters.

[0122] Finally, the loss function of the main network is minimized based on the calculated target Q value to update the main network parameters:

[0123]

[0124] in, denoted as the target Q-value; Q is the Q-value estimate of the current main network for the current state and action. Let N be the current state and action of the k-th sample; N is the sample size.

[0125] Furthermore, after every C steps, the model copies the parameters of the main network to the target network to ensure the stability of the target value estimation.

[0126] θ - ←θ(14)

[0127] The exploration rate decreases linearly during training, assuming an initial exploration rate of ε. start The minimum exploration rate is ε end The total number of decay steps is M, and the current training step count is s. t The exploration rate ε at step t is:

[0128]

[0129] Or simplified to:

[0130]

[0131] 5. Reasoning Phase and Layout Output:

[0132] After training, the trained final target network is used for layout inference. First, the model initializes the current state s0, then enters a loop to continuously make layout decisions until the conditions for layout completion are met. In each loop, the agent, based on the current state s0... t Choose an optimal action from the already trained target network, that is, determine the layout movement direction of a certain component:

[0133]

[0134] Where θ * This is the final target network obtained during the training phase.

[0135] After performing this action, a new state is obtained, and the action is recorded for later use in building a complete layout scheme. Then, the current state is updated.

[0136] s t ←s t+1 (18)

[0137] The model then proceeds to the next round of state evaluation and action selection. After inference is complete, the model outputs the final layout scheme, along with the total unit reuse complexity, total fluid transport path length, and bioassay completion time.

[0138] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for FPVA biochip layout considering reagent type differences in unit reuse, characterized in that, Specifically, the following steps are included: Modeling the biochip placement task as a Markov decision process: Based on the input FPVA chip size, the dependencies of each operation in the biochemical reaction timing diagram, the types of reagents contained in each operation, and the binding scheduling scheme, the FPVA grid model is initialized and the state space, action space, and reward function are defined. The binding scheduling scheme includes the component number and size bound to each operation and the timing information of each operation. Initializing the FPVA mesh model includes: modeling the FPVA biochip as a two-dimensional matrix structure according to the size of the FPVA chip, in order to obtain the layout information of all components on the FPVA chip based on the two-dimensional matrix structure, including the coordinates and dimensions of the components; Define the state space: The environment state of the agent at each step includes the total unit reuse complexity of the current layout scheme and the layout information of all components; Define the action space: the agent's action corresponds to selecting a component and determining the direction of the component's movement; The environment updates the layout scheme based on the agent's actions and determines whether the current layout scheme is valid. If the current layout scheme is valid, the layout state is updated and the total unit reuse complexity of the new layout scheme is calculated. Otherwise, the action is invalid and the layout scheme is rolled back to the state before the action. Define the reward function: Set the reward function based on the total cell reuse complexity of the current layout scheme, the legality of the current component layout, and the distance of the component from the center point of the FPVA chip; The FPVA biochip layout was completed using a dual deep Q-network based on Markov decision processes.

2. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, The modeling of FPVA biochips is as follows: Based on the size of the FPVA biochip, the FPVA biochip is modeled as an m×n two-dimensional matrix structure. Each position in the two-dimensional matrix structure corresponds to a basic functional unit on the FPVA chip. The position of the basic functional unit is represented by coordinates (x, y), where 0 ≤ x ≤ m-1 and 0 ≤ y ≤ n-1. x represents the unit being located in the x-th column from left to right in the two-dimensional matrix structure, and y represents the unit being located in the y-th row from bottom to top in the two-dimensional matrix structure. Each basic functional unit includes several microvalves and flow channels. The fluid transport path is constructed by setting the opening and closing of microvalves between adjacent basic functional units; The basic functional unit with coordinates (0,0) is set as the input unit, and the basic functional unit with coordinates (m,n) is set as the output unit. The fluid is input from the (0,0) unit and output from the (m,n) unit. The position and size of components in a two-dimensional matrix structure using (locx) i ,locy i ,lenx i ,leny i ) indicates that locx i locy i lenx i leny i These represent the horizontal and vertical coordinates of component i, and the number of units occupied by component i in the horizontal and vertical directions, respectively.

3. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, The calculation of the total unit reuse complexity of the current layout scheme is as follows: Where P represents the total unit reuse complexity of the current layout scheme; p i,j Measure the unit reuse complexity of the cell at coordinate (i,j); N a,b α represents the number of different types of reagents in the two components numbered a and b; a,b α is a weight parameter; when the blend operations bound to the two components have no dependency, α a,b =1, when there is a dependency between the mixed operations bound to the two components, 0 < α a,b <1, and the smaller the distance between the operations bound to the two components in the sequence diagram, the lower α is. a,b The smaller the value, the better; c i,j,a,b Let c be a binary variable. When both components numbered a and b have occupied the (i,j) unit, c i,j,a,b =1, otherwise c i,j,a,b =0.

4. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, The state space S is represented as: S={s t |s t =[P,x0,y0,x1,y1,x2,y2,…,x i ,y i ,…,x l-1 ,y l-1 ],0≤x i <m,0≤y i <n} Among them, s t Let x represent the environment state of the agent at step t, P represent the total unit reuse complexity of the current layout scheme, and x represent the environment state of the agent at step t. i y i Let x and y be the x and y coordinates of component i, respectively, where 0 ≤ i < l, and l is the number of components.

5. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, Action space A is represented as: A={a t |0≤a t <l×5} Among them, a t This represents the component selected by the agent at step t and its direction of movement; each component can move or remain stationary in one of the four directions: up, down, left, or right, and can move at most one unit at a time.

6. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 2, characterized in that, The current layout scheme is legal if it meets all constraints, including layout area restrictions and concurrent component non-overlapping constraints. The layout area constraint is that components cannot occupy units that are not suitable for component layout, as expressed as: (locx,locy)≠(0,0) locx + lenx < m || locy + leny < n Where locx and locy represent the horizontal and vertical coordinates of the component; lenx and leny represent the number of units occupied by the component in the horizontal and vertical directions; The constraint that concurrent components do not overlap means that concurrent components in the scheduling scheme cannot overlap with each other.

7. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 6, characterized in that, The non-overlapping constraint of concurrent components specifically means that for any two concurrent components (locx) in the scheduling scheme... i ,locy i ,lenx i ,leny i ) and (locx j ,locy j ,lenx j ,leny j It needs to satisfy any one of the following four formulas: locx i +lenx i ≤locx j locy i +leny i ≤locy j locx j +lenx j ≤locx i locy j +leny j ≤locy i Where locx j locy j lenx j leny j These represent the horizontal and vertical coordinates of component j, and the number of units occupied by component j in the horizontal and vertical directions, respectively.

8. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, The reward function is specifically set as follows: Where, r t This indicates that the agent is in state s. t Take action a t The immediate reward obtained afterward; P pre and P now λ represents the total unit reuse complexity corresponding to the layout schemes before and after the action; λ is a weighting coefficient used to balance the relative importance of unit reuse complexity and component distance from the center point; β is a ternary variable, which is the value of the component's position when action a... t When the component is moved away from the center of the chip, β = -1; when action a t Move the component closer to the center of the chip, β = 1; when action a t Keep the components stationary, β = 0; C2 and C1 are both constants.

9. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 1, characterized in that, The FPVA biochip layout based on Markov decision process and dual deep Q-network includes the following steps: S1. Initialization: Initialize the main network parameters and target network parameters, and clear the experience replay pool; S2, Environment Interaction: In the current state s t Below, the main network uses the ε-greedy policy to select action a. t The exploration rate decreases gradually in a linear decay manner; after the selected action is performed, the environment returns to a new state s. t+1 and instant rewards t , will (s t ,a t ,r t ,s t+1 Stored in the experience replay pool; S3. Sampling and Training: Once the number of samples in the experience replay pool reaches a certain size, a small batch of samples is randomly sampled for training, and the target Q-value for each sample is calculated. Where, r t It is the immediate reward obtained at the current moment; γ is the discount factor; s t+1 The next state; Q target It is the Q-value estimated by the target network; Q online θ is the Q-value estimated by the current main network; a′ is the possible next action; θ is the main network parameter. - These are the target network parameters; The main network parameters are updated by minimizing the loss function of the main network based on the calculated target Q value: in, Let Q be the target Q-value for the k-th sample; Q is the Q-value estimate of the current main network for the current state and action. Let N represent the current state and action of the k-th sample; N is the sample size. Every C steps, the main network parameters are copied to the target network parameters to ensure the stability of the target value estimation. S4. Inference Phase and Layout Output: After training is completed, the trained final target network is used for layout inference; the current state is initialized, and then a loop is entered to continuously make layout decisions until the conditions for layout completion are met; In each loop, the agent selects an optimal action based on the current state and the trained target network, that is, determines the layout movement direction of a certain component; after executing the optimal action, a new state is obtained, and the optimal action is recorded for subsequent construction of a complete layout scheme, and then the current state is updated; then the next round of state evaluation and action selection begins. After reasoning is complete, the final layout scheme, the total unit reuse complexity of the final layout scheme, the total length of the fluid transport path, and the completion time of the bioassay are output.

10. The FPVA biochip layout method considering reagent type differences in unit reuse according to claim 9, characterized in that, The total length of the fluid transport path and the completion time of the bioassay were estimated by calculating the Manhattan distance.

Images