Pollution sensing physical design method of microfluidic FPVA biochip

Abstract

The invention discloses a pollution sensing physical design method of a microfluidic FPVA biochip, and relates to the technical field of biochips. The method comprises the following steps: respectively defining framework descriptions of a pollution sensing layout model and a pollution sensing wiring model based on a microfluidic FPVA biochip; obtaining a trained layout model and a trained wiring model; by synchronizing and coordinating the trained layout model and wiring model, the trained layout model and wiring model can globally perceive changes of all resources on the FPVA biochip when executing respective tasks, and a physical design scheme of pollution perception is obtained. By designing a new physical design framework based on deep reinforcement learning, the overall performance of the chip is remarkably improved, the total cleaning cost of bioassay is reduced to the maximum extent, and the overall length of a fluid transportation path is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of biochips, and in particular to a contamination perception physical design method of a microfluidic FPVA biochip. Background Art

[0002] Microfluidic biochips are able to automate various biochemical experiments, such as immunoassays and DNA analysis, by precisely manipulating microscale fluid samples / reagents. These devices, often referred to as "labs on a chip," have significant advantages such as high precision, throughput, and reliability. As such, they are expected to become an alternative to traditional manual laboratories.

[0003] Combining the advantages of flow microfluidic biochips and digital microfluidic biochips, a new generation of fully programmable valve array (FPVA) biochips was proposed, realizing the "digital" operation of continuous fluids. Figure 1 The general logic structure of the FPVA biochip is shown, where valves are regularly arranged in the fluid channels along the horizontal and vertical directions on the chip. These valves are connected to a multiplexing control system based on Boolean logic at the periphery of the biochip to switch between open and closed states. By opening any two valves in the unit and closing the rest of the valves, the fluid can be flexibly directed to flow in any direction, such as Figure 1 shown.

[0004] Despite significant technological advances, these biochips still face the challenge of fluid cross-contamination, which can affect the accuracy and reliability of experimental results. To alleviate this problem, residues need to be dissolved in a buffer and rinsed out. This washing operation effectively removes residues from the channel surface, allowing the contaminated flow channel to be quickly reused for other fluids. However, the washing time is mainly determined by the dissolution time of the residues; if the required washing time is long, it will delay the completion time of the entire bioassay. Therefore, effectively allocating on-chip channel resources during the physical design process to minimize the impact of residues that require a long time to dissolve, thereby reducing the cost of washing, is a key issue that needs to be addressed.

[0005] Currently, only a few automated methods have been proposed to address the cross-contamination problem of FPVA biochips. The prior art proposes an efficient cleaning algorithm that groups contaminated areas together for uniform cleaning. At the same time, an efficient flow layer architecture synthesis has been developed to minimize the total number of contamination points, thereby reducing cleaning overhead and bioassay completion time. In addition, a cleaning optimization method that considers buffer volume consumption has been developed to ensure that all contaminants are removed in the shortest cleaning time. However, they do not systematically consider the actual cost of contamination removal for each contamination point generated during the physical design process. Therefore, although efforts have been made to reduce the total number of contamination points, the serious problem of long cleaning time caused by the presence of difficult-to-clean contamination points has not been solved. Summary of the invention

[0006] In view of the deficiencies in the above-mentioned background technologies, the present invention is aimed at the problem of contamination-aware physical design of FPVA biochips. The present invention provides a contamination-aware physical design method for microfluidic FPVA biochips, which is mainly based on a deep reinforcement learning framework to effectively improve the chip design quality. The method involves three key technologies, including a contamination-aware layout algorithm based on deep reinforcement learning, a contamination-aware wiring algorithm based on deep reinforcement learning, and a contamination-aware physical design algorithm based on collaborative design, with the goal of minimizing the total length of the fluid transport path and the total cleaning cost of the bioassay.

[0007] The purpose of the present invention is to provide a contamination sensing physical design method for a microfluidic FPVA biochip, comprising:

[0008] Based on the microfluidic FPVA biochip, the framework descriptions of the pollution-aware layout model and the pollution-aware wiring model are defined respectively; wherein the framework descriptions include action space, state space, mask generation and reward function;

[0009] According to the framework description of the pollution-aware layout model and the framework description of the pollution-aware routing model, the pollution-aware layout model and the pollution-aware routing model are trained based on a deep reinforcement learning algorithm to obtain a trained layout model and a trained routing model respectively.

[0010] By synchronizing and coordinating the trained layout model and routing model, the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

[0011] Preferably, the framework description of the pollution-aware layout model includes a first action space, a first state space, a first mask generation, and a first reward function;

[0012] Wherein, the first action space is composed of all layout positions;

[0013] The first state space is composed of four core parts, namely, the information of the components of the current layout, the information of the components, the dissolution time of the fluid, and the cleaning cost matrix;

[0014] The first mask generation is to mask the action when the component exceeds the boundary of the biochip or there is overlap between units of different types of components.

[0015] Preferably, the first reward function is:

[0016]

[0017] Where α, β, γ and ψ are hyperparameters; F o It is a binary flag, a value of "1" indicates that the source component and the target component have overlapping units, and a value of "0" indicates that there is no overlap; S r Represents the minimum rectangular area that contains the laid out components; C w is the cleaning cost generated after executing the layout action; △ represents the value difference between time step t and time step t-1.

[0018] Preferably, the framework description of the pollution-aware wiring model includes a second action space, a second state space, a second mask generation, and a second reward function;

[0019] Among them, the second action space is composed of all flow directions of the fluid on the FPVA biochip;

[0020] The second state space is composed of three core parts, namely, the information of the fluid being routed, the target location of the fluid, and the cleaning cost of each unit on the chip;

[0021] The second mask generation is to cover up the action when the fluid exceeds the boundary of the biochip or the fluid flows back along its historical path.

[0022] Preferably, the second reward function is:

[0023]

[0024] Where ε, θ, μ, τ are hyperparameters; Represents the total length of the path planned for the transportation task; represents the total cleaning cost; -ε represents a negative reward; θ represents the endpoint reward; (fx, fy) represents the position reached by the fluid at the current time step; Indicates the target position of the fluid.

[0025] Preferably, the deep reinforcement learning algorithm adopts a proximal strategy optimization algorithm with an Actor-Critic architecture.

[0026] Preferably, obtaining a pollution-aware physical design solution includes:

[0027] When the layout subtask is completed, the resource update status on the chip is input into the routing environment. The routing agent perceives the changes in the layout scheme, cell occupancy and pollution status, and performs the routing task based on the updated information. Similarly, after completing the routing subtask, the resource update status is fed back to the layout environment.

[0028] Preferably, the pollution-aware layout model is used to determine the on-chip locations of all components; and the pollution-aware routing model is used to find a set of flow paths for all transport tasks.

[0029] The present invention also provides a contamination sensing physical design system for a microfluidic FPVA biochip, comprising:

[0030] A definition module is used to define the framework description of the pollution-aware layout model and the pollution-aware wiring model based on the microfluidic FPVA biochip, wherein the framework description includes an action space, a state space, a mask generation, and a reward function;

[0031] A training module, used for training the pollution-aware layout model and the pollution-aware wiring model based on a deep reinforcement learning algorithm according to the framework description of the pollution-aware layout model and the framework description of the pollution-aware wiring model, respectively, to obtain a trained layout model and a trained wiring model;

[0032] The collaborative design module is used to synchronize and coordinate the trained layout model and routing model so that the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention provides a contamination-aware physical design method for a microfluidic FPVA biochip. Aiming at the contamination-aware physical design problem of FPVA biochips, the present invention proposes a framework based on deep reinforcement learning to effectively improve the chip design quality. The method involves three key technologies, including a contamination-aware layout algorithm based on deep reinforcement learning, a contamination-aware wiring algorithm based on deep reinforcement learning, and a contamination-aware physical design algorithm based on collaborative design, with the goal of minimizing the total length of the fluid transport path and the total cleaning cost of the bioassay. Specifically, the following steps are included:

[0035] The pollution-aware layout algorithm based on deep reinforcement learning in the present invention: a pollution-aware layout algorithm based on deep reinforcement learning is proposed, a reinforcement learning environment for multiple layout tasks is designed, and a neural network model that can efficiently perform layout tasks is trained using the PPO algorithm. The optimization goals are to minimize the total cleaning cost, reduce fluid wiring tasks, and reduce the area of ​​the approximate shortest path region, thereby generating a high-quality pollution-aware layout solution.

[0036] The pollution-aware wiring algorithm based on deep reinforcement learning in the present invention: a pollution-aware wiring algorithm based on deep reinforcement learning is proposed, a reinforcement learning environment for multiple wiring tasks is designed, and a neural network model that can efficiently perform wiring tasks is trained using the PPO algorithm, with minimizing the total cleaning cost and reducing the total path length as the optimization goals, to generate a high-quality pollution-aware wiring solution.

[0037] The pollution-aware physical design algorithm based on collaborative design in the present invention: by synchronizing and coordinating two convergent layout neural network models and wiring neural network models, the two models can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, so as to make the most favorable decision for the entire physical design, thereby improving the quality of physical design and reducing the total cleaning cost and total path length of the entire physical design.

[0038] The present invention provides a contamination-aware physical design method for FPVA biochips, which comprehensively considers layout and wiring issues, and significantly improves the overall performance of the chip by designing a new physical design framework based on deep reinforcement learning, thereby minimizing the total cleaning cost of bioassays and optimizing the total length of the fluid transport path. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is the logical structure of the FPVA biochip.

[0040] Figure 2 The timing diagram of the biometrics (a) and the binding and scheduling scheme (b) corresponding to the biometrics shown in (a);

[0041] Figure 3 for Figure 2 Two physical designs of the bioassay shown in (a);

[0042] Figure 4 The proposed DRL-based collaborative physical design process. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand and implement the technical solution of the present invention, the present invention is further described below in conjunction with specific embodiments and drawings, but the embodiments are not intended to limit the present invention.

[0044] The present invention is mainly aimed at reducing the number of contamination points, which seems to intuitively shorten the total cleaning time. In practice, even a few contamination points may require a long time to clean. Each contaminated channel can be cleaned with a buffer before being reused by any functional fluid. To ensure complete removal, the contaminant must be completely dissolved in the buffer and discharged from the biochip. The dissolution time of the contaminant is mainly determined by its diffusion coefficient. The lower the diffusion coefficient, the longer the dissolution time. Since the dissolution time is the core part of the total cleaning time, ignoring the influence of the diffusion coefficient of the fluid during physical design may extend the time of multiplexing the flow channel, which will lead to a longer completion time of the bioassay. Suboptimal physical design may lead to increased cleaning costs, including more contamination points. More importantly, the total dissolution time of these contamination points may be greatly extended. Therefore, the present invention proposes a contamination-aware physical design method under FPVA biochip, which takes layout and wiring issues into consideration, and significantly improves the overall performance of the chip by designing a new physical design framework based on deep reinforcement learning, minimizes the total cleaning cost of the bioassay, and optimizes the total length of the fluid transport path.

[0045] In order to achieve the above object, the present invention provides a contamination sensing physical design method for a microfluidic FPVA biochip, comprising:

[0046] S1. Based on the microfluidic FPVA biochip, a framework description of a pollution-aware layout model and a pollution-aware wiring model is defined respectively; wherein the framework description includes an action space, a state space, a mask generation, and a reward function;

[0047] The framework description of the pollution-aware layout model includes a first action space, a first state space, a first mask generation, and a first reward function;

[0048] Wherein, the first action space is composed of all layout positions;

[0049] The first state space is composed of four core parts, namely, the information of the components of the current layout, the information of the components, the dissolution time of the fluid, and the cleaning cost matrix;

[0050] The first mask generation is to mask the action when the component exceeds the boundary of the biochip or there is overlap between units of different types of components.

[0051] The first reward function is:

[0052]

[0053] Where α, β, γ and ψ are hyperparameters; F oIt is a binary flag, a value of "1" indicates that the source component and the target component have overlapping units, and a value of "0" indicates that there is no overlap; S r Represents the minimum rectangular area that contains the laid out components; C w is the cleaning cost generated after executing the layout action; △ represents the value difference between time step t and time step t-1.

[0054] The framework description of the pollution-aware wiring model includes a second action space, a second state space, a second mask generation, and a second reward function;

[0055] Among them, the second action space is composed of all flow directions of the fluid on the FPVA biochip;

[0056] The second state space is composed of three core parts, namely, the information of the fluid being routed, the target location of the fluid, and the cleaning cost of each unit on the chip;

[0057] The second mask generation is to cover up the action when the fluid exceeds the boundary of the biochip or the fluid flows back along its historical path.

[0058] The second reward function is:

[0059]

[0060] Where ε, θ, μ, τ are hyperparameters; Represents the total length of the path planned for the transportation task; represents the total cleaning cost; -ε represents a negative reward; θ represents the endpoint reward; (fx, fy) represents the position reached by the fluid at the current time step; Indicates the target position of the fluid.

[0061] S2. According to the framework description of the pollution-aware layout model and the framework description of the pollution-aware routing model, the pollution-aware layout model and the pollution-aware routing model are trained based on a deep reinforcement learning algorithm to obtain a trained layout model and a trained routing model respectively.

[0062] The deep reinforcement learning algorithm adopts a proximal policy optimization algorithm with an Actor-Critic architecture.

[0063] The pollution-aware placement model is used to determine the on-chip locations of all components; the pollution-aware routing model is used to find a set of flow paths for all transport tasks.

[0064] S3. By synchronizing and coordinating the trained layout model and routing model, the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

[0065] When acquiring pollution-aware physical design solutions, include:

[0066] When the layout subtask is completed, the resource update status on the chip is input into the routing environment. The routing agent perceives the changes in the layout scheme, cell occupancy and pollution status, and performs the routing task based on the updated information. Similarly, after completing the routing subtask, the resource update status is fed back to the layout environment.

[0067] In order to illustrate the contamination perception physical design method of a microfluidic FPVA biochip provided by the present invention, it is described in conjunction with the accompanying drawings.

[0068] In one embodiment, a contamination-aware physical design method for a microfluidic FPVA biochip is provided, based on an FPVA simulation environment: a FPVA biochip of size W×H is considered as a collection of multifunctional units with W units in each row and H units in each column. Each unit consists of a flow channel and four valves that control the flow direction of the fluid, such as Figure 1 shown.

[0069] The component is modeled as a four-tuple δ = (dx, dy, dw, dh), where (dx, dy) is the coordinate of the upper left corner of the component. (dw, dh) is the width and height of the component.

[0070] The fluid is modeled as a quaternion ζ = (fx, fy, fp, fd), where (fx, fy) is the position reached by the fluid at the current time step, fp is the path of fluid transport, and fd is the dissolution time of the fluid.

[0071] Let (x, y) be the coordinates of the FPVA unit; C x,y ∈R ≥0 is its contamination degree, where 0 means the unit is completely clean and greater than 0 means the unit is contaminated.

[0072] It is worth noting that the degree of contamination of the unit is equal to the dissolution time of the residual liquid. Represents the cleaning cost matrix of the FPVA biochip.

[0073] Specific pollution-aware physical design methods include:

[0074] (1) Pollution-aware layout algorithm based on deep reinforcement learning:

[0075] This application provides an efficient DRL-based layout to determine the on-chip locations of all components. The layout design is described as follows:

[0076] Action space: Since the action at time step t is to place components, the action space consists of all possible layout positions and is defined as:

[0077] A 1 ={a t |a t ∈{P 1 ,P 2 ,…,P w×h},P i =(px i ,py i )|px i ∈[1,W],py i ∈[1,H]}(1)

[0078] Where (px i ,py i ) are the coordinates of the upper left corner cell of the on-chip cell occupied by the component.

[0079] State space: In order for the agent to make effective layout decisions by perceiving valuable information in the environment, the state space should consist of four core parts, namely the components m(o i ) Component m(o k )(e k,i ∈E) information Fluid (o i ) dissolution time and the cleaning cost matrix C t . Therefore, the entire state space is expressed as:

[0080]

[0081] Mask Generation: The agent must mask actions when it encounters the following situations: a) a component is outside the biochip boundary (i.e., dx+dw-1>W∨dy-dh<0); b) there is overlap between units of different types of components (e.g., ). Among them, U m and U q are the sets of cells occupied by m and q respectively. m and t q are the functional types of m and q respectively. Therefore, the mask matrix M 1 ∈{0,1} W×H It can be defined as:

[0082]

[0083] Reward function: The primary goal of placement is to properly arrange the placement of each pair of source and target components. In this case, the performance metrics include the on-chip area of ​​the minimum rectangle containing all components and the cleaning cost caused by cell reuse.

[0084] Accordingly, in state s t Rewards obtained after executing the layout action It can be calculated as:

[0085]

[0086] where α, β, γ and ψ are hyperparameters. The variable F o It is a binary flag. A value of "1" indicates that the source component and the target component have overlapping units, and a value of "0" indicates that there is no overlap. When the source component and the target component overlap, there is no need to set up a flow channel for the fluid transport task between them, thereby reducing the use of channel resources. r S represents the area of ​​the approximate shortest path region

[11] , defined as the area of ​​the smallest rectangle containing the laid out components. r The smaller it is, the more conducive it is to generate a routing solution with a smaller total path length. w is the cleaning cost after executing the layout action. △ represents the value difference between time step t and time step t-1.

[0087] (2) Pollution-aware wiring algorithm based on deep reinforcement learning:

[0088] This application provides an efficient DRL-based router to find a set of flow paths for all transport tasks. The framework of the router is described as follows:

[0089] Action space: Since the action at time step t is to realize the flow channel wiring, the action space consists of all possible flow directions of the fluid on the FPVA biochip and is defined as:

[0090] A 2 ={a t |a t ∈{D 1 ,D 2 ,D 3 ,D 4},D i =i}(7)

[0091] Among them, D 1 , D 2 , D 3 , D 4 They represent the four flow directions of the flow channel: up, down, left and right.

[0092] State space: In order to enable the agent to perceive valuable information from the environment and make efficient routing decisions, the state space at time step t should consist of three core parts, namely, the fluid being routed out (o i ) Fluid out(o i ) And the cleaning cost of each unit on the chip. Therefore, the entire state space is expressed as:

[0093]

[0094] Mask generation: After executing an action, the agent must mask the action if it encounters the following situations: a) The fluid exceeds the boundaries of the biochip (i.e. [1,H]); b) the fluid along its historical path (i.e. (fx t ,fy t )∈fp t-1 ) backflow. Therefore, the mask matrix M 2 ∈{0,1} W×H It can be defined as:

[0095]

[0096] Reward function: The main goal of routing is to properly allocate flow channels for each transport task. In this case, the performance indicators include the total length of the transport path and the cleaning cost caused by the reuse unit. t Rewards received after performing a wiring operation It can be calculated as:

[0097]

[0098] Where ε, θ, μ, τ are hyperparameters. In order to converge the exploration process of the routing agent and optimize the path length, the agent is given a negative reward ε as a penalty for each step. The end point reward θ helps to converge faster by linking reaching the target location with additional rewards. After completing each routing task, the total length of the path planned for this transportation task is calculated and total cleaning cost Therefore, negative rewards are needed to encourage the agent to optimize these two objectives.

[0099] The placer and router are trained respectively by using the pollution-aware layout algorithm based on deep reinforcement learning and the pollution-aware routing algorithm based on deep reinforcement learning. The training process is described in Algorithm 1: Training process of deep reinforcement learning agent

[0100]

[0101] Among them, the relevant configuration settings during the training process are as follows:

[0102] Training configuration: In order to increase sample diversity and enhance the generalization ability of the agent, random sampling is performed based on the {washing time}-{diffusion coefficient} graph [7] to determine the dissolution time of various fluids, which are then used as input to the placement and routing environment. When the environment is initialized, a three-dimensional matrix is ​​used to represent the state of component layout or fluid routing. Each dimension of the matrix records the properties, location, and potential cleaning cost of the component or fluid. Convolutional neural networks (CNNs) extract features from these matrices, enhancing the ability of the DRL model to handle complex states.

[0103] PPO Architecture: During the training phase, we use the Proximal Policy Optimization (PPO) algorithm with an Actor-Critic architecture

[12] ,

[13] . The PPO architecture first extracts high-level features of the input state through a four-layer CNN. Then, the extracted features are unfolded and input into two multi-layer perceptron (MLP) modules: actor and critic. The actor network predicts the probability distribution of actions based on the current state, while the critic network evaluates the value of the current state. ReLU is used as the activation function between each hidden layer. The actor network is optimized using the PPO algorithm, while the critic network helps the actor to be trained by providing state value estimates. The overall architecture combines convolutional feature extraction with the actor-critic structure to achieve policy learning and state evaluation.

[0104] Training process: The placement and routing agents use a unified training framework. Algorithm 1 aims to train a DRL agent that can effectively perform placement / routing tasks. The input of the algorithm includes a set of tasks and an FPVA simulation environment. The final output is a trained and converged DRL agent. The algorithm first initializes the policy network parameters θ 0 , value function parameter φ 0 and a set of key hyperparameters. Then, the algorithm performs n iterations, each of which aims to collect state-action samples from the environment for subsequent training. At the beginning of each iteration, all parameters of the environment are reset. During the sampling phase, the agent and the environment interact T s Step (Lines 5-9 in Algorithm 1). At each step, the agent passes through the current policy network Select an action. After the action is executed, the system generates rewards based on the current state and action. and transition to the next state s t+1 This process is repeated for T s Step, all state-action samples Store in a buffer for future training use.

[0105] Based on the data sampled from the current policy, calculate the advantage function for each sample To estimate whether a particular action performs better or worse than the expected action under the policy. The advantage function can be calculated using methods such as time difference or generalized advantage estimation. At the same time, the discounted return Based on instant rewards and discount factor λ, which is used to evaluate the total reward that the agent will receive in the future starting from the current state. Finally, the collected samples are divided into smaller batches, and each small batch of samples is used for multiple updates (lines 12-15 in Algorithm 1). In each update, the policy θ is updated by maximizing the PPO objective function k+1 , which is calculated as follows:

[0106]

[0107] Among them, argmax represents the variable that maximizes the function value. Represents the probability ratio between the new and old strategies. In the PPO algorithm, r c It is a hyperparameter of the policy clipping range, which is used to limit the degree of change between the new and old policies during the update. Its role is to prevent policy collapse or overfitting during training, thereby improving the stability and performance of the training process.

[0108] In addition, the parameters of the value network are updated by minimizing the mean squared error of the value function, thereby improving its ability to estimate future rewards. Therefore, the value network parameter φk+1 can be calculated as follows:

[0109]

[0110] Among them, argmin represents the variable that minimizes the function value, V φ (s t ) is the value function of the current state. This training process continues until the agent converges, enabling it to make effective decisions based on the current state. (3) Pollution-aware physical design algorithm based on collaborative design:

[0111] See also Figure 4 As shown in Figure 1, the DRL-based placer and router globally perceive the changes in on-chip resources through the collaborative design module, thereby guiding the agent to make efficient decisions. When the placement subtask is completed, the resource updates on the chip are input into the routing environment. The routing agent perceives the changes in the placement scheme, cell occupancy, and pollution, and performs the routing task based on the updated information. Similarly, after completing the routing subtask, the resource updates are fed back to the placement environment.

[0112] It should be noted that the DRL placer and DRL router proposed in the present invention can only process their own placement / routing tasks respectively, so the individual modules cannot consider resource usage globally and are prone to fall into local optimal solutions. The collaborative design module adopted can synchronize the chip resources occupied by the placer and router in real time, and reasonably model the bioassay as their respective problem models, and then schedule the placer and router according to the progress of the physical design, so that the two stages of placement and routing are combined into one stage, allowing the placer and router to efficiently cooperate to complete the physical design task.

[0113] Among them, the pollution-aware physical design algorithm based on collaborative design, the collaborative design process is shown in Algorithm 2.

[0114] Algorithm 2: Pollution-aware physical design algorithm based on collaborative design

[0115]

[0116]

[0117] It should be noted that a low-cost physical design solution that is pollution-aware through cleaning cost is finally output, including a pollution-aware layout solution and a pollution-aware wiring solution. The present invention mainly performs pollution awareness through cleaning cost. Pollution awareness is a method of considering the difficulties of the subsequent cleaning stage in advance during the physical design stage, thereby performing advance optimization, which can reduce the completion time of the bioassay of the biochip.

[0118] Five real bioassays and five synthetic benchmarks were conducted on FPVA biochips of different sizes to verify the effectiveness of the proposed cleaning cost-based contamination-aware technology and the collaborative-design-based contamination-aware physical design algorithm. In addition, the proposed method was compared with the existing advanced heuristic FPVA physical design to verify its high efficiency.

[0119] The information of the 10 groups of standard test cases used in the experiment is shown in Table 1. Each group of test cases includes parameters such as the number of fluids, the number of mixing operations, and the array scale. The experiments were measured at FPVA scales of 8×8, 10×10, 12×12, and 13×13, respectively.

[0120] Table 1 10 groups of standard test cases

[0121]

[0122]

[0123] Since there is no research on the physical design of FPVA biochips considering the cleaning cost, we verified the performance of several optimization strategies in the proposed design flow through two sets of experiments. First, we modified the proposed method by removing the consideration of the cleaning cost of each contamination point and named it WCAS. In the second set of experiments, we implemented another DRL-based algorithm NCDS to perform placement and routing design without using the contamination-aware physical design algorithm based on co-design. We compared these methods by running them 10 times on each benchmark and taking the average as the final result. Tables 2 and 3 show the corresponding comparison results, respectively, where the "Imp(%)" column provides the improvement of the proposed method over the other two methods. Compared with WCAS, the total path length of fluid transport in this method is shortened by 6.8% on average. Specifically, the total path length of IVD2 and Synthetic4 is slightly increased, which is acceptable for most biochemical experiments. In addition, this method can reduce the cleaning cost by 2.9% to 24.7%, with an average reduction rate of 10.4%. Similarly, the proposed method achieves an average reduction of 7.9% and 10.2% in total path length and cleaning cost compared to NCDS. These results demonstrate that our proposed DRL-based design framework performs well in path planning and contamination control, providing a new solution for the physical design of FPVA biochips.

[0124] Table 2 Comparison results before and after the introduction of the cleaning cost strategy

[0125]

[0126]

[0127] Table 3 Comparison results before and after the introduction of collaborative design strategy

[0128]

[0129] In order to verify the effectiveness of the proposed DRL-based physical design method, we compared it with the state-of-the-art heuristic physical design method proposed in the literature: Liu G, Zhu Y, Guo W, et al. Fault-tolerance-oriented physical design for fully programmable valve array biochips [C] / / 2023 60th ACM / IEEE Design Automation Conference (DAC). IEEE, 2023: 1-6. This method uses a PSO-based layout algorithm and an A*-based routing algorithm. We modified this method by removing the fault tolerance consideration and adding the cleaning cost perception, and named it CAPD. We compared the two methods, running them 10 times on each benchmark and taking the average as the final result. The comparison results are shown in Table 4. As can be seen from Table 4, the DRL-based physical design method reduces the total path length of fluid transport by 0.7% to 43.4%, with an average reduction of 24.5%. In addition, the total cleaning cost was reduced by 37.5% on average in all benchmarks, which significantly shortened the total dissolution time of the buffer in the actual fluid scheduling process. These results demonstrate the effectiveness of the physical design approach.

[0130] Table 4 Comparison results with the state-of-the-art heuristic physical design methods

[0131]

[0132] Table 5 Figure 2 Diffusion coefficients and dissolution times for each fluid in the bioassay shown in (a)

[0133]

[0134] by Figure 2 The bioassay described in is used as an example. The diffusion coefficients of the input / intermediate fluids and their corresponding dissolution times are given in Table 1. Figure 3 It indicated Figure 2 Two physical designs of the bioassay are shown, with darker contaminated cells requiring longer lysis times. Figure 3 (a) is a non-contamination-aware physical design, where the number of cross-contamination points is 14 and the total dissolution time of all cross-contamination points is 89 s. 2 The fluid (i.e. in(o 2 )) The residue left behind takes 6 seconds to dissolve, while in(o 2 ) The flow channel used needs to be entered into m 3 The fluid (i.e. in(o 3)) is used, so the delay time is very long. In addition, m 2 The four units are m 6 Multiplexing, m 2 The output fluid (i.e. the outflow (o 2 )) The residue left behind takes 10 seconds to dissolve. This results in 6 The input fluid is delivered to m 6 Previously there were long delays, which ultimately extended the overall execution time of the bioassay. Figure 3 The solution shown in (b) takes into account both the number of cross-contamination points and the corresponding dissolution time, where the number of cross-contamination points is 8 and the total dissolution time of all cross-contamination points is 27 s. 1 and m 3 The residues left by the input fluids need to be dissolved in 3 seconds, and the flow channels used by these input fluids need to be m 2 and m 4 The input fluid is used without causing long delays.

[0135] The present invention provides a contamination sensing physical design system for a microfluidic FPVA biochip, comprising:

[0136] A definition module is used to define the framework description of the pollution-aware layout model and the pollution-aware wiring model based on the microfluidic FPVA biochip, wherein the framework description includes an action space, a state space, a mask generation, and a reward function;

[0137] A training module, used for training the pollution-aware layout model and the pollution-aware wiring model based on a deep reinforcement learning algorithm according to the framework description of the pollution-aware layout model and the framework description of the pollution-aware wiring model, respectively, to obtain a trained layout model and a trained wiring model;

[0138] The collaborative design module is used to synchronize and coordinate the trained layout model and routing model so that the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

Claims

1. A contamination sensing physical design method for a microfluidic FPVA biochip, characterized in that: include: Based on the microfluidic FPVA biochip, the framework descriptions of the pollution-aware layout model and the pollution-aware wiring model are defined respectively; wherein the framework descriptions include action space, state space, mask generation and reward function; According to the framework description of the pollution-aware layout model and the framework description of the pollution-aware routing model, the pollution-aware layout model and the pollution-aware routing model are trained based on a deep reinforcement learning algorithm to obtain a trained layout model and a trained routing model respectively. By synchronizing and coordinating the trained layout model and routing model, the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

2. The contamination sensing physical design method of the microfluidic FPVA biochip according to claim 1, characterized in that: The framework description of the pollution-aware layout model includes a first action space, a first state space, a first mask generation, and a first reward function; Wherein, the first action space is composed of all layout positions; The first state space is composed of four core parts, namely, the information of the components of the current layout, the information of the components, the dissolution time of the fluid, and the cleaning cost matrix; The first mask generation is to mask the action when the component exceeds the boundary of the biochip or there is overlap between units of different types of components.

3. The pollution sensing physical design method of the microfluidic FPVA biochip according to claim 2 is characterized in that: The first reward function is: Where α, β, γ and ψ are hyperparameters; F o It is a binary flag, a value of "1" indicates that the source component and the target component have overlapping units, and a value of "0" indicates that there is no overlap; S r Represents the minimum rectangular area that contains the laid out components; C w is the cleaning cost generated after executing the layout action; △ represents the value difference between time step t and time step t-1.

4. The contamination sensing physical design method of the microfluidic FPVA biochip according to claim 1, characterized in that: The framework description of the pollution-aware wiring model includes a second action space, a second state space, a second mask generation, and a second reward function; Among them, the second action space is composed of all flow directions of the fluid on the FPVA biochip; The second state space is composed of three core parts, namely, the information of the fluid being routed, the target location of the fluid, and the cleaning cost of each unit on the chip; The second mask generation is to cover up the action when the fluid exceeds the boundary of the biochip or the fluid flows back along its historical path.

5. The contamination sensing physical design method of the microfluidic FPVA biochip according to claim 4, characterized in that: The second reward function is: Where ε, θ, μ, τ are hyperparameters; Represents the total length of the path planned for the transportation task; represents the total cleaning cost; -ε represents a negative reward; θ represents the endpoint reward; (fx, fy) represents the position reached by the fluid at the current time step; Indicates the target position of the fluid.

6. The contamination sensing physical design method of the microfluidic FPVA biochip according to claim 1, characterized in that: The deep reinforcement learning algorithm adopts a proximal policy optimization algorithm with an Actor-Critic architecture.

7. The contamination sensing physical design method of the microfluidic FPVA biochip according to claim 1, characterized in that: When acquiring pollution-aware physical design solutions, include: When the layout subtask is completed, the resource update status on the chip is input into the routing environment. The routing agent perceives the changes in the layout scheme, cell occupancy and pollution status, and performs the routing task based on the updated information. Similarly, after completing the routing subtask, the resource update status is fed back to the layout environment.

8. The contamination-aware physical design method of a microfluidic FPVA biochip according to claim 1, characterized in that: The pollution-aware placement model is used to determine the on-chip locations of all components; the pollution-aware routing model is used to find a set of flow paths for all transport tasks.

9. A contamination sensing physical design system for a microfluidic FPVA biochip, characterized in that: include: A definition module is used to define the framework description of the pollution-aware layout model and the pollution-aware wiring model based on the microfluidic FPVA biochip, wherein the framework description includes an action space, a state space, a mask generation, and a reward function; A training module, used for training the pollution-aware layout model and the pollution-aware wiring model based on a deep reinforcement learning algorithm according to the framework description of the pollution-aware layout model and the framework description of the pollution-aware wiring model, respectively, to obtain a trained layout model and a trained wiring model; The collaborative design module is used to synchronize and coordinate the trained layout model and routing model so that the trained layout model and routing model can globally perceive the changes of all resources on the FPVA biochip when performing their respective tasks, and obtain a contamination-aware physical design solution.

Images