Continuous microfluidic biochip multi-link joint optimization design method based on deep reinforcement learning
By integrating binding, scheduling, placement, and wiring processes using deep reinforcement learning, the problem of insufficient information interaction in microfluidic biochip design was solved, improving design quality and efficiency, and optimizing biochemical reaction time and channel length.
Patent Information
- Application Number
- CN202511087756.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-05
- Publication Date
- 2025-11-14
AI Technical Summary
The lack of information exchange between stages in traditional step-by-step design methods leads to low design quality and long cycle time for continuous microfluidic biochips, as well as the difficulty in reconciling solution quality and scalability in existing one-step architecture frameworks.
A deep reinforcement learning-based approach is adopted to construct a state space and action space. Node topology relationships are extracted through a graph convolutional neural network, a multi-objective reward function is defined, and a near-end policy optimization algorithm is combined to achieve joint optimization of binding, scheduling, placement, and routing, thereby generating a chip flow layer architecture scheme.
It improves the overall quality of chip architecture solutions, shortens the design cycle, ensures the generation of feasible solutions on large-scale chips, and optimizes biochemical reaction time, flow channel length, and valve number.
Smart Images
Figure CN120952045A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of electronic automation technology for microfluidic biochips, specifically relating to a multi-stage joint optimization design method for continuous-flow microfluidic biochips based on deep reinforcement learning. It can also be understood as a one-step architecture synthesis method for continuous-flow microfluidic biochips (CFMBs). Background Technology
[0002] Continuous-flow microfluidic biochips (CFMBs) are widely used in the biomedical field due to their advantages such as miniaturization, high reliability, and low sample consumption. However, with the increase in chip integration, their design complexity has increased significantly. Traditional step-by-step design methods handle tasks such as binding, scheduling, placement, and wiring in steps, resulting in insufficient information exchange between each step, leading to low solution quality and long design cycles.
[0003] With advancements in microfabrication technology, the feature size of CFMBs continues to shrink. It is now possible to fabricate thousands of valves onto a chip plane, achieving densities approaching 1 million / cm². 2 This has led to a rapid increase in the complexity of biochip design. Therefore, in recent years, researchers have focused on developing corresponding design automation solutions to achieve a fully automated process supporting large-scale integration of biochips. The design flow of CFMBs typically consists of multiple stages, including bonding, scheduling, placement, and routing, to systematically optimize key performance indicators such as biochemical reaction completion time, chip area, channel length, and channel intersections. However, there are strong interrelationships between design steps, and the exchange of information between design results and constraints can lead to gaps in the design process. These design gaps inevitably limit the quality of the resulting chip architecture solution, prolong the corresponding design cycle, and may even limit the feasibility of realizing the entire biochip. To eliminate the gaps between design steps, researchers have proposed a one-step architecture synthesis framework. However, the quality and scalability of existing continuous microfluidic biochip solutions are difficult to reconcile. Summary of the Invention
[0004] To address the shortcomings and deficiencies of existing technologies, this invention provides a multi-stage joint optimization design method for continuous microfluidic biochips based on deep reinforcement learning. This method aims to solve the problems of low solution quality and long design cycles caused by insufficient information interaction between stages in traditional step-by-step design, as well as the incompatibility between solution quality and scalability in existing one-step architecture frameworks. The method constructs a state space containing operational device binding information, spatial coordinates, device size, and operational priorities; it then uses a graph convolutional neural network to extract node topological relationships and connectivity features. A three-dimensional action space is defined, including device position adjustment, corresponding type device binding switching, and operational priority adjustment. Based on the dynamic updates of the state space and action space, a device binding scheme, a priority-based scheduling sequence, and a placement and routing scheme based on a path planning algorithm are generated simultaneously. A multi-objective reward function is designed, which includes a placement validity reward determined by the number of devices occupied at chip grid coordinate points, and normalized weighted rewards for biochemical reaction time, total channel length, and number of valves. A proximal policy optimization algorithm is employed, maximizing the above reward function through a policy gradient update mechanism, achieving joint optimization of binding, scheduling, placement, and routing, ultimately outputting a chip fluid layer architecture scheme. Experiments show that, compared with existing methods, this method optimizes aspects such as biochemical reaction time, total flow channel length, and number of valves, and can still generate feasible solutions on large-scale chips.
[0005] The specific technical solution adopted by this invention to solve its technical problem is as follows:
[0006] A multi-stage joint optimization design method for continuous microfluidic biochips based on deep reinforcement learning includes:
[0007] A state space is constructed, which is based on node features including operating device binding information, spatial location coordinates, device size and operation priority. The topological relationships and connectivity features of the nodes are extracted by a graph convolutional neural network.
[0008] Define a three-dimensional action space, which includes the device's position adjustment along the coordinate direction, the binding and switching of operations with corresponding device types, and the numerical adjustment of operation priorities;
[0009] Based on the dynamic updates of the state space and action space, an operation device binding scheme, a priority-based scheduling sequence, and a layout and routing scheme based on a path planning algorithm are generated synchronously.
[0010] Calculate a multi-objective reward function, which includes a layout legality reward and normalized weighted rewards for biochemical reaction time, total flow channel length, and number of valves, wherein the layout legality reward is determined based on the number of devices occupied at chip grid coordinate points;
[0011] A near-end policy optimization algorithm is adopted, which maximizes the multi-objective reward function through a policy gradient update mechanism to achieve joint optimization of binding, scheduling, placement and routing, and outputs a chip flow layer architecture scheme.
[0012] Furthermore, the node features in the state space specifically include node type, device x-coordinate, y-coordinate, device number, device width, device height, and operation priority. The graph convolutional neural network creates an initial representation of the node through the node features and extracts the node connectivity into a low-dimensional vector representation.
[0013] Furthermore, in the three-dimensional action space: the device position is adjusted to move in units along the four directions of up, down, left, and right; the device binding is switched to select the previous, current, or next device from the corresponding device list; and the operation priority is adjusted to a priority value of +1, +0, or -1.
[0014] Furthermore, the path planning algorithm is the A* algorithm, which is used to generate the layout and wiring path of the flow channel based on the device location information; the scheduling adopts the list scheduling algorithm, which is used to generate the specific operation execution sequence based on the operation priority.
[0015] Furthermore, the calculation method for the layout legality reward is as follows: when the number of devices occupied by any coordinate point (i,j) in the chip grid, A(i,j)≤1, the layout is legal and the reward value is a preset positive value; when A(i,j)>1, the layout is illegal and the reward value is a negative value of the sum of the number of devices occupied by that coordinate point.
[0016] Furthermore, the proximal policy optimization algorithm improves training stability by limiting the policy update magnitude, and maximizes the objective function based on the cumulative reward value through the policy gradient update mechanism, thereby achieving iterative optimization of the policy network parameters.
[0017] Furthermore, the state space corresponds to the complete design flow of a continuous microfluidic biochip, containing real-time state information of all operations, and the policy network achieves dynamic decision-making by learning the mapping from the state space to the action space.
[0018] Furthermore, the device list is categorized by device type, and the operation selects and binds only from the device list that matches its type, in order to avoid mismatch between the operation type and the device type.
[0019] Furthermore, the iterative process of joint optimization includes: initializing the network parameters of deep reinforcement learning, updating the architecture solution by selecting actions through the policy network, determining the legality of the layout and calculating the reward value, updating the network parameters based on the reward value through the proximal policy optimization algorithm, and repeating the iteration until an optimized chip flow layer architecture scheme is generated.
[0020] Furthermore, the number of valves is determined based on the number of flow channel intersections.
[0021] And a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.
[0022] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.
[0023] Compared with the prior art, the present invention and its preferred embodiments have at least the following beneficial effects:
[0024] Compared to existing technologies, this invention and its preferred solutions integrate design stages such as binding, scheduling, placement, and routing into an organic whole through multi-stage joint optimization design based on deep reinforcement learning. This solves the problem of insufficient information interaction between stages in traditional step-by-step design, which is conducive to improving the overall quality of chip architecture solutions and shortening the design cycle. At the same time, it alleviates the contradiction between solution quality and scalability in the existing one-step architecture synthesis framework, enabling the generation of feasible solutions in large-scale chip design.
[0025] The application of graph convolutional neural networks helps to extract node features and topological relationships more accurately, providing more effective information support for decision-making; the design of the three-dimensional action space enables dynamic adjustment of device position, binding relationship, and priority, improving the flexibility and adaptability of the design process; the construction of a multi-objective reward function takes into account key indicators such as layout legality, biochemical reaction time, flow channel length, and number of valves, making the optimization direction more comprehensive; the adoption of a near-end policy optimization algorithm enhances the stability of policy updates and improves the exploration efficiency of complex decision spaces; the device binding type matching mechanism reduces the problem of mismatch between operation and device type, further ensuring the rationality of the design scheme. Attached Figure Description
[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:
[0027] Figure 1 This is a complete synthesis flowchart of CFMBs according to an embodiment of the present invention;
[0028] Figure 2 This is a schematic diagram of the strategy and value network in an embodiment of the present invention. Detailed Implementation
[0029] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:
[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.
[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.
[0032] This invention proposes a multi-stage joint optimization design method for continuous microfluidic biochips based on deep reinforcement learning. First, a graph convolutional neural network is used to extract state features, effectively capturing information about nodes and their relationships. Second, the A* algorithm and list scheduling algorithm are combined in the proximal policy optimization algorithm to obtain a specific architecture design scheme. Finally, a multi-objective reward function is designed, which normalizes and weights the biochemical reaction time, total flow channel length, and number of valves, and achieves efficient exploration of the complex decision space through the policy gradient update mechanism of the proximal policy optimization algorithm. Experiments show that, compared with existing methods, the proposed method optimizes the biochemical reaction time by 2.1%, reduces the total flow channel length by 21.3%, and reduces the number of valves by 65.0% on benchmark test cases, and can still generate feasible solutions on large-scale chips.
[0033] The essence of this invention is a one-step architecture synthesis method for CFMBs based on deep reinforcement learning. It integrates all design steps into an organic whole, considering all steps simultaneously, thereby completely eliminating gaps between design steps and solving the problems of quality and scalability compatibility. Through the proposed method, biochips can be directly implemented on the chip in a global manner, while minimizing the total time of biochemical reactions, flow channel length, and the number of additional valves introduced, achieving joint optimization of binding, scheduling, placement, and wiring within a given chip cost.
[0034] The following specific embodiment illustrates and describes the content of the present invention:
[0035] like Figure 1 , Figure 2As shown, this embodiment proposes a DRL-based method to solve the one-step architecture synthesis problem of continuous microfluidic biochips. A policy modeled by a neural network is trained, learning to take actions that maximize cumulative rewards by repeating sequences of states, actions, and rewards. Based on a graph convolutional neural network architecture, the network extracts connectivity information of relevant nodes into low-dimensional vector representations that can be used downstream. Given the cumulative reward for each state, the method employs Proximal Policy Optimization (PPO) to update the parameters of the policy network. PPO improves stability by limiting the magnitude of policy updates; its core idea is to maximize the objective function.
[0036] This invention creates an initial representation of each node based on its characteristics, including node type, x-coordinate, y-coordinate, device number, device width and height, and priority. Then, the architecture solution of the chip flow layer is iteratively updated based on the action selected by the model.
[0037] The one-step architecture proposed in this invention synthesizes a complete design flow by having each state space corresponding to a specific state. The model learns the mapping from state to action through a policy network, where each action involves adjustments in three dimensions: device binding selection, the spatial location of the device on the chip, and the operation priority. Based on the operation information in the state space, specific binding results can be obtained; priorities can be combined with a list scheduling algorithm to generate a specific scheduling process; and specific placement and routing schemes can be further generated using device location information and the A* algorithm. Therefore, the various design flows are constantly interacting with each other.
[0038] The training process of the model is as follows: First, the network information in the DRL method is initialized, and then the model autonomously selects actions to obtain the next state s. t+1 The system obtains device binding information, device location, and priority for each operation. Next, it determines the validity of the layout based on the device location information. If valid, it combines the scheduling algorithm and A* algorithm to obtain a detailed scheduling, layout, and routing scheme, calculating the total reward value. If invalid, it directly calculates the total reward value. Finally, it begins the next learning iteration. After completing the specified iterations, it generates an optimized flow layer architecture solution, minimizing the completion time of the biochemical reaction, the total length of the flow channels, and the total number of additional valves introduced.
[0039] The implementation of the above solution in this embodiment is also based on the following specific elements:
[0040] 1. State Space
[0041] In each step, the state space corresponds to a complete chip flow layer architecture solution. The state space contains the state information of all operations. The status of each operation includes the following information.
[0042] (1)
[0043] Where, x i and y i d represents the x-coordinate and y-coordinate of the top-left corner of the device bound to the operation on the chip. i This indicates the device serial number to which the operation is bound, w i and h i p represents the width and height of the device to which this operation is bound, respectively. i This indicates the priority of the operation.
[0044] 2. Action Space
[0045] The fluid layer design task of continuous microfluidic biochips involves four stages: binding, scheduling, placement, and wiring. During the execution of these stages, the operation binding information, device location information, and operation priority information all need to be dynamically adjusted. The action sequence of each operation is shown in Equation (1).
[0046] (2)
[0047] in, This indicates the action of selecting the device location. This indicates the action of selecting a device. This represents an action that changes the priority of an operation. It can be seen that the action space is mainly divided into three categories, with values... , , The specific information is as follows:
[0048] (a) Position adjustment: The device bound to each operation can move in four directions: up, down, left, and right.
[0049] The device's position is changed through action selection, meaning that each time there are four possible movement options: move up, down, left, or right by one space.
[0050] (b) Device selection: Each operation can select the previous, current or next device to bind.
[0051] To avoid mismatches between operation type and device type when changing device binding information through action selection, different types of devices are divided into multiple device lists. Each operation selects the device from the corresponding device list based on its own operation type.
[0052] (c) Priority adjustment: The priority of each operation can be adjusted to +1, +0 or -1.
[0053] The main function of priority information is to select the operation to be executed first when multiple operations are ready at the same time, based on their priority. This can change the scheduling process and thus optimize the completion time of the entire biochemical reaction.
[0054] 3. Reward Function
[0055] The goal of high-level integration and physical design is to minimize the completion time of the biochemical reaction, the total length of the flow channel, and the total number of additional valves introduced. Therefore, multiple objectives are combined into a single reward function. First, the individual sub-rewards are normalized, and then the total reward is the weighted sum of the total biochemical reaction time, the total length of the flow channel, and the number of additional valves introduced. The weights can be used to explore the trade-offs between the indicators. The specific reward function is shown in Equation (2).
[0056] (3)
[0057] (4)
[0058] (5)
[0059] Where u is the reward for whether the layout is valid, as shown in formula (3). Let (i,j) represent the number of devices in the chip grid at coordinates (i,j). N represents an illegal reward. s The reward represents the normalized total time of the biochemical reaction. The total time of the biochemical reaction is derived from the state space obtained by DRL combined with a list scheduling algorithm. The specific algorithm flow is shown in Algorithm 1; r l This represents the reward after normalizing the total length of the flow channels. The flow channel length is mainly obtained by calculating the path between operations using the A* algorithm; r v The bonus represents the additional number of valves introduced, calculated by determining the number of intersections in the flow channels.
[0060] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.
[0061] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0062] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.
[0064] This invention is not limited to the above-described preferred embodiments. Anyone inspired by this invention can derive other forms of multi-stage joint optimization design methods for continuous microfluidic biochips based on deep reinforcement learning. All equivalent variations and modifications made within the scope of the claims of this invention shall fall within the scope of this invention.
Claims
1. A multi-stage joint optimization design method for continuous microfluidic biochips based on deep reinforcement learning, characterized in that, include: A state space is constructed, which is based on node features including operating device binding information, spatial location coordinates, device size and operation priority. The topological relationships and connectivity features of the nodes are extracted by a graph convolutional neural network. Define a three-dimensional action space, which includes the device's position adjustment along the coordinate direction, the binding and switching of operations with corresponding device types, and the numerical adjustment of operation priorities; Based on the dynamic updates of the state space and action space, an operation device binding scheme, a priority-based scheduling sequence, and a layout and routing scheme based on a path planning algorithm are generated synchronously. Calculate a multi-objective reward function, which includes a layout legality reward and normalized weighted rewards for biochemical reaction time, total flow channel length, and number of valves, wherein the layout legality reward is determined based on the number of devices occupied at chip grid coordinate points; A near-end policy optimization algorithm is adopted, which maximizes the multi-objective reward function through a policy gradient update mechanism to achieve joint optimization of binding, scheduling, placement and routing, and outputs a chip flow layer architecture scheme.
2. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The node features in the state space specifically include node type, device x-coordinate, y-coordinate, device number, device width, device height, and operation priority. The graph convolutional neural network creates an initial representation of the nodes through the node features and extracts the node connectivity into a low-dimensional vector representation.
3. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: In the three-dimensional action space: the device position is adjusted to move in units along the four directions of up, down, left, and right; the device binding is switched to select the previous, current, or next device from the corresponding device list; the operation priority is adjusted to a priority value of +1, +0, or -1.
4. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The path planning algorithm is the A* algorithm, which is used to generate the layout and wiring path of the flow channel based on the device location information; the scheduling adopts the list scheduling algorithm, which is used to generate the specific operation execution sequence based on the operation priority.
5. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The calculation method for the layout legality reward is as follows: when the number of devices occupied at any coordinate point (i,j) in the chip grid, A(i,j)≤1, the layout is legal and the reward value is a preset positive value; when A(i,j)>1, the layout is illegal and the reward value is the negative value of the sum of the number of devices occupied at that coordinate point.
6. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The proximal policy optimization algorithm improves training stability by limiting the policy update magnitude. Based on the cumulative reward value, it maximizes the objective function through the policy gradient update mechanism to achieve iterative optimization of the policy network parameters.
7. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 6, characterized in that: The state space corresponds to the complete design flow of a continuous microfluidic biochip, containing real-time state information of all operations. The policy network achieves dynamic decision-making by learning the mapping from the state space to the action space.
8. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 3, characterized in that: The device list is categorized by device type. Operations select and bind devices only from the device list that matches the device type, in order to avoid mismatches between operation type and device type.
9. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The iterative process of joint optimization includes: initializing the network parameters of deep reinforcement learning, updating the architecture solution by selecting actions through the policy network, determining the legality of the layout and calculating the reward value, updating the network parameters based on the reward value through the near-end policy optimization algorithm, and repeating the iteration until an optimized chip flow layer architecture scheme is generated.
10. The method for joint optimization design of multiple components of a continuous microfluidic biochip based on deep reinforcement learning according to claim 1, characterized in that: The number of valves is determined based on the number of flow channel intersections.